Motion capture

Motion capture, Motion Tracking or Mocap, is a technique of digitally recording movements for entertainment, sports and medical applications.

Methods and Systems

Motion tracking or motion capture started as a photogrametric analysis tool in biomechanics research, and expanded into education, training, sports and recently computer animation for cinema and video games as the technology matured. A performer wears markers near each joint to identify the motion by the positions or angles between the markers. Acoustic, inertial, LED, magnetic or reflective markers, or combinations of any of these, are tracked, optimally at least two times the rate of the desired motion, to submillimeter positions. The motion capture computer software records the positions, angles, velocities, accelerations and impulses, providing an accurate digital representation of the motion.

In entertainment applications this can reduce the costs of animation which otherwise requires the animator to draw each frame, or with more sophisticated software, key frames which are interpolated by the software. Motion capture saves time and creates more natural movements than manual animation, but is limited to motions that are anatomically possible. Some applications might require additional impossible movements like animated super hero martial arts or stretching and squishing that are not possible with real actors.

In biomechanics, sports and training, real time data can provide the necessary information to diagnose problems or suggest ways to improve performance, requiring motion capture technology to capture motions up to 140 miles per hour for a golf swing.

Optical systems

Optical systems triangulate the 3D position of a marker between one or more cameras calibrated to provide overlapping projections. Tracking a large number of markers or multiple performers or expanding the capture area is accomplished by the addition of more cameras. These systems produce data with 3 degrees of freedom for each marker, and rotational information must be inferred from the relative orientation of three or more markers; for instance shoulder, elbow and wrist markers providing the angle of the elbow...

Passive optical

Passive optical system use markers coated with a Retroreflective material to reflect light back that is generated near the cameras lens. The cameras sensitivity can be adjusted taking advantage of most cameras narrow range of sensitivity to light so only the bright markers will be sampled ignoring skin and fabric.

The centroid of the marker is estimated as a position within the 2 dimensional image that is captured. The grayscale value of each pixel can be used to provide sub-pixel accuracy.

An object with markers attached at known positions is used to calibrate the cameras and obtain their positions and the lens distortion of each camera is measured. Providing two calibrated cameras see a marker, a 3 dimensional fix can be obtained. Typically a system will consist of around 6 to 24 cameras. Systems of over two hundred cameras exist. Extra cameras are required for full coverage around the capture subject. Typical eight camera systems are about $150,000 US.

Professional vendors have sophisticated constraint software to reduce problems from marker swapping since all markers appear identical. Unlike active marker systems and magnetic systems, passive systems do not require the user to wear wires or electronic equipment. It is important not to touch the markers or get them dirty as this changes the reflective properties and can cause errors. The markers must be visible away from the surface of the body, making them prone to being knocked off with any contact. This type of system can capture large numbers of markers at frame rates as high as 2000fps with high accuracy. The frame rate for a given system is often traded off between resolution and speed so a 4 megapixel system runs at 160 hertz normally but can reduce the resolution to 1 megapixel and then run a 640 hertz for a constant 640 million pixels per second.

Modulated active marker Optical

Active marker systems can further be refined by strobing one marker on at a time, or tracking multiple markers over time and modulating the amplitude or pulse width to provide marker ID. 12 megapixel spatial resolution modulated systems show more subtle movements than 4 megapixel optical systems by having both higher spatial and temporal resolution. Directors can see the actors performance in real time, and watch the results on the mocap driven CG character. The unique marker IDs reduce the turnaround, by eliminating marker swapping and providing much cleaner data than other technologies. LEDs with onboard processing and a radio synchronization allow motion capture outdoors in direct sunlight, while capturing at 480 frames per second due to a high speed electronic shutter. Computer processing of modulated Ids allows less hand cleanup or filtered results for lower operational costs. This higher accuracy and resolution requires more processing than passive technologies, but the additional processing is done at the camera to improve resolution via a subpixel or centroid processing, providing both high resolution and high speed. These motion capture systems are typically under $50,000 for an eight camera, 12 megapixel spatial resolution 480 hertz system with one actor.

Active marker Optical

Active optical systems triangulate positions by illuminating one LED at a time very quickly or multiple LEDs but sophisticated software to identify them by their relative positions, somewhat akin to celestial navigation. Rather than reflecting light back that is generated externally, the markers themselves are powered to emit their own light. Since Inverse Square law provides 1/4 the power at 2 times the distance, this can increase the distances and volume for capture. ILM used active Markers in Van Helsing to allow capture of the Harpies on very large sets. The power to each marker can be provided sequentially in phase with the capture system providing a unique identification of each marker for a given capture frame at a cost to the resultant frame rate. The ability to identify each marker in this manner is useful in realtime applications. The alternative method of identifying markers is to do it algorithmically requiring extra processing of the data. Eight camera systems are about $400,000 US.

Inertial systems

Inertial Motion Capture technology is based on miniature inertial sensors, biomechanical models and sensor fusion algorithms. It's an easy to use and cost-efficient way for full-body human motion capture. The motion data of the inertial sensors (Inertial_guidance_system) is transmitted wirelessly to a PC or laptop, where the full body motion is recorded or viewed. No external cameras, emitters or markers are needed for relative motions, but for absolute positioning, an external magnetic or optical system is used. Inertial mocap systems capture the full 6 degrees of freedom body motion of a human in real-time. These systems are similar to the WII controllers but much more sensitive and having much greater resolution and update rate. They can accurately measure the direction to the ground to within a degree. To achieve a higher accuracy for positioning, a combination with an optical or magnetic system can be used. Base suits tend to be in the $50,000 range.

Mechanical motion

Mechanical motion capture systems directly track body joint angles and are often referred to as exo-skeleton motion capture systems, due to the way the sensors are attached to the body. A performer attaches the skeletal-like structure to their body and as they move so do the articulated mechanical parts, measuring the performer’s relative motion. Mechanical motion capture systems are real-time, relatively low-cost, free-of-occlusion, and wireless (untethered) systems that have unlimited capture volume. Typically, they are rigid structures of jointed, straight metal or plastic rods linked together with potentiometers that articulate at the joints of the body. These suits tend to be in the $25,000 to $75,000 range plus an external absolution positioning system.

Magnetic systems

Magnetic systems calculate position and orientation by the relative magnetic flux of three orthogonal coils on both the transmitter and each receiver. The relative intensity of the voltage or current of the three coils allows these systems to calculate both range and orientation by meticulously mapping the tracking volume. Since the sensor output is 6DOF, useful results can be obtained with two-thirds the number of markers required in optical systems; one on upper arm and one on lower arm for elbow position and angle. The markers are not occluded by nonmetallic objects but are susceptible to magnetic and electrical interference from metal objects in the environment, like rebar (steel reinforcing bars in concrete) or wiring, which affect the magnetic field, and electrical sources such as monitors, lights, cables and computers. The sensor response is nonlinear, especially toward edges of the capture area. The wiring from the sensors tends to preclude extreme performance movements. The capture volumes for magnetic systems are dramatically smaller than they are for optical systems. With the magnetic systems, there is a distinction between “AC” and “DC” systems: one uses square pulses, the other uses sine wave pulse.

Performance capture

Performance capture differs from standard motion capture due to the interactive nature of the performance, capturing the body, the hands and facial expression all at the same time, as opposed to capturing data for reference motion and editing the motions together later. The actor usually interacts with models of the objects in the scene. The recorded performance data can be used to animate different actors. In The Polar Express Tom Hanks played five roles - an 8-year old boy, his father, the train's conductor, a hobo, and Santa Claus.

For Active Appearance Model, Principal components analysis or Eigen Tracking to work, Cameras need to see the desired movement approximately every 30 degrees or the actor must perform to the camera or the software will not be able to see enough information to process the image.

Although performance capture has been used in some earlier films and computer games, The Polar Express was the first movie made solely with the process. This film was directed by Robert Zemeckis, who had a long history of technical innovations in film making (historical composites in Forrest Gump and the combination of animation and live action in Who Framed Roger Rabbit) and became a self-professed fan of performance capture (he produced the 2006 thriller Monster House made using the same technique) because of the creative freedom it gives the director. Zemeckis is currently using performance capture in an adaptation of Beowulf scheduled for 2007 release.

The procedure

In the motion capture session, the movements of one or more actors are sampled many times per second. High resolution optical motion capture systems can be used to sample body, facial and finger movement at the same time.

A motion capture session records only the movements of the actor, not his visual appearance. These movements are recorded as animation data which are mapped to a 3D model (human, giant robot, etc.) created by a computer artist, to move the model the same way. This is comparable to the older technique of rotoscope where the visual appearance of the motion of an actor was filmed, then the film used as a guide for the frame by frame motion of a hand-drawn animated character.

If desired, a camera can pan, tilt, or dolly around the stage while the actor is performing and the motion capture system can capture the camera and props as well. This allows the computer generated characters, images and sets, to have the same perspective as the video images from the camera. A computer processes the data and displays the movements of the actor, as inferred from the 3D position of each marker. If desired, a virtual or real camera can be tracked as well, providing the desired camera positions in terms of objects in the set.

A related technique match moving can derive 3D camera movement from a single 2D image sequence without the use of photogrammetry, but is often ambiguous below centimeter resolution, due to the inability to distinguish pose and scale characteristics from a single vantage point. One might extrapolate that future technology might include full-frame imaging from many camera angles to record the exact position of every part of the actor’s body, clothing, and hair for the entire duration of the session, resulting in a higher resolution of detail than is possible today.

After processing, the software exports animation data, which computer animators can associate with a 3D model and then manipulate using normal computer animation software. If the actor’s performance was good and the software processing was accurate, this manipulation is limited to placing the actor in the scene that the animator has created and controlling the 3D model’s interaction with objects.

History

Motion capture began as a military technology used for tracking targets, using AC magnetic trackers. In 1986, marketer Jack Scully and engineer Ernie Blood were fired from Polhemus Navigation Services. They founded Ascension Technology, supplier of mocap devices. This began a revolution in motion capture for use in cartoons in the private sector. Chris Walker, an animator and inventor of animation technology, further developed motion capture, and used it in TV cartoons under his company Modern Cartoons in the 1990s. The mocap suit he used then cost $300,000 USD.[[1]]

Final Fantasy: Spirits Within was the first movie made solely with the motion capture.

Advantages

Mo cap offers several advantages over traditional computer animation of a 3D model:

• More rapid, sometimes even real time results can be obtained.

• The amount of work does not vary with the complexity or length of the performance to the same degree when using traditional techniques.

• Complex movement and realistic physical interactions such as secondary animation, weight and exchange of forces can be more easily recreated in a physically accurate manner.

• Mocap technology allows one actor to play multiple roles within a single film.

Disadvantages

• Specific hardware and special programs are required to obtain and process the data.

• The cost of the software and equipment, personnel required can be prohibitive for small productions.

• The capture system may have specific requirements for the space it is operated in.

• When problems occur it is sometime easier to reshoot the scene rather than trying to manipulate the data. Only a few systems allow real time viewing of the data to decide if the take needs to be redone.

• Applying motion to quadruped characters can be difficult.

• The technology can become obsolete every few years as better software and techniques are invented.

• The results are limited to what can be performed within the capture volume without extra editing of the data.

• Movement that does not follow the laws of physics generally cannot be represented.

• Traditional animation techniques such as added emphasis on anticipation and follow through, secondary motion or manipulating the shape of the character as with squash and stretch animation techniques are generally not applicable.

• If the computer model has different proportions from the capture subject artifacts may occur. For example, if a cartoon character has large, over-sized hands, these may intersect strangely with any other body part when the human actor brings them too close to his body.

• The real life performance may not translate on to the computer model as expected.

Applications

Video games use motion capture for football, baseball and basketball players or the combat moves of a martial artist as well as in action sports.

Movies use motion capture for CG effects, in some cases replacing traditional cell animation, and for completely computer-generated creatures, such as Gollum, The Mummy, and King Kong.

Virtual Reality and Augmented Reality require real time input of the user’s position and interaction with their environment, requiring more precision and speed than older motion capture systems could provide. Noise and errors from low resolution or low speed systems, and overly smoothed and filtered data with long latency contribute to “simulator sickness” where the lag and mismatch between visual and vestibular cues and computer generated images caused nausea and discomfort.

High speed—high resolution active marker systems can provide smooth data at low latency, allowing real time visualization in virtual and augmented reality systems. The remaining challenge that is almost possible with powerful graphic cards is mapping the images correctly to the real perspectives to prevent image mismatch.

Motion capture technology is frequently used in digital puppetry systems to aid in the performance of computer generated characters in real-time.

Related techniques

Facial motion capture is utilized to record the complex movements in a human face, especially while speaking with emotion. This is generally performed with an optical setup using multiple cameras arranged in a hemisphere at close range, with small markers glued or taped to the actor’s face.

Inertial systems use devices such as accelerometers or gyroscopes to measure positions and angles. They are often used in conjunction with other systems to provide updates and global reference, since they only measure relative changes, not absolute position.

RF (radio frequency) positioning systems are becoming more viable as higher frequency RF devices allow greater precision than older RF technologies. The speed of light is 30 centimeters per nanosecond (billionth of a second), so a 10 gigahertz (billion cycles per second) RF signal enables an accuracy of about 3 centimeters. By measuring amplitude to a quarter wavelength, it is possible to improve the resolution down to about 8 mm. To achieve the resolution of optical systems, frequencies of 50 gigahertz or higher are needed, which are almost as line of sight and as easy to block as optical systems. Multipath and reradiation of the signal are likely to cause additional problems, but these technologies will be ideal for tracking larger volumes with reasonable accuracy, since the required resolution at 100 meter distances isn’t likely to be as high.

An alternative approach was developed where the actor is given an unlimited walking area through the use of a rotating sphere, similar to a hamster ball, which contains internal sensors recording the angular movements, removing the need for external cameras and other equipment. Even though this technology could potentially lead to much lower costs for mocap, the basic sphere is only capable of recording a single continuous direction. Additional sensors worn on the person would be needed to record anything more.

A studio in the Netherlands is using a 6DOF (Degrees of freedom) motion platform with an integrated omni-directional treadmill with high resolution optical motion capture to achieve the same effect. The captured person can walk in an unlimited area, negotiating different uneven terrains. Applications include medical rehabilitation for balance training, biomechanical research and virtual reality.

Programs

• Organic Motion: By Organic Motion INC, real-time markerless motion capture. The first system that doesn't require a body suit, markers, or any data clean up. Similar to Mandala of the 1980s on the Amiga but having 3D body tracking. Seems to be good to about a centimeter resolution, ignoring hands and feet which are difficult to track with vision systems. [2] The Max Plank Institute and Stanford have some research and discussions on these processes.
• Imocap: By ILM, a proprietary on-set computer-vision based full-body motion capture system first used in Pirates of the Caribbean 2. The actors wear checkered bands.
• Mova Contour system: Facial motion capture, phosphorescent green makeup worn by actor.
• Image Metrics: Proprietary facial motion capture service, no makeup, no spandex suit, no special hardware.
• PhaseSpace Recap: Motion capture editor, no makeup, black spandex suit, needs hardware.
• Pure Data + Pidip: motion capture, open source program
• Eyesweb: motion capture, freeware

See also

• Animation
• Rotoscope
• Match moving, also known as “motion tracking”
• Gollum, a character in The Lord of the Rings movies which was partially animated with motion capture

Wikipedia information about Motion capture This article is licensed under the GNU Free Documentation License. It uses material from the Wikipedia article "Motion capture". It may not have been reviewed by professional editors (see full disclaimer). Help contribute to Americola and edit this article.