Datenbank Glossar – Special Effects

Point-cloud projection

Once the camera position has been determined for every frame it is then possible to estimate the position of each feature in real space by inverse projection. The resulting set of points is often referred to as a point cloud because of its raw appearance like a nebula. Since point clouds often reveal some of the shape of the 3D scene they can be used as a reference for placing synthetic objects or by a reconstruction program to create a 3D version of the actual scene.

Ground-plane fixing

The camera and point cloud need to be oriented in some kind of space. Therefore, once calibration is complete, it is necessary to define a ground plane. Normally, this is a unit plane that determines the scale, orientation and origin of the projected space. Some programs attempt to do this automatically, though more often the user defines this plane. Since shifting ground planes does a simple transformation of all of the points, the actual position of the plane is really a matter of convenience.

Reconstruction

Reconstruction is the interactive process of recreating a photographed object using tracking data. This technique is related to photogrammetry. In this particular case we are referring to using match moving software to reconstruct a scene from incidental footage.

A reconstruction program can create three-dimensional objects that mimic the real objects from the photographed scene. Using data from the point cloud and the user’s estimation, the program can create a virtual object and then extract a texture from the footage that can be projected onto the virtual object as a surface texture.

2D vs. 3D

Match moving has two forms. Compositing programs, such as Shake, Adobe After Effects and Discreet Combustion, have two dimensional motion tracking capabilities. This feature translates images in two-dimensional space and can add effects such as motion blur in an attempt to eliminate relative motion between two features of two moving images. This technique is sufficient to create verisimilitude when the two images do not include major changes in camera perspective. For example a billboard deep in the background of a shot can often be replaced using two-dimensional tracking.

Three dimensional match moving tools make it possible to extrapolate three-dimensional information from two-dimensional photography. Programs capable of 3D match moving include:

Voodoo (freeware)

Scenespector VooCAT

Icarus (University of Manchester research project, now discontinued but still popular)

Maya Live

The Pixel Farm PFTrack

PFHoe (Cost effective match mover based on PFTrack algorithms)

REALVIZ MatchMover (autodesk bought and buried)

Science.D.Visions 3DEqualizer (which won an Academy Award for Technical Achievement)

Andersson Technologies SynthEyes

2d3 boujou (which won an Emmy award in 2002)

These programs allow users to derive camera movement and other relative motion from arbitrary footage. The tracking information can be transferred to computer graphics software such as Blender, Maya or LightWave and used to animate virtual cameras and CGI objects.

Automatic vs. interactive tracking

There are two methods by which motion information can be extracted from an image. Interactive tracking relies on the user to follow features through a scene. The points tracked by the user are then used to calculate the camera movement. Automatic tracking relies on computer algorithms to identify and track features through a shot.

The advantage of interactive tracking is that a human user can follow features through an entire scene and will not be confused by features that are not rigid. The disadvantage is that the user will inevitably introduce small errors as they follow objects through the scene, which can lead to drift.

The advantage of automatic tracking is that the computer can create many more points than a human can. A large number of points can be analyzed with statistics to determine the most reliable data. The disadvantage of automatic tracking is that, depending on the algorithm, the computer can be easily confused as it tracks objects through the scene.

Professional-level motion tracking is usually achieved using a combination of interactive and automatic techniques. An artist can remove points that are clearly anomalous and use tracking mattes to block confusing information out of the automatic tracking process.

Tracking mattes

A tracking matte is similar in concept to a garbage matte used in traveling matte compositing. However, the purpose of a tracking matte is to prevent tracking algorithms from using unreliable, irrelevant or non-rigid tracking points. For example, in a scene where an actor walks in front of a background, the tracking artist will want to use only the background to track the camera through the scene, knowing that motion of the actor will throw off the calculations. In this case, the artist will construct a tracking matte to follow the actor through the scene, blocking that information from the tracking process.

Refining

Since there are often multiple possible solutions to the calibration process and a significant amount of error can accumulate, the final step to match moving often involves refining the solution by hand. This could mean altering the camera motion itself or giving hints to the calibration mechanism. This interactive calibration is referred to as refining.

Most match moving applications seem based on similar algorithms for tracking and calibration. Often, the initial results obtained are similar. However, it seems that each program has different refining capabilities. Therefore, when choosing software, look closely at the refining process.

For more information :

Art of tracking

http://www.fxguide.com/modules.php?name=News&file=print&sid=209
http://www.fxguide.com/modules.php?name=News&file=print&sid=212

Ref book

Matchmoving: The Invisible Art of Camera Tracking — Tim Dobbert