Visual effects have the power to reshape our perception of reality, crafting scenes that defy the laws of physics and elevate storytelling to new heights. The magic that unfolds on screen is the culmination of countless hours of meticulous work, with matchmove and 3D tracking serving as the foundation upon which this visual sorcery is built. It has been around since the 90s and has been relatively slow to make its way into the creative pipeline. But today, it’s an indispensable part of a VFX pipeline, and when it’s not done right, it simply breaks the shot. Especially where the Visual Effects is hidden in plain sight.
In the VFX industry, Matchmove is generally another accepted term for Tracking and is interchangeably used. But there is a very subtle difference between both the terms, and hence the lines are blurred between them. In this post, we shall explore that along with different forms of Tracking and Matchmove.
Why is Tracking or Matchmove done?
In compositing, the primary focus is to match the motion of an object in one shot with another object in a different shot in a seamless manner. By tracking the motion of the object being composited, the compositor can accurately depict the object’s movement in the scene. This process is beneficial for creating realistic visual effects, such as making it appear as if a creature or vehicle is interacting with the environment in a shot. It is a complex process but an essential tool for creating genuinely believable visual effects.
Difference between Tracking & Matchmove
In a Nutshell, VFX Tracking is done to locate a point or series of points from frame to frame in a sequence. This definition covers the entire spectrum of VFX tracking. The information of Tracking is later used to stabilize, solve object or camera movement in the shot.
Matchmove, on the other hand, is the method used to get the final results of a 3D tracking output only. The accuracy of a 3D tracking output is crucial in seamlessly integrating 3D elements with Live Action. Hence the style of Matchmove becomes very important as well.
The different processes of Tracking
There are various ways to do motion tracking in live-action footage, and each process has its purpose. Let’s explore a few of them.
One Point Track
Suitable Use Case: Rapid 2D tracking situations centered around a single point with no warping.
As the name suggests, one point track captures the motion data from a single point in the footage or composition. One Point Track the simplest form of Tracking and is a foundation for every tracking process that is known. But this form only yields position information of one point. With no rotation or scale information, this kind of Tracking has minimal usage and is used for achieving a simple task.
Two Point Track
Suitable Use Case: Mild to moderate levels of camera shake in the shot
The two-point track builds upon its predecessor. Two Point Track enables the user to get scale and rotation information along with the position that isn’t possible with One point track. This kind of Tracking is beneficial in capturing simple camera movement and incorporate warping.
Corner Pin Track
Suitable Use Case: Basic screen replacement.
This kind of Tracking is an artist doing screen replacement best friend. If you work on commercials or projects related to phones, tv, laptop or anything that resemble screens, you will be using this a lot. Corner Pin Tracking matches the perspective movement of an element in a shot quickly and effectively. This is achieved using 4 point track.
Planar Tracking
Suitable Use Case: Complex Tracking on flat surfaces.
Typically, when tracking points, you track a single pixel or group of pixels inside a user-defined search area. With Planar tracking, the concept remains the same, but you are tracking contrast and texture over time instead of pixels. That is why it’s also known as texture tracking. Its innovative approach that excellent track when done on a flat surface with textures.
3D Tracking
Suitable Use Case: Set extensions, Matte painting, and compositing.
3D Tracking is the most complex and superior kind of tracking that we are capable of doing today. It’s so powerful that it enables users to get information in Z-Axis that other tracking forms can’t. It generates a virtual camera from the live-action footage to place the 3D assets into the scene accurately. Because of its complexity, it comes with a deep learning curve. But it is the most effective technique of tracking to date.
Spline Tracking
Suitable Use Case: Challenging camera tracking tasks.
Spline tracking is a more advanced form of Planar Tracking. With this form, the user can track any shape because the user needs to create a custom tracking plane for it to work. Fundamentally, the user draws a roto shape around the object the user wants to track, and then it will to the best of its ability, capture the track of the object.
Additional Resources
If you are curious to know about leading Matchmove software used in a VFX Pipeline today, you can read about it here. For an in-depth read on what the other software are out there, you should try this informative article.
To know more about the tracking concepts in-depth, here is a video to assist you with them.
The different styles of Matchmove
Matchmove and 3D Tracking are synonymously used. But the purpose of 3D Tracking is to get the best possible camera solution that seamlessly integrates 3D elements into live-action footage. Matchmove is the method that makes it happens. There are two effective methods.
Automatic
In this method, computer algorithms track the features or points in a shot. The results thus generated are much faster through this method as human intervention is minimal. But for an accurate camera solution, it needs to be fed with as much camera information possible such as lens distortion, focal points, etc.
Supervised
Also referred to as ‘interactive tracking.’ In this method, an artist tracks certain features or points in the shot to support the algorithm-generated ones. This method of matchmove is used in shots where the motion blur levels are very high, and the computer algorithm cannot automatically capture the features or points.
Conclusion
Matchmove is the most technical aspect of a VFX pipeline compared to other elements. It’s one of those aspects in a VFX pipeline that is hard to salvage midway if there are any errors. The only way to fix the mistakes would be mostly to redo it. A compositor might be able to work their magic to some extent, but it’s constrained and may or may not work.
So it’s essential to have experienced matchmove artists on your project to save time and cost. At FXiation Digitals, we have artists who have years of production experience before joining us and handled complex shots with ease that our clients bought for us. Hence, If your next project calls for any tracking requirements, please check with us. We are sure your search will end here.
