For this post, I’m going to give you an overview of how I rendered a wall of boxes into some video footage of my garage. To get started, you’ll need some footage that you want to add elements to. I shot my scene handheld which makes tracking a bit more difficult. A more efficient way to do this would be to shoot the footage on a tripod and then add in the camera shake after everything is composed. Either way, start by adding in some high-contrast tracking points. These can be taped-down markers or simply objects in a scene that stands out from the background.
Tracking the Footage
In Blender, go to the Motion Tracking layout and open your "image" and choose the video footage. Go to the Toolbar Tab Track / Clip and click Set Scene Frame. This matches your blender timeline to the video clip.
Now to add the tracking points go to Markers Tab, choose Detect Features to automatically find points, or manually pick your points by choosing Add and click on what you want to be a marker in the footage. Try to add 10 or more markers. You can always delete the ones that don’t solve cleanly. Once the markers are set you can track those points by clicking the track forward button on the timeline.
Solving the Track Points
In the Solve tab, toggle on Keyframes so that Blender chooses the keyframes for you and click the Solve Camera Motion button. Once the Solve is done you’ll see the Solve Error range in the top right of your viewport. Try and get the error under 0.5 by deleting any tracking points that drift or go off target. You can also look at the waveform and select and delete any points that don’t fit the main motion path.
Once you are happy with the Solve Erro number. Go to The Scene Setup and click Set as Background. This aligns your video footage with your camera.
Defining the Scenes Axis
With everything tracked, the video footage and camera movements match but Blender doesn’t know what way is up. To align the axis of the scene to the footage, go to the Orientation tab, choose 3 points on a flat surface, and choose either wall or floor. Obviously, pick the option that fits your scene. For my footage I chose Floor. Next, check your 3D viewport and see if things align right. If it’s giving you trouble, try choosing different tracking points and reapply the Orientation.
Modeling the scene
Now that the technical work is out of the way, it’s time to start modeling things. For my scene, I’m adding boxes inside the garage so we will need to model anything that will be between our 3D props and the camera. That means the walls and garage door. After that, you can add in your props.
A big part of making your 3D objects fit the scene is to make sure your lighting matches. How you set up your lights all depends on your footage. Do some side-by-side tests with your basic geometry and compare it to your footage until you have something close. Pay close attention to how defined the shadows are.
Getting ready to render
The last part before we render the scene is to hide the rough geometry while still keeping that Cast Shadows. To do this select your geometry, go to Object Properties, Visibility tab, and toggle on Shadow Catcher.
Now render out your scene and see how it all went! If you want to see you can also render out the scene without the footage and composite it in another program like after effects but that’s for another post.