Doing screen replacement in a video with live action footage is one of the most commonly used features of Fliva rendering platform. Be it handheld devices like phones and tablets or large screens like computers and even tv’s.
Even though we do support many features of After Effects, we do not support everything. And some features are just slower to render. We advise you to use as few After Effects features as possible while still getting the result you want, this usually improves render times and ultimately the cost. One of the ways to accomplish this is to pre-render as much as possible. When doing screen replacements, we usually do this, to ensure that the edges of the screens look ultra sharp, and blended nicely into the background.
Setup the project
Start by creating a single comp and insert your live action footage.
Track the screen
Next up you need the screen tracked so that you can replace the content. We decided to use Mocha for this since it is bundled with AE.
Now we have a relatively simple scene here, with nothing in front of the screen and almost no movement of the camera, so it was rather easy to track. We created an X-Spline layer around the screen. Then we clicked “Show Planar Surface”, and moved the corners to the corners of the screen. Finally, we just started from the beginning, and Mocha tracked every frame for us.
Make sure the track is perfect before copying the settings. Just a few pixels of in a frame or two will ruin the effect when rendered. There are many great tutorials out there on how to do this.
Import the tracking data into After Effects
Next, we create a solid. We decided to make it light green, this is a sharp contrast to the rest of the scene, so it is easier to see what is going on.
And then we add the Corner Pin effect from the Effects -> Distortion menu.
Then we paste the tracking data from Mocha into the corner pin settings. Now you can add masks and perfect the feathering to make the screen look natural.
Face in the hole technique
The final step before making the real screen content is to pre-render. We usually do this by the “face in the hole”- method. This refers to letting the screen be 100% transparent, and then pre-render the final footage. Then we can place the pre-rendered footage in a layer above the screen content. The masks and feathering from before will become semitransparent.
First on the footage create a track matte from the solid, with the type “Alpha Inverted Matte”. All the places that were green before are now transparent.
Finally, we pre-render. We use the Quick Time video codec “Animation” with alpha channel enabled. We do this so often that we have a preset for it in Media Encoder.
The final scene
Now to the composition that you can send to us. First, place the footage at the topmost layer, and put a composition under it. Copy the Corner Pin effect, and it’s tracking data you used for the green solid before to this new composition. (You could also pre-compose the solid).
This composition is the screen content. You could place anything here. We decided to go with simple text on a black background.
And there you have it, the screen replacement in a video, in 9 easy steps. Now we could make the “Dynamic Screen Content” layer variable – give it a good name. And we could create thousands of these videos with custom text per recipient.