In this project we were given a video of garage in which we have to carry out various VFX techniques such as rotoscopy, color grading, color correction, composition etc. Along with this we have to add a 3D objects in the scene which would be located inside the room that we can see in the video below.
In the beginning, as per the video, we were given the distorted grid, which was exactly according to the plate. Thanks to this grid we were able to take its reference and undistort the plate itself. Undistorting the plate is extremely crucial as it flatten ups the plate resulting into a better working output. With the help of this undistort node we can add the exact distortion back to the video after all the editing and composition is done so that it is back to its original way.
After undistorting the plate, I started by 3D tracking the video. 3D tracking is the key to the videos in which we have to do an addition of 3D Geometries. With the help of 3D track we get these track points (refer to the image below) on which we can set up the object. For doing so we have to select the point and create a card/geometry which would be aligned in the 3D space. In these 3D space we can add as many as 3D objects/images on the plate. All of these geometries as well as cards are then combined together into a scene node. These scene nodes collect all the data of 3D space and combine it together to get the merged result.
After 3D the entire sequence, the next process of rotoscoping and cleanup. Thanks to the 3D trackers the process reduces the time it usually takes up for a normal/manual rotoscope. The main process begins by adding a frame hold. These frame hold should illustrate the clearest vision of the cleanup place, as the cleanup can look blurred if taken on the wrong frame. By selecting the point where we want to add a card (specifically on the area where we want to do the cleanup process), we can create a card. On this card, we do our desired cleanup. I have specifically used roto paint, as it gave me much more flexibility for doing the cleanups. After doing the desired cleanup we copy these onto the card which we created prior. Project3D is the node that we use to copy these cleanup roto paint onto the card. After this, we add all the cleanups to the scene so that we can get the final look at our cleanup by combining them.
As we have already tracked the video, we can carry forward with rotoscoping without worrying about cleanup. For rotoscoping, I used a similar workflow to that of cleanup.
In this technique, we add project3D, card, and scanline render prior to the roto which I added later on in the case of cleanup. Even though I had proper and well-defined tracking points, I had to make slight changes to the roto, so that it matched the borders of the walls throughout the video. These slight changes were done a few frames apart, having approximately a 15-35 frame gap in between. This was a necessary step so that the roto sticks to the wall and it doesn’t slip off.
The difficult part of rotoing this particular video was the lamp handle. Due to the poor lighting at the back, the handle of the lamp disappears at some point which was difficult to see. For this, I increased the exposure of the screen and drew an approximation of the future position of the handle.
And after adding all the roto to a scene I got the result given below. (please refer to the video below)
A powerful computer program called Nuke is utilized in the film as well as broadcast industries for compositing and visual effects. Making 3D shadows is just one of its many features, which gives an object a feeling of depth and realism. These shadows can be created through a variety of methods, including shadow maps and ray tracing. In order to produce precise and lifelike shadows calculates the path of light rays from a virtual light source to the object in the picture. Shadow maps imitate the effect of shadows using a 2D texture map, which leads to quicker rendering times but possibly less accurate results. With Nuke’s 3D shadow tools, artists can modify the shadows’ intensity, color, and softness as well as their location and size.
Over here, I Created a scene and added 3D elements to track points with the help of axis which determined the Track pints in the “3D Space”. For getting these 3D objects, the color according to the scene, I Created a spot light, according to that of the scene.
After the step of lighting the scene, Comes the main part, i.e. Casting shadows. Basically, for getting this Shadow, we need to create the shadow pass. For this, the step was to do a setup of various cards/grids on which the shadow would be cast. these grids are aligned onto the track points, according to the position of walls and ground of the plate. Following this, I created a new “Shadow Mask” which would make a new “Shadow Pass” for all the 3D objects that I imported and aligned to the scene.
With the help of this step, I could extract the “Shadows” of all the elements (3D objects), and merge it with the actual scene, to get more defined output.