I’ve been experimenting lately with ways of generating point cloud data from moving footage. This effect has been very popular lately and i wanted to find a way of generating a similar look without expensive equipment or time consuming 3D builds. Above is an example of some footage i shot over the summer with a Canon 7D and a KAMERAR SD-1 Mark II slider.
This look was generated in the compositing package Nuke utilizing its camera tracker. When creating a camera track, nuke natively spits out a point cloud. This is intended to give you a representation of the scene in front of the camera. Usually these points are not used at all…. but we are going to do something cool with them :)
Bellow is an example of how simple the set up is. All you need is you footage you’d wish to generate a point cloud from and a camera tracker node.
Typically the default settings on the camera tracker will not give you a dense enough point cloud. I found these settings gave the best results on HD footage.
Sometimes i’d do a few different tracks with different amounts of “Number of Features” and “Detection Threshold”
Once a track is complete next steps are to solve the camera. Go to the CameraTracker tab and click the solve button.
Once the solve is complete go to the “Export” section of the attribute window and click the camera drop down. Select Scene+ in this dropdown menu and click create. This should build what you see bellow which is everything you need to create the effect. Your camera and a nice dense point cloud that represents the footage.
Now all we do is disconnect the original footage. To do this i usually just replace the footage with a black “constant” set to the same resolution as the footage.
This creates a very heavy scene… at this point id recommend rendering the output of the “scanline” to a “.exr” sequence. I'll set my write node to save EXR’s with all channels. We will need depth information included from the scanline to add depth of field back into our image. This render can take a while so go chill out or go for a walk. you deserve it.
Once the render is complete the creative juices start flowing.
I usually start with a blank scene, read in the exr’s we rendered and create a Zdefocus node.
Depth of field helps show the overall shape of the scene and creates focus for your eye. I play around with its depth and the size until i can make shapes out in the scene.
Once you find a cool look all that’s left is color grading and adding elements to give the shot some creative flair. In some i added dust elements drifting through the air. In another I emphasized the rain fall by making it look like they emit light.
Go create some cool looking clips.