Technique BreakDown Future City VR Shot
While I was getting my MFA from Academy of Art University, I was compositing a VR shot as my final thesis.
Here is how is the final looks like:
I shot it at a rooftop of a parking lot using a cannon 5D Mark iii with 15mm lens and a nodal ninja tripod. I shot 6 multi exposure image in every 90 degrees, and make it HDR image using Photoshop.
Bring them in Nuke use CARA VR to stitch them to a panorama image. Use C_CameraSolver node to track the similar points in each of these shots, then use C_Stitcher to stitch it. After pre-stitching I found out that there are some ghost issue at some area, so I split all the views and add roto to adjust the alpha to remove the ghost part. Join them together and bring it into an C_Blender node to final it
As soon as I get the panorama from CARA, I did some cleaning work ( take out the tripod and some buildings ). It should be easier in photoshop, but I want to get familiar with the workflow of CARA, so I used the toolset in CARA called latlong-rotopaint. To fully understand this process, there are four kinds of projection in CARA VR display - Latlong, Cubemap, Rectilinear and Fisheye. What it does is transform latlong projection to rectilinear projection ( it looks like a regular 2d shot and the focal lens is adjustable ), then we should able to paint whatever we need. It's the same principle to comp a 2d element into VR shot.
Then I need some CG buildings, so I modeled and textured some buildings in C4D. also rebuilt the ground and some of walls to cast shadow and light, so at the end I could multiply the textures on to get the right lighting. Here are what I have after render. I put lights on where I planned to place my hologram so I could have the lighting separately.
This is only a still image, cause the render time was taking too long I decided to render animated elements separately too. I also want to experience mattepainting in VR, so I left some of the area open for MP. Here is where I met difficult. It was extremely hard to match the lens distortion in latlong projection format. After some tests, I ending up rendering two boxes with grid in a cubemap projection in C4D, so that I could easily do MP with the accurate perspective and lens distortion. Here is what I did.
I multiplied the ground and wall texture onto the lighting condition, and merge them together.
It's straight forward process to comp in 2d elements. I transformed latlong to retilinear again, turn the camera to where I want to put them, merge the elements and transform back to latlong format. I also rendered a couple of hover cars and slogans in C4D at the end.
so this is what I got in final.
It was an interesting experience to me. The look of it is not perfect, but the point is I almost went though every kind of situation in both VR compositing and CG side, and it's good to explore things that I didn't know and learning it by doing this kind of project.