This strategy is meant to help create a more convincing and accurate viewer experience.
Google researchers have developed a strategy to automate the 3D modeling process while enhancing the results it generates through photo crowdsourcing.
The tech giant is collecting the images of landmark locations to create convincing replications.
The idea of the 3D modeling technique is to use pictures of the real places including both the landmarks and their lighting. The technology employed is called neural radiance fields (NeRF). It extracts three-dimensional depth data from 2D images by measuring the termination point of light rays. This complex technique that can generate realistic, textured three-dimensional landmark models.
The Google NeRF in the Wild (NeRF-W) system steps into even more sophisticated territory. The reason is that it uses “in-the-wild photo collections” as its input material. This broadens the software’s capacity to view various landmarks from different angles. From there, it evaluates the images it receives for structures, separating environmental and photographic variations such as lighting on the scene, image exposure, and weather conditions. It also conducts comparisons from one shot to the next, to spot differences in objects such as people who may be seen in one image but aren’t in another.
With that data, the 3D modeling tech recreates scenes based on combinations of static elements.
The static elements are composed for structure geometry and textures. Those are combined with transient elements that offer volumetric radiance.
What all this means is that the outcome of NeRF-W is a three-dimensional image that provides smooth viewing from multiple angles without an artifacted or jittery distortion. At the same time, the lighting system applies the detected variations to offer radiance guidance for the scene so the three-dimensional version produced will contain appropriate lighting and shadows.
NeRF-W is also able to designate object differences from one image to the next as uncertainty fields. In this way, they can be de-emphasized or removed altogether. This is different from conventional NeRF 3D modeling, which turns those image-to-image differences into cloudlike occlusion artifacts as it does not have the capacity to differentiate between them and the structures upon receipt of the images. Google produced a video about its use of NeRF-W, indicating that its landmark recreation is 
adequately convincing that AR and VR users will be able to gain a solid experience of complex architecture’s true appearance.
