R&D Study - VPS Test of the Niantic Spatial Platform with Unity and Scaniverse

Team and Context

Our mobile developer Noa handled most of the implementation, assisted by Maël, an intern, who actively participated in the field surveys.

The tests took place in Bourg-en-Bresse, at several iconic locations in the city. Our main target was the statue of Dr. Xavier Bichat (1771-1802), a renowned French physician, located in the Jardin du Bastion, just a few minutes from our office.

The flagship feature of the Niantic Spatial Platform is the Visual Positioning System (VPS). While GPS alone delivers accuracy to a few meters - sufficient for navigation, but unsuitable for accurately placing a virtual object - VPS can shrink that margin to just a few centimeters.

To achieve this, it combines GPS data - which provides an initial location - with visual data captured in real time by the smartphone camera, then compares this information to a set of visual and geometric data from the environment stored in the cloud. Thanks to computer vision algorithms, the device can then determine its exact position and orientation, enabling very stable positioning of virtual elements in the camera view.

Evolution of Niantic Tools

Until recently, creating a VPS point with Niantic tools was done almost exclusively on iOS, ideally with a device equipped with a LiDAR sensor. The old Lightship scan app directly leveraged this depth data, combined with camera images, to produce a precise, textured 3D mesh in real time. This method provided excellent geometric accuracy, even in low-contrast environments, but it remained the preserve of a minority of high-end devices.

With the arrival of Scaniverse on Android, scanning relies on photogrammetry: a technique that reconstructs geometry by analyzing a series of images taken from different angles. The process first produces a point cloud - each point representing a position and color in space - which is then transformed into a mesh to be used in engines like Unity. More recent approaches, such as "Gaussian Splatting," make it possible to render these points directly as colored "splats," but for now, the AR ecosystem mainly relies on the mesh format to ensure compatibility and collision management.

This method change makes scanning accessible to a much wider range of devices, without needing a dedicated depth sensor. In return, it requires a bit more rigor in the field: slower movements, complete coverage of the subject, and good lighting conditions to ensure the model's accuracy and thus VPS reliability.

As a reminder, Niantic is the company behind the AR games Ingress and Pokémon GO. Founded within Google in 2010 and becoming independent in 2015, they opened their AR SDK to developers in 2021. In early 2024, Niantic sold their video game business (including Pokémon GO) to Scopely for $3.85 billion, now refocusing on geospatial AI under the name Niantic Spatial Inc.

The company has a clear goal:

“To become the leader in geospatial AI by developing spatial intelligence that helps people better understand, navigate, and interact with the physical world.”

Scan Method and Equipment

For this study, we scanned several sites in community mode with Scaniverse: the Xavier Bichat statue, the Vila Marguerite, the NDSE column (Notre Dame de la Sainte Espérance), a wooden turnstile, a turtle sculpture, a Roman column, as well as a stele and other local points of interest.

VPS relies on a large number of persistent "visual points." These points are most reliable when the system has images captured under varied conditions: in different lighting, at various times of day, and from multiple angles. By scanning a place repeatedly, you enrich the visual database, which greatly improves recognition and positioning stability.

We used Android smartphones: Google Pixel 7 Pro, Pixel 8 Pro, and Samsung S24.

Prototype Setup

After the statue was carefully scanned and its VPS point was activated by Niantic, we were able to integrate the resulting mesh into a Unity 3D project. Our goal was to measure both the speed of VPS detection and the accuracy of the placement of 3D objects, whether static or animated.

Next, we designed an AR scene showcasing the statue, with a virtual wall featuring an opening revealing the real background, a chain barrier encircling the plinth, and several strategically placed dynamic 3D elements. The tests showed that the virtual objects remained stably aligned, confirming the centimeter-level precision observed in the initial measurements.

Observed Results

In the field, positioning accuracy remained consistent, with a deviation of just a few centimeters from the target points. VPS lock-on occurred quickly, within one to three seconds, which is smooth enough to preserve immersion. We noticed slight visual instability upon the initial display, which disappeared as soon as the scene stabilized. To further improve the experience, we implemented a subtle transition during the first two seconds of AR startup - allowing the scene to configure and the rendering to become perfectly stable.

Challenges and Lessons Learned

One major challenge remains occlusion. Properly hiding a virtual object when a real element is in front of it is complex, especially in environments with dense vegetation or very fine details. In these settings, depth mapping can struggle to reconstruct exact outlines, leading to visual inconsistencies.

Finally, each site has its own unique aspects - lighting, background, structural shapes - which may require specific configuration to optimize rendering.

This experience confirms that performing multiple scans greatly improves VPS reliability. High-end devices not only allow for faster lock-on but also provide better texture definition, making visual recognition easier. Densely vegetated environments require an adapted occlusion strategy, and fine-tuning the pivots of virtual objects is essential to avoid floating effects.

Potential Applications

The possibilities for using the Niantic Spatial Platform are numerous. In the cultural field, it can enhance tourist visits by displaying contextual information or historical reconstructions directly on monuments. For entertainment, it paves the way for outdoor games and interactive treasure hunts. Brands can use it to create geolocated installations or marketing campaigns at strategic locations. Architects and urban planners will see it as a valuable tool for visualizing a project in a real-world setting, while in industry, it can serve for training and maintenance assistance.

Today, AR on smartphones is the main point of access. Tomorrow, with the democratization of AR glasses, geolocated experiences developed today could become a standard practice.