Snapchat Elevates Augmented Reality

Snapchat launched a basket of new features at its Partner Summit, and lowered the boom for AR.

I wasn’t expecting it.

When I give presentations about augmented reality I point to Snapchat Lenses, Pokemon Go and IKEA - the “usual suspects” when you’re trying to make the point that AR is already here.

So Snapchat should have been higher up on my list of things to watch. And so yesterday when they held their Partner Summit, I wasn’t really paying attention until news started popping up on Twitter (mind you, it was immediately drowned out by PS5 reactions):

The Rear-Facing Camera and AR Everywhere

The best way to sum up the announcements they made yesterday? The rear camera is now just as important as the selfie-creating one.

Now, you can try on virtual shoes, scan your dog (to find out what breed it is), and will soon be able to scan food products to find out their nutritional value.

But more broadly, Snapchat is bringing together three main pillars that are needed to supercharge AR everywhere:

The AR Cloud

Rich AR experiences that are persistent and shared need a ‘point cloud’: a shared reference model of the physical world so that objects can be precisely placed and seen through multiple devices.

Snapchat now has an “AR Cloud” which has been developed for major landmarks based on thousands of user photos. They showed a sample point cloud:

And it will be extended through the slow roll-out of Local Lenses. Snapchat is on its way to creating a digital twin of the entire world.

Machine Learning

Placing digital objects in the physical world is what AR is all about. But having those physical objects KNOW what they’re near is when the real magic will happen. For example, Unity MARS is providing tools to help make this happen. And now Snapchat is letting you bring your own ML models into its platform.

Embedded Everywhere

Finally, you want to bring AR into lots of experiences and places. And Snapchat is supercharging its presence elsewhere by letting developers both embed experiences within their own apps, and embedding their own apps into Snapchat.

Their Camera Kit, for example, can be used within iOS, Android or Web.

Elevating AR

Snapchat is elevating augmented reality. It will educate the larger industry on what AR can truly be, the value of point clouds and digital twins, and will provide tools that can be used outside of Snapchat itself to bring AR experiences to more apps and devices.

And it has me thinking: maybe it’s time to take a closer look at Spectacles and to start speculating on what a third generation might be able to, well, see.