The dream of graffiti artists everywhere is now a reality – vandals have daubed the whole of Carnaby Street in red and blue paint.
Luckily, this vandalism is easy to clean and totally invisible. Today, Snapchat has launched Local Lenses – a new feature that is one of the first persistent, large scale, collaborative uses of augmented reality.
The technology was first announced in June, and promised to transform whole neighbourhoods into digital canvases. It has now been launched in a small area around London’s Carnaby Street.
This first Local Lens is called City Painter. The concept is simple – using the camera on the Snapchat app to view Carnaby Street, City Painter lets you spray great fountains of red and blue paint above Carnaby’s shops, decorating the bricks with predesigned graffiti murals. Anyone on the street can see others spraying their paint, and users can battle to cover the shops in their colours.
Local Lenses are the latest app from Snap's camera engineering team based in London, who also created Landmarkers. Landmarkers was the company’s first foray into AR at scale, allowing people to place user-created lenes onto famous landmarks – the Eiffel Tower shot rainbows, for instance, and cute characters appeared above Buckingham Palace.
Local Lens is more complex, says Qi Pan, senior manager of research engineering at Snap London. There was far less data to work with – 3D data of prominent public landmarks was easy to obtain, whereas Carnaby Street – which was chosen as a fully pedestrianised area where users could aim their phone without fear of traffic, and which also happens to be right next to Snap’s London offices – was a different animal. “It’s very unlikely that people are going to be taking photos of every single part of a street, just because, generally, there’s not necessarily something interesting on every single part of any street,” he explains.
This raises another difficulty – the sheer size of the space. Landmarkers involved users pointing their camera at a single fixed landmark, generally from a single, postcard-like point of view.
City Painter, on the other hand, involved 3D mapping the whole of Carnaby Street, so that users could paint it from any angle. The company had to extract visual data from several sources, including analysing public Snaps of Carnaby Street shared by users. “For Local Lenses, we’re leveraging 360-degree camera imagery,” says Pan. “Someone can just walk down the street to map it, as well as combining that with any public story Snaps that we may have of the area.”
City Painter’s second novelty is that the experience is shared. In Landmarkers, everyone was limited to their own augmented reality – what you saw on your phone was different from what everyone else was seeing. Local Lens, in contrast, is persistent and synchronous.
“We have a single shared reality,” says Pan. “And when you do something to this world, someone else can see that result almost instantly. Those changes also persist if everyone leaves the experience and new people pop up the next day – they can see the space that’s been altered by yourself and others.”
Local Lenses represents the latest AR project for Snap, including last year’s third iteration of its smartglassess Spectacles. Snap claims that there have already been over one million lenses created using Lens Studio, the company’s publicly available tool for creating augmented reality experiences, which Snapchatters have played with billions of times, and more than 75 per cent of Snap’s daily active users interact with AR every day. All of this activity, explains Pan, trains the Snapchat camera. “We want the Snapchat camera to really understand and be intelligent about what it sees in the world.”
As the camera improves, its applications will grow, from advertising to video games, to live experiences, to mapping and directions. City Painter represents a basic example in the development of a ‘mirror world’ – the idea that everty street, building and room in a city will have a digital twin in the mirror world. Virtual experiences might be built around whole neighbourhoods, or entire digital versions of London, for example.
“We envisage doing new things that couldn’t be done before, for example AR tours of neighbourhoods guided by famous people who used to live there, leaving notes, photos and videos for your friends to discover later, or playing immersive AR games together with your friends, altering the shared digital space,” says Pan. “We’re also coming up to Halloween, in the future, we could imagine a total transformation of Carnaby Street to make it feel spooky.”
Local Lenses and City Painter launch today on Carnaby Street in London. People near Carnaby Street with Snapchat open will see a marker appear on their Snap Map letting them know the experience is there.
Updated October 8, 2020 11.39 BST: This article was updated to clarify that Local Lenses were built by Snap's camera platform team in London
Will Bedingfield is a staff writer for WIRED. He tweets from @WillBedingfield
This article was originally published by WIRED UK
