Even the most tech-averse among us have probably heard of augmented reality (or AR) – if not because we’ve dabbled in a spot of Pokémon Go ourselves, then perhaps because tech giants like Facebook and Apple are heavily investing in the technology. And like it or not, where they go, the world tends to follow.
So for our 14th Campfire, we wanted to find out more about the opportunities and repercussions AR might have for the world of experience design.
And who better to open our eyes to the new dimensions on offer than Rob Morgan: a game writer, immersion designer and AR specialist who has designed AR PlayStation games with JK Rowling, created a location-based grime hip-hop opera (hip-hopera?), and worked with the National Gallery to pivot during the pandemic using the power of AR.
Here are eight lessons we learned from Rob’s Firestarter Talk – and to find out what our Campfire made of it all, head to Campfire 14: The Sparks.
1. Punchdrunk meets Pokémon Go – possible in AR
Question: What happens when you combine the real-world set dressing and actors of an immersive theatre spectacle like Punchdrunk with the location-based layers of an AR game like Pokémon Go?
Answer: Multiple narratives unfolding in one space, blending the best of the physical and digital realms.
Rob gave an example of a location-based AR hip-hop grime opera called Consequences Greater Than Collaboration, in which a branching narrative was manifested inside a specific place using location-triggered audio content powered by iBeacons. As participants explore the space, they’re tracked from room to room and able to make narrative decisions with their feet, with the consequence of their decisions following them through the story.
Interestingly, Rob explained that he likes these experiences to end on a communal musical moment regardless of these branching narratives. This reminds us of Claus Raasted making the same point about his LARPing adventures all the way back in Campfire 1, suggesting that technology is just a tool to help us explore and create the most effective narratives for transformation.
2. In AR, the most important thing to augment is the player.
On the difference between VR and AR, Rob said:
“In VR, you can override the player’s senses and create an entirely new world around them. But in AR, you can’t override what the player sees. Instead, in AR the user is the center of the experience: they are their own protagonist. And the most important reality that you are augmenting is the player themselves.”
In other words, as we explored in Campfire 13, in AR you can’t tell someone that they’re in a dragon cave; but you can tell them that they’re a dragon in disguise.
“Even though 95% of what the player perceives in an AR experience is going to be the real world seasoned with a digital narrative, it can still be more compelling, because you have direct access to activate the players’ real emotions out in the real world, where the emotional and social stakes – and the rewards – are high.”
3. Ambient storytelling creates customized realities.
Rob pointed out that humans are able to experience different stories in the same spaces.
“You can ambiently experience a story that’s secretively happening inside your own head, but still be able to engage with the space around you in a normal fashion without colliding with other people. This unlocks the possibility of multiple, parallel, contextual layers that allow different users to receive very different information in the same space – and use it in radically different ways.”
The implications for this are both exciting and scary. As we bring the tools of the internet out onto the streets, we face the very real dilemma of being able to filter out what we don’t want to see.
“It’s already hard enough to agree on what’s real and what isn’t. And the internet has made that harder, not easier. The very idea of a base consensus reality that we can all agree on might very soon seem pretty old fashioned.”
4. Non-zero sum information design allows us to use unlimited digital “hyper labels” on physical objects.
“Zero sum” means there’s an upper limit on information – but in AR, you’re able to edit and add things to the digital dimensions within real spaces. And there’s no upper limit on how much you can do that – no wall space or word count. This means you can also select and filter that information just like you do in a web browser, allowing for unlimited amounts of linked information.
“In AR the only limitation is the interface you use to access the information. Once you can create a digital reference point for a real object, there’s no limit to the number of labels you can associate with that object in the digital layer. You can create and curate connections between those labels in the form of what I’ve started calling ‘hyper labels’.”
5. The museum of the future will not be a single-channel public broadcaster, but a gateway to multiple perspectives.
AR might also offer a solution to museums trying to decolonise their collections, as well as appeal to younger digital natives who want to be able to self-curate their experience.
“Non-zero sum information design means AR could provide an alternative to the traditional challenge of the museum label-writing process. It could provide a larger space on which to inscribe information. And it could allow us to play around with multi-channel curation.”
Rob’s work with the National Gallery, National Portrait Gallery, Royal Academy and Sky Arts, which placed colourful AR markers on landmarks around the West End – as profiled by the New York Times in Using Augmented Reality, London Takes Its Art to the Streets – is a great example of this potential. When people view these markers through an app on their device, they can choose from several “channels” of snackable content giving different perspectives on the artworks, from gallery curators to entertainers like Stephen Mangan or James Acaster.
6. In this new era of digital placemaking, the narrative is controlled by everyone, and no one.
In this new realm, public spaces could become nexuses for different, non-competing perspectives, not disputed territory for who gets to define the sole narrative. Rather than removing statues, we could layer unlimited perspectives on them, for example.
“An installation or destination could be a site of public participation in the ongoing process of creating culture and interpreting art: a kind of fifth plinth for culture and art commentary. This might include users being able to leave their own messages tagged in the digital margin of reality.
At this point, we have to start asking, ‘What is the difference between artwork and graffiti? Is it the artist’s intent? Or who owns the wall that the artwork is painted onto?’ Because both intent and ownership of public space are bound to become much less stable in the digital dimension.”
According to Rob the answer lies in not trying to assert ownership over the digital dimension, but embrace becoming a “polylith”.
“Instead, provide insight into your digital dimension, as well as accessibility, navigability, and relevance. And for today’s audiences, relevance means offering them more than a sole pillar of content.”
7. Huge potential for cacophony: curation more important than ever.
There’s a danger that this unlimited number of perspectives could lead to information overload – but Rob believes curation is the solution to this.
“If the digital lab becomes a wild west of disordered content and piracy, that’ll give me nostalgia for the internet in the 2000s. I pirated all my entertainment then, and so did everyone I know. But once Netflix came along, the vast majority of consumers were more than happy to pay for a convenient, quality product. Because consumers don’t want cacophony.
Quality, convenient, targeted curation of content in the digital layer is more important than ever. For venues and brands, figuring out how to take control of the digital dimensions of their spaces is going to be the next great challenge. Because if they don’t, people will just use their spaces in their own digital context anyway, and in ways they can’t access or even be aware of.”
8. With AR, brands can follow their customers home.
It might sound a tad Black Mirror, but the blend of physical and digital dimensions could extend the reach of experience designers and brands into people’s everyday lives.
“In the coming era, if you have a physical premises, that also means you have a very powerful digital platform. This presents an opportunity to extend your reach and expand your digital footprint out beyond the physical limitations of those premises and into users’ own contexts.
What changed the games industry was mobile games after smartphones came out about 12 years ago: small games that fit in people’s pockets and fit around their lives. I believe placemaking in AR has the potential to cause a similar sort of revolution and fundamentally change the way that we walk down the street.”
Want to know how our Campfire responded to Rob’s Firestarter talk? Read more about the sparks that flew afterwards at Campfire 14: The Sparks.
