What are the Ethical Considerations of Augmented Reality?


This time last year, I gave a talk on the ethics of VR at TOA Berlin. Because VR can be so immersive, I argued, we need ground rules about simulating situations that could be considered torture, and making sure we get explicit consent before putting people in headsets to show them content that could be triggering. This all seems fairly sensible and black-and-white, and there was a general sense of agreement in the room.

But what are the ethical considerations for augmented reality? As it stands now, that’s a trickier question — because AR is a layer on top of the physical world, at this point there is still some grounding for whatever the experience might be. Things will come to life and animate, but there is a still a sense that it’s make-believe; that’s not a knock against the tech, that’s just the way it is. But several recent conversations, as well as a talk I gave at Northside Festival last week and a great piece in Slate about augmented reality and property rights, are starting to raise some concerns.


At an ARCore workshop at Google last week, the moderators had us demo a fun app called “Just a Line.” You could “draw” in spaces in AR and then share those drawings with friends — seems harmless and fun, right? Ah, but this is the internet, my friends, and this is why we can’t have nice things. It’s super easy to imagine a world where this gets more popular, and you can’t walk by certain buildings without seeing hateful graffiti. Because the graffiti isn’t real, it’s almost more dangerous — we can confront things out in the open, at least, but if things are only visible to a handful of people in an augmented world, we have little hope of stopping them.

The Slate piece also raises an interesting point about consent and AR — if AR ads are projected on your home or office, for instance, what are your rights if you want them taken down? On the flipside, could you theoretically sell your blank spaces as territory for AR ads? And if AR ads are all individualized, as they might be in the future, what if you only want to show some ads and not others? Will huge companies outbid smaller players for this privilege, returning us to the status quo?


When Pokemon Go became a sensation two years ago, it had to grapple with some of these problems — people were playing the game at memorials and in cemeteries, as well as on busy roads. The team solved this by geo-fencing certain areas where the game wouldn’t work, and that seemed to repair things. But simply creating blocks won’t do much or address the bigger questions of how to remix the world responsibly as we move forward.

The worst case scenario is that of a digital Potemkin village, where people only see what they want to see, and can block out social ills. The internet and social media have pulled us in that direction already — we need to start having these conversations now so the AR won’t move us further in the wrong direction.

What Is Holding AR and VR Back?


Augmented World Expo was last week, and spending a few days wandering the halls of the convention center, you could easily come away with the notion that immersive media is the biggest thing since sliced bread. Talk after talk and panel after panel featured people rhapsodizing about the future of AR and VR, and on one hand, it certainly was exciting. It’s fun to hang out with folks just as passionate as you are about the next wave of storytelling and communication — but on the other hand, it can present attendees with a bit of a false positive. Because as much as many of us hate to admit it, AR and VR haven’t gone mainstream quite yet — and unless they start to move in that direction, we might be in deep trouble.

A few years ago I wrote a piece called the Music Startup Meltdown, where I posited the notion that music startup founders couldn’t grasp the idea that not everyone was as passionate about music as they were, because people tend of self-select social groups that reinforce their biases. And I feel like the AR/VR world is starting to go this way as well — there are so many events and conferences and meetups that it gets very easy to just hang with likeminded people and think everyone is as excited about the latest updates to ARCore as you are. But unless we start talking to the wider world, and getting them to buy in (and pay up), we’re going to get stuck.


At AWE, I was all set to give a talk about immersive storytelling, and at the last minute I threw my notes away and gave a talk about the two things we need to overcome to help AR and VR grow. The first is that we need to create a sense of urgency for brands around adoption of immersive tech — right now it is a “nice to have,” but not something that everyone needs for every campaign. This is a long process, although luckily there is a growing body of research that backs up the effectiveness of AR for advertising (VR for ads is going to be a much longer term play). We need to start defining real ROI and showing exactly how AR can pay off from a financial point of view. If we just stick to the art and magic side of things, I feel we won’t get very far.

The second thing that holds us back is bad content and people who don’t know what they’re doing. I gave an example of a project that cost a million dollars, took two years, and resulted in a few articles and some free downloads, but nothing of real value for the brand. Those of us who are experts in the space need to be much more vocal about what makes content good and get brands to listen — very few brands and agencies have any in-house talent at this point, and they need to start hiring now. If we create a bad impression and brands don’t see returns, they won’t want to continue creating in the space, and we’re all pretty screwed.


The bottom line is this: spending time with likeminded technologists and creatives is great and fun, but unless we start broadening the conversation, events like AWE will move from filling convention centers to the backrooms of coffeeshops. If we can’t convince brands and agencies to start investing, we’ll run out gas before we even hit the road.

Augmenting Art and Activism


Museums are now filled with people looking at art through their phones, but one collective in New York has taken the concept to another level. Visitors to the Jackson Pollack gallery at MoMA in NYC can download the MoMAR appand see a layer of virtual art on top of the iconic paintings — it’s the first in what will hopefully be an explosion of people using AR to remix classic works and add commentary as a digital layer on top of the physical world.

The idea of using AR for activism is still fairly new, but if the technology allows us to reshape the world we see, there’s no reason we can use it to try to educate others about social issues. Gabo Arora, best known as an award-winning VR director, created an AR piece that allowed users to learn more about a civil rights leader in Baltimore while exploring the neighborhood where she did most of her work. Other potential use cases could involve reclaiming spaces — if you can’t remove a Confederate statue, for instance, you could encourage people to use AR to overlay statues of more positive figures and share those.

As cities change and morph, AR could also be used to make sure history isn’t forgotten. Many neighborhoods have gentrified and those changes have meant erasure for some communities; using AR to allow viewers to project images of what once existed is a way to confront history. In New York, it would help preserve the identities of what were once vibrant arts and ethnic communities and are now a sea of Duane Reades and bank branches.


In terms of art, AR could almost be the seen as the next wave of the Guerrilla Girls movement, which used activist tactics to call attention to the lack of female artists in major museums. If you can’t get real art in a space, adding it virtually might be the next best thing. As more people use AR to customize what they see, patrons could potentially create their own museums out of blank spaces, and then curate virtual spaces for others.

Of course, the downside of all this is that AR could be used to spread messages of hate and intolerance. It’s one thing to remix art, but it’s another thing to alter the message or destroy it, even if the destruction is virtual. Graffiti on the side of a temple or mosque in an AR world presents a different challenge than graffiti in the real world, but those messages could be hidden and if most people don’t see them, the hate could keep hiding in the shadows.

Despite this, augmented reality provides real opportunities to bring remix culture to a mass audience. We’re more able to customize our realities now than we’ve ever been, and AR will take this ten steps further. In a few years it’ll be common for us to use AR to beautify and change our world, and hopefully usher in a new wave of education and growth.

Augmented reality — the next wave of reporting


The following is based on a workshop I did at the Associated Press a few weeks ago, and comments from a Matter.vc panel I spoke on last month.

While a number of media outlets have experimented with virtual reality and 360 video, far fewer have used augmented reality as a storytelling device. The New York Times has launched a few features, including a piece on Olympic athletes and another on David Bowie; the BBC launched a companion app for Civilizations; and USA Today built an AR experience around a rocket launch. But more news outlets should start playing in the space, as AR is far more readily accessible to the average consumer and can used to illustrate stories in a way that helps visual learners see data and helps further the understanding of complex topics.


Take a health related news piece, for instance a feature about heart disease. Although it is a topic that touches almost every family, too few people are educated enough to understand how the heart really works and how disease manifests, and words on a page and flat illustrations can only go so far. Now imagine that an image of the heart can be placed in a space, blown up, explored, and pulled apart and then put back together, with detailed descriptions of exactly how everything works. Especially for younger readers, who have grown up on far more visual content than printed matter, this can help tell the story in a new way.

A more data driven story can also find a new audience with AR features. Charts and pie graphs can make even the most compelling information seem deathly dull, and augmented reality allows readers to interact with the data and visualize it in new ways, especially if it can be presented as a layer on top of the physical world for context. Site-specific local reporting could be fascinating and have a long shelf life, and keep new people returning to a location and extend the reach of an article.


For many people, reporting on environmental issues feels far removed from their daily reality — we all complain about the wonky weather, but almost no one has been able to explore melting polar ice caps, for example. Allowing readers to put them in an intimate space and pull them apart to explore makes something that was once theoretical feel more tangible and real, and can spur action or more learning.

While the upfront costs of creating AR assets is an expense, many of them will wind up being reusable with minor tweaking — the equivalent of a great stock photo that gets used over and over. For now, these assets have to live in an app; given that many news sites already have a designated app where users consume content, this isn’t a major hurdle. In a few months, augmented reality will be able to be viewed via a web browser, and then anyone with a link and smartphone can partake in the content.

Augmented reality will fundamentally transform journalism as we know it, and we’ve barely started to scratch the surface. If you’re interested in learning more, reach out via the Friends With Holograms site.

Augmented Reality and the Death of the Billboard


If you find yourself with some spare time and want an odd challenge, try counting how many mass-consumer ads you see every day. Exclude the micro-targeted online ads that seem to defy logic (Amazon: just because I searched for garbage cans doesn’t mean I want to only buy garbage cans from here on out) and focus on the billboards and subway posters and ads on the sides of long defunct telephone boxes. The numbers start to mount quickly. Now, think of how many made an impression. Sure, there might have been one reminding you that a favorite TV show is coming back, or a new product from a brand you like is in stores — but the vast majority were just wasted space, visual pollution in an already crowded landscape.

For years we’ve lived with this because no other alternatives existed. But augmented reality now has the power to take all those online recommendations into the real world, and radically change the way we consume information and design cities. Imagine Times Square without the pulsing, glowing neon orgy of branding and consumption. Visualize a world with white space, nature, and public art, instead of ads for things you probably don’t want or need.


This transition will start on our phones and then explode as we shift to the AR glasses predicted to reach consumers in a few years. The way it works is pretty simple — as you use AR to navigate through your day, you’ll be served contextual, relevant, interactive AR experiences customized just for you. When you pass a store window, for instance, you’ll see mannequins that reflect your body type, price range, and style profile — so if you’re a thirtysomething professional woman, like me, you’ll see one set out of outfits, and the twenty year old club kid will see another.

Ditto for food products and restaurant recommendations. I haven’t eaten fast food in years and I’m likely never to again, unless I find myself trapped in an airport and starving. So McDonalds is wasting money every time they advertise to me, because I’m a lost cause. But there are plenty of other food brands I love and would be happy to hear about and consume, and they’ll have more of a voice in the new and scaleable world of AR.


When everyone is being served content they’re likely to respond to, ROI will be exponentially higher and consumers will be much more engaged. AR content already has an average dwell time of 75 seconds and much higher retention rates, so users are likely to remember what they’ve seen and act accordingly.

Beyond that, think of all the public space that will be freed up. Blank space will be made available as a spot to simply rest your eyes and take a break. There will be room for art and commentary and trees and flowers, places where people can relax and meditate and learn. We’re still a ways out from this new reality, but advertisers need to start preparing, lest they get left behind when the shift happens.