Augmented reality demos hint at the future of immersion

Twit­ter is awash with impres­sive demos of aug­ment­ed real­i­ty using Apple’s ARK­it or Google’s ARCore. I think it’s cool that there’s a pal­pa­ble sense of excite­ment around AR—I’m pret­ty excit­ed about it myself—but I think that there’s per­haps a lit­tle too much ear­ly hype, and that what the demos don’t show is per­haps more sug­ges­tive of the gen­uine­ly excit­ing future of AR.

Below is an exam­ple of the demos I’m talk­ing about — a mock­up of an AR menu that shows each of the indi­vid­ual dish­es as a ren­dered 3D mod­el, dig­i­tal­ly placed into the envi­ron­ment (and I want to make clear I’m gen­uine­ly not pick­ing on this, just using it as an illus­tra­tion):

This rais­es a few ques­tions, not least around deliv­ery. As a cus­tomer of this restau­rant, how do I access these mod­els? Do I have to down­load an app for the restau­rant? Is it a WebAR expe­ri­ence that I see by fol­low­ing  a URL?

There’s so much still to be defined about future AR plat­forms. Ben Evans’ post, The First Decade of Aug­ment­ed Real­i­ty, grap­ples with a lot of the issues of how AR con­tent will be deliv­ered and accessed:

Do I stand out­side a restau­rant and say ‘Hey Foursquare, is this any good?’ or does the device’s OS do that auto­mat­i­cal­ly? How is this bro­kered — by the OS, the ser­vices that you’ve added or by a sin­gle ‘Google Brain’ in the cloud?

The demo also rais­es impor­tant ques­tions about util­i­ty; for exam­ple, why is see­ing a 3D mod­el of your food on a table bet­ter than see­ing a 3D mod­el in the web page you vis­it, or the app you down­load? Or, why is it bet­ter even than see­ing a reg­u­lar pho­to, or just read­ing the descrip­tion on the menu? Do you get more infor­ma­tion from see­ing a mod­el in AR than from any oth­er medi­um?

Matt Mies­niks’ essay, the prod­uct design chal­lenges of AR on smart­phones, details what’s nec­es­sary to make AR tru­ly use­ful, and it pro­ceeds from a very fun­da­men­tal basis:

The sim­ple ques­tion “Why do this in AR, wouldn’t a reg­u­lar app be bet­ter for the user?” is often enough to cause a rethink of the entire premise.

And a series of tweets by Steven John­son nails the issue with a lot of the demos we’re see­ing:

Again, I’m not set­ting out to crit­i­cise the demos; I think exper­i­men­ta­tion is crit­i­cal to the devel­op­ment of a new technology—even if, as Mies­nieks points out in a sep­a­rate essay, a lot of this exper­i­men­ta­tion has already hap­pened before

I’m see­ing lots of ARK­it demos that I saw 4 years ago built on Vufo­ria and 4 years before that on Layar. Devel­op­ers are re-learn­ing the same lessons, but at much greater scale.

But plac­ing 3D objects into phys­i­cal scenes is just one nar­row facet of the greater poten­tial of AR. When we can extract spa­cial data and infor­ma­tion from an image, and also manip­u­late that image dig­i­tal­ly, aug­ment­ed real­i­ty becomes some­thing much more inter­est­ing.

In Matthew Panzarino’s review of the new iPhones he talks about the Por­trait Light­ing feature—which uses machine learn­ing smarts to cre­ate stu­dio-style photography—as aug­ment­ed real­i­ty. And it is.

AR isn’t just putting a vir­tu­al bird on it or drop­ping an Ikea couch into your liv­ing room. It’s alter­ing the fab­ric of real­i­ty to enhance, remove or aug­ment it.

The AR demos we’re see­ing now are fun and some­times impres­sive, but my intu­ition is that they’re not real­ly rep­re­sen­ta­tive of what AR will even­tu­al­ly be, and there are going to be a few inter­est­ing years until we start to see that revealed.

Also pub­lished on Medi­um.