Blogmented Reality

So today’s video illustrates two points that have been made previously on the blog. The first is a throw away Bruce Sterling quote that one of the ways you’ll know AR has made it and isn’t just a fad is that you’ll be able to use AR interfaces to construct other things in Augmented Reality spaces. Well, we’re doing this man. We’re making this happen.

The second point is, why is none of the cool AR happening in the states? Certainly not in any of our educational institutions. It really makes you wonder what’s going on with the various interface and tech research labs that we have at various universities around the United States that every time you see a really impressive demo like this it tends to have a .au or a .jp or a .kr or half a dozen European country web codes. What exactly are we missing here?

Presented without commentary as I’ve already discussed most of the ideas this brings up elsewhere on the blog. Really underrated movie though, and not just because of the quasi-AR intro.

Okay so I lied, this is not actually a dance project really.

.txt interactive digital performance – excerpt#6 from ponto txt on Vimeo.

But it is close enough that I really want to see some genuine AR dance projects. I’m talking people traipsing through fields of information like this and not just disrupting it but creating it with their motion. There’s something about the artform of dance that lends itself especially well to augmented reality, and I think it’s the sense of negotiating between actual real space and information. If we’re overlaying our world with the platonic world of information (and I swear one of these days I’ll sit down and explain exactly what I mean by that) then there’s going to be a LOT of spacial negotiation. Unlike the traditional internet there are limitations to the amount of data per square foot that people are going to knock up against, hard. And dance, just like synesthesia, is going to be a great road-map for how we do that spacial negotiation.

So it’s a bit unfair to single out System Flaw, a fun looking game with a fairly slick interface for this criticism but let’s take a moment to look at their demo video that is indicative of a general problem with mobile AR right now.

Don’t we all still look a little rediculous starting into our tiny screens and acting like the world we see though it is oh so much more important than the rest of the world? It’s like crackberry addiction with the added bonus of embarrassing movements. The interesting question to watch AR answer (or not) over the next few years is how they’re going to deal with that embarrassment. Is mobile AR just going to collapse under the weight of how silly we look using AR interfaces until we get heads-up displays or some other interface breakthrough that makes us look less silly, or is it just going to be so ubiquitous that we’re forced to accept it?

This week in augmented reality mashups: Projected AR gaming

Pinwall | interactive facade pinball | urban screening from urbanscreen on Vimeo.

I honestly can’t tell how things are going to go with projected AR. Is the novelty going to wear off if it’s overused or are we going to start having some sort of projection mapping project going on in any public space anybody can get their hands on? At this point it’s still a difficult enough process to scare away the uninitiated, but I can’t imagine that lasting much longer. Especially if people are introduced to it as super fun AR games on the side of buildings.

One of the reasons I personally find AR so fascinating is because it’s been hiding in plain sight. Even though it seems like it’s popped up out of nowhere it’s actually all over the place we just haven’t thought of it as AR before. Take these HP ads that have been airing for quite some time now. I honestly hadn’t really thought of them as AR fiction until I stumbled across them today in an attempt to find some illustrations for an AR Fiction Weekend post about something totally different.

I picked this Pharrell ad for two reasons, one mundane and one theoretical. The mundane reason is that out of all of these HP ads Pharrell is the one who’s most clearly using some sort of gestural interface to control his little AR fiction universe. Which, if you think about it, is actually kinda insane. When you watch the other ads you’ll see pictures and other documents pop up as if by magic the moment their owner thinks about them, but Pharrell (or the director of this spot more likely) has created an interface limitation for a pretend computer system. They’re striving for some sort of near-future AR realism aesthetic.

The second reason is the concept of AR as synesthesia. Pharrell says he sees music and in the context of the ad we see it too. AR is going to allow people to literally look at the world in a different way and if we’re going to accept that as one of our central metaphors then why shouldn’t we take design cues from synesthetes, a real world example of people who see information that isn’t “really” there overlayed onto the real world?

Of course, in synesthesia the information we’re overlaying is limited to sensory data whereas in AR we’ve got basically any information we feel like but that’s a whole different post.

Haptics! Now that we can see digital stuff scattered around our environment and hear digital stuff scattered around our environment it’s time to be able to touch it.

There’s something sorta tragic about this video though. The way the virtual Teddy Bear constantly reaches out for contact when it’s not being interacted with and then bumps around like it’s being beaten when the interaction does come. It’s the sad half life of an object of affection still in beta.

It’s not technically difficult to build gloves with a whole bunch of these things built in. In fact it’s probably a more modest goal than a good heads-up AR display. But right now it feels like we’re in a kind of uncanny valley of virtual objects where the object is real enough to inspire emotion but not interactive enough to offer any kind of emotional resolution. So you end up with the world’s saddest virtual teddy bear or some geolocative hellscape of 5000 t-rexs that just stand around roaring while you’re trying to visit the golden gate bridge.

I was almost tempted to make this another edge-case AR Fiction weekend post. Augmented Reality Tourism makes up a significant subplot in Bruce Sterling’s recent The Caryatids. So I was not entirely surprised to find this video on his blog this morning.

Bruce points out a lot of the important points in his entry about this, for instance, HEY this is monetizable unlike the entire rest of the internet. But one thing I want to point out about this is how much it all seems like entertainment and not education. There’s something decidedly not academic about AR, and that’s not just because barely anybody in the academy is looking into it. If there’s traditional academic rigor in AR it’s certainly stuff like this, and yet it doesn’t feel like the academy. It feels like Disney Land. And I somehow doubt that’s just the result of growing pains. When we see something that looks like academic AR I don’t think it’s going to look the way we imagine AR or the way we imagine academic networks. I think it’s going to end up looking a lot more like the invisible college. Honestly, even I’m not sure if I mean the abstract concept or the historical institution.

Finally, I’m once again making good on my promise of terrible terrible puns as we discuss heads up displays for AR and how, for now at least, you’ll only be seeing good ones in the AR Fiction Weekend posts. Here, have a slideshow about it.

Just like last time I posted a slide show, you can find a lot more supplementary material at the original site I found the sideshow embedded in. But here’s the general gist: heads up displays like the kind you see in Dennou Coil or other really cool AR fiction are still a long way out. But they AREN’T held back by unsurpassable technical hurdles like flying cars or jet-packs. Instead, most of the basic traits of wearable, functional, dare I say fashionable, AR glasses have been shown to work in proof of concept products like the ones you see in that slide show. The question is how long it will take to integrate those traits into something that can be made available at a reasonable price. I’ve seen 2015 thrown around a lot as a date for that, which seems about right to me.

So we’re stretching the definition of AR fiction this weekend because the AR is real but it’s being used with a fictional band. For those that don’t know, the Gorillaz are a collaboration between Damon Albarn of Blur and comic artist Jamie Hewlett. The idea is that Albarn and whoever he feels like working with create new music that is then attributed to the 2D band drawn by Hewlett. As you can imagine this makes touring a somewhat strange proposition as the band you’re going to see live is not actually the band making the music. Albarn and Hewlett have tried several variations of touring to solve this problem but the version I’m going to show you today is a strange combination of projection mapping and an old theme park trick called Pepper’s Ghost.

I show you that specific video not because it’s the best example of the technology being demoed but because the camera operator is so obsessed with showing it off as a tech demo, almost to the determent of the actual “performance.” Watch how the camera repeatedly goes out of focus and uses the audience to block out the characters for short periods. None of that’s useful thematically it’s to show you that what’s being projected on stage is actually physically there in some sense. That they didn’t just 3D animate the characters and throw them into the broadcast but instead they threw them onto the stage and then broadcast THAT. It’s a great example of the kind of fascination people tend to have with a new medium and the challenges it presents. Also worth looking at is the mix of live performance and the pre-animated characters about halfway in.