WebAR available now

A million dollars, live forever, or travel back in time? WebAR is now accessible to everyone who has a mobile phone . Try it in Safari or Chrome https://apps.8thwall.com/8w/jini/ powered by 8th Wall

"Making your wildest dreams come true." - 🤖🧞‍♂️
WebAR is something I dreamt would be available next year. Are my dreams aren't crazy enough? Now we need to dream bigger.

Notes taken on a mobile device. Pardon any auto-corrections or incorrection.

G4C Brain Jam Presentations

g4c_xr_brain_jam_logo_1280x637.jpg

Video presentations of four of the 8 experiences created during the Brian Jam. The Brain Jam is part of Games 4 Change at Parsons School of Design in New York City.
It merges neuroscientists with engineers and game developers to create applications and expeirences that connect with the brain.

"Mind Bending  - a universal controller"

"Helping families with autism see eye to eye through augmented reality."

Screen Shot 2018-07-05 at 9.53.59 AM.png

[Video not embedding - view on YouTube here]
"SenseScape - an immersive VR sensory processing challenge that is intended to aid diagnosis and monitoring of concussion"
 

Me, myself, and I

 

Recorded via mobile. Please forgive the gorilla arm.

Notes taken on a mobile device. Pardon any auto-corrections or incorrection.

Games 4 Change - XR 4 Change Day Highlights

  • Vijay Ravindiran of FloreoTech.com created lessons and simulations for autistic children to practice interfacing in challenging, real life situations like crossing the road or with police. Great insight into how user testing informs product decisions such as adding logic in the system for simulations to have more or less steps depending on the child’s actions. Very well-thought-out experience from the logic in interactions, to required fidelity of a simulation, to the multi-user experience (where the kid is in the headset and the teacher is managing the test on a tablet - each with an adapted interface into the simulation). VR is not only for empathy. It can  impact people in a positive manner through practical (or magical) experiences. 

  • A wonderful 360 film piece My Africa funded by the Tiffany Foundation. I want to say it was the best linear storytelling I have experienced immersively. Raised in Africa, I may be biased to this film.

  • Brain Jam builds - functioning prototypes that explored the interplay between the brain and virtual or augmented experiences. Like exploring the implications of changing the camera view from 1st person to 3rd person to omniscient (planar view) to explore the potential cognitive impacts. Demonstrated in a simulation of collaborating with industrial robots. The brain jam produced 8 different builds (or simulations) built in 48 hours. Some exploring neural input like using EEG data via headband to control simulated physics (Mind bending). Another used eye tracking to encourage autistic children to make eye contact (beyond the silly “social media” lenses we currently use - this is the definition of a “social lens” encouraging and fostering real social interaction). 

  • Gaia Dempsy - cofounder of Daqari on a new hierarchy of needs. Revising Maslow’s hierarchy of needs (which was based on ethnographic studies of Blackfoot culture) based on new input form first nation tribal councils using xR to envision the future and pass down cultural knowledge. 

  • A more thorough origin of VR and guidelines for the future from Tom Furness who created developed cockpit systems and HUDs on HMDs. It gave pilots “the ability to do things that had never been done before.” Not because the display changed the physics of the aircraft, but because the it changed the perception of the aircraft and merged the pilot and the machine closer together. Spatial awareness (enemy location, your location and orientation) and state awareness (weapons systems, fuel, speed, etc.) merged into situational awareness that integrated into the pilot’s visual perception. 

    Private research from decades ago is still more advanced than what is on the market now. Which makes me wonder “What is being worked on now?”

 Tom Furness of the Virtual World Society displaying visualization methods where pointing at the God's Eye View can highlight the same point in the first person view HUD.

Tom Furness of the Virtual World Society displaying visualization methods where pointing at the God's Eye View can highlight the same point in the first person view HUD.

Notes taken on a mobile device. Pardon any auto-corrections or incorrection.

AI Rap Battle Bots

It is important to know how technology works if you are to design with or for that new technology. A hackathon is one way to explore new worlds.A hackathon is a context where it’s ok to break things without the expectation of creating any final output – but this hackathon had all of the ingredients for creatives to hack something new together.

rap-battlebots-team.png

Teaming up with R/GA’s Chatbot Authority Brad Jacobson and software engineer Sam Royston, we created a Rap Battle Bot. Training the Rap Battle Bot on thousands of lines of lyrics compiled across decades of rap allowed it to generate entirely new lines of poetry. Leveraging Reply.AI we created a chat interface on Facebook, so people can battle for title of best rapper, alive, dead, or digital. We ended the night with a live demo – a freestyle throwdown between R/GA's CTO and the Rap Battle Bot, leaving the crowd asking for an encore.


Team:
Sam Brewton - Rap Battle Experience Designer
Brad Jacobson - Strategist & Resident Chatbot Authority
Sam Royston - Machine Learning Engineer
Michael Hirsch - Data Scientist - Machine Learning

Photos by Annabel Ruddle 
TBT November 2016

 

Notes taken on a mobile device. Pardon any auto-corrections or incorrection.

Dancing Bear

Magic Lepa's digital eyewear was shown on Brazilian TV during the World Cup.

Alan Cooper and I are face palming, but are in awe.

Because it is a dancing bear projected into your eyes on the real, physical space in front of you. 

And Neymar responds to gestures and voice commands. 

There are other AR experiences for the World Cup ranging from Hololens experiences to Volumetric Replay using computer vision to render 3D displays from single source RGB cameras.

Screen Shot 2018-06-26 at 6.32.38 AM.png
upload.png

What we have now is a dancing bear.

That is Alan Cooper's metaphor. The bear doesn't dance well, but the town folk will come out to watch the bear dance and throw coins at its feet because it near magical that the lumbering bear can dance at all. This metaphor now applies to digital eyewear and most Augmented Reality. Because it does not work well. Either technically - the tracking takes a while to start as the system maps the space on a bright single textured surface. Or socially - you wear a heavy headset and look awkward as you carry around a tethered computing device and battery pack. But it is magical that it works at all. 

Cooper uses the metaphor of the poorly thought out keyless car entry system being a dancing bear. "The wonder isn't that he keyless entry system works well, but that they keyless entry system works at all. I am very willing to put up with the interaction problem in order to gain the benefit of remote entry to my vehicle."

The wonder isn’t that the keyless entry system works well, but that they keyless entry system works at all. I am very willing to put up with the interaction problem in order to gain the benefit of remote entry to my vehicle.
— Dr. Alan Cooper

Cooper then highlights how we have grown to put up with poorly thought out technological experiences to the point where we accept the collateral costs.  He equates these technology products by comparing them to a bad rescue ship."If you are stranded on a deserted island, you don't care much that your rescue ship is leaky, rat-infested hull."

“The difference between having a software solution for your problem and not having any solution is so great that we accept any hardship or difficulty that the solution might force on us.”
— Dr. Alan Cooper

We cannot just accept the poor interaction models of augmented reality experiences and spatial computing. The collateral cost is usability, high cognitive friction in launching and interacting with the experience, or social integration in long session use and low social adoption in long term diffusion. While it is seemingly magical they work at all, we need experiences that are more than dancing bears or leaky rescue ships.

Magic Leap Dancing Bear.gif

 

GIF via Business Insider  
Quotes via Cooper, Alan. The Inmates Are Running the Asylum: Why High-Tech Products Drive Us Crazy and How to Restore the Sanity. Sams, 1999.

 

Notes taken on a mobile device. Pardon any auto-corrections or incorrection.

New at R/GA

I am very happy to share that two of my students from the Interaction Design class I teach at the School of Visual Arts in New York City are now summer apprentices with R/GA. Xin An is at R/GA Shanghai. Laney Lynn is at R/GA LA.

After teaching one session on how to prototype AR & VR with web technologies, they went to town and created the works below.

XinMaze.gif
XinAn-Maze2.gif

Maze by Xin An
A VR Experience based on dreams.

LaneyLynn-GalleryVision.gif

Gallery Vision by Laney Lynn.
A business card enhanced through AR connected to a larger way of making the Museum of Art and Design more interactive.

I believe this shows the value of being able to prototype for AR and VR. When working on the cutting edge in a space where there are not many examples of best practices, it is essential to be able to work in a form of that medium and not be bound by legacy tools that cannot convey the foundational methods of experiencing the medium.

So good job and welcome two new team members to R/GA. 

Undergraduate Class: Interaction Design at SVA

Prototypes made in Ottifox.

Notes taken on a mobile device. Pardon any auto-corrections or incorrection.