Games 4 Change - XR 4 Change Day Highlights

  • Vijay Ravindiran of created lessons and simulations for autistic children to practice interfacing in challenging, real life situations like crossing the road or with police. Great insight into how user testing informs product decisions such as adding logic in the system for simulations to have more or less steps depending on the child’s actions. Very well-thought-out experience from the logic in interactions, to required fidelity of a simulation, to the multi-user experience (where the kid is in the headset and the teacher is managing the test on a tablet - each with an adapted interface into the simulation). VR is not only for empathy. It can  impact people in a positive manner through practical (or magical) experiences. 

  • A wonderful 360 film piece My Africa funded by the Tiffany Foundation. I want to say it was the best linear storytelling I have experienced immersively. Raised in Africa, I may be biased to this film.

  • Brain Jam builds - functioning prototypes that explored the interplay between the brain and virtual or augmented experiences. Like exploring the implications of changing the camera view from 1st person to 3rd person to omniscient (planar view) to explore the potential cognitive impacts. Demonstrated in a simulation of collaborating with industrial robots. The brain jam produced 8 different builds (or simulations) built in 48 hours. Some exploring neural input like using EEG data via headband to control simulated physics (Mind bending). Another used eye tracking to encourage autistic children to make eye contact (beyond the silly “social media” lenses we currently use - this is the definition of a “social lens” encouraging and fostering real social interaction). 

  • Gaia Dempsy - cofounder of Daqari on a new hierarchy of needs. Revising Maslow’s hierarchy of needs (which was based on ethnographic studies of Blackfoot culture) based on new input form first nation tribal councils using xR to envision the future and pass down cultural knowledge. 

  • A more thorough origin of VR and guidelines for the future from Tom Furness who created developed cockpit systems and HUDs on HMDs. It gave pilots “the ability to do things that had never been done before.” Not because the display changed the physics of the aircraft, but because the it changed the perception of the aircraft and merged the pilot and the machine closer together. Spatial awareness (enemy location, your location and orientation) and state awareness (weapons systems, fuel, speed, etc.) merged into situational awareness that integrated into the pilot’s visual perception. 

    Private research from decades ago is still more advanced than what is on the market now. Which makes me wonder “What is being worked on now?”

Tom Furness of the Virtual World Society displaying visualization methods where pointing at the God's Eye View can highlight the same point in the first person view HUD.

Tom Furness of the Virtual World Society displaying visualization methods where pointing at the God's Eye View can highlight the same point in the first person view HUD.

Notes taken on a mobile device. Pardon any auto-corrections or incorrection.

AI Rap Battle Bots

It is important to know how technology works if you are to design with or for that new technology. A hackathon is one way to explore new worlds.A hackathon is a context where it’s ok to break things without the expectation of creating any final output – but this hackathon had all of the ingredients for creatives to hack something new together.


Teaming up with R/GA’s Chatbot Authority Brad Jacobson and software engineer Sam Royston, we created a Rap Battle Bot. Training the Rap Battle Bot on thousands of lines of lyrics compiled across decades of rap allowed it to generate entirely new lines of poetry. Leveraging Reply.AI we created a chat interface on Facebook, so people can battle for title of best rapper, alive, dead, or digital. We ended the night with a live demo – a freestyle throwdown between R/GA's CTO and the Rap Battle Bot, leaving the crowd asking for an encore.

Sam Brewton - Rap Battle Experience Designer
Brad Jacobson - Strategist & Resident Chatbot Authority
Sam Royston - Machine Learning Engineer
Michael Hirsch - Data Scientist - Machine Learning

Photos by Annabel Ruddle 
TBT November 2016


Notes taken on a mobile device. Pardon any auto-corrections or incorrection.

New at R/GA

I am very happy to share that two of my students from the Interaction Design class I teach at the School of Visual Arts in New York City are now summer apprentices with R/GA. Xin An is at R/GA Shanghai. Laney Lynn is at R/GA LA.

After teaching one session on how to prototype AR & VR with web technologies, they went to town and created the works below.


Maze by Xin An
A VR Experience based on dreams.


Gallery Vision by Laney Lynn.
A business card enhanced through AR connected to a larger way of making the Museum of Art and Design more interactive.

I believe this shows the value of being able to prototype for AR and VR. When working on the cutting edge in a space where there are not many examples of best practices, it is essential to be able to work in a form of that medium and not be bound by legacy tools that cannot convey the foundational methods of experiencing the medium.

So good job and welcome two new team members to R/GA. 

Undergraduate Class: Interaction Design at SVA

Prototypes made in Ottifox.

Notes taken on a mobile device. Pardon any auto-corrections or incorrection.

Thinking in Pictures

I ran across this book in the kid's table of the Strand Book Store.

It is about a girl who thinks in pictures.

Grandin outlines three different minds

Screen Shot 2018-05-29 at 11.04.16 PM.png

1. "Photo Realistic Visual Thinkers
   - Poor at algebra"

2. "Pattern Thinkers
   - music and math"

3. "Verbal Mind
    - poor at drawing."
These appear to be a select section of Gardner's Theory of Multiple Intelligences

Unique traits for minds that are closer towards Aspergers or autism:

- " The autistic mind tends to be a specialist mind"

- The trade off between cognitive thinking and social interaction.
           Or strengths weighted to one of those areas.

- Sensory issue

-  Strong ability to categorize
with an example of quicky being able to define problems as equipment problems or personelle problems.

- Fixation

The primary visual cortex of a control brain (age, gender control) versus her brain.

“But one of the things that I was able to do in my design work is I could test-run a piece of equipment in my mind, just like a virtual reality computer system.”

Comparing her mind to a "VR computer system" is coincidental.
This was in 2010, before any consumer VR system of this millennium was even in dev kit.
But it is related to how we all think because we live in moving, volumetric system.

We think in imagery.
We test-run or simulate scenarios and processes in our mind.
We describe them visually for others to see.

Notes taken on a mobile device. Pardon any auto-corrections or incorrection.


The ability for the screen to move further from the user's eye means that it is healthier.

It allows users to naturally focus their eye in the device.

One of the drawbacks of current HMD systems is that if a virtual object appears far or close, your eyes are always focused at the same rate. 

Even when you use a laptop or mobile phone for extended periods of time, you should take a break to look out of the window and focus on something 20 meters away to exercise your visual focus.

Uploaded by Oculus on 2018-05-24.

Notes taken on a mobile device. Pardon any auto-corrections or incorrection.

Three differences in 3D

There are three fundamental differences when creating for AR & VR.
These are differences that both designers and engineers encounter when developing 3D experiences.


Notes taken on a mobile device. Pardon any auto-corrections or incorrection.

High-speed AR for cars, airplanes, and drones

"(CTECH)  High-speed augmented reality (AR) startup Edgybees Ltd. announced the completion of a $5.5 million Seed round. Motorola Solutions Venture Capital, the investment arm of data communication company Motorola Solutions, Inc., Verizon Ventures, and equity crowdfunding company OurCrowd Management Ltd. participated in the round, as did venture capital firms 8VC, NFX, and Aspect Ventures."

"Founded in 2016 and headquartered in Israel, Edgybees develops AR technologies for fast-moving platforms such as cars, airplanes, and drones. Edgybees’s technology is used by developers to create realistic, immersive AR experiences that help maximize the effectiveness of aerial operations. The company was founded by Menashe Haskin, who formerly managed the Israeli development office of Amazon Prime Air."




Notes taken on a mobile device. Pardon any auto-corrections or incorrection.

Winslow Turner Porter III - New Reality Co. during AIGA at MAD

Winslow Turner Porter III - New Reality Co. during AIGA at MAD

See how Winslow created “Tree” an official selection of Sundance Film Festival New Frontier 2017.

Speaker description from AIGA:
"Winslow Porter / Co-Founder, Director / New Reality Company
Winslow is a Brooklyn based director, producer and creative technologist specializing in virtual reality and large-scale immersive installations. Winslow has always been fascinated with the possibilities of how the intersection of art and technology can elevate storytelling. In 2014 he produced the Tribeca Film Festival Transmedia Award-winning documentary CLOUDS, among other acclaimed new media projects. Winslow formed studio the New Reality Company with Milica Zec in 2016, creating the critically acclaimed cinematic VR experiences Giant and Tree. They were both named designers in residence at A/D/O, a BMW sponsored design center in Greenpoint Brooklyn; last fall the two were selected to Adweek’s Top 100 creatives as digital innovators.  And this past February Tree won the Lumiere award for Best Location Based VR Short at Warner Brothers for the Advanced Imaging Society.

Sabrina Dridje / Managing Director / The Telly Awards
Sabrina is the newly appointed Managing Director of The Telly Awards, the leading award honoring excellence in video and television across all screens. In this role, Sabrina is leading efforts to build upon the 38 year legacy of the award by revitalizing new partnerships, editorial strategy and digital presence. Sabrina was most recently the Director of New York City’s leading incubator and and collaborative workspace for media startups, entrepreneurs, and artists – the Made in NY Media Center team.

Natasha Jen / Partner / Pentagram
Natasha is an award-winning designer, an educator, and a partner at Pentagram. Born in Taipei, Taiwan, she joined Pentagram’s New York office in 2012. A three-time National Design Award nominee, Natasha’s work is recognized for its innovative use of graphic, verbal, digital, and spatial interventions that challenge conventional notions of media and cultural contexts. Her work is immediately recognizable, encompassing brand identity systems, packaging, exhibition design, digital interfaces, signage and wayfinding systems, print and architecture.  Natasha is a faculty member at the School of Visual Arts and is a guest critic at Harvard Graduate School of Design, Yale University School of Art, Cooper Union, Rhode Island School of Design, and Maryland Institute College of Art.

Rori DuBoff / Managing Director, Content Innovation – Extended Reality / Accenture Interactive
Rori is a strategic and innovations thought leader in the extended reality space, spanning VR & AR experiences. In her current role as Managing Director, head of content innovation for Accenture Interactive, Rori focuses on strategically applying immersive technologies for business and brand transformation. Rori has over 20 years experience working in digital marketing, integrated media & creative brand advertising. Prior to Accenture, Rori was global head of digital strategy, EVP at Havas media group, where she led and managed strategic planning worldwide. Rori has also held senior positions as Ogilvy, IPG & Omnicom. Rori is regular public speaker and writer on marketing innovation and strategy. She holds an MBA from NYU Stern School of Business and a BA from the University of Pennsylvania."

Event description from AIGA:

We have experienced, designed for and lived in the device economy for decades. We watch, consume and create for screens of all sizes. However, we are now moving into the era of a world where content can be experienced in the absence of a true screen – a 360 video that invites us to interact with new environments, virtual reality that redefines the user as an active agent in creating a narrative and AR/VR/MR (augmented reality/virtual reality/mixed reality) that is blurring the line between “reality” and designed experiences all together.

The questions now include what does it mean to be a designer when working in AR/VR/MR? Does a designer become a director? What are the titles for the roles that are needed? What tools and new types of collaborators lay ahead?

With the help of Sabrina Dridje, managing director of the Telly Awards; Natasha Jen, a partner at Pentagram; Winslow Porter, an award-winning VR director and Rori DuBoff, a thought leader in extended reality space, this panel will dive into the strategic and practical insights needed to design for a screenless world.

This workshop is open to all, but designed for creatives just starting to understand the AR/VR/MR space."


Follow AIGA and sign up for more events in the Future of Work series 

Notes taken on a mobile device. Pardon any auto-corrections or incorrection.