RoboRaid

A basic experience where holograms or AR entities appear to break through the physical walls.
This simple experience only has one method of gesture input - the single Air Tap.
 

Air Tap. Source: https://docs.microsoft.com/en-us/windows/mixed-reality/gestures

When wearing the HMD, the hands occlude the visualizations, so your hands appear on top of the robot holograms.

The second method of input is audible, saying "X-ray" shows you where the opponent are "inside the wall."

See more about RoboRaid and some of Microsoft's concept work here: https://www.microsoft.com/en-us/hololens/apps/roboraid

Outside of this simple application, you can give audible commands to Cortana, the Hololens system AI agent (like Siri but smarter). And you can use the only other Hololens recognized hand gesture - Bloom - to call up the main menu.

Dematerialization to create applications for 3D experiences

Notes from Jody Medich's "Real-world VR applications beyond entertainment - a look at our rapidly dematerializing world."

Visual Cortex
"Visual sense is 70% of your sensory intake."
Visual cortex is even used by blind people
everything from spatial memory to spatial location - all happens in your visual cortex.

Your visual cortex is about the size of your hand. It works for both hemispheres of your brian. It is integral to both sides.

A synthetic visual cortex.

More on
Dematerialization
and application to
Kinesthetic Learning, for creating muscle memory and new nueral pathways.

Follow Jody Medich @nothelga

View the original source https://www.oreilly.com/ideas/real-world-vr-applications-beyond-entertainment

Full video on Safari Books https://www.safaribooksonline.com/library/view/oreilly-design-conference/9781491976180/video302847.html

IA for AR

Rony's example

Three different levels of zoom of volumetric content you can walk around and view from any angle.

LOF
Consists of proximate, angle and contextual information 

AR Prototyping Tools

The main prototyping tools I have used:

• Torch 3D

• Ottifox

• Halo Labs

There are not many prototyping tools for Augmented Reality. There are very few.

These are select tools that I have discovered and vetted in my own time.
They are an advancement on WebVR that are converging with the adoption of mobile AR.

Designing and Prototyping for AR

Designing and Prototyping Augmented Reality

How to understand how to design for AR? Prototyping multi-scene augmented reality.
Understanding, designing, and prototyping, augmented reality.

There was the age of personal computing, the dot com era, internet when we realized we could connect people to each other, and more recently interpersonal computing that defined more than a decade of digital experience with the notion of “mobile first” and VC firms that only invested in mobile, or designers who were “mobile designers.” We are entering the age of mixed reality or spatial computing.

With innovations like autonomous vehicles that you do not need to drive or that can drive you,  blockchain and cryptocurrency, I believe augmented reality will be the interface to all of this.

AR will be the interface to everything that you cannot see. Anything that lacks a dedicated physical interface such as autonomous vehicles, deliveries, service process, blockchain and crypto-currency. AR will also replace many digital interfaces.

As we enter an age of spatial computing, or designing across space agencies, brands and designers are asking “How to we design for virtual and augmented reality?” When informed that AR experiences are made in a game development engines - this often misunderstood term confuses people, leading them to believe that gaming is the only application. Or that the end goal is only for entertainment’s sake and that Augmented Reality cannot provide real business value. These game development engines with extra heavy interfaces can intimidate many designers, or the simple notion of an integrated development environment where you design and code scares away designer and new creative adopters.

A few select companies are bridging the skill, knowledge and tool gap that currently separates designing and developing 3D experiences. By creating prototyping tools for VR & AR, these companies are bringing these frontier technologies closer to fruition through their creator tools. Currently there is a battle between 2D design and prototyping tools as the phases between design and prototyping merge and the iterations become quicker for more agile processes.  If you are a digital product designer, you are well aware of this battle and keeping track of which tools are better bets to master for your process, teams, and assets. If you are a Venture Capitalist with bets on frontier technologies, you have looked at the tool layer of augmented and virtual realities, because you understand it is the basis of creating experiences and content for reality. And you also understand that this frontier is rapidly approaching. These AR prototyping tools not only bring this frontier closer, they are empowering designers to make things beyond the screen.

What are these companies? Where are they founded, how did they start? How do they bring us as creators closer to this future where frontier technologies are no longer experimental but are instrumental in solving problems and visualizing what is currently invisible? I will cover that in a series of posts covering the main prototyping tools, a process of design for development, informations architecture for virtual and augmented experiences.

Stay tuned.

Face Blur. Computer Vision

While blurring something out in After Effects recently, I realized the relationship between the blur, with tracking (including Z axis tracking) and computer vision.

““Hold Infinity in the palm of your hand...A fibre from the Brain does tear...The Game Cock clipd & armd for fight...Has left the Brain that wont Believe””

— William Blake - Auguries of Innocence

Security in AR and the dark side of the future.

"The user interface you think of today will radically change over the next five years for the apps you buy and for the applications you build." - Gartner Top 10 Strategic Technology Trends 2018

It is understood or expected that AR is in Gartner's horizon for 2018.
What is surprising is the emphasis on net security and specifically setting up decoys for digital intruders. Netsec very large and over looked market by interaction designers, largely because it is not a consumer facing experience. It is an experience no one wants to face.
As AR becomes the interface to innovations that we cannot see, such as the blockchain, UAVs and UMS, machine learning, and ambient computing, it will also become an interface or seam to our lives that can be hacked.
I gave a talk on the dark side of AR UX and cited Neal Stephenson's novel the Diamond Age and how someone was infected with " a meme that ran advertisements for roach motels, in Hindi, superimposed on the bottom right-hand corner of his visual field, twenty-four hours a day."

We may not face threat of 24/7 visual hack anytime soon as AR contact lens ventures have been invested in but not innovated upon. But what about "pop-ups" or visual attacks while driving, walking, or operating machinery? We understand the work place, manufacturing, and other enclosed environments to be the first contexts for digital eyewear to reiceve adoption.

As we design the interface to ubiquitous computing we not only need to make it usable (something we struggle with), useful (something we constantly aspire to), but also safe. That's something we're not aware of. People question how advertising will be non-intrusive with AR. But there are deeper sides that public discourse has not touched upon yet.
 

 

Three Types of Design

John Maeda dissected design into three types of design.
This model was first displayed in KPCB's 2016 Design in Tech Report.
It has since been reused in 2017's DIT Report.
 

I find it useful in conveying to people the types of design I specialize in. Design Thinking to define new revenue streams for better business models. And Computational Design, such as using WebVR to code functional Augmented Reality prototypes.

Madea writes how this simple chart is controversial. Much the model can be debated. For example, websites and mobile sites could be seen as falling into "Classical Design" because we have been making them for decades.

See more on the Design in Tech report here.

xR Announcements - 2017 Q2

Recent xR announcements
include but are beyond head mounted displays

Facebook: 
Use cases for AR
- information
- digital objects
- enhancements

Focused on the camera as the AR platform

https://www.youtube.com/watch?v=OFkEHEWPZSU 

Building Blocks
- Precise Location
- (SLAM) Simultaneous Localization & Mapping
- Mapping 3D scenes from still photos
- 3D Effects
- Object recognition 

Use Cases visualized
Leave directions or notes for friends.

Give people more information about where they are

Drop digital objects other people can find and interact with right there

https://www.youtube.com/watch?v=K_bTzosFGUs

---

MSFT

- Use by airlines to make jet engines
- To manager factories with data overlaid
- For medicine
- In education
Released hand held controllers (they are the only 6DOF multi-hand controllers you can set down on a flat surface).
 

---

Google Lens:
“Vision based computing capabilities”
Tango is now ARCore - 
based on:
- Camera movement
- Approximate Lighting
- Planar tracking (horizontal only)

---

Apple: 
ARKit
- Approximate lighting
- Amazon has a game development engine.
The implications for commerce are huge.

---

They don’t have to be in HMDs
Newly announced experiences are mobile based.

But… whatever devices we wear will offer functionality of replacing everything that we do with current mobile phones.

Torch 3D has people ask, "This prototyping tool your creating. If you don’t think people will use it for games…what will they use it for?"
Torch 3D's answer is “For everything else. Everything that we currently use our phones for.”

It is immediately important to understand how it affects your business beyond the CMO role and across your organization.

Highlights from the preview of ARiA

From Charles' Finks preview of Augmented Reality in Action on Forbes.

1. It’s already here.

“What's visible to the consumer right now are tricks like masks, filters, and games that nibble at the edges of visual computing, but what's invisible is the way AR has been integrated into every day life. For example, most new cars have a heads up display: data projected onto the driver's windshield. Broadcast news and sports now routinely lay data, graphics, and animation onto the physical world. AR has become ubiquitous in ways that have nothing to do with smart glasses.”

2. It’s going to get bigger

“ 'AR is on the verge. It’s exciting to watch it take shape. In the end, it won’t even be called AR. That it will disappear,’ says Bob Metcalfe, Professor of Innovation at the University of Texas, Founder of 3Com, who famously formulated Metcalfe's Law, which states that the value of a telecommunications network is proportional to the square of the number of connected users of the system.”

3. It’s going to be really big

“ Many industry insiders see AR/VR as the fourth wave of modern computing following mobile computing, the internet, and personal computing. With each subsequent wave dwarfing the previous in market size and social impact, this is one wave you may want to catch. AR is likely to replace not only the majority of screens in use but alter the way we interact with computers forever.”

Read in full here: https://www.forbes.com/sites/charliefink/2018/01/15/aria-the-ar-conference-at-mit-is-the-anti-ces/#6f3bdd133e7c