VR Needs a “Mouse” for Navigation

What if your PC/laptop had no mouse or trackpad? You’d find it a pain, wouldn’t you?

Navigating VR worlds has a similar problem, but you (your whole body) are the navigation tool for VR; you are the “mouse roller-ball”.  Whereas a roller-ball on a Formica table or leg of jeans (mouse specifications – Steve Jobs) is great for computer mice.  It’s a different proposition for real presence VR.

Designing a human interface device for VR navigation requires inputs from physiology, psychology, neuroscience, physics, material science plus a whole bunch of other disciplines.  So that a mouse for VR worlds complements the strengths and abilities of people, and reduces the effects of human and technology limitations.

We walk to explore our world

Why not just go with the flow and enjoy seated VR? – because that’s not life as we know it.  It’s not great VR.

Yes, we fly and drive to explore our world.  But mostly we walk and we know our position within space and directions from one place to another by walking. Architects and clients walk around buildings.  Saboteurs stealthily walk around starships and enemy buildings. We know the size and relative positions of where we live because we navigate by walking. These are normal expectations (predictions) of real life.  To replicate these we need something that allows us to walk comfortably and safely, blindfolded, without fear of collision, in open worlds.

It’s natural to think that to feel we are walking in VR.  We actually need to lift and place our feet forward, one after the other, ‘step-and-place’ ….just as in real life.  This “intuition” seems reasonable because our whole life’s experience of walking, from toddler ‘til now, has been that ‘step-and-place’, equals walking.

However realistically navigating VR worlds by walking is really difficult.  Attempts over the past 30 years to achieve ‘step-and-place’ walking in VR.  Even military and NASA budgets failed to deliver a practical solution.  So where’s the problem?

It turns out our intuition is wrong!

We don’t carry memories of how we move our legs when we walk

That’s crucial if you’re looking to design a practical VR treadmill.

Understanding how the Brain Functions When Walking

It’s not just true for walking but running, climbing stairs, holding onto things.  Many everyday actions are subconscious and often fully autonomous.  Done without a moment’s conscious thought.  Guided by a desire to do something.  Crossing a room, climbing stairs, catching the bus, and even driving home! In all of these everyday activities, the control of leg movement is done subconsciously.

Understanding that we don’t carry memories of how we walk is a first critical step.  However equally important is realizing the brain receives sensory data (cues) to keep the process running smoothly. It turns out these brain “cues” are critical feedback signals, necessary for our brains to accept the instruction to walk has taken place and the action is proceeding to plan. And so, the walking process is pushed to the subconscious.  With the sensory feedback cues being crucial for our sense of wellbeing in real life just as they are in VR.

Think about it now – you probably walked to where you’re reading this – now think hard: How did you move your legs to get to this place?  – There is no memory!

Understanding the Brain

Walking, climbing stairs, holding objects, plus lots of other similar actions are subconsciously governed by at least the following parts of the brain; the motor cortex (priming of muscles), basal ganglia (repetitive action) and cerebellum (coordination). The basal ganglia have the particular capacity to quickly remember and issue repeat motor cortex commands required for walking, climbing stairs, etc.  Some rhythmic motions of walking are governed by neurons in the spinal cord which may be regulated by the cortex via the cerebellum to increase or decrease pace and gait. But crucially, nothing gets stored in memory!

We provide an executive command “I need a drink” our brain’s motor cortex, cerebellum, basal ganglia, spinal cord plus a few other bits do what’s necessary to take us to the kitchen. Even stepping over the dog is done subconsciously!

This is complex stuff and there are many brilliant people who have worked extensively in deciphering the brain’s interactions and inter-dependencies often using practical assessment of a person’s actions and limitations. Because of the amazing complexity of the brain and its critical function for life and consciousness, the majority of understanding and knowledge has come from a comparison of injured and uninjured brains in living people. From this research, increasingly detailed maps of function and process are being built.

The Solution

In 2001 Julian Williams, in the first place without too much thought as to the impact on the realism of walking, thought reciprocating or sliding of feet in a dish would be a visually realistic compromise of walking –  great for knee-upward television shots. Contestants on a dish could be directed by an audience through a maze – the audience seeing a plan view but the contestant seeing only their local VR scene. The dish and sliding feet would look like walking and keep the contestant in place.

With more thought and experimentation Williams discovered an amazing realism to walking in this fashion and with the prescience of a time-traveler, he predicted the coming of consumer head-mounted display, the motivation for exercise in virtual worlds and the almost limitless potential applications of both. He applied for a US patent in 2003.  It was granted in 2008.

Considerable further research and development led to an understanding of the crucial bits of data (brain cues) necessary for leg movement to become subconscious (autonomous) and for us to subsequently think we must have been walking, running, etc. and these cues are:

• We must be weight bearing on our legs
• Legs must move in simple back and forth motion
• Our vision must match how our body feels and how our brain predicts from the motion we’ve initiated
• Direction is self-determined and unhindered

Of amazing importance considering the 100’s of millions of dollars thrown at VR treadmills over the past 30 years:

There is no need for feet to lift and be placed forward to be accepted as walking. 

In spite of this, several VR treadmills launched in the last 5 years rely on step-and-place as an equivalence of walking. Unfortunately as previously discovered by the military and NASA there is considerable user effort and cost to deliver on the physics for this mode of movement.

For the Wizdish team delivering a working ergonomically designed VR treadmill using slide-walking has also proven to have special challenges that have gradually been solved over the past eight years.

Despite consumer VR headsets appearing in mid-2016, at least 50,000 people in 30 countries have experienced ROVR VR. The number of users continues to grow as misconceptions and bad-intuitions are overcome by businesses searching for a practical solution to full physicality, real presence VR.

Words of Frederick Brooks Jr

In 1992 Frederick Brooks Jr (Fred Brooks) founder of the Computer Science faculty at the University of North Carolina, in his final report of “The (VR) Walkthrough Project” to the U.S. National Science Foundation (NSF) says ….

“For a dynamic Walkthrough experience, the illusion of motion is crucial. Good motion with almost any polygonal image that has perspective and obscuration will be perceived as a walkthrough of some validity. Poor motion is an illusion killer, no matter how rich the visual image.”

Fred Brooks’ statement, restated hundreds of times by other researchers over the past 30 years, is indisputable and is a salutary note to VR content developers and publishers, headset manufacturers, providers of entertainment, education, healthcare, training and the rest – physicality is essential for compelling real presence VR.

During 2017 there was a gradual acceptance of Fred Brook’s statement and that’s great news for consumers and VR market growth.

For the Players – The findings presented here offer a future full of wonder for VR enthusiasts.  Because there’s a solution that transforms VR into real presence amazing experiences.

For Story-tellers and Content Developers – Because what we see now matches what we feel.  There are no software constraints.  Open worlds are possible and with no breaks in presence.  You can walk forever. Every imaginative idea is open for exploration.

The very real possibility of public demands for VR game sequels is opened.

Contact us at: [email protected]