Why eye tracking is to VR what touch was to smartphones

VR and eye tracking - taking it back to natural

From the very start of your life, you have been using your eyes to “aim your reach”. Sure, getting it right early on was tricky. But today, that short glance helps your body to coordinate everyday stuff, like picking something up or pressing a button. Your eyes helps your body to adjust and coordinate movements (remember your football/golf/and so on … coach telling you: “Keep your eyes on the ball” ; -)

But, over the last 30 or 40 years, using computers, mankind has taken a detour.

To interact with an object on your desktop, you are using a mouse (or touchpad) to translate your intention into action. It is a three-step process:

  1. You look at the icon (or object, window … .)
  2. You maneuver the mouse to the object (mouse travel)
  3. You take action (click, double click, drag and so on)

Certainly, the mouse, as an innovation by Doug Engelbart way back in 1963, has had a huge impact (as the graphical user interface) to propel usability of computers replacing toggling and menu clicking in character based UI. But still, mouse travel is an awkward, to first time users, and not a natural method of interaction.

The introduction of touch in the mobile phone market and the rise of the smartphones is a great example of the power of aligning with natural behavior!

If you ever owned an ultra-powerful N95 by Nokia – you know what I mean!

The N95, technology packed, early generation smartphone was obliterated by touch – when Apple introduced the iPhone

Suddenly, you went from a three-step process to a two-step process:

  1. You look at the icon
  2. You touch (click)

This is the same way you, as a baby, learned to interact. And it is a great analogy explaining what eye tracking is to VR.

But first.

In over the last year or so, major brands (Acer, Alienware, MSi) are releasing consumer-grade devices with eye tracking technology embedded in them. And the user scenario enhancement is all about taking the user experience back to natural human behavior.

Obviously, this is very exciting and creates great opportunities for game developers. Today they can detach game camera “aim” from what you are looking at (run in one direction while aiming in another, using your eyes) and so much more. And in Windows 10, work in two separate windows and not having to click the window you want to make active. Your gaze tells the computer your intention and it becomes “active” for scrolling or whatever it is you want to do. And so on.

For computers, your eyes are a great complement to existing input devices, like keyboard, mouse, gamepad, joystick and so on.

But in VR, it is another story. Methods of input are limited. 

The controllers you hold in your hand may have buttons, but learning them and have them translate into an intuitive user experience is far from easy. And the simple task of “aiming” could be tricky. Today, in a head mounted display (HMD), you are using your forehead or the controller(s) you hold in your hand. Aiming with your forehead is it not very accurate and over time quite exhausting to your neck. Aiming with the controllers is like the mouse all over, but probably a bit trickier, since it is in a three-dimensional world.

That is why eye tracking is to VR what touch was to smartphones. It translates so many three steps tasks into two step tasks. It makes it more natural.

And that is why eye tracking is the next natural step for VR. Eye tracking provides an opportunity to simply tap into existing behavior, making VR, even more, align with how human behavior works in reality. This will help developers create applications for VR. And this will help drive VR towards its full potential.

“I was even more impressed during the next portion of my demo, which let me interact with a whole bunch of objects on a virtual patio. With tons of objects laid before me, my eyes suddenly became a mouse cursor that let me scroll through them by simply looking around”

MICHAEL ANDRONICO March 2, 2017 (GDC 2017) – read the whole article here.

Thanks for reading! Great if you would share. And even greater if you would leave a comment!

To learn more about eye tracking and gaming, go here. To see what consumer devices that have eye tracking built in - go here.

To learn more about eye tracking and VR, download our white paper How Eye Tracking Drives Immersive VR.

Download: How Eye Tracking Drives Immersive VR

What to read next

The Immersive Gaze

Read next: The Immersive Gaze