From Hack to Product, Microsoft Empowers People with Eye Control for Windows 10

Sometimes, a simple email can lead to a life-changing breakthrough.

In 2014, former NFL player Steve Gleason, who has a neuromuscular disease called amyotrophic lateral sclerosis (ALS), sent an email to Microsoft, challenging employees at the company’s first hackathon.

Steve’s ask to this diverse array of thinkers, doers and dreamers was to develop technology that could address some of the constraints he faces living with ALS, a disease which causes the death of neurons that control muscle movement, resulting in difficulty moving, speaking, swallowing, and eventually, breathing. For most people with ALS, the eyes are the only muscle not impacted by the disease.

Steve wanted to be able to play with his son, talk more easily with his wife and to move his wheelchair by himself: goals that for someone with ALS seemed like an impossible dream.

“I realized pretty quickly after my diagnosis that technology would have to become an extension of myself. Until there is a medical cure for ALS, technology will be that cure,“ Steve said.

The Microsoft Enable Research team meets to discuss the Eye Gaze Wheelchair.

Steve’s challenge was a natural fit for Microsoft, especially during One Week Hackathon, which encourages employees to focus on passion projects that advance the company’s mission of empowering people around the planet. A team calling themselves Ability Eye Gaze took on Steve’s ask, excited by the opportunity to create technology that could genuinely affect a person’s life, and quickly got to work.

The Ability Eye Gaze team focused on developing a tool based on Steve’s requests, and after three days of hacking, one of the projects stood out: the Eye Gaze Wheelchair. This unique solution allowed Steve to drive his wheelchair with only the movement of his eyes as he looked at controls on his Surface.

The judges reviewed over 3,000 projects from teams across the company, and Microsoft CEO Satya Nadella named the Eye Gaze Wheelchair the grand prize winner of the 2014 hackathon. The Eye Gaze Wheelchair received such enthusiasm from employees and the ALS community that a new Microsoft Research team was created to understand the potential of eye tracking technology. However, this was only the first chapter of an inspiring story.

The new research team started to work closely with Steve’s nonprofit, Team Gleason as well as Evergreen ALSA, an organization that supports people living with ALS and their loved ones through services and education. They also reached out to individuals locally in the Seattle ALS community to understand their experiences and needs, and how technology could help them.

When the Windows team came across this technology, they immediately saw the potential for eye tracking to change people’s lives as well. Eric Badger, Principle Software Engineer Lead on Windows, and Harish Kulkarni, Principal Software Development Engineer in Microsoft Research, quickly started prototyping new eye tracking scenarios together. The results were promising and they dedicated a team of engineers to bring eye tracking support to Windows.

Last week, Satya returned to the One Week hackathon, to share projects that had inspired him over the years, and to announce that Windows 10 will include built-in eye tracking support and an experience called Eye Control, inspired from the winning hack in 2014.

YouTube Video

An audio description of the video is available here

Eye Control makes Windows 10 more accessible by empowering people with disabilities to operate an onscreen mouse, keyboard, and text-to-speech experience using only their eyes. The experience requires a compatible eye tracker, like the Tobii 4C, which unlocks access to the Windows operating system to be able to do the tasks one could previously accomplish with a physical mouse and keyboard.

Right now, Eye Control is in beta and people interested in early testing and providing feedback can sign up to be a Windows Insider.

Eye Control is demoed by a woman looking at a Surface tablet.
A demo of Eye Control at the 2017 Microsoft OneWeek hackathon event.

The source of the motivation is simple. It’s our opportunity to embody the mission of the company, to ‘empower every person and organization to achieve more’ combined with the power of One Week. Now in its fourth year, One Week brings together thousands of Microsoft employees from across the globe to generate new ideas that push the boundaries of technology and solve real-world challenges. During this year’s hackathon, we asked our employees, “Who will you empower?” That mentality, combined with a passionate belief in the transformational power of technology, is a magic combination.

For two Microsoft employees, the passion to empower people drove them to help make Eye Control a reality.

Jon Campbell is a Senior Research Software Development Engineer and was a member of the original Ability Eye Gaze hackathon team. “There was this great vibe at One Week, it was like summer camp for innovative ideas rather than work. It was exciting to be challenged to follow our passions and think creatively with so many talented people from across the company,” said Campbell. “While not all ideas are winners, the thing that makes the hackathon great is that other ideas we kicked around had real potential – like eye tracking technology to help people with ALS and how Windows can bring it to more people.”

Jake Cohen, Program Manager on the Windows Interaction Platform team, is one of the Program Managers responsible for designing and integrating Eye Control into Windows 10. “When I heard about the Ability Eye Gaze team and what they were creating, it was super exciting to think about the possibilities of what could be done next,” said Cohen. “And it really made a dramatic difference to me when we started meeting with people living with ALS. I began to understand all the challenges they live with every single day. We’ve done a lot of great work across our teams, and we have a lot more to do, but we’re at a point where it has the potential to really start changing lives.”

The Ability Eye Gaze hackathon team accepted Steve Gleason’s challenge and as a result, Windows 10 will be more accessible for a broader set of users. Now Steve, as well as those with ALS and other disabilities will be able to live fuller lives. For them, the seemingly impossible dream is now becoming more of a possibility.

“When I approached Microsoft three years ago, I asked them to help develop innovative technology programs that would allow people with disabilities to stay productive and purposeful,” said Steve Gleason. “They agreed without hesitation. At that time, I had no idea they would continue expanding access throughout all of Windows 10. Having Eye Control in Windows 10 continues to bridge the gap between widely used technology and people with disabilities. It’s simply liberating.”

NFL player, Steve Gleason, with his son
Former NFL player, Steve Gleason, with his son.

If you have questions or feedback on how we can continue to improve our products and services, you can contact us through the Disability Answer Desk (now with ASL support) and Accessibility User Voice Forum.

You can learn more about accessibility at Microsoft by visiting, http://microsoft.com/accessibility.