Microsoft’s annual Hackathon is a company-wide event that encourages employees to partner with colleagues from different teams and dream up projects that normally would never land on their desks.
Sunny Zhang works for Microsoft in Vancouver. Sunny’s friend, Brennan Jones works for Microsoft in Cambridge, UK. Inspired by the desire to feel connected despite the physical space that separated them, Sunny and Brennan teamed up during Hackathon 2019 to design VROOM.
So, what is VROOM? Think about how many times you wished you could blink and be somewhere else. VROOM simulates this experience by combining three existing technologies – robotic telepresence, hyper-realistic avatars and mixed reality – in one virtual space where holograms of two people, in different places, can interact with one another.
Through robotic telepresence, Sunny can drive a video conferencing-enabled-robot located in Brennan’s Cambridge office while sitting at her desk in Vancouver. The VROOM team wanted to elevate this concept by overlaying live avatars of real people over a telepresence robot. To do this, they used the Microsoft mixed reality device, HoloLens.
Step one in the project was mastering HoloLens. No one on the VROOM team had worked with the device before, but Sunny and her teammates were able to learn the ins and outs of the technology from scratch – she was amazed at how quickly her team picked up a new skill. Hacking, to this group of creative thinkers ‘feels like a superpower being discovered’.
The major difference between VROOM and other current methods of video conferencing is autonomy for the remote user. In a normal screen to screen video conference, the remote user depends on the local user to adjust their point of view. The VROOM team hypothesized that users would feel more connected when long-distance communicating if their interactions were of full-body scale and not contained within a screen.
This led them to the next step in designing VROOM – streaming live video from the telepresence robot’s 360°’ camera into a VR headset display. Sunny describes this challenge as ‘trying to fit a flat rectangle (streaming) into a sphere (360 video)’. Their last step involved figuring out how to remotely drive a telepresence robot using VR controls. The VROOM hackers were tasked with finding a way for these two sets of distinct technologies, each preprogramed with their own languages, to talk to one another.
Here’s the irony. Sunny and her team had to face the very challenge their project was aiming to solve while designing VROOM – gaps in connection. Their team consisted of two main groups, a team in Vancouver and a team in Cambridge. With an 8-hour time difference between these two locations, one group was always picking up where the other left off. The teams handed over notes and filmed video messages to keep each other up to date, but working while disconnected is not without its challenges. What better way to remain inspired by a project’s purpose than to be actively hacking towards a solution while working within the problem itself. For Sunny, this is ‘the charm of being a hacker’.
By the end of Hackathon, the VROOM team had a working product in their hands. Wearing their mixed reality headsets, Sunny and Brennan gave each other a hug and a celebratory high five. Though Brennan was sitting at his desk in MS Cambridge and Sunny was hanging out with her VROOM teammates at MS Vancouver, they were able to connect in one shared virtual space.
