Color Binoculars, a new mobile app from Microsoft Garage, helps colorblind ‘see’ the world the way others do

Nov 11, 2016   |   Athima Chansanchai

Three vertical screenshots of what apples, flowers and peppers

Look outside. Fall is in full swing, and trees are filled with leaves of different shades of brown, red, yellow and orange. But eight percent of the population can’t distinguish between those vibrant hues.

“Anything with red or green messes me up,” says Tom Overton, a Microsoft software engineer whose focus is on Cortana experiences. He’s also colorblind. “It’s not so terrible, but it does affect you. For instance, fall leaves don’t look any different for me than other leaves. They look like they always do. It takes a lot of color out of my life – metaphorically, that is.”

Inspired to create a solution to this lifelong problem, Overton teamed up with Tingting Zhu, another Microsoft software engineer, and came up with Color Binoculars, an iOS app being released through the Microsoft Garage.

Camera captures of apples without the filter, and with the filter
Camera captures of apples without the filter, and with the filter

“It’s an app that helps colorblind people distinguish color combinations that they would normally have trouble telling apart,” Overton explains. “For example, since I have difficulty distinguishing between red and green, our app makes reds brighter and greens darker so that the difference is more obvious. It replaces difficult color combinations, like red and green, with more easily distinguishable combinations, like pink and green.”

The name of the app is also a metaphor, says Overton.

“You can see the world more zoomed in, you can use this app to see the world in a different way,” he says. You’re not saving images or video, but using the phone’s camera as a way to translate images.

If you’re colorblind, it’s hard to tell apart wavelengths of light, so what the app does is use filters and the iPhone’s camera, which is objective, to see the differences between colors, if not the colors themselves. With this app, you could pick out flowers, choose matching clothes, tell the difference between colored alerts onscreen – and appreciate autumn’s glorious transition.

“For me at least, it’s such a personal project,” Overton says. “I showed it off to my family. I have a cousin who is also colorblind, and he really enjoyed it. Also, when I’m cooking and I need to brown meat, I can bust it out so I can tell when it’s not pink anymore!”

Tom Overton and Tingting Zhu of Microsoft app Color Binoculars on January 21, 2016. (Photography by Scott Eklund/Red Box Pictures)
Tom Overton and Tingting Zhu of Microsoft app Color Binoculars on January 21, 2016. (Photography by Scott Eklund/Red Box Pictures)

The duo conceived the app for the 2015 Hackathon, an annual global event for Microsoft employees.

“We read about special lenses that helped colorblind people distinguish colors,” says Zhu, who works on the Azure active directory. “However, that’s expensive. So we thought, let’s code something on a phone for free.”

For them, it was also fun to see whether or not they could do it, since neither had ever touched iOS development before. But they both have iPhones and neither wanted to do C# server development, since they do that every day already. And it was a great opportunity to work together. Even though they had shared an office for 10 months, they had never really collaborated before this.

The two came up with several ideas at once, and split several small tasks. They quickly realized they needed to make the phone camera work for them, capturing the live preview and then adjusting every picture by applying specific filters. Overton’s colorblindness made him a good tester, as well as developer.

Then they took their project to The Garage’s pitch day, and that process helped take it from a basic prototype to a more full-featured app that adapts to three different kinds of colorblindness and has an on/off mechanism for the filters to compare what the world normally looks like vs. what the colorblind see.

Carrots, with and without the Color Binoculars filter
Carrots, with and without the Color Binoculars filter

“It was pretty cool when we actually nailed it,” Zhu says. “I read articles about how those without color blindness see the world vs. how colorblind people do. Then when we applied the filter, I could see how it looks so different for others.”

Both Zhu and Overton praised The Garage for shepherding them through the final stages to ship the app, such as making sure it was legally buttoned up.

The Garage is the outlet for Microsoft teams around the world to get experimental apps and projects out to the public, such as several recently developed by interns, as well as Video BreakdownArrow LauncherTrip TrackerSprightlyKaizalaNews Pro 3.0 and Hub Keyboard.

Overton and Zhu learned lessons, too, and as a result the app can support any screen size on iOS.

“We’re more confident, and planning to develop something more in the future,” says Zhu, explaining her takeaway from the experience.

“You take the first step and practice, practice, practice,” Overton adds. “Then you can step back and be impressed you made that, and try to raise the bar a little bit.”

For Overton, working on a solution for something so personal was gratifying.

“It was really empowering to push through the challenges and take it all the way to the app store,” says Overton.

Download Color Binoculars and find out more about the Microsoft Garage.

Athima Chansanchai
Microsoft News Center Staff