As we celebrate the International Day of Persons with Disabilities (IDPD), we are launching the newest version of Seeing AI, powered by the latest advances in AI at Microsoft, and sharing that it is now available for the first time on Android devices via the Google Play Store. With today’s Android availability, Seeing AI is now available in 18 languages* with a plan to increase to 36 languages in 2024.
Seeing AI is a free app that narrates the world for blind and low vision people, including myself, right from your mobile device. It supports individuals with a variety of daily tasks such as reading mail, identifying everyday products, hearing descriptions of photos, and much more.
Thanks to feedback from those who use the app, the new Android version includes the latest generative AI features recently released on iOS:
- Richer Descriptions of photos: In addition to providing a brief summary of photos on the Scene channel, you can now tap ‘more info’ and a rich description will be generated, including far greater detail about what is in the image.
- Chat to your documents: After scanning a document, in addition to hearing it read aloud, you can also chat to Seeing AI to ask questions, such as about items on a menu, the price of an item on a receipt, or to summarize an article.
At Microsoft, we use our technology to innovate with and for people with disabilities. Seeing AI was first released for iOS as a research project in 2017 when a team of engineers and I worked with the blind community to understand the areas where technology may be able to provide even greater independence and enjoyment.
With Seeing AI, you can just point the camera, or take a photo, and hear a description. Switch between channels to hear focused information:
- Short Text: Speaks text as soon as it appears in front of the camera.
- Documents: Provides audio guidance to capture a printed page, and reads the content aloud, along with its original formatting. Chat with Seeing AI to quickly find information.
- Products: Scans barcodes, using audio beeps to guide you; hear the name, and package information when available.
- Scenes: Hear a description of the scene captured and tap ‘more info’ to generate a rich description. Explore the photo by moving your finger over the screen to hear the location of different objects.
- People: Identify friends around you.
- Currency: Recognizes currency notes.
- Colors: Identifies the perceived color.
- Handwriting: Reads handwritten text like in greeting cards (available in a subset of languages).
- Light: Generates an audible tone corresponding to the brightness in the surroundings.
- Images in other apps: Share a photo with Seeing AI to recognize it.
There are over 3 billion active Android users worldwide, and bringing Seeing AI to this platform will provide so many more people in the blind and low vision community the ability to utilize this technology in their everyday lives. We will continue to work with the community to understand feedback to improve the app. As the motto goes, “nothing about us, without us.” And as additional versions roll out, customer feedback will continue to be critical for new AI-powered enhancements to future versions of the Seeing AI app.
My team and I are eager for you to try Seeing AI and let us know what you think.
*Languages available in iOS and Android: Czech, Danish, Dutch, English, Finnish, French, German, Greek, Hungarian, Italian, Japanese, Korean, Norwegian Bokmal, Polish, Portuguese, Russian, Spanish, Swedish, Turkish. https://www.seeingai.com/Android