Reimagining the Future of Accessible Education with AI (Part 2)

This month, the Microsoft AI for Accessibility program is calling for project proposals that advance AI-powered innovations in education that will empower people with disabilities. Through a two-part series, we are highlighting projects we are supporting. Part 1 of “Reimagining the Future of Accessible Education with AI” can be found here.

Imagine you are performing exceedingly well in high school, only to find out that you will have to take one of the most prestigious university entrance exams with a reader who does not fully understand the technical content. This is the story of thousands of students in Kenya and India who are not provided an equal opportunity to pursue the academic courses of their choice.

The majority of students who are blind or low vision in countries in Africa, Latin America, and parts of Asia and the Middle East rely on in-person readers and writers to take exams and complete homework. However, this leads to several detrimental impacts on performance, including lack of availability of test readers and writers with domain-specific knowledge, frustrations of test readers and writers during long exams or with unfamiliar material, poor working relationships due to confusion, and lack of communication, among others. Furthermore, local universities have been forced to focus more on technology and reduce face-to-face accommodations due to COVID-19 protocols.

inABLE, a nonprofit organization in Kenya that empowers the blind and low vision through computer assistive technology, partnered with I-STEM, a not-for-profit company in India that develops STEM technical solutions, received a Microsoft AI for Accessibility grant to revolutionize how blind students take exams.

“The I-Assistant is an artificial intelligence (AI) solution that gives blind and low vision people an alternative to an in-person test reader and writer with an automated tech-powered conversational experience,”, explains Irene Mbari-Kirika, Founder and Executive Director of inABLE. I-Assistant leverages a custom model built on top of Azure Computer Vision, particularly Text Extraction (OCR), that detects various document structures and layouts (e.g., tables, headings) and processes text, including math expressions. Then a readable and accessible document is created – thereby converting any exam into an accessible, digital format. Using Azure Custom Speech Recognition, Text-to-Speech, and Language Understanding (LUIS), users can intelligently navigate the exam or document and perform context-aware actions, like checking exam status by asking questions such as “return to the question I skipped”, “how many questions have I answered?”, or “how much time do I have left?”

Kartik Sawhney, I-Stem cofounder, emphasizes the strengths of the AI through this work: “We are very pleased that we were able to build a fully functioning prototype with multiple AI-powered components coming together to provide a natural conversational experience to the users. What is even more interesting is that many of the individual components such as OCR and custom speech by themselves have great promise.”

To further promote research into content accessibility that was a core component of the I-Assistant project, the project team is sharing an open-source OCR and document analysis dataset so that other teams can build new innovations on top of it.

I-Assistant’s natural conversational experience considers the specific needs of the blind community while interacting with the computer, such as navigating fast and effectively and accurate speech-to-text. The system provides the blind or low vision student with the same experience as a sighted person, with the added advantages of consistency, accuracy, and comfort. I-Assistant helps blind and low vision students take tests more independently – a true testament to empowering people with disabilities.

Learn more about past grantees, how AI works, and the tools Microsoft provides to enable inventors of all skill levels to bring their projects to life.

As we look to support novel and innovative ideas to empower people with disabilities in their education journey, AI technology in the classroom is particularly capable of having an impact in these scenarios thanks to advancements in transcription, translation, and language understanding capabilities. Given the importance of accessible curricula and content and the possibilities to advance assistive technology for students with disabilities, there is no better time than the present to reimagine inclusion and accessibility in education and consider how AI technology can:

• Innovatively support the transition from high school to college for students with disabilities.

• Make it easier to learn to use Assistive Technology.

• Provide flexible, multimodal, and personalized solutions.

• Provide accommodations for students with disabilities including those impacted by:

  • Mental health conditions

  • Cognitive disabilities

  • Intersectional identities

More importantly, how will you use the power of AI to take your innovative idea for the future of inclusive and accessible education to the next level?

The AI for Accessibility program offers grants through data science skills, Azure compute credits, and cash to help innovators design and build the future of AI-driven, accessible, and inclusive technology. We support innovative AI projects led by individuals or teams rooted in the disability communities that they serve. Universities, non-profits, startups, and other organizations are encouraged to submit project proposals that leverage AI-powered innovations to build and advance equity for students with disabilities inside and outside the classroom.

If you are interested in applying for a grant award, our Education Request for Proposals is open now until the application deadline on March 12, 2021. Check out our application FAQs for more details on the program.