AI-Mimi is building inclusive TV experiences for Deaf and Hard of Hearing user in Japan
Around the world, there is an increased demand for subtitles. In the United Kingdom for instance, the BBC reports that subtitles are primarily intended to serve viewers with loss of hearing, but they are used by a wide range of people: around 10% of broadcast viewers use subtitles regularly, increasing to 35 percent for some online content. The majority of these viewers are not hard of hearing.”
Similar trends are being recorded around the world for television, social media and other channels that provide video content.
Is it estimated that in Japan, over 360,000 people are Deaf or Hard of Hearing – 70,000 of them use sign language as their primary form of communication, while the rest prefer written Japanese as the primary way of accessing content. Additionally, with nearly 30 percent of people in Japan aged 65 or older, the Japan Hearing Aid Industry Association estimates 14.2 million people have a hearing disability.
Major Japanese broadcasters have subtitles for a majority of their programs, which requires a process that includes dedicated staff and the use of specialized equipment valued at tens of millions of Japanese yens. “Over 100 local TV channels in Japan face barriers in providing subtitles for live programs due to the high cost of equipment and limitations of personnel” said Muneya Ichise from SI-com. The local stations are of high importance to the communities they serve, with the local news programs conveying significant updates concerning the area and its population.
To address this accessibility need, starting 2018, SI-com and its parent company, ISCEC Japan, have been piloting with local TV stations innovative and cost-efficient ways of introducing subtitles to live broadcasting. Their technical solution to offer subtitles for live broadcasting, AI Mimi, is an innovative pairing between human input and the power of Microsoft Azure Cognitive Service, creating a more accurate and faster solution through the hybrid format. Furthermore, ISCEC is able to compensate for the shortage of people inputting subtitles locally by leveraging their own specialized personnel. AI-Mimi has also been introduced at Okinawa University and the innovation was recognized and awarded a Microsoft AI for Accessibility grant.
Based on extensive testing and user feedback, themed around the need for bigger fonts and better display of the subtitles on the screen, SI-com is able to create a model with over 10 lines of subtitles on the right side of the TV screen, moving away from the more commonly used version with only two lines in display at the bottom. In December 2021, they demoed the technology for the first time, in a live broadcast, partnering with a local TV channel in Nagasaki.
The live demo was evaluated by the Deaf and Hard of Hearing community, rating the experience highly and confirming their accessibility needs and wishes were met. Equally, the local TV station emphasized the benefits of using ISCEC’s model without the requirement or cost of specialized equipment, instead providing increased flexibility. Mr. Ichise said: “We are so surprised that we got positive feedback from so many people, commending the tech innovation that plays an important role in promoting the use of subtitles across all live broadcasts and ensuring accessible experiences for the communities.”
