Translation tech powers automatic subtitles for everyday life


Editor’s note: The following is a post from Athima Chansanchai, Microsoft News Center Staff

The world is a big place, but advancements in language translation are making it a little smaller every day. A shared understanding emphasizes what we have in common, rather than the chasms that keep us apart. While we’re not living with “Star Trek”-like universal translators yet, we are able to point smartphones at unknown signs with words instantly translating on the screen before our eyes. It’s not magic, but it sure feels like it.

It’s a pretty great place in time to be as we celebrate International Translation Day Monday. It’s an amazing time to revel in the reach of our translation services, no matter where you are in the world, and no matter what device you’re using.

Microsoft’s translation research is powering advancements all around you, from phone to desktop and beyond. For more than two decades, Microsoft has made deep investments in machine learning and natural interfaces that has produced tools people use every day. Language is no longer an obstacle in understanding emails and web pages.

Even online relationships can spark without a common tongue. You may not realize it, but you’ve probably already been using the machine translation technology from Microsoft Research. The machine translation team originated in the Natural Language Processing group, which was responsible for the spelling and grammar tools within Microsoft Office and includes scientists, lexicographers, linguists, software engineers, and machine-learning experts.

Breaking down language barriers is an incredibly complex problem, but it has also spawned experiences that touch our lives every day. And the future promises even more natural ways for people to communicate.

Bing Translator is used daily in 187 countries — 95 percent of the world’s countries. More than 80,000 developers are using Microsoft’s Translation API. And every day, millions of users use the Bing Bar to instantly translate Web pages. That’s a lot of shared understanding that we’re proud to be delivering to our customers around the world.

For those who use Word, the Translator app in the Office Store is launching Monday in conjunction with International Translation Day. Expanding on top of built-in functionality within the products, it allows you to select any text and instantly translate it. Using this app, organizations can utilize within Office the custom translation systems they built with the Microsoft Translator Hub. Already, Office users translate over 1.3 million characters each hour using the built-in translation features.

The Microsoft Translator Hub empowers businesses and communities to build, improve, and deploy their own automatic language translation systems, which continues fine-tuning the translation quality of established languages and bringing in other languages not yet supported by major translation providers. An average of 99 customized translation systems are built each month with the Microsoft Translator Hub.

In the aftermath of the devastating earthquake in Haiti, Microsoft’s translation technology helped bridge the language gap by helping people communicate in Haitian Creole and first responder’s native tongues, and has also worked with communities around the world to preserve languages (such as Mayan and Hmong).

“Be it a farmer in India able to understand better farming practices from one in China, or a mother in Africa able to learn about child nutrition, or a student of history in Canada following a revolution in Egypt, or a consumer in Russia buying a stylish hat from someone in the U.S., or even a kid in Israel becoming a pen pal with one in Iran – translation technologies can help bring down language barriers for information and communication,” says Vikram Dendi, whose Microsoft Research Machine Translation team does the lion’s share of the behind-the-scenes work powering the machine translation technology behind Bing, SharePoint, Office, Yammer, Lync, and many other Microsoft technologies.

Some of that work includes analyzing how computers can learn from the massive amounts of data being generated by online interactions, devices, services, social networks, and sensors — and how that big data can be transformed into real-time language translations.

Just think about being in a foreign land, trying to navigate your way, and you see a sign that may point you in the right direction. You whip out your Windows Phone, open the Bing Translator app and point your camera at the sign. Within seconds a translation of the text overlaid on the original text appears. The technology behind this is made possible through Translator’s partnership with Microsoft’s Augmented Reality team using a real-time OCR engine on Windows Phone. As the video frames the image, it’s able to recognize text, extract it and translate using Translator.

“I love this app,” says one customer who was using it in Germany. “Translating is as easy pointing the camera at text and waiting a few seconds. It’s free and works pretty darn well.”

Even if you don’t have an internet connection, you can still travel with confidence by downloading the offline language packs in advance. With the offline language packs, you are still able to leverage the robust power of the Translator app for type, speech, and camera translation for a number of existing supported languages with more being added in future releases.

And, you don’t even need to be out of the country to find a use for such an app – especially in a multi-lingual salad bowl like the U.S.

“I’m at the Smithsonian in D.C. and wanted to test how it translates English to Spanish,” says another customer. “In short, it works impressively well. Apps like this make me feel like I’m living in ‘the future.’”

“Apps serve a very obvious human need, very real, which is especially important when we’re traveling, when we’re most reliant on information technology to help us get around,” says Blaise Aguera y Arcas, a distinguished engineer who heads a team that works on next-generation experiences and technologies and were instrumental in creating the magic in the Bing Translator apps.

To help developers create apps that appeal to people who speak other languages, the Translator Control allows developers to harness the power of Translator for their own applications. Likewise, for webmasters who want their sites to reach a much wider audience, the Translator Widget allows visitors to a site to see real-time content translation in any of more than 40 supported languages. Webmasters can also enable the collaborative translation framework (CTF) to harness the power of their user community to improve translations over time, along with building customized machine translation systems with the Hub.

Translation comes down to understanding, and with online social networking increasingly tied to spreading cross-cultural information, getting those basics down is a must. Facebook, Socl, and Twitter all use Microsoft’s translation API – and what ties them together is that core of understanding.

“If you are a native speaker, you won’t be very impressed. But the goal is not to try to compete with a native speaker, the goal is to make a person understand,” says Dendi. “With the advent of Facebook and Twitter, suddenly our social networks don’t have physical boundaries anymore and translator removes the language ones as well.” Dendi says people don’t have to rely on communicating in a common language. “Now you can expand your network even further or still follow your friend when they are saying something in their native language.”

Dendi has seen social networking moving into business, like Yammer, where he says people also are much more comfortable talking in their own language, creating a much more vibrant environment and improved productivity with the addition the new translation features in Yammer. In addition, enterprise customers like eBay, Adobe, Autodesk, Ernst & Young, LinkedIn, HP, and Intel are just some of the companies that are leveraging machine translation to reach their customers around the world.

Whether you are using translation for work or pleasure, Microsoft has been able to tailor this technology for all kinds of applications and audiences. In developing the translation technology itself, the teams kept their focus on delivering developer friendly APIs, and tools that allowed organizations to build custom translation systems that would best fit their needs. Developers relying on this cloud based translation service will appreciate the emphasis the translator team placed on simplicity and scalability, resulting in greater agility for both the team and those that use the service.

The next big step in language translation could include a not-too-distant future where something like the “Star Trek” universal translator is possible. (After all, Bing Translator already supports Klingon!)

“It is certainly one of the most natural, high impact steps. Speech is, by and far the most natural form of interaction – and it makes a lot of sense for us to combine the great research we have done in speech technology with the world class machine translation work we are doing,” Dendi says. “It is also one of the most interesting challenges, which includes solving problems not just in these two research areas, but also coming up with exceptional user experiences that are needed to deliver this seamlessly to the user. We expect we will be able to advance the state of the art in all of these areas as we move towards solving the speech-to-speech translation challenge. We have made a tremendous amount of progress, as you know – and there is a lot of interest in this.”

Aguera y Arcas sees an even more ambitious future.

“I think that translation is one of those things that belongs more as a super power than as an app. It should just happen for you,” he says. “I meet a handful of people every day, who know who I am, and I would like the super power of knowing the names of everybody else I’m meeting. Translation I see as similar – it’s a very frequent necessity while you’re traveling, or working in a global environment. In the moments, you need it, you want it to just work.”

So, as we celebrate International Translation Day, there’s never been more opportunities to break down language barriers and connect with others.