Bing adds new intelligent visual search features

As the saying goes, a picture is worth a thousand words. Microsoft’s new intelligent visual search technology allows users to discover information about objects captured in images without having to pick and choose a handful of keywords to fit into a search box.

The AI-powered visual search feature is available on Bing mobile apps.

“Sometimes, it is almost impossible to describe what you want to search for using words,” explained Vince Leung, product lead for Bing Images at Microsoft.

For example, imagine hiking through a meadow and seeing a flower that you’ve never seen before. You want to know what it is and whether you can get it at your local garden store to plant at home. Bing’s Visual Search can help you identify and find more information from your snapshot of the flower.

Or, perhaps you’re in the market for a new couch and spot one you like in a high-end home furnishing store, but the price tag is beyond your budget. By taking a picture of the couch, Bing’s Visual Search can help you find couches that match the style with prices that may meet your budget.

The visual search feature uses Microsoft’s computer vision algorithms, which are trained with datasets containing vast amounts of labeled images, as well as images from around the web. From the training images, the algorithms learn to recognize dogs from cats, for example, and roses from daisies.

What’s more, the learning process is never done; the performance of the algorithms improves as they get more data.

“While there have been strides for many years to get to this point,” noted Leung, “with the advent of cloud computing we are able to accelerate our ability to make sense out of pixels.”


John Roach writes about Microsoft research and innovation. Follow him on Twitter.