Azure adds real-time analytics for Hadoop and new machine learning capabilities

Today at Strata + Hadoop World in New York, one of the industry’s premier big data events, we’re announcing the expansion of our Azure data services. These announcements and our participation in the event are part of our commitment to bring big data to everyone by leveraging the power, flexibility and scale of the cloud.

We’re announcing support of real-time analytics for Apache Hadoop in Azure HDInsight and new machine learning capabilities in the Azure Marketplace. Our partner and Hadoop vendor Hortonworks also announced how they are integrating with Microsoft Azure with the latest release of the Hortonworks Data Platform. On Friday, my colleague Corporate Vice President of Machine Learning Joseph Sirosh, will give a keynote at the conference where he will discuss how removing the barriers to broad adoption of advanced analytics creates new opportunities for our customers and data scientists.

Big data, including Hadoop and advanced analytics, is changing the way our customers do business. As they collect and store more data than ever before, they expect more from their data and want more insights from it, including being able to do real-time analytics over streams of data to complement their existing Hadoop deployments. Microsoft’s approach is to make it easier for our customers to work with data of any type and size — using the tools, languages and frameworks they want — in a trusted cloud environment.

To do this, we are delivering a comprehensive set of cloud solutions — from Azure Machine Learning, Azure HDInsight and our other Azure data services — to managed data services from our partners. We’re committed to supporting the broadest data platform so our customers find real value in the cloud, on their terms. With our big data and cloud solutions, customers like MediaCom, Pier 1 Imports and ThyssenKrupp Elevator have more options to enable new scenarios to gain deeper insight from their data and improve the way they deliver services and products to their customers.

Azure HDInsight combines the best of Hadoop open source technology with the elasticity and manageability enterprises require. Available in preview today, we are supporting Apache Storm in HDInsight, allowing our customers to process millions of items of Hadoop data from their Internet of Things devices in near real time using a fully managed Hadoop service. By bringing real-time analytics capabilities to HDInsight, we are opening up new customer scenarios such as the ability to analyze operational data in real time for predictive maintenance.

apache-storm

On Wednesday, Microsoft introduced Apache Storm for real-time analytics for Hadoop.

The preview availability of Storm in HDInsight continues Microsoft’s investment in the Hadoop ecosystem and HDInsight. Recently, we announced support for HBase clusters and the availability of HDInsight as the first global Hadoop big data service in China. And together with Hortonworks, we continue to contribute code and engineering hours to many Hadoop projects.

As part of their integration with the Azure platform, Hortonworks Data Platform (HDP) has achieved Azure Certification. In the next version of HDP, Hortonworks is also delivering hybrid data connectors so customers can extend their on-premise Hadoop deployments to Azure and leverage the cloud for backup, scale and testing.

apache-storm-2

Announced Wednesday, Hortonworks HDP 2.2 includes hybrid connectors to move data to Microsoft Azure. It will be available in November.

Introduced this summer and available now in preview, Microsoft Azure Machine Learning helps customers and partners rapidly design, test, automate and manage predictive analytics solutions in the cloud. For example, search engines, online product recommendations, credit card fraud prevention systems, GPS traffic directions and mobile phone personal assistants all use the power of machine learning to provide people with valuable insight.

Today, we are introducing new machine learning capabilities in the Azure Marketplace enabling customers and partners to access machine learning capabilities as Web services. These include a recommendation engine for adding product recommendations to a website, an anomaly detection service for predictive maintenance or fraud detection and a set of R packages, a popular programming language used by data scientists. These new capabilities will be available as finished examples for anyone to try.

On Thursday, Joseph Sirosh will share more about these new capabilities on the machine learning blog. To find out more about how Microsoft’s data platform is helping customers create new solutions that bring together big data insights, predictive analytics and powerful visualizations, go here.