Solving a common corporate conundrum: Making sense of all that data

Most companies today are collecting enormous amounts of data, and chances are they know that data contains crucial insights about everything from what customers want to purchase at 10 p.m. on Friday versus 7 a.m. on Wednesday to how they could run their businesses more efficiently any day of the week. What’s more, that data is all available in real time.

But too often companies can’t hear those signals, or they hear them too late.

“It would be an understatement to say we’re able to see just the tip of that iceberg. It’s more like we are analyzing a single ice cube out of that iceberg,” said Daniel Yu, director of product marketing, Azure data and artificial intelligence at Microsoft.

Part of the problem is that there’s so much data, and it’s so difficult to understand. Much of this valuable information is in what’s called unstructured or semi-structured data — generated from customer interactions on the web, software as a service apps, social media, mobile apps or IoT devices such as connected refrigerators and intelligent assistants. It is then stored in the cloud, where many of the tools for analyzing it are still maturing.

Yu said that’s left companies feeling they have two choices: They can either have powerful systems that can do really sophisticated analysis but require them to know exactly what their needs are upfront, or they can opt for more flexible systems that don’t give them as many choices for sophisticated analysis and are more time-consuming to manage .

“We think that’s a false choice,” Yu said. “You can have both power and flexibility in analytics, at a reasonable cost.”

Two women sit at a desk, chatting with a man standing next to them

From left, Lidia Rozhentsova , Sofia Iasonidou and Niklas Arbin of BookBeat discuss data analytics tools they have used to make sense of the overwhelming amounts of raw data they gather every day. Photo by Alexander Donka for Microsoft.

On Thursday, Microsoft announced that its customer offerings are getting an upgrade, with the general availability of Azure Data Explorer (ADX) and Azure Data Lake Storage (ADLS).

Microsoft says its Azure-based cloud analytics platform delivers the industry’s best price-performance ratio, a standard that measures the speed of a system against its hourly cost. According to independent testing by GigaOM, analytics with Azure SQL Data Warehouse is up to 14 times faster and costs 94 percent less than other cloud analytics offerings.

Microsoft said ADX can analyze 1 billion records of streaming data per second, using simple query language, while leaving the data and its metadata in its original state. ADLS provides a repository for storing massive amounts of structured or unstructured data, with the efficiency and security features of Azure Blob Storage. This combination is optimized for analytics.

That’s exactly the kind of offering the company says everyone from small startups to big established businesses need.

A good example is BookBeat, a European-based streaming audiobook service that runs its business on the Azure platform. The company uses data analytics to serve customers crisp recommendations based on their own reading history and those of customers with similar interests. It also relies on data to launch new business models, like a shared family account that it rolled out after its data predicted, correctly, that it would succeed.

“All our teams are data-driven,” said Niklas Arbin, head of developers at BookBeat. “We use it for everything.”

Arbin said that, with Azure managing its server infrastructure, BookBeat’s technical specialists are free to work on other essential tasks. That includes mining the data to deliver real insight and value and building internal tools for managing the vast streams of books it offers.

“Azure enables us to use the highest standards in application development, which has been very hard to do in any business intelligence toolset,” Arbin said. “It gives our developers freedom to choose the best tool for the job.”

A desktop with a collection of books neatly stacked against the cubicle wall

Many of the desks at BookBeat’s offices in Stockholm, Sweden, are loaded with analog books as well. Photo by Alexander Donka for Microsoft.

The company doesn’t even have a traditional IT department.

“We don’t have to worry about things like uptime for servers,” Arbin said. “We don’t like working in the middle of the night.”

To John Chirapurath, general manager of Azure data, Blockchain and AI at Microsoft, that sounds like success. He said Microsoft’s goal is to remove complexity for customers wherever possible, from ingesting data to presenting it.

“We always strive to make it very easy for IT staff to adopt analytics and for line of business people to utilize and deliver powerful insights using beautiful products,” Chirapurath said.

Microsoft says another selling point for customers is the company’s long history of securing its Azure cloud, which includes helping customers conform to privacy standards and regulation such as the General Data Protection Regulation, or GDPR.

To Yu, the ongoing advances in cloud analytics technology, and the relentless flow of useful data, are bringing customers to a tipping point. For many, he said it’s no longer going to make sense to host all that data on premises, and devote so many resources to managing those resources, when there are so much more interesting analytics that can be done in the cloud.

“Data and analytics are changing everything for businesses,” Yu said. “There’s not a single company that isn’t thinking about this.”

Related:

Top imageAnna Taban-Franz, center, of BookBeat looks at a Power BI visualization of book sales per market in Sweden, Germany and Finland. Photo by Alexander Donka for Microsoft.