Processing Data From the Edge: A Platform for the Internet of Your Things

 |   Microsoft Corporate Blogs

techEd
At TechEd in Barcelona

Last week in Barcelona at Microsoft TechEd Europe, we had the opportunity to talk with customers and partners about the Internet of Things (IoT) and the Microsoft technologies that can help companies realize IoT’s potential today.

We all know what IoT is conceptually: a physical network of basic sensors and intelligent edge devices that have become a rich source of data. By connecting these to each other and cloud services, we can understand their state, generate context about their use, and utilize cloud-based computing resources like Azure to drive analysis and extract insight.

It’s a simple approach that can generate powerful results. A recent report from GE suggests that even small improvements in efficiency can make a big difference. The study points out tens of billions that could be saved across industries such as aviation, energy, healthcare and transportation with just a one percent improvement in fuel use, process efficiency or capital expenditures.

How can a single company achieve the small efficiencies that add up to such huge savings? When tuned to your environment, with your assets and your data, IoT becomes “the Internet of Your Things“—a powerful tool that can help you finesse processes, identify anomalies and make better, more proactive decisions.

During the event last week, we talked about the architecture for the Internet of Your Things, and what Microsoft provides today to make this type of change possible. The following are some of the main considerations we discussed.

Getting started with data telemetry

First, it’s important to understand your goals and the insights you want from your IoT implementation—the questions to answer. Do you have a specific business challenge, or are you merely investigating? Is the data time-based, event-based or other? Is the scenario real time, in which device interaction is necessary, or over a period of time such as month to month? Is the goal to simply detect anomalies, or is it focused around another measurement like growth or average values?

With a strategy for the basis and goals of your collection efforts, the organization can begin to understand the frequency and type of data it needs. The insight to be gained may call for a certain amount of volume, momentum or time progression to bring clarity. Finally, being able to determine how much of the data is needed can help in understanding what types of compute, processing and storage resources will be required.

Digital Enterprise Vision_png

Once a plan for obtaining the right data is determined, the architecture for the Internet of Your Things can be broken down into 3 primary areas:

Data sources at the network edge

This is the edge of IoT. Data sources here can be the most basic of sensors, limited functionality devices or fully functional devices with operating systems. The edge of IoT will often not be an OS-based device, but the data sourced from those points is critical to the larger context of your architecture.

These sources of data may directly connect to cloud services like Event Hubs, or they may utilize gateway style technology. Gateways may exist to translate from short haul to long haul protocols (BLe, ZWave etc) onto IP addressable ones, assist in data formatting, structuring, or assist in command and control or provisioning capabilities to edge devices.

Within an Azure IoT architecture, Microsoft provides sample code, SDK documentation, community and enterprise grade assistance to implement connectivity that is language or development tool agnostic.  .NET, Java, Ruby, Python, Node.js, or iOS, Android and Windows.

Cloud-based infrastructure to receive, process and store your data

When connecting your devices to Azure, Event Hubs are the destinations at which data arrives to be processed. Within this, Service Bus namespaces use queues, topics and relays to process events on their way to storage. These are configured within the Azure Management portal, and can be processed by custom code, Storm (HDInsight) or Stream Analytics — or they may pass through adapters preparing the data directly for storage.

The destination for the data could be Azure SQL, Table, Blob, HDInsight, or additional Service Bus processing depending on the nature of the events. In addition, tools like Azure Machine Learning may be connected to drive algorithmic approaches to predictive maintenance scenarios.

Presentation experiences, data analysis and dashboards
The value of an IoT investment is realized through data insight — the ability to understand not just the data from your devices, but to also understand it in context by combining other data sources. Through the use of portals, apps and productivity tools like Power BI for Office 365, data analysis can be performed to show trends, patterns and deeper analysis than a traditional event log could show.

With these tools connected to cloud platforms and services through secure data endpoints or OData surfaces, there is no limit to the ability to combine rich UI with data telemetry.

Read more about Microsoft’s recent innovations on Azure here.

Tags: , , , , , ,