Key to Unlock the Power of Internet of Things (IoT) - Big Data & Analytics

In today's digital age, each one of us is surrounded by a variety of data which often comes in various types and forms like newspapers, magazines, emails, advertisements, social media, books, etc. With the onset of digital age and  smartphones in particular, the onslaught of digital data is continues and by far has overtaken other forms of data. As we move closer to realizing the next digital revolution - Internet of Things (IoT), we need to prepare ourselves for the digital data insanity that is going to hit us very soon. In my last blog, I had briefly touched upon the huge potential as well as implications of the amount of data that is going to be generated by Internet of Things (IoT) and will discuss this in greater detail here.

It is estimated that IoT will massively increase the amount of data available for analysis by all manner of organizations. Enormous volumes of machine-generated data from the  Internet of Things (IoT) will emerge. If exploited properly, this data - often dubbed machine or sensor data, and often seen as the next evolution in Big Data - can fuel a wide range of data-driven business process improvements across numerous industries. However, there are significant barriers to overcome before the potential benefits are fully realized.

How Internet of Things, Big Data & Analytics are related?

It is becoming very evident in past few years that technologies have stopped living in a vacuum. They are now more likely to be connected or integrated mostly by sharing data, to provide more enriched and seamless customer experience. The Internet of Things (IoT) are incomplete without a mention of big data. Connected devices, sensors, and algorithms all operate in ways that involve massive amounts of data  generation and exchange.

The success or failure of the Internet of Things hinges on big data," says Brian Hopkins, an analyst with Forrester Research.

As organizations step into IoT, they must understand the symbiotic relationship between IoT and Big Data. For IoT deployments to really make an impact, they must provide some sort of useful tool or service, while also collecting relevant data. Just like with any big-data play, merely collecting the data isn't enough. The data must be processed and analyzed to glean insights, and those insights must drive actionable steps that can improve the business decisions.

The central idea behind IoT is that sensors and microchips can be placed anywhere and everywhere to create a collective network that connects devices and generates data. Instead of that data sitting in an information silo where it is accessible to only a few specialists, it becomes part of a Big Data "lake" where it can be analyzed in the context of other information. To capitalize on IoT and implement data-driven business models, organizations need a platform that helps them generate connected intelligence by collecting, managing and analyzing huge volumes of sensor data in a cost-effective and scalable manner.

Surge in Real-time Big Data

Gartner reported in September 2014 that 73% of respondents in a third quarter 2014 survey had already invested or planned to invest in big data in the next 24 months. This was an increase from 64% in 2013. According to IDC,  big data will grow at a 27% CAGR to $32.4 B through 2017 about 6 times the growth rate of the overall information and communication technology market. Other analyst like Wikibon, are even more bullish, predicting revenues of  $53.4B by 2017, as more businesses begin to realize real benefits from Big Data analytics. 2015 will continue to see solid growth in Big Data analytics tools like SAP HANA and Hadoop, which can deliver results in a matter of minutes or hours as opposed to days, but it is also moving beyond batch and into a real-time big data analytics approach. Preconfigured converged and hyper-converged platforms will speed implementation of Big Data applications.

The 'internet of things' (IoT) and 'big data' are two of the most-talked-about technology topics in recent years, which is why they occupy places at or near the peak of analyst firm Gartner's most recent Hype Cycle for Emerging Technologies

Gartner's 2014 Hype Cycle for Emerging Technologies

Big data, is characterized by 'four Vs': volume, variety, velocity and veracity. That is, big data comes in large amounts (volume), is a mixture of structured and unstructured information (variety), arrives at (often real-time) speed (velocity) and can be of uncertain provenance (veracity). Such information is unsuitable for processing using traditional SQL-queried relational database management systems (RDBMSs), which is why a constellation of alternative tools -- notably Apache's open-source Hadoop distributed data processing system, plus various NoSQL databases and a range of business intelligence platforms -- has evolved to service this market.

The IoT and big data are clearly going to connect billions of internet-connected 'things', generating massive amounts of data. However, that in itself won't be cause of new age digital revolution, or transform our day-to-day digital lives, or deliver a life-saving advance medical system. As EMC and IDC point out in their latest Digital Universe report, organizations need to determine high-value, 'target-rich' data that is (1) easy to access (2) available in real time (3) has a large footprint (affecting major parts of the organization or its customer base) and/or (4) can effect meaningful change, given the appropriate analysis and follow-up action.

Consider this, When you take IoT into the realm of intelligent vehicles, the amount of data begins to boggle the mind. But the upside is potentially colossal. The Internet of Things helps us to connect your vehicle to other vehicles, as well as traffic lights and parking spaces, which can add up to no more traffic jams, fewer red lights and no need to drive the vehicle. As per Cisco, Having autonomous cars which drive themselves would eliminate 80 percent of crash scenarios. However, Big Data is the fuel of these connected vehicles and only real time analytics will unlock the true potential & savings. When you scale up the data being generated  to every vehicle running autonomous, traffic jams would shift from the roadways to the airwaves. Thus, it becomes very critical that the data generated by connected IoT is put through a smart analytical process that is able to identify useful & actionable data from the garbage data. This brings us to another key factor of, Business or IoT Analytics, which will play a pivotal role in shaping up the future of IoT along with Big Data.

Need for effective Business Analytics for IoT

Organizations want real-time big data and analytics capability because of an emerging need for big data that can be immediately actionable in business decisions. In the long term, this ability to apply real-time analytics to business problems will grow as the Internet of Things (IoT) becomes a bigger factor in daily life. An example is the use of big data in online advertising, which immediately personalizes ads & sends specific coupons to customers as soon as they walk into a retail store or visit its websites based on their customer profiles & shopping habits, that big data analytics have captured.

Duke Chung, co-founder and chief marketing officer of customer service support technology company Parature, says a data analytics platform will be a must-have. Chung’s Harvard Business Review blog post actually focuses on how the Internet of Things will affect customer service, but much of it is also applicable to or will require IT.

The fact that millions of devices will soon be Wi-Fi enabled will cause a flood of user data for companies to sift through,” he writes.  “Businesses can use this data to understand where issues are happening on their products, how frequently, and best resolutions — but only if they have the means to analyze it.”

All major players, including IBM, Pentaho/ThinkBig and, industrial players such as GE, are coming up with predictive analytics solutions for the Internet of Things harnessing cloud computing. For instance, GE's "Industrial Internet." The phrase refers to the technologies of cloud computing and the "Internet of Things" applied across a broad range of GE's businesses in an effort to squeeze better performance and efficiency from the operations of everything from computer-controlled manufacturing equipment to gas turbine engines and power plants. It's an ambitious effort that GE is hoping to eventually sell to other companies as a cloud service—branded as Predix. Likewise, IBM is also looking to bring its Watson "cognitive cloud" service to help people understand data from IoT devices. In short, every single one of them is trying to integrate their business analytic capabilities to the large amount of data generated by connected machines and magnitude of sensors. The thought behind this is to devise a new way of running  industries & manufacturing in the future, driven by real-time data, that will fundamentally alter the way corporates run their businesses.

Eventually, analytical systems could make decisions about logistics, plant configuration, and other operational details with little human intervention other than creativity, intuition, and fine motor skills. And even in industries where there is no production plant, analytics could make people more efficient by getting them where they need to be at the right time with the right tools. Before you analyze the data, you will need some way to collect, integrate, aggregate, model and distribute it. Creating that world requires some demanding management of data and the modeling of systems and processes in the physical world that create that data to give it meaningful shape.

It's when data from networked sensors is merged with other sources that it becomes valuable. However, this huge amount of raw data emerging from various connected devices is useless unless there is good analytics process in place that could build data models on any kind of data that represents physical phenomena. Corporations like GE is trying to build certain preventive data models which would fed on thousands of terabytes of maintenance history data and remote diagnostic data recorded from every jet engine in GE's managed fleet—data periodically dumped into the cloud during between-flight maintenance. Models can then calculate projected wear on turbine blades and other components over time, figuring out when it's time to pull them. There are numerous advantages of using a cloud platform to deliver the analytics as well as the data. The cloud is enabling businesses to get up and running more quickly, easily and cost-effectively. The cloud as a deployment model makes years of intellectual property immediately available to anyone that seeks to incorporate IoT into their big data strategy. In short,  the cloud enables companies to tie all its data resources together - structured and unstructured - to generate connected intelligence from all types of data, including Internet of Things data in greater context.

Challenges and Journey ahead

The journey has already started and is getting exciting by the day. However, this also poses tremendous problems in the realms of Big Data and analytics. Where is this data going to be stored? How is it going to be pooled? Where will the analysis be done? Questions on the availability of sustainable IoT infrastructure, Security & Privacy challenges, availability of skills, etc.

At the current technological pace, the IoT is going to become a reality before even we realize. However, we can't be caught unprepared on Big Data and Business Analytics front, if we were to truly harness the potential of IoT – and basic data discovery tools aren’t going to help. Instead we’ll need BI platforms that are powerful enough to handle mind blowing amounts of data. Given what we know of Big Data and Analytics today, are we on the right path to tap into the potential of IoT? 

References: www.forbes.com, www.zdnet.com www.techrepublic.com