No Comments

IoT and Data challenges


by , , No Comments


On one hand IT systems are grappling with deluge of data and are scrambling to put systems in place to make sense out of it. On the other hand we have now almost every object surrounding us capable of generating data. As an example take a look at these seemingly trivial objects : Toothbrush, shoe, helmet, power hammer, digital photo frames, medicine dispensing box, on site construction machinery, room heaters, washing machine, refrigerators, pollution sensors and the list goes on and on, all of these now have sensors and are generating data.

In 2010, Eric Schmidt said that every 2 days we were generating as much data as human civilisation had ever generated till 2003 from its beginning. To my mind, this data was largely generated due to human interactions. Now we have an additional dimension of sensors and devices generating data on their own.

The prospect of every object around us turning into data generating device is a new challenge that science and technology has never encountered. In a broad sense this trend is being called as Internet Of Things. It is so big that to encompass all the objects it is simply being referred to as ‘Thing’. The real challenge is in establishing successfully a chain of – gathering data, converting it into information, transforming it in knowledge and looping back this knowledge to make these devices smarter. This challenge would require collective work of data storage technologies, data analysts, domain experts and visionaries who would be able to think of a new world given these possibilities. The design approach for developing futuristic devices and solutions would be significantly different than used in the past. The decision making would be based on real time data availability. Yet just capturing and storing the data in real time would not mean anything unless domain experts are able to convert it into actionable decisions.

I think we are in the 1st wave of new age decision support system. This wave is about creating technologies to help capture and store this vast amount of data which is often referred to as having 3 characteristics – velocity, variety and volume. We are not yet there where we can leverage this data for real time decision system (say, of last 10 min or 10 sec of data set). As we move towards this goal, we would need newer approach to data storage (database management) system.

There are emerging approaches in the form of Hadoop, MongoDB etc that are trying to address these needs. Even the existing RDBMS technologies are offering NoSQL structure to address these needs. Only time would tell if we need even different approach than being offered by current and emerging technologies. We have lived with RDBMS structure for over 30 years, but the pace of data generation is so fast that I think these emerging data storage technologies would not get to live that long, it would take different form than existing one within a shorter period of say 5 to 7 years.

Interesting times are ahead.

– Sachin Dabir, CEO and Director | Ashnik, UK


More from  Sachin Dabir :