ISE Blog

Garbage In, Garbage Out: Have You Met My Friend IoT Yet?

You can have an amazing system that solves all your problems, but if you feed it bad information you should expect bad results. IoT is no exception.

Business CatIoT is hot, it’s everywhere, and it’s going to solve all your organization’s problems! All you have to do is get a few sensors, a network connection, and a generic big data service, and you will triple company profits! 

You start your project convinced it’s better than sliced bread, only to eat endless crow sandwiches staying up late at night with your team trying to figure out where the profits went.  While you’ve sliced and diced your data like a 7th grader playing Fruit Ninja, all you’ve got is a tin of sardines. It slowly dawns on you that IoT isn’t an elixir that gives your organization +20 Profit points in your march toward market dominance. While you sprinted to catch the best data scientists your organization could find, you may have ignored the first, and arguably most important, link in the IoT chain. I’m talking about the sensors and the data they provide. The reality is that the sensors and the data they provide can suffer from a variety of problems, a few of which we’ll cover in this blog post.  

Wait…what’s a sensor?

Most readers have some understanding of what a sensor is. For example, they recognize there is some piece of hardware in their activity tracker that monitors heart rate or movement. But for the purposes of this post let’s provide a definition as provided by Wikipedia

"In the broadest definition, a sensor is an object whose purpose is to detect events or changes in its environment and sends the information to the computer which then tells the actuator (output devices) to provide the corresponding output. A sensor is a device that converts real world data (Analog) into data that a computer can understand using ADC (Analog to Digital converter)"

Translating the above, a sensor takes an environment variable – pressure, temperature, humidity, etc – from an analog state and turns it to a digital state that a computer system can understand in the form of a voltage.  If it’s 80 degrees out, your thermostat may report that as 3.5 volts. When it’s 90 degrees, maybe it’s 3.75 volts. 

Now that we have a basic definition, let’s move on to what can go wrong with sensor and sensor data for IoT.

That’s a bad sensor, bad!

Bad DogYou can teach an old dog new tricks, but sadly your sensors aren’t that teachable. Here’s a well-known truism that often gets forgotten; sensors lie. I’m not saying your sensor is a sentient being that doesn’t think you can handle the truth, I’m saying they break, short, have drift, and can be impacted by the environment around them. When this happens, the data reported is inaccurate. Maybe it’s actually 80 degrees out, but your temperature sensor reports 83 due to drift. Maybe the sensor is broke and will only report 80 degrees no matter what the real temperature. If you do not have a comprehensive system in place to detect situations like this, your analysis will be based on faulty data and you’ll get a visit from the garbage in, garbage out fairy. 

Seriously, how many times are you going to tell me this story?

Annoyed WomanEven if your sensor is reporting accurate information it could be reporting too much information, and that could be clouding your analysis. Big Data along with IoT is all the rage. All you need to make Big Data great is even more data! In fact, you need to know everything from every sensor on your device that has happened ever or you’re a failure.  Actually, not so much. What’s really critical is getting the right data at the right time, not just information for information’s sake. All too often systems report and store data that adds no value and clutters up analysis.

A simple example is that of a door sensor on a refrigerated truck. If you were monitoring such a door you would only care when there has been a change in state; when it is opened or when it is closed. If it closes at 1:00 PM and opens again at 4:00 PM do you need sensor data reporting every minute that it’s still closed? Of course not, a report at 1:01 of ‘door closed’ tells you nothing new and just add toxic sludge to your data lake. When thinking about your sensor network, make sure you are getting the ‘just right’ amount of data for your analysis.

That’s not what I was asking for!

One last thing to note about sensor data is that one can rarely take action based on a single piece of data. Two types of data relevant to IoT are observation data and transaction data. Observational data is something we measure such as temperature, humidity, pressure, etc. Transactional data tells us elements such as information about our sensor network topology, message rates, message size, etc. When creating an IoT solution that can add value both types of data need to be considered. Put another way, rarely is it just a piece of data from a sensor that makes a decision or action easy to determine.

Milk JugReturning to our door sensor on a refrigerated truck, if the door opens unexpectedly do we care? Maybe, maybe not. If there is a trailer full of milk one would certainly want the door shut and the temperature maintained, but if the trailer is empty and sitting at a rest stop it is unlikely to be a concern if the door reports as open. It’s only by pulling in the right types of data at the right time that we can know when some data matters and when it can be ignored. This data will come from a network of sensors that can be thought of as the nervous system of your device. The host side is the brains and can interact with the rest of the system. 


So what do you think? Do you have a better grasp on what sensors are and how they can be used? We hope so! If you're interested in exploring how IoT can help your organization thrive, contact us at ISE
Hudson Ludvigson, Senior Software Engineer

Hudson Ludvigson, Senior Software Engineer

Hudson Ludvigson is a Senior Software Engineer and the Practice Lead in Vehicle Telematics at Innovative Software Engineering. He has been with ISE since March 2006. He enjoys the diversity of the software engineering field and how it impacts and improves peoples' everyday life, particularly in the domains of agriculture, business, finance, and medicine. In his downtime he can be found enjoying college football and outdoor recreational sports, or spending time with his family.