Advertisement

Blog

The Internet of Things: Sensor Fusion (or Confusion?)

What keeps the Internet of Things from becoming a tangled jumble of incoming data from various connected devices? And within a single device that feeds information into the IoT, how does sensor fusion work?

Implemented in a wide variety of product categories, sensor fusion is a necessity for applications like wearable gadgets supporting health and fitness and the body motion tracking devices used in the production of advanced CGI movies and gaming. Even products as “simple” as your smartphone require multiple sensors with lots of degrees of freedom and a powerful MCU, to collect, coordinate, process, analyze, filter, and communicate data.

The latest versions of sensor fusion are a set of adaptive prediction and filtering algorithms based on extended Kalman Filter theory that uses quaternion concepts to avoid mathematical singularity and deliver more reliable results. These algorithms “make sense” of all of the complex information coming from multiple sensors, including accelerometers, gyroscopes, compasses, and pressure sensors, by taking each sensor’s measurement data as input, and compensating for drift and other effects and limitations of each individual sensor, to output accurate and responsive dynamic results.

For one example, let’s take a look at the application of pedestrian dead reckoning (PDR) and the fusion of four sensors and five inputs. You’ve got your GPS input as well as your accelerometer, gyro, magnetometer, and pressure-sensor data coupled to an MCU.

Working together, a three-axis accelerometer and three-axis gyroscope function as a strapdown inertial navigation or pedometer-based portable navigation device. The accelerometer provides step detection, and the tilt-compensated compass — a three-axis magnetometer — if disturbed, allows the gyro to make heading adjustments. The compass calculates magnetic field and compensates for the gyro’s zero-rate drift over time. Meanwhile, the pressure sensor, working with the accelerometer, acts as an altimeter and conveys floor changes for indoor navigation.

So much data, so little time…

The MCU houses the Kalman Filter, which predicts sensor errors from the equations of inertial navigation. It estimates and compensates for the gyro’s long-term bias drift, magnetic anomalies, and provides data for dead-reckoning applications when GPS info is unavailable.

Et voila! Sensor fusion.

In the next few years, the market pundits project significant growth of sensors fueled by sensor fusion. According to many analysts, that growth in annual sales could increase from the multimillion units recorded in 2012 to reach more than a couple billion by 2016.

This analysis suggests the IoT will be pretty limited to the kinds of applications we’re all familiar with. But if you close your eyes and dream a bit, perhaps you can see, as I do, components of future sensor modules expanding beyond the current fusion of pressure sensors, accelerometers, gyros, and magnetometers to such applications as PDR and cardiac monitoring by including UV index sensors, gas, and volatile organic compound sensing for applications such as safety SmartWear. In my mind, robotic autonomous surgery functioning through sensor fusion is no longer the stuff of science fiction, and bionics, nano-biosensors fusing organic tissues with sensor technology, is the next logical step in sensor fusion.

21 comments on “The Internet of Things: Sensor Fusion (or Confusion?)

  1. Netcrawl
    July 23, 2014

    @Jay that was great! thanks for that. Sensors are undergoing a renaissance of sorts as MEMS technology becomes less expensive and further minituarized,  fueling the deep penetrations of sensors into new applications and new market. Sensors are used almost everywhere- mobile space, home automation, automotive, health care, oil exploration, manufacturing and military.

    Sensor fusion is taking a leading role in IoT, advancement in sensor fusion could led to some new exciting new applications and big development, today's sensor technology is taking a big leap, its now starting to mimic the ultimate sensing machine- the human being.

  2. samicksha
    July 23, 2014

    What makes me confused in sensor fusion is their behaviour of centralized versus decentralized which refers to where the fusion of the data occurs.

  3. Myled
    July 24, 2014

    “Implemented in a wide variety of product categories, sensor fusion is a necessity for applications like wearable gadgets supporting health and fitness and the body motion tracking devices used in the production of advanced CGI movies and gaming.”

    Jay, in IoT, sensors are playing a vital role for data collection and processing. In other ways we can say sensors are the interface for collecting the input datas in real time manner, on which further processing are happens. For wearable technologies, especially in health monitoring system sensors playing the same role.

  4. Myled
    July 24, 2014

    “What makes me confused in sensor fusion is their behaviour of centralized versus decentralized which refers to where the fusion of the data occurs.”

    Samicksha, now a day's some of the companies are offering sensor as a service through cloud. they will deploy the sensors at customer place and it get connected to the central processing unit through cloud.

  5. Myled
    July 24, 2014

    “Sensors are used almost everywhere- mobile space, home automation, automotive, health care, oil exploration, manufacturing and military.”

    Netcrawl, sensors are an integral part of the automation system, irrespective of whether it's a home automation or healthcare or military. The place where real time datas has to be captured can make use of various types of sensors.

  6. Netcrawl
    July 24, 2014

    @myanalog, I agree with you about that confusion you seen here, but for me the single biggest issues facing the sensor industry is the lack of standardization across various operating systems, this limit the use of the full capabilities of the sensors. Sensor fusion could provides us with a whole new host of capabilities that could enables a variety of services. But we're not able to achive that because of standardization issues we're facing today. 

  7. Myled
    July 24, 2014

    “but for me the single biggest issues facing the sensor industry is the lack of standardization across various operating systems, this limit the use of the full capabilities of the sensors.”

    Netcrawl, I have to agree for this.  What we are following is just matching the parameters with associated components.

  8. Netcrawl
    July 24, 2014

    @Myanalog in a cloud-based systems, big data algorithms can be leveraged to provide a more elaborate and accurate response, cloud could make a huge impact in sensor technology in terms of data processing and storage. 

  9. Myled
    July 25, 2014

    “in a cloud-based systems, big data algorithms can be leveraged to provide a more elaborate and accurate response, cloud could make a huge impact in sensor technology in terms of data processing and storage. “

    Netcrawl, you are right. centralized processing are happening and hence a low investments for such things from the customer side.

  10. samicksha
    July 28, 2014

    This is very interesting sensor as service, i,e, now we don't have to search for sensors separately, all of them could be found under one roof. Any company in your knowledge already providing this kind of service.

  11. geek
    July 28, 2014

    “This analysis suggests the IoT will be pretty limited to the kinds of applications we're all familiar with”


    I think that's the beauty with IoT. The applications will be the ones we're very familiar with but the integration will enhance the usablity greatly. It will involve simple applications like informing your garage door when your car is approaching, or keeping a track of your refridgerator inventory. Yet, despite these familiar applications it will aim to make lives better.

  12. geek
    July 28, 2014

    “Sensor fusion is taking a leading role in IoT, advancement in sensor fusion could led to some new exciting new applications and big development, today's sensor technology is taking a big leap, its now starting to mimic the ultimate sensing machine- the human being”

    @Netcrawl: Sensor technology is indeed aiming towards getting closer to how human beings perceive objects. The technology is not only aiming for higher accuracy but is also looking to broaden the range of data that can be sensed. And, of course, the beauty lies in the integration part.

  13. Myled
    July 29, 2014

    “This is very interesting sensor as service, i,e, now we don't have to search for sensors separately, all of them could be found under one roof. Any company in your knowledge already providing this kind of service.”

    Samicksha, so far it's not commercialized like cloud services. But can happen within a year.

  14. Sachin
    July 31, 2014

    Sensors are undergoing a renaissance of sorts as MEMS technology becomes less expensive and further minituarized,

    @Netcrawl, totally agree with you. As sensors sizes shrink we can add more and more functionality to the existing system. I am sure MEMS technology will find applications in many other fields which so far was considered highly unlikely to implement.

  15. Sachin
    July 31, 2014

    but for me the single biggest issues facing the sensor industry is the lack of standardization across various operating systems, this limit the use of the full capabilities of the sensors.

    @Netcrawl, I agree with you. I am not sure why there is no standardization across vearious operating systems. I am curious to know if the process of standardization has already started or do you think the same trend will continue ?

  16. Netcrawl
    July 31, 2014

    @SachinEE I believe its already started but still in the early stages. The lack of standardization makes programming for sensor devices an arduous task, after a sensor-based program is completed and released, and its usually dependent on the type of hardware.

    Using one or more solutions to enable deloyment of sensor-based system has limited and slowed the integration of sensors with hardware. There must be interoperability, Can you imagine a world without standard? cars ould not fit in the driving lane, apps would not work on mobile devices,each manufacturers would need to invent and build solutions to its problem. 

  17. yalanand
    July 31, 2014

    @Tzubair : Ineed. Beauty lies in the integration of such sensors into device grod (where sensors and devices can talk to each other without operational latency), and this device grid is the concern of most standardization boards lik IEEE who constantly hold meetings and conferences on how should the IOT grod ( and the underlying basic architecture itself) be designed for assembly lining.

  18. yalanand
    July 31, 2014

    @Myanalog : that depends upon the area you are in. Most SaaS (Software as a service) and PaaS )platform as a service) vendors are looking at a bring future where sensors market will boom, and the change is being bought by Samsung, GE, LG and other electronic supply chain giants. The rate they are moving, it might be before a year when sensors would be the household name.

  19. Netcrawl
    August 2, 2014

    @yalaland yes you're right most cloud vendors are looking for bright future here but sensor market is still in early stage and its going to takes some time before we can see some result.

    But there's some great advances like the recently launched Kinesis service, a fully managed service for real-time processing of streaming data at a massive scale, Amazon's description of Kinesis service is pitched mainly to its AWS customers who are looking for an effective way to capture and process live data. It's going to a year or even a decade before we can fully see the impact of sensor technology.

  20. Myled
    August 11, 2014

    “Most SaaS (Software as a service) and PaaS )platform as a service) vendors are looking at a bring future where sensors market will boom, and the change is being bought by Samsung, GE, LG and other electronic supply chain giants. The rate they are moving, it might be before a year when sensors would be the household name.”

    Yalanand, there is no doubt for that. Cloud and 'sensor as a serve' can complement each other.  Wherever there is automation or IoT based systems, sensor can be an integral part of that.

  21. Davidled
    August 11, 2014

    There is an unlimited application for Cloud based IoT.  However, it is possible that sensor technology could not meet the IoT requirement. For example, let us build smart city or building. I image how many sensors could be required for making the smart city. Alternatively, sensor function could be extended.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.