Analog Angle Article

When your data is accurate but misleading

I recently saw an ad for a device which measures a person's core-body temperature by using infrared (IR) to sense the person's forehead and its major artery heat emissions. Checking the forehead is, of course, a well-established qualitative technique, used by mothers to say “you feel hot, you're sick” or “you're fine, go to school” and as such, it has its validity and its limitations.

But the ad for the Exergen Temporal Scanner said it was as reliable and valid as a measure of body temperature as the more rigorous “internal contact” methods of measurement. And that made me wonder if it was the case.

Long story short: it probably is, but not necessarily. The problem is not the accuracy of the infrared temperature measurement itself; IR measurement can be very accurate, especially if the unit is properly calibrated and the IR emissivity of the object is known. The issue here is that the relationship between the critical core-body temperature and the observable forehead temperature varies between people and even for the same person.

Even a slight difference in this correlation wipes out much of the needed data credibility. After all, a shift of half a degree to a full °F (one-quarter to one-half °C) may be medically significant in some cases. (You can read an overview of IR scanning in this application here and a medical study report here.) Still, coarser accuracy is all that is needed in other cases.

The situation made me think about the many times we have accurate, precise, and repeatable data, but the measured point or parameter is only an indirect assessment of what we really wanted to measure. (Note: I am not talking about times when we interpret the collected data incorrectly though misapplication of statistical analysis; that's a whole different subject.) Often, for example, in physics experiments spanning sub-atomic to cosmic, all we can “see” is a second- or even third-order effect of the real parameter of interest.

Even if we think we can closely correlate and link what we want to measure with what we are capable of observing, there may fixed errors or dynamic changes between the two. The many digits our test instrument shows may lull us into thinking we know more and better than we really do.

It's the old story, but one that is easy to forget: What am I really trying to measure here? What are sources of uncertainty, besides the front-end sensor and instrument itself? What accuracy, precision, and repeatability do I really expect or need? Am I so wrapped up in what I am doing that I am not stepping back objectively and doing a reality check?

We've all had the bad experience of leading–or misleading–ourselves down a wrong path due to these kinds of acquired-data problems, especially in the heat of debug and product test. Is this temporal scanner a “good” instrument? As in so many cases, the answer is simple: “yes, but it also depends on what you are looking for, and what accuracy you expect.” ♦

0 comments on “When your data is accurate but misleading

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.