A few weeks ago, my smartphone refused to pick up emails, although access was fine on my desktop and laptop PCs (No need to bore you with the details, as I am sure you have your own similar stories). It’s especially frustrating and scary to fix this class of problems since access and visibility in the system (the phone and email provider) are extremely limited with only a tiny “keyhole” through which you can see and debug. It’s like building a ship in a bottle, except the bottle is painted black and the only point of view or access is via the bottle opening (Figure 1).
Figure 1 Troubleshooting and debugging is often complicated by physical-access issues; you feel like you are working on a ship in a bottle, sometimes made worse as it seems that the bottle is painted, so the insides are not even visible. Source: www.shipinbottles.com
Even more scary, unlike most analog-circuit debugging, I knew it was all-too-easy to do something which not only makes things worse but is possibly irreversible, resulting in loss of all email access (don’t ask how I know this risk).
Fortunately, after several hours of trying “this and that” I did get the smartphone email to work again. Then I realized that due to the pressure and frustration of my efforts, I had totally neglected to document what didn’t work and what eventually did.
The entire situation reminded me of all the engineering skills that we develop and master, where debugging and troubleshooting are often the most difficult and challenging. It takes a combination of practice, experience, mistakes, self-discipline, persistence, calmness, clarity, out-of-the box-thinking, attention to details, a mentor, and lucky breaks. Even if it is taught in school (usually not) or learned on the first real job, debugging is largely a formal way of approaching and working through a challenge. Yet that is hard to do when you are dealing with management and schedule pressure.
When I talk about troubleshooting and debugging, I mean hands-on circuit-related investigation, not just sitting at a keyboard and changing lines of code. Real troubleshooting and debugging require connecting test and measurement units as well as determining if the test arrangement is valid and not adversely affecting performance.
Of course, there are different debug scenarios, including bringing up a prototype, which is the most difficult because the design may have some fundamental flaws or misconceptions. There’s the other extreme where something that has worked well for a long time has developed problems. Then there’s a middle ground where a design with a modest track record begins to operate erratically. The causes can range from something as mundane as an intermittent or broken connection to something as aggravating and subtle as a part which doesn’t meet its datasheet specifications; those are just two of the literally countless possibilities.
It’s a messy business, figuratively, and often literally with test probes, leads, fixtures, and more. In many cases, you soon have an accumulation of changes which haven’t solved the problem, while at the same time, you are not sure where you are or how you got there. At least the changes are usually reversible, unlike medical procedures and surgery, which generally are not.
Although it may seem that debugging is a skill that comes only with experience rather than online guides or books, that’s not the case, as clearly demonstrated by the book, “DeBugging: The 9 Indispensable Rules for Finding Even the Most Elusive Software and Hardware Problems” by David J. Agans, published by the American Management Association (Figure 2).
Figure 2 This book is hands-on, fully practical, full-length work on debugging that I have ever encountered. Source: Goodreads
Despite what publisher’s name may suggest, this is not an academic, theoretical, or ivory-tower tutorial. Instead, it’s a very hands-on, down-in-the-trenches guide with firm suggestions to many techniques and approaches you should use as well as some to avoid. It’s also definitely not a simplistic “cookbook” guide.
I am sure you all have had your challenging debugging situations. But put the problem and resolution aside for a moment, and try to remember: what was the most memorable mistake you made, and/or smart tactic you employed when debugging a difficult, mostly hardware situation?
- What that first engineering job teaches
- How the Analog Challenge Has Changed
- Just give me a decent data sheet, please
- When Your Project Ends Up on the Scrap Heap
- Contrary to Rumor, Analog Circuit Design Is Alive & Exciting