Advertisement

Blog

How Do You Debug a Custom Analog IC?

We've talked about lots of different integrated solutions to various design needs. We've had blogs about analog front ends, power management devices, hearing aids, factory automation, and health-related devices.

Assuming you've followed the intent of these blogs and had custom integrated analog ICs made to your exacting specifications, you now have an IC on your circuit board that performs amazing technological tasks in a tiny amount of board space. Unless it doesn't.

Then what? Warm up the scope and the coffee pot and start troubleshooting the board. Sadly, because you put all that functionality inside, you can't tell where the signal gets mucked up or where the power supply voltages get all weird. Here's an example.

(Source: Microchip)

(Source: Microchip)

If you used a device like this and the MCU wasn't working right, maybe the internal bias supply (or similar LDO) was not supplying the right voltage. Can you tell? Perhaps adding one pin to bring that voltage out for test purposes would have been a good idea. Here's another example.

(Source: QEX Magazine, July/August 2002)

(Source: QEX Magazine, July/August 2002)

This is a discrete IC version of a portion of a software-defined radio. Assuming you integrated the Johnson counter, the Tayloe detector, and the op-amp circuitry into one piece of silicon, consider the utility of adding test points (brought out on pins) for the counter and detector outputs. That would greatly simplify troubleshooting.

Here's a block diagram of sorts from one of Maxim Integrated's devices. (Maxim is the sponsor of this site.) This one could be part of a data acquisition system used in some factory automation equipment. As before, if you plan ahead and bring out test points from the interconnects between each stage, life will be so much easier once production starts.

(Source: Maxim Integrated)

(Source: Maxim Integrated)

Here's one more simple example of the right way to do this. In this case, the device is indeed pretty simple. As integrated analog goes, this barely qualifies, since it's simply an instrumentation amplifier.

(Source: Intersil)

(Source: Intersil)

At first glance, it looks quite conventional, but there are two additional signals brought out — the connections to the input stage op-amps. This allows you to see what is going on at these points, and it can supply very useful information.

What complex integrated devices have you used? Did you wish you had access to additional signals or test points?

Related posts:

— Brad Albing, Editor-in-Chief, Planet Analog and Integration Nation Circle me on Google+

40 comments on “How Do You Debug a Custom Analog IC?

  1. CarlWH
    November 21, 2013

    Hi Brad,

    You debug a custom IC just like you would any IC.

    The evaluation should be planned, usually with engineering samples, which will be evaluated to the ASIC spec, (by the provider), evaluated in the system, (by the customer), and qualified to standards, (this can be provider, customer, test house/body or any suitable party).

    If a problem is identified, you can wafer probe or even do fib mods to get access to relevant nodes.

    Simulations and models are very good nowadays, however you can still get unforeseen system errors and behaviour.

    Of course in the digital domain there are well trodden paths of scan and code coverage to try and guarantee functionality.

     

    Carl

     

  2. Scott Elder
    November 21, 2013

    Hi Brad,

    Many (most?) system level analog ICs have “back doors” for testing.  It becomes very difficult, as you point out, to validate (final test) an IC system solely at the pins with fixed assigned signals.  

    Many years ago, it was practical (if not necessary) to probe internal nodes for troubleshooting.  But with today's semiconductor feature sizes so small, it is very easy to add multiple signal multiplexers throughout the analog sections to a common analog wire that is then multiplexed out to a pin.

    If during the design, several key points are brought out to the external pins through analog multiplexers, things like dropping probes or bringing up buried nodes with focused ion beam metalization (FIB) becomes less of a requirement.  This is where great troubleshooting skills come into play.  Just like a medical doctor.  Run a few tests, study the results, refine the tests, come up with a conclusion, and then validate the conclusion.

    Certainly some times it takes dropping a probe or “FIB” a line (validate a conclusion), but anytime that is required the troubleshooting process slows way down.  Much much easier to simply look at the signals on the application board while wiggling the internal nodes.

    For the curious among us, it might be an interesting exercise to take some parts and try to find those back doors!!  Don't expect that to be an easy process though.  No manufacturer wants their parts to accidentally trip into the test mode during normal usage.  So protection measures are in place to mitigate this possiblity.

    I also wonder if disclosing these features in the data sheet would help the end users also.  It certainly can be dangerous (i.e. stopping the switching regulator from switching with one switch permanently on!).

     

  3. D Feucht
    November 23, 2013

    As Brad points out, the observability of semi-discrete design is one of its advantages over higher levels of integration, and for IC users, can make product test and diagnosis easier too.

    By now, there must be some discussion in the industry that I am not privy to about coming up with an analog JTAG. If it is disclosed publicly, it takes away some of the intrigue of Scott's challenge, the IC equivalent of looking for unadvertised button sequences in early HP calculators (HP-35 and HP-45 in particular) that displayed the internal timer rate or other semi-useful functions.

    If the hidden features of analog back-doors could be quite useful to users, then suppliers are squandering added value that could be presented (and sold) to the user. But if the user is a blockhead, the supplier wastes valuable resources on too much customer service, and it is a replay of the sorcerer's apprentice, with silicon instead of water.

  4. Brad_Albing
    November 24, 2013

    @CarlWH — all valid approaches and methods that one should pursue. Initially, I was thinking more about trouble-shooting with an integrated analog IC where the IC might not be defective. Instead, the overall [board] design might have problems — so the addition of these test pins could let you see further into the design so that you could figure out why it wasn't working.

  5. Brad_Albing
    November 24, 2013

    @Scott – good point about stopping the switching regulator from switching. if it's a buck regulator and you leave the upper FET on at 100% duty cylce, there will be serious smoke and flames. Pro'ly undesireable….

  6. goafrit2
    November 24, 2013

    The best design is to avoid debugging custom analog IC because whenever debug happens, it means a new run has to take place. In other words, another  6 months is needed to redesign, layout and then go to the foundry. I hate debugging analog IC and spend really good time before tapeout to minimize that experience

  7. goafrit2
    November 24, 2013

    >> evaluated in the system, (by the customer)

    Never send an IC to a customer if you have not completed the debugging process. There is no relationship in the mix for the customer to help debug a product. That must be done by the OEM before any sample is sent out for a customer to test. So, a debugging process must never involve a customer.

  8. goafrit2
    November 24, 2013

    >> But with today's semiconductor feature sizes so small, it is very easy to add multiple signal multiplexers throughout the analog sections to a common analog wire that is then multiplexed out to a pin.

    In the industry, they call it DFT (design for test). You build those redunancies to simply the process of making sense of signals when there are problems. Nearly all major analog projects use that. It makes sense as it does no good fixing problems in bits when you can pipe in stimuli to evaluate the performance of different blocks or circuit units. The ease of making digital blocks also makes this process quicker as anything is customizable.

  9. goafrit2
    November 24, 2013

    >>observability of semi-discrete design is one of its advantages over higher levels of integration, and for IC users, can make product test and diagnosis easier too.

    There is also the advantage that if one block is broken, you can replace it with another one when everything is not integrated. But the practical challenge is that the cost of one of those blocks is largely the same for the whole blocks. So, in the money matters, integrating more is a better business.

  10. goafrit2
    November 24, 2013

    >>  the hidden features of analog back-doors could be quite useful to users, then suppliers are squandering added value that could be presented (and sold) to the user

    I do not see the value in turning your products into users' testbenches. If the product does not work as advertized or sold, there is no need to buy it. I do not see the gain from the user's point of view since they cannot even fix the integrated product. The focus from the design angle is to have the product operate in the normal designed state and spare customers the pain of wasting time looking for insights that will not add any value.

  11. Brad_Albing
    November 24, 2013

    @goafrit – I agree. My example here hypothesizes that you (as a product design engineer) had designed an analog IC and had it fabricated. We will assume that the IC company made just what you asked for and it met the test specs you sent to them. But when you put it on your circuit board, things didn't quite work out the way you had planned.

    That's when you will wish you had added those test pins to which I referred.

  12. jkvasan
    November 25, 2013

    Brad,

    Debugging an analog IC at design team is the most daunting task. However, once implemented, to monitor its functionality, one could use some auxiliary circuit to provide test points. Extra high input impedance opamps could monitor some of the feedback voltages. It involves cost, any way.

  13. fasmicro
    November 25, 2013

    That is true – the addition of multiplexed pins could help in looking at some signals so that troubleshooting can be faster and done effectively. Yet, the best strategy will be to spend more time to develop better real-life models at the design phase over the constructs of hoping that proping at bench can help. When you have a chip from fab, and there is problem, you are about 3-9 months behind market introduction. It is unlike software where you can make mistakes and fix over the night. This is one of the reasons why our business is challenging.

  14. fasmicro
    November 25, 2013

    I think the point is that by giving users more access to look at your product, they could potentially find uses which you might have thought about. Though it s a risky thing to give users access to “re-test” your product, it could give some peace of mind as they deploy the products in some applications. Some customers require you present some block designs as they want to understand the fundamental architecture upon which design is made. Providing those access points could go a long in the process.

  15. fasmicro
    November 25, 2013

    >> Extra high input impedance opamps could monitor some of the feedback voltages. It involves cost, any way.

    More than 60% of analog IC cost goes into testing. When a designer simplifies the process of testing, you help the company. The problem is that testers are not valued in the chain despite them burning the highest combined cost.

  16. fasmicro
    November 25, 2013

    In my office, we have a business process in our design and development. When a deigner finishes, he must have a TESTING meeting with the test guys. He explains what he may need to see should troubleshooting be required. He will help the testers plan the whole testing strategy before layout. That way, everyone is on the same page and cost of testing is reduced. 

    The problem most times is that testers are never involved in the development phase until the product is back from the fab . By then some of the contributions they might have offered will be lost.

  17. fasmicro
    November 25, 2013

    //You debug a custom IC just like you would any IC.

    I am not sure if that statement is correct. In digital, you can automate and scan. But in analog, that is a different ballgame. I think the strategy for analog, mixed signal and digital IC debuggings are different.

  18. CarlWH
    November 25, 2013

    Some interesting points are being raised here.

    When undertaking a custom IC, (note not defining the flavour), the test strategy will be defined at the outset. This will of course be dependent on the device, but in essence it's a custom device and you do have plenty of options, although not all will be appropriate for every situation.

    Let's say you have defined your test strategy and committed to silicon. You will first get back engineering samples. The customer will want to test these in their system and this will be part of the evaluation of the device. Of course, the vendor will have completed some preliminary debug but in essence the customer is paying for this device and they will be required to sign it off and as such it has to work in their system.

    If you have got everything correct, you can then move into production silicon, if not there will be some modification. Most drastic, would be a complete re-spin, but it could be a metal only change. Normally wafers are held at various stages to allow tweaks.

    So now we are in production.  I think this is the situation Brad was alluding to. The device is on your board and how do you debug it. You cannot bring out internal nodes as the device is made.

    Carl

  19. Brad_Albing
    November 25, 2013

    @CarlWH – So now we are in production.  I think this is the situation Brad was alluding to . Exactly right.

  20. goafrit2
    November 27, 2013

    >>  I think this is the situation Brad was alluding to. The device is on your board and how do you debug it.

    Good explanation. Of course the assumption here is that this is a product being developed by a specific customer. If this is going to the mass market with no specific customer in mind, it is incumbent on the OEM to make sure everything works fine.

  21. goafrit2
    November 27, 2013

    >> Exactly right.

    Thanks for the update and explanation. In chip development, there are many phases. The last being the production run when all the tests and fixes are done. Being in that phase means the product is ready for primetime upon production.

  22. SunitaT
    November 28, 2013

    Debugging an analog IC at design team is the most daunting task.

    @Jayaraman, I totally agree with you on this. Debugging an analog IC is definitely dauting task. I think adding more and more test points definitely helps us to resolve this complexity.

  23. SunitaT
    November 28, 2013

    In digital, you can automate and scan. But in analog, that is a different ballgame.

    @fasmicro, I totally agree with your opinion. Debugging Analog IC is not as easy as debugging Digital IC. I think debugging analog IC requires lot more experience and intution to debug the circuit.

  24. SunitaT
    November 28, 2013

    When a deigner finishes, he must have a TESTING meeting with the test guys.

    @fasmicro, I think this is the strategy being followed by many companies. Interaction between designer and testing team definitely helps to build proper testing methodology and helps the team to find more bugs in the testing phase.

     

  25. jkvasan
    November 29, 2013

    To debug analog circuits, we have always been used to monitor the functional performance and interpret the behaviour. Traditionally, we use test points and not any special interface. Implementing a analog JTAG could be a great step.

  26. Brad_Albing
    November 30, 2013

    @goafrit2 – Agreed – with a mass-market device, it must be thoroughly tested.

  27. Brad_Albing
    November 30, 2013

    @JK – That's the sort of thing I was thinking of. You have to be careful when monitoring feedback nodes (summing junctions), but it can be done. Regarding the cost of adding an extra op-amp here or there, cost should be negligible considering the cost of the whole IC.

  28. Brad_Albing
    November 30, 2013

    @fasmicro – that touches to some extent on the issues that Jason Bowden raised in his blog “A Bad Case of Cost 'Frusfusion'.”

  29. Brad_Albing
    November 30, 2013

    @JK – that idea of an anlog JTAG port has come up in conversation here before. Not a bad idea; we just need to figure out the rules and implement a standard.

  30. yalanand
    November 30, 2013

    Debugging takes lot of time in total product cycle. Now a days , because of these CAD tools the debugging  is so easy , but in earlier days it was very complex to debug  from start to end. In analog debug we have to consider so many parameters which can go wrong.

  31. yalanand
    November 30, 2013

    Implementing a analog JTAG could be a great step.

    @Jayaraman, true.  JTAG technology can be applied to the whole product life cycle including product design, prototype debugging, production, and field service.

  32. goafrit2
    December 6, 2013

    >> To debug analog circuits, we have always been used to monitor the functional performance and interpret the behaviour. 

    Our industry needs to invest in modelling and sims. It is still unacceptable that we still live in a design world where measurements deviate heavily from simulations. The sophistication in analog testing can be reduced if we improve the correlation between prototypes and sims.

  33. fasmicro
    December 6, 2013

    I think adding more and more test points definitely helps us to resolve this complexity.

    You can also use the method of Design for Testability. This makes all the difference when the time comes for bench. The problem is that we design and rarely think through the testing until the chip returns from the fab

  34. fasmicro
    December 6, 2013

    @SunitaT, do not take it for granted. Many firms and specifically many designers do not consider testers such a key part of the product development. It turns out that testing guys are brought in after the product is already in the fab. I have been in meetings where testers had wished they were engaged earlier in the process.

  35. fasmicro
    December 6, 2013

    Regarding the cost of adding an extra op-amp here or there,

    @Brad, but it could turn out that the Opamps, OTAs could assume more than 60% of the whole product. So, if you eliminate them, you have truly engineered a nimble product. Do not take it for granted when you can save OTA or opamps. They are huge cost factors in any analog design. The stablity caps are not cheap because they are huge for compensation. Take out opamps in most ADCs, you can save 60% of the cost!

  36. fasmicro
    December 6, 2013

    >> Debugging takes lot of time in total product cycle. Now a days , because of these CAD tools the debugging  is so easy

    Debugging at the bench is different from the one you are referring here. With CAD, you are still at simulation phase. That is a child's cry as you can fix things easily. The most challenging is bench where the cost is another 3-6 months of new wafers and runs.

  37. jkvasan
    December 9, 2013

    @Brad,

    Again, the problem with a JTAG port is it would understand and communicate digital data. It is not the implementation the very JTAG port is a problem but converting signals measured at various points into digital data and then feeding to the port would be the actual problem, in my opinion.

  38. jkvasan
    December 9, 2013

    @yalanand,

    Analog JTAG, in my opinion, is easier said than done. It needs several conversion layers before each of these signals are level converted and transformed to digital signals within the measurement range.

  39. jkvasan
    December 9, 2013

    fasmicro,

    You have a point there. DFT can exponentially reduce the number of problems we face at a very later stage as it gives the power of finding out what went wrong.

     

  40. fasmicro
    December 16, 2013

    >> Analog JTAG, in my opinion, is easier said than done. 

    A mux may be better in a more complex system. I am not used to analog JTAG.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.