Monte Carlo Musings

Monte Carlo Analysis in SPICE simulation is a quite useful tool for many things relating to manufacturing yield, field issues, customer returns, and performance issues.

One of my first real exposures to how this might apply went something like this — I was out test flying a radio installation with a customer, and we flew over a patrol car — the radio transmission from the car (it was a law enforcement helicopter) was unintelligible. The communications officer onboard mentioned they needed to swap out that radio (the radio in the car) and get it repaired. I noticed as the day warmed up to 110 degrees on a later test flight that the companies own receiver design in this install did not sound too good on the secondary receiver.

Eventually, I got a decent SPICE simulator after a very large corporation bought the division, and I was able to model the circuit. At first in Monte Carlo it looked OK — it was a 10th order high-pass filter set to strip off the signaling tones that was implemented with inductors and capacitors; the parts were precision devices. The design was one that was re-used from several very old designs dating back to possibly the early 1970s.

However, once I studied the datasheets more closely, I determined that the devices being used did not show specs over temperature. The data sheets also did not have info for the effects of ageing, humidity, shock, vibration, etc. — it was a long list. I was able to get the information from the vendors eventually. Then my manager pointed out that these 2 percent parts that were used were likely from the same production line as the 1 percent parts. The 1 percent parts were “cherry picked” off of the line first, and the remaining parts, as long as they met the 2 percent spec, were sold as such. This is proper, but it results in a skewed distribution for the 2 percent devices — flatter distribution for the spread (rather than bell-curved). With all this information, I kicked off a simulation run that was set up just to step through the worst case limits on everything. This did not really show anything especially terrible. Still curious, I switched the simulator to a run of 200 on Monte-Carlo type simulation and let the workstation run overnight until it was done.

Low and behold, a few real stinkers popped out. It was time to look at the economics of buying the better parts. The simulator gave me the percent of units that would have issues, and I gave the manager the results to try and help run up the intangible cost of customer service calls, repairs, etc.

Have you seen similar issues with Monte Carlo analysis and datasheets these days?

12 comments on “Monte Carlo Musings

  1. eafpres
    March 19, 2013

    Hi William–Monte Carlo simulation is incredibly useful but often overlooked.  Nowadays you can do a lot in a spreadsheet as the calculations are so fast.  One concern is the goodness of the random number generation.  In your SPICE setup with Monte Carlo, did you ever look at the distribution of random numbers and confirm it was properly Gaussian?

  2. SunitaT
    March 19, 2013

    In a circuit analysis, Monte Carlo analysis gives a useful picture of actual circuit performance. However, as a circuit designer, I'm most often interested in worst-case performance.  That is, I want to know how a circuit performs at the extremes of component values, to ensure that  I've met design specification.  Monte Carlo analysis can tell me what is the performance at these extremes, but if the circuit contains many components it can take quite a lot of runs before its random selection of parameter values happens to simultaneously correspond to the worst-case limits of all of the components (and it's quite possible that I'll never see the true worst-case limits — after all, it's a matter of chance).

  3. SunitaT
    March 19, 2013

    Monte Carlo simulation furnishes the decision-maker with a range of possible outcomes and the probabilities they will occur for any choice of action. The technique was first used by scientists working on the atom bomb; it was named for Monte Carlo, the Monaco resort town renowned for its casinos. Since its introduction in World War II, Monte Carlo simulation has been used to model a variety of physical and conceptual systems.

  4. Brad Albing
    March 19, 2013

    That reminds me – I should go back and learn more about this. I used to almost understand it, but never used it enough to retain the knowledge. More's the pity.

    March 19, 2013

    I use simulations to check a few things, but take answers with a grain of salt. Simulation is used for a rough check until the bench testing takes place. But I have used the Monte Carlo feature to see what sort of ranges I would expect. If involving temperature, it is important to have the right models for every part. And, I agree, a large design can take hours to run. Just have the hard disk space, or limit what you can review later.

  6. amrutah
    March 21, 2013

    @eafpres:“In your SPICE setup with Monte Carlo, did you ever look at the distribution of random numbers and confirm it was properly Gaussian?”

    My understanding is that the monte-carlo always distributes all its input random variables in a guassian fashion and within the sigma ragne that we specify.  Once this is done, then we can then expect what the output distribution will be.. mostly it has to gaussain, and may be shifted.  observing the output pattern we can signoff the design results.

  7. eafpres
    March 21, 2013

    @amrutah:  My question is really about does the software generate good random distributions and does anybody ever check them?  If you use a function to generate random points in a Gaussian distribution, have you ever just taken the output of the function and created a histogram to see if it is normal?

    For instance, I was doing a simulation recently and used Excel; to generate random inputs I used the RAND() function.  This function is supposed to generate random numbers distributed uniformly from 0 to 1.  I ran 40,000 trials and looked at the distribution collected into 20 bins.  If perfectly uniform they should each be 5%; the lowest was 4.735% and the highest was 5.260%, or a total of 0.525% max to min.  This might be good enough, but most people just assume the functions are perfect.

  8. amrutah
    March 21, 2013

    @eafpres: The montecarlo algorithm inside the spice should spread the input parameters in gauddian or normal form(random spread).

      In case of excel RAND() function, I think it does not use montecarlo algorithm to generate the random numbers but some random number generation algorithm.

  9. Brad Albing
    March 27, 2013

    >But I have used the Monte Carlo feature to see what sort of ranges I would expect. If involving temperature, it is important to have the right models for every part.

    I would expand on that  a bit – the anaysis should be useful for doing a sensitivity study to examine what an analog ckt will do as different components' values are varied away from their “perfect” value.

  10. RedDerek
    March 27, 2013

    I forgot the component sensitivity analysis. That is the other option to the MonteCarlo.

  11. Skyfall
    April 24, 2013

    Pre-Excel2003 had acknowledged issues with it's RAND function not being random enough. Look up the “DieHard” tests.. The name stuck with me since I found out it had something to do with Ministry of Health statistics. There is no such thing as “perfect” in statistics.

    With MC Spice analysis you get a uniform distribution (square) vs a gaussion unless you take steps to change this. That is conservative design. Most people account for this as vendor margin when they see much better results for factory parameters. 

    I think it is urban legend that the passive vendors bin their output and the 2-5% are missing all the 1%'s. I have talked to senior FAE's at several passive companies and they say no one 100% tests at least at a SMT chip level and no one could manage supply if their process control was dependant on binning. 

  12. eafpres
    April 24, 2013

    @Skyfall re: Excel–I am using Excel 2010 and the 40000 data points for the RAND() function were generated in that version, still having some variation from uniform distribution.  However, as it is sufficient for my purposes I have not run tests of the distribution error vs. sample size to see if it is better in the limit of very large number of samples.

    re: component values and simulation–in truth, you would need to understand the manufacturing process for a component to try to estimate the distribution shape, even without sorting.  I agree that the uniform distribution over a tolerance range is very conservative.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.