In my last few blogs we have been looking at the screening steps employed for space products offered by Analog Devices. I had planned to continue with this topic this month, but a recent trip to the cyclotron at Texas A&M, along with the current time of year, altered that course just a bit. Instead, this month I thought I would talk a little about giving thanks. It is something we often neglect to do but could benefit greatly from. I did an internet search to find an appropriate quote to reflect on for this blog and found this quote from Ralph Waldo Emerson to best summarize what I would like to convey:
“Cultivate the habit of being grateful for every good thing that comes to you, and to give thanks continuously. And because all things have contributed to your advancement, you should include all things in your gratitude.” – Ralph Waldo Emerson
If you would like to explore this topic of gratitude and are interested in a few more good quotes I’d encourage you to check out this article: 21 quotes of thanksgiving and gratitude. It has the quote I mentioned above along with several other great quotes and thoughts on the topic of gratitude. It is an appropriate thought to ponder as we sit in the middle of the Thanksgiving season.
To set the stage a bit for this discussion it would be good to offer some perspective on radiation testing for single event effects. I have talked about the importance of radiation testing when qualifying a product for space applications in a few of my previous blog posts. It is imperative to understand how a device or system will respond when subjected to radiation so that its operation in the harsh, radiation-rich environment of space can be predicted. While we cannot generate the energy levels of radiation experienced in space what we can do is generate enough radiation energy to help predict a response. One of the places in the US where this can be done is at the Cyclotron facility at Texas A&M University. I had written about this facility back in December of 2016. It has been a while, so I will provide a few thoughts here as a reminder.
The facility is quite impressive. From the control room to the cave it still has me in awe when I visit the facility and see the complexity of the system. This first photo is of the control table. I still cannot help but think back to one of the movies from my childhood when I see this. It makes me think of War Games every time. The control room scenes there look similar so that is where my mind goes.
Cyclotron Control Room – Control Table
Here is another shot from the control room of the various monitors of system functions. I assume these are for various process control monitors to report back the status and measurements of various system parameters for the Cyclotron. It is impressive to me that this is the technology behind such a complex system.
Cyclotron Control Room – System Monitors
Devices under test go into what is referred to as the cave which houses the end stage of the cyclotron. Just to the left in the image below is where devices are placed and the ion beam impacts the device. A staging area with X, Y, and Z positioning is there that allows for precise placement of the device to make sure the ion beam impacts the device in the correct area and at the correct distance for proper ion penetration.
End Stage of the Cyclotron in the Cave
Obviously when testing with radiation it is not desirable to be down in the cave. Thus, there is a data room that sits above the cave with a protection layer in between. To communicate with the device under test in the cave 30 feet of USB cable is ran through piping from the data room into the cave. This allows for software automation of test equipment and system boards down in the cave to perform the testing. One can also run extended length VGA cables from oscilloscopes and/or spectrum analyzers from the cave up to the data room as well. The data room has extra monitors that can be used to display measurements from such equipment. Once the control room has the ion beam set up properly it can be turned on and off from the data room.
Well, I think that sets the stage to give an idea of what the Cyclotron facility looks like. It should start to give a picture as well as to how complex the setup is for testing devices. The cave is locked up when the ion beam is turned on to protect users from radiation. This means that all controls for the test must be in the data room, thus all testing is remote access. What I have not shown here is that all equipment is placed onto a cart and sent down into the cave via an elevator. These facts alone are enough to make one appreciate the complexity of the test setup in and of itself. Now, let’s look at some things that can make the testing interesting.
First and foremost, there is simply not a fool proof method of predicting the response of a device when it is irradiated by the ion beam. This does not mean that when planning one does not to try to anticipate what a device will do, but it just means that every possible behavior cannot be known beforehand. This makes things interesting during the test! There are other things that can happen such as damaged or faulty boards, cables, USB hubs, USB-GPIB adapters, etc. In our recent trip I think we encountered nearly everything that could go wrong.
We were fortunate to have several time-slots this month with one slot occurring two weeks prior to the second and third time slots. Each time slot was 16 hours with the second and third time slots separated by one day. There were some unexpected behaviors seen during the first time slot on one of the devices we tested so we had to move on to other devices and devise a work around between the first time slots and the next time slots. When we moved on to another device we also ran into unexpected behaviors. Unlike the first device though, the behaviors required active debug while exposing the device to the ion beam. This put added pressure to work quickly and efficiently because beam time is limited. We were able to work through the first set of unexpected behavior on the second device, but we then encountered additional obstacles with the response time of the Python scripts controlling the second device. We found a workaround to this by lowering the flux (ions/second) to a level that allowed the software enough time to respond.
We took the lessons from the first time-slot and went in for the second set of time slots two weeks later. Once again, unexpected behaviors cropped up. We had a device failure with one of the devices which was not expected. We then ran into problems communicating with the equipment over USB. We had issues with one of our controller boards down in the cave. Luckily, we had planned with extra devices and equipment. We were able to troubleshoot the equipment communication problems to a bad USB cable and a piece of test equipment with an intermittent USB port. We found that one of the controller boards we had been using was also not working properly and replaced it.
By this point you are probably asking what there is to be thankful for. Well, the answer to that is simple. Despite all the issues we ran into our planning paid off since we had brought additional backup devices, boards, and equipment and were able to carry on despite the issues. The lesson here is that sometimes we run into issues, but if we plan properly and keep a positive attitude, we can be able to give thanks even in times of distress. We were thankful we planned ahead and had the extra items so that we could replace faulty items. We were thankful that we had multiple time slots to work with the devices to better understand how they would perform when irradiated and work around that with our software. In all these events we learned from the things that went right as well as the things that did not go as expected. Just as the quote above says it is through all the experiences not just the positive ones that we learn and grow. We should be thankful even when things are not going exactly as planned because it is a learning opportunity.
As we are in the midst of the Thanksgiving season think of a time (or times) when things did not go as planned and, even though perhaps a bit painful, were good learning opportunities. As an engineer it is imperative to learn from all our experiences, even the bad ones. We should be thankful that we have a mindset as an engineer that keeps us plowing forward and digging continually to find the solution.
If you would like to see more on radiation performance of various Analog Devices space level components I encourage you to check on Analog Devices: Space and scroll down to the All Resources section to look for either of these three subgroups on the page: 1) Single Event Effects Radiation Reports 2) High Dose Rate Radiation Reports or 3) Low Dose Rate Radiation Reports. Now that you have some insight into how single event testing can pose challenges, hopefully you will have a better appreciation for the single event effects radiation reports there. Of the three tests, single event effects are probably the most difficult to test. That is part of why we chose to be engineers though, right? We love to face down a challenge and come up with a solution. The world needs engineers, let us give them engineers that are thankful for the challenges we face and let’s meet those challenges head on ready to solve them and learn from the experience. I hope you have enjoyed the topic in this installment and I encourage you to leave a note in the comments and tell folks about a challenge you have overcome as an engineer.