Neuronics: Distributed-Memory Addressing, Part 4

The tradeoff for less memory is that a given memory location with CMAC addressing might be used for multiple input states, resulting in collisions for some memory locations. Because a larger input space has been mapped into the smaller address space, interference is inevitable. By adding outputs from a unique combination of memory locations for each input state, interference adds a small amount of noise to the output value if the input space is sparse — that is, if few collisions occur. Then the interference is of an acceptable amount. A plot of the tradeoff is shown below.

The optimum K might be taken as the value where the noise effect on the output function value is reduced to be numerically equal to the probability of interference between input states, or…

In practice, among robot arm researchers, good results have been obtained in the dynamic control of arms for K  = 80 and N  = 12, with RN  = 4096. Then Kopt  = 60 which is not too different from 80. If memory contains 8-bit byte values, then the total range on the output function, in bits, is k  + 8. The maximum error that a shared location can contribute to the output value is thus 8 bits out of k  + 8. Typically it will be a small fraction of 8.

By choosing an overlapping address coding scheme like Gray coding, where only one bit changes in the code among adjacent states, then states that are close in N -space to learned states will code to most of the same memory locations and put out a function value that is close to that of the learned state. In other words, for similar states, similar values will be output, even though behavior using the new states has never been used in training the memory. This means that the CMAC memory can generalize from what has been learned for continuous functions. In any localized region of N -space, training a state in the region will also provide some training for all the points in the region.

In summary, CMAC distributed-memory address decoding relative to ordinary linear decoding reduces the address space by decomposing it into K address spaces with fewer address bits (qN ) per space. A 16-bit memory space (a  = 16), for instance, could be decomposed by using the four least-significant bits for k . For N  = 3, then…

…and the least-significant byte of the 16-bit address decomposes into 4 LSBs for K and 4 MSBs for Q . The channel encoding (Q ) no longer has the higher resolution added by the lower 4 bits and can address only 16 bytes per input vector component. For 3 dimensions, the address-spaces of Q resolution consists of qN  = 12 bits and there are K  = 16 of them.

In applications of neural networks, such as robot arm control or machine vision, N > 1 and the advantages of CMAC memory increase. In electronics historically, electronic circuits have commonly entailed single-variable input-output systems. Most analog measurement instruments and waveform sources have N  = 1 or 2 (such as stereo or video color encoding). Distributed memory illustrates an opportunity to expand to multiple-I/O analog systems. As N becomes large, the number of circuit connections dominates over the processing in complexity.

Computing becomes largely an addressing process instead of an ALU process and is connectionist in emphasis. IC implementation of connection-dominant circuits relies heavily on monolithic multilayer interconnect capability. Emerging 3D IC technology is bound to greatly aid the connection problem and allow conceptually interesting neuronic analog circuits to be implemented. Meanwhile, existing “2+ -D” IC technology will allow for lower-N circuits to be built. Still needed are analog circuits that do CMAC coding, and that is the topic of an upcoming neuronics article.

14 comments on “Neuronics: Distributed-Memory Addressing, Part 4

  1. Davidled
    May 3, 2014

    ->In practice, among robot arm research, good results have been obtained in dynamic control arm.

    It sounds like CMAC coding algorithm is a good fit to Neural Network. I am wondering if there is any reference article for robot arm researchers whose paper is using CMAC Distributed-Memory Address. It is possible that this could be a good application for Genetic Algorithm (GA).

  2. etnapowers
    May 6, 2014

    “In practice, among robot arm researchers, good results have been obtained in the dynamic control of arms for K = 80 and N = 12, with RN = 4096. Then Kopt = 60 which is not too different from 80.”


    I guess that Kopt = 64 right?

    I wonder what is the difference between the two cases K = 80 and K = 64 in terms of efficiency of the robot arm.

  3. D Feucht
    May 6, 2014

    Yes, it appears that you are right. The sqrt(4096) = sqrt(2^12) = 2^6 = 64. Thanks for spotting that. In this case, the value is even closer to the empirical optimum among roboticists of 80.

    As  for arm control “efficiency”, the graph in the article predicts the effects. With a somewhat higher number of memory locations per input state, there will be somewhat more interference but it will be attentuated in effect by the additional number of locations adding to the combined output.

  4. etnapowers
    May 7, 2014

    “By choosing an overlapping address coding scheme like Gray coding, where only one bit changes in the code among adjacent states, then states that are close in N -space to learned states will code to most of the same memory locations and put out a function value that is close to that of the learned state”

    When utilizing a Grey code , a XOR decoder is requested , this adds circuitry and increases the costs. Are these additional costs compensated by the increased efficiency of the system?

  5. Sachin
    May 7, 2014

    It sounds very interesting how you put it that when it comes to applications of neural networks then CMAC coding is a good way forward. I am not that sure or let me just say I don't know of any robot arm pioneers who are using CMAC distributed memory address. Therefore please, I am looking to getting information about any reference article or paper dealing with this, and anybody with an idea to let me know.

  6. D Feucht
    May 7, 2014

    The Gray-code encoder was illustrative of the basic concept, not the implementation. At present, CMAC implementation is on digital computers in software. In my next (and final?) neuronics article (series), I hope to present some ideas on how to implement CMAC in analog circuitry.

  7. D Feucht
    May 7, 2014

    James S. Albus, inventor of the CMAC scheme, was a major robot pioneer. Do a websearch on his name and on “CMAC” and you should find leads to other articles on the topic. For instance, W. Thomas Miller III, et. al. , at the U. of NH wrote one of the papers about CMAC.They have a paper in the Int'l Journal of Robotics Research , vol. 6, no. 2, 1987 from which some of my information was taken. Pentii Kanerva's book, Sparse Distributed Memory , MIT Pfress, 1988 is another. A third instance is the compendium, Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches , edited by David White and Donald Sofge, Van Nostrand Reinhold, 1992, which has  chapters on CMAC control.

  8. Davidled
    May 7, 2014

    For instance, when typing James S. Albus in Website, there are some lists. I am wondering if there are some articles that could be provided in the website link, to let us review the article or paper on this site.

  9. etnapowers
    May 8, 2014

    Thank you for This Dennis, I look forward to read about the implementation of Cipher-based MAC alghorithm in analog circuitry, there could be some interesting solutions, I wonder if the efficiency may vary depending on the particular implementation chosen.

  10. D Feucht
    May 8, 2014

    Albus wrote extensively and published many papers in various journals. So did members of his team at NIST. You might look at the website for papers under robotics automation by Albus.

    My single best recommendation is his book, put out by BYTE Books in 1981 titled Brains, Behavior, and Robotics.

  11. Davidled
    May 9, 2014

    NIST Website provides a lot of technical paper and Journal.  Albus involved with Robot Crane Project with mechanical Dynamic using Math Simulation and Modeling which is out of topics as below:

    I could not article Distributed Memory Address related to Robot Arm. As back to original question, if you could link paper related website simply, that will be great.

  12. yalanand
    May 10, 2014

    D Feucht, I am not saying that you give me a number of lists of books, but I only wanted you to give me a specific source that will give my something on robot arms with respect to CMAC distributed memory address. As Daej is saying, at least make it easier for us, give as a direct link. I think is like you want us to go and do our own research. We want to review that paper on this site. Thank you. 

  13. D Feucht
    May 10, 2014

    Not much research is involved in finding Albus's papers on the Web. I did a websearch on “James S. Albus CMAC” and found these weblinks:

    For an overview, also with more links:

    “New Approach to Manipulator Control: The Cerebellar Model Articulation Controller (CMAC)” from NIST at

    At the bottom of the webpage is a link to a downloadable PDF copy.

    A quick CMAC explanation is at

    References to Albus's most important publications are given here (near bottom):

    This should get you going.

  14. Davidled
    May 11, 2014

    This article title “New Approach to Manipulator Control: The Cerebellar Model Articulation Controller,” provides a little bit clarification, even though there is a little bit deviation compared with the hardware implementation related paper or article regarding on addressing mapping. Potentially, this CMAC could be used to other application.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.