Neuronics: Long-Term Memory, Part 1

A central problem — perhaps the central problem — in the advancement of neuronics is the lack of a technology that stores analog quantities indefinitely. Sample and hold circuits will not quite do it, not indefinitely. While it is possible to make one-bit hysteretic switches out of comparators or store a couple of bits of charge in a MOSFET gate, large-scale analog storage has yet to be devised. The following idea might not achieve this goal either, but maybe it can be a stepping-stone for someone’s bright idea that does.

In the 1980s, Barrie Gilbert was exploring multiple differential pairs of BJTs by combining their outputs while offsetting their input voltages from each other. This resulted in what he called multitanh technology: circuits with multiple hyperbolic tangent functions. The tanh function describes the familiar large-scale transmittance of diff-pairs, and (amazingly) nobody had yet explored it, though the BJT had been around for decades by the time Barrie did his research. An example of a multitanh circuit is shown below, extracted from one of his patents (US4476538). The constant current sources, I , and resistors, R , create the offset voltages at the input bases of the diff-pairs.

Multitanh circuits extend the linear input voltage range of the single diff-pair. Beyond this, the basis for the patent was the discovery that at the output, there is a slight ripple in the resulting function that, upon closer examination, is a sinusoid, shown below as the next figure in the same patent.

The sine function repeats through more than one cycle, and with some modification can be extended for multiple cycles. So what we have at this point is a function that is cyclically nonmonotonic in that for a given y value there are multiple x values.

Next, this function is inverted by swapping its axes (by flipping it over and rotating it 90°). This can be performed electronically by placing the function in the feedback path of an op-amp. A simpler example of op-amp function inversion is to connect the inputs of a multiplier together so that it becomes a squaring function, then place it in the feedback path of an op-amp, as shown below.

The multiplier has the transmittance (not transfer function, because this is not a linear circuit),

and the feedback path transmittance is

The input voltage, vI , is not attenuated because of the multiplier current-source output, and the error voltage at the inverting input of the op-amp is

The forward path through the op-amp has a gain of

Then the closed-loop transmittance is

Solving for vo /vi (by removing vo from the right side),

The squaring function, y = x2 in H results in a closed-loop function that is the inverse, x = y½ , y ≥ 0 . The op-amp inversion circuit works because the square-root function is monotonic, as is its inverse. The multitanh function is not.

In the next part of this blog, we will look at a different inverse-function technique with regard to the multitanh circuit.

Related posts:

9 comments on “Neuronics: Long-Term Memory, Part 1

  1. amrutah
    November 15, 2013

    Dennis, Thanks for the blog post.

        In the blog we are discussing about the squaring function, how is that  related to the memory, or am I missing something (“probably many things”)?

  2. D Feucht
    November 17, 2013

    The op-amp square-root circuit using a squarer in the feedback path was intended to illustrate the idea of function inversion in circuitry. The memory idea cannot work quite the same way because the function is nonmonotonic and must be in order to have multiple stable voltage values at which it can represent a digitized quantity.

    To implement this circuit (which was not worked out in any detail in the article, thereby leaving the reader something to ponder further) something like a multipath feedback hysteresis circuit is required. In this article, I wanted mainly to convey the idea for further thought and discussion rather than try to present a tutorial on what is really an exploratory subject.

  3. etnapowers
    November 19, 2013

    @Dennis: To invert a function, the monotonic property has to be satisfied, so this circuit makes a local inversion, is this correct?   

  4. etnapowers
    November 19, 2013

    The closed loop trasmittance is given by the V0/VI expression you described only if the overall system works as linear, is this that case?

  5. D Feucht
    November 19, 2013

    I am not sure what a “local inversion” is, but because of the cyclic nonmonotonicity, such a circuit would work like a hystersis switch – a Schmitt trigger circuit. It would have positive feedback that moves it to a stable operating (equilibrium) point. For multiple cycles in the function, there are multiple stable points and to select which one, the circuit has to be cycled through to the desired point, which is the stored value of interest.

  6. D Feucht
    November 19, 2013

    Loop transmittance is nonmonotonic and thus the functional inverse of it is not only nonlinear but is not even a function, just as a hysteresis loop is not a function because for any x value there are two y values. In this case, there are 2^n y values, wheren is the number of bits that can be stored.

    I confess that I have not worked out (no less built) this circuit idea in any detail, though it seems like an attractive scheme to develop. It would be something like a well-known hysteresis circuit but with multiple loops.

  7. etnapowers
    November 25, 2013

    Hi Dennis,

    your answer contains the concept of local inversion, if a function is monotonic in a range , then it can be inverted in that range, so this means that the monotonic property is locally satisfied.


  8. etnapowers
    November 25, 2013


    I agree with you, this situation is a typical case of  hysteresis circuit with multiple loops. The loops are not separable , and it's very difficult to cut the feedback loop to calculate the direct loop gain transfer function.

    The only chance to reach stability is to give a stimulus to one loop per time and then simulate all the stability of all the loops simultaneously. 



  9. Mikael
    April 17, 2020

    It’s a pity that all the equations seems to have gone missing.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.