Comparative Research of Neuron Circuits

Abstract

Spiking neurons can be implemented in hardware, for example, to model large neural systems, simulate real-time behaviour, and interface bi-directionally between brains and machines. Circuit solutions used to implement silicon neuron circuits depend on the application requirements. Various neuron circuits are presented in this thesis, including spike-event generators (Axon Hillock neuron circuits), above-threshold neuron circuits (Quadratic Integrate and Fire neuron circuits), and differential pair integrator circuits. Cadence\'s tool simulates these circuits using 180nm technology. Comparing these circuits is based on their working properties and simulation results, and their features are demonstrated with experiments.

Introduction

I. INTRODUCTION

Sensory perception, cognitive processes, decision-making, and motor control can be performed by biological systems with low energy consumption. It is a topic of ongoing research to emulate some of the brain's intelligent processing in silicon [1-4]. The field of neuromorphic engineering aims to mimic the mechanisms of biological (brain) systems. The term "Neuromorphic Engineering" was introduced by Carver Mead. He noted many functional similarities between analog CMOS devices and neural processing, as opposed to digital processing [5-7].

Several spike-based neural network simulators have been developed, and much research has focused on software tools and strategies to simulate spike neural networks [8], but with real-time behaviour. It is not suitable for large-scale detailed simulation or system design. Custom digital systems such as graphics processing units (GPUs) and field-programmable gate arrays (FPGAs) can provide such functionality, but these systems can approach neural density and synapses, energy efficiency, and resilience. It is unclear whether this can be done in a model of the central nervous system [9].

Silicon neurons (SiNs) are very large-scale analog/digital integration (VLSI) that simulates the electrophysiological behaviour of real neurons that can be used in neural system hardware implementations. These simulations are much more energy efficient than those performed on general-purpose computers capable of large-scale real-time neural simulations [10, 11]. Depending on the area of ??application of interest, SiN circuits become more or less complex, all large neural networks are integrated on the same chip and a single neuron is mounted on a single chip or spread.

In this task, we used the Cadence tool in 180 nm technology to simulate various circuits commonly used for SiN design in CMOS technology. Through this work, we sought to provide insight into the most representative silicon neural circuit design by addressing the concordance of different neurons in different situations and comparing them. Compare the different approaches taken in the design.

II. RELATED WORK

A. Neuromorphic Computing Systems

Neuromorphic computing system demonstrates some degree of neurobiological inspiration that differentiates them from mainstream conventional computing systems [9]. The term `Neuromorphic Computing' was introduced by Carver Mead in 1990 [12]. Over the last 3 decades, the agenda has been to conceptualize and design substrates that are able to emulate the dynamics of biological networks so as to perform energy-efficient, fault-tolerant, and real-time processing of neural information reminiscent of the mammalian cortex [14].   

In neuromorphic systems, analogous behaviour of sub-threshold electronics and neural ion channel behaviour were analysed, and both implement the neural equations in continuous time whereas the digital systems use some form of discrete-time approximation which is close to mead’s approach [12, 13].

The analog systems were 100 times more efficient in their use of silicon, and they use 10000 times less power than comparable to digital systems, and also more robust to component degradation and failure than more conventional systems.

B. Wafer-Scale Integration of Analog Neural Networks

A novel design of an artificial neural network tailored for wafer-scale integration was implemented which contains continuous-time analog neurons with up to 16k inputs which allow the mapping of network models derived from biology on the VLSI neural network [11]. A single 20-cm wafer contains about 60 million synapses that have been implemented which have been highly accelerated compared to biological real-time. An asynchronous low voltage signalling scheme is introduced that makes the wafer-scale approach feasible by limiting the total power consumption while simultaneously providing a flexible, programmable network topology.

C. Different Tools and Strategies For Simulation Spiking Neurons Network

Different types of simulations strategies and simulation tools were used to implement the neuron networks. Series of benchmark simulations for different types of networks of spiking neurons, including Hodgkin–Huxley type, integrate-and-fire models, interacting with current-based or conductance-based synapses, using clock-driven or event-driven integration strategies were analysed [9]. Simulators were classified into 4 categories according to their most relevant range of applications, and the complete simulation was made on them including the graphical interface and sophisticated tools, and the result was analysed. For different environments, simulations were made on different models, but, unfortunately, codes are not compatible with each other, which underline the need for a more transparent communication channel between simulators.

D. Analog VLSI for Neuromorphic Circuits

Analog VLSI technology looks attractive to the efficient implementation of artificial neural systems for the following reasons.

1. Parallelism: Massively parallel neural systems are efficiently implemented in analog VLSI technology allowing high processing speed. The neural processing elements are smaller than their digital equivalent, so it is possible to integrate on the same chip a large number (i.e., thousands) of interconnections (i.e., synapses).
2. Fault Tolerance: To ensure fault tolerance to the hardware level, it is necessary to introduce redundant hardware and in analog VLSI technology, the cost of additional nodes is relatively low.
3. Low Power: The use of subthreshold MOS transistors reduces the synaptic and neuron power consumption, thus offering the possibility of low-power neural systems.
4. Real-world Interface: Analog neural networks eliminate the need for analog to digital and digital to analog converters and can be directly interfaced with sensors and actuators. This advantage is evident when the data given to the neural network is massive and parallel.
5. Low Values of S/N Ratio: This corresponds to a low precision (i.e., number of bits) in performing the computation. This is not a problem since in the neural system, the overall precision in the computation is determined not by the single computational nodes, but by the number of nodes and interconnections between nodes [17].

Thus, Analog VLSI is best suited for the implementation of the Neuromorphic / Biomimetic Circuits.

III. NEURON STRUCTURE

A. Differential Pair Integrator circuit

Fig. 1 above shows a Differential Pair Integrator (DPI) circuit that integrates the input pulse voltage and follows a current mode approach. However, instead of using a single pFET to generate the appropriate current Iw , it uses a differential pair in a negative feedback configuration. This allows the circuit to achieve Low-Pass Filter functionality with adjustable actuation: the input voltages have been integrated to produce a current of maximum amplitude Isyn defined by Vw, Vt, and Vthr.

B. Spike-Event Generator

The biologically realistic neuron model generates a smooth and continuous analog waveform over time, even when an action potential is generated, but in many other neurons the action potential is an event. Discontinuous and discrete are generated every time a certain threshold is crossed. [7]. Fig. 2 shows a schematic diagram of the proposed Axon-Hillock circuit for generating discrete events. The amplifier used in this circuit consists of two inverters connected in series.

C. The Quadratic Integrator-and-Fire Neuron

The quadratic I&F neuron circuit [21], shown in Fig. 3, is an example of a generalized I&F circuit above the threshold [22]. The required nonlinear oscillation behavior is achieved using a separate post-spike reset mechanism and differential equations of two-state variables. The circuit implementation is not intended to reproduce the precision of the nonlinear equations, but rather to use the simplest possible circuit that can produce the functional behavior of the coupled system of nonlinear equations. The two-state variables are "membrane potential (V)" expressed as voltage across Cv and “slow variable (U)”. The membrane potential consists of the transistors NM1, NM2, NM5, PM1, PM2 and the membrane capacitor Cv. This capacitor (Cv) integrates the spike-generating positive-feedback current of PM2, and the leakage current generated by NM5 [8, 23]. The NM1 transistor generates the positive-feedback current that is mirrored by the transistors PM1-PM2 and is approximately quadratic dependent on the membrane potential. The comparator circuit (PM5, PM6, and NM5 – NM8) detects spikes and provides a reset pulse to the gate of the transistor NM4, causing the membrane potential to rise extremely rapidly to the value determined by the voltage at the node C. the slow variable is built using the transistors NM1, PM1, PM3, NM3, and PM4. The membrane potential determines the magnitude of the current supplied by the same transistor PM3 as a membrane circuit. Transistor NM3 provides the non-linear leakage current. The comparator generates a pulse to turn on the transistor PM4 so that an additional amount of charge is transferred.

IV. RESULT AND DISCUSSION

A. Simulation result of differential pair integrator circuit

Fig. 4. Display the test bench and transient response of the DPI circuit. This circuit consists of four n-FETs, two p-FETs and a capacitor. The n-FETs form a differential pair whose branch current Iin represents the synaptic input during the charging phase. This circuit is less compact than other synaptic circuits that can produce the exponential dynamics seen in excitatory and inhibitory postsynaptic currents of biological synapses, without requires additional input pulse expansion circuit. The DPI synapse shown here has independent control of time constant, synaptic weight, and synaptic extension parameters. DPI provides an additional degree of freedom through Vthr bias which helps in implementing additional adaptation and plasticity schemes [19].

Fig. 5. Shows  the test-bench and Transient response of Axon-hillock circuit. Input current Iin is integrated on the membrane capacitor Cmem. At this point Vout quickly rises to Vdd, switching on the reset transistor and activating positive feedback through the capacitor divider implemented by Cmem and feedback capacitor Cfb. The membrane capacitor discharges when the reset current which is set by Vpw is larger than the input current until it reaches the threshold and Vout swings back to 0 and the cycle repeats. The inter-spike interval is inversely proportional to the input current, while the pulse duration period depends on both the input and reset currents. The action potential is a discontinuous and discrete event which is generated whenever the threshold is crossed. The main advantage of this circuit is that self-resetting property.

C. Simulation result of quadratic integrate-and-fire neuron circuit

Fig. 6. Shows experimental  and transient responses of Izhikevich's neurons. Neurons in the mammalian brain have been classified into several types based on the spike pattern seen in the intracellular recording. From the waveform, we can observe the regular spiking of excitatory cortical cells. When a neuron is stimulated for a long time (injected into the DC step), the neurons fire a few spikes with short intervals between spikes, after which this cycle increases, called is the frequency matching point. Increasing the magnitude of the applied DC will increase the frequency between the spikes, but it will never be too fast due to the large hyperpolarizations after the spikes. In the model, this corresponds to C and D nodes. This neural circuit has a different activation pattern and rapid response compared to other neural circuits.

V. ACKNOWLEDGMENT

The authors would like to thank Padmaja k, Nathasha Vepriyana, Abhinandan L, and Sohan G Naik for their assistance in designing, simulating, and creating layouts.

Conclusion

In this work, we have described different neural circuits that have been developed over the years, using different design approaches and for a variety of application scenarios. We simulated all neurons presented in 180 nm technology using the cadence tool. In particular, we described Quadratic I&F neurons [20]. Sub-threshold current-mode circuits have a higher degree of mismatch than those above-thresholds, but they have lower noise energy and higher source efficiency [8, 15, 16]. The DPI synapse shown here has independent control of time constant, synaptic weight, and synaptic extension parameters. DPI provides an additional degree of freedom through Vthr bias which helps in implementing additional flexibility and adaptive plans [19]. Quadratic I&F neurons have fast peak power compared to other neurons, and the Axon-Hillock circuit has good self-healing and adaptive properties. There is no specific choice of SiN circuit style and design. Depending on its use application, a variety of circuit designs are already there. We can say that there is no absolute optimal design, since there are many types of neurons in biology, there are many design and circuit choices for SiN [8].

References

[1] G. Indiveri; E. Chicca; R. Douglas “ A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity ”, IEEE Transactions on Neural Networks, Jan. 2006, Volume: 17, Issue: 1. pp 211 – 221. [2] Adria Bofill-i-petit, Alan F Murray “ Synchrony detection and amplification by silicon neurons with STDP synapses ” IEEE Trans Neural Netw. 2004 Sep;15(5):1296-304 [3] Shih-Chii Liu; R. Douglas “ Temporal coding in a silicon network of integrate-and-fire neurons ” IEEE Transactions on Neural Networks, Volume: 15, Issue: 5, Sept. 2004, pp. 1305 – 1314 [4] Chicca; D. Badoni; V. Dante; M. D\'Andreagiovanni; G. Salina; L. Carota; S. Fusi; P. Del Giudice “ A VLSI recurrent network of integrate-and-fire neurons connected by plastic synapses with long-term memory ” IEEE Transactions on Neural Networks, Volume: 14, Issue: 5, Sept. 2003, pp. 1297 – 1307 [5] Runchun Wang “ Neuromorphic Implementations of Polychronous Spiking Neural Networks ” neural networks (computer science) polychronous networks neuromorphic engineering, 2013 [6] Mead, C. A. “ Neuromorphic electronic systems ”. Proc. IEEE 78, (1990), 1629–1636. [7] Carver Mead “ Analog VLSI and neural systems ” Addison-Wesley Longman Publishing Co., Inc.75 Arlington Street, Suite 300 Boston, MAUnited States, jan-1989, pp. 371 [8] Giacomo Indiveri1, Bernabé Linares-Barranco, Tara Julia Hamilton, André van Schaik, Ralph Etienne-Cummings, Tobi Delbruck, Shih-Chii Liu, Piotr Dudek6, Philipp Häfliger, Sylvie Renaud, Johannes Schemmel, Gert Cauwenberghs, John Arthur, Kai Hynna, Fopefolu Folowosele, Sylvain Saighi, Teresa Serrano-Gotarredona, Jayawan Wijekoon, Yingxue Wang and Kwabena Boahen “ Neuromorphic silicon neuron circuits ”, Frontiers in Neuroscience, 31 May 2011 [9] Brette, R., Rudolph, M., Carnevale, T., Hines, M., Beeman, D., Bower, J. M., Diesmann, M., Morrison, A., Goodman, F. C., Harris P. H. Jr., Zirpe, M., Natschläger, T., Pecevski, D., Ermentrout, B., Djurfeldt, M., Lansner, A., Rochel, O., Vieville, T., Muller, E., Davison, A. P., El Boustani, S., and Destexhe, A.. “ Simulation of networks of spiking neurons: a review of tools and strategies ”. J. Comput. Neurosci. 23, (2007), 349–398. [10] Silver, R., Boahen, K., Grillner, S., Kopell, N., and Olsen, K. L.. ” Neurotech for neuroscience: unifying concepts, organizing principles, and emerging tools ”. J. Neurosci. 27, (2007), 11807–11819. [11] Schemmel, J., Fieres, J., and Meier, K.“ Wafer-scale integration of analog neural networks ”, in Proceedings of the IEEE International Joint Conference on Neural Networks, Hong Kong, (2008), 431–438.. [12] John Zachary Young, Larry R. Square “ The History of Neuroscience in Autobiography ”, Volume -1, In 1937, 157-772 pp. [13] Indiveri, G., Chicca, E., and Douglas, R. (2009). “ Artificial cognitive systems: from VLSI networks of spiking neurons to neuromorphic cognition ”. Cognit. Comput. 1, (2009), 119–127. [14] Steve Furber 2016 J. Neural Eng. 13 051001. “ Large-scale neuromorphic computing systems ”. J. Neural Eng. 13 (2016) 051001 (14pp). [15] Attwell and S. B. Laughlin, “ An energy budget for signaling in the grey matter of the Brain ”,Journal of Cerebral Blood Flow & Metabolism, 2001, vol. 21, no. 10, pp. 1133-1145. [16] . V. Arthur and K. Boahen, “ Recurrently connected silicon neurons with active dendrites for one-shot learning\", IEEE International Joint Conference on Neural Networks (IEEE Cat. No. 04CH37541), vol. 3. IEEE, 2004, pp. 1699-1704. [17] P. Northmore and J. G. Elias, “ Building silicon nervous systems with dendritic tree neuromorphs ”, Pulsed neural networks, 1998 pp. 135-156. [18] Qiangfei Xia and J. Joshua Yang, “ Memristive crossbar arrays for brain-inspired computing ”. Natures Material, April 2009,(15pp) https://doi.org/10.1038/s41563-019-0291-x. [19] A. Vittoz, “ Analog VLSI Signal Processing: Why,Where, and How? ”, Journal of VLSI Signal Processing, 1994, vol. 8, pp. 27–44. [20] Bartolozzi, C., and Indiveri, G. (2007). “ Synaptic dynamics in analog VLSI “. Neural Comput. 19, 2581–2603. [21] Culurciello, E., Etienne-Cummings, R., and Boahen, K. (2003). “ A biomorphic digital image sensor ”. IEEE J. Solid State Circuits 38, 281–294. [22] Wijekoon, J. H. B., and Dudek, P. (2008). “ Compact silicon neuron circuit with spiking and bursting behavior ”. Neural Netw. 21, 524–534. [23] Izhikevich, E. M. (2003). “ Simple model of spiking neurons ”. IEEE Trans. Neural Netw. 14, 1569–1572. [24] van Schaik, A., Jin, C., Hamilton, T. J., Mihalas, S., and Niebur, E. (2010a). “A log-domain implementation of the Mihalas-Niebur neuron model,” in International Symposium on Circuits and Systems, ISCAS 2010 (Paris: IEEE), 4249–4252. [25] van Schaik, A., Jin, C., and Hamilton. T. J. (2010b). “A log-domain implementation of the Izhikevich neuron model,” in International Symposium on Circuits and Systems, ISCAS 2010 (Paris: IEEE), 4253–4256. [26] Rangan, V., Ghosh, A., Aparin, V., and Cauwenberghs, G. (2010). “A subthreshold aVLSI implementation of the Izhikevich simple neuron model,” In IEEE Engineering in Medicine and Biology Conference EMBC (Buenos Aires: IEEE).

Copyright

Copyright © 2022 Nisarga G S, Dr. Punith Kumar M B, Dr. Mahesh, M Subramanyam. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.