Neuron building blocksInput signals received by neurons are forwarded to their axons that are long and thin extensions of the main body. The first axon part is called axon hillock and typically initiates the generation of a spiking voltage signal called action potential. This action potential is transmitted via the remaining axon that is typically insulated except for distinct points. The latter are called nodes of Ranvier and exhibit ion channels that regenerate the incoming action potential. In this work, we simply call the parts of an axon that connect nodes of Ranvier axon segments. The electrical signal transmission stops at the end of an axon, where a chemical transmission to the next neuron is carried out via synapses. In the following, we discuss the modeling of these elementary building blocks of a neuron.Axon hillock, node of Ranvier, and axon segmentAxon hillocks and nodes of Ranvier are functionally similar in that they are both capable of generating action potentials. In this work, the only difference between the two is that only axon hillocks receive external input currents j. They can in general be modeled by any neuronal oscillator. In this work, we use the equivalent electrical circuit of the Morris-Lecar model, because it is mathematically less complex than e.g. the Hodgkin-Huxley model, but still biologically reasonable, cf.8. Since typical neurons contain sodium and potassium channels, we replace the original model’s calcium channels by sodium channels , cf.8,27,29. The resulting model equations read $$C\dot{u} = j – G_{{\text{L}}} \left[ {u – E_{{\text{L}}} } \right] – g_{{{\text{Na}}}} (u)\left[ {u – E_{{{\text{Na}}}} } \right] – W_{{\text{K}}} \left[ {u – E_{{\text{K}}} } \right]{\mkern 1mu} ,\quad g_{{{\text{Na}}}} (u) = \frac{{G_{{{\text{Na}}}} }}{2}\left[ {1 + {\text{tanh}}\left( {\frac{{u – U_{{{\text{Na}}1}} }}{{U_{{{\text{Na}}2}} }}} \right)} \right]{\mkern 1mu} ,\quad W_{{\text{K}}} = z_{{\text{K}}} G_{{\text{K}}}$$
(1a)
$$\begin{aligned} {\dot{z}}_{\textrm{K}}&= \left[ f(u) -z_{\textrm{K}} \right] F_{\textrm{K}}\textrm{cosh}\left( \frac{u-U_{\textrm{K}1}}{2U_{\textrm{K}2}}\right) \,,&f(u)&= \frac{1}{2}\left[ 1+ \textrm{tanh}\left( \frac{u-U_{\textrm{K}1}}{U_{\textrm{K}2}}\right) \right] \,. \end{aligned}$$
(1b)
Here, u is the membrane potential. The sodium channels are described by the conductance function \(g_{{{\text{Na}}}} (u)\), and the potassium channels are characterized by the memristance \(W_{{\text{K}}}\) governed by the fraction of open channels \(z_{{\text{K}}}\). The voltage-dependent opening and closing of these channels is described by f(u). Here, \(f(u) = 0\) indicates that all channels of the considered ion type are closed, \(0< f(u) < 1\) means that some of the channels are open, and \(f(u)= 1\) corresponds to all channels being open. The combined channel equations govern a threshold-based oscillation behavior: When the input current j is large enough to increase u above a certain threshold voltage, \(g_{{{\text{Na}}}}\) increases and triggers a rapid and strong rise of u due to the positive resting potential \(E_{{{\text{Na}}}}\). This leads to positive voltage values that result in an increase of \(W_{{\text{K}}}\) and an activation of \(E_{{\text{K}}}\). Since \(E_{{\text{K}}}\) is negative, u drops back to the resting potential that is close to \(E_{{\text{L}}}\). Parameters utilized in this work are shown in Table 1 and are based on29. The corresponding equivalent electrical circuit is depicted in Fig. 3b. Note that we only apply an external current j to axon hillock models that are to grow from the beginning.Table 1 Parameters for node of Ranvier and axon hillock.To model an axon segment, one can use a constant resistor that interconnects the Morris-Lecar circuits in series8. We call the alternating interconnection of Morris-Lecar circuits and resistors a static axon model, because it cannot change its structure, but exhibits a signal transmission delay. The latter stems from the fact that subsequent oscillators can only begin their oscillation once previous ones started, leading to a delay increasing with each oscillator added.SynapseSynapses form when an axon contacts the dendrites of another neuron. Synaptic transmission occurs when an action potential reaches the end of an axon, leading to the opening of ion channels at the dendrites of the postsynaptic neuron. As this work focuses on axon guidance, we use a synapse model mimicking the channel opening and neglect short- and longterm memory aspects typically associated with synapses30. In particular, we deploy a memristor model comparable to the potassium channel memristor of the Morris-Lecar model. Similar to an axon segment model, the synapse model is connected in series with a node of Ranvier and an axon hillock, see Fig. 3. The model equations read$$W_{{\text{s}}} (z_{{\text{s}}} ) = G_{{{\text{s1}}}} z_{{\text{s}}} + G_{{{\text{s0}}}} ,\quad \dot{z}_{{\text{s}}} = \left[ {\frac{1}{2}\left[ {1 + {\text{tanh}}\left( {\frac{{u – U_{{{\text{s}}1}} }}{{U_{{{\text{s}}2}} }}} \right)} \right] – z_{{\text{s}}} } \right]F_{{\text{s}}} ,$$
(2)
where \(W_\mathrm{s}\) is the synaptic memductance and \(z_\mathrm{s}\) is its state variable. \(G_\mathrm{s1}=20\) μS is the maximum conductance, \(G_\mathrm{s1}=5\) μS is the minimum conductance, \(U_\mathrm{s1}=40\,\textrm{mV}\) is the threshold voltage, \(U_\mathrm{s2}=1\,\textrm{mV}\) is the edge steepness, and \(F_\mathrm{s}=3\,\textrm{kHz}\) denotes the opening rate of synaptic channels. Manufactured memristors with similar resistance ranges are e.g. reported on in31,32.Growth conceptIn biology, the growth of axons is guided by guidance cues, where axons follow a path along concentration gradients33. We denote these growth-controlling cues as growth cues. The aim of this work is to implement this growth behavior in a bio-inspired way by an electrical circuit. For this purpose, we assume a regular grid structure as the environment in which growth can occur, see Fig. 1b.Figure 1Illustration of the growth concept and pruning concept. Here, large circles represent axon hillocks, where only the yellow ones are active. Small circles are potential nodes of Ranvier, with the yellow ones denoting grown nodes of Ranvier. Thin lines are potential axon segments, where the yellow ones denote the actually grown ones. Thick orange lines are potential synapses that are formed when connected to a grown axon segment. (a) Directional derivatives of the growth map are highlighted in red at the edges of the grid. (b) Growth map with grid structure due to gradient-based network growth. (c) Pruning map with the same grid structure as in (b). Note that \(t_0\) is the start time, \(t_\mathrm{growth}\) is the time at which growth is completed, and \(t_\mathrm{end}\) is the time at which pruning is completed.Each grid point is either a potential node of Ranvier (small circles) or an axon hillock (large circles), where the latter are only placed at start and target positions. Edges between potential nodes of Ranvier are potential axon segments (thin lines), and edges between axon hillocks and potential nodes of Ranvier are either potential axon segments or potential synapses (thick orange lines). The positions of the targets and synapses need to be determined in advance. If the sensed directional derivatives of the concentration map, see Fig. 1a, are larger than a certain threshold, the axon segments grow towards the next node of Ranvier. This is illustrated in Fig. 1b for a threshold of 0. Here, active axon hillocks as well as grown nodes of Ranvier and axon segments are highlighted in yellow.Pruning conceptWhile we associate the addition of new axon segments with growth, we refer to the removal of axon segments as pruning. According to34,35, one often distinguishes between small-scale and large-scale axonal pruning. The former describes the reduction of the number of interconnections between two neurons, while large-scale pruning is associated with removing the entire link between two neurons. This type of pruning is assumed to be predetermined and occurs e.g. for removing unnecessary or false interconnections. In this work, we focus on the functional connection structure of neurons, i.e., whether a connection exists or not. Since small scale pruning has no influence on this, we only consider large scale pruning.As a simple modeling approach, we assume that pruning cues are generated at the target neuron only when it is connected to the outgrowing neuron. This is illustrated in Fig. 1b. This approach is inspired by an axon pruning example observed from the neocortex of mice, see34. Note that different neurons can be sensitive to different pruning cues, see e.g.34. For reasons of simplicity, however, we account for a single pruning cue that can prune axons of different neurons.Growth- and pruning-cue sensitive axon segmentBased on8, axon growth in the proposed grid structure can be interpreted as the length increase of an axon segment until a conductive path between two nodes of Ranviers emerges. In a similar sense, pruning is then associated with the length decrease of an axon segment. The growth of axon segments should depend on the sensed directional derivatives of the growth concentration and should only occur at consecutive positions. The latter can be ensured by transmitting a control signal to the current ending position(s) of the axon. To allow for pruning, the axon segment should also be able to sense the pruning cue concentration.From an electrical circuit point of view, the length increase or decrease of an axon segment is equivalent to a decreasing or increasing resistance, respectively. An electrical circuit model for the axon segment is hence required to dynamically adjust its resistance value. This adjustment should depend on (i) a control signal triggering the growth, (ii) external stimuli representing directional derivatives of a growth cue concentration, and (iii) external stimuli that account for a pruning cue concentration. For these reasons, we use a memristor in combination with multiple sensors, which we call memsensor, cf.36. Note that memristors acting as sensors themselves have also been reported, see37. We neglect this option in this work, since we believe that with the current state of art, memristors in combination with sensors offer a better implementable and flexible approach due to less limitations of available devices.Figure 2(a) Memsensor as axon segment model and (b) its equivalent circuit consisting of memristors, gradient sensors (blue), and a general sensor (red).The memsensor model considered in this work is shown in Fig. 2 and consists of two anti-parallel memristors, two anti-parallel gradient sensors, and one general sensor. Note that the anti-parallel interconnection of memristors and gradient sensors enables a growth in both directions. Moreover, memristors account for a non-volatile growth of axon segments. The memristor model used in this work yields $$M(z_{{\text{a}}} ) = M_{1} + z_{{\text{a}}} \left[ {M_{0} – M_{1} } \right]{\mkern 1mu} ,\quad \dot{z}_{a} = w(z_{a} )\left[ {S\left[ {\sigma \left( {u – U_{ + } } \right) – \sigma \left( { – u – U_{ – } } \right)} \right] – S_{{\text{r}}} } \right],$$
(3a)
with the memristance M, the memristor state \(z_\mathrm{a}\), the heaviside function \(\sigma (\cdot )\), and a window function \(w(z_\mathrm{a})\) ensuring that \(z_\mathrm{a}\in [0,1]\). \(S_\mathrm{r}\) denotes the slope of a retention characteristic, which enables an increase of the memristance and thus implements the length decrease due to pruning. The gradient sensor is described by$$\begin{aligned} R_\mathrm{g}(\Delta \eta _\mathrm{g})&= R_{\textrm{g},0} + R_\mathrm{g,1}\left[ 1 – \textrm{tanh}\left( \frac{\Delta \eta _\mathrm{g}-\Delta \eta _\mathrm{g1}}{\Delta \eta _\mathrm{g2}}\right) \right] , \end{aligned}$$
(3b)
where \(\Delta \eta _\mathrm{g}\) is a normalized, dimensionless gradient sensor signal representing the directional derivative of the sensed growth cue concentration \(\eta _\mathrm{g}\). The general sensor model reads$$\begin{aligned} R_\mathrm{p}(\eta _\mathrm{p})&= R_\mathrm{p,0} + R_\mathrm{p,1}\left[ 1 + \textrm{tanh}\left( \frac{\eta _\mathrm{p}-\eta _\mathrm{p1}}{\eta _\mathrm{p2}}\right) \right] \,. \end{aligned}$$
(3c)
Here, \(\eta _\mathrm{p}\) is a normalized, dimensionless sensor signal that we associate with the pruning cue concentration. For a technical implementation, the external stimuli representing growth and pruning cue concentrations can be given by e.g. light sources with two different wave lengths. In this sense, two light sensors for two different wave lengths are required.Under ideal conditions, the functionality of the complete memsensor can be characterized as follows: To disable signal transmission between two oscillators, memristors are initially in their high resistance states \(M_1\). In the absence of strong growth cue gradients, gradient sensors are also high ohmic. Choosing \(R_{\textrm{p},1}>M_{1}\), the majority of the voltage is present at the sensor. This prevents memristors from switching to \(M_1\) and thus signals being transmitted when no growth indication is given. Growth starts when three conditions are met: (i) One of the oscillators connected to the memsensor must be active. (ii) Growth cue gradients are sensed by one of the sensors \(R_\mathrm{g}\) and have to exceed a certain threshold value. (iii) The growth cue gradient has to be directed towards the inactive oscillator. If these conditions are met, one memristor switches towards \(M_1\), because \(R_\mathrm{g}\) of the same direction is now decreased, enabling the memristor voltage to exceed its threshold. This way, the total resistance of the memsensor becomes \(M_0+R_{\textrm{g},0}+R_{\textrm{p},0}\). Given that \(M_0+R_{\textrm{g},0}+R_{\textrm{p},0}\) is below the minimum resistance that prevents signal transmission, the previously inactive oscillator begins to oscillate. For the chosen Morris-Lecar oscillators, this resistance threshold is roughly \(100\,\mathrm {k\Omega }\). Pruning takes place when pruning cues are sensed, leading to an increased \(R_p=R_{\textrm{p},1}\). Setting \(R_{\textrm{p},1}>M_1\), the complete voltage is now present at the pruning sensor. This way, the memristor with \(M_0\) is driven back to its high resistance state \(M_1\) via its retention characteristic. The parameters utilized for the memsensor are shown in Table 2. Memristors observed in experimental works with comparable parameters, especially in terms of resistances, can for instance be found in38,39,40. Similar threshold voltages can be achieved when rescaling the oscillator circuit parameters.Table 2 Memsensor parameters.Pruning-cue generation circuitIn order to get closer to biology with our circuit model, we incorporate a dynamic pruning cue generation. For modeling purposes, we assume that neurons only act on local information, i.e. if a synaptic connection at their dendrites exist, or if action potentials are generated at their axon hillock. Based on this, we assume that pruning cues are only generated at the target neuron once they become active, since this is a signal-based indication that a connection to at least one of the outgrowing neurons has been established. Since we, for reasons of simplicity, consider only a single pruning cue that should prune only undesired connections, a directional emission of the cues is also necessary. A model for the pruning cue generation yields $$\begin{aligned} {\dot{\eta }}_{\textrm{p},0}&= \alpha \sigma (u-u_\mathrm{th})-\beta \eta _{\textrm{p},0}\,,{} & {} {}{} & {} \end{aligned}$$
(4a)
where \(\eta _{\textrm{p},0}\) is the pruning cue concentration generated at the axon hillock of a target neuron located at \((x_0,y_0)\) and u is the membrane potential of this axon hillock. \(\alpha =120\) is the pruning cue generation rate, \(\beta =3\) is the pruning cue decay rate, and \(u_\mathrm{th}=15\,\textrm{mV}\) is the threshold voltage for generating pruning cues. The damping of the pruning cue emission is described via$$\begin{aligned} \eta _\mathrm{p}&= \frac{1}{2}\left[ \frac{{\textbf{s}}^T{\textbf{r}}}{|{\textbf{s}}||{\textbf{r}}|^2} + 1\right] \frac{\eta _{\textrm{p},0}}{1 + |{\textbf{r}}|}\,,&{\textbf{s}}&= \begin{pmatrix} x_\mathrm{s}-x_0\\ y_\mathrm{s}-y_0 \end{pmatrix}\,,&{\textbf{r}}&= \begin{pmatrix} x-x_0\\ y-y_0 \end{pmatrix}. \end{aligned}$$
(4b)
Here, (x, y) is the observed position on the grid, \((x_\mathrm{s},y_\mathrm{s})\) is the position of the synapse whose connected axon should be pruned. The vectors \({\textbf{s}}\) and \({\textbf{r}}\) indicate the directions from target axon hillock to the synapse and observed grid point, respectively, as well as the corresponding distances.In a second step, we represent the differential equation (4a) responsible for the cue generation with an equivalent electrical circuit shown in Fig. 3a. Redefining \(\alpha = \alpha _1\alpha _2\), with \(\alpha _1=3 \cdot 10^3\) and \(\alpha _2=40\cdot 10^{-3}\), the circuit is governed by$$\begin{aligned} C_\mathrm{p}{\dot{u}}_{\textrm{p},0}&= \sigma (u-u_\mathrm{th})I – \frac{1}{R_\mathrm{p}}u_{0}\,,&u_{\textrm{p},0}&= \eta _{\textrm{p},0}U_0\,,&R_\mathrm{p}&= \frac{\alpha _1}{\beta }R_0\,,&C_\mathrm{p}&= \frac{1}{\alpha _1}C_0\,,&I&=\alpha _2 I_0\,. \end{aligned}$$
(5)
Here, \(C_\mathrm{p}\) and \(R_\mathrm{p}\) are the capacitance and the resistance of the RC circuit, \(u_{\textrm{p},0}\) is the voltage representing the pruning cue concentration at the point of cue generation \((x_0,y_0)\), and I is the input current amplitude. \(I_0 = 1\,\textrm{A},\, U_0 = 1\,\textrm{V},\, R_0=1\,\mathrm {\Omega }\), and \(C_0=1\,\textrm{F}\) are a normalization current, voltage, resistance, and capacitance. With regard to a technical implementation based on light sensors, the RC circuit could be replaced by a light-emitting diode. Note that in our model, the spread of the cue concentration is still mathematically calculated via Eq. (4b).
A memristive circuit for self-organized network topology formation based on guided axon growth
