Recurrent neural network consisting of FitzHugh–Nagumo systems: characteristics required for training

The purpose of this study is to determine the feasibility and features of training a recurrent neural network consisting of FitzHugh-Nagumo systems with delayed feedback to predict an impulse (spike signal). Methods. The network under consideration consisted of N=60 FitzHugh–Nagumo systems with different lag times. During training, the problem of which neuron should be activated and with what strength of lagged feedback was solved. The network was trained using gradient descent from different initial conditions. In the process of research, it was found that the use of standard recurrent network training characteristics such as mean squared error or mean absolute error was not applicable to this task, so an alternative method for computing the loss function was proposed. Results. The proposed combined loss function is the sum of MSE error and interspike interval error, and therefore has the following advantages: 1 – includes the information about spike periodicity and interspike intervals, 2 – responds adequately to the absence of a network output signal, 3 – takes into account small amplitude fluctuations in addition to impulse dynamics, which allows predicting complex quasi-periodic signals. It has been shown that gradient descent can be used for the problem, but several initial conditions must be used because of the nonlinearity of the loss function. The more initial conditions, the more accurate the result. Conclusion. The problem of predicting a pulse (spike) signal using a self-closed recurrent neural network consisting of FitzHughNagumo systems with delayed feedback has been successfully solved. It was clearly shown what features should be taken into account during loss function calculation and how the gradient descent should be realized.  

neural networks, recurrent neural networks, FitzHugh–Nagumo system, time-delayed feedback