Adaptive chaotic injection to reduce overfitting in artificial neural networks

Abstract

Artificial neural networks (ANNs) have become an integral tool in various fields of research. ANNs are mathematical models which can be trained to perform various prediction tasks. The effectiveness of an ANN can be impacted by overfitting which occurs when the ANN overfits to the training data. As a result, the ANN does not generalize well to novel data. In our research, we assess the feasibility of using a chaotic strange attractor to generate sequences of values to inject into an ANN to reduce overfitting. An adaptive method was developed to scale and inject the values into the neurons throughout training. The chaotic injection (CI) was tested on three benchmark datasets using different ANN models. The results were compared against the baseline ANN, dropout (DO), and Gaussian noise injection (GNI). The CI improved the performance of the ANN and converged faster than DO and GNI.