|
Many techniques have been developed in recent years in order to speed up the training process of Multilayer Perceptrons (MLPs), such as accelerated versions of gradient descent, second order methods, Kalman related algorithms, and block learning methods; among them, the latter offer a reasonable way of reducing training effort and getting good results with MLPs. However, they usually find suboptimal solutions and, sometimes, the values obtained for the weights are excessively large. In this paper, we analyze the drawbacks of these block methods using a very simple test model, we propose several modifications by discarding samples (DS- LSB), using controlled perturbation (CP-LSB), and by means of a Reduced Sensitivity version (RS-LSB) and we also extend their forms to include some especial characteristics: we propose a robust training algorithm relying on Total Least Squares (TLS) minimizations (the so-called RS-TLS algorithm), useful for noisy training patterns, and we also present a method for training MLPs with general output cost function (the `RS-WLSB' algorithm). The advantages of these methods with respect to related algorithms are illustrated using several test problems. Finally, we extract some conclusions and propose, as a further work, the development of recursive implementations, able to learn on-line and to deal with non-stationary problems, and the application of block training methods to recurrent networks.
|
