Techniques for avoiding local minima in gradient-descent-based ID algorithms

Joan L. Brierton

doi:10.1117/12.276095

10 June 1997 Techniques for avoiding local minima in gradient-descent-based ID algorithms

Joan L. Brierton

Proceedings Volume 3066, Radar Sensor Technology II; (1997) https://doi.org/10.1117/12.276095
Event: AeroSense '97, 1997, Orlando, FL, United States

Abstract

One nonlinear adaptive approach to generating target recognition algorithms, the distributed connectionist approach, is also referred to as a neural network. These algorithms frequently employ a gradient descent technique, such as the back propagation learning algorithm, to find a mapping that separates the n dimensional feature space into m recognizable classes. Gradient descent techniques are known to be limited by a characteristic referred to as the `local minima' problem. During the search for an optimum solution or global minima, these techniques can encounter local minima from which they cannot escape due to the `steepest descent' nature of the approach. However, several training techniques used to speed up training or to otherwise optimize these adaptive learning algorithms have side effects which can obviate this local minima problem. We will define a local minima problem with respect to the 1D target ID problem. Appropriate terminology and an error space relevant to the 1D range profile problem will be presented. Four techniques, dynamic architecture definition, weight pruning, adaptive learning rate selection and dynamic training set generation used to optimize training for the multilayer perceptron will be summarized. An analytical explanation of a common underlying mechanism which allows escape from local minima and is shared by these techniques is presented. Some additional advantages are provided by one of the four techniques, the dynamic training set technique. Evidence of these advantages, consistently high quality results, the automatic identification of anomalous signatures in the data base and simple implementation, will be presented.

Citation Download Citation

Joan L. Brierton "Techniques for avoiding local minima in gradient-descent-based ID algorithms", Proc. SPIE 3066, Radar Sensor Technology II, (10 June 1997); https://doi.org/10.1117/12.276095

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available