Graduation Semester and Year

2013

Language

English

Document Type

Dissertation

Degree Name

Doctor of Philosophy in Electrical Engineering

Department

Electrical Engineering

First Advisor

Michael T Manry

Abstract

Training a feedforward multilayer perceptron (MLP) requires obtaining train- ing data and solving a non-convex optimization problem to calculate the network's weights. Various problems can arise during training that ultimately can limit a MLP's usefulness such as slow convergence and high computational complexity. Additionally, when training one needs to have confidence that the chosen algorithm is working optimally for the chosen coordinate system.We introduce novel second order training algorithms to overcome these difficulties. In the process, a piecewise affine model of the multilayer perceptron is introduced which shows that objective functions for training are poorly modeled by quadratic functions of network weights. One step and multistep second order training algorithms are derived which avoid the problems implied by the model.The new second order algorithms are shown to have a form of affine invariance which ensures that they are optimal in the sense that they cannot be improved by affine transformation.In simulation, their training and validation performance is comparable to Levenberg- Marquardt, yet they have the advantage of reduced computational complexity.

Disciplines

Electrical and Computer Engineering | Engineering

Comments

Degree granted by The University of Texas at Arlington

Download

Share

COinS