Graduation Semester and Year




Document Type


Degree Name

Doctor of Philosophy in Electrical Engineering


Electrical Engineering

First Advisor

Michael T Manry


Training a feedforward multilayer perceptron (MLP) requires obtaining train- ing data and solving a non-convex optimization problem to calculate the network's weights. Various problems can arise during training that ultimately can limit a MLP's usefulness such as slow convergence and high computational complexity. Additionally, when training one needs to have confidence that the chosen algorithm is working optimally for the chosen coordinate system.We introduce novel second order training algorithms to overcome these difficulties. In the process, a piecewise affine model of the multilayer perceptron is introduced which shows that objective functions for training are poorly modeled by quadratic functions of network weights. One step and multistep second order training algorithms are derived which avoid the problems implied by the model.The new second order algorithms are shown to have a form of affine invariance which ensures that they are optimal in the sense that they cannot be improved by affine transformation.In simulation, their training and validation performance is comparable to Levenberg- Marquardt, yet they have the advantage of reduced computational complexity.


Electrical and Computer Engineering | Engineering


Degree granted by The University of Texas at Arlington