Graduation Semester and Year
2016
Language
English
Document Type
Thesis
Degree Name
Master of Science in Electrical Engineering
Department
Electrical Engineering
First Advisor
Michael T Manry
Abstract
A Multilayer perceptron typically has a fixed nonlinear activation function for each hidden unit. In this thesis, an adaptive activation function for individual hidden unit is designed, where the network learns these activation functions at every iteration using a modern second order algorithm. Methods and algorithms for these adaptive activation functions along with several other techniques for training a multilayer perceptron’s weights are discussed. Comparisons between a multilayer perceptron with sigmoidal activation functions and a multilayer perceptron with piecewise linear activation functions are also discussed. The common activation function used is the sigmoidal activations, but it is still not proven that the sigmoidal activations works best for all the applications. Hence the adaptive activation technique described in this thesis can be used, which learns independently as it passes through the data.
Keywords
Neural networks, Multilayer perceptron, Adaptive activation functions, Output weight optimization, Hidden weight optimization, Piecewise linear activation function, Multiple optimal learning factor
Disciplines
Electrical and Computer Engineering | Engineering
License
This work is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 4.0 International License.
Recommended Citation
Rane, Chinmay Appa, "MULTILAYER PERCEPTRON WITH ADAPTIVE ACTIVATION FUNCTIONS" (2016). Electrical Engineering Theses. 336.
https://mavmatrix.uta.edu/electricaleng_theses/336
Comments
Degree granted by The University of Texas at Arlington