ORCID Identifier(s)

0000-0002-1373-819X

Graduation Semester and Year

2016

Language

English

Document Type

Thesis

Degree Name

Master of Science in Electrical Engineering

Department

Electrical Engineering

First Advisor

Michael T Manry

Abstract

A Multilayer perceptron typically has a fixed nonlinear activation function for each hidden unit. In this thesis, an adaptive activation function for individual hidden unit is designed, where the network learns these activation functions at every iteration using a modern second order algorithm. Methods and algorithms for these adaptive activation functions along with several other techniques for training a multilayer perceptron’s weights are discussed. Comparisons between a multilayer perceptron with sigmoidal activation functions and a multilayer perceptron with piecewise linear activation functions are also discussed. The common activation function used is the sigmoidal activations, but it is still not proven that the sigmoidal activations works best for all the applications. Hence the adaptive activation technique described in this thesis can be used, which learns independently as it passes through the data.

Keywords

Neural networks, Multilayer perceptron, Adaptive activation functions, Output weight optimization, Hidden weight optimization, Piecewise linear activation function, Multiple optimal learning factor

Disciplines

Electrical and Computer Engineering | Engineering

Comments

Degree granted by The University of Texas at Arlington

Download

Share

COinS