ORCID Identifier(s)

0000-0002-1373-819X

Graduation Semester and Year

2021

Language

English

Document Type

Dissertation

Degree Name

Doctor of Philosophy in Electrical Engineering

Department

Electrical Engineering

First Advisor

Dr Michael T Manry

Abstract

Deep learning training training algorithms are a huge success in recent years in many fields including speech, text,image video etc. Deeper and deeper layers are proposed with huge success with resnet structures having around 152 layers. Shallow convolution neural networks(CNN's) are still an active research, where some phenomena are still unexplanined. CNN's are assumed to be invariant to shift due to its architecture, but recent studies have shown other wise. Apart from shift invariance, activation functions used in the network are of utmost importance, as they provide non linearity to the networks. Relu's are the most commonly used activation function. We show a shallow network which is specifically used for classifying images with shifted objects. Completed Tasks are shown for analyzing and improving shallow networks shift invariance in convolutional neural networks. We demonstrate commonly used downsampling technique and show if these downsampling techniques work for shallow CNN's. We also show a way to factorize the output weights in the feature layer. A traditional segmentation example is shown for the shifted objects and subsequent results are also given. We also show a complex piece-wise linear(PWL) activation in the hidden layer. We show that these PWL activations work much better than relu activations in our networks for convolution neural networks and multilayer perceptrons. Result comparison in MATALB and PYTORCH for shallow and deep CNNs are given to further strengthen our case. A naive growing and pruning algorithm for these PWL activation is shown and compared with original results.

Keywords

Neural networks, Convolutional neural networks, Activation functions, Adaptive activation functions, Segmentation, Shift invariance

Disciplines

Electrical and Computer Engineering | Engineering

Comments

Degree granted by The University of Texas at Arlington

30015-2.zip (5714 kB)

Share

COinS
 
 

To view the content in your browser, please download Adobe Reader or, alternately,
you may Download the file to your hard drive.

NOTE: The latest versions of Adobe Reader do not support viewing PDF files within Firefox on Mac OS and if you are using a modern (Intel) Mac, there is no official plugin for viewing PDF files within the browser window.