Graduation Semester and Year
2021
Language
English
Document Type
Dissertation
Degree Name
Doctor of Philosophy in Electrical Engineering
Department
Electrical Engineering
First Advisor
Dr Michael T Manry
Abstract
Deep learning training training algorithms are a huge success in recent years in many fields including speech, text,image video etc. Deeper and deeper layers are proposed with huge success with resnet structures having around 152 layers. Shallow convolution neural networks(CNN's) are still an active research, where some phenomena are still unexplanined. CNN's are assumed to be invariant to shift due to its architecture, but recent studies have shown other wise. Apart from shift invariance, activation functions used in the network are of utmost importance, as they provide non linearity to the networks. Relu's are the most commonly used activation function. We show a shallow network which is specifically used for classifying images with shifted objects. Completed Tasks are shown for analyzing and improving shallow networks shift invariance in convolutional neural networks. We demonstrate commonly used downsampling technique and show if these downsampling techniques work for shallow CNN's. We also show a way to factorize the output weights in the feature layer. A traditional segmentation example is shown for the shifted objects and subsequent results are also given. We also show a complex piece-wise linear(PWL) activation in the hidden layer. We show that these PWL activations work much better than relu activations in our networks for convolution neural networks and multilayer perceptrons. Result comparison in MATALB and PYTORCH for shallow and deep CNNs are given to further strengthen our case. A naive growing and pruning algorithm for these PWL activation is shown and compared with original results.
Keywords
Neural networks, Convolutional neural networks, Activation functions, Adaptive activation functions, Segmentation, Shift invariance
Disciplines
Electrical and Computer Engineering | Engineering
License
This work is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 4.0 International License.
Recommended Citation
Rane, Chinmay Appa, "ADAPTIVE ACTIVATIONS AND SHIFT INVARIANCE IN SHALLOW CONVOLUTIONAL NEURAL NETWORKS" (2021). Electrical Engineering Dissertations. 86.
https://mavmatrix.uta.edu/electricaleng_dissertations/86
Comments
Degree granted by The University of Texas at Arlington