ORCID Identifier(s)

0000-0002-1381-6581

Graduation Semester and Year

2018

Language

English

Document Type

Thesis

Degree Name

Master of Science in Computer Science

Department

Computer Science and Engineering

First Advisor

Vassilis Athitsos

Abstract

Sign languages are used by deaf people for communication. In sign languages, humans use hand gestures, body, facial expressions and movements to convey meaning. Humans can easily learn and understand sign languages, but automatic sign language recognition for machines is a challenging task. Using recent advances in the field of deep-learning, we introduce a fully automated deep-learning architecture for isolated sign language recognition. Our architecture tries to address three problems: 1) Satisfactory accuracy with limited data samples 2) Reducing chances of over-fitting when the data is limited 3) Automating recognition of isolated signs. Our architecture uses deep convolutional encoder-decoder architecture for capturing spatial information and LSTM architecture for capturing temporal information. With a vocabulary of 14 one-handed signs chosen from LSA64 Dataset, our architecture achieves an accuracy of 96.02% for top 3 predictions in signer dependent settings and an accuracy of 77.85% for top 3 predictions in signer independent settings.

Keywords

DeepSign, Sign language recognition, Neural networks, Deep learning, LSTM network, Unidirectional LSTM, Bidirectional LSTM, Encoder-decoder, Convolutional neural networks

Disciplines

Computer Sciences | Physical Sciences and Mathematics

Comments

Degree granted by The University of Texas at Arlington

Download

Share

COinS