Design And Performance Evaluation Of Noise Resistant Neural Networks For Analog Intelligent Edge Computing

Author

Omobayode Ifedayo Fagbohungbe, Prairie View A&M University

Date of Award

5-2022

Document Type

Dissertation - Campus Access Only

Degree Name

Doctor of Philosophy (PhD)

Degree Discipline

Electrical Engineering

Abstract

Analog hardware-implemented deep neural network (DNN) models are promising for computation in energy-constrained systems such as edge computing devices due to their high computational speed and low power consumption. However, the analog nature of the devices and the associated noise sources will cause changes to the value of the weights in the trained DNN models deployed on such devices. These changes can cause significant degradation in the performance of the DNN models during inferencing. Hence, there is a need to design robust DNN models that suer little or no degradation in performance due to noise associated with analog hardware. In this dissertation, a systematic evaluation of the inference performance of trained popular DNN models for image classification deployed on analog devices has been carried out, where additive white Gaussian noise has been added to the weights of the trained models during inference. It is observed that deeper models and models with more redundancy in design, such as VGG, are more robust to the noise in general. In addition, it is also observed that the robustness is also affected by the design philosophy of the model, the detailed structure of the model, the exact machine learning task, as well as the datasets. To further study the DNN model design and hyperparameter choices on the noise-resistant property of DNN models, the impact of batch normalization (BatchNorm) and learning rate are examined. Specically, a comparison study of the impact of L1 BatchNorm, TopK BatchNorm, and L2 BatchNorm on the noise-resistant property of DNN has been performed. The result showed that L1 BatchNorm and TopK BatchNorm has better noise-resistant property than that of L2 BatchNorm, in addition to their lower computational overhead. Furthermore, it is observed that L2 BatchNorm negatively impacts the noise-resistant property of DNN models, especially with increased number of L2 BatchNorm in the model architecture. In order to mitigate this, a novel training philosophy that maintains the delicate balance between superior model inference performance and model noise-resistant property has been proposed, particularly in cases where model training becomes unstable without L2 BatchNorm. The impact of the learning rate, a very critical hyperparameter, on the noise-resistant property of DNN is also studied. The results show that there exists a range of learning rate values that maintain a good balance between model inference performance and model noise-resistant property. Furthermore, this range of values diers from model to model and task to task. A theoretical explanation is also provided based on mathematical justification for the impact of these design choices on the noise-resistant property of DNN models.

Committee Chair/Advisor

Lijun Qian

Committee Member

John Fuller

Committee Member

Xiangfang Li

Committee Member

Xishuang Dong

Committee Member

Nick Duffield

Publisher

Prairie View A&M University

Rights

© 2021 Prairie View A & M University

Date of Digitization

11/13/2024

Contributing Institution

John B Coleman Library

City of Publication

Prairie View

MIME Type

Application/PDF

Recommended Citation

Fagbohungbe, O. I. (2022). Design And Performance Evaluation Of Noise Resistant Neural Networks For Analog Intelligent Edge Computing. Retrieved from https://digitalcommons.pvamu.edu/pvamu-dissertations/88