Enhancing Prediction Reliability Of Deep Learning By Data Confidence For Recommendation Systems: A Case Study On Named Entity Recognition

Author

Prianka Banik, Prairie View A&M University

Date of Award

5-2023

Document Type

Thesis

Degree Name

Master of Science

Degree Discipline

Computer Science

Abstract

Reliability is crucial for industrial recommendation systems. Recent advancement in deep neural networks has greatly improved the performance of modern recommendation systems. However, there is a lack of research on estimating how reliable such recommendation systems are in practical scenarios. Due to the blackbox nature of the deep learning-based systems, many times additional labor has to be involved to examine the prediction accuracy manually, which is costly and time-consuming. To address the problem, we propose a novel approach to estimate the model confidence for a deep learning-based recommendation system. Our approach utilized data statistics to improve the traditional model confidence estimation and maintain the model’s high performance. We further proposed a new evaluation metric to properly compare different prediction confidence estimation approaches. Experimental results showed that the external data statistics could effectively improve the prediction reliability by increasing confidence score, which will lead to significant reduction of the time and labors on the system’s prediction result examination.

Index Terms—Prediction Reliability, Recommendation Systems, Deep Learning, Data Confidence, Named Entity Recognition

Committee Chair/Advisor

Lin Li

Committee Member

Xishuang Dong

Committee Member

Lijun Qian

Committee Member

Ahmed Ahmed

Committee Member

Kiranmai Bellam

Publisher

Prairie View A&M University

Rights

Date of Digitization

6/08/2023

Contributing Institution

John B Coleman Library

City of Publication

Prairie View

MIME Type

Application/PDF

Recommended Citation

Banik, P. (2023). Enhancing Prediction Reliability Of Deep Learning By Data Confidence For Recommendation Systems: A Case Study On Named Entity Recognition. Retrieved from https://digitalcommons.pvamu.edu/pvamu-theses/1517