Date of Award


Document Type


Degree Name

Master of Science

Degree Discipline

Computer Science


Reliability is crucial for industrial recommendation systems. Recent advancement in deep neural networks has greatly improved the performance of modern recommendation systems. However, there is a lack of research on estimating how reliable such recommendation systems are in practical scenarios. Due to the blackbox nature of the deep learning-based systems, many times additional labor has to be involved to examine the prediction accuracy manually, which is costly and time-consuming. To address the problem, we propose a novel approach to estimate the model confidence for a deep learning-based recommendation system. Our approach utilized data statistics to improve the traditional model confidence estimation and maintain the model’s high performance. We further proposed a new evaluation metric to properly compare different prediction confidence estimation approaches. Experimental results showed that the external data statistics could effectively improve the prediction reliability by increasing confidence score, which will lead to significant reduction of the time and labors on the system’s prediction result examination.

Index Terms—Prediction Reliability, Recommendation Systems, Deep Learning, Data Confidence, Named Entity Recognition

Committee Chair/Advisor

Lin Li

Committee Member

Xishuang Dong

Committee Member

Lijun Qian

Committee Member

Ahmed Ahmed

Committee Member

Kiranmai Bellam


Prairie View A&M University


© 2021 Prairie View A & M University

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Date of Digitization


Contributing Institution

John B Coleman Library

City of Publication

Prairie View





To view the content in your browser, please download Adobe Reader or, alternately,
you may Download the file to your hard drive.

NOTE: The latest versions of Adobe Reader do not support viewing PDF files within Firefox on Mac OS and if you are using a modern (Intel) Mac, there is no official plugin for viewing PDF files within the browser window.