Proceedings of The 14th International Conference on Educational Data Mining

Behavioral Testing of deep neural knowledge tracing models

by Minsam Kim, Yugeun Shim, Seewoo Lee, Hyunbin Loh, Juneyoung Park

Recently, in the task of Knowledge Tracing(KT), Deep Neural Networks (DNN) showed superb performance over classical methods on multiple dataset benchmarks. While most Deep Learning based Knowledge Tracing (DLKT) models are optimized for general objective metrics such as accuracy or AUC on benchmark data, proper deployment of the service in a real-world environment requires additional qualities.


Deep Learning based Knowledge Tracing (DLKT) uses Deep Neural Networks (DNN) to perform Knowledge Tracing(KT) which predicts how a student will respond to an unsolved question.
In this context, we adopt the idea of behavioral testing from software engineering and define desirable KT model behaviors. We propose an analysis framework to diagnose the KT model’s behavioral quality. This ensures that the model meets expectations and resigns from abnormal behaviors. Having test-run the framework on various datasets, the results highlight the impact of dataset size and model architecture upon the model’s behavioral quality.


Loading PDF…

Get your score in 40 min!

Just do 1/4 of a full test and get actionable insights.

R.test is an AI-powered diagnostic test platform that evaluates student’s test readiness. Our mission is to get rid of inefficiency and inequality from test prep industry by making assessments more adaptive, accessible, and reliable.

â“’ 2023 Riiid, Inc. All Rights Reserved

521, Teheran-ro, Gangnam-gu, Seoul, Korea

College Board® is a trademark registered by the College Board, which is not affiliated with, and does not endorse, this website.

Neither Riiid, Inc. or R.test is affiliated with College Board® and do not have access to College Board's proprietary data.

ACT® is a registered trademark of ACT, inc. This website is not endorsed or approved by ACT, inc.

Neither Riiid, Inc. or R.test is affiliated with ACT, Inc. and do not have access to ACT’s proprietary data.