An In-depth Interactive and Visualized Platform for Evaluating and Analyzing MRC Models

Abstract

Machine Reading Comprehension (MRC) has made leaps and bounds when focusing on answering questions. However, since the ex- isting accuracy-based evaluation metrics are agnostic to the nu- ances of neural networks, the true understanding and inferenc- ing abilities of MRC models remain largely unknown. To address the above limitations, InDepth-Eva-MRC, an interactive and vi- sualized platform, is proposed to provide analysis from cognitive fine-grained for MRC models. Concretely, the platform makes post- hoc systems to explain the behavior of MRC models. On the one hand, it analyzes the linguistic bias via performances with different linguistic properties. On the other hand, it performs skill-based analysis methods based on the modified test samples and semi- automatically generated test samples. Furthermore, through its detailed and interactive visualizations, the platform offers in-depth results analysis and model comparison from cognitive fine-grained. A screencast video and additional external material are available on https://github.com/thuiar/InDepth-Eva-MRC.