Local and frame-synchronous confidence measures for automatic speech recognition
Jean-Paul Haton (Professor) and Odile Mella (Associate Professor)
October 9th 2007
In automatic speech recognition, confidence measures aim at estimating the confidence we can give to a result (phone, word, sentence) provided by the speech recognition engine; for example, the contribution of the confidence measure allows to highlight the misrecognized or out-of-vocabulary words.
In this thesis, we propose several confidence measures which are able to provide this estimation for applications using large vocabulary and on-the-fly recognition, as keyword indexation, broadcast news transcription, and live teaching class transcription for hard of hearing children.
In this framework, we have defined two types of confidence measures. The first, based on likelihood ratio, are frame-synchronous measures which can be computed simultaneously with the recognition process of the sentence. The second ones are based on an estimation of the posterior probability limited to a local neighborhood of the considered word, and need only a short delay before being computed on the sub word graph extracted from the recognition process.
These measures were assessed and compared to a state-of-the-art one, which is also based on posterior probability but which requires the recognition of the whole sentence. Two evaluations were performed on a real broadcast news corpus provided by the ESTER campaign. The first one used the Equal Error Rate criterion in an automatic transcription task. The second evaluation was performed in a keyword spotting task. We achieved performance close to our reference measure with our local measures and a delay of less than one second.
We also integrated one of our frame-synchronous measures in the decoding process of the recognition engine in order to improve the solution provided by the system and then to decrease the word error rate. We achieved to decrease the word error rate of around 1%.
Moreover, one of our confidence measure acheived to improve comprehension of hard of hearing children.