NEW METRICS


Is WER really what we need?

  • For isolated word systems makes sense

  • Not effective for spontaneous speech or news broadcasts

  • Named-Entity scoring is used in IE systems


Suggested Methodology - MITRE

  • MUC Scoring + Word Alignment based scoring

  • Word Alignment

  • Matching of Named Entities

  • Comparison of Named Entity Alignments

    • Type of NE

    • Extent of NE - tolerance allowed

    • Content - closest to WER