Cheap and easy entity evaluation

Authors: Ben Hachey, Joel Nothman, Will Radford - Annual Meeting of the Association for Computational Linguistics, ACL 2014 cited by 24

The AIDA-YAGO dataset is a popular tar-get for whole-document entity recogni-tion and disambiguation, despite lacking a shared evaluation tool. We review eval-uation regimens in the literature while comparing the output of three approaches, and identify research opportunities. This utilises our open, accessible evaluation tool. We exemplify a new paradigm of distributed, shared evaluation, in which evaluation software and standardised, ver-sioned system outputs are provided online. 1