Expected Ratio of Relevant Units: A Measure for Structured Information
Retrieval
Source:
INitiative for the Evaluation of XML Retrieval (INEX). Proceedings
of the Second INEX Workshop, LIP6, Université Paris 6, Dagstuhl, France (2003)
Keywords:
piwowarski, prum
Abstract:
Since the 60's, evaluation has been a key problem for Information
Retrieval (IR) systems and has been extensively discussed in the
IR community. New IR paradigms, like Structured Information Retrieval
(SIR), make classical evaluation measures inappropriate. A few tentative
extensions to these measures has been proposed but are also inadequate.
We do propose in this paper a new measure which is a generalisation
of recall. This measure takes into account the specificity of SIR,
when elements to be retrieved are linked by structural relationships.
We show an instantiation of this measure on the INEX database and
present experiments to show how well it is adapted to SIR evaluation.
