Providing Consistent and Exhaustive Relevance Assessments for XML
Retrieval Evaluation
Source:
Proceedings of the Thirteenth Conference on Information and Knowledge
Management (CIKM 2004), Washington D.C., U.S.A. (2004)
Keywords:
selection, piwowarski
Abstract:
Comparing retrieval approaches requires test collections, which consist
of documents, queries and relevance assessments. Obtaining consistent
and exhaustive relevance assessments is crucial for the appropriate
comparison of retrieval approaches. Whereas the evaluation methodology
for flat text retrieval approaches is well established, the evaluation
of XML retrieval approaches is a research issue. This is because
XML documents are composed of nested components, which cannot be
considered as independent in terms of relevance. This paper describes
the methodology adopted in INEX (the INitiative for the Evaluation
of XML Retrieval) to ensure consistent and exhaustive relevance assessments.
