Apply to become a certified provider
CLEF Question Answering Test Suites (2003-2008) – Evaluation Package
The Cross-Language Evaluation Forum (CLEF) promotes R&D in multilingual information access (MLIA) by (i) developing an infrastructure for the testing, tuning and evaluation of information retrieval systems operating on European languages in both monolingual and cross-language contexts, and (ii) creating test-suites of reusable data which can be employed by system developers for benchmarking purposes. The CLEF Question Answering Suites (2003-2008) contain the data used for the Question Answering (QA) track of the CLEF campaigns carried out from 2003 to 2008. This track tested the performance of monolingual, bilingual and multilingual Question Answering systems on multilingual collections of news documents. The CLEF Test Suite is composed of: • News Data Collections • Questions • Guidelines • Relevance assessments • Official campaign results • Working notes papers The News Data Collections consist of the following datasets: • Bulgarian o Sega 2002 (33,356 documents, 120 Mb) o Standart 2002 (35,839 documents, 93 Mb) o Novinar 2002 (18,086 documents, 48 Mb) • Dutch o NRC Handelsblad 1994/95 (84,121 documents, 299 Mb) o Algemeen Dagblad 1994/95 (106,483 documents, 241 Mb) • English o Glasgow Herald 1995 (56,472 documents, 154 Mb) o Los Angeles Times 1994 (113,005 documents, 425 Mb) • Finnish o Aamulehti late-1994/95 (55,344 documents, 137 Mb) • French o Le Monde 1994 (44,013 documents, 157 Mb) o Le Monde 1995 (47,646 documents, 156 Mb) o SDA French 1994 (43,178 documents, 86 Mb) o SDA French 1995 (42,615 documents, 88 Mb) • German o Frankfurter Rundschau 1994 (139,715 documents, 320 Mb) o Der Spiegel 1994/95 (13,979 documents, 63 Mb) o SDA German 1994 (71,677 documents, 144 Mb) o SDA German 1995 (69,438 documents, 141 Mb) • Italian o La Stampa 1994 (58,051 documents, 193 Mb) o SDA Italian 1994 (50,527 documents, 85 Mb) o SDA Italian 1995 (48,980 documents, 85 Mb) • Portuguese o Público 1994 (51,751 documents, 164 Mb) o Público 1995 (55,070 documents, 176 Mb) o Folha de São Paulo 1994 (51,875 documents, 108 Mb) o Folha de São Paulo 1995 (52,038 documents, 116 Mb) • Romanian o Wikipedia • Spanish o EFE 1994 (215,738 documents, 509 Mb) o EFE 1995 (238,307 documents, 577 Mb) The full package is stored on 1 DVD.
ELRA