Apply to become a certified provider
ARCADE II Evaluation Package
The ARCADE II Evaluation Package was produced within the French national project ARCADE II (Evaluation of parallel text alignment systems), as part of the Technolangue programme funded by the French Ministry of Research and New Technologies (MRNT). The ARCADE II project enabled to carry out a campaign for the evaluation in the field of multilingual alignment, with even more ambitious objectives than in the ARCADE I project (within the AUPELF campaigns (Actions de recherche Concertées, 1996-1999), by including a finer alignment and by coping with many other languages (extension to French-distant languages). Thus, ARCADE II is not only an extension of ARCADE I, but also presents innovative and exploratory aspects, for instance by integrating French-distant languages, such as Arabic, Russian, Chinese, etc. This package includes the material that was used for the ARCADE II evaluation campaign. It includes resources, protocols, scoring tools, results of the campaign, etc., that were used or produced during the campaign. The aim of these evaluation packages is to enable external players to evaluate their own system. The campaign is distributed over two actions: 1) Sentence alignment: it consists in evaluating the alignment of French language with Latin-script languages on one side, and with non Latin-script languages on the other side. 2) Translation of named entities: it consists in identifying in the parallel Arabic corpus the translation corresponding to the named entities phrases annotated in the French corpus. The ARCADE II evaluation package contains the following data and tools: 1) The JOC Corpus (Official Journal of the European Community) with Latin-script languages (English, French, German, Italian, Spanish) contains 1 million words per language (5 million words in all). The texts are aligned at the sentence level and produced in XML and UTF-8 format. 2) The MD Corpus (Le Monde Diplomatique) with non-Latin-script languages (Arabic, Chinese, Greek, Japanese, Persian, Russian,) contains manually-aligned texts at the sentence level, encoded in XML and UTF-8. The size of the different parts varies according to the language pair. A subset for the Arabic-French part was manually annotated with named entities. The size in words was calculated in the French part. The calculation is different depending on the language (such as for Arabic where many clitics are agglutinated, which reduces the number of words), and sometimes impossible (such as for Chinese, where there is no graphical separation between words): <table border="0" width="100%" cellspacing="0" cellpadding="2" class="infoBoxContents"> <tr align=center><td></td><td>Arabic-French</td><td>Chinese-Fr</td><td>Greek-Fr</td><td>Japanese-Fr</td><td>Persian-Fr</td><td>Russian-Fr</td></tr> <tr align=center><td align=left><strong>Number of articles</strong></td><td>150 x 2</td><td>59 x 2</td><td>50 x 2</td><td>52 x 2</td><td>53 x 2</td><td>50 x 2</td></tr> <tr align=center><td align=left><strong>Number of words in French</strong></td><td>316,000</td><td>100,000</td><td>90,000</td><td>100,000 </td><td>108,000</td><td>91,000</td></tr> </table> A description of the project is available at the following address: http://www.technolangue.net/article.php3?id_article=201 (in French language)
ELRA