AmericasNLP 2023 Shared Task on Machine Translation into Indigenous Languages
- Mailing list for AmericasNLP 2023 shared task participants
- AmericasNLP 2023 shared task GitHub (data+evaluation script)
- Registration form
What?
The AmericasNLP 2023 Shared Task on Machine Translation into Indigenous Languages is a competition aimed at encouraging the development of machine translation (MT) systems for Indigenous languages of the Americas. Participants will build systems that translate between Spanish and an Indigenous language.Why?
Many of the Indigenous languages of the Americas are so-called low-resource languages: parallel data with other languages as needed to train MT systems is limited. This means that many approaches designed for translating between high-resource languages, such as English and Chinese, are not directly applicable or perform poorly. Additionally, many Indigenous languages exhibit linguistic properties uncommon among languages frequently studied in natural language processing (NLP). For instance, many are polysynthetic. This constitutes an additional difficulty. The goal of AmericasNLP is to motivate researchers to take on the challenge of developing MT systems for Indigenous languages.How?
AmericasNLP invites the submission of MT results obtained by systems built for Indigenous languages. Participants can use the training and development data we provide, but there are no limits on what participants can use. If participants want to translate additional data to improve their systems, that's great! If they want to use pretrained models, that's great, too! The only limitation is that we ask participants to not have the test input translated by hand or train on the development or test sets.The main metric of the shared task is ChrF++ (Popović, 2017). Participants can enter the competition with as many language pairs as they like, and systems for every language pair will be evaluated separately.
We provide an evaluation script and a baseline MT system to help participants getting started quickly. If you are interested in this shared task, please register here.
System Submission
Please send all your system outputs to americas.nlp.workshop@gmail.com. The subject of your email should be "AmericasNLP2023; Shared Task Submission; <TEAM NAME>". The content of your submission email should be as follows:- Line 1: Team name
- Line 2: Names of all team members
- Line 3: Language codes for all languages you are sending submissions for in order of your choice (we will use that to double-check that we got all files you intended to send)
- [optional] Line 4: A link to a GitHub repository with code that can be used to reproduce your results. This is not required in order to participate in the shared task, but we strongly encourage it.
Please attach all output files to your email as a single zip file, named after your team, e.g., "TheTranslators.zip". Within that zip file, the individual files should be named "<LANGUAGE_CODE>.results.<VERSION>". The language code should be the same as used in the corresponding training set names. The version number is in case you want to submit the outputs of multiple systems; it should be a single-digit (please don't submit more than 9 options per language!). Each output file should contain one sentence per line. Sentences should not be tokenized.
Which languages?
The following language pairs are featured in the AmericasNLP 2023 shared task:- Hñähñu–Spanish
- Wixarika–Spanish
- Nahuatl–Spanish
- Guaraní–Spanish
- Bribri–Spanish
- Rarámuri–Spanish
- Quechua–Spanish
- Aymara–Spanish
- Shipibo-Konibo–Spanish
- Asháninka–Spanish
- Chatino–Spanish
Important Dates
- Release of initial languages and evaluation scipt: March 16, 2023
- Release of baseline system and baseline results: March 20, 2023
- Release of surprise language data: April 21, 2023
- Submission of translations (shared task deadline):
May 07, 2023May 09, 2023 - Announcements of results:
May 09, 2023May 10, 2023 - Submission of system description papers: May 16, 2023 (submission portal)
- Notification of acceptance: May 20, 2023
- Camera-ready papers due: May 26, 2023
- Workshop: July 14, 2023
Organizers
Manuel Mager, Arturo Oncevay, Abteen Ebrahimi, Enora Rice, Cynthia Montaño, John Ortega, Shruti Rijhwani, Ivan Vladimir Meza Ruiz, Alexis Palmer, Katharina KannContact: americas.nlp.workshop@gmail.com
