AmericasNLP 2024 Shared Task 2: Creation of Educational Materials for Indigenous Languages

What?

The AmericasNLP 2024 shared task on the creation of educational materials for Indigenous languages is a competition aimed at encouraging the development of natural language processing systems (NLP) to help with the teaching and diffusion of Indigenous languages of the Americas. Participants will build systems that can automatically create exercises by converting a base sentence into another sentence that’s changed with regards to one specific property (such as negation or tense).

Why?

Many of the Indigenous languages of the Americas are vulnerable or endangered. This means that, depending on the language, no or only a few children are learning them and, generally, they are only spoken by a few small groups of people. Because of this, these languages are at a high risk of becoming extinct in the near future. Many communities are carrying out revitalization efforts, including teaching their languages to their community members. Creating materials to teach these languages is an urgent priority, but this process is expensive and time consuming. NLP presents an opportunity to help with these efforts.
In addition to being endangered, most Indigenous languages of the Americas are so-called low-resource languages: the data needed to train any NLP systems, let alone deep learning-based systems, is severely limited. This means that many approaches used for high-resource languages, such as English and Chinese, are not directly applicable or perform poorly. Finally, many Indigenous languages exhibit linguistic properties uncommon among languages frequently studied in NLP. This constitutes an additional difficulty. The goal of AmericasNLP is to motivate researchers to take on the challenge of developing systems for these Indigenous languages.

How?

AmericasNLP invites the submission of results obtained by systems built for the creation of educational materials for Indigenous languages. Participants can use the training and development data we provide and there are no limits on what additional resources participants may use. If participants want to leverage additional data to improve their systems, that's great! If they want to use pretrained models, that's great, too! The only limitation is that we ask participants to not create the test outputs manually or train on the development or test sets.
In this shared task, participants will be given a dataset with base sentences. The dataset will also contain an indication of the change we expect systems to make to each base sentence. Systems will transform the base sentence into a target sentence according to the indicated change.
Base sentence: Ye' shka' (Bribri for "I walked")
Expected change: Polarity: Negative
Target sentence: Ye' kë̀ shkàne̠ (Bribri for “I didn't walk")
The main metric of the shared task is accuracy. Participants can enter the competition for as many languages as they like, and systems for every language will be evaluated separately, in addition to the overall average score, which will be used to determine the shared task’s winner. We provide an evaluation script and a baseline system to help participants get started quickly.

System Submission

Please send all your system outputs to americas.nlp.workshop@gmail.com. The subject of your email should be "AmericasNLP2024_SharedTask2; Shared Task Submission; <TEAM NAME>". The content of your submission email should be as follows:

Please attach all output files to your email as a single zip file, named after your team, e.g., "TheTranslators.zip". Within that zip file, the individual files should be named "<LANGUAGE_CODE>.results.<VERSION>". The language code should be the same as used in the corresponding training set names. The version number is in case you want to submit the outputs of multiple systems; it should be a single-digit (please don't submit more than 9 options per language!). Each output file should contain one sentence per line in the style of the test input file, but with an additional column (similar to the train and dev sets).

Which languages?

The following language pairs are featured in the shared task:

Important Dates

All deadlines are 11:59 pm UTC -12h (AoE).

Organizers

Manuel Mager, Pavel Denisov, Silvia Fernandez Sabido, Samuel Canul Yah, Alejandro Molina-Villegas, Lorena Hau Ucán, Arturo Oncevay, Rolando Coto-Solano, Luis Chiruzzo, Marvin Agüero-Torales, Aldo Alvarez, Katharina von der Wense

Contact: americas.nlp.workshop@gmail.com
Design: Rebeca Guerrero and Manuel Mager