Forecasting Scenarios from the Use of AI in Diplomacy

Commentary by Mucktarr Darboe MY

Scenario 1

Tess Baker, a diplomat from the Netherlands, is tasked with leading a multilateral negotiation aimed at finalizing a trade agreement between her country and Greece. During the negotiation, a large language model (LLM) is used to translate the discussion in real time. Due to several translation errors and miscategorizations caused by the system, confusion regarding the stipulations of the agreement spreads in the room. This leads to further debate and shakes stakeholders’ confidence in the process.

Drivers:

1. Complex language and terminology
2. Contextual and sociocultural sensitivities
3. Technological limitations (real-time system constraints)
4. Negotiation text ambiguity
5. Lack of feedback loops

Comments:

Tess Baker’s scenario highlights the challenges that can arise when using LLMs for real-time translation in multilateral negotiations. Complex language and terminology, context and cultural nuances, and the pressure of real-time constraints and system limitations are all barriers that contribute to potential errors and misunderstandings. Additionally, ambiguity in negotiation text and the lack of feedback loops for translation improvement has further complicated the process, leading to confusion among stakeholders. Despite this, with careful consideration of these barriers and proactive measures, such as using trained translators and providing contextual information to the translation system, the risks associated with real-time translation using LLMs can be mitigated.

Confidence Score: 0.75

Scenario 2

Selam Hailu is a young translator working at Canada’s diplomatic mission in Ethiopia. With a background in software engineering, he oversees the rollout of a project which uses LLMs for daily translation of thousands of documents of Ethiopian media, news, and intelligence for Canadian diplomats’ daily briefs. In testing, the system achieves 99.5 percent translation accuracy. Hailu’s team members, who are proficient in Amharic, Oromo, French, and English, then manually review outputs and correct the errors they find. According to a three-month study by an external audit firm, the new workflow frees up 30 percent more time and resources for Hailu’s team and increases overall mission performance.

Drivers:

1. Background in software engineering
2. Manual review by proficient team members
3. External audit firm study
4. Use of LLMs for daily translation
5. Increased efficiency and resource allocation

Comments:

Hailu’s software engineering background and expertise enables him to oversee the LLM project effectively, ensuring its seamless integration into the mission’s workflow. The manual review by his proficient team members adds a critical layer of quality control, augmenting the accuracy of translations. The practice of external audit firm study provides objective evidence of the project’s success, boosting the confidence of stakeholders. The decision to use LLMs for daily translation significantly improves efficiency and accuracy, as demonstrated by the rate of high translation accuracy achieved. This, coupled with the manual review process, results in a 30 percent increase in time and resources for Hailu’s team, leading to overall improvements in mission performance.

Confidence Score: 0.85