SPOTLIGHT ON… PIERRETTE BOUILLON

Pierrette Bouillon is Dean of the Faculty of Translation and Interpreting (FTI) after several years as Vice-Dean. She has been a professor at the FTI since 2007 and is also head of the Department of Translation Technology. She remains highly research-active, collaborating on numerous Swiss and European projects exploring the social benefits of machine translation. BabelDr was one such project in collaboration with the Geneva University Hospitals (Hôpitaux universitaires de Genève, HUG). Since 2021, she has been working on the subsidiary project PROPICTO, which she agreed to talk about for the e-bulletin.

Lisez cet entretien en français

Could you give a brief outline of the BabelDr project?

The BabelDr project was launched in 2017 as the result of a collaboration between the FTI's Translation Technology department and the accident and emergency unit at HUG. The FTI has made accessibility a focus for research projects for many years, particularly in the context of medical communication. At HUG, 52% of patients are foreign and 12% do not speak any French at all. To mitigate the problem, we set up BabelDr, a medical translation software programme which translates a number of set phrases into ten or so languages, including Tigrinya (widely spoken in Eritrea), Arabic, Spanish, and Farsi. The system was also adapted for use by d/Deaf and hard-of-hearing patients, with a version offering sign-language video translation. What makes BabelDr special is that it lets users ask questions freely in their own words for the system to translate into the closest human-translated set phrase, guaranteeing accuracy. The software is still in use at the HUG accident and emergency unit when other solutions are unavailable. Many studies have shown that in this context, automatic interpreting systems are not the most appropriate. The project let us build corpora and conduct several studies on medical communication, evaluating the perception of machine translation in this context, and comparing various methods for the task.

Can you tell us about PROPICTO?

PROPICTO (PROjection de la parole vers des PICTOgrammes / Projecting words to pictograms) arose from a collaboration between the FTI Translation Technology department and GETALP (Groupe d’Étude en Traduction Automatique / Traitement Automatisé des Langues et de la Parole) at the University of Grenoble computer laboratory. The four-year joint French-Swiss project was launched in 2021 with funding from the Fonds national suisse de la recherche scientifique (FNS) and the Agence nationale de la recherche française (ANF). The aim is to establish word-to-pictogram translation systems using French as the entry language, to improve access to communication for patients who do not speak French and / or who have cognitive difficulties. The tool can also be used by the elderly and people with low literacy skills. It tackles a wide range of social needs in the fields of disability, facilitating communication for people with cognitive difficulties, and medicine, easing communication between medical staff and patients who don't share a common language. It also meets the requirements of Switzerland's 2002 Federal law on eliminating inequalities for people with disabilities, the UN convention on the rights of persons with disabilities, ratified in Switzerland in 2014, and France's laws of 2 January 2002 and 11 February 2005 enshrining equal rights for people with disabilities.

What are the project's aims?

Augmentative and alternative communication (AAC) systems are crucial for people with disabilities. AAC uses signs and communication boards with symbols and computer systems that let people transcribe their message in precise detail. However, such systems can be long to learn and difficult to use. To solve this problem, the project starts from the hypothesis that systems translating words into pictograms can help people who use AAC. Pictograms are simple graphic signs often used in AAC because they improve understanding of the message being transmitted. In the medical field, particularly in emergency situations, it is vital to bring down barriers to communication. The problems they cause can impact the quality of care and patient health and safety. PROPICTO sets out to explore solutions building on machine translation for use in AAC systems with a view to improving communication between medical staff and patients.

How did you go about it?

Developing a tool like this calls for extended research in several areas of natural speech and language processing (NSLP) – voice recognition, automatic simplification, disambiguating and extracting concepts for use in pictograms, and pictogram translation.

The first module involved a word recognition system that takes a recorded audio segment and generates the associated transcription as text. The second module then simplifies that text and extracts concepts to turn into pictograms. The third stage then takes the simplified, disambiguated output as the starting point to generate the actual pictograms.

Our final challenge is building a pipeline system that can automatically translate the spoken word into pictograms. We have experimented with various potential approaches and built demonstrators in a range of general and specialist fields. The PictoDr system for use in medical contexts uses neural technology to extract given medical concepts using the UMLS ontology and then produce the corresponding pictograms.

What have been your biggest challenges?

The project has had to overcome three major hurdles – the difficulty of depicting medical pictograms, the lack of non-copyright pictograms, especially in the medical field, and a shortage of parallel corpora connecting expressions with the corresponding corpora, which we need to train the translation models.

What results do you expect?

We will begin evaluating the tool with target groups in the course of 2024. PROPICTO will make the entire set of project resources available to the scientific community, including the corpora of questions translated into pictograms, the database connecting pictograms and their semantic meaning, and the various pictogram translation applications. Our publications, resources and applications are all available on the project website (in French).