2023Activity reportProject-TeamALMANACH

3.1.1 Research axis 1

Our first objective is to stay at a state-of-the-art level in key NLP tasks such as shallow processing, part-of-speech tagging and (syntactic) parsing, which are core expertise domains of ALMAnaCH members. This will also require us to improve the generation of semantic representations (semantic parsing), and to begin to explore tasks such as machine translation, which now relies on neural architectures also used for some of the above-mentioned tasks. Given the generalisation of neural models in NLP, we will also be involved in better understanding how such models work and what they learn, something that is directly related to the investigation of language variation (Research axis 2). We will also work on the integration of both linguistic and non-linguistic contextual information to improve automatic linguistic analysis. This is an emerging and promising line of research in NLP. We will have to identify, model and take advantage of each type of contextual information available. Addressing these issues will enable the development of new lines of research related to conversational content. Applications include improved information and knowledge extraction algorithms. We will especially focus on challenging datasets such as domain-specific texts (e.g. financial, legal) as well as historical documents, in the larger context of the development of digital humanities. We currently also explore the even more challenging new direction of a cognitively inspired NLP, in order to tackle the possibility to enrich the architecture of state-of-the-art algorithms, such as RNNGs, based on human neuroimaging-driven data.

3.1.2 Research axis 2

Language variation must be better understood and modelled in all its forms. In this regard, we will put a strong emphasis on four types of language variation and their mutual interaction: sociolinguistic variation in synchrony (including non-canonical spelling and syntax in user-generated content), complexity-based variation in relation to language-related disabilities, and diachronic variation (computational exploration of language change and language history, with a focus on Old to all forms of Modern French, as well as Indo-European languages in general). In addition, the noise introduced by Optical Character Recognition and Handwritten Text Recognition systems, especially in the context of historical documents, bears some similarities to that of non-canonical input in user-generated content (e.g. erroneous characters). This noise constitutes a more transverse kind of variation stemming from the way language is graphically encoded, which we call language-encoding variation. Other types of language variation will also become important research topics for ALMAnaCH in the future. This includes dialectal variation (e.g. work on Arabic varieties, something on which we have already started working, producing the first annotated data set on Maghrebi Arabizi, the Arabic variants used on social media by people from North-African countries, written using a non-fixed Latin-script transcription) as well as the study and exploitation of paraphrases in a broader context than the above-mentioned complexity-based variation.

Both research axes above rely on the availability of language resources (corpora, lexicons), which is the focus of our third, transverse research axis.

3.1.3 Research axis 3

Language resource development (raw and annotated corpora, lexical resources) is not just a necessary preliminary step to create both evaluation datasets for NLP systems and training datasets for NLP systems based on machine learning. When dealing with datasets of interest to researchers from the humanities (e.g. large archives), it is also a goal per se and a preliminary step before making such datasets available and exploitable online. It involves a number of scientific challenges, among which (i) tackling issues related to the digitalisation of non-electronic datasets, (ii) tackling issues related to the fact that many DH-related datasets are domain-specific and/or not written in contemporary languages; (iii) the development of semi-automatic and automatic algorithms to speed up the work (e.g. automatic extraction of lexical information, low-resource learning for the development of pre-annotation algorithms, transfer methods to leverage existing tools and/or resources for other languages, etc.) and (iv) the development of formal models to represent linguistic information in the best possible way, thus requiring expertise at least in NLP and in typological and formal linguistics. Such endeavours are domains of expertise of the ALMAnaCH team, and a large part of our research activities will be dedicated to language resource development. In this regard, we aim to retain our leading role in the representation and management of lexical resource and treebank development and also to develop a complete processing line for the transcription, analysis and processing of complex documents of interest to the humanities, in particular archival documents. This research axis 3 will benefit the whole team and beyond, and will benefit from and feed the work of the other research axes.

3.2.1 Processing of natural language at all levels: morphology, syntax, semantics

Our expertise in NLP is the outcome of more than 10 years in developing new models of analysis and accurate techniques for the full processing of any kind of language input since the early days of the Atoll project-team and the rise of linguistically informed data-driven models as put forward within the Alpage project-team.

Traditionally, a full natural language process (NLP) chain is organised as a pipeline where each stage of analysis represents a traditional linguistic field (in a structuralism view) from morphological analysis to purely semantic representations. The problem is that this architecture is vulnerable to error propagation and very domain sensitive: each of these stage must be compatible at the lexical and structure levels they provide. We arguably built the best performing NLP chain for French 84, 144 and one of the best for robust multilingual parsing as shown by our results in various shared tasks over the years 141, 136, 145, 100. So we pursue our efforts on each of our components we developed: tokenisers (e.g. SxPipe), part-of-speech taggers (e.g. MElt), constituency parsers and dependency parsers (e.g. FRMG, DyALog-SR) as well as our recent neural semantic graph parsers 136.

In particular, we continue to explore the hybridisation of symbolic and statistical approaches, and extend it to neural approaches, as initiated in the context of our participation to the CoNLL 2017 multilingual parsing shared task4 and to Extrinsic Parsing Evaluation Shared Task5.

Fundamentally, we want to build tools that are less sensitive to variation, more easily configurable, and self-adapting. Our short-term goal is to explore techniques such as multi-task learning (cf. already  126) to propose a joint model of tokenisation, normalisation, morphological analysis and syntactic analysis. We also explore adversarial learning, considering the drastic variation we face in parsing user-generated content and processing historical texts, both seen as noisy input that needs to be handled at training and decoding time.

3.2.2 Integrating context in NLP systems

While those points are fundamental, therefore necessary, if we want to build the next generation of NLP tools, we need to push the envelop even further by tackling the biggest current challenge in NLP: handling the context within which a speech act is taking place.

There is indeed a strong tendency in NLP to assume that each sentence is independent from its siblings sentences as well as its context of enunciation, with the obvious objective to simplify models and reduce the complexity of predictions. While this practice is already questionable when processing full-length edited documents, it becomes clearly problematic when dealing with short sentences that are noisy, full of ellipses and external references, as commonly found in User-Generated Content (UGC).

A more expressive and context-aware structural representation of a linguistic production is required to accurately model UGC. Let us consider for instance the case for Syntax-based Machine Translation of social media content, as is carried out by the ALMAnaCH-led ANR project Parsiti (PI: DS). A Facebook post may be part of a discussion thread, which may include links to external content. Such information is required for a complete representation of the post's context, and in turn its accurate machine translation. Even for the presumably simpler task of POS tagging of dialogue sequences, the addition of context-based features (namely information about the speaker and dialogue moves) was beneficial 103. In the case of UGC, working across sentence boundaries was explored for instance, with limited success, by 82 for document-wise parsing and by 125 for POS tagging.

Taking the context into account requires new inference methods able to share information between sentences as well as new learning methods capable of finding out which information is to be made available, and where. Integrating contextual information at all steps of an NLP pipeline is among the main research questions addressed in this research strand. In the short term, we focus on morphological and syntactic disambiguation within close-world scenarios, as found in video games and domain-specific UGC. In the long term, we investigate the integration of linguistically motivated semantic information into joint learning models.

From a more general perspective, contexts may take many forms and require imagination to discern them, get useful data sets, and find ways to exploit them. A context may be a question associated with an answer, a rating associated with a comment (as provided by many web services), a thread of discussions (e-mails, social media, digital assistants, chatbots—on which see below–), but also meta data about some situation (such as discussions between gamers in relation with the state of the game) or multiple points of views (pictures and captions, movies and subtitles). Even if the relationship between a language production and its context is imprecise and indirect, it is still a valuable source of information, notwithstanding the need for less supervised machine learning techniques (cf. the use of LSTM neural networks by Google to automatically suggest replies to emails).

3.2.3 Information and knowledge extraction

The use of local contexts as discussed above is a new and promising approach. However, a more traditional notion of global context or world knowledge remains an open question and still raises difficult issues. Indeed, many aspects of language such as ambiguities and ellipsis can only be handled using world knowledge. Linked Open Data (LODs) such as DBpedia, WordNet, BabelNet, or Framebase provide such knowledge and we plan to exploit them.

However, each specialised domain (economy, law, medicine...) exhibits its own set of concepts with associated terms. This is also true of communities (e.g. on social media), and it is even possible to find communities discussing the same topics (e.g. immigration) with very distinct vocabularies. Global LODs weakly related to language may be too general and not sufficient for a specific language variant. Following and extending previous work in ALPAGE, we put an emphasis on information acquisition from corpora, including error mining techniques in parsed corpora (to detect specific usages of a word that are missing in existing resources), terminology extraction, and word clustering.

Word clustering is of specific importance. It relies on the distributional hypothesis initially formulated by Harris, which states that words occurring in similar contexts tend to be semantically close. The latest developments of these ideas (with word2vec or GloVe) have led to the embedding of words (through vectors) in low-dimensional semantic spaces. In particular, words that are typical of several communities (see above) can be embedded in a same semantic space in order to establish mappings between them. It is also possible in such spaces to study static configurations and vector shifts with respect to variables such as time, using topological theories (such as pretopology), for instance to explore shifts in meaning over time (cf. the ANR project Profiterole concerning ancient French texts) or between communities (cf. the ANR project SoSweet). It is also worth mentioning on-going work (in computational semantics) whose goal is to combine word embeddings to embed expressions, sentences, paragraphs or even documents into semantic spaces, e.g. to explore the similarity of documents at various time periods.

Besides general knowledge about a domain, it is important to detect and keep trace of more specific pieces of information when processing a document and maintaining a context, especially about (recurring) Named Entities (persons, organisations, locations...) —something that is the focus of future work in collaboration with Patrice Lopez on named entity detection in scientific texts. Through the co-supervision of a PhD funded by the LabEx EFL (see below), we are also involved in pronominal coreference resolution (finding the referent of pronouns). Finally, we plan to continue working on deeper syntactic representations (as initiated with the Deep Sequoia Treebank), thus paving the way towards deeper semantic representations. Such information is instrumental when looking for more precise and complete information about who does what, to whom, when and where in a document. These lines of research are motivated by the need to extract useful contextual information, but it is also worth noting their strong potential in industrial applications.

3.3.1 Theoretical and empirical synchronic linguistics

Permanent members involved: all

We aim to explore computational models to deal with language variation. It is important to get more insights about language in general and about the way humans apprehend it. We will do so in at least two directions, associating computational linguistics with formal and descriptive linguistics on the one hand (especially at the morphological level) and with cognitive linguistics on the other hand (especially at the syntactic level).

Recent advances in morphology rely on quantitative and computational approaches and, sometimes, on collaboration with descriptive linguists—see for instance the special issue of the Morphology journal on “computational methods for descriptive and theoretical morphology”, edited and introduced by 79. In this regard, ALMAnaCH members have taken part in the design of quantitative approaches to defining and measuring morphological complexity and to assess the internal structure of morphological systems (inflection classes, predictability of inflected forms...). Such studies provide valuable insights on these prominent questions in theoretical morphology. They also improve the linguistic relevance and the development speed of NLP-oriented lexicons, as also demonstrated by ALMAnaCH members. We shall therefore pursue these investigations, and orientate them towards their use in diachronic models (cf. Section 3.3.3).

Regarding cognitive linguistics, we have the perfect opportunity with the starting ANR-NSF project “Neuro-Computational Models of Natural Language” (NCM-NL) to go in this direction, by examining potential correlations between medical imagery applied on patients listening to a reading of “Le Petit Prince” and computation models applied on the novel. A secondary prospective benefit from the project will be information about processing evolution (by the patients) along the novel, possibly due to the use of contextual information by humans.

3.3.2 Sociolinguistic variation

Because language is central in our social interactions, it is legitimate to ask how the rise of digital content and its tight integration in our daily life has become a factor acting on language. This is even more actual as the recent rise of novel digital services opens new areas of expression, which support new linguistic behaviours. In particular, social media such as Twitter provide channels of communication through which speakers/writers use their language in ways that differ from standard written and oral forms. The result is the emergence of new language varieties.

A very similar situation exists with regard to historical texts, especially documentary texts or graffiti but even literary texts, that do not follow standardised orthography, morphology or syntax.

However, NLP tools are designed for standard forms of language and exhibit a drastic loss of accuracy when applied to social media varieties or non-standardised historical sources. To define appropriate tools, descriptions of these varieties are needed. However, to validate such descriptions, tools are also needed. We address this chicken-and-egg problem in an interdisciplinary fashion, by working both on linguistic descriptions and on the development of NLP tools. Recently, socio-demographic variables have been shown to bear a strong impact on NLP processing tools (see for instance 97 and references therein). This is why, in a first step, jointly with researchers involved in the ANR project SoSweet (ENS Lyon and Inria project-team Dante), we will study how these variables can be factored out by our models and, in a second step, how they can be accurately predicted from sources lacking these kinds of featured descriptions.

3.3.3 Diachronic variation

Language change is a type of variation pertaining to the diachronic axis. Yet any language change, whatever its nature (phonetic, syntactic...), results from a particular case of synchronic variation (competing phonetic realisations, competing syntactic constructions...). The articulation of diachronic and synchronic variation is influenced to a large extent by both language-internal factors (i.e. generalisation of context-specific facts) and/or external factors (determined by social class, register, domain, and other types of variation).

Very few computational models of language change have been developed. Simple deterministic finite-state-based phonetic evolution models have been used in different contexts. The PIElexicon project 118 uses such models to automatically generate forms attested in (classical) Indo-European languages but is based on an idiosyncratic and unacceptable reconstruction of the Proto-Indo-European language. Probabilistic finite-state models have also been used for automatic cognate detection and proto-form reconstruction, for example by 80 and 99. Such models rely on a good understanding of the phonetic evolution of the languages at hand.

In ALMAnaCH, our goal is to work on modelling phonetic, morphological and lexical diachronic evolution, with an emphasis on computational etymological research and on the computational modelling of the evolution of morphological systems (morphological grammar and morphological lexicon). These efforts will be in direct interaction with sub-strand 3b (development of lexical resources). We want to go beyond the above-mentioned purely phonetic models of language and lexicon evolution, as they fail to take into account a number of crucial dimensions, among which: (1) spelling, spelling variation and the relationship between spelling and phonetics; (2) synchronic variation (geographical, genre-related, etc.); (3) morphology, especially through intra-paradigmatic and inter-paradigmatic analogical levelling phenomena, (4) lexical creation, including via affixal derivation, back-formation processes and borrowings.

We apply our models to two main tasks. The first task, as developed for example in the context of the ANR project Profiterole, consists in predicting non-attested or non-documented words at a certain date based on attestations of older or newer stages of the same word (e.g., predicting a non-documented Middle French word based on its Vulgar Latin and Old French predecessors and its Modern French successor). Morphological models and lexical diachronic evolution models will provide independent ways to perform the same predictions, thus reinforcing our hypotheses or pointing to new challenges.

The second application task is computational etymology and proto-language reconstruction. Our lexical diachronic evolution models will be paired with semantic resources (wordnets, word embeddings, and other corpus-based statistical information). This will allow us to formally validate or suggest etymological or cognate relations between lexical entries from different languages of a same language family, provided they are all inherited. Such an approach could also be adapted to include the automatic detection of borrowings from one language to another (e.g. for studying the non-inherited layers in the Ancient Greek lexicon). In the longer term, we will investigate the feasibility of the automatic (unsupervised) acquisition of phonetic change models, especially when provided with lexical data for numerous languages from the same language family.

These lines of research will rely on etymological data sets and standards for representing etymological information (see Section 3.4.2).

Diachronic evolution also applies to syntax, and in the context of the ANR project Profiterole, we are beginning to explore more or less automatic ways of detecting these evolutions and suggest modifications, relying on fine-grained syntactic descriptions (as provided by meta-grammars), unsupervised sentence clustering (generalising previous works on error mining, cf.  127), and constraint relaxation (in meta-grammar classes). The underlying idea is that a new syntactic construction evolves from a more ancient one by small, iterative modifications, for instance by changing word order, adding or deleting functional words, etc.

3.3.4 Accessibility-related variation

Language variation does not always pertain to the textual input of NLP tools. It can also be characterised by their intended output. This is the perspective from which we investigate the issue of text simplification (for a recent survey, see for instance  142). Text simplification is an important task for improving the accessibility to information, for instance for people suffering from disabilities and for non-native speakers learning a given language 119. To this end, guidelines have been developed to help writing documents that are easier to read and understand, such as the FALC (“Facile À Lire et à Comprendre”) guidelines for French. 6

Fully automated text simplification is not suitable for producing high-quality simplified texts. Besides, the involvement of disabled people in the production of simplified texts plays an important social role. Therefore, following previous works 96, 134, our goal will be to develop tools for the computer-aided simplification of textual documents, especially administrative documents. Many of the FALC guidelines can only be linguistically expressed using complex, syntactic constraints, and the amount of available “parallel” data (aligned raw and simplified documents) is limited. We will therefore investigate hybrid techniques involving rule-based, statistical and neural approaches based on parsing results (for an example of previous parsing-based work, see  75). Lexical simplification, another aspect of text simplification 104, 120, will also be pursued. In this regard, we have already started a collaboration with Facebook's AI Research in Paris, the UNAPEI (the largest French federation of associations defending and supporting people with intellectual disabilities and their families), and the French Secretariat of State in charge of Disabled Persons.

Accessibility can also be related to the various presentation forms of a document. This is the context in which we have initiated the OPALINE project, funded by the Programme d'Investissement d'Avenir - Fonds pour la Société Numérique. The objective is for us to further develop the GROBID text-extraction suite7 in order to be able to re-publish existing books or dictionaries, available in PDF, in a format that is accessible by visually impaired persons.

3.4.1 Construction, management and automatic annotation of Text Corpora

Corpus creation and management (including automatic annotation) is often a time-consuming and technically challenging task. In many cases, it also raises scientific issues related for instance with linguistic questions (what is the elementary unit in a text?) as well as computer-science challenges (for instance when OCR or HTR are involved). It is therefore necessary to design a work-flow that makes it possible to deal with data collections, even if they are initially available as photos, scans, wikipedia dumps, etc.

These challenges are particularly relevant when dealing with ancient languages or scripts where fonts, OCR techniques, language models may be not extant or of inferior quality, as a result, among others, of the variety of writing systems and the lack of textual data. We will therefore work on improving print OCR for some of these languages, especially by moving towards joint OCR and language models. Of course, contemporary texts can be often gathered in very large volumes, as we already do within the ANR project SoSweet, resulting in different, specific issues.

ALMAnaCH pays a specific attention to the re-usability8 of all resources produced and maintained within its various projects and research activities. To this end, we will ensure maximum compatibility with available international standards for representing textual sources and their annotations. More precisely we will take the TEI (Text Encoding Initiative) guidelines as well the standards produced by ISO committee TC 37/SC 4 as essential points of reference.

From our ongoing projects in the field of Digital Humanities and emerging initiatives in this field, we observe a real need for complete but easy work-flows for exploiting corpora, starting from a set of raw documents and reaching the level where one can browse the main concepts and entities, explore their relationship, extract specific pieces of information, always with the ability to return to (fragments of) the original documents. The pieces of information extracted from the corpora also need to be represented as knowledge databases (for instance as RDF “linked data”), published and linked with other existing databases (for instance for people and locations).

The process may be seen as progressively enriching the documents with new layers of annotations produced by various NLP modules and possibly validated by users, preferably in a collaborative way. It relies on the use of clearly identified representation formats for the annotations, as advocated within ISO TC 37/SC 4 standards and the TEI guidelines, but also on the existence of well-designed collaborative interfaces for browsing, querying, visualisation, and validation. ALMAnaCH has been or is working on several of the NLP bricks needed for setting such a work-flow, and has a solid expertise in the issues related to standardisation (of documents and annotations). However, putting all these elements in a unified work-flow that is simple to deploy and configure remains to be done. In particular, work-flow and interface should maybe not be dissociated, in the sense that the work-flow should be easily piloted and configured from the interface. An option will be to identify pertinent emerging platforms in DH (such as Transkribus) and to propose collaborations to ensure that NLP modules can be easily integrated.

It should be noted that such work-flows have actually a large potential besides DH, for instance for exploiting internal documentation (for a company) or exploring existing relationships between entities.

3.4.2 Development of Lexical Resources

ALPAGE, the Inria predecessor of ALMAnaCH, has put a strong emphasis in the development of morphological, syntactic and wordnet-like semantic lexical resources for French as well as other languages (see for instance 133, 90). Such resources play a crucial role in all NLP tools, as has been proven among other tasks for POS tagging 129, 131, 145 and parsing, and some of the lexical resource development will be targeted towards the improvement of NLP tools. They will also play a central role for studying diachrony in the lexicon, for example for Ancient to Contemporary French in the context of the Profiterole project. They will also be one of the primary sources of linguistic information for augmenting language models used in OCR systems for ancient scripts, and will allow us to develop automatic annotation tools (e.g. POS taggers) for low-resourced languages (see already 146), especially ancient languages. Finally, semantic lexicons such as wordnets will play a crucial role in assessing lexical similarity and automating etymological research.

Therefore, an important effort towards the development of new morphological lexicons will be initiated, with a focus on ancient languages of interest. Following previous work by ALMAnaCH members, we will try and leverage all existing resources whenever possible such as electronic dictionaries, OCRised dictionaries, both modern and ancient 128, 101, 130, while using and developing (semi)automatic lexical information extraction techniques based on existing corpora 132, 135. A new line of research will be to integrate the diachronic axis by linking lexicons that are in diachronic relation with one another thanks to phonetic and morphological change laws (e.g. XIIth century French with XVth century French and contemporary French). Another novelty will be the integration of etymological information in these lexical resources, which requires the formalisation, the standardisation, and the extraction of etymological information from OCRised dictionaries or other electronic resources, as well as the automatic generation of candidate etymologies. These directions of research are already investigated in ALMAnaCH 101, 130.

An underlying effort for this research will be to further the development of the GROBID-dictionaries software, which provides cascading CRF (Conditional Random Fields) models for the segmentation and analysis of existing print dictionaries. The first results we have obtained have allowed us to set up specific collaborations to improve our performances in the domains of a) recent general purpose dictionaries such as the Petit Larousse (Nénufar project, funded by the DGLFLF in collaboration with the University of Montpellier), b) etymological dictionaries (in collaboration with the Berlin Brandenburg Academy of sciences) and c) patrimonial dictionaries such as the Dictionnaire Universel de Basnage (an ANR project, including a PhD thesis at ALMAnaCH, has recently started on this topic in collaboration with the University of Grenoble-Alpes and the University Sorbonne Nouvelle in Paris).

In the same way as we signalled the importance of standards for the representation of interoperable corpora and their annotations, we will keep making the best use of the existing standardisation background for the representation of our various lexical resources. There again, the TEI guidelines play a central role, and we have recently participated in the “TEI Lex 0” initiative to provide a reference subset for the “Dictionary” chapter of the guidelines. We are also responsible, as project leader, of the edition of the new part 4 of the ISO standard 24613 (LMF, Lexical Markup Framework)  123 dedicated to the definition of the TEI serialisation of the LMF model (defined in ISO 24613 part 1 `Core model', 2 `Machine Readable Dictionaries' and 3 `Etymology'). We consider that contributing to standards allows us to stabilise our knowledge and transfer our competence.

3.4.3 Development of Annotated Corpora

Along with the creation of lexical resources, ALMAnaCH is also involved in the creation of corpora either fully manually annotated (gold standard) or automatically annotated with state-of-the-art pipeline processing chains (silver standard). Annotations will either be only morphosyntactic or will cover more complex linguistic levels (constituency and/or dependency syntax, deep syntax, maybe semantics). Former members of the ALPAGE project have a renowned experience in those aspects (see for instance 140, 122, 138, 107) and will participate to the creation of valuable resources originating from the historical domain genre.

Under the auspices of the ANR Parsiti project, led by ALMAnaCH (PI: DS), we aim to explore the interaction of extra-linguistic context and speech acts. Exploiting extra-linguistics context highlights the benefits of expanding the scope of current NLP tools beyond unit boundaries. Such information can be of spatial and temporal nature, for instance. They have been shown to improve Entity Linking over social media streams 89. In our case, we decided to focus on a closed world scenario in order to study context and speech acts interaction. To do so, we are developing a multimodal data set made of live sessions of a first person shooter video game (Alien vs. Predator) where we transcribed all human players interactions and face expressions streamlined with a log of all in-game events linked to the video recording of the game session, as well as the recording of the human players themselves. The in-games events are ontologically organised and enable the modelling of the extra-linguistics context with different levels of granularity. Recorded over many games sessions, we already transcribed over 2 hours of speech that will serve as a basis for exploratory work, needed for the prototyping of our context-enhanced NLP tools. In the next step of this line of work, we will focus on enriching this data set with linguistic annotations, with an emphasis on co-references resolutions and predicate argument structures. The midterm goal is to use that data set to validate a various range of approaches when facing multimodal data in a close-world environment.

Node infrastructure:

We have access to two types of GPU node in Jean Zay:12

1. Nodes comprising 4 GPU Nvidia Tesla V100 SXM2 32GB, 192GB of RAM, and two Intel Cascade Lake 6248 processors. One Nvidia Tesla V100 card is rated at around 300W,13 while the Intel Cascade Lake processor is rated at 150W.14 For the DRAM we can use the work of 85 to estimate the total power draw of 192GB of RAM at approximately 20W. The total power draw of one Jean Zay node at peak use therefore adds up to around 1520W.
2. Nodes comprising 8 GPU Nvidia A100 SXM4 80GB, 512GB of RAM, and two AMD Milan EPYC 7543 processors. One Nvidia A100 card is rate at around 400W.15 while the AMD Milan processor is rated at 225W.16 Following 85, we estimate the total power draw of 512GB of RAM at approximately 50W. The total power draw of one A100 node at peak use therefore adds up to around 3700W.

With this information, we use the formula proposed by 143 and compute the total power required for each setting:

Where $c$ and $g$ are the number of CPUs and GPUs respectively, $p_c$ is the average power draw (in W) from all CPU sockets, $p_r$ the average power draw from all DRAM sockets, and $p_g$ the average power draw of a single GPU. We estimate the total power consumption by adding GPU, CPU and DRAM consumption, and then multiplying by the Power Usage Effectiveness (PUE), which accounts for the additional energy required to support the compute infrastructure. We use a PUE coefficient of 1.20, which is the value reported by IDRIS for the Jean Zay supercomputer. For the real time $t$ we have to divide the reported time for each Jean Zay project by 4 for V100 nodes and 8 for A100 nodes, as Jean Zay reports the computing time of each project in GPU hours and not in per-node hours. In Table 1 we report the training times in hours, as well as the total power draw (in kWh) of each Jean Zay project associated with the ALMAnaCH team during 2023.17 We use this information to compute the total power consumption (multiplying by the PUE) of each project, also reported in Table 1.

We can further estimate the CO${}_2$ emissions in kilograms of each single project by multiplying the total power consumption by the average CO${}_2$ emissions per kWh in France, which were around 50g/kWh on average for 2023.18 The total CO${}_2$ emissions in kg for one single model can therefore be computed as:

All emissions are also reported in Table 1. The total emission estimate for the team is 4,168.57kg of CO${}_2$. The carbon footprint of a single passenger on a round trip from Paris to New York (Boeing 787), flying economy, amounts to around 2300kg of CO${}_2$.19 This means that our computing emissions from Jean Zay for 2023 amount to just above one Paris-New York trip, and therefore the largest source of emissions from the team are from flights to conferences, which are a necessary part of communicating about our research to the international community.

7.1.1 KaMI-Lib

• Name:
  KaMI (Kraken Model Inspector) - Python Library
• Keywords:
  HTR, OCR, Python, Handwritten Text Recognition, Image segmentation, Library
• Functional Description:

  KaMI-lib (Kraken as Model Inspector) is a Python library for evaluating transcription models (handwritten text recognition and optical character recognition) trained either with the Kraken engine (http://kraken.re) or without it.

  It provides a single class for comparing strings (e.g. extracted from text files) and for generating scores in order to evaluate the automatic transcription’s performance. The Kraken engine is implemented in KaMI-lib in order to produce a prediction with a pre-trained transcription model and to compare it to a ground truth (in PAGE XML or XML ALTO format) associated with its image.

  KaMI-lib uses different metrics to evaluate a transcription model: the Word Error Rate (WER), the Character Error Rate (CER), and the Word Accuracy (Wacc). In addition, KaMI-lib provides the edit distances and the operations performed on the different strings. It is also possible to weigh the cost of operations in order to adjust scores.

  It is also possible to get different scores with text pre-processing functions applied to the ground truth and the prediction, such as deleting all diacritics, ponctuations, or numbers, ignoring upper case, etc. By doing so, KaMI-lib aims to give a better understanding of text features’ impacts on transcription results. This functionality also aims to make users adapt the creation of training data according to their texts’ specificites, and optimize the training process.

  Documentation is available here: https://gitlab.inria.fr/dh-projects/kami/kami-lib

• URL:
  https://­github.­com/­KaMI-tools-project/­KaMi-lib
• Publications:
  hal-03495762, hal-03008579
• Contact:
  Lucas Terriel
• Participants:
  Alix Chague, Lucas Terriel, Hugo Scheithauer

7.1.2 Ungoliant

• Name:
  Ungoliant
• Keyword:
  Natural language processing
• Functional Description:
  Ungoliant is a high-performance pipeline that provides tools to build corpus generation pipelines from CommonCrawl. It currently is the generation pipeline for OSCAR corpus. Ungoliant is a replacement of the goclassy pipeline.
• URL:
  https://­github.­com/­oscar-project/­ungoliant
• Publications:
  hal-03301590, hal-03536361
• Contact:
  Julien Abadji
• Participants:
  Julien Abadji, Pedro Ortiz Suarez, Benoit Sagot

7.1.3 HTR-United

• Keywords:
  HTR, OCR
• Functional Description:

  HTR-United is a Github organization without any other form of legal personality. It aims at gathering HTR/OCR transcriptions of all periods and styles of writing, mostly but not exclusively in French. It was born from the mere necessity for projects- to possess potentiel ground truth to rapidly train models on smaller corpora.

  Datasets shared or referenced with HTR-United must, at minimum, take the form of: (i) an ensemble of ALTO XML and/or PAGE XML files containing either only information on the segmentation, either the segmentation and the corresponding transcription, (ii) an ensemble of corresponding images. They can be shared in the form of a simple permalink to ressources hosted somewhere else, or can be the contact information necessary to request access to the images. It must be possible to recompose the link between the XML files and the image without any intermediary process, (iii) a documentation on the transcription practices followed for the segmentation and the transcription. In the cases of a Github repository, this documentation must be summarized in the README.

  A corpus can be sub-diveded into smaller ensembles if it seems necessary.

• Release Contributions:
  First version.
• URL:
  https://­htr-united.­github.­io/
• Contact:
  Alix Chague

7.1.4 VGAMT

• Name:
  Visually Guided and Adapted Machine Translation system
• Keyword:
  Machine translation
• Functional Description:
  Machine translation model that lets the user adding an image as input in addition to the sentence to be translated.
• Publication:
  hal-03977982
• Contact:
  Matthieu Futeral-peter

7.1.5 CamemBERTa

• Name:
  a DeBERTa v3-based French language model
• Keywords:
  Language model, French
• Functional Description:
  CamemBERTa was initially evaluated on five distinct downstream tasks for French: part-of-speech (POS) tagging, dependency parsing, named entity recognition (NER), FLUE (French Language Understanding Evaluation), including natural language inference (NLI). It improves the state of the art for most tasks compared to previous monolingual and multilingual approaches, which again confirms the effectiveness of large pretrained language models for French. CamemBERTa is particularly effective for high-level tasks.
• URL:
  https://­gitlab.­inria.­fr/­almanach/­CamemBERTa
• Contact:
  Wissam Antoun

7.1.6 CamemBERT-bio

• Keywords:
  Language model, Deep learning, NLP, Transformer
• Functional Description:
  CamemBERT-bio is a state-of-the-art french biomedical language model built using continual-pretraining from camembert-base. It was trained on a french public biomedical corpus of 413M words containing scientific documments, drug leaflets and clinical cases extrated from theses and articles. It shows significant improvement on multiple biomedical named entity recognition tasks compared to camembert-base.
• URL:
  http://­camembert-bio-model.­fr/
• Contact:
  Rian Touchent

7.1.7 CoMMuTE

• Name:
  Contrastive multilingual and multimodal translation evaluation
• Keywords:
  Machine translation, Evaluation, Image analysis
• Functional Description:
  CoMMuTE is a contrastive evaluation dataset designed to assess the ability of multimodal machine translation models to exploit images in order to disambiguate the sentence to be translated. In other words, given a sentence containing a word that can be translated in several ways, the additional image determines the meaning of the word to be translated. The model must then take the image into account to propose a correct translation. CoMMuTE is available from English into French, German and Czech.
• URL:
  https://­github.­com/­MatthieuFP/­CoMMuTE
• Contact:
  Matthieu Futeral-peter

7.1.8 MANTa-LM

• Name:
  Language Model on top of a Module for Adaptive Neural TokenizAtion
• Keyword:
  NLP
• Functional Description:
  Language Model on top of a Module for Adaptive Neural TokenizAtion
• URL:
  https://­huggingface.­co/­almanach/­manta-lm-base
• Contact:
  Nathan Godey

7.1.9 RoCS-MT

• Name:
  Robust Challenge Set for Machine Translation
• Keywords:
  Machine translation, NLP, Evaluation, Robustness, User-generated content
• Functional Description:
  RoCS-MT, a Robust Challenge Set for Machine Translation (MT), is designed to test MT systems’ ability to translate user-generated content (UGC) that displays non-standard characteristics, such as spelling errors, devowelling, acronymisation, etc. RoCS-MT is composed of English comments from Reddit, selected for their non-standard nature, which have been manually normalised and professionally translated into five languages: French, German, Czech, Ukrainian and Russian. The challenge set was included as a test suite at the WMT 2023 conference. This repository therefore also includes automatic translations from the submissions to the general MT task.
• URL:
  https://­github.­com/­rbawden/­RoCS-MT
• Publication:
  hal-04300824
• Contact:
  Rachel Bawden

7.1.10 3MT_French Dataset

• Name:
  3 Minutes Thesis Corpus
• Keywords:
  Multimodal Corpus, Video annotation
• Functional Description:
  This new resource will be useful to computer science and social science researchers working on public speaking assessment and training. It will help refine the analysis of speaking from a fresh perspective based on social-cognitive theories rarely studied in this context, such as first impressions and theories of primacy and recency.
• URL:
  https://­zenodo.­org/­records/­7603511#.­Y904l3CZOUk
• Publication:
  https://hal.science/hal-04366763
• Contact:
  Chloe Clavel

7.1.11 feats2notes

• Keywords:
  Natural language processing, Generative AI, Oenology, Wine, Text generation
• Functional Description:
  The aim of this software is to generate comments about an object based on a list of properties associated with a quality and intensity. It initially organises relevant structured data according to the input, transforms it into coherent sentences and ensures their quality. It was initially developed to produce oenological comments from formalised descriptions of the organoleptic properties of wines, in collaboration with the company Winespace.
• URL:
  https://­github.­com/­anna-chepaikina/­feats2notes
• Contact:
  Benoit Sagot

7.1.12 CATMuS Medieval

• Name:
  Consistent Approach to Transcribing ManuScripts - Medieval model
• Keyword:
  Handwritten Text Recognition
• Functional Description:
  CATMuS (Consistent Approach to Transcribing ManuScript) Medieval is a model for automatically transcribing medieval manuscripts using Latin scripts, in particular Old and Middle French, Latin, Spanish (and other languages of Spain), and Italian. The model was trained on the largest and most diverse dataset known for medieval manuscripts in Latin scripts, with more than 110 000 lines of training data.
• Contact:
  Thibault Clerice
• Partners:
  University of Toronto, Ecole nationale des chartes, CIHAM UMR 5648, VeDPH - Ca' Foscari, Université de Genève, ENS Lyon

7.1.13 HTRomance

• Keyword:
  Handwritten Text Recognition
• Functional Description:
  The ground truth produced as part of the HTRomance project aims to provide diverse data, from the 12th century to the 19th century, for training handwritten text recognition models. It covers the following languages: Latin, various states of French, Spanish, Occitan and Italian.
• URL:
  https://­htromance-project.­github.­io/
• Contact:
  Thibault Clerice
• Partners:
  VeDPH - Ca' Foscari, Ecole nationale des chartes, CIHAM UMR 5648, ENS Lyon

7.1.14 OSCAR

• Name:
  Open Super-large Crawled ALMAnaCH coRpus
• Keywords:
  Raw corpus, Multilingual corpus
• Functional Description:

  OSCAR is a huge multilingual corpus obtained by language classification and filtering of the Common Crawl corpus using the goclassy architecture.

  OSCAR is currently shuffled at line level and no metadata is provided. Thus it is mainly intended to be used in the training of unsupervised language models for natural language processing.

  Data is distributed by language in both original and deduplicated form. There are currently 166 different languages available.

• Release Contributions:
  Version 21.09 was generated using Ungoliant version v1, a new generation tool, faster and better documented/tested than the previous one, goclassy, used for OSCAR 1.0 (aka OSCAR 2019). As per OSCAR Schema v1.1, each document/record now has associated metadata. New languages with respect to version 2019: Manx, Rusyn, Scots and West Flemish. Their size and quality still has to be assessed. Removed languages with respect to version 2019: Central Bikol and Cantonese. Cantonsese was of a very low quality. Central Bikol corpus is still available on OSCAR 2019.
• URL:
  https://­oscar-corpus.­com/
• Publications:
  hal-02148693, hal-03301590, hal-03536361, hal-03177623
• Contact:
  Pedro Ortiz Suarez
• Participants:
  Pedro Ortiz Suarez, Benoit Sagot, Julien Abadji

7.1.15 Expresso

• Name:
  Expresso - A Benchmark and Analysis of Discrete Expressive Speech Resynthesis
• Keywords:
  Speech processing, Speech Synthesis, Expressive rendering, Multimodal Corpus
• Functional Description:
  Expresso est un corpus de parole expressive de haute qualité (48 kHz) qui comprend à la fois de la parole lue de manière expressive (8 styles, au format wav mono) et des dialogues improvisés (26 styles, au format wav stéréo). L'ensemble de données inclut 4 locuteurs (2 hommes, 2 femmes) et totalise 40 heures (11h de lecture, 30h d'improvisation). Les transcriptions de la parole lue sont également fournies. La tâche du Benchmark Expresso est de resynthétiser l'audio entrant en utilisant un code discret à faible débit qui a été obtenu sans supervision à partir du texte.
• Contact:
  Tu Nguyen
• Partner:
  META

7.1.16 SONAR

• Keywords:
  Sentence embeddings, Natural language processing, Multimodality, Speech, Text, Machine translation, Zero-shot
• Functional Description:
  SONAR (for Sentence-level multimOdal and laNguage-Agnostic Representations) is a multilingual and multimodal fixed-size sentence embedding space, with a full suite of speech and text encoders and decoders. It substantially outperforms existing sentence embeddings such as LASER3 and LabSE on the xsim and xsim++ multilingual similarity search tasks. Speech segments can be embedded in the same SONAR embedding space using language-specific speech encoders trained in a teacher-student setting on speech transcription data. We also provide a single text decoder, which allows us to perform text-to-text and speech-to-text machine translation, including for zero-shot language and modality combinations.
• URL:
  https://­github.­com/­facebookresearch/­SONAR
• Publication:
  hal-04264028
• Contact:
  Paul-Ambroise Duquenne

7.1.17 SpeechMatrix

• Keywords:
  Speech processing, Parallel corpus
• Functional Description:
  SpeechMatrix provides massive parallel speech which is mined from VoxPopuli in 17 languages: Czech (cs), German (de), English (en), Spanish (es), Estonian (et), Finnish (fi), French (fr), Croatian (hr), Hungarian (hu), Italian (it), Lithuanian (lt), Dutch (nl), Polish (pl), Portuguese (pt), Romanian (ro), Slovak (sk) and Slovenian (sl).
• URL:
  https://­github.­com/­facebookresearch/­fairseq/­tree/­ust/­examples/­speech_matrix
• Contact:
  Paul-Ambroise Duquenne
• Partner:
  META

7.1.18 T-modules

• Keywords:
  Machine translation, Natural language processing, Speech processing, Zero-shot, Multimodality
• Functional Description:
  A new approach to zero-shot cross-modal transfer between speech and text for translation tasks. Multilingual speech and text are encoded in a joint fixed-size representation space, and then decoded to enable zero-shot translation between languages and modalities. All our models are trained without the need of cross-modal labelled translation data. Despite a fixed-size representation, we achieve very competitive results on several text and speech translation tasks, outperform the state of the art for zero-shot speech translation and introduce the first results for zero-shot direct speech-to-speech and text-to-speech translation.
• Publication:
  hal-03834732
• Contact:
  Paul-Ambroise Duquenne

8.1.1 OSCAR: Collecting Text Corpora from the Web

Since the introduction of large language models (LLMs) in Natural Language Processing (NLP), large raw corpora have played a crucial role in computational linguistics. However, most of these large raw corpora are either available only for English or not available to the general public due to copyright issues. There are some examples of freely available multilingual corpora for the training of deep learning NLP models, such as the Paracrawl corpus 76 and our own large-scale multilingual corpus OSCAR 116.20 However, they have quality issues, especially for low-resource languages, an issue investigated in a large-scale study we were involved in and whose initial publication in 2021 102 was followed by a publication in the Transactions of the Association for Computational Linguistics102.

Recreating and updating these corpora is very complex. Since 2021, we have been developing Ungoliant 73, 72, a pipeline that improves in a number of ways upon the goclassy pipeline used to create the original OSCAR corpus (known as OSCAR v1 or OSCAR 2019). Ungoliant is faster, modular, parameterisable and well documented. We have used it to create several new versions of OSCAR that are larger and based on more recent data 73, 72, and it was also used as the basis for the pipeline to create the ROOTS corpus on which the large multilingual language model BLOOM was trained during the BigScience project (38% of ROOTS is based on OSCAR) 70.

In addition to the improved pipeline, we also have also added additional information to OSCAR. Since version 22.01, OSCAR has included updated metadata, with language identification performed at the document level, resulting in a new subcorpus containing documents with sentences in multiple languages in significant proportions. Ongoing work includes (amongst other aspects) large-scale improvement of the language identification mechanism, additional annotations. Ungoliant is released under an open-source licence and we publish the corpus under a research-only licence.

In early 2023 we published OSCAR 23.01, which is based on the November/December 2022 dump of Common Crawl. While being quite similar to OSCAR 22.01, it contains several new features, including KenLM-based 94 adult content detection, precomputed Locality-Sensitive Hashes for near deduplication, and blocklist-based categories. OSCAR 23.01 has also moved from gzip to Zstandard compression.

In 2023 we have also initiated the development of a new, multimodal version of OSCAR, in collaboration with engineers working at GENCI on the Jean Zay supercomputer. Our goal is to collect a large number of images from the internet associated with snippets of text (such as, but not limited to, their captions), the type of data required to train models that combine text and image information (e.g. for multimodal machine translation, where image-based information is used to improve machine translation quality, for which see Section 8.4.1).

This work is still being carried out in close collaboration with Pedro Ortiz, although he left ALMAnaCH mid-2022 after a successful PhD defence 113 and in 2023 was working at DFKI Berlin. Other people outside of ALMAnaCH regularly interact with the “OSCAR team”, in particular Sebastian Nagel from CommonCrawl.

8.1.2 COLaF: Collecting High-quality Text Corpora for French and Other Languages of France

In close interaction with the (informal) OSCAR project and in collaboration with our colleagues from the MULTISPEECH team in the Nancy Inria Center (Emmanuel Vincent, Slim Ouni) and the former ALMAnaCH member Laurent Romary, as well as with the Inria headquarters' support (in particular Jean-Frédéric Gerbeau), we finalised the creation of COLaF (“Corpus et Outils pour les Langues de France” — corpora and tools for the languages of France), an Inria “DEFI” (Inria-internal multi-team project) jointly led by Benoît Sagot and Slim Ouni. The goal of COLaF is to contribute to the development of free corpora and tools for French and other languages of France, in close collaboration with academic and institutional partners.

The scope of COLaF includes both:

• Text data, for which ALMAnaCH is responsible, with Benoît Sagot as COLaF's co-PI and Thibault Clérice as COLaFALMAnaCH's project manager,
• Speech and sign language data, for which MULTISPEECH is responsible.

COLaF aims to cover French and the languages of France in all its diversity:

• It aims to have as diverse a coverage as possible: French from France and elsewhere, regional languages, French-based creoles (including outside France), indigenous languages, migrant languages and French sign language.
• All aspects of variation will be studied, beyond the standard state of the language, including specialised languages, diachrony, non-standard states (user-generated content, learner language, etc.).
• Activity within the project notably covers the acquisition and structuring of texts from non-textual sources (books, audio recordings, etc.), the classification by language and linguistic variety of large volumes of texts (in close connection with the OSCAR project), the development of annotation and transformation models (translation, normalisation, voice synthesis, sign language generation) serving the development of corpora and the exploitation of newly created resources.

Since its creation in August, and even prior to that (thanks to funding from Benoît Sagot's PRAIRIE chair), we have been working in multiple directions, including:

• We have interacted with a number of relevant external partners:
  • language-specific institutions, including Académie Régionale de la Langue Picarde (Picard) and Lo Congrès (Occitan),
  • research laboratories such as LILPa (Université de Strasbourg and CNRS, Alsacian and other languages), LISA (Université de Corse and CNRS, Corsican), MoDyCo (Université Paris-Ouest and CNRS, Breton and other languages).
• We have started to work on text corpus representation based on the TEI guidelines, with a special interest in a complete and homogeneous metadata scheme.
• We have resumed our work on language identification, at the intersection between the OSCAR and COLaF project, in the context of Rasul Dent's newly started PhD (funded by COLaF).

8.3.1 MT for specific domains and low-resource scenarios

One of the major challenges for the development of high quality MT models is adapting them to specific domains, or even ideally to multiple domains at once. We have made several contributions to MT for specific domains this year.

Firstly, Rachel Bawden was one of the co-organisers of two shared tasks at WMT 2023, the main conference on MT: the general MT task, which evaluates MT models over a range of domains 34, and the biomedical MT task, which evaluates MT models on their ability to translate biomedical texts 36. ALMAnaCH is also a partner of the MaTOS ANR project (PI François Yvon, CNRS), dedicated to the MT of scientific documents for English–French and French–English 44. Two PhD students are co-supervised by Rachel Bawden jointly with François Yvon in the projet: Nicolas Dahan on the evaluation of MT for scientific documents and Ziqian Peng (recruited by the CNRS) on document-level MT. The first research carried out in the project has focused on the NLP domain, using titles and abstracts of articles from the HAL platform and postediting automatic translations from a variety of MT models. We analysed the performance of the different models and studied the differences between postedits produced by professional translators and those produced by authors in the NLP community.26

Adapting neural MT models to specific domains was the aim of the DadaNMT project led by Rachel Bawden and involving Jesujoba Alabi (engineer in 2022) and Seth Aycock. We explored two different strategies for adaptation: (i) the integration of matched terms from bilingual lexicons to overcome tye presence of otherwise unknown terms, and (ii) the selection of few-shot examples when translating using LLMs, guided by topic models. In the first work,27 we explored the commonly used strategy of adding additional information inline in source sentences in a bid to offer insights into whether this can be more than a copy mechanism. We found that gains were limited using this method but that it is typically more than just a copy mechanism, and models are capable of selecting or ignoring terms amongst those given, as appropriate. In the second approach,28 we explored several strategies of using topic models to perform domain adaptation of LLMs for MT, including appending topic-based keywords, topic labels and few-shot examples from specific topics.

This second approach is linked to the topic of Armel Zebaze's PhD on analogy for low-resource NLP, supervised by Rachel Bawden and Benoît Sagot, which began in November 2023. He has been studying the use of different language models for few-shot example selection, particularly to help the translation of low-resource language pairs. The topics of domain adaptation and low-resource processing are in fact highly linked for two main reasons: (i) many domains are low-resource and therefore many of the techniques used in low-resource MT can be used for adaptation to specific domains and (ii) adapting a model to a new language can be seen as analogous to adapting a model to a new domain, particularly in multilingual models. A further contribution to low-resource MT was the use of multilingual language models for Indian languages. In 40, in the context of Sonal Sannigrahi's 2021 internship, we studied the use of transliteration and segmentation strategies to facilitate cross-lingual transfer and few-shot transfer.

8.3.2 MT applied to non-standard texts

User-generated content (UGC) such as texts found on social media are characterised by multiple phenomena not typically present in standard edited texts and which present challenges for MT (e.g. spelling mistakes, acronyms, truncations, and contractions). It is important to develop MT models that are robust, which means they are able to translate these kinds of text just as well as if the texts had not displayed non-standard variation. In 2023, we explored both the development of robust models and the creation of evaluation dataset specific to non-standard texts.

In the context of José Rosales Nuñez's PhD thesis co-supervised by Djamé Seddah with the Laboratoire de Linguistique Formelle (Université de Paris),29 we explored the use of variational MT for the translation of UGC and proposed an extension using mixture density networks and normalising flows, based on the hypothesis that having independent latent distributions could help model different UGC specificities 39. Our proposed model showed results comparable or superior to standard state-of-the-art variational MT models, whilst improving performance on UGC translation. We provide analysis into the behaviour of neural learning representations when processing non-standard data. For example, our analysis of the models reveals that the representations of non-standard sentences have a higher cosine similarity with respect to their normalised versions than for the standard variational model. Finally, we confirm that the learned embeddings are more robust, since they can be used to initial the embeddings of a a vanilla transformer model to produce a more robust model.

An alternative strategy to developing more robust models is to normalise the non-standard text before applying tools trained on standard data. The lexical normalisation step is itself challenging, as it requires determining which words require normalising and how to choose standard variants to replace them with. In the context of her PhD on robustness MT, supervised by Rachel Bawden and Benoît Sagot, Lydia Nishimwe explored the challenges faced by this task, by surveying the literature, providing preliminary experiments using various masked language models combined with levenshtein distance, and discussing the problems of evaluation for the task 47. Her paper won the best paper award at the RECITAL 2023 conference.30 She has since been working on building robust sentence embeddings, using the test case of Laser, and, inspired by T-modules (research carried out in the context of Paul-Ambroise Duquenne's PhD) using distillation to reduce the distance between non-standard sentences and their normalised versions.31

8.3.3 MT Evaluation

Research into evaluation is important for the development of models. On the subject of MT robustness and evaluation, we developed a new evaluation set designed to test MT models' capacity to handle specifically non-standard texts 20. RoCS-MT (Robust Challenge Set for Machine Translation) is composed of non-standard English texts from the Reddit forum, which we manually normalised and had professionally translated into 5 languages, French, German, Czech, Ukrainian and Russian (See the resource in Section 7.1.9), funded by Rachel Bawden's PRAIRIE chair position. RoCS-MT was submitted as a test suite to the WMT 2023 general translation task, which meant that we could evaluate translations produced by all models submitted to the task. We found that most models still struggle with many non-standard phenomena, in particular those that alter the spelling of words.

In a follow-up to the BigScience Workshop 70, we also completed the evaluation of the BLOOM language model in collaboration with François Yvon (CNRS) 21. We chose to study a range of domains, language pairs, resourcedness levels, as well as other aspects such as prompt choice, number of few-shot examples, the use of cross-lingual few-shot examples and the influence of linguistic context. As an extension to this work and as described in Section 8.2, we have also been working on the evaluation of BLOOM for a range of French NLP tasks.

Finally, as mentioned above, Nicolas Dahan, supervised by Rachel Bawden and François Yvon in the context of the MaTOS ANR project, began a PhD on the evaluation of MT for scientific documents. He has begun to explore the current state of the art in MT metrics and analyse how well adapted they are to evaluating document-level aspects such as lexical cohesion.

8.4.1 Image-enhanced MT

In the context of Matthieu Futeral-Peter's PhD thesis, co-supervised by Rachel Bawden, Benoît Sagot and Cordelia Schmid (WILLOW project-team) and in collaboration with Ivan Laptev (ex-member of WILLOW), we have continued developing new approaches to multimodal (text-image) MT. Our main focus has been on ambiguity, which presents one of the major challenges in MT, and on how to exploit visual context to resolve it. Previous work in the field has shown that obtaining improvements from the addition of images is challenging, limited not only by the difficulty of building effective cross-modal representations but also by the lack of specific evaluation and training data.

Our new multimodal method, named VGAMT (Visually Guided and Adapted Machine Translation) 2 (see the model in Section 7.1.4) is based on a strong pre-trained MT model, fine-tuned using adapters on two training objectives: the multimodal MT objective (exploiting multimodal parallel sentences, each accompanied by an image) and visually conditioned masked language modelling (using English captioning data). We found that combining these two objectives was necessary to force the model to effectively use visual context. The model's performance is further boosted by the use of a novel guided self-attention mechanism, by which irrelevant connections between the text and the image parts are masked, thereby guiding the model to consider the relevant connections. An important aspect of this work was also evaluation, standard metrics being ill-adapted and misleading for reporting progress on this task, due to the lack of adequate test data. We released CoMMuTE (see the resource in Section 7.1.7), a Contrastive Multilingual Multimodal Translation Evaluation dataset, composed of ambiguous sentences and their possible translations, accompanied by disambiguating images corresponding to each translation. For three language directions (English–{French, Czech and German}), we show that our method remains competitive against text-only models on standard benchmarks and performs above all other models by a large margin on CoMMuTE, whereas the previous state-of-the-art models do not beat the baseline.

8.4.2 Multimodal MT with speech

This subsection is directly inspired by the introduction of Paul-Ambroise Duquenne's PhD thesis (not yet published).

In the context of Paul-Ambroise CIFRE PhD thesis, co-supervised by Benoît Sagot and Holger Schwenk (META), we had previously investigated the use of fixed-sized sentence embeddings spaces for multiple purposes, with a focus on text-to-text, speech-to-text, text-to-speech and speech-to-speech MT. We first introduced multilingual and multimodal speech/text sentence embeddings using a teacher/student approach with the existing LASER sentence embedding space 88. We demonstrated that we could perform semantic similarity estimation between speech and text in different languages and introduce speech mining as an extension of bi-text mining for the speech modality. We trained speech translation systems using the mined data and demonstrate significant gains with this additional data. Based on these promising results, we scaled speech-to-speech mining to 136 language pairs to introduce the SpeechMatrix corpus and trained several speech translation systems on this mined data 28.

We then explored how to efficiently decode these fixed-size representations into multiple languages and modalities, and how to perform zero-shot cross-modal machine translation in this framework. We demonstrated that we can combine independently trained encoders and decoders from different languages and modalities in a zero-shot way to perform cross-modal translation 32. As a second step, published this year, we explore multilingual training in such modular framework, to benefit from cross-lingual learning 29.

This year, we drew conclusions from our previous work and introduced SONAR, a state-of-the-art massively multilingual speech/text sentence embedding space for both cross-lingual and cross-modal similarity search as well as decoding capabilities 63. We recently complemented these semantic sentence representations with a modality specific representation encoding non-semantic speech properties of an audio signal.

8.6.1 Analysing Zero-Shot Transfer Scenarios across Spanish Variants

In previous work on multilingual zero-shot hate-speech detection 109, we highlighted the need to alleviate the cultural gap found in cross-lingual scenarios. An example of the issue of the cultural gap is the use of the slur puta, which can translate to English “prostitute”, as a non-offensive intensifier in Spanish, which we found to be a strong trigger for our models trained on an English misogyny-oriented dataset. While this cultural gap has been highlighted for different languages from different countries and has been under increasing scrutiny recently, the question of its impact on NLP models when applied to dialects of the same language remains. Language variation between geolects such as American English and British English, Latin-American Spanish, and European Spanish is still a problem for NLP models, which often rely on (latent) lexical information for their classification tasks. More importantly, the cultural aspect just mentioned, which is crucial for hate speech detection, is often overlooked.

Along this line of research, we therefore carried out a thorough analysis of hate speech detection models for different variants of Spanish and introduced a new dataset of hate speech directed toward immigrants from Twitter. Using mBERT 117 and Beto 81, a monolingual Spanish BERT-based language model, as the basis of our transfer learning architecture, our results indicate that hate speech detection models for a given Spanish variant are affected when different variations of the language are not considered. When compared to multilingual model such as mBERT, these differences are accentuated by the specific nature of its training data (Wikipedia, no discussions). Our intuitions were confirmed with an error analysis conducted with the SHAP interpretability framework 106, which highlighted the vulnerability of language models fine-tuned on another geolect to cultural-specific hateful terms. In an era where cross-cultural issues in NLP are becoming increasingly important 98, 112, 95, our work and methodology constitute an interesting step in this process. This work was published in a EACL2023's specialised workshop on language variation (VARDIAL 2023) 23.

8.6.2 Multilingual Radicalisation Detection in the CounteR Project

Since 2021 ALMAnaCH has been the leader of the NLP part of the CounteR H2020 European project devoted to radicalisation detection (grant agreement n°101021607). Part of this work consists in building a radicalisation classifier to be integrated in the CounteR platform, a prototype targeting Law Enforcement Agencies. The classifier is trained on two sources of data: data acquired by the consortium from a third-party covering French, English and Arabic, and data targeting six others languages (Portuguese, German, Romania, Latvian, Greek and Bulgarian) produced through the translation and adaption of the English data. A third source of data, currently used for evaluation, was produced by prompting LLMs following different user scenarios.

We based our classification architecture on the multilingual multitask framework we developed from 92 and detailed in 109. As the basis for our multi-task classifier, we used XLMR-T, a large multilingual XLM-R model 83 fine-tuned on a masked-language modelling task on 200M tweets 77. We then jointly trained the classifier on our radicalisation dataset and on a variety of auxiliary tasks (part-of-speech tagging, dependency parsing, named entity recognition (NER) and sentiment analysis). In order to cope with code-switching and to avoid having to run language identification, we concatenated the training data for all languages into one training set. The performance gain compared to the use of monolingual models greatly exceeded expectations (+6 points for Arabic for example, despite it being written in a different script). As our deliverables are UE restricted, we unfortunately cannot provide further details of our results in a publicly available report.

An important part of our work this year was the improvement of our model by producing and including domain-specific NER annotations for our core languages (Arabic, English, French). We took advantage of this time-consuming task to also pseudo-anonymise all entities and non-public URLs. Unlike the current state of the art, we decided to keep all socio-demographic variables that can be expressed through aliases, the topic of the URLs, hashtags and Telegram channels in order to preserve the meaning. The idea is to make sure that a classifier trained on pseudo-anonymised data would still benefit from lexical information while being able to distribute the corpus in a GDPR-compliant way. Note that no other corpus of this type exists with this granularity of annotation and in particular for this target domain. Its release will most certainly establish a notable milestone.

Regarding the potential biases of our classifiers, we are involved in a bias assessment of our models with one of the partners of the project, Ethicas Tech, an auditing firm. Based on target profiles we identified, we generated a synthetic dataset using the Vicuna Uncensored LLM,34 for about 1800 documents for French and English and annotated them following the same guidelines used to build the initial dataset. Of course, being purely generated, there is no intent in this type of content per se, so analysing biases against non-existent users is arguable. However, given GDPR restrictions and the volatile nature of social media content (most of the most radical content is quickly deleted by the platforms), using such synthetic data can be considered a plausible proxy for such studies. The content we generated will also be used to train an LLM-based radical content detector using the same methodology we demonstrated in 42, where we released the first adversarial dataset for the detection of ChatGPT content.

8.6.3 Enriching the Narabizi Treebank

One of the target languages of the CounteR project is Arabic in all its forms, including dialectal Arabic. Given our expertise on handling user-generated content, we focused on the North-African Arabic dialect, which displays significant variability across regions and predominantly exists in spoken form and lacks standardised spelling when written. Many Arabic speakers use the Latin script to write their dialects online, using digits and symbols for phonemes not easily mapped to Latin letters 139. This written form, called Arabizi and its North African variant, NArabizi, often showcases code-switching, with French with up to 36% of French words in the NArabizi dataset 139.

In order to increase the usefulness of the NArabizi treebank in a multitask framework where task transfer is better realised through higher level tasks (e.g. NER, Sentiment analysis) than lower level ones (e.g. morpho-syntactic analysis via part-of-speech tagging and dependency parsing) 109, 121, we added two more annotation layers to the treebank: named-entities and offensive language annotation. We also conducted an extensive consistency checking that led to re-annotating many linguistic phenomena. In particular, we switched from the light tokenisation inspired by the French Social Media Bank 140 to a more classic morphology-based tokenisation. This step was made necessary by the inclusion of the named entity annotation layer, as we wanted to avoid creating multiple entities differing only in the the use of determiners. This step enabled us to conduct (surprisingly) the first experiments on NER with non-gold tokenisation. This work was published in a specialised workshop (Linguistic Annotation Workshop) colocated with ACL 2023 38.

Verbatim Analysis

Participants: Benoît Sagot.

• Partner type:
  Inria start-up
• Leader for ALMAnaCH:
  Benoît Sagot.
• Dates:
  1 Jul 2009–31 Dec 2024
• Description:
  Verbatim Analysis is an Inria start-up co-created in 2009 by enoît Sagot. It uses some of ALMAnaCH's free NLP software (SxPipe) as well as a data mining solution co-developed by Benoît Sagot, VERA, for processing employee surveys with a focus on answers to open-ended questions. Its activities have been progressively taken over by opensquare (see below), and the company will be discontinued at the end of Jan 2024.

opensquare

Participants: Benoît Sagot.

• Partner type:
  Inria start-up
• Leader for ALMAnaCH:
  Benoît Sagot.
• Dates:
  1 Dec 2016–present
• Description:
  Opensquare was co-created in December 2016 by Benoît Sagot with 2 senior specialists of human resources (HR) consulting. It is dedicated to designing, carrying out and analysing employee surveys as well as HR consulting based on these results. It uses a new employee survey analysis tool, enqi, which is still under development. This tool being co-owned by opensquare and Inria, both parties have signed a Software Licence Agreement in exchange for a yearly fee paid by opensquare to Inria based on its turnover. Benoît Sagot currently contributes to opensquare, under the “Concours scientifique” scheme.

META AI

Participants: Benoît Sagot, Tú Anh Nguyen, Paul-Ambroise Duquenne.

• Partner type:
  Company
• Leader for ALMAnaCH:
  Benoît Sagot.
• Dates:
  1 Jan 2018–present
• Funding received:
  210,000€
• Description:

  Our collaboration with META AI is centered around the joint supervision of CIFRE PhD theses. A first collaboration (Louis Martin's PhD thesis), co-supervised by Benoît Sagot, Éric de La Clergerie and Antoine Bordes (META) was dedicated to text simplification (“français Facile À Lire et à Comprendre”, FALC), in collaboration with UNAPEI. This collaboration was part of a larger initiative called Cap'FALC involving (at least) these three partners as well as the relevant ministries. Louis defended his PhD in 2022. Two other joint PhD theses started in 2021 and will be defended in 2024. Paul-Ambroise Duquenne's PhD, co-supervised by Benoît Sagot and Holger Schwenk (META), is dedicated to sentence embeddings for massively multilingual speech and text processing. Tú Anh Nguyen's PhD, co-supervised by Benoît Sagot and Emmanuel Dupoux (META), is dedicated to the unsupervised learning of linguistic representations from speech data, with a focus on textless dialogue modelling and speech generation.

  In addition, Benoît Sagot was one of the very few academic researchers who received a $50,000 gift grant from META AI.

Winespace

Participants: Benoît Sagot, Anna Chepaikina.

• Partner type:
  Start-up
• Leader for ALMAnaCH:
  Benoît Sagot.
• Dates:
  1 Sept 2019–30 Sep 2023
• Funding received:
  32,391€
• Description:
  The collaboration with this start-up company, dedicated to information extraction from wine descriptions to develop a wine recommendation system, was carried out in 2020 following previous discussions, in collaboration with Inria Bordeaux's “InriaTech” structure. In 2021, we designed and prepared a second step for this collaboration, which involved the hiring of Anna Chepaikina, a dedicated research engineer who started at the end of March 2022. After twelve months working in the framework of a “plan de relance” scheme, she was hired at Inria for a further six months fully funded by Winespace in the context of a new bilateral agreement. Since then, she has been hired by Winespace as an NLP expert.

INPI

Participants: You Zuo, Benoît Sagot, Éric de La Clergerie.

• Partner type:
  Public institution
• Leader for ALMAnaCH:
  Benoît Sagot.
• Dates:
  1 Oct 2021–30 Nov 2023
• Funding received:
  99,999.99€
• Description:
  A collaboration with the Institut National de la Propriété Industrielle (France's patent office) started in 2021. A research engineer, You Zuo, was hired for a one-year contract to work on patent classification, later extended to 14 months. This project also involved an informal collaboration with the qatent startup.

Qatent

Participants: You Zuo, Benoît Sagot, Éric de La Clergerie.

• Partner type:
  Inria start-up
• Leader for ALMAnaCH:
  Benoît Sagot.
• Dates:
  1 Jan 2021–present
• Description:
  Qatent is a startup supported by the Inria Startup Studio and ALMAnaCH that applies NLP to help write better patents faster. It follows the 18-month stay at ALMAnaCH of Kim Gerdes, one of the three founders of the company, and benefits from ALMAnaCH's scientific expertise and the Inria Startup Studio's counselling and financial support. You Zuo's CIFRE PhD thesis, co-supervised by Benoît Sagot and Kim Gerdes (now at qatent), is the latest development of this collaboration.

Orange

Participants: Éric de La Clergerie, Lionel Tadjou Tadonfouet.

• Partner type:
  Company
• Leader for ALMAnaCH:
  Éric de La Clergerie.
• Dates:
  1 Mar 2020–28 Feb 2023
• Funding received:
  30,000€
• Description:
  The collaboration between ALMAnaCH and Orange is centered around a joint CIFRE PhD thesis dedicated to conversation disentanglement. The CIFRE convention ended in February 2023, and Lionel Tadjou defended his PhD in September 2023.

AXA ReV

Participants: Djamé Seddah.

• Partner type:
  Company
• Leader for ALMAnaCH:
  Djamé Seddah.
• Dates:
  1 Oct 2020–present
• Description:
  AXA ReV is the R&D Lab of the Axa Insurance group, located in Paris. This collaboration focuses on neural model interpretability and establishing “explainable” benchmarks as an end-goal for research on question answering.

LightON

Participants: Djamé Seddah.

• Partner type:
  Start-up
• Leader for ALMAnaCH:
  Djamé Seddah.
• Dates:
  22 Sept 2020–present
• Description:
  LightON builds Optical Processor Units, a specialised line of processor able to outperform GPUs on certains tasks. We are working with them to see if we can use their technology to speed up the training of large language models. This informal collaboration has already resulted in the design, training and publication of the PAGnol generative language model for French.

zaion

Participants: Chloé Clavel, Lorraine Vanel.

• Partner type:
  Company
• Leader for ALMAnaCH:
  Chloé Clavel.
• Dates:
  1 Feb 2022–1 Mar 2025
• Funding received:
  16,000€
• Description:
  CIFRE PhD thesis between Telecom-Paris37 and Zaion in order to develop conversational systems integrating socio-emotional strategies in an explicit way. The CIFRE contract being with TelecomParis, it is mentioned in this section rather than in the previous one.

10.1.1 H2020 projects

ANR BASNUM

Participants: Benoît Sagot.

• Duration:
  1 Oct 2018–30 Jun 2023.
• PI:
  Geoffrey Williams (Université de Grenoble).
• Coordinator for ALMAnaCH:
  Laurent Romary.
• Partners:
  • Université de Bretagne Sud
  • Université Grenoble Alpes
  • LaTTICe
• Summary:
  Digitalisation and computational annotation and exploitation of Henri Basnage de Beauval's encyclopedic dictionary (1701).

ANR MaTOS

Participants: Rachel Bawden, Éric de La Clergerie, Nicolas Dahan.

• Duration:
  1 Jan 2023–31 Dec 2026.
• PI:
  François Yvon.
• Coordinator for ALMAnaCH:
  Rachel Bawden.
• Partners:
  • Université de Paris
  • CNRS
• Summary:
  The MaTOS (Machine Translation for Open Science) project aims to develop new methods for the machine translation (MT) of complete scientific documents, as well as automatic metrics to evaluate the quality of these translations. Our main application target is the translation of scientific articles between French and English, where linguistic resources can be exploited to obtain more reliable translations, both for publication purposes and for gisting and text mining. However, efforts to improve MT of complete documents are hampered by the inability of existing automatic metrics to detect weaknesses in the systems and to identify the best ways to remedy them. The MaTOS project aims to address both of these issues.

ANR TraLaLaM

Participants: Rachel Bawden, Benoît Sagot, Thibault Clérice, Juliette Janès.

• Duration:
  1 Oct 2023–30 Sept 2026.
• PI:
  Josep Crego (SYSTRAN).
• Coordinator for ALMAnaCH:
  Rachel Bawden.
• Partners:
  • SYSTRAN
  • CNRS
• Summary:
  The aim of TraLaLaM is to explore the use of large language models (LLMs) for machine translation, by asking two main questions: (i) in what scenarios can contextual information be effectively used via prompting? and (ii) for low-resource scenarios (with a focus on dialects and regional languages), can LLMs be effectively trained without any parallel data?

10.2.1 Competitivity Clusters and Thematic Institutes

10.2.2 Other National Initiatives

10.2.3 Regional Initiatives

11.1.1 Scientific events: organisation

11.1.2 Scientific events: selection

11.1.3 Journal

11.1.4 Invited talks

• Rachel Bawden:
  • Laboratoire Informatique de Grenoble, Université de Grenoble, France (28 Mar 2023): “From Linguistic to Visual Context in Machine Translation”.
  • ANITI-PRAIRIE workshop, Toulouse, France (27 Jun 2023): “From Linguistic to Visual Context in Machine Translation”.
• Benoît Sagot:
  • Journée d'étude “La littérature au prisme des humanités numériques” (One-day workshop organised by the ObTIC - Sorbonne Université) (16 Mar 2023): “Une approche computationnelle pour l'étude scriptométrique du français du XVIIème siècle”.
  • European Masters Program “Language & Communication Technologies” Annual Meeting 2023 (19 Jun 2023): “Language models and their training data: experiments and challenges”.
  • PRAIRIE colloquium, PariSanté Campus (16 Mar 2023): “Combining modalities: two experiments on multimodal NLP”. Website
  • George Mason University, Fairfax, Virginia, USA (21 Apr 2023): “NLP beyond text”. GMU Linguistics Colloquium Series
• Hugo Scheithauer:
  • Journée d'étude “Traitements automatique pour les humanités numériques” - Université Paris-Nanterre (15 Mar 2023): “DataCatalogue : un projet pour la restructuration automatique des catalogues de vente”. Website
• Chloé Clavel:
  • Seminar LORIA Nancy (17 Nov 2023): “Socio-conversational AI: Modelling the socio-emotional component of interactions using neural models”. Website
  • Amphi21 Sciences Po (30 Nov 2023): “Nouveaux outils, nouveaux usages. Quels enjeux pour une intelligence artificielle transparente et responsable ?”. Website
• Alix Chagué and Floriane Chiffoleau:
  • Institut historique allemand (DHIP/IHA), Paris (7 Sept 2023): “ATR: What can eScriptorium do for you?”. From Source to Full Text: Workshop on Using Automatic Character Recognition (ATR)
  • Institut historique allemand (DHIP/IHA), Paris (8 Sept 2023): “What can you do next? Choice of output and reuse of your transcription”. From Source to Full Text: Workshop on Using Automatic Character Recognition (ATR)
• Alix Chagué:
  • Institut historique allemand (DHIP/IHA), Paris (8 Sept 2023): “Image Acquisition and Layout Analysis”. Joint with Hippolyte Souvay. From Source to Full Text: Workshop on Using Automatic Character Recognition (ATR)
  • Atelier Antologie Grecque, Sorbonne Univ. Paris (6 Mar 2023): “EScriptorium and automatic text recognition”. Collaborative edition of the Greek Anthology - practical workshops
• Justine Cassell:
  • MBZ University of Artificial Intelligence, Abu Dhabi (12 Dec 2023): “What future for human-computer interaction in the era of AI”.
  • Grenoble Institute Polytechnique (28 Nov 2023): “Sociality in Language and Thought”.
  • World Economic Forum, Davos (16 Dec 2023): “Empowering Humans and Machines in Industry”. Various talks in the context of the Davos meeting
  • INSEI (Suresnes): Laboratoire Graphes (17 Nov 2023): “Programmable Virtual Peers for Autism”.
  • Canopé: Réseaux de Formation des Enseignants (15 Nov 2023): “L'IA Generative et l'Education”.
• Thibault Clérice:
  • Institut Élie Cartan de Lorraine, Nancy (21 Nov 2023): “Par-delà le mot: détecter l'expression de la sémantique sexuelle dans un corpus latin en diachronie longue”. Journée Fédération Charles Hermite : Représentations du Langage
  • Centre de Recherche Inter-Universitaire en Humanités Numériques, Montréal, Canada (13 Oct 2023): “Détection de la sémantique sexuelle en langue latine”. Colloque du CRIHN 2023
• Djamé Seddah:
  • Jelinek Summer Workshop on Speech and Language Technology, JSALT, Le Mans Universités (26 Jun 2023): “Faut-il avoir peur du grand méchant GPT ? Démystifions les modèles de langues”. Website
• Floriane Chiffoleau:
  • Institut historique allemand (DHIP/IHA), Paris (8 Sept 2023): “Text recognition and correction”. Joint with Ariane Pinche, Hippolyte Souvay. From Source to Full Text: Workshop on Using Automatic Character Recognition (ATR)
  • University of Oslo, Norway (online) (21 Sept 2023): “TEI Publisher, a platform for sustainable digital editions”. Workshop Digital Scholarly Editions in the Nordic Region - Where are we now? Where will we be in 10 years?
• Lauriane Aufrant:
  • Bibliothèque nationale de France (24 Nov 2023): “Fiabiliser sans entraver, comprendre pour utiliser : les normes en soutien à la réglementation européenne sur l'IA”.
• Éric de La Clergerie:
  • Université d'Orléans (14 Dec 2023): “Grands modèles de langues, revisite d'une discipline ? (ou grand chamboulement ?)”. Intervention dans le master SDL

11.1.5 Scientific expertise

• Alix Chagué:
  • Expert for the Fondation des arts et des sciences.
  • Member of Comité de Perfectionnement du Master Technologies Numériques Appliquées à l'Histoire de l'Ecole nationale des chartes (Paris).
• Rachel Bawden:
  • Reviewer of thematic committee for Genci projects (Reviewing projects for the allocation of computation resources).
• Benoît Sagot:
  • Member of the scientific advisory board of ERIC CLARIN.
• Djamé Seddah:
  • Expert for the Cascade funding call published for the EU project UTTER.

11.1.6 Research administration

• Benoît Sagot:
  • Member of the scientific board of Inria Paris's Comité des Projets (Inria Paris research centre's Bureau du Comité des Projets).
  • Member of the board of the Société de Linguistique de Paris (Administrateur).
• Rachel Bawden:
  • Member of the board of the Société de Linguistique de Paris (Administratrice).

11.2.1 Teaching

• Hugo Scheithauer:
  • Master's course (M2) as part of the Master “Documentation et Humanités Numériques”. Méthodologie de la recherche et préprofessionalisation (18 hours), coorganised with Françoise Dalex. École du Louvre, France.
• Arij Riabi:
  • Master's course (M1) as part of the Cycle d'ingénieur. Applied statistics project (20 hours). ENSAE.
• Benoît Sagot:
  • Master's course (M2) as part of the Master “Mathématiques, Vision Apprentissage”. Speech and Language Processing (20 hours), coorganised with Emmanuel Dupoux and Robin Algayres. ENS Paris-Saclay, France.
  • Computational Linguistics 1 (Spring 2023), Language and Computers, invited lecture, George Mason University, Fairfax, Virginia, USA. Introduction to NLP for historical linguistics (3 hours). George Mason University, Fairfax, Virginia, USA. 20 Apr 2023.
  • Classes for the general public as part of the Chaire annuelle “Informatique et sciences numériques”. Apprendre les langues aux machines (9 hours). Collège de France. "Leçon inaugurale" (Inaugural lecture) on the 30th Nov 2023
• Alix Chagué:
  • Bachelor; Master; Doctorate's course (1er, 2e et 3e cycles) as part of the Faculté des Arts et des Sciences. HNU-6059: Humanités Numériques: Langages de Programmation (15 hours). Université de Montréal, Canada.
• Nathan Godey:
  • Bachelor's course (L2) as part of the Licence. Python. Université Pierre et Marie Curie.
  • Master's course (M2) as part of the Master SCIA. Advanced NLP (22 hours). EPITA.
• Rian Touchent:
  • Eq. Bachelor's course (1st year) as part of the Cycle ingénieur. Information systems and programming (22.5 hours). CentraleSupélec.
• Chloé Clavel:
  • Master's course (M2) as part of the Artificial Intelligence & Advanced Visual Computing Master. Sentiment analysis, speech emotion recognition, speech synthesis and conversational systems. (12 hours). Polytechnique.
• Lauriane Aufrant:
  • Master's course (M2) as part of the Master “Linguistique informatique”. Information extraction and automated knowledge graph construction (12 hours). Université Paris Cité.
  • Executive education's course (Executive education) as part of the Formation continue. Traitement automatique du langage naturel (NLP) et des données textuelles (text-mining) (7 hours). Telecom Executive Education.
• Thibault Clérice, Alix Chagué and Hugo Scheithauer:
  • Workshop on HTR-United for ADHO's DH2023 conference, University of Graz, RESOWI, Graz, Austria. Workshop HTR-United: metadata, quality control and sharing process for HTR training data (4 hours). University of Graz, RESOWI, Graz, Austria. 12 Jul 2023.

11.2.2 Supervision

11.2.3 Juries

11.3.1 Articles and contents

11.3.2 Education

• Rachel Bawden:
  • participated in the Rendez-Vous des Jeunes Mathématiciennes et Informaticiennes Inria 2023 (Presentation on NLP to high school girls), “Traitement Automatique des Langues (TAL). Que se cache-t-il derrière les modèles de langue ?” Inria Paris, 24 Oct 2023.
• Benoît Sagot:
  • gave a talk at Colloque « L'IA et ses défis » (Colloque ouvert au public du Campus de l'Innovation pour les Lycées), “L'apprentissage profond, au cœur de l'IA moderne”. Collège de France, Paris, 28 Sept 2023.
• Chloé Clavel:
  • gave a talk at Super Demain, Le grand laboratoire de l'éducation aux médias numériques, www.superdemain.fr (Seminar), “Robot et émotion : la reconnaissance des manifestations émotionnelles”. Lyon, 24 Nov 2023.
• Floriane Chiffoleau, Hugo Scheithauer and Lydia Nishimwe:
  • were mentors for work shadowing (Seconday “3ème” school students), Inria Paris, 19 Dec 2023.
• Alix Chagué and Lydia Nishimwe:
  • ran a workshop at the Rendez-vous des Jeunes Mathématiciennes et Informaticiennes, Inria Paris, 24-25 Oct 2023.

11.3.3 Interventions

• Benoît Sagot:
  • gave a talk at PRAIRIE evening (Professional workshop), “A few words on BLOOM (and ChatGPT)”. Inria Paris, 17 Jan 2023.
  • participated in FD3, co-organised by PRAIRIE and France Digitale (Professional workshop), “Round table on “Is Generative AI a revolution and what to expect next?””. Station F, 29 Mar 2023.
  • gave a talk at Dauphine Digital Days (Professional workshop), “Quelles données pour entraîner les grands modèles de langue ?”, Opening keynote for the round table “Quels régimes de régulation des données pour entraîner les intelligences artificielles ?”. Université Paris Dauphine, 20 Nov 2023.
  • gave a talk at Inria Alumni Meeting (Professional workshop), “Une brève introduction à ChatGPT”. Inria Paris, 23 Nov 2023.
• Hugo Scheithauer (with EHRI):
  • participated in Yad Vashem (EHRI General Partneer Meeting), Jerusalem, Israël, 29 May 2023.

International journals

• 12 articleT.Thibault Clérice, M.Malamatenia Vlachou-Efstathiou and A.Alix Chagué. CREMMA Medii Aevi: Literary manuscript text recognition in Latin.Journal of Open Humanities Data9April 2023, 4HALDOIback to text
• 13 articleT.Thibault Clérice. You Actually Look Twice At it (YALTAi): using an object detection approach instead of region segmentation within the Kraken engine.Journal of Data Mining and Digital HumanitiesHistorical Documents and...December 2023HALDOIback to text
• 14 articleT. A.Tu Anh Nguyen, E.Eugene Kharitonov, J.Jade Copet, Y.Yossi Adi, W.-N.Wei-Ning Hsu, A.Ali Elkahky, P.Paden Tomasello, R.Robin Algayres, B.Benoît Sagot, A.Abdelrahman Mohamed and E.Emmanuel Dupoux. Generative Spoken Dialogue Language Modeling.Transactions of the Association for Computational Linguistics11March 2023, 250–266HALDOIback to text

National journals

• 15 articleS.Simon Gabay, P.Philippe Gambette, R.Rachel Bawden and B.Benoît Sagot. Old or Modern? Towards a Computational Graphematic Analysis of 17th Century French Texts.Linx85February 2023HALDOI

Invited conferences

• 16 inproceedingsH.Hugo Scheithauer. DataCatalogue : Un projet pour la restructuration automatique de catalogues de vente.Traitements automatiques pour les humanités numériques - corpus d’histoire de l’art, d’enseignement, d’urbanismeNanterre, FranceMay 2023HALback to text

International peer-reviewed conferences

• 17 inproceedingsR.Robin Algayres, Y.Yossi Adi, T. A.Tu Anh Nguyen, J.Jade Copet, G.Gabriel Synnaeve, B.Benoît Sagot and E.Emmanuel Dupoux. Generative Spoken Language Model based on continuous word-sized audio tokens.The 2023 Conference on Empirical Methods in Natural Language ProcessingSingapore, SingaporeDecember 2023HALback to text
• 18 inproceedingsR.Robin Algayres, P.Pablo Diego-Simon, B.Benoît Sagot and E.Emmanuel Dupoux. XLS-R fine-tuning on noisy word boundaries for unsupervised speech segmentation into words.Findings of the Conference on Empirical Methods in Natural Language Processing (EMNLP)Singapore, SingaporeDecember 2023HALback to text
• 19 inproceedingsW.Wissam Antoun, B.Benoît Sagot and D.Djamé Seddah. Data-Efficient French Language Modeling with CamemBERTa.Findings of the Association for Computational Linguistics: ACL 2023Toronto, CanadaJuly 2023, 5174–5185HALDOIback to text
• 20 inproceedingsR.Rachel Bawden and B.Benoît Sagot. RoCS-MT: Robustness Challenge Set for Machine Translation.Proceedings of the Eighth Conference on Machine TranslationWMT23 - Eighth Conference on Machine TranslationProceedings of the Eighth Conference on Machine TranslationSingapore, Singapore2023, 198--216HALback to text
• 21 inproceedingsR.Rachel Bawden and F.François Yvon. Investigating the Translation Performance of a Large Multilingual Language Model: the Case of BLOOM.Proceedings of the 24th Annual Conference of the European Association for Machine TranslationEAMT 2023 - 24th Annual Conference of the European Association for Machine TranslationTampere, FinlandMarch 2023HALDOIback to text
• 22 inproceedingsJ.-B.Jean-Baptiste Camps, N.Nicolas Baumard, P.-C.Pierre-Carl Langlais, O.Olivier Morin, T.Thibault Clérice and J.Jade Norindr. Make Love or War? Monitoring the Thematic Evolution of Medieval French Narratives.Proceedings of the Computational Humanities Research Conference 2023Computational Humanities Research (CHR 2023)CEUR Workshop ProceedingsParis, France2023HALback to text
• 23 inproceedingsG.Galo Castillo-López, A.Arij Riabi and D.Djamé Seddah. Analyzing Zero-Shot transfer Scenarios across Spanish variants for Hate Speech Detection.Tenth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial 2023)Dubrovnik, CroatiaAssociation for Computational LinguisticsMay 2023, 1-13HALDOIback to text
• 24 inproceedingsA.Alix Chagué and T.Thibault Clérice. "I'm here to fight for ground truth": HTR-United, a solution towards a common for HTR training data.Digital Humanities 2023: Collaboration as OpportunityGraz, Austria2023HAL
• 25 inproceedingsA.Alix Chagué, T.Thibault Clérice, J.Jade Norindr, M.Maxime Humeau, B.Baudoin Davoury, E. V.Elsa Van Kote, A.Anaïs Mazoue, M.Margaux Faure and S.Soline Doat. Manu McFrench, from zero to hero: impact of using a generic handwriting recognition model for smaller datasets.Digital Humanities 2023: Collaboration as OpportunityGraz, AustriaNovember 2022HALback to text
• 26 inproceedingsT.Thibault Clérice, A.Alix Chagué and H.Hugo Scheithauer. Workshop HTR-United: metadata, quality control and sharing process for HTR training data.DH 2023 - Digital Humanities Conference: Collaboration as OpportunityGraz, AustriaNovember 2022HALback to text
• 27 inproceedingsT.Thibault Clérice and A.Anthony Glaise. Twenty-One* Pseudo-Chrysostoms and more: authorship verification in the patristic world.Proceedings of the Computational Humanities Research Conference 2023Computational Humanities Research Conference 2023Proceedings of the Computational Humanities Research Conference 2022Paris, France2023HALback to text
• 28 inproceedingsP.-A.Paul-Ambroise Duquenne, H.Hongyu Gong, N.Ning Dong, J.Jingfei Du, A.Ann Lee, V.Vedanuj Goswami, C.Changhan Wang, J.Juan Pino, B.Benoît Sagot and H.Holger Schwenk. SpeechMatrix: A Large-Scale Mined Corpus of Multilingual Speech-to-Speech Translations.Proceedings of the 61st Annual Meeting of the Association for Computational LinguisticsACL 2023 - 61st Annual Meeting of the Association for Computational LinguisticsToronto, CanadaAugust 2023HALDOIback to textback to text
• 29 inproceedingsP.-A.Paul-Ambroise Duquenne, H.Holger Schwenk and B.Benoît Sagot. Modular Speech-to-Text Translation for Zero-Shot Cross-Modal Transfer.Proceedings of INTERSPEECH 2023INTERSPEECH 2023Dublin, IrelandAugust 2023HALDOIback to text
• 30 inproceedings A.Ali Elkahky, W.-N.Wei-Ning Hsu, P.Paden Tomasello, T. A.Tu Anh Nguyen, R.Robin Algayres, Y.Yossi Adi, J.Jade Copet, E.Emmanuel Dupoux and A.Abdelrahman Mohamed. Do Coarser Units Benefit Cluster Prediction-Based Speech Pre-Training? 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2023) ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) Ixia-Ialyssos, Greece IEEE 2023 HAL DOI
• 31 inproceedingsM.Matthieu Futeral, C.Cordelia Schmid, I.Ivan Laptev, B.Benoît Sagot and R.Rachel Bawden. Tackling Ambiguity with Images: Improved Multimodal Machine Translation and Contrastive Evaluation.Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)Toronto, CanadaJuly 2023, 5394–5413HALDOI
• 32 inproceedingsI.Itai Gat, F.Felix Kreuk, T.Tu Anh Nguyen, A.Ann Lee, J.Jade Copet, G.Gabriel Synnaeve, E.Emmanuel Dupoux and Y.Yossi Adi. Augmentation Invariant Discrete Representation for Generative Spoken Language Modeling.20th International Conference on Spoken Language Translation (IWSLT 2023)Toronto, CanadaAssociation for Computational LinguisticsJuly 2023, 465-477HALDOIback to text
• 33 inproceedingsE. H.El Haff Karim, W.Wissam Antoun, F.Florence Le Ber and V.Véronique Pitchon. Reconnaissance des entités nommées pour l'analyse des pharmacopées médiévales.EGC 2023 - Extraction et Gestion des ConnaissancesLyon, FranceJanuary 2023HAL
• 34 inproceedingsT.Tom Kocmi, E.Eleftherios Avramidis, R.Rachel Bawden, O.Ondřej Bojar, A.Anton Dvorkovich, C.Christian Federmann, M.Mark Fishel, M.Markus Freitag, T.Thamme Gowda, R.Roman Grundkiewicz, B.Barry Haddow, P.Philipp Koehn, B.Benjamin Marie, C.Christof Monz, M.Makoto Morishita, K.Kenton Murray, M.Masaaki Nagata, T.Toshiaki Nakazawa, M.Martin Popel, M.Maja Popović, M.Mariya Shmatova and J.Jun Suzuki. Findings of the 2023 Conference on Machine Translation (WMT23): LLMs Are Here But Not Quite There Yet.Proceedings of the Eighth Conference on Machine TranslationWMT23 - Eighth Conference on Machine TranslationSingapore, Singapore2023, 198--216HALback to text
• 35 inproceedingsS.Simon Meoni, T.Theo Ryffel and E.Eric Villemonte de La Clergerie. Large Language Models as Instructors: A Study on Multilingual Clinical Entity Extraction.The 22nd Workshop on Biomedical Natural Language Processing and BioNLP Shared TasksThe 22nd Workshop on Biomedical Natural Language Processing and BioNLP Shared TasksToronto, CanadaAssociation for Computational LinguisticsJuly 2023, 178-190HALDOIback to text
• 36 inproceedingsM.Mariana Neves, A.Antonio Jimeno Yepes, A.Aurélie Névéol, R.Rachel Bawden, G. M.Giorgio Maria Di Nunzio, R.Roland Roller, P.Philippe Thomas, F.Federica Vezzani, M. V.Maika Vicente Navarro, L.Lana Yeganova, D.Dina Wiemann and C.Cristian Grozea. Findings of the WMT 2023 Biomedical Translation Shared Task: Evaluation of ChatGPT 3.5 as a Comparison System.Proceedings of the Eighth Conference on Machine TranslationWMT23 - Eighth Conference on Machine TranslationSingapore, Singapore2023, 43--54HALback to text
• 37 inproceedingsT. A.Tu Anh Nguyen, W.-N.Wei-Ning Hsu, A.Antony d'Avirro, B.Bowen Shi, I.Itai Gat, M.Maryam Fazel-Zarani, T.Tal Remez, J.Jade Copet, G.Gabriel Synnaeve, M.Michael Hassid, F.Felix Kreuk, Y.Yossi Adi and E.Emmanuel Dupoux. Expresso: A Benchmark and Analysis of Discrete Expressive Speech Resynthesis.INTERSPEECH 2023 - 24th Annual Conference of the International Speech Communication AssociationDublin, IrelandISCAAugust 2023, 4823-4827HALDOIback to text
• 38 inproceedingsA.Arij Riabi, M.Menel Mahamdi and D.Djamé Seddah. Enriching the NArabizi Treebank: A Multifaceted Approach to Supporting an Under-Resourced Language.Proceedings of the 17th Linguistic Annotation Workshop (LAW-XVII)Toronto, CanadaAssociation for Computational LinguisticsJuly 2023, 266-278HALDOIback to textback to text
• 39 inproceedingsJ. C.José Carlos Rosales Núñez, D.Djamé Seddah and G.Guillaume Wisniewski. Multi-way Variational NMT for UGC: Improving Robustness in Zero-shot Scenarios via Mixture Density Networks.NoDaLiDa 2023 - 24th Nordic Conference on Computational LinguisticsTorshavn, Faroe IslandsMay 2023HALback to text
• 40 inproceedingsS.Sonal Sannigrahi and R.Rachel Bawden. Investigating Lexical Sharing in Multilingual Machine Translation for Indian Languages.Proceedings of the 24th Annual Conference of the European Association for Machine TranslationEAMT 2023 - 24th Annual Conference of the European Association for Machine TranslationTampere, FinlandMay 2023HALback to textback to text
• 41 inproceedingsV.Valentin Taillandier, D.Dieuwke Hupkes, B.Benoît Sagot, E.Emmanuel Dupoux and P.Paul Michel. Neural Agents Struggle to Take Turns in Bidirectional Emergent Communication.ICLR 2023 - 11th International Conference on Learning RepresentationKigali, RwandaMay 2023HAL

National peer-reviewed Conferences

• 42 inproceedingsW.Wissam Antoun, V.Virginie Mouilleron, B.Benoît Sagot and D.Djamé Seddah. Towards a Robust Detection of Language Model-Generated Text: Is ChatGPT that easy to detect?Actes de CORIA-TALN 2023. Actes de la 30e Conférence sur le Traitement Automatique des Langues Naturelles (TALN), volume 1 : travaux de recherche originaux -- articles longs18e Conférence en Recherche d'Information et Applications -- 16e Rencontres Jeunes Chercheurs en RI -- 30e Conférence sur le Traitement Automatique des Langues Naturelles -- 25e Rencontre des Étudiants Chercheurs en Informatique pour le Traitement Automatique des LanguesParis, FranceATALA2023, 14-27HALback to textback to text
• 43 inproceedingsN.Niyati Bafna, C.Cristina España-Bonet, J.Josef van Genabith, B.Benoît Sagot and R.Rachel Bawden. Cross-lingual Strategies for Low-resource Language Modeling: A Study on Five Indic Dialects.Actes de CORIA-TALN 2023. Actes de la 30e Conférence sur le Traitement Automatique des Langues Naturelles (TALN), volume 1 : travaux de recherche originaux -- articles longs18e Conférence en Recherche d'Information et Applications -- 16e Rencontres Jeunes Chercheurs en RI -- 30e Conférence sur le Traitement Automatique des Langues Naturelles -- 25e Rencontre des Étudiants Chercheurs en Informatique pour le Traitement Automatique des LanguesParis, FranceATALA2023, 28-42HALback to textback to text
• 44 inproceedingsM.Maud Bénard, A.Alexandra Mestivier, N.Natalie Kubler, L.Lichao Zhu, R.Rachel Bawden, E.Eric De La Clergerie, L.Laurent Romary, M.Mathilde Huguin, J.-F.Jean-François Nominé, Z.Ziqian Peng and F.François Yvon. MaTOS : Machine Translation for Open Science.Actes de CORIA-TALN 2023. Actes de l'atelier "Analyse et Recherche de Textes Scientifiques" (ARTS)@TALN 202318e Conférence en Recherche d'Information et Applications -- 16e Rencontres Jeunes Chercheurs en RI -- 30e Conférence sur le Traitement Automatique des Langues Naturelles -- 25e Rencontre des Étudiants Chercheurs en Informatique pour le Traitement Automatique des LanguesParis, FranceATALA2023, 8-15HALback to text
• 45 inproceedingsS.Simon Meoni, T.Théo Ryffel and E.Eric De La Clergerie. Annotation d'entités cliniques en utilisant les Larges Modèles de Langue.Actes de CORIA-TALN 2023. Actes de la 30e Conférence sur le Traitement Automatique des Langues Naturelles (TALN), volume 1 : travaux de recherche originaux -- articles longs18e Conférence en Recherche d'Information et Applications -- 16e Rencontres Jeunes Chercheurs en RI -- 30e Conférence sur le Traitement Automatique des Langues Naturelles -- 25e Rencontre des Étudiants Chercheurs en Informatique pour le Traitement Automatique des LanguesParis, FranceATALA2023, 190-203HALback to text
• 46 inproceedingsS.Simon Meoni, R.Rian Touchent and E.Eric De La Clergerie. Passe ta pharma d'abord !Actes de CORIA-TALN 2023. Actes du Défi Fouille de Textes@TALN202318e Conférence en Recherche d'Information et Applications -- 16e Rencontres Jeunes Chercheurs en RI -- 30e Conférence sur le Traitement Automatique des Langues Naturelles -- 25e Rencontre des Étudiants Chercheurs en Informatique pour le Traitement Automatique des LanguesParis, FranceATALA2023, 68-76HAL
• 47 inproceedingsL.Lydia Nishimwe. Lexical normalisation of user-generated content on social media.Actes de CORIA-TALN 2023. Actes des 16e Rencontres Jeunes Chercheurs en RI (RJCRI) et 25e Rencontre des Étudiants Chercheurs en Informatique pour le Traitement Automatique des Langues (RÉCITAL)18e Conférence en Recherche d'Information et Applications -- 16e Rencontres Jeunes Chercheurs en RI -- 30e Conférence sur le Traitement Automatique des Langues Naturelles -- 25e Rencontre des Étudiants Chercheurs en Informatique pour le Traitement Automatique des LanguesActes des 16e Rencontres Jeunes Chercheurs en RI (RJCRI) et 25e Rencontre des Étudiants Chercheurs en Informatique pour le Traitement Automatique des Langues (RÉCITAL)Paris, FranceATALA2023, 160-183HALback to textback to text
• 48 inproceedingsL.Lionel Tadonfouet Tadjou, E.Eric De La Clergerie, F.Fabrice Bourge and T.Tiphaine Marie. Constitution de sous-fils de conversations d'emails.Actes de CORIA-TALN 2023. Actes de la 18e Conférence en Recherche d'Information et Applications (CORIA)18e Conférence en Recherche d'Information et Applications -- 16e Rencontres Jeunes Chercheurs en RI -- 30e Conférence sur le Traitement Automatique des Langues Naturelles -- 25e Rencontre des Étudiants Chercheurs en Informatique pour le Traitement Automatique des LanguesParis, FranceATALA2023, 157-171HAL
• 49 inproceedingsR.Rian Touchent, L.Laurent Romary and E.Eric De La Clergerie. CamemBERT-bio : Un modèle de langue français savoureux et meilleur pour la santé.Actes de CORIA-TALN 2023. Actes de la 30e Conférence sur le Traitement Automatique des Langues Naturelles (TALN), volume 1 : travaux de recherche originaux – articles longs18e Conférence en Recherche d'Information et Applications 16e Rencontres Jeunes Chercheurs en RI 30e Conférence sur le Traitement Automatique des Langues Naturelles 25e Rencontre des Étudiants Chercheurs en Informatique pour le Traitement Automatique des LanguesParis, FranceATALA2023, 323-334HALback to text
• 50 inproceedingsY.You Zuo, K.Kim Gerdes, H.Houda Mouzoun, S.Samir Ghamri Doudane and B.Benoît Sagot. Exploring Data-Centric Strategies for French Patent Classification: A Baseline and Comparisons.Actes de CORIA-TALN 2023. Actes de la 30e Conférence sur le Traitement Automatique des Langues Naturelles (TALN), volume 1 : travaux de recherche originaux -- articles longs18e Conférence en Recherche d'Information et Applications -- 16e Rencontres Jeunes Chercheurs en RI -- 30e Conférence sur le Traitement Automatique des Langues Naturelles -- 25e Rencontre des Étudiants Chercheurs en Informatique pour le Traitement Automatique des LanguesParis, FranceATALA2023, 349-365HALback to text

Conferences without proceedings

• 51 inproceedingsH.Hugo Scheithauer, S.Sarah Bénière, J.-P.Jean-Philippe Moreux and L.Laurent Romary. DataCatalogue : rétro-structuration automatique des catalogues de vente.Webinaire Culture-InriaParis, FranceNovember 2023HALback to text
• 52 inproceedingsC.Chahan Vidal-Gorène, J.-B.Jean-Baptiste Camps and T.Thibault Clérice. Synthetic lines from historical manuscripts: an experiment using GAN and style transfer.Visual Processing of Digital Manuscripts: Workflows, Pipelines, Best Practices. ICIAP 2023 Workshops. ICIAP 2023.Udine, ItalySeptember 2023HALback to text

Doctoral dissertations and habilitation theses

• 53 thesisJ.José Rosales Núñez. Machine Translation of User-Generated Contents : an Evaluation of Neural Translation Systems under Zero-shot Conditions.Université Paris-SaclayOctober 2023HALback to text

Reports & preprints

• 54 miscW.Wissam Antoun, D.Djamé Seddah and B.Benoît Sagot. From Text to Source: Results in Detecting Large Language Model-Generated Content.September 2023HAL
• 55 miscN.Niyati Bafna, C.Cristina España-Bonet, J.Josef van Genabith, B.Benoît Sagot and R.Rachel Bawden. A Simple Method for Unsupervised Bilingual Lexicon Induction for Data-Imbalanced, Closely Related Language Pairs.May 2023HALback to text
• 56 miscB.Beatrice Biancardi, M.Mathieu Chollet and C.Chloé Clavel. Introducing the 3MT_French Dataset to Investigate the Timing of Public Speaking Judgements.October 2022HALDOI
• 57 misc A.Alix Chagué and F.Floriane Chiffoleau. ATR: What can eScriptorium do for you? September 2023 HAL back to text
• 58 miscA.Alix Chagué and F.Floriane Chiffoleau. What can you do next? Choice of output and reuse of your transcription.September 2023HALback to text
• 59 miscA.Alix Chagué and T.Thibault Clérice. Données ouvertes, données propres, et autres vies : Testaments de Poilus et CREMMA.December 2023HALback to text
• 60 miscA.Alix Chagué and H.Hippolyte Souvay. Image Acquisition and Layout Analysis.September 2023HALback to text
• 61 miscF.Floriane Chiffoleau. TEI Publisher, a platform for sustainable digital editions.September 2023HALback to text
• 62 miscT.Thibault Clérice. Detecting Sexual Content at the Sentence Level in First Millennium Latin Texts.September 2023HALback to text
• 63 miscP.-A.Paul-Ambroise Duquenne, H.Holger Schwenk and B.Benoît Sagot. SONAR: Sentence-Level Multimodal and Language-Agnostic Representations.October 2023HALback to textback to text
• 64 miscN.Nathan Godey, E.Eric Villemonte de La Clergerie and B.Benoît Sagot. Headless Language Models: Learning without Predicting with Contrastive Weight Tying.September 2023HALback to text
• 65 misc N.Nathan Godey, E.Eric Villemonte de La Clergerie and B.Benoît Sagot. Is Anisotropy Inherent to Transformers? June 2023 HAL back to text
• 66 misc T. A.Tu Anh Nguyen, M. D.Maureen De Seyssel, R.Robin Algayres, P.Patricia Rozé, E.Ewan Dunbar and E.Emmanuel Dupoux. Are word boundaries useful for unsupervised language learning? 2022 HAL DOI back to text
• 67 reportY.Yannick Parmentier, S.Sylvain Pogodalla, R.Rachel Bawden, M.Matthieu Labeau and I.Iris Eshkol-Taravella. Procédure de diffusion des publications de l'ATALA sur les archives ouvertes.ATALASeptember 2023, 17HAL
• 68 miscA.Ariane Pinche, T.Thibault Clérice, A.Alix Chagué, J.-B.Jean-Baptiste Camps, M.Malamatenia Vlachou-Efstathiou, M.Matthias Gille Levenson, O.Olivier Brisville-Fertin, F.Federico Boschetti, F.Franz Fischer, M.Michael Gervers, A.Agnès Boutreux, A.Avery Manton, S.Simon Gabay, P.Patricia O'Connor, W.Wouter Haverals, M.Mike Kestemont and C.Caroline Vandyck. CATMuS-Medieval: Consistent Approaches to Transcribing ManuScripts: A generalized set of guidelines and models for Latin scripts from Middle Ages (8th--16th century).December 2023HALback to text
• 69 miscC. A.C. Annemieke Romein, T.Tobias Hodel, F.Femke Gordijn, J.Joris Zundert, A.Alix Chagué, M. V.Milan Van Lange, H. S.Helle Strandgaard Jensen, A.Andy Stauder, J.Jake Purcell, M.Melissa Terras, P. v.Pauline van Den Heuvel, C.Carlijn Keijzer, A.Achim Rabus, C.Chantal Sitaram, A.Aakriti Bhatia, K.Katrien Depuydt, M. A.Mary Aderonke Afolabi-Adeolu, A.Anastasiia Anikina, E.Elisa Bastianello, L. V.Lukas Vincent Benzinger, A.Arno Bosse, D.David Brown, A.Ash Charlton, A. N.André Nilsson Dannevig, K. V.Klaas Van Gelder, S. C.Sabine C.P.J. Go, M. J.Marcus J.C. Goh, S.Silvia Gstrein, S.Sewa Hasan, S. v.Stefan von Der Heide, M.Maximilian Hindermann, D.Dorothee Huff, I.Ineke Huysman, A.Ali Idris, L.Liesbeth Keijzer, S.Simon Kemper, S.Sanne Koenders, E.Erika Kuijpers, L.Lisette Rønsig Larsen, S.Sven Lepa, T.Tommy Link, A. V.Annelies Van Nispen, J.Joe Nockels, L.Laura Noort, J. J.Joost Johannes Oosterhuis, V.Vivien Popken, M.María Estrella Puertollano, J.Joosep Puusaag, A.Ahmed Sheta, L.Lex Stoop, E.Ebba Strutzenbladh, N. v.Nicoline van Der Sijs, J. P.Jan Paul van Der Spek, B. B.Barry Benaissa Trouw, G.Geertrui van Synghel, V.Vladimir Vučković, H.Heleen Wilbrink, S.Sonia Weiss, D. J.David Joseph Wrisley and R.Riet Zweistra. Exploring Data Provenance in Handwritten Text Recognition Infrastructure: Sharing and Reusing Ground Truth Data, Referencing Models, and Acknowledging Contributions. Starting the Conversation on How We Could Get It Done.2023HALDOIback to text
• 70 miscT. L.Teven Le Scao, A.Angela Fan, C.Christopher Akiki, E.Ellie Pavlick, S.Suzana Ilić, D.Daniel Hesslow, R.Roman Castagné, A. S.Alexandra Sasha Luccioni, F.François Yvon, M.Matthias Gallé, J.Jonathan Tow, A. M.Alexander M. Rush, S.Stella Biderman, A.Albert Webson, P. S.Pawan Sasanka Ammanamanchi, T.Thomas Wang, B.Benoît Sagot, N.Niklas Muennighoff, A. V.Albert Villanova del Moral, O.Olatunji Ruwase, R.Rachel Bawden, S.Stas Bekman, A.Angelina Mcmillan-Major, I.Iz Beltagy, H.Huu Nguyen, L.Lucile Saulnier, S.Samson Tan, P.Pedro Ortiz Suarez, V.Victor Sanh, H.Hugo Laurençon, Y.Yacine Jernite, J.Julien Launay, M.Margaret Mitchell, C.Colin Raffel, A.Aaron Gokaslan, A.Adi Simhi, A.Aitor Soroa, A. F.Alham Fikri Aji, A.Amit Alfassy, A.Anna Rogers, A. K.Ariel Kreisberg Nitzav, C.Canwen Xu, C.Chenghao Mou, C.Chris Emezue, C.Christopher Klamm, C.Colin Leong, D.Daniel van Strien, D. I.David Ifeoluwa Adelani, D.Dragomir Radev, E. G.Eduardo González Ponferrada, E.Efrat Levkovizh, E.Ethan Kim, E. B.Eyal Bar Natan, F.Francesco de Toni, G.Gérard Dupont, G.Germán Kruszewski, G.Giada Pistilli, H.Hady Elsahar, H.Hamza Benyamina, H.Hieu Tran, I.Ian Yu, I.Idris Abdulmumin, I.Isaac Johnson, I.Itziar Gonzalez-Dios, J.Javier de la Rosa, J.Jenny Chim, J.Jesse Dodge, J.Jian Zhu, J.Jonathan Chang, J.Jörg Frohberg, J.Joseph Tobing, J.Joydeep Bhattacharjee, K.Khalid Almubarak, K.Kimbo Chen, K.Kyle Lo, L.Leandro von Werra, L.Leon Weber, L.Long Phan, L. B.Loubna Ben Allal, L.Ludovic Tanguy, M.Manan Dey, M. R.Manuel Romero Muñoz, M.Maraim Masoud, M.María Grandury, M.Mario Šaško, M.Max Huang, M.Maximin Coavoux, M.Mayank Singh, M.-J. T.Mike Tian-Jian Jiang, M. C.Minh Chien Vu, M. A.Mohammad A. Jauhar, M.Mustafa Ghaleb, N.Nishant Subramani, N.Nora Kassner, N.Nurulaqilla Khamis, O.Olivier Nguyen, O.Omar Espejel, O.Ona de Gibert, P.Paulo Villegas, P.Peter Henderson, P.Pierre Colombo, P.Priscilla Amuok, Q.Quentin Lhoest, R.Rheza Harliman, R.Rishi Bommasani, R. L.Roberto Luis López, R.Rui Ribeiro, S.Salomey Osei, S.Sampo Pyysalo, S.Sebastian Nagel, S.Shamik Bose, S. H.Shamsuddeen Hassan Muhammad, S.Shanya Sharma, S.Shayne Longpre, S.Somaieh Nikpoor, S.Stanislav Silberberg, S.Suhas Pai, S.Sydney Zink, T. T.Tiago Timponi Torrent, T.Timo Schick, T.Tristan Thrush, V.Valentin Danchev, V.Vassilina Nikoulina, V.Veronika Laippala, V.Violette Lepercq, V.Vrinda Prabhu, Z.Zaid Alyafeai, Z.Zeerak Talat, A.Arun Raja, B.Benjamin Heinzerling, C.Chenglei Si, E.Elizabeth Salesky, S. J.Sabrina J. Mielke, W. Y.Wilson Y. Lee, A.Abheesht Sharma, A.Andrea Santilli, A.Antoine Chaffin, A.Arnaud Stiegler, D.Debajyoti Datta, E.Eliza Szczechla, G.Gunjan Chhablani, H.Han Wang, H.Harshit Pandey, H.Hendrik Strobelt, J. A.Jason Alan Fries, J.Jos Rozen, L.Leo Gao, L.Lintang Sutawika, M. S.M Saiful Bari, M. S.Maged S. Al-Shaibani, M.Matteo Manica, N.Nihal Nayak, R.Ryan Teehan, S.Samuel Albanie, S.Sheng Shen, S.Srulik Ben-David, S. H.Stephen H. Bach, T.Taewoon Kim, T.Tali Bers, T.Thibault Fevry, T.Trishala Neeraj, U.Urmish Thakker, V.Vikas Raunak, X.Xiangru Tang, Z.-X.Zheng-Xin Yong, Z.Zhiqing Sun, S.Shaked Brody, Y.Yallow Uri, H.Hadar Tojarieh, A.Adam Roberts, H. W.Hyung Won Chung, J.Jaesung Tae, J.Jason Phang, O.Ofir Press, C.Conglong Li, D.Deepak Narayanan, H.Hatim Bourfoune, J.Jared Casper, J.Jeff Rasley, M.Max Ryabinin, M.Mayank Mishra, M.Minjia Zhang, M.Mohammad Shoeybi, M.Myriam Peyrounette, N.Nicolas Patry, N.Nouamane Tazi, O.Omar Sanseviero, P.Patrick von Platen, P.Pierre Cornette, P. F.Pierre François Lavallée, R.Rémi Lacroix, S.Samyam Rajbhandari, S.Sanchit Gandhi, S.Shaden Smith, S.Stéphane Requena, S.Suraj Patil, T.Tim Dettmers, A.Ahmed Baruwa, A.Amanpreet Singh, A.Anastasia Cheveleva, A.-L.Anne-Laure Ligozat, A.Arjun Subramonian, A.Aurélie Névéol, C.Charles Lovering, D.Dan Garrette, D.Deepak Tunuguntla, E.Ehud Reiter, E.Ekaterina Taktasheva, E.Ekaterina Voloshina, E.Eli Bogdanov, G. I.Genta Indra Winata, H.Hailey Schoelkopf, J.-C.Jan-Christoph Kalo, J.Jekaterina Novikova, J. Z.Jessica Zosa Forde, J.Jordan Clive, J.Jungo Kasai, K.Ken Kawamura, L.Liam Hazan, M.Marine Carpuat, M.Miruna Clinciu, N.Najoung Kim, N.Newton Cheng, O.Oleg Serikov, O.Omer Antverg, O.Oskar van der Wal, R.Rui Zhang, R.Ruochen Zhang, S.Sebastian Gehrmann, S.Shani Pais, T.Tatiana Shavrina, T.Thomas Scialom, T.Tian Yun, T.Tomasz Limisiewicz, V.Verena Rieser, V.Vitaly Protasov, V.Vladislav Mikhailov, Y.Yada Pruksachatkun, Y.Yonatan Belinkov, Z.Zachary Bamberger, Z.Zdeněk Kasner, A.Alice Rueda, A.Amanda Pestana, A.Amir Feizpour, A.Ammar Khan, A.Amy Faranak, A.Ana Santos, A.Anthony Hevia, A.Antigona Unldreaj, A.Arash Aghagol, A.Arezoo Abdollahi, A.Aycha Tammour, A.Azadeh Hajihosseini, B.Bahareh Behroozi, B.Benjamin Ajibade, B.Bharat Saxena, C. M.Carlos Muñoz Ferrandis, D.Danish Contractor, D.David Lansky, D.Davis David, D.Douwe Kiela, D. A.Duong A. Nguyen, E.Edward Tan, E.Emi Baylor, E.Ezinwanne Ozoani, F.Fatima Mirza, F.Frankline Ononiwu, H.Habib Rezanejad, H.Hessie Jones, I.Indrani Bhattacharya, I.Irene Solaiman, I.Irina Sedenko, I.Isar Nejadgholi, J.Jesse Passmore, J.Josh Seltzer, J. B.Julio Bonis Sanz, L.Livia Dutra, M.Mairon Samagaio, M.Maraim Elbadri, M.Margot Mieskes, M.Marissa Gerchick, M.Martha Akinlolu, M.Michael Mckenna, M.Mike Qiu, M.Muhammed Ghauri, M.Mykola Burynok, N.Nafis Abrar, N.Nazneen Rajani, N.Nour Elkott, N.Nour Fahmy, O.Olanrewaju Samuel, R.Ran An, R.Rasmus Kromann, R.Ryan Hao, S.Samira Alizadeh, S.Sarmad Shubber, S.Silas Wang, S.Sourav Roy, S.Sylvain Viguier, T.Thanh Le, T.Tobi Oyebade, T.Trieu Le, Y.Yoyo Yang, Z.Zach Nguyen, A. R.Abhinav Ramesh Kashyap, A.Alfredo Palasciano, A.Alison Callahan, A.Anima Shukla, A.Antonio Miranda-Escalada, A.Ayush Singh, B.Benjamin Beilharz, B.Bo Wang, C.Caio Brito, C.Chenxi Zhou, C.Chirag Jain, C.Chuxin Xu, C.Clémentine Fourrier, D. L.Daniel León Periñán, D.Daniel Molano, D.Dian Yu, E.Enrique Manjavacas, F.Fabio Barth, F.Florian Fuhrimann, G.Gabriel Altay, G.Giyaseddin Bayrak, G.Gully Burns, H. U.Helena U. Vrabec, I.Imane Bello, I.Ishani Dash, J.Jihyun Kang, J.John Giorgi, J.Jonas Golde, J. D.Jose David Posada, K. R.Karthik Rangasai Sivaraman, L.Lokesh Bulchandani, L.Lu Liu, L.Luisa Shinzato, M. H.Madeleine Hahn de Bykhovetz, M.Maiko Takeuchi, M.Marc Pàmies, M. A.Maria A Castillo, M.Marianna Nezhurina, M.Mario Sänger, M.Matthias Samwald, M.Michael Cullan, M.Michael Weinberg, M.Michiel de Wolf, M.Mina Mihaljcic, M.Minna Liu, M.Moritz Freidank, M.Myungsun Kang, N.Natasha Seelam, N.Nathan Dahlberg, N. M.Nicholas Michio Broad, N.Nikolaus Muellner, P.Pascale Fung, P.Patrick Haller, R.Ramya Chandrasekhar, R.Renata Eisenberg, R.Robert Martin, R.Rodrigo Canalli, R.Rosaline Su, R.Ruisi Su, S.Samuel Cahyawijaya, S.Samuele Garda, S. S.Shlok S Deshmukh, S.Shubhanshu Mishra, S.Sid Kiblawi, S.Simon Ott, S.Sinee Sang-Aroonsiri, S.Srishti Kumar, S.Stefan Schweter, S.Sushil Bharati, T.Tanmay Laud, T.Théo Gigant, T.Tomoya Kainuma, W.Wojciech Kusa, Y.Yanis Labrak, Y. S.Yash Shailesh Bajaj, Y.Yash Venkatraman, Y.Yifan Xu, Y.Yingxin Xu, Y.Yu Xu, Z.Zhe Tan, Z.Zhongli Xie, Z.Zifan Ye, M.Mathilde Bras, Y.Younes Belkada and T.Thomas Wolf. BLOOM: A 176B-Parameter Open-Access Multilingual Language Model.November 2023HALback to textback to textback to text