2023Activity reportProject-TeamMULTISPEECH

3.1.1 Axis 1.1 — Integrating domain knowledge

State-of-the-art methods for speech and audio processing are based on discriminative neural networks trained for the targeted task. This paradigm faces major limitations: lack of interpretability, large data requirements and inability to generalize to unseen classes or tasks. Our approach is to combine the representation power of deep learning with our acoustic expertise to obtain smaller generative models describing the probability distribution of speech and audio signals. Particular attention will be paid to designing physically-motivated input layers, output layers, and unsupervised representations that capture complex-valued, multi-scale spectro-temporal dependencies. Given these models, we derive computationally efficient inference algorithms that address the above limitations. We also explore the integration of deep learning with symbolic reasoning and common-sense knowledge to increase the generalization ability of deep models.

3.1.2 Axis 1.2 — Learning from little/no labeled data

While supervised learning from fully labeled data is economically costly, unlabeled data are inexpensive but provide intrinsically less information. Our goal is to learn representations that disentangle the attributes of speech by equipping the unsupervised representation learning methods above with supervised branches exploiting the available labels and supervisory signals, and with multiple adversarial branches overcoming the usual limitations of adversarial.

3.1.3 Axis 1.3 — Preserving privacy

To preserve privacy, speech must be transformed to hide the users’ identity and other privacy-sensitive attributes (e.g., accent, health status) while leaving intact those attributes which are required for the task (e.g., phonetic content for automatic speech recognition) and preserving the data variability for training purposes. We develop strong attacks to evaluate privacy. We also seek to hide personal identifiers and privacy-sensitive attributes in the linguistic content, focusing on their robust extraction and replacement from speech signals.

3.2.1 Axis 2.1 — Linguistic speech content

Speech recognition is the main means to extract linguistic information from speech. Although it is a mature research area, performance drops in real-world environments pursue the development of speech enhancement and source separation methods to effectively improve robustness in such real-world scenarios. Semantic content analysis is required to interpret the spoken message. The challenges include learning from little real data, quickly adapting to new topics, and robustness to speech recognition errors. The detection and classification of hate speech in social media videos will also be considered as a benchmark, thereby extending the work on text-only detection. Finally, we also consider extracting phonetic and prosodic information to study the categorization of speech sounds and certain aspects of prosody by learners of a foreign language.

3.2.2 Axis 2.2 — Speaker identity and states

Speaker identity is required for the personalization of human-computer interaction. Speaker recognition and diarization are still challenging in real-world conditions. The speaker states that we aim to recognize include emotion and stress, which can be used to adapt the interaction in real time.

3.2.3 Axis 2.3 — Speech environment information

We develop audio event detection methods that exploit both strongly/weakly labeled and unlabeled data, operate in real-world conditions, can discover new events, and provide a semantic interpretation. Modeling the temporal, spatial and logical structure of ambient sound scenes over a long duration is also considered.

3.3.1 Axis 3.1 - Multimodality modeling and analysis

The study of multimodality concerns the interaction between modalities, their fusion, coordination and synchronization for a single speaker, as well as their synchronization across the speakers in a conversation. We focus on audiovisual speech enhancement to improve the intelligibility and quality of noisy speech by considering the speaker’s lip movements. We also consider the semi/weakly/self-supervised learning methods for multimodal data to obtain interpretable representations that disentangle in each modality the attributes related to linguistic and semantic content, emotion, reaction, etc. We also study the contribution of each modality to the intelligibility of spoken communication.

3.3.2 Axis 3.2 - Multimodal speech generation

Multimodal speech generation refers to articulatory, acoustic, and audiovisual speech synthesis techniques which output one or more facets. Articulatory speech synthesis relies on 2D and 3D modeling of the dynamics of the vocal tract from real-time MRI (rtMRI) data. We consider the generation of the full vocal tract, from the vocal folds to the lips, first in 2D then in 3D. This comprises the generation of the face and the prediction of the glottis opening. We also consider audiovisual speech synthesis. Both the animation of the lower part of the face related to speech and of the upper part related to the facial expressions are considered, and development continues towards a multilingual talking head. We investigate further the modeling of expressivity for both audio-only and audiovisual speech synthesis, for a better control of expressivity, where we consider several disentangled attributes at the same time.

3.3.3 Axis 3.3 — Interaction

Interaction is a new field of research for our project-team that we will approach gradually. We start by studying the multimodal components (prosody, facial expressions, gestures) used during interaction, both by the speaker and by the listener, where the goal is to simultaneously generate speech and gestures by the speaker, and generating regulatory gestures for the listener. We will introduce different dialog bricks progressively: Spoken language understanding, Dialog management, and Natural language generation. Dialog will be considered in a multimodal context (gestures, emotional states of the interlocutor, etc.) and we will break the classical dialog management scheme to dynamically account for the interlocutor’s evolution during the speaker’s response.

7.1.1 ASTALI

• Name:
  Automatic Speech-Text Alignment Software
• Keyword:
  Speech-text alignment
• Functional Description:
  ASTALI is a software for aligning a speech signal with its corresponding orthographic transcription (given in simple text file for short audio signals or in .trs files as generated by transcriber for longer speech signals). Using a phonetic lexicon and automatic grapheme-to-phoneme converters, all the potential sequences of phones corresponding to the text are generated. Then, using acoustic models, the tool finds the best phone sequence and provides the boundaries at the phone and at the word levels. The web application makes the service easy to use, without requiring any software downloading.
• News of the Year:
  We migrated the software to a more robust server and fixed a few bugs. We also reorganized the code hierarchy under Gitlab to facilitate easier updates in the future.
• URL:
  http://­astali.­loria.­fr/
• Contact:
  Theo Biasutto-Lervat

7.1.2 Voice Transformer 2

• Keywords:
  Speech, Privacy
• Scientific Description:

  The implemented method is inspired from the speaker anonymisation method proposed in [Fan+19], which performs voice conversion based on x-vectors [Sny+18], a fixed-length representation of speech signals that form the basis of state-of-the-art speaker verification systems. We have brought several improvements to this method such as pitch transformation, and new design choices for x-vector selection

  [Fan+19] F. Fang, X. Wang, J. Yamagishi, I. Echizen, M. Todisco, N. Evans, and J.F. Bonastre. “Speaker Anonymization Using x-vector and Neural Waveform Models”. In: Proceedings of the 10th ISCA Speech Synthesis Workshop. 2019, pp. 155–160. [Sny+18] D. Snyder, D. Garcia-Romero, G. Sell, D. Povey, and S. Khudanpur. “X-vectors: Robust DNN embeddings for speaker recognition”. In: Proceedings of ICASSP 2018 - 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2018, pp. 5329–5333.

• Functional Description:
  Voice Transformer increases the privacy of users of voice interfaces by converting their voice into another person’s voice without modifying the spoken message. It ensures that any information extracted from the transformed voice can hardly be traced back to the original speaker, as validated through state-of-the-art biometric protocols, and it preserves the phonetic information required for human labelling and training of speech-to-text models.
• News of the Year:
  A transfer contract was signed with the startup Nijta.
• Contact:
  Nathalie Vauquier
• Participants:
  Brij Mohan Lal Srivastava, Nathalie Vauquier, Emmanuel Vincent, Marc Tommasi

7.1.3 Web Multimodal Annotation

• Name:
  Web-based Multimodal Data Annotation
• Keywords:
  Annotation tool, Audio segmentation, Audiovisual, Web Application, Speech
• Scientific Description:
  Web Multimodal Annotation offers the reading, segmentation, and annotation of audio and video files, while providing a visualization of the associated waveform and spectrogram. Annotations can be manual or automatic on audio or video type data. In addition, the tool allows for manual correction of annotations, whether they have been added automatically or manually by annotators. To ensure extensive compatibility, the application uses the TextGrid (Praat) format.
• Functional Description:
  Web Multimodal Annotation is a web application that was designed to manually segment and label audio or video data and to visualize the audio data. In addition, this web application also allows for multi-level labeling and collaboration between multiple annotators. This web application can be useful to annotators, researchers in the field of multimodal speech or gestures.
• Release Contributions:
  The software is deployed on a web server. This version is quite mature, but requires evaluation by users.
• News of the Year:
  During this year, we implemented an automatic alignment system for the automatic labeling of audio or video data. In addition, we simplified the deployment of our Web Multimodal Annotation solution on servers, particularly by developing Docker images. We also worked on improving the performance and speed of this tool, significantly reducing the loading times required for the visualization of audio and video data. To overcome challenges related to the size of audio and video files, we integrated a streaming system. To ensure compatibility, we added the TextGrid format, which is widely used in the fields of linguistics, phonetics, and automatic language processing research.
• Contact:
  Slim Ouni
• Participants:
  Slim Ouni, Sofiane Azzouz
• Partners:
  CNRS, Université de Lorraine

7.1.4 VAC

• Name:
  Virtual Assistant Creator
• Keywords:
  Artificial intelligence, Audio signal processing, Speech processing
• Functional Description:
  VAC is a framework for creating interconnectable components for speech and natural language processing. It also offers a set of standard components for creating virtual assistants such as noise reduction, speech recognition and synthesis, and natural language understanding.
• Release Contributions:
  Middleware for communication between components - Speech processing components - Access to microphone and speakers (via python-sounddevice and libportaudio2) - Denoising (ConvTasNet model) - Activity detection (via webrtcvad) - Speech recognition (K2/Sherpa zipformer model, NeMo ConformerTransducer model) - Speech synthesis (BalacoonTTS, FastPitch and HifiGAN from NeMo) - Natural language processing components - Text completion (via llama-cpp) - Chat (via llama-cpp) - Video processing components - Face detection (via dlib) - Facial landmark detection (via dlib) - Body detection (via opencv2) - Body pose estimation (via opencv2)
• URL:
  https://­gitlab.­inria.­fr/­multispeech/­vac
• Contact:
  Theo Biasutto-Lervat

7.1.5 PercEval

• Name:
  Perceptive Evaluation Platform
• Keywords:
  Evaluation, Speech processing
• Functional Description:
  PercEval offers users the ability to create surveys with multimedia content, publish them, and analyze their results.
• Release Contributions:
  - survey administration webpage - survey creation webpage - survey response webpage - survey analytics webpage
• URL:
  https://­perceval.­inria.­fr
• Contact:
  Theo Biasutto-Lervat

7.2.1 Virtual Assistant Creator

Voice assistants and voice interfaces have become a key technology, simplifying the user experience and increasing the accessibility of many applications, and their use will intensify in the coming years. However, this technology poses two major problems today: on the one hand, the quasi-hegemony of large technology companies (mainly American) raises questions about European digital sovereignty, and on the other hand, the commonly used client-server architecture raises privacy risks. To simultaneously address these two problems, we are currently developing an open-source platform for the creation of embedded virtual assistants.

This platform will provide the main speech processing and natural language processing bricks that are necessary to build a voice interface, such as denoising, recognition or speech synthesis. The generated assistant can be fully embedded in the users' terminal. The data being processed locally, we ensure the protection of their private lives. We envisage a multiplatform solution on PC (Windows, Linux, MacOS) as well as on mobile (Android, iOS).

During the second year of development, we mainly focused on the final design of the middleware API, relying on Protobuf and ZeroMQ, and on the implementation of the cross-platform python framework.

Moreover, we also add several key components into the python library, such as speech processing components (microphone and speaker access, voice activity detection with webrtcvad, speech enhancement with ConvTasNet, speech recognition with ZipFormer and ConformerTransducer, speech synthesis with BalacoonTTS, FastPitch and HifiGAN), video processing components (Face bounding-box and landmarks detection with dlib, Body bounding-box and pose estimation with opencv2) and text processing components (text completion and chat with llama-cpp).

7.3.1 MRI Dataset

The study of articulatory gestures has a wide range of applications, particularly in the study of speech production and automatic speech recognition. Real-time MRI data is particularly interesting because it offers good temporal resolution and complete coverage of the medio-sagittal section of the vocal tract. Existing MRI databases focus mainly on English, although the articulation of sounds depends on the language concerned. We therefore acquired MRI data for 10 native French speakers with no speech production or perception problems. A corpus consisting of sentences was used to ensure good phonetic coverage of French. Real-time MRI technology with a temporal resolution of 20 ms was used to acquire images of the vocal tract of the participants during speech production. The Sound was recorded simultaneously, denoised and temporally aligned with the images. The Speech was transcribed to obtain the phonetic segmentation. We also acquired static 3D MRI images of the French phonemes. In addition, we included annotations on spontaneous swallowing. Data are available together with the presentation of the database https://­www.­nature.­com/­articles/­s41597-021-01041-3.

8.1.1 Axis 1.1 — Integrating domain knowledge

8.1.2 Axis 1.2 - Learning from little/no labeled data

8.1.3 Axis 1.3 - Preserving privacy

Speech signals convey a lot of private information. To protect speakers, we pursued our investigation of x-vector based voice anonymization, which relies on splitting the speech signal into the speaker (x-vector), phonetic and pitch features and resynthesizing the signal with a different target x-vector. To reduce the amount of residual speaker information in the phonetic and pitch features, we explored the use of Laplacian noise 14 inspired from differential privacy. Pierre Champion defended his PhD 48, and we released the report 8 of the interdisciplinary Dagstuhl Seminar organized in 2022 on this topic.

8.2.1 Axis 2.1 — Linguistic speech content

8.2.2 Axis 2.2 — Speaker identity and states

8.2.3 Axis 2.3 — Speech in its environment

8.3.1 Axis 3.1 — Multimodality modeling and analysis

8.3.2 Axis 3.2 — Multimodal speech generation

8.3.3 Axis 3.3 — Interaction

9.1.1 Vivoka

• Company: Vivoka (France)
• Duration: Oct 2021 – Oct 2024
• Participants: Can Cui, Mostafa Sadeghi, Emmanuel Vincent
• Abstract: This contract funds the PhD of Can Cui on joint and embedded automatic speech separation, diarization and recognition for the generation of meeting minutes.

9.1.2 Meta AI

• Company: Meta AI (France)
• Duration: May 2022 – Apr 2025
• Participants: Robin San Roman, Antoine Deleforge, Romain Serizel
• Abstract: This CIFRE grant funds the PhD of Robin San Roman on self-supervised disentangled representation learning of audio data for compression and generation.

9.1.3 Orange Labs

• Company: Orange Labs (France)
• Duration: March 2023 – Feb 2026
• Participants: Taous Iatariene, Romain Serizel
• Abstract: This CIFRE grant funds the PhD of Taous Iatariene on sound source tracking.

10.1.1 Participation in other International Programs

10.2.1 Visits of international scientists

10.3.1 Horizon Europe

10.3.2 Other european programs/initiatives

ANR DEEP-PRIVACY

• Title:
  Distributed, Personalized, Privacy-Preserving Learning for Speech Processing
• Duration:
  Jan 2019 - Jun 2023
• Coordinator:
  Denis Jouvet until Aug 2022; Emmanuel Vincent from Sep 2022
• Partners:
  LIUM (Le Mans), Inria MAGNET (Lille), LIA (Avignon)
• Participants:
  Pierre Champion, Denis Jouvet, Hubert Nourtel, Emmanuel Vincent
• Abstract:
  The objective of the DEEP-PRIVACY project is to elaborate a speech transformation that hides the speaker identity for an easier sharing of speech data for training speech recognition models; and to investigate speaker adaptation and distributed training.

ANR ROBOVOX

• Title:
  Robust Vocal Identification for Mobile Security Robots
• Duration:
  Mar 2019 – Apr 2024
• Coordinator:
  Laboratoire d'informatique d'Avignon (LIA)
• Partners:
  Inria (Nancy), LIA (Avignon), A.I. Mergence (Paris)
• Participants:
  Antoine Deleforge, Sandipana Dowerah, Denis Jouvet, Romain Serizel
• Abstract:
  The aim of ROBOVOX project is to improve speaker recognition robustness for a security robot in real environment. Several aspects will be particularly considered such as ambient noise, reverberation and short speech utterances.

ANR BENEPHIDIRE

• Title:
  Stuttering: Neurology, Phonetics, Computer Science for Diagnosis and Rehabilitation
• Duration:
  Mar 2019 - Jun 2024
• Coordinator:
  Praxiling (Montpellier)
• Partners:
  Praxiling (Montpellier), LORIA (Nancy), INM (Montpellier), LiLPa (Strasbourg).
• Participants:
  Yves Laprie, Slim Ouni, Shakeel Ahmad Sheikh
• Abstract:
  The BENEPHIDIRE project brings together neurologists, speech-language pathologists, phoneticians, and computer scientists specializing in speech processing to investigate stuttering as a speech impairment and to develop techniques for diagnosis and rehabilitation.

ANR LEAUDS

• Title:
  Learning to understand audio scenes
• Duration:
  Apr 2019 - Mar 2023
• Coordinator:
  Université de Rouen Normandie
• Partners:
  Université de Rouen Normandie, Inria (Nancy), Netatmo (Paris)
• Participants:
  Félix Gontier, Mauricio Michel Olvera Zambrano, Romain Serizel, Emmanuel Vincent
• Abstract:
  LEAUDS aims to make a leap towards developing machines that understand audio input through breakthroughs in the detection of audio events from little annotated data, the robustness to “out-of-the lab” conditions, and language-based description of audio scenes. MULTISPEECH is responsible for research on robustness and for bringing expertise on natural language generation.

Inria Project Lab HyAIAI

• Title:
  Hybrid Approaches for Interpretable AI
• Duration:
  Sep 2019 - Aug 2023
• Coordinator:
  Inria LACODAM (Rennes)
• Partners:
  Inria TAU (Saclay), SEQUEL, MAGNET (Lille), MULTISPEECH, ORPAILLEUR (Nancy)
• Participants:
  Irina Illina, Emmanuel Vincent, Georgios Zervakis
• Abstract:
  HyAIAI is about the design of novel, interpretable artificial intelligence methods based on hybrid approaches that combine state of the art numeric models with explainable symbolic models.

ANR HAIKUS

• Title:
  Artificial Intelligence applied to augmented acoustic Scenes
• Duration:
  Dec 2019 - Nov 2023
• Coordinator:
  Ircam (Paris)
• Partners:
  Ircam (Paris), Inria (Nancy), IJLRA (Paris)
• Participants:
  Antoine Deleforge, Prerak Srivastava, Emmanuel Vincent
• Abstract:
  HAIKUS aims to achieve seamless integration of computer-generated immersive audio content into augmented reality (AR) systems. One of the main challenges is the rendering of virtual auditory objects in the presence of source movements, listener movements and/or changing acoustic conditions.

ANR JCJC DENISE

• Title:
  Tackling hard problems in audio using Data-Efficient Non-linear InverSe mEthods
• Duration:
  Oct 2020 – Sep 2024
• Coordinator:
  Antoine Deleforge
• Participants:
  Antoine Deleforge, Tom Sprunck, Marina Krémé
• Collaborators:
  UMR AE, Institut de Recherche Mathématiques Avancées de Strasbourg, Institut de Mathématiques de Bordeaux
• Abstract:
  DENISE aims to explore the applicability of recent breakthroughs in the field of nonlinear inverse problems to audio signal reparation and to room acoustics, and to combine them with compact machine learning models to yield data-efficient techniques.

Action Exploratoire Inria Acoust.IA

• Title:
  Acoust.IA: l'Intelligence Artificielle au Service de l'Acoustique du Bâtiment
• Duration:
  Oct 2020 - Sep 2023
• Coordinator:
  Antoine Deleforge
• Collaborators:
  UMR AE (Cerema Est, Strasbourg).
• Participants:
  Antoine Deleforge, Stéphane Dilungana, and Cédric Foy (CEREMA)
• Abstract:
  This project aims at radically simplifying and improving the acoustic diagnosis of rooms and buildings using new techniques combining machine learning, signal processing and physics-based modeling.

ANR Full3DTalkingHead

• Title:
  Synthèse articulatoire phonétique
• Duration:
  Apr 2021 - Sep 2024
• Coordinator:
  Yves Laprie
• Partners:
  Gipsa-Lab (Grenoble), IADI (Nancy), LPP (Paris)
• Participants:
  Slim Ouni, Vinicius Ribeiro, Yves Laprie
• Abstract:
  The objective is to realize a complete three-dimensional digital talking head including the vocal tract from the vocal folds to the lips and the face, and integrating the digital simulation of the aero-acoustic phenomena.

ANR Lorraine Artificicial Intelligence – LOR-AI LOR-AI

• Title:
  Lorraine Artificicial Intelligence Cofinancement de thèses en IA
• Duration:
  Sep 2020- Dec 2025
• Coordinator:
  Yves Laprie
• Partners:
  CNRS, Inria, Regional University Hospital Centre (CHRU)
• Participants:
  Doctoral schools of Université de Lorraine
• Abstract:
  This project about Artificial Intelligence, led by the Université de Lorraine (UL), has a double objective by providing 12 co-funding for doctoral theses: on the one hand, to strengthen UL areas of excellence in AI and domains tightly connected to IA, i.e. particularly Health, and on the other hand, to open other research areas to AI with the objective of leading to scientific breakthroughs.

ANR REFINED

• Title:
  Real-Time Artificial Intelligence for Hearing Aids
• Duration:
  Mar 2022 - Mar 2026
• Coordinator:
  CEA List (Saclay)
• Partners:
  CEA List (Saclay), Institut de l'audition (Paris), LORIA (Nancy)
• Participants:
  Paul Magron, Nasser-Eddine Monir, Romain Serizel
• Abstract:
  The Refined project brings together audiologists, computer scientists and specialists about hardware implementation to design new speech enhancement algorithms that both fit the needs of patients suffering of hearing losses and the computational constraints of hearing aid devices.

ANR LLM4all

• Title:
  Large Language Models for All
• Duration:
  Oct 2023 - Mars 2027
• Coordinator:
  Synalp Loria (Nancy)
• Partners:
  LORIA-Synalp, LORIA-Multispeech, LIX, Linagora, Ap-HP, HuggingFace
• Participants:
  Irina Illina, Emmanuel Vincent
• Abstract:
  Large Language Models (LLM) of sufficient size exhibit outstanding emergent abilities, such as learning from their input context and decomposing a complex problem into a chain of simpler steps. The LLM4all project will thus focus on such large models, or on models at the same level of generic performances, and will propose methods to solve two related fundamental issues: how to update these LLMs automatically, and how to reduce their computing requirements in order to facilitate their deployment.

PEPR Cybersécurité, projet iPOP

• Title:
  Protection des données personnelles
• Duration:
  Oct 2022 – Sep 2028
• Coordinator:
  Vincent Roca (Inria PRIVATICS)
• Partners:
  Inria PRIVATICS (Lyon), COMETE, PETRUS (Saclay), MAGNET, SPIRALS (Lille), IRISA (Rennes), LIFO (Bourges), DCS (Nantes), CESICE (Grenoble), EDHEC (Lille), CNIL (Paris)
• Participant:
  Emmanuel Vincent
• Summary:
  The objectives of iPOP are to study the threats on privacy introduced by new digital technologies, and to design privacy-preserving solutions compatible with French and European regulations. Within this scope, Multispeech focuses on speech data.

DGA DEEP MAUVES

• Title:
  Deep automatic aircraft speech recognition for non native speakers
• Duration:
  Dec 2022 – Dec 2026
• Coordinator:
  Irina Illina
• Participant:
  Irina Illina, Raphaël Bagat, Emmanuel Vincent
• Summary:
  This project proposes methods and tools that increase the usability of ASR systems for non-native speakers in noisy conditions in the aeronautical domain.

Défi Inria COLaF

• Title:
  Corpus et Outils pour les Langues de France
• Duration:
  Aug 2023 – Jul 2027
• Coordinator:
  Slim Ouni and Benoît Sagot (Inria ALMANACH)
• Partners:
  Inria ALMANACH (Paris)
• Participant:
  Slim Ouni, Sam Bigeard, Vincent Colotte, Emmanuel Vincent
• Summary:
  This project aims to increase the inclusiveness of speech technologies by releasing open data, models and software for accented French and for regional, overseas and non-territorial languages of France.

11.1.1 Scientific events: selection

11.1.2 Journal

11.1.3 Invited talks

• A brief introduction to multichannel noise reduction with deep neural networks, International Symposium on Auditory and Audiological Research (ISAAR), Denmark, August 2023 (R. Serizel)

11.1.4 Leadership within the scientific community

• Member of the Steering Committee of ISCA's Special Interest Group on Security and Privacy in Speech Communication (E. Vincent)
• Board member of Le VoiceLab, the association of French voice tech players (E. Vincent)
• Member of the IEEE Technical Committee on Audio and Acoustic Signal Processing (A. Deleforge)
• Secretary/Treasurer, executive member of AVISA (Auditory-VIsual Speech Association), an ISCA Special Interest Group (S. Ouni)
• Vice-president of AFCP - Association Francophone de la Communication Parlée (S. Ouni)
• Board member of AFCP - Association Francophone de la Communication Parlée (V. Colotte, Y. Laprie)
• Chair of the Steering Group of DCASE (R. Serizel)

11.1.5 Scientific expertise

• Expert for the Data Governance Working Group of the Global Partnership on AI (GPAI) (E. Vincent)
• Expert for HORIZON Europe 2023 (S. Ouni, Y. Laprie)
• Expertise for the Cognition Institute Carnot (A. Bonneau)
• Expertise for Collaborative Research in Computational Neuroscience (CRCNS) (R. Serizel)
• Expertise for the the Czech Science Foundation (I. Illina)

11.1.6 Research administration

• Head of Science of Inria Nancy – Grand Est (E. Vincent)
• Member of Inria's Evaluation Committee (E. Vincent)
• Member of the Comité Espace Transfert of Inria Nancy – Grand Est (E. Vincent)
• Vice-chair of the hiring committee for Junior Research Scientists, Inria Nancy – Grand Est (E. Vincent)
• Member of the admission committee for Inria Starting Faculty Positions, Inria Nancy – Grand Est (E. Vincent)
• Co-Chair of the local commission for sustainable development, Inria Nancy – Grand Est (A. Deleforge)
• Referent for research data, Inria Nancy – Grand Est (A. Deleforge)
• Member of the Commission de Développement Technologique, Inria Nancy – Grand Est (R. Serizel)
• Member of the Comipers, Inria Nancy – Grand Est (R. Serizel)
• Member of the Comité Utilisateurs des Moyens de Calculs, Inria Nancy – Grand Est (T. Biasutto–Lervat)
• Referent Plateformes-Outils, Inria Nancy – Grand Est (T. Biasutto–Lervat)
• Head of pole scientifique Automatique, Mathematiques, Informatique et leurs interactions (AM2I) (Y. Laprie)
• Member of the board, Université de Lorraine (Y. Laprie)
• Member of the bureau du pole scientifique Automatique, Mathematiques, Informatique et leurs interactions (AM2I) (I. Illina)
• Member of the Comité du pole scientifique Automatique, Mathematiques, Informatique et leurs interactions (AM2I) (I. Illina)
• Member of the evaluation committee of Haut Conseil de l'évaluation de la recherche et de l'enseignement supérieur (HCERES) for STIH (I. Illina)
• Member of the RIPEC jury, UL (I. Illina)
• Member of the admission committee for Nantes University (I. Illina)
• Member of the selection committee for the position of senior lecturer in phonetics at the University Paul Valéry of Montpellier (Y. Laprie)
• Academic representative of Université de Lorraine at AIDA (Artificial Intelligence Doctoral Academy) (S. Ouni)

11.2.1 Teaching

• BUT: I. Illina, Java programming (100 hours), Linux programming (58 hours), and Advanced Java programming (40 hours), Data Structures (50h), L1, Université de Lorraine, France
• BUT: I. Illina, Supervision of student projects and internships (50 hours), L2, Université de Lorraine, France
• BUT: R. Serizel, Introduction to office tools (108 hours), Multimedia and web (20 hours), Documents and databases (20 hours), L1, Université de Lorraine, France
• BUT: R. Serizel, Multimedia content and indexing (14 hours), Content indexing and retrieval software (20 hours), L2, Université de Lorraine, France
• BUT: S. Ouni, Programming in Java (24 hours), Web Programming (24 hours), Graphical User Interface (96 hours), L1, Université de Lorraine, France
• BUT: S. Ouni, Advanced Algorithms (24 hours), L2, Université de Lorraine, France
• Licence: A. Bonneau, Phonetics (18 hours), L1, École d’orthophonie, Université de Lorraine, France
• Licence: Y. Laprie, Phonetics (16 hours), L2, École d’audioprothèse, Université de Lorraine, France
• Licence: V. Colotte, Digital literacy and tools (hybrid courses, 50 hours), L1, Université de Lorraine, France
• Licence: V. Colotte, System (80 hours), L2-L3, Université de Lorraine, France
• Master: V. Colotte, Integration project: multimodal interaction with Pepper Robot (17 hours), M2, Université de Lorraine, France
• Master: V. Colotte, Multimodal oral communication (24 hours), M2, Université de Lorraine, France
• Master: V. Colotte, AI introduction (6 hours), M2 - intellectual property rights, Université de Lorraine, France
• Master: V. Colotte, Introduction to speech processing (24 hours), M1, Université de Lorraine, France
• Master: Y. Laprie, Speech corpora (30 hours), M1, Université de Lorraine, France
• Master: S. Ouni, Multimedia in Distributed Information Systems (31 hours), M2, Université de Lorraine, France
• Master: R. Serizel, S. Ouni, P. Magron and V. Ribeiro, Oral speech processing (24 hours), M2, Université de Lorraine
• Master: E. Vincent and P. Magron, Neural networks (40 hours), M2, Université de Lorraine
• Master: A. Deleforge, Initiation to Machine Learning (24 hours), M1-M2, Télécom Physique Strasbourg
• Master: A. Deleforge, Artificial Intelligence, Machine Learning, Deep Learning (24 hours), M1, Télécom Physique Strasbourg
• Other: V. Colotte, Co-Responsible for NUMOC (Digital literacy by hybrid courses) for Université de Lorraine, France (for 7,000 students)
• Other: S. Ouni, Co-Responsible of parcours Ingénierie Logiciel, BUT, Université de Lorraine, France

11.2.2 Supervision

• PhD: Tulika Bose, “Transfer Learning for Abusive Language Detection”, Jan 2023, I. Illina and D. Fohr. 47
• PhD: Shakeel Sheikh, “Deep learning for stuttering detection”, Feb 2023, S. Ouni 50.
• PhD: Georgios Zervakis, “Enriching large language models with semantic lexicons and analogies”, Mar 2023, M. Couceiro (LORIA) and E. Vincent 52.
• PhD: Sandipana Dowerah, “Deep Learning-based Speaker Verification In Real Conditions”, May 2023, D. Jouvet and R. Serizel 49.
• PhD: Prerak Srivastava, “Realism in virtually supervised learning for acoustic room characterization and sound source localization”, Nov 2023, A. Deleforge and E. Vincent 51.
• PhD: Nicolas Zampieri, “Détection des discours haineux dans les réseaux sociaux : apport des expressions polylexicales”, Dec 2023, I. Illina and D. Fohr.
• PhD: Vinicius Ribeiro, “Deep Supervision of the Vocal Tract Shape for Articulatory Synthesis of Speech”, Dec 2023, Y. Laprie
• PhD in progress: Stéphane Dilungana, “New Algorithms for Automated Room Acoustic Diagnosis”, Oct 2020, A. Deleforge and C. Foy and S. Faisan.
• PhD in progress: Can Cui, “Joint and embedded automatic speech separation, diarization and recognition for the generation of meeting minutes”, Oct 2021, M. Sadeghi and E. Vincent.
• PhD in progress: Sewade Olaolu Ogun, “Multi-factor data augmentation and transfer learning for embedded automatic speech recognition”, Oct 2021, V. Colotte and E. Vincent.
• PhD in progress: Raphaël Bagat, “Automatic speech recognition for non-native speakers in a noisy environment”, Oct 2023, I. Illina and E. Vincent.
• PhD in progress: Louis Abel, “Expressive audio-visual speech synthesis in an interaction context”, Oct 2021, S. Ouni and V. Colotte
• PhD in progress: Mickaëlla Grondin, “Modeling gestures and speech in interactions”, Nov 2021, S. Ouni and F. Hirsch (Praxiling)
• PhD in progress: François Effa, “Caractérisation de la détectabilité d'alarmes dans le bruit”, Dec 2020, J.-P. Arz ( Institut National de Recherche et de Sécurité), Nicolas Grimault (Centre de Recherche en Neurosciences de Lyon) and R. Serizel
• PhD in progress: Robin San Roman, “Self supervised disentangled representation learning of audio data for compression and generation”, Jun 2022, Y. M. Adi (Meta AI), A. Deleforge, R. Serizel and G. Synnaeve (Meta AI)
• PhD in progress: Nasser-Eddine Monir, “Multichannel speech enhancement for patients with auditory neuropathy spectrum disorders”, Dec 2022, P. Magron and R. Serizel
• PhD in progress: Taous Iatariene, “Spatial tracking of multiple moving sound sources”, Mar 2023, A. Guérin (Orange Labs) and R. Serizel
• PhD in progress: Jean-Eudes Ayilo, “Audio-visual Speech Enhancement: Bridging the Gap between Supervised & Unsupervised Approaches”, Oct 2023, M. Sadeghi and R. Serizel
• PhD Sofiane Azzouz, November 2023, “Acoustic to Articulatory Inversion by using dynamic MRI images”, Y. Laprie and P-A. Vuissoz
• PhD in progress: Zahra Benslimane, “Neural Networks For Improving Speech Understanding In Real-Time Embedded Systems”, Dec 2023, T. Allenet (CEA-List), F. Auzanneau (CEA-List) and R. Serizel

11.2.3 Juries

• Participation in the PhD jury of Mohamed Nabih Ali Mohamed Nawar (University of Trento, Jan 2023), E. Vincent, reviewer
• Participation in the PhD jury of Ranya Aloufi (Imperial College London, Mar 2023), E. Vincent, opponent
• Participation in the PhD jury of Yann Teytaut (Sorbonne Université, Jul 2023), E. Vincent, chair
• Participation in the PhD jury of Xiaoyu Bie (Université de Grenoble, Oct 2023), A. Deleforge, examiner
• Participation in the PhD jury of Vladimir Iashin, (Tampere University, Apr 2023), R. Serizel, reviewer
• Participation in the PhD jury of Samuele Cornell (Università Politecnica delle Marche, May 2023), R. Serizel, reviewer
• Participation in the PhD jury of Arjun Pankajakshan (Queen Mary University, Jul 2023), R. Serizel, reviewer
• Participation in the PhD jury of Marc-Antoine Georges (Université Grenoble Alpes, May 2023), Y. Laprie, reviewer
• Participation in the PhD jury of Martin Lebourdais (Université du Mans, Oct 2023), R. Serizel, reviewer
• Participation in the PhD jury of Arjun Pankajakshan (Queen Mary University, Jul 2023), R. Serizel, reviewer
• Participation in the PhD jury of Alexis Dehais-Underdown (Université de la Sorbonne Nouvelle, Sept 2023), Y. Laprie, reviewer
• Participation in the PhD jury of Pablo Zinemanas (Universitat Pompeu Fabra, Oct 2023), R. Serizel, reviewer
• Participation in the PhD jury of Mohammad Mohammadamini (Avignon University), I. Illina, examiner
• Participation in the HDR jury of Marie Tahon (Université du Mans, Jan 2023), E. Vincent, reviewer
• Participation in the HDR jury of Nicolas Obin (Sorbonne Université, Sep 2023), E. Vincent, reviewer

11.3.1 Education

• MATh.enJEANS - Presentation to college students of thr MULTIMOD platform (for multimodal data acquision) (L. Abel, T. Biasutto–Lervat)

11.3.2 Interventions

• IA : Quels défis pour demain ?, Quoi de Neuf Chercheur?, Twitch, Jul 2023 (E. Vincent)
• Interview on speech anonymization for L'Esprit Sorcier, Dec 2023 (E. Vincent)
• Video Interview "Let's talk about research with..." for Inria, Sep 2023 (A. Deleforge)
• L’éthique et l’IA, Podcast Monde Numérique, Sept 2023 (L. Abel)
• Showcase of the DeepLipsync technology, VivaTech, Jun 2023 (T. Biasutto–Lervat, L. Abel)

International journals

• 8 articleS.Simone Fischer-Hübner, D.Dietrich Klakow, P.Peggy Valcke and E.Emmanuel Vincent. Privacy in Speech and Language Technology.Dagstuhl Reports128March 2023, 60-102HALDOIback to text
• 9 articleK.Karyna Isaieva, F.Freddy Odille, Y.Yves Laprie, G.Guillaume Drouot, J.Jacques Felblinger and P.-A.Pierre-André Vuissoz. Super-Resolved Dynamic 3D Reconstruction of the Vocal Tract during Natural Speech.Journal of Imaging910October 2023, 233HALDOIback to text
• 10 articleZ.Zhiqi Kang, M.Mostafa Sadeghi, R.Radu Horaud and X.Xavier Alameda-Pineda. Expression-preserving face frontalization improves visually assisted speech processing.International Journal of Computer Vision1315May 2023, 1122-1140HALDOIback to text
• 11 articleO.Ondřej Mokrý, P.Paul Magron, T.Thomas Oberlin and C.Cédric Févotte. Algorithms for audio inpainting based on probabilistic nonnegative matrix factorization.Signal ProcessingMay 2023HALDOIback to text
• 12 articleV.Vinicius Ribeiro, K.Karyna Isaieva, J.Justine Leclere, J.Jacques Felblinger, P.-A.Pierre-André Vuissoz and Y.Yves Laprie. Automatic segmentation of vocal tract articulators in real-time magnetic resonance imaging.Computer Methods and Programs in Biomedicine2432January 2024HALDOIback to text
• 13 articleM.Mostafa Sadeghi, X.Xavier Alameda-Pineda and R.Radu Horaud. Unsupervised Performance Analysis of 3D Face Alignment with a Statistically Robust Confidence Test.Neurocomputing564January 2024HALDOIback to text
• 14 articleA. S.Ali Shahin Shamsabadi, B. M.Brij Mohan Lal Srivastava, A.Aurélien Bellet, N.Nathalie Vauquier, E.Emmanuel Vincent, M.Mohamed Maouche, M.Marc Tommasi and N.Nicolas Papernot. Differentially private speaker anonymization.Proceedings on Privacy Enhancing Technologies20231January 2023HALDOIback to text
• 15 articleS. A.Shakeel Ahmad Sheikh, M.Md Sahidullah, F.Fabrice Hirsch and S.Slim Ouni. Advancing Stuttering Detection via Data Augmentation, Class-Balanced Loss and Multi-Contextual Deep Learning.IEEE Journal of Biomedical and Health InformaticsFebruary 2023HALDOIback to text
• 16 articleS.Shakeel Sheikh, M.Md Sahidullah, F.Fabrice Hirsch and S.Slim Ouni. Stuttering Detection Using Speaker Representations and Self-supervised Contextual Embeddings.International Journal of Speech TechnologyJune 2023HALDOIback to text
• 17 articleI. A.Imran Ahamad Sheikh, E.Emmanuel Vincent and I.Irina Illina. Training RNN Language Models on Uncertain ASR Hypotheses in Limited Data Scenarios.Computer Speech and Language83January 2024, 101555HALDOIback to text
• 18 articleX.Xiaolei Zhang, L.Lei Xie, E.Eric Fosler-Lussier and E.Emmanuel Vincent. Guest editorial: Special issue on advances in deep learning based speech processing.Neural Networks158January 2023HALDOIback to text

International peer-reviewed conferences

• 19 inproceedingsJ.-E.Jean-Eudes Ayilo, M.Mostafa Sadeghi and R.Romain Serizel. Diffusion-based speech enhancement with a weighted generative-supervised learning loss.International Conference on Acoustics Speech and Signal Processing (ICASSP)Seoul (Korea), South KoreaApril 2024HALback to text
• 20 inproceedingsL.Louis Bahrman, M.Marina Krémé, P.Paul Magron and A.Antoine Deleforge. Signal Inpainting from Fourier Magnitudes.European Signal Processing Conference (EUSIPCO)EUSIPCO 2023Helsinki, FinlandJune 2023HALback to text
• 21 inproceedingsC.Can Cui, I. A.Imran Ahamad Sheikh, M.Mostafa Sadeghi and E.Emmanuel Vincent. End-to-end Multichannel Speaker-Attributed ASR: Speaker Guided Decoder and Input Feature Analysis.2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU 2023)Taipei, TaiwanDecember 2023HALback to text
• 22 inproceedingsA.Alexis Dehais-Underdown, P.Paul Vignes, L.Lise Crevier-Buchman, D.Didier Demolin, P.-A.Pierre-André Vuissoz, K.Karyna Isaieva, M.Marc Fauvel, Y.Yves Laprie and J.Jacques Felblinger. Non-pulmonic initiation in human beatboxing: a real-time MRI study.20th International Congress of Phonetic Sciences (ICPhS 2023)Prague, Czech RepublicAugust 2023HALback to text
• 23 inproceedingsL.Louis Delebecque and R.Romain Serizel. BinauRec: A dataset to test the influence of the use of room impulse responses on binaural speech enhancement.European Signal Processing Conference (EUSIPCO)EUSIPCO23Helsiinki, FinlandSeptember 2023HALback to text
• 24 inproceedingsS.Sandipana Dowerah, A.Ajinkya Kulkarni, R.Romain Serizel and D.Denis Jouvet. Self-supervised learning with diffusion-based multichannel speech enhancement for speaker verification under noisy conditions.Proceedings of Interspeech 2023INTERSPEECH 2023Dublin (Ireland), IrelandISCAAugust 2023, 3849-3853HALDOIback to text
• 25 inproceedingsS.Sandipana Dowerah, R.Romain Serizel, D.Denis Jouvet, M.M Mohammadamini and D.Driss Matrouf. Joint optimization of diffusion probabilistic-based multichannel speech enhancement with far-field speaker verification.IEEE SLT 2022Doha, QatarJanuary 2023HALback to text
• 26 inproceedingsJ.Janek Ebbers, R.Reinhold Haeb-Umbach and R.Romain Serizel. Post-Processing Independent Evaluation of Sound Event Detection Systems.DCASE 2023 - 8th Workshop on Detection and Classification of Acoustic Scenes and EventsTampere, FinlandSeptember 2023HALDOIback to text
• 27 inproceedingsF.François Effa, R.Romain Serizel, J.-P.Jean-Pierre Arz and N.Nicolas Grimault. Lightweight Annotation and Class Weight Training for Automatic Estimation of Alarm Audibility in Noise.ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal ProcessingRhodes Island, GreeceIEEEJune 2023, 1-5HALDOI
• 28 inproceedingsA.Ali Golmakani, M.Mostafa Sadeghi, X.Xavier Alameda-Pineda and R.Romain Serizel. A weighted-variance variational autoencoder model for speech enhancement.International Conference on Acoustics Speech and Signal Processing (ICASSP)Seoul (Korea), South KoreaApril 2024HAL
• 29 inproceedingsA.Ali Golmakani, M.Mostafa Sadeghi and R.Romain Serizel. Audio-visual speech enhancement with a deep kalman filter generative model.International Conference on Acoustics Speech and Signal Processing (ICASSP)Rhodes island, GreeceJune 2023HALback to text
• 30 inproceedingsF.Félix Gontier, R.Romain Serizel and C.Christophe Cerisara. SPICE+: Evaluation of automatic audio captioning systems with pre-trained language models.Proceedings of the 2023 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2023)IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2023)Rhodes Island, GreeceJune 2023HALback to text
• 31 inproceedingsI.Irina Illina and D.Dominique Fohr. Semantic Information Investigation for Transformer-based Rescoring of N-best Speech Recognition.LTC 2023Proceedings of Language and Technology 2023Poznan, PolandApril 2023HAL
• 32 inproceedingsY.Yves Laprie, V.Vinicius Ribeiro, K.Karina Isaeva, J.Justine Leclere, J.Jacques Felblinger and P.-A.Pierre-André Vuissoz. Modeling the temporal evolution of the vocal tract shape with deep learning.20th International Congress on Phonetic SciencesPrague (CZ), Czech RepublicAugust 2023HALback to text
• 33 inproceedingsS.Simon Leglaive, L.Léonie Borne, E.Efthymios Tzinis, M.Mostafa Sadeghi, M.Matthieu Fraticelli, S.Scott Wisdom, M.Manuel Pariente, D.Daniel Pressnitzer and J. R.John R. Hershey. The CHiME-7 UDASE task: Unsupervised domain adaptation for conversational speech enhancement.7th International Workshop on Speech Processing in Everyday Environments (CHiME)Dublin, IrelandJuly 2023HAL
• 34 inproceedingsP.Paul Magron and T.Tuomas Virtanen. Spectrogram Inversion for Audio Source Separation via Consistency, Mixing, and Magnitude Constraints.Proc. European Signal Processing Conference (EUSIPCO)EUSIPCO 2023Helsinki, FinlandSeptember 2023HALback to text
• 35 inproceedingsI.Ilyass Moummad, R.Romain Serizel and N.Nicolas Farrugia. Pretraining Representations for Bioacoustic Few-Shot Detection using Supervised Contrastive Learning.Detection and Classification of Acoustic Scenes and Events 2023TAMPERE, Finland2023HAL
• 36 inproceedingsB.Berné Nortier, M.Mostafa Sadeghi and R.Romain Serizel. Unsupervised speech enhancement with diffusion-based generative models.International Conference on Acoustics Speech and Signal Processing (ICASSP)Seoul (Korea), South KoreaApril 2024HALback to text
• 37 inproceedings S.Sewade Ogun, V.Vincent Colotte and E.Emmanuel Vincent. Can we use Common Voice to train a Multi-Speaker TTS system? The 2022 IEEE Spoken Language Technology Workshop (SLT 2022) Doha, Qatar January 2023 HAL back to text
• 38 inproceedingsS.Sewade Ogun, V.Vincent Colotte and E.Emmanuel Vincent. Stochastic Pitch Prediction Improves the Diversity and Naturalness of Speech in Glow-TTS.InterSpeech 2023Dublin, IrelandAugust 2023HALback to text
• 39 inproceedingsC.Cennet Oguz, P.Pascal Denis, E.Emmanuel Vincent, S.Simon Ostermann and J.Josef van Genabith. Find-2-Find: Multitask Learning for Anaphora Resolution and Object Localization.2023 Conference on Empirical Methods in Natural Language ProcessingSingapore, Singapore2023HALback to text
• 40 inproceedingsR. S.Robin San Roman, Y.Yossi Adi, A.Antoine Deleforge, R.Romain Serizel, G.Gabriel Synnaeve and A.Alexandre Défossez. From Discrete Tokens to High-Fidelity Audio Using Multi-Band Diffusion.NeurIPS 2023 - Conference on Neural Information Processing SystemsNew Orleans, United States2023HALDOIback to text
• 41 inproceedingsM.Mostafa Sadeghi and R.Romain Serizel. Fast and efficient speech enhancement with variational autoencoders.International Conference on Acoustics Speech and Signal Processing (ICASSP)Rhodes island, GreeceJune 2023HALback to textback to text
• 42 inproceedingsM.Mostafa Sadeghi and R.Romain Serizel. Posterior sampling algorithms for unsupervised speech enhancement with recurrent variational autoencoder.International Conference on Acoustics Speech and Signal Processing (ICASSP)Seoul (Korea), South KoreaApril 2024HALback to textback to text
• 43 inproceedingsR.Romain Serizel, S.Samuele Cornell and N.Nicolas Turpault. Performance above all ? energy consumption vs. performance for machine listening, a study on dcase task 4 baseline.ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)Rhodes Island, FranceIEEE; IEEE2023, 1-5HALDOIback to textback to text
• 44 inproceedingsP.Prerak Srivastava, A.Antoine Deleforge, A.Archontis Politis and E.Emmanuel Vincent. How to (Virtually) Train Your Speaker Localizer.INTERSPEECH 2023Dublin, IrelandAugust 2023HALback to text
• 45 inproceedingsN.Nicolas Zampieri, I.Irina Illina and D.Dominique Fohr. Improving Hate Speech Detection with Self-Attention Mechanism and Multi-Task Learning.LTC'23 - 10th Language & Technology Conference: Human Language Technologies as a Challenge for Computer Science and LinguisticsPoznan, PolandApril 2023HALback to text

Conferences without proceedings

• 46 inproceedings C.Constance Douwes, F.Francesca Ronchini and R.Romain Serizel. Monitoring environmental impact of DCASE systems: Why and how ? Detection and Classification of Acoustic Scene and Events (DCASE) Workshop Tampere (Finlande), Finland September 2023 HAL back to text

Doctoral dissertations and habilitation theses

• 47 thesisT.Tulika Bose. Transfer learning for abusive language detection.Université de LorraineJanuary 2023HALback to textback to textback to text
• 48 thesisP.Pierre Champion. Anonymizing Speech : Evaluating and Designing Speaker Anonymization Techniques.Université de LorraineApril 2023HALback to text
• 49 thesisS.Sandipana Dowerah. Deep Learning-based Speaker Verification In Real Conditions.Université de Lorraine; CNRS, Inria, LORIAMay 2023HALback to text
• 50 thesisS. A.Shakeel Ahmad Sheikh. Deep learning for stuttering detection.Université de LorraineFebruary 2023HALback to textback to text
• 51 thesisP.Prerak Srivastava. Realism in virtually supervised learning for acoustic room characterization and sound source localization.Université de LorraineNovember 2023HALback to textback to text
• 52 thesisG.Georgios Zervakis. Enriching large language models with semantic lexicons and analogies.Université de LorraineMarch 2023HALback to textback to text

Reports & preprints

• 53 miscC.Can Cui, I. A.Imran Ahamad Sheikh, M.Mostafa Sadeghi and E.Emmanuel Vincent. End-to-end Joint Rich and Normalized ASR with a limited amount of rich training data.September 2023HALback to text
• 54 reportI.Ilyass Moummad, R.Romain Serizel and N.Nicolas Farrugia. Supervised contrastive learning for pre-training bioacoustic few-shot systems.IMT Atlantique; LORIAMay 2023HAL

Other scientific publications

• 55 inproceedingsC.Can Cui, I. A.Imran Ahamad Sheikh, M.Mostafa Sadeghi and E.Emmanuel Vincent. End-to-end Multichannel Speaker-Attributed ASR: Speaker Guided Decoder and Input Feature Analysis.Rencontre des Jeunes Chercheurs en Parole 2023 - 10E EditionGrenoble, FranceNovember 2023HALback to text