3.1.1 Axis 1.1 — Integrating domain knowledge

State-of-the art methods for speech and audio processing are based on discriminative neural networks trained for the targeted task. This paradigm faces major limitations: lack of interpretability, large data requirements, inability to generalize to unseen classes or tasks. Our approach is to combine the representation power of deep learning with our acoustic expertise to obtain smaller generative models describing the probability distribution of speech and audio signals. Particular attention will be paid to designing physically-motivated input layers, output layers, and unsupervised representations that capture complex-valued, multi-scale spectro-temporal dependencies. Given these models, we derive computationally efficient inference algorithms that address the above limitations. We also explore the integration of deep learning with symbolic reasoning and common-sense knowledge to increase the generalization ability of deep models.

3.1.2 Axis 1.2 — Learning from little/no labeled data

While supervised learning from fully labeled data is economically costly, unlabeled data are inexpensive but provide intrinsically less information. Our goal is to learn representations that disentangle the attributes of speech by equipping the unsupervised representation learning methods above with supervised branches exploiting the available labels and supervisory signals, and with multiple adversarial branches overcoming the usual limitations of adversarial.

3.1.3 Axis 1.3 — Preserving privacy

To preserve privacy, speech must be transformed to hide the users’ identity and other privacy-sensitive attributes (e.g., accent, health status) while leaving intact those attributes which are required for the task (e.g., phonetic content for automatic speech recognition) and preserving the data variability for training purposes. We develop strong attacks to evaluate the privacy. We also seek to hide personal identifiers and privacy-sensitive attributes in the linguistic content, focusing on their robust extraction and replacement from speech signals.

3.2.1 Axis 2.1 — Linguistic speech content

Speech recognition is the main means to extract linguistic information from speech. Although it is a mature research area, performance drops in real-world environments pursue the development of speech enhancement and source separation methods to effectively improve robustness in such real-world scenarios. Semantic content analysis is required to interpret the spoken message. The challenges include learning from little real data, quickly adapting to new topics, and robustness to speech recognition errors. The detection and classification of hate speech in social media videos will also be considered as a benchmark, thereby extending the work on text-only detection. Finally, we also consider extracting phonetic and prosodic information to study the categorization of speech sounds and certain aspects of prosody by learners of a foreign language.

3.2.2 Axis 2.2 — Speaker identity and states

Speaker identity is required for personalization of human-computer interaction. Speaker recognition and diarization are still challenging in real-world conditions. The speaker states that we aim to recognize include emotion and stress, which can be used to adapt the interaction in real time.

3.2.3 Axis 2.3 — Speech environment information

We develop audio event detection methods that exploit both strongly/weakly labeled and unlabeled data, operate in real-world conditions, can discover new events, and provide a semantic interpretation. Modeling of the temporal, spatial and logical structure of ambient sound scenes over a long duration is also considered. We are also interested in the lesser studied problem of inferring the acoustic properties of the environment from impulse response measurements or from multichannel recordings of unknown sound sources.

3.3.1 Axis 3.1 - Multimodality modeling and analysis

The study of multimodality concerns the interaction between modalities, their fusion, coordination and synchronization for a single speaker, as well as their synchronization across the speakers in a conversation. We focus on audiovisual speech enhancement to improve the intelligibility and quality of noisy speech by considering the speaker’s lip movements. We also consider the semi/weakly/self-supervised learning methods for multimodal data so as to obtain interpretable representations that disentangle in each modality the attributes related to linguistic and semantic content, emotion, reaction, etc. We also study the contribution of each modality to the intelligibility of spoken communication.

3.3.2 Axis 3.2 - Multimodal speech generation

Multimodal speech generation refers to articulatory, acoustic, and audiovisual speech synthesis techniques which output one or more facets. Articulatory speech synthesis relies on 2D and 3D modeling of the dynamics of the vocal tract from real-time MRI (rtMRI) data. We consider the generation of the full vocal tract, from the vocal folds to the lips, first in 2D then in 3D. This comprises the generation of the face and the prediction of the glottis opening. We also consider audiovisual speech synthesis. Both the animation of the lower part of the face related to speech and of the upper part related to the facial expression are considered, and development continues towards a multilingual talking head. We investigate further the modeling of expressivity for both audio-only and audiovisual speech synthesis, for a better control of expressivity, where we consider several disentangled attributes at the same time.

3.3.3 Axis 3.3 — Interaction

Interaction is a new field of research for our project-team that we will approach gradually. We start by studying the multimodal components (prosody, facial expressions, gestures) used during interaction, both by the speaker and by the listener, where the goal is to simultaneously generate speech and gestures by the speaker, and generating regulatory gestures for the listener. We will introduce different dialog bricks progressively: Spoken language understanding, Dialog management, and Natural language generation. Dialog will be considered in a multimodal context (gestures, emotional states of the interlocutor, etc.) and we will break the classical dialog management scheme to dynamically account for the interlocutor’s evolution during the speaker’s response.

7.1.1 ASTALI

• Name:
  Automatic Speech-Text Alignment Software
• Keyword:
  Speech-text alignment
• Functional Description:
  ASTALI is a software for aligning a speech signal with its corresponding orthographic transcription (given in simple text file for short audio signals or in .trs files as generated by transcriber for longer speech signals). Using a phonetic lexicon and automatic grapheme-to-phoneme converters, all the potential sequences of phones corresponding to the text are generated. Then, using acoustic models, the tool finds the best phone sequence and provides the boundaries at the phone and at the word levels. The web application makes the service easy to use, without requiring any software downloading.
• News of the Year:
  The application has been migrated to a new, more robust web server. The application is still operational and actively used.
• URL:
  http://­astali.­loria.­fr/
• Contact:
  Theo Biasutto–lervat

7.1.2 Asteroid

• Name:
  Asteroid: The PyTorch-based audio source separation toolkit for researchers.
• Keywords:
  Source Separation, Deep learning
• Functional Description:
  Asteroid is an open-source toolkit made to design, train, evaluate, use and share neural network based audio source separation and speech enhancement models. Inspired by the most successful neural source separation systems, Asteroid provides all neural building blocks required to build such a system. To improve reproducibility, Kaldi-style recipes on common audio source separation datasets are also provided. Experimental results obtained with Asteroid’s recipes show that our implementations are at least on par with most results reported in reference papers.
• Release Contributions:

  [0.6.0] - 2022-06-28 Upgraded source code and recipes to PyTorch-Lighting 1.5+.

  [0.5.3] - 2022-06-26 Added: [egs&tests] MixIT loss function (#595)

  Changed: [docs] Update RTD version to 0.5.1

  Fixed: [src] Fix FasNetTAC loading (#586) [src] Fix device in padding for sudormrf. Fix #598 (#603) [docs] Fix deep_clustering_loss docs example (#607) [src] Remove torch.complex32 usage for torch 1.11.0 (#609) [egs] Fix package install script for WHAMR! (#613) [src] Fix shape mismatching in SuDORMRF's masknn (#618) [CI] Fix CI Restrict torchmetrics version to under 0.8.0 (#619) [docs] Fix docs Restrict jinja2>=3.0.0,<3.1.0 (#620)

• News of the Year:
  - Upgraded source code and recipes to PyTorch-Lighting 1.5+. - Added the MixIT loss function
• URL:
  https://­github.­com/­asteroid-team/­asteroid
• Contact:
  Antoine Deleforge
• Participants:
  Manuel Pariente, Mathieu Hu, Joris Cosentino, Sunit Sivasankaran, Mauricio Michel Olvera Zambrano, Fabian Robert Stoter

7.1.3 VocabLearn

• Name:
  Web-based Pronunciation Learning Application
• Keywords:
  Pronunciation training, Talking head, Second language learning
• Scientific Description:
  This platform highlights our work on realistic animation of a talking head from speech (also called lipsync). Our lipsync system is operational for German. The evaluation of pronunciation is based on our work on speech recognition. The work on evaluation is not fully completed.
• Functional Description:
  This web-based application is dedicated to foreign language pronunciation learning (current version was developed for the German language). It is intended for high school and middle school students. There are two types of exercises that are integrated in this application. (1) Flashcards: Cards are presented, then a virtual teacher (a 3D talking head) pronounces the words and sentences corresponding to these cards. Students can practice and make an evaluation of their word comprehension. (2) Speech recognition. The application displays a list of words/phrases that the student pronounces and the system gives feedback on the quality of the pronunciation. This application is composed of two modules: one for students (described above) and one for teachers, allowing them to create lessons, and to follow the results and progress of student evaluations.
• News of the Year:
  A first online version of the flash cards application is deployed. Both versions: Student and teacher are fully running. New features have been added to the administration interface to add cards and to generate new videos of the virtual teacher.
• Contact:
  Slim Ouni
• Participants:
  Theo Biasutto–lervat, Slim Ouni, Denis Jouvet, Louis Abel

7.1.4 Voice Transformer 2

• Keywords:
  Speech, Privacy
• Scientific Description:

  The implemented method is inspired from the speaker anonymisation method proposed in [Fan+19], which performs voice conversion based on x-vectors [Sny+18], a fixed-length representation of speech signals that form the basis of state-of-the-art speaker verification systems. We have brought several improvements to this method such as pitch transformation, and new design choices for x-vectors selection

  [Fan+19] F. Fang, X. Wang, J. Yamagishi, I. Echizen, M. Todisco, N. Evans, and J.F. Bonastre. “Speaker Anonymization Using x-vector and Neural Waveform Models”. In: Proceedings of the 10th ISCA Speech Synthesis Workshop. 2019, pp. 155–160. [Sny+18] D. Snyder, D. Garcia-Romero, G. Sell, D. Povey, and S. Khudanpur. “X-vectors: Robust DNN embeddings for speaker recognition”. In: Proceedings of ICASSP 2018 - 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2018, pp. 5329–5333.

• Functional Description:
  Voice Transformer increases the privacy of users of voice interfaces by converting their voice into another person’s voice without modifying the spoken message. It ensures that any information extracted from the transformed voice can hardly be traced back to the original speaker, as validated through state-of-the-art biometric protocols, and it preserves the phonetic information required for human labelling and training of speech-to-text models.
• Release Contributions:
  This version only retains the anonymization method using x-vectors and neural models from the previous version. The previous architecture inherited from kaldi based on bash scripts has been completely revised for a modular architecture, based on REST micro-services, implemented only in python and C++.
• Contact:
  Nathalie Vauquier
• Participants:
  Brij Mohan Lal Srivastava, Nathalie Vauquier, Emmanuel Vincent, Marc Tommasi

7.1.5 Web Multimodal Annotation

• Name:
  Web-based Multimodal Data Annotation
• Keywords:
  Annotation tool, Audio segmentation, Audiovisual, Web Application, Speech
• Scientific Description:
  Web Multimodal Annotation is a web application that was designed to manually segment and label audio or video data and to visualize the audio data. In addition, this web application also allows for multi-level labeling and collaboration between multiple annotators. This web application can be useful to annotators, researchers in the field of multimodal speech or gestures.
• Functional Description:
  Web Multimodal Annotation is a web application that was designed to manually segment and label audio or video data and to visualize the audio data. In addition, this web application also allows for multi-level labeling and collaboration between multiple annotators. This web application can be useful to annotators, researchers in the field of multimodal speech or gestures.
• Release Contributions:
  The software is deployed on a web server. This version is quite mature, but requires evaluation by users.
• Contact:
  Slim Ouni
• Participants:
  Slim Ouni, Sofiane Azzouz
• Partners:
  CNRS, Université de Lorraine

7.2.1 Multimodal Voice assistant

Voice assistants and voice interfaces have become a key technology, simplifying the user experience and increasing the accessibility of many applications, and their use will intensify in the coming years. However, this technology poses two major problems today: on the one hand, the quasi-hegemony of large technology companies (mainly American) raises questions about European digital sovereignty, and on the other hand, the commonly used client-server architecture raises privacy risks. To simultaneously address these two problems, we are currently developing an open-source platform for the creation of embedded virtual assistants.

This platform will provide the main speech processing and natural language processing bricks that are necessary to build a voice interface, such as denoising, recognition or speech synthesis. The generated assistant will be fully embedded in the users' terminal. The data being processed locally, we ensure the protection of their private lives. We envisage a multiplatform solution on PC (Windows, Linux, MacOS) as well as on mobile (Android, iOS).

During this first year of development, many points have been addressed and evaluated, in particular:

• benchmarking and choice of software tools (wasm, ONNX...)
• benchmarking (performance and computing power) of speech recognition and synthesis models (from NVIDIA's NeMo repository)
• prototyping of a cross-platform PC solution (Python)
• prototyping of an Android solution (Kotlin)
• preliminary design of the API

8.1.1 Axis 1.1 — Integrating domain knowledge

8.1.2 Axis 1.2 - Learning from little/no labeled data

8.1.3 Axis 1.3 - Preserving privacy

Speech signals convey a lot of private information. To protect speakers, we pursued our investigation of x-vector based voice anonymization, which relies on splitting the speech signal into speaker (x-vector), phonetic and pitch features and resynthesizing the signal with a different target x-vector. We conducted an extensive study of the impact of the target x-vector selection process (speaker distance metric, target region of x-vector space, target gender, speaker- or utterance-level target selection) on privacy and utility 19. To reduce the amount of residual speaker information in the phonetic and pitch features, we explored the use of vector quantization 28, 27 and Laplacian noise 16 inspired from differential privacy. We also evaluated the impact of voice anonymization on emotional speech data 45. Finally, we showed that slicing utterances into shorter segments further improves privacy at no cost in utility 42.

In a complementary line of work, we studied the adaptation of ASR language models trained on anonymized text data to the statistics of the original text data 57.

We analyzed the results of the 1st Voice Privacy Challenge which we had organized in 2020 in an article 20 and a detailed technical report 78. We co-organized the 2nd Voice Privacy Challenge 77, which improves the evaluation of voice anonymization systems in several ways, namely ranking based on multiple privacy-utility tradeoffs, semi-informed attack models, and an intonation preservation constraint.

8.2.1 Axis 2.1 — Linguistic speech content

8.2.2 Axis 2.2 — Speaker identity and states

8.2.3 Axis 2.3 — Speech in its environment

8.3.1 Axis 3.1 — Multimodality modeling and analysis

8.3.2 Axis 3.2 — Multimodal speech generation

8.3.3 Axis 3.3 — Interaction

9.1.1 Meta AI

• Company: Meta AI (France)
• Duration: Nov 2018 – Feb 2022
• Participants: Adrien Dufraux, Emmanuel Vincent
• Abstract: This CIFRE grant funded the PhD of Adrien Dufraux on cost-effective weakly supervised learning for automatic speech recognition.

9.1.2 Vivoka

• Company: Vivoka (France)
• Duration: Oct 2021 – Oct 2024
• Participants: Can Cui, Mostafa Sadeghi, Emmanuel Vincent
• Abstract: This contract funds the PhD of Can Cui on joint and embedded automatic speech separation, diarization and recognition for the generation of meeting minutes.

9.1.3 Meta AI

• Company: Meta AI (France)
• Duration: May 2022 – Apr 2025
• Participants: Robin San Roman, Antoine Deleforge, Romain Serizel
• Abstract: This CIFRE grant funds the PhD of Robin San Roman on self-supervised disentangled representation learning of audio data for compression and generation.

10.1.1 Visits of international scientists

• Archontis Politis, assistant professor at Tampere University (Finland), visited the team from Sep. 12 to Nov. 4, 2022.

10.2.1 H2020 & Horizon Europe

10.2.2 Other european programs/initiatives

ANR JCJC DiSCogs

• Title:
  Distant speech communication with heterogeneous unconstrained microphone arrays
• Duration:
  Sep 2018 – Aug 2022
• Coordinator:
  Romain Serizel (LORIA, Nancy)
• Participants:
  Louis Delebecque, Nicolas Furnon, Irina Illina, Romain Serizel, Emmanuel Vincent
• Collaborators:
  Télécom ParisTech, 7sensing
• Abstract:
  The objective is to solve fundamental sound processing issues in order to exploit the many devices equipped with microphones that populate our everyday life. The solution proposed is to apply deep learning approaches to recast the problem of synchronizing devices at the signal level as a multi-view learning problem.

ANR DEEP-PRIVACY

• Title:
  Distributed, Personalized, Privacy-Preserving Learning for Speech Processing
• Duration:
  Jan 2019 - Jun 2023
• Coordinator:
  Denis Jouvet until Aug 2022; Emmanuel Vincent from Sep 2022
• Partners:
  LIUM (Le Mans), Inria MAGNET (Lille), LIA (Avignon)
• Participants:
  Pierre Champion, Denis Jouvet, Hubert Nourtel, Emmanuel Vincent
• Abstract:
  The objective of the DEEP-PRIVACY project is to elaborate a speech transformation that hides the speaker identity for an easier sharing of speech data for training speech recognition models; and to investigate speaker adaptation and distributed training.

ANR ROBOVOX

• Title:
  Robust Vocal Identification for Mobile Security Robots
• Duration:
  Mar 2019 – Apr 2024
• Coordinator:
  Laboratoire d'informatique d'Avignon (LIA)
• Partners:
  Inria (Nancy), LIA (Avignon), A.I. Mergence (Paris)
• Participants:
  Antoine Deleforge, Sandipana Dowerah, Denis Jouvet, Romain Serizel
• Abstract:
  The aim of ROBOVOX project is to improve speaker recognition robustness for a security robot in real environment. Several aspects will be particularly considered such as ambiant noise, reverberation and short speech utterances.

ANR BENEPHIDIRE

• Title:
  Stuttering: Neurology, Phonetics, Computer Science for Diagnosis and Rehabilitation
• Duration:
  Mar 2019 - Dec 2023
• Coordinator:
  Praxiling (Montpellier)
• Partners:
  Praxiling (Montpellier), LORIA (Nancy), INM (Montpellier), LiLPa (Strasbourg).
• Participants:
  Yves Laprie, Slim Ouni, Shakeel Ahmad Sheikh
• Abstract:
  The BENEPHIDIRE project brings together neurologists, speech-language pathologists, phoneticians, and computer scientists specializing in speech processing to investigate stuttering as a speech impairment and to develop techniques for diagnosis and rehabilitation.

ANR LEAUDS

• Title:
  Learning to understand audio scenes
• Duration:
  Apr 2019 - Mar 2023
• Coordinator:
  Université de Rouen Normandie
• Partners:
  Université de Rouen Normandie, Inria (Nancy), Netatmo (Paris)
• Participants:
  Félix Gontier, Mauricio Michel Olvera Zambrano, Romain Serizel, Emmanuel Vincent
• Abstract:
  LEAUDS aims to make a leap towards developing machines that understand audio input through breakthroughs in the detection of audio events from little annotated data, the robustness to “out-of-the lab” conditions, and language-based description of audio scenes. MULTISPEECH is responsible for research on robustness and for bringing expertise on natural language generation.

Inria Project Lab HyAIAI

• Title:
  Hybrid Approaches for Interpretable AI
• Duration:
  Sep 2019 - Aug 2023
• Coordinator:
  Inria LACODAM (Rennes)
• Partners:
  Inria TAU (Saclay), SEQUEL, MAGNET (Lille), MULTISPEECH, ORPAILLEUR (Nancy)
• Participants:
  Irina Illina, Emmanuel Vincent, Georgios Zervakis
• Abstract:
  HyAIAI is about the design of novel, interpretable artificial intelligence methods based on hybrid approaches that combine state of the art numeric models with explainable symbolic models.

ANR HAIKUS

• Title:
  Artificial Intelligence applied to augmented acoustic Scenes
• Duration:
  Dec 2019 - Nov 2023
• Coordinator:
  Ircam (Paris)
• Partners:
  Ircam (Paris), Inria (Nancy), IJLRA (Paris)
• Participants:
  Antoine Deleforge, Prerak Srivastava, Emmanuel Vincent
• Abstract:
  HAIKUS aims to achieve seamless integration of computer-generated immersive audio content into augmented reality (AR) systems. One of the main challenges is the rendering of virtual auditory objects in the presence of source movements, listener movements and/or changing acoustic conditions.

ANR JCJC DENISE

• Title:
  Tackling hard problems in audio using Data-Efficient Non-linear InverSe mEthods
• Duration:
  Oct 2020 – Sep 2024
• Coordinator:
  Antoine Deleforge
• Participants:
  Antoine Deleforge, Tom Sprunck, Marina Krémé
• Collaborators:
  UMR AE, Institut de Recherche Mathématiques Avancées de Strasbourg, Institut de Mathématiques de Bordeaux
• Abstract:
  DENISE aims to explore the applicability of recent breakthroughs in the field of nonlinear inverse problems to audio signal reparation and to room acoustics, and to combine them with compact machine learning models to yield data-efficient techniques.

Action Exploratoire Inria Acoust.IA

• Title:
  Acoust.IA: l'Intelligence Artificielle au Service de l'Acoustique du Bâtiment
• Duration:
  Oct 2020 - Sep 2023
• Coordinator:
  Antoine Deleforge
• Collaborators:
  UMR AE (Cerema Est, Strasbourg).
• Participants:
  Antoine Deleforge, Stéphane Dilungana, and Cédric Foy (CEREMA)
• Abstract:
  This project aims at radically simplifying and improving the acoustic diagnosis of rooms and buildings using new techniques combining machine learning, signal processing and physics-based modeling.

InriaHub ADT PEGASUS

• Title:
  PEGASUS: rehaussement de la ParolE Généralisé par Apprentissage SUperviSé
• Duration:
  Nov 2020 - Oct 2022
• Coordinator:
  Antoine Deleforge
• Participants:
  Joris Cosentino, Antoine Deleforge, Manuel Pariente, Emmanuel Vincent
• Abstract:
  This engineering project aims at further developing, expanding and transfering the Asteroid speech enhancement and separation toolkit recently released by the team 80.

ANR Full3DTalkingHead

• Title:
  Synthèse articulatoire phonétique
• Duration:
  Apr 2021 - Sep 2024
• Coordinator:
  Yves Laprie
• Partners:
  Gipsa-Lab (Grenoble), IADI (Nancy), LPP (Paris)
• Participants:
  Slim Ouni, Vinicius Ribeiro, Yves Laprie
• Abstract:
  The objective is to realize a complete three-dimensional digital talking head including the vocal tract from the vocal folds to the lips, the face and integrating the digital simulation of the aero-acoustic phenomena.

PEPR Cybersécurité, projet iPOP

• Title:
  Protection des données personnelles
• Duration:
  Oct 2022 – Sep 2028
• Coordinator:
  Vincent Roca (Inria PRIVATICS)
• Partners:
  Inria PRIVATICS (Lyon), COMETE, PETRUS (Saclay), MAGNET, SPIRALS (Lille), IRISA (Rennes), LIFO (Bourges), DCS (Nantes), CESICE (Grenoble), EDHEC (Lille), CNIL (Paris)
• Participant:
  Emmanuel Vincent
• Summary:
  The objectives of iPOP are to study the threats on privacy introduced by new digital technologies, and to design privacy-preserving solutions compatible with French and European regulations. Within this scope, Multispeech focuses on speech data.

ANR DFG M-PHASIS

• Title:
  Migration and Patterns of Hate Speech in Social Media - A Cross-cultural Perspective
• Duration:
  March 2018 – August 2022
• Coordinator:
  Angeliki Monnier and Christian Schemer
• Partners:
  CREM Université de Lorraine, LORIA, JGUM Johannes Gutenberg-Universität Mainz, SAAR Saarland University Saarbrücken
• Participant:
  Irina Illina, Dominique Fohr, Ashwin Geet D'sa
• Summary:
  Focusing on the social dimension of hate speech, the project M-PHASIS seeks to study the patterns of hate speech related to migrants in user-generated content.

ANR REFINED

• Title:
  Real-Time Artificial Intelligence for Hearing Aids
• Duration:
  Mar 2022 - Mar 2026
• Coordinator:
  CEA List (Saclay)
• Partners:
  CEA List (Saclay), Institut de l'audition (Paris), LORIA (Nancy)
• Participants:
  Paul Magron, Nasser-Eddine Monir, Romain Serizel
• Abstract:
  The Refined project brings together audiologists, computer scientists and specialists about hardware implementation to design new speech enhancement algorithms that fit both the needs of patients suffering of hearing losses and the computational constraints of hearing aid devices.

PIA2 ISITE LUE

• Title:
  Lorraine Université d’Excellence
• Duration:
  Sep 2019 - Jan 2023
• Coordinator:
  Université de Lorraine
• Participants:
  Tulika Bose, Dominique Fohr, Irina Illina
• Abstract:
  LUE (Lorraine Université d’Excellence) was designed as an “engine” for the development of excellence, by stimulating an original dialogue between knowledge fields. The IMPACT initiative OLKI (Open Language and Knowledge for Citizens) funds the PhD thesis of Tulika Bose on the detection and classification of hate speech.

11.1.1 Scientific events: organisation

11.1.2 Scientific events: selection

11.1.3 Journal

11.1.4 Invited talks

• Expert session “Speech anonymization”, 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (E. Vincent)
• Panel session “Beyond Words: Recognition, Spoofing, and Anonymization of Individual Traits in Speech”, 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (E. Vincent)
• Keynote “Speech anonymization”, 2022 Annual Conference of the UKRI CDT in Speech and Language Technologies and their Applications (E. Vincent)
• Seminar “Phase recovery for audio demixing: contributions and perspectives”, Neural DSP Technologies (Helsinki, Finland), Aug 2022 (P. Magron)
• Seminar "Prédiction de la forme géométrique du conduit vocal à partir de la suite de phonèmes à articuler, GDR ISIS seminar "Voix", Oct 2022 (Y. Laprie)
• Seminar "Hearing the Shape of a Room: When Signal Processing Meets Acoustics" Institut de Recherche Mathématique Avancée, Univ. Strasbourg, Oct 2022 (A. Deleforge)

11.1.5 Leadership within the scientific community

• Member of the Steering Committee of ISCA's Special Interest Group on Security and Privacy in Speech Communication (E. Vincent)
• Member of the Steering Committee for the CHiME Challenge series (E. Vincent)
• Secretary/Treasurer, executive member of AVISA (Auditory-VIsual Speech Association), an ISCA Special Interest Group (S. Ouni)
• Vice-president of AFCP - Association Francophone de la Communication Parlée (S. Ouni)
• Board member of AFCP - Association Francophone de la Communication Parlée (V. Colotte, Y. Laprie)
• Member of the Steering Group on Detection and Classification of Acoustic Scenes and Events (R. Serizel)
• Chair of the "Datasets and Challenges" subcommittee of the IEEE Technical Committee for Audio and Acoustics Signal Processing (A. Deleforge)

11.1.6 Scientific expertise

• Member of the Advisory Board of H2020 FVLLMONTI (E. Vincent)
• Coordinator of a task-force on large language models for the French National AI Research Plan (E. Vincent)
• Member of ANR Evaluation Committee 23 on Artificial Intelligence (D. Jouvet)
• Reviewer of an ANR CIFRE proposal (D. Jouvet)
• Reviewer of a project for the Czech Science Foundation (D. Jouvet, I. Illina)
• Reviewer of ANR generic call proposals (A. Deleforge, Y. Laprie, S. Ouni)
• Reviewer of a project for the Swiss National Science Foundation (S. Ouni)
• Reviewer of a project for the Canadian program MITACS (P. Magron)

11.1.7 Research administration

• Head of Science of Inria Nancy – Grand Est (E. Vincent)
• Scientific Director for the partnership between Inria and DFKI (E. Vincent, until Aug 2022)
• Member of Inria's Evaluation Committee (E. Vincent)
• Member of the Comité Espace Transfert of Inria Nancy – Grand Est (E. Vincent)
• Member of the hiring committee for Inria Senior Research Scientists (E. Vincent)
• Member of the hiring committee for Junior Research Scientists, Inria Lille – Nord Europe (E. Vincent)
• Member of the hiring committee for a Junior Professor Chair, Inria Nancy – Grand Est (E. Vincent)
• Member of the admission committee for Inria Starting Faculty Positions, Inria Nancy – Grand Est (E. Vincent)
• Member of the working group “Research visibility and attractiveness” of Métropole du Grand Nancy (E. Vincent)
• Member of the working group “Stimulate the production and use of health data” at City Healthcare 2022 in Nancy (E. Vincent)
• Member of pole scientifique automatique, mathématiques, informatique et leurs interactions (AM2I) of Université de Lorraine (I. Illina)
• Member of the CNU 27 (Conseil National des Universités) - Computer Science (S. Ouni)
• Member of the Commission du personnel scientifique (COMIPERS), Inria Nancy – Grand Est (R. Serizel)
• Member of the Commission de développement technologique (CDT), Inria Nancy – Grand Est (R. Serizel)
• Head of pole scientifique automatique, mathématiques, informatique et leurs interactions (AM2I) of Université de Lorraine (Y. Laprie)
• Member of Commission paritaire of Université de Lorraine (Y. Laprie)
• Member of the board, Université de Lorraine (Y. Laprie)
• President of the Commission pour l'Action et la Responsabilité Ecologique (CARE) Inria/Loria (A. Deleforge).
• Member of the Information & Edition Scientifique (IES) commision of INRIA (A. Deleforge, P. Magron)
• Participation in the hiring committee for the professor of general phonetics at Université de la Sorbonne Nouvelle (Y. Laprie)

11.2.1 Teaching

• BUT: I. Illina, Java programming (100 hours), Linux programming (58 hours), and Advanced Java programming (40 hours), L1, Université de Lorraine, France
• BUT: I. Illina, Supervision of student projects and internships (50 hours), L2, Université de Lorraine, France
• BUT: R. Serizel, Introduction to office tools (108 hours), Multimedia and web (20 hours), Documents and databases (20 hours), L1, Université de Lorraine, France
• BUT: R. Serizel, Multimedia content and indexing (14 hours), Content indexing and retrieval software (20 hours), L2, Université de Lorraine, France
• BUT: S. Ouni, Programming in Java (24 hours), Web Programming (24 hours), Graphical User Interface (96 hours), L1, Université de Lorraine, France
• BUT: S. Ouni, Advanced Algorihms (24 hours), L2, Université de Lorraine, France
• Licence: A. Bonneau, Phonetics (16 hours), L2, École d’audioprothèse, Université de Lorraine, France
• Licence: V. Colotte, Digital literacy and tools (hybrid courses, 50 hours), L1, Université de Lorraine, France
• Licence: V. Colotte, System (80 hours), L2-L3, Université de Lorraine, France
• Master: V. Colotte, Integration project: multimodal interaction with Pepper Robot (17 hours), M2, Université de Lorraine, France
• Master: V. Colotte and S. Ouni, Multimodal oral comunication (24 hours), M2, Université de Lorraine, France
• Master: V. Colotte, Introduction to speech processing (24 hours), M1, Université de Lorraine, France
• Master: Y. Laprie, Speech corpora (30 hours), M1, Université de Lorraine, France
• Master: S. Ouni, Multimedia in Distributed Information Systems (31 hours), M2, Université de Lorraine, France
• Master: R. Serizel, S. Ouni, P. Magron and V. Ribeiro, Oral speech processing (24 hours), M2, Université de Lorraine
• Master: E. Vincent and P. Magron, Neural networks (40 hours), M2, Université de Lorraine
• Master: A. Deleforge, Initiation to Machine Learning (24 hours), M1-M2, Télécom Physique Strasbourg
• Other: V. Colotte, Co-Responsible for NUMOC (Digital literacy by hybrid courses) for Université de Lorraine, France (for 7,000 students)
• Other: S. Ouni, Responsible of Année Spéciale DUT, Université de Lorraine, France

11.2.2 Supervision

• PhD: Adrien Dufraux, “Exploitation of noisy transcriptions for automatic speech recognition”, Apr 2022, E. Vincent, A. Brun (LORIA) and M. Douze (Meta AI) 64.
• PhD: Ashwin Geet D'Sa, "Expanding the training data for neural network based hate speech classification", May 2022, I. Illina, D. Fohr 65
• PhD: Ajinkya Kulkarni, “Expressivity transfer in deep learning based text-to-speech synthesis”, Jul 2022, V. Colotte and D. Jouvet 66.
• PhD: Mauricio Michel Olvera Zambrano, “Robust audio event detection”, Dec 2022, E. Vincent and G. Gasso (LITIS).
• PhD in progress: Tulika Bose, "Trandfert learning for abusive language detection", Sept 2019, I. Illina, D. Fohr
• PhD in progress: Pierre Champion, “Privacy preserving and personalized transformations for speech recognition”, Oct 2019, D. Jouvet and A. Larcher (LIUM).
• PhD in progress: Sandipana Dowerah, “Robust speaker verification from far-field speech”, Oct 2019, D. Jouvet and R. Serizel.
• PhD in progress: Shakeel Ahmad Sheikh, “Identifying disfluency in speakers with stuttering, and its rehabilitation, using DNN”, Oct 2019, S. Ouni
• PhD in progress: Georgios Zervakis, “Integration of symbolic knowledge into deep learning”, Nov 2019, M. Couceiro (LORIA) and E. Vincent.
• PhD in progress: Nicolas Zampieri, "classification des discours haineux dans les réseaux sociaux", Nov 2019, I. Illina, D. Fohr
• PhD in progress: Prerak Srivastava, “Hearing the walls of a room: machine learning for audio augmented reality”, Oct 2020, A. Deleforge and E. Vincent.
• PhD in progress: Stéphane Dilungana, "ACOUST.IA: artificial intelligence for building acoustics", Oct 2020, A. Deleforge, C. Foy, S. Faisan
• PhD in progress: Seyed Ahmad Hosseini, “3D sign language generation”, Feb 2021, S. Ouni and M. Sadeghi
• PhD in progress: Can Cui, “Joint and embedded automatic speech separation, diarization and recognition for the generation of meeting minutes”, Oct 2021, M. Sadeghi and E. Vincent
• PhD in progress: Sewade Olaolu Ogun, “Multi-factor data augmentation and transfer learning for embedded automatic speech recognition”, Oct 2021, V. Colotte and E. Vincent
• PhD in progress: Louis Abel, “Expressive audio-visual speech synthesis in an interaction context”, Oct 2021, S. Ouni and V. Colotte
• PhD in progress: Mickaëlla Grondin, “Modeling gestures and speech in interactions”, Nov 2021, S. Ouni and F. Hirsch (Praxiling)
• PhD in progress: Tom Sprunck, "Hearing the Shape of a Room: Towards Acoustic Super-Resolution", Nov 2021, A. Deleforge, C. Foy, Y. Privat
• PhD in progress: Robin San Roman, "Self supervised disentangled representation learning of audio data for compression and generation", Jun 2022, R. Serizel, A. Deleforge, Y. M. Adi, G. Synnaeve
• PhD in progress: Nasser-Eddine Monir, “Multichannel speech enhancement for patients with auditory neuropathy spectrum disorders”, Dec 2022, R. Serizel and P. Magron

11.2.3 Juries

• Participation in the HDR jury of Antoine Laurent (Le Mans Université, Jan 2022), D. Jouvet, reviewer
• Participation in the HDR jury of Jean-Luc Rouas (Université de Bordeaux, Mar 2022), E. Vincent, member
• Participation in the PhD jury of Adrien Llave (Centrale Supélec, Mar 2022), R. Serizel, member.
• Participation in the PhD jury of Nicolas Olivier (Univ. de Rennes, Mar 2022), S. Ouni, reviewer
• Participation in the PhD jury of Marie-Anne Lacroix (Université de Rennes 1, Apr 2022), R. Serizel, reviewer
• Participation in the PhD jury of Jiri Martinek (University of West Bohemia, Apr 2022), D. Jouvet, reviewer
• Participation in the PhD jury of Manon Macary (Le Mans Université, Jun 2022), D. Jouvet, reviewer
• Participation in the PhD jury of Yingming Gao (Technische Universität Dresden, Jun 2022), Y. Laprie, reviewer.
• Participation in the PhD jury of Pierre-Hugo Vial (IRIT, Toulouse, Nov 2022), P. Magron, member
• Participation in the PhD jury of Thomas BARGUIL, (Univ de Bretagne sud, Nov 2022), I. Illina, reviewer

11.3.1 Articles and contents

• The race to hide your voice, Wired, June 1, 2022 (E. Vincent)
• Interview for "Dynalips, solution de lipsync de pointe, anime MetaHuman !", Loria Actualité, and Facutel (université de Lorraine), July 2022 (S. Ouni)
• Les assistants vocaux haussent le ton, 01net, August 24, 2022 (E. Vincent)
• La reconnaissance vocale : vous allez tout comprendre, Monde Numérique, August 22, 2022 (E. Vincent)
• Interview for "Comment le numérique peut aider au diagnostic et à la prise en charge du bégaiement par les orthophonistes ?", Inria Actualité, Dec 2022 (S. Ouni)
• Interview for "REFINED : vers des prothèses auditives plus efficaces grâce à l’IA", Inria Actualité, Dec 2022 (R. Serizel)

11.3.2 Interventions

• Talk on speech anonymization at the Cycle de la Voix #3 seminar organized by Le VoiceLab, Feb 2022 (E. Vincent)
• Talk on speech neural synthesis at the Cycle de la Voix #9 seminar organized by Le VoiceLab, Nov 2022 (V. Colotte)
• Panel discussion at BpiFrance's Deeptech Tour in Metz, Mar 2022 (E. Vincent)
• Chiche, Lycée Saint-Exupéry, Fameck (2 classes), Mar 2022 (E. Vincent)
• Panel discussion at Viva Technology in Paris, Jun 2022 (E. Vincent)
• Chiche, Lycée Kastler, Stenay (3 classe), Mar 2022 (R. Serizel)
• Representations of the theater piece "Drone Control" with "Les sens des mots" at Reine Blanche Theater, Paris, Mar 2022, and Jardin Botanique, Villers-lès-Nancy, Sep 2022 (A. Deleforge)
• Representation of the theater piece "Le Procès du Robot" with "Crache Texte" at Faculté des Sciences et Technologies, Villers-lès-Nancy, Oct 2022 (A. Deleforge)

International journals

• 8 articleS.Sajjad Amini, M.Mohammad Soltanian, M.Mostafa Sadeghi and S.Shahrokh Ghaemmaghami. Non-Smooth Regularization: Improvement to Learning Framework through Extrapolation.IEEE Transactions on Signal Processing702022, 1213 - 1223
  HALDOI
• 9 articleS.Spandan Dey, M.Md Sahidullah and G.Goutam Saha. Cross-corpora spoken language identification with domain diversification and generalization.Computer Speech and LanguageFebruary 2023
  HAL
• 10 articleI. K.Ioannis K Douros, Y.Yu Xie, C.Chrysanthi Dourou, K.Karyna Isaieva, P.-A.Pierre-Andre Vussoz, J.Jacques Felblinger and Y.Yves Laprie. 3D dynamic spatiotemporal atlas of the vocal tract during consonant-vowel production from 2D real time MRI.Journal of Imaging89August 2022, 227
  HALDOI
• 11 articleZ.Zhiqi Kang, M.Mostafa Sadeghi, R.Radu Horaud and X.Xavier Alameda-Pineda. Expression-preserving face frontalization improves visually assisted speech processing.International Journal of Computer VisionJanuary 2023
  HALback to text
• 12 articleP.Paul Magron and C.Cédric Févotte. A majorization-minimization algorithm for nonnegative binary matrix factorization.IEEE Signal Processing LettersJune 2022
  HAL
• 13 articleP.Paul Magron and C.Cédric Févotte. Neural content-aware collaborative filtering for cold-start music recommendation.Data Mining and Knowledge Discovery2022
  HAL
• 14 articleO.Ondřej Mokrý, P.Paul Magron, T.Thomas Oberlin and C.Cédric Févotte. Algorithms for audio inpainting based on probabilistic nonnegative matrix factorization.Signal ProcessingMay 2023
  HAL
• 15 articleV.Vinicius Ribeiro, K.Karyna Isaieva, J.Justine Leclere, P.-A.Pierre-André Vuissoz and Y.Yves Laprie. Automatic generation of the complete vocal tract shape from the sequence of phonemes to be articulated.Speech Communication141April 2022, 1-13
  HALDOIback to text
• 16 articleA. S.Ali Shahin Shamsabadi, B. M.Brij Mohan Lal Srivastava, A.Aurélien Bellet, N.Nathalie Vauquier, E.Emmanuel Vincent, M.Mohamed Maouche, M.Marc Tommasi and N.Nicolas Papernot. Differentially private speaker anonymization.Proceedings on Privacy Enhancing Technologies20231January 2023
  HALback to text
• 17 articleS.Shakeel Sheikh, M.Md Sahidullah, F.Fabrice Hirsch and S.Slim Ouni. Machine Learning for Stuttering Identification: Review, Challenges & Future Directions.Neurocomputing5142022October 2022, 17
  HALDOIback to text
• 18 articleT.Tom Sprunck, A.Antoine Deleforge, Y.Yannick Privat and C.Cédric Foy. Gridless 3D Recovery of Image Sources from Room Impulse Responses.IEEE Signal Processing LettersNovember 2022
  HALDOIback to text
• 19 articleB. M.Brij Mohan Lal Srivastava, M.Mohamed Maouche, M.Md Sahidullah, E.Emmanuel Vincent, A.Aurélien Bellet, M.Marc Tommasi, N.Natalia Tomashenko, X.Xin Wang and J.Junichi Yamagishi. Privacy and utility of x-vector based speaker anonymization.IEEE/ACM Transactions on Audio, Speech and Language ProcessingJune 2022
  HALback to text
• 20 articleN.Natalia Tomashenko, X.Xin Wang, E.Emmanuel Vincent, J.Jose Patino, B. M.Brij Mohan Lal Srivastava, P.-G.Paul-Gauthier Noé, A.Andreas Nautsch, N.Nicholas Evans, J.Junichi Yamagishi, B.Benjamin O'Brien, A.Anaïs Chanclu, J.-F.Jean-François Bonastre, M.Massimiliano Todisco and M.Mohamed Maouche. The VoicePrivacy 2020 Challenge: Results and findings.Computer Speech and Language74July 2022, 101362
  HALDOIback to text
• 21 articleP.-H.Pierre-Hugo Vial, P.Paul Magron, T.Thomas Oberlin and C.Cedric Fevotte. Learning the Proximity Operator in Unfolded ADMM for Phase Retrieval.IEEE Signal Processing Letters292022, 1619-1623
  HALDOIback to text
• 22 articleX.Xiaolei Zhang, L.Lei Xie, E.Eric Fosler-Lussier and E.Emmanuel Vincent. Guest editorial: Special issue on advances in deep learning based speech processing.Neural Networks158January 2023
  HALDOIback to text

International peer-reviewed conferences

• 23 inproceedingsM.Mohammad Abdollahi, R.Romain Serizel, A.Alain Rakotomamonjy and G.Gilles Gasso. Integrating isolated examples with weakly-supervised sound event detection: a direct approach.7th Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE)Nancy, FranceNovember 2022
  HAL
• 24 inproceedingsT.Tulika Bose, N.Nikolaos Aletras, I.Irina Illina and D.Dominique Fohr. Domain Classification-based Source-specific Term Penalization for Domain Adaptation in Hate-speech Detection.COLING 2022 - Proceedings of the 29th International Conference on Computational LinguisticsGyeongju, South KoreaOctober 2022
  HALback to text
• 25 inproceedingsT.Tulika Bose, N.Nikolaos Aletras, I.Irina Illina and D.Dominique Fohr. Dynamically Refined Regularization for Improving Cross-corpora Hate Speech Detection.ACL 2022 - 60th meeting Association for Computational Linguistics FindingsDublin, IrelandMay 2022
  HALDOIback to text
• 26 inproceedingsT.Tulika Bose, I.Irina Illina and D.Dominique Fohr. Transferring Knowledge via Neighborhood-Aware Optimal Transport for Low-Resource Hate Speech Detection.Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (AACL-IJCNLP)Online, TaiwanNovember 2022
  HALback to text
• 27 inproceedings P.Pierre Champion, D.Denis Jouvet and A.Anthony Larcher. Are disentangled representations all you need to build speaker anonymization systems? Proc. Interspeech 2022 INTERSPEECH 2022 - Human and Humanizing Speech Technology incheon, South Korea September 2022
  HAL back to text
• 28 inproceedingsP.Pierre Champion, D.Denis Jouvet and A.Anthony Larcher. Privacy-Preserving Speech Representation Learning using Vector Quantization.JEP 2022 - Journées d'Études sur la ParoleJournées d'Études sur la ParoleÎle de Noirmoutier, FranceJune 2022
  HALback to text
• 29 inproceedingsS.Stéphane Dilungana, A.Antoine Deleforge, C.Cédric Foy and S.Sylvain Faisan. Geometry-Informed Estimation of Surface Absorption Profiles from Room Impulse Responses.30th European Signal Processing Conference (EUSIPCO)Belgrade, SerbiaIEEEAugust 2022, 867-871
  HALDOIback to text
• 30 inproceedings S.Sandipana Dowerah, R.Romain Serizel, D.Denis Jouvet, M.Mohammad Mohammadamini and D.Driss Matrouf. How to Leverage DNN-based speech enhancement for multi-channel speaker verification? 4th International Conference on Advances in Signal Processing and Artificial Intelligence (ASPAI' 2022) Corfu, Greece October 2022
  HAL back to text
• 31 inproceedingsS.Sandipana Dowerah, R.Romain Serizel, D.Denis Jouvet, M.M Mohammadamini and D.Driss Matrouf. Joint optimization of diffusion probabilistic-based multichannel speech enhancement with far-field speaker verification.IEEE SLT 2022Doha, QatarJanuary 2023
  HALback to text
• 32 inproceedingsJ.Janek Ebbers, R.Reinhold Haeb-Umbach and R.Romain Serizel. Threshold independent evaluation of sound event detection scores.ICASSP 2022 - IEEE International Conference on Acoustics, Speech and Signal ProcessingSingapore, SingaporeMay 2022
  HALDOI
• 33 inproceedingsA.Ashwin Geet d'Sa, I.Irina Illina, D.Dominique Fohr and A.Awais Akbar. Exploration of Multi-Corpus Learning for Hate Speech Classification in Low Resource Scenarios.TSD 2022 - 25th International Conference on Text, Speech and Dialogueproceedings of TSD 2022Brno, Czech RepublicSeptember 2022
  HALback to text
• 34 inproceedingsI.Irina Illina and D.Dominique Fohr. BERT Semantic Context Model for Efficient Speech Recognition.ICCAS 2022 - International Conference on Cognitive Aircraft Systemsproceedings of ICCAS 2022Toulouse, FranceJune 2022
  HALback to text
• 35 inproceedingsI.Irina Illina and D.Dominique Fohr. Semantic Information Investigation for Transformer-based Rescoring of N-best Speech Recognition.LTC 2023Proceedings of Language and Technology 2023Poznan, PolandApril 2023
  HAL
• 36 inproceedingsZ.Zhiqi Kang, M.Mostafa Sadeghi, R.Radu Horaud, X.Xavier Alameda-Pineda, J.Jacob Donley and A.Anurag Kumar. The Impact of Removing Head Movements on Audio-visual Speech Enhancement.ICASSP 2022 - IEEE International Conference on Acoustics, Speech and Signal ProcessingSingapore, SingaporeIEEEFebruary 2022, 1-5
  HALDOIback to text
• 37 inproceedingsR.Romain Karpinski, V.Vinicius Ribeiro and Y.Yves Laprie. Accelerating the Centerline Processing of Vocal Tract Shapes for Articulatory Synthesis.ICA 2022- 24th International Congress on AcousticsGyeongyu, South KoreaOctober 2022
  HALback to text
• 38 inproceedingsA. M.Ama Marina Kreme, B.Bruno Torrésani and A.Antoine Deleforge. Local time-frequency fading.Proceedings of the 24th International Congress on AcousticsICA 22 - International Congress on Acoustics 2022Gyeongju, South Korea2022
  HAL
• 39 inproceedingsA.Ajinkya Kulkarni, V.Vincent Colotte and D.Denis Jouvet. Analysis of expressivity transfer in non-autoregressive end-to-end multispeaker TTS systems.INTERSPEECH 2022Incheon, South KoreaSeptember 2022
  HALback to text
• 40 inproceedingsA.Ajinkya Kulkarni, V.Vincent Colotte and D.Denis Jouvet. Multi-stage attention for fine-grained expressivity transfer in multispeaker text-to-speech system.EUSIPCO 2022Belgrade, SerbiaAugust 2022
  HALback to text
• 41 inproceedingsM.-A.Marie-Anne Lacroix, N.Nancy Bertin, R.Romuald Rocher and P.Pascal Scalart. Vers un système embarqué de classification d'événements sonores : étude de l'impact de la quantification des descripteurs.GRETSI 2022 XXVIIIème Colloque Francophone de Traitement du Signal et des ImagesGRETSI 2022 XXVIIIème Colloque Francophone de Traitement du Signal et des ImagesNancy, FranceSeptember 2022
  HALback to text
• 42 inproceedingsM.Mohamed Maouche, B. M.Brij Mohan Lal Srivastava, N.Nathalie Vauquier, A.Aurélien Bellet, M.Marc Tommasi and E.Emmanuel Vincent. Enhancing speech privacy with slicing.Interspeech 2022 - Human and Humanizing Speech TechnologyIncheon, South KoreaSeptember 2022
  HALback to text
• 43 inproceedingsM.Mohammad Mohammadamini, D.Driss Matrouf, J.-F.Jean-François Bonastre, S.Sandipana Dowerah, R.Romain Serizel and D.Denis Jouvet. A Comprehensive Exploration of Noise Robustness and Noise Compensation in ResNet and TDNN-based Speaker Recognition Systems.EUSIPCO 2022 - 30th European Signal Processing ConferenceBelgrade, SerbiaAugust 2022
  HALback to text
• 44 inproceedingsM.Mohammad Mohammadamini, D.Driss Matrouf, J.-F. A.Jean-François A Bonastre, S.Sandipana Dowerah, R.Romain Serizel and D.Denis Jouvet. Barlow Twins self-supervised learning for robust speaker recognition.Interspeech 2022 - Human and Humanizing Speech TechnologyIncheon, South KoreaSeptember 2022
  HALDOIback to text
• 45 inproceedingsH.Hubert Nourtel, P.Pierre Champion, D.Denis Jouvet, A.Anthony Larcher and M.Marie Tahon. Evaluation of Speaker Anonymization on Emotional Speech.JEP 2022 - Journées d'Études sur la ParoleActes des 34e Journées d’Études sur la Parole — JEP2022 « Parole, Geste, Musique : des unités à leur organisation »Île de Noirmoutier, FranceJune 2022
  HALback to text
• 46 inproceedings S.Sewade Ogun, V.Vincent Colotte and E.Emmanuel Vincent. Can we use Common Voice to train a Multi-Speaker TTS system? The 2022 IEEE Spoken Language Technology Workshop (SLT 2022) Doha, Qatar January 2023
  HAL back to text
• 47 inproceedingsC.Cennet Oguz, I.Ivana Kruijff-Korbayová, P.Pascal Denis, E.Emmanuel Vincent and J.Josef van Genabith. Chop and change: Anaphora resolution in instructional cooking videos.Findings of AACL-IJCNLP 2022 - 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics - 12th International Joint Conference on Natural Language ProcessingTaipeh, TaiwanNovember 2022
  HALback to text
• 48 inproceedingsM.Michel Olvera, E.Emmanuel Vincent and G.Gilles Gasso. On the impact of normalization strategies in unsupervised adversarial domain adaptation for acoustic scene classification.ICASSP 2022 - IEEE International Conference on Acoustics, Speech and Signal ProcessingSingapore, SingaporeMay 2022
  HALDOIback to textback to text
• 49 inproceedingsV.Vinicius Ribeiro and Y.Yves Laprie. Autoencoder-Based Tongue Shape Estimation During Continuous Speech.23rd INTERSPEECH Conference on "Human and Humanizing Speech Technology”Incheon, South KoreaSeptember 2022
  HALback to text
• 50 inproceedingsF.Francesca Ronchini and R.Romain Serizel. A benchmark of state-of-the-art sound event detection systems evaluated on synthetic soundscapes.ICASSP 2022 - IEEE International Conference on Acoustics, Speech and Signal ProcessingSingapore/Virtual, SingaporeMay 2022
  HALDOI
• 51 inproceedingsD.Dana Ruiter, L.Liane Reiners, A.Ashwin Geet d'Sa, T.Thomas Kleinbauer, D.Dominique Fohr, I.Irina Illina, D.Dietrich Klakow, C.Christian Schemer and A.Angeliki Monnier. Placing M-Phasis on the Plurality of Hate: A Feature-Based Corpus of Hate Online.LREC 2022 – 13th Language Resources and Evaluation ConferenceLREC 2022 ProceedingsMarseille, FranceJune 2022, 791-804
  HALback to text
• 52 inproceedingsM.Mostafa Sadeghi and P.Paul Magron. A Sparsity-promoting Dictionary Model for Variational Autoencoders.INTERSPEECH 2022Incheon, South KoreaSeptember 2022
  HALback to text
• 53 inproceedingsS. A.Shakeel A Sheikh, M.Md Sahidullah, F.Fabrice Hirsch and S.Slim Ouni. End-to-End and Self-Supervised Learning for ComParE 2022 Stuttering Sub-Challenge.ACM Multimedia 2022 Computational Paralinguistics Challenge (ComParE)Lisbon, PortugalOctober 2022
  HALback to text
• 54 inproceedingsS.Shakeel Sheikh, M.Md Sahidullah, F.Fabrice Hirsch and S.Slim Ouni. Robust Stuttering Detection via Multi-task and Adversarial Learning.EUSIPCO 2022 - 30th European Signal Processing ConferenceBelgrade, SerbiaAugust 2022
  HALback to text
• 55 inproceedingsI. A.Imran Ahamad Sheikh, E.Emmanuel Vincent and I.Irina Illina. Transformer versus LSTM Language Models Trained on Uncertain ASR Hypotheses in Limited Data Scenarios.LREC 2022 - 13th Language Resources and Evaluation ConferenceMarseille, FranceJune 2022
  HALback to text
• 56 inproceedingsP.Prerak Srivastava, A.Antoine Deleforge and E.Emmanuel Vincent. Realistic sources, receivers and walls improve the generalisability of virtually-supervised blind acoustic parameter estimators.17th International Workshop on Acoustic Signal Enhancement (IWAENC)17th International Workshop on Acoustic Signal Enhancement (IWAENC)Bamberg, GermanySeptember 2022
  HALback to text
• 57 inproceedingsM. A.Mehmet Ali Tugtekin Turan, D.Dietrich Klakow, E.Emmanuel Vincent and D.Denis Jouvet. Adapting Language Models When Training on Privacy-Transformed Data.LREC 2022 - 13th Language Resources and Evaluation ConferenceMarseille, FranceJune 2022
  HALback to text
• 58 inproceedingsN.Nicolas Zampieri, C.Carlos Ramisch, I.Irina Illina and D.Dominique Fohr. Identification des Expressions Polylexicales dans les Tweets.RECITAL 2022- Traitement Automatique des Langues Naturelles (TALN)actes de TALN-RECITAL 2022Avignon, FranceJune 2022
  HALback to text
• 59 inproceedingsN.Nicolas Zampieri, C.Carlos Ramisch, I.Irina Illina and D.Dominique Fohr. Identification of Multiword Expressions in Tweets for Hate Speech Detection.LREC 2022 - 13th Edition of its Language Resources and Evaluation ConferenceLREC 2022 ProceedingsMarseille, FranceJune 2022
  HALback to text
• 60 inproceedingsG.Georgios Zervakis, E.Emmanuel Vincent, M.Miguel Couceiro, M.Marc Schoenauer and E.Esteban Marquer. An analogy based approach for solving target sense verification.NLPIR 2022 - 6th International Conference on Natural Language Processing and Information RetrievalBangkok, ThailandDecember 2022
  HALback to text

Conferences without proceedings

• 61 inproceedingsD.Domitille Caillat, L.Ludovic Marin, C.Christelle Dodane, F.Fabrice Hirsch, S.Slim Ouni, P.Pierre Slangen, P.Patrice Guyot, V.Vincent Colotte, A.Aliyah Morgenstern, L.Louis Abel, M.Mickaëlla Grondin-Verdon and J. L.Juliette Lozano Goupil. Synchronization of speech and gestures in an interactional context (SyncoGest Project).ISGS 2022 - 9th Conference of the International Society for Gesture StudiesChicago, United States2022
  HAL
• 62 inproceedingsS.Stéphanie Deckert, A.Agnès Piquard-Kipffer and A.Anne Bonneau. Perception of German fricatives by French dyslexic subjects.New Sounds 2022 Book of abstractsNew Sounds 2022, 10th International Symposium on the Acquisition of Second Language SpeechBarcelone, SpainApril 2022
  HALback to text

Edition (books, proceedings, special issue of a journal)

• 63 proceedingsM.Mathieu LagrangeA.Annamaria MesarosT.Thomas PellegriniG.Gael RichardR.Romain SerizelD.Dan StowellProceedings of the 7th Workshop on Detection and Classification of Acoustic Scenes and Events (DCASE 2022).Tampere UniversityNovember 2022, 1-225
  HAL

Doctoral dissertations and habilitation theses

• 64 thesisA.Adrien Dufraux. Leveraging noisy transcriptions for automatic speech recognition.Université de LorraineApril 2022
  HALback to textback to text
• 65 thesisA.Ashwin Geet d'Sa. Expanding the training data for neural network based hate speech classification.Université de LorraineMay 2022
  HALback to textback to text
• 66 thesisA.Ajinkya Kulkarni. Expressivity transfer in deep learning based text-to-speech synthesis.Université de LorraineJuly 2022
  HALback to textback to text
• 67 thesisR.Romain Serizel. Contributions to speech processing and ambient sound analysis.Université de LorraineMarch 2022
  HAL

Reports & preprints

• 68 miscL.Louis Bahrman, M.Marina Krémé, P.Paul Magron and A.Antoine Deleforge. Signal Inpainting from Fourier Magnitudes.October 2022
  HALback to text
• 69 miscL.Louis Delebecque, R.Romain Serizel and N.Nicolas Furnon. Towards an efficient computation of masks for multichannel speech enhancement.March 2022
  HAL
• 70 miscA.Ali Golmakani, M.Mostafa Sadeghi, X.Xavier Alameda-Pineda and R.Romain Serizel. Weighted variance variational autoencoder for speech enhancement.October 2022
  HAL
• 71 miscA.Ali Golmakani, M.Mostafa Sadeghi and R.Romain Serizel. AUDIO-VISUAL SPEECH ENHANCEMENT WITH A DEEP KALMAN FILTER GENERATIVE MODEL.October 2022
  HAL
• 72 miscF.Félix Gontier, R.Romain Serizel and C.Christophe Cerisara. SPICE+: EVALUATION OF AUTOMATIC AUDIO CAPTIONING SYSTEMS WITH PRE-TRAINED LANGUAGE MODELS.2022
  HAL
• 73 miscM.Marina Krémé and B.Bruno Torrésani. Étude d'un algorithme d'optimisation pour le fading temps-fréquence.June 2022
  HAL
• 74 miscM.Mostafa Sadeghi and R.Romain Serizel. FAST AND EFFICIENT SPEECH ENHANCEMENT WITH VARIATIONAL AUTOENCODERS.October 2022
  HAL
• 75 miscR.Romain Serizel, S.Samuele Cornell and N.Nicolas Turpault. Performance above all ? energy consumption vs. performance for machine listening, a study on dcase task 4 baseline.November 2022
  HALback to textback to text
• 76 miscP.Prerak Srivastava, A.Antoine Deleforge, A.Archontis Politis and E.Emmanuel Vincent. How to (Virtually) Train Your Sound Source Localizer.November 2022
  HALback to text
• 77 miscN.Natalia Tomashenko, X.Xin Wang, X.Xiaoxiao Miao, H.Hubert Nourtel, P.Pierre Champion, M.Massimiliano Todisco, E.Emmanuel Vincent, N.Nicholas Evans, J.Junichi Yamagishi and J.-F.Jean-François Bonastre. The VoicePrivacy 2022 Challenge Evaluation Plan.September 2022
  HALback to text
• 78 miscN.Natalia Tomashenko, X.Xin Wang, E.Emmanuel Vincent, J.Jose Patino, B. M.Brij Mohan Lal Srivastava, P.-G.Paul-Gauthier Noé, A.Andreas Nautsch, N.Nicholas Evans, J.Junichi Yamagishi, B.Benjamin O'Brien, A.Anaïs Chanclu, J.-F.Jean-François Bonastre, M.Massimiliano Todisco and M.Mohamed Maouche. Supplementary material to the paper The VoicePrivacy 2020 Challenge: Results and findings.September 2022
  HALback to text