Challenge 1: Developing frugal methods with robust expressivity.

The idea of frugal approaches means algorithms relying on a controlled use of computing resources, but also methods whose expressivity and flexibility provably relies on the versatile notion of sparsity. This is expected to avoid the current pitfalls of costly over-parameterizations and to robustify the approaches with respect to adversarial examples and overfitting. More specifically, it is essential to contribute to the understanding of methods based on neural networks, in order to improve their performance and most of all, their efficiency in resource-limited environments.

Challenge 2: Integrating models in learning algorithms.

To make statistical machine learning both more frugal and more interpretable, it is important to develop techniques able to exploit not only high-dimensional data but also models in various forms when available. When some partial knowledge is available about some phenomena related to the processed data, e.g. under the form of a physical model such as a partial differential equation, or as a graph capturing local or non-local correlations, the goal is to use this knowledge as an inspiration to adapt machine learning algorithms. The main challenge is to flexibly articulate a priori knowledge and data-driven information, in order to achieve a controlled extrapolation of predicted phenomena much beyond the particular type of data on which they were observed, and even in applications where training data is scarce.

Challenge 3: Guarantees on interpretability, explainability, and privacy.

The notion of sparsity and its structured avatars –notably via graphs– is known to play a fundamental role in ensuring the identifiability of decompositions in latent spaces, for example for high-dimensional inverse problems in signal processing. The team's ambition is to deploy these ideas to ensure not only frugality but also some level of explainability of decisions and an interpretability of learned parameters, which is an important societal stake for the acceptability of “algorithmic decisions”. Learning in small-dimensional latent spaces is also a way to spare computing resources and, by limiting the public exposure of data, it is expected to enable tunable and quantifiable tradeoffs between the utility of the developed methods and their ability to preserve privacy.

7.1.1 FAuST

• Keywords:
  Learning, Sparsity, Fast transform, Multilayer sparse factorisation
• Scientific Description:
  FAuST allows to approximate a given dense matrix by a product of sparse matrices, with considerable potential gains in terms of storage and speedup for matrix-vector multiplications.
• Functional Description:

  FAUST is a C++ toolbox designed to decompose a given dense matrix into a product of sparse matrices in order to reduce its computational complexity (both for storage and manipulation).

  Faust includes Matlab and Python wrappers and scripts to reproduce the experimental results of the following papers: - Le Magoarou L. and Gribonval R,. "Flexible multi-layer sparse approximations of matrices and applications", Journal of Selected Topics in Signal Processing, 2016. - Le Magoarou L., Gribonval R., Tremblay N. "Approximate fast graph Fourier transforms via multi-layer sparse", IEEE Transactions on Signal and Information Processing over Networks, 2018 - Quoc-Tung Le, Rémi Gribonval. Structured Support Exploration For Multilayer Sparse Matrix Factorization. ICASSP 2021 – IEEE International Conference on Acoustics, Speech and Signal Processing, Jun 2021, Toronto, Ontario, Canada. pp.1-5. - Sibylle Marcotte, Amélie Barbe, Rémi Gribonval, Titouan Vayer, Marc Sebban, et al.. Fast Multiscale Diffusion on Graphs. 2021.

• Release Contributions:

  Faust 1.x contains Matlab routines to reproduce experiments of the PANAMA team on learned fast transforms.

  Faust 2.x contains a C++ implementation with preliminary Matlab / Python wrappers.

  Faust 3.x includes Python and Matlab wrappers around a C++ core with GPU acceleration, new algorithms.

• News of the Year:

  In 2021, new algorithms bringing improved precision and/or accelerations were incorporated into Faust, GPU support was completed together with a systematic optimization of the code (including the ability to run it in float instead of double precision), and PIP packages were made available to ease the installation of faust.

  In 2020, major efforts were put into finalizing Python wrappers, producing tutorials using Jupyter notebooks and Matlab livescripts, as well as substantial refactoring of the code to optimize its efficiency and exploit GPUs.

  In april 2018, a Software Development Initiative (ADT REVELATION) started in for the maturation of FAuST. A first step was to complete and robustify Matlab wrappers, to code Python wrappers with the same functionality, and to setup a continuous integration process. A second step was to simplify the parameterization of the main algorithms. The roadmap for next year includes showcasing examples and optimizing computational efficiency.

  In 2017, new Matlab code for fast approximate Fourier Graph Transforms have been included. based on the approach described in the papers:

  -Luc Le Magoarou, Rémi Gribonval, "Are There Approximate Fast Fourier Transforms On Graphs?", ICASSP 2016 .

  -Luc Le Magoarou, Rémi Gribonval, Nicolas Tremblay, "Approximate fast graph Fourier transforms via multi-layer sparse approximations", IEEE Transactions on Signal and Information Processing over Networks,2017.

• URL:
  https://­faust.­inria.­fr/
• Publications:
  hal-01416110, hal-01627434, hal-01167948, hal-01254108, tel-01412558, hal-01156478, hal-01104696, hal-01158057, hal-03132013
• Contact:
  Remi Gribonval
• Participants:
  Luc Le Magoarou, Nicolas Tremblay, Remi Gribonval, Nicolas Bellot, Adrien Leman, Hakim Hadj-Djilani

8.1.1 Works on Gromov-Wasserstein: graph dictionary learning

Participants: Titouan Vayer.

Collaborations with Cédric Vincent-Cuaz (PhD student, MAASAI, Université Côte d'Azur), Rémi Flamary (CMAP, Ecole Polytechnique), Marco Corneli (MAASAI, Université Côte d'Azur) and Nicolas Courty (IRISA, Université Bretagne Sud).

The Gromov-Wasserstein (GW) distance is derived from optimal transport (OT) theory. The interest of OT lies both in its ability to provide relationships, connections, between sets of points and distances between probability distributions. By modeling graphs as probability distributions GW has become an important tool in many ML tasks involving structured data. Based on GW as a fidelity term, we proposed in 34 an efficient graph dictionary learning algorithm that allows to describe graphs as a simple composition of smaller graphs (atoms of the dictionary). We proposed a stochastic algorithm capable of learning a dictionary-like representation in the complex setting where the graphs in the dataset arrive progressively in time. We showed that these representations are particularly efficient for tasks such as change detection for structured data and clustering of graphs. We proposed an alternative approach in 48 which goal is to learn a single graph of large size whose subgraphs will best match (according to the GW criterion) the graphs of the dataset.

In another line of works, in collaboration with Clément Bonet (PhD student, IRISA, Université Bretagne-Sud), Nicolas Courty, François Septier (LMBA, Université de Bretagne Sud) and Lucas Drumetz ( Lab-STICC OSE, IMT Atlantique), we proposed an extension of the GW framerwork for shape matching problems 11. It consists in finding an optimal plan between the measures projected on a wisely chosen subspace and then completing it in a nearly optimal transport plan on the whole space. The advantage is to lower the computational complexity of the GW distance.

8.1.2 Diffused Wasserstein Distance for Optimal transport between attributed graphs

Participants: Paulo Gonçalves, Rémi Gribonval, Amélie Barbe, Titouan Vayer.

This work is a collaboration with Pierre Borgnat (CNRS) from the the Physics Lab of ENS de Lyon, Marc Sebban, Professor at the LabHC of University Jean Monet, and Sibylle Marcotte (student at ENS de Rennes).

In a series of recent articles, we proposed the Diffusion Wasserstein (DW) distance, a generalization of the standard Wasserstein to undirected and connected graphs where nodes are described by feature vectors. Using the heat diffusion equation constructed on the exponential kernel of the graph's Laplacian, we locally average the attributes of the nodes over a neighborhood that is controlled by the diffusion time. Like the fused Gromov-Wasserstein distance, this mixed distance allows to compute an optimal transport plan that captures both the structural and the feature information of the graphs. A big advantage of the Diffusion Distance though, is its computational cost that remains significantly inferior to that of the fused Gromov Wasserstein distance. Moreover, applied to different domain adaptation tasks, we experimentally showed that in many difficult situations, the DW distance was able to outperform the most recent concurrent methods.

To further reduce the computational cost of the diffusion Wasserstein distance, we proposed to use a Chebyshev approximation of the diffusion operator applied to the features vectors. In the course of this work, we were also able to tighten the theoretical approximation bounds, which in turn permits to significantly improve estimates of the polynomial order for a prescribed error 31.

Finally, to address the classical problem of tuning the diffusion time, the unique free parameter of DW distance, we devised a triplet loss based method that permits to find the best diffusion time in the context of domain adaptation tasks 27.

8.2.1 Mathematics of deep learning: approximation theory, scale-invariance, and regularization

Participants: Rémi Gribonval, Pierre Stock, Antoine Gonon, Elisa Riccietti, Vincent Schellekens.

Collaborations with Facebook AI Research, Paris, with Nicolas Brisebarre (ARIC team, ENS de Lyon), and with Yann Traonmilin (IMB, Bordeaux) and Samuel Vaiter (JAD, Dijon)

Our paper studying the expressivity of sparse deep neural networks from an approximation theoretic perspective and highlighting the role of depth to enable efficient approximation of functions with very limited smoothness was published this year 8. Motivated by the importance of quantizing networks besides pruning them to achieve sparsity, we started to investigate the approximation theoretic properties of quantized deep networks, with the objective of defining and comparing the corresponding approximation classes with the unquantized ones.

Neural networks with the ReLU activation function are described by weights and bias parameters, and realized a piecewise linear continuous function. Natural scalings and permutations operations on the parameters leave the realization unchanged, leading to equivalence classes of parameters that yield the same realization. These considerations in turn lead to the notion of identifiability – the ability to recover (the equivalence class of) parameters from the sole knowledge of the realization of the corresponding network. We studied this problem in depth throught the lens of a new embedding of ReLU neural network parameters of any depth. The proposed embedding is invariant to scalings and provides a locally linear parameterization of the realization of the network.Leveraging these two key properties, we derived some conditions under which a deep ReLU network is indeed locally identifiable from the knowledge of the realization on a finite set of samples. We studied the shallow case in more depth, establishing necessary and sufficient conditions for the network to be identifiable from a bounded subset 22.

An important challenge in deep learning is to promote sparsity during the learning phase using a regularizer. In the classical setting of linear inverse problems, it is well known that the ${\ell }^1$ norm is a convex regularizer lending itself to efficient optimization and endowed with stable recovery guarantees.

A particular challenge is to understand to what extent using an ${\ell }^1$ penalty in this context is also well-founded theoretically, and to possibly design alternate regularizers if possible. On the one hand, we started investigating the properties of minimizers of the ${\ell }^1$ norm in deep learning problems. On the other hand, we considered the abstract problem of recovering elements of a low-dimensional model set from under-determined linear measurements. Considering the minimization of a convex regularizer subject to a data fit constraint, we explored the notion of a "best" convex regularizer given a model set. This was formalized as a regularizer that maximizes a compliance measure with respect to the model. Several notions of compliance were studied and analytical expressions were obtained for compliance measures based on the best-known recovery guarantees with the restricted isometry property. This lead to a formal proof of the optimality of the ${\ell }^1$-norm for sparse recovery and of the nuclear norm for low-rank matrix recovery for these compliance measures. We also investigated the construction of an optimal convex regularizer using the example of sparsity in levels 46.

8.2.2 Algorithms for quantized networks

Participants: Rémi Gribonval, Pierre Stock, Elisa Riccietti.

Collaboration with Facebook AI Research, Paris

8.2.3 Deep sparse factorizations: hardness, algorithms and identifiability

Participants: Rémi Gribonval, Elisa Ricietti, Marion Foare, Léon Zheng, Quoc-Tung Le.

Collaboration with Valeo AI, Paris

Matrix factorization with sparsity constraints plays an important role in many machine learning and signal processing problems such as dictionary learning, data visualization, dimension reduction.

Last year, from an algorithmic perspective, we analyzed and fixed a weakness of proximal algorithms in sparse matrix factorization. We also described a new tractable proximal operator called Generalized Hungarian Method, associated to so-called $k$-regular matrices, which are useful for the factorization of a class of matrices associated to fast linear transforms. We further illustrated the effectiveness of our proposals by numerical experiments on the Hadamard Transform and magnetoencephalography matrix factorization. This work was published this year in a conference 29, and the new proximal operator was implemented in the FA$\mu$ST software library (see Section 7).

From a theoretical perspective, we considered the hardness and uniqueness properties of sparse matrix factorization. First, even with only two factors and a fixed, known support, we showed that optimizing the coefficients of the sparse factors can be an NP-hard problem. Besides, we studied the landscape of the corresponding optimization problem and exhibited "easy" instances where the problem can be solved to global optimality with an algorithm demonstrated to be orders of magnitude faster than classical gradient based methods 43. In complement, we investigated the essential uniqueness of sparse matrix factorizations, both with two factors 50 and in a multi-layer setting 49. We combined these results with a focus on so-called butterfly supports to achieve a multilayer sparse factorization algorithm able to learn fast transforms essentially at the cost of a single matrix-vector multiplication, with exact recovery guarantees 30. A first version of the corresponding algorithm was incorporated in the FA$\mu$ST software library (see Section 7) and is subject to software optimizations to further speed it up.

8.3.1 Theoretical and algorithmic foundations of compressive learning: sketches, kernels, and optimal transport

Participants: Rémi Gribonval, Titouan Vayer, Ayoub Belhadji, Vincent Schellekens, Luc Giffon, Léon Zheng.

Collaborations with Gilles Blanchard (Univ. Paris-Saclay), Yann Traonmilin (IMB, Bordeaux),Laurent Jacques and Vincent Schellekens (U. Louvain, Belgium), Nicolas Keriven (GIPSA-lab, Grenoble), Phil Schniter (Ohio State Univ.), and with Valeo AI

The compressive learning framework proposes to deal with the large scale of datasets by compressing them into a single vector of generalized random moments, called a sketch, from which the learning task is then performed. Our papers establishing statistical guarantees on the generalization error of this procedure, first in a general abstract setting illustrated on PCA 6, then for the specific case of compressive $k$-means and compressive Gaussian Mixture Modeling 16, were published this year. A tutorial paper on the principle and the main guarantees of compressive learning was also finalized and published this year 7.

Theoretical guarantees in compressive learning fundamentally rely on comparing certain metrics between probability distributions. This year we established some conditions under which the Wasserstein distance can be controlled by Maximum Mean Discrepancy (MMD) norms, which are defined using reproducing kernel Hilbert spaces. Based on the relations between the MMD and the Wasserstein distance, we provide new guarantees for compressive statistical learning by introducing and studying the concept of Wasserstein learnability of the learning task 47.

Dimension reduction in compressive learning exploits the ability to approximate certain kernels by finite dimensional quadratures. We studied a quadrature, proposed by Ermakov and Zolotukhin in the sixties, through the lens of kernel methods. The nodes of this quadrature rule follow the distribution of a determinantal point process, while the weights are defined through a linear system, similarly to the optimal kernel quadrature. We showed how these two classes of quadrature are related, and we proved a tractable formula of the expected value of the squared worst-case integration error on the unit ball of an RKHS of the former quadrature. In particular, this formula involves the eigenvalues of the corresponding kernel and leads to improving on the existing theoretical guarantees of the optimal kernel quadrature with determinantal point processes 28.

From a more empirical perspective, we pursued our efforts to make sketching for compressive learning more versatile and efficient. This notably involved exploring how to adapt the sketching pipeline to exploit optical processing units (OPUs) for energy-efficient fast random projection, and investigating the ability to exploit sketching in large-scale deep self-supervised learning scenarios.

Finally, making the connection between graph learning and sketching methods, we have recently started to study the practical possibility and theoretical limitations of using a sketching technique to estimate the precision matrix involved in the Graphical Lasso algorithm.

8.3.2 Privacy preservation

Participants: Rémi Gribonval, Clément Lalanne.

Collaborations with Aurélien Garivier (UMPA, ENS de Lyon) and SARUS, Paris; and with Laurent Jacques and Vincent Schellekens (U. Louvain, Belgium), Florimond Houssiau and Yves-Alexandre de Montjoye (Imperial College, London)

In the context of the Ph.D. thesis of Antoine Chatalic (in the PANAMA team in Rennes, defended last year) we showed 13 that a simple perturbation of the sketching mechanism with additive noise is sufficient to satisfy differential privacy, a well-established formalism for defining and quantifying the privacy of a random mechanism. We combined this with a feature subsampling mechanism, which reduces the computational cost without damaging privacy. The framework was applied to the tasks of Gaussian modeling, k-means clustering and principal component analysis (PCA), for which sharp privacy bounds were derived. Empirically, the quality (for subsequent learning) of the compressed representation produced by this mechanism is strongly related with the induced noise level, for which we gave analytical expressions.

This year we also addressed problem of differentially private estimation of multiple quantiles (MQ) of a dataset, a key building block in modern data analysis. We showed how to implement the non-smoothed Inverse Sensitivity (IS) mechanism for this specific problem and established that the resulting method is closely related to the recent JointExp algorithm, sharing in particular the same computational complexity and a similar efficiency. We also identified pitfalls of the two approaches on certain peaked distributions, and proposed a fix Numerical experiments showed that the empirical efficiency of the resulting algorithms is similar to the non-smoothed methods for non-degenerate datasets, but orders of magnitude better on real datasets with repeated values.

10.1.1 ANR IA Chaire : AllegroAssai

Participants: Rémi Gribonval [correspondant], Paulo Gonçalves, Elisa Ricietti, Marion Foare, Mathurin Massias, Léon Zheng, Quoc-Tung Le, Antoine Gonon, Titouan Vayer, Ayoub Belhadji, Luc Giffon, Clement Lalanne.

Duration of the project: 2020 - 2024.

AllegroAssai focuses on the design of machine learning techniques endowed both with statistical guarantees (to ensure their performance, fairness, privacy, etc.) and provable resource-efficiency (e.g. in terms of bytes and flops, which impact energy consumption and hardware costs), robustness in adversarial conditions for secure performance, and ability to leverage domain-specific models and expert knowledge. The vision of AllegroAssai is that the versatile notion of sparsity, together with sketching techniques using random features, are key in harnessing these fundamental tradeoffs. The first pillar of the project is to investigate sparsely connected deep networks, to understand the tradeoffs between the approximation capacity of a network architecture (ResNet, U-net, etc.) and its “trainability” with provably-good algorithms. A major endeavor is to design efficient regularizers promoting sparsely connected networks with provable robustness in adversarial settings. The second pillar revolves around the design and analysis of provably-good end-to-end sketching pipelines for versatile and resource-efficient large-scale learning, with controlled complexity driven by the structure of the data and that of the task rather than the dataset size.

10.1.2 ANR DataRedux

Participants: Paulo Gonçalves [correspondant], Rémi Gribonval, Marion Foare, Israel Campero Jurado.

Duration of the project: February 2020 - January 2024.

DataRedux puts forward an innovative framework to reduce networked data complexity while preserving its richness, by working at intermediate scales (“mesoscales”). Our objective is to reach a fundamental breakthrough in the theoretical understanding and representation of rich and complex networked datasets for use in predictive data-driven models. Our main novelty is to define network reduction techniques in relation with the dynamical processes occurring on the networks. To this aim, we will develop methods to go from data to information and knowledge at different scales in a human-accessible way by extracting structures from high-resolution, diverse and heterogeneous data. Our methodology will involve the identification of the most relevant subparts of time-resolved datasets while remapping the remaining parts of the system, the simultaneous structural-temporal representations of time-varying networks, the development of parsimonious data representations extracting meaningful structures at mesoscales (“mesostructures”), and the building of models of interactions that include mesostructures of various types. Our aim is to identify data aggregation methods at intermediate scales and new types of data representations in relation with dynamical processes, that carry the richness of information of the original data, while keeping their most relevant patterns for their manageable integration in data-driven numerical models for decision making and actionable insights.

10.1.3 ANR Darling

Participants: Paulo Gonçalves [correspondant], Rémi Gribonval, Marion Foare.

Duration of the project: February 2020 - January 2024.

This project meets the compelling demand of developing a unified framework for distributed knowledge extraction and learning from graph data streaming using in-network adaptive processing, and adjoining powerful recent mathematical tools to analyze and improve performances. The project draws on three major parallel directions of research: network diffusion, signal processing on graphs, and random matrix theory which DARLING aims at unifying into a holistic dynamic network processing framework. Signal processing on graphs has recently provided a comprehensive set of basic instruments allowing for signal on graph filtering or sampling, but it is limited to static signal models. Network diffusion on the opposite inherently assumes models of time varying graphs and signals, and has pursued the path of proposing and understanding the performance of distributed dynamic inference on graphs. Both areas are however limited by their assuming either deterministic graph or signal models, thereby entailing often inflexible and difficult-to-grasp theoretical results. Random matrix theory for random graph inference has taken a parallel road in explicitly studying the performance, thereby drawing limitations and providing directions of improvement, of graph-based algorithms (e.g., spectral clustering methods). The ambition of DARLING lies in the development of network diffusion-type algorithms anchored in the graph signal processing lore, rather than heuristics, which shall systematically be analyzed and improved through random matrix analysis on elementary graph models. We believe that this original communion of as yet remote areas has the potential to path the pave to the emergence of the critically needed future field of dynamical network signal processing.

10.1.4 GDR ISIS project MOMIGS

Participants: Elisa Riccietti [correspondant], Marion Foare, Trieu Vy Le Hoang, Paulo Gonçalves.

Duration of the project: September 2021 - September 2023.

This project focuses on large scale optimization problems in signal processing and imaging. A natural way to tackle them is to exploit their underlying structure, and to represent them at different resolution levels. The use of multiresolution schemes, such as wavelets transforms, is not new in imaging and is widely used to define regularization strategies. However, such techniques could be used to a wider extent, in order to accelerate the optimization algorithms used for their solution and to tackle large datasets. Techniques based on such ideas are usually called multilevel optimization methods and are well-known and widely used in the field of smooth optimization and especially in the solution of partial differential equations. Optimization problems arising in image reconstruction are however usually nonsmooth and thus solved by proximal methods. Such approaches are efficient for small-scale problems but still computationally demanding for problems with very high-dimensional data. The ambition of this project is thus to combine proximal methods and multiresolution analysis not only as a regularization, but as a solution to accelerate proximal algorithms.

10.2.1 Labex CominLabs LeanAI

Participants: Elisa Riccietti [correspondant], Rémi Gribonval.

Duration of the project: October 2021-December 2024.

Collaboration with Silviu-Ioan Filip and Olivier Sentieys (IRISA, Rennes), Anastasia Volkova (LS2N Nantes)

The LeanAI project aims at developing a comprehensive and flexible framework for mixed-precision optimization. The project is motivated by the increasing demand for intelligent edge devices capable of on-site learning, driven by the recent developments in deep learning. The realization of such systems is a massive challenge due to the limited resources available in an embedded context and the massive training costs for state-of-the-art deep neural networks. In this project we attack these problems at the arithmetic and algorithmic levels by exploring the design of new mixed numerical precision algorithms, energy-efficient and capable of offering increased performance in a resource-restricted environment. The ambition of the project is to develop more flexible and faster techniques than existing reduced-precision gradient algorithms, by determining the best numeric formats to be used in combination with this kind of methods, rules to dynamically adjust the precision and extension of such techniques to second-order and multilevel strategies.

10.2.2 Labex Emerging Topics

Participants: Marion Foare [correspondant].

Duration of the project: April 2019-December 2022.

Collaboration with Eric Van Reeth (Creatis, Lyon)

Magnetic Resonance Imaging (MRI) is an extremely important anatomical and functional imaging technique, widely used by physicists to establish medical diagnosis. Acquiring high resolution volumes is desirable in many clinical and pre-clinical applications to accurately adapt the treatment to the measurements, or simply obtain highly resolved images of small anatomical structures. However, directly acquiring high-resolution volumes implies: i) long scanning times, which are often not tolerated by patients and children, and ii) images with low signal-to-noise ratio. Therefore, it is of particular interest to quickly acquire low-resolution volumes, and enhance their resolution as a post-processing step.

This project aims at developing new techniques to build super-resolution images for 3D MRI, that can take into account more physical constraints, such as prior medical knowledge, and to derive efficient machine learning algorithms suited for large scale data, with theoretical guarantees. In particular, we explore specialized piecewise smooth reconstruction variational methods, like the Mumford-Shah (MS) and the Total Variation (TV) variants, and to adapt their fitting terms as well as their optimization algorithms. The main originality of this project is to combine resolution enhancement and segmentation in MRI (usually performed as two distinct post-processing steps), starting from the MS model, a seminal tool originally designed for image denoising and segmentation tasks. This approach will improve the quality of the reconstruction both in terms of sharpness and smoothness, and help the doctors with reaching a diagnosis.

11.1.1 Scientific events: organisation

11.1.2 Scientific events: selection

11.1.3 Journal

11.1.4 Invited talks

• R. Gribonval was a keynote speaker at the international conference EUSIPCO 2021 and an invited speaker at the national conference CAp21.
• E. Riccietti was an invited speaker at 13th JLESC Workshop.

11.1.5 Leadership within the scientific community

• Rémi Gribonval is a member of the Scientific Committee of RT MIA (formerly GDR MIA)
• Rémi Gribonval is a member of the Comité de Liaison SIGMA-SMAI
• Rémi Gribonval is a member of the Cellule ERC of INS2I, mentoring for ERC candidates in the STIC domain

11.1.6 Scientific expertise

• Rémi Gribonval is a member of the Scientific Advisory Board (vice-president) of the Acoustics Research Institute of the Austrian Academy of Sciences, and a member of the Commission Prospective of Institut de Mathématiques de Marseille
• Rémi Gribonval, member of the EURASIP Special Area Team (SAT) on Signal and Data Analytics for Machine Learning (SiG-DML) since 2015.

11.1.7 Research administration

• Paulo Gonçalves is Deputy Scientific Director of the new research center of Inria in Lyon.

11.2.1 Teaching

• Master :
  • Rémi Gribonval: Inverse problems and high dimension; Mathematical foundations of deep neural networks; Concentration of measure in probability and high-dimensional statistical learning; M2, ENS Lyon
• Engineer cycle (Bac+3 to Bac+5):
  • Paulo Gonçalves: Traitement du Signal (déterministe, aléatoire, numérique), Estimation statistique. 80 heures Eq. TD. CPE Lyon, France
  • Marion Foare: Traitement du Signal (déterministe, numérique, aléatoire), Traitement et analyse d'images, Optimisation, Compression, Projets. 280 heures Eq. TD. CPE Lyon, France
  • Elisa Riccietti: M1 course Optimization and Approximation (28h) and 19h of tutor responsibility at ENS Lyon

11.2.2 Supervision

All PhD students of the team are co-supervised by at least one team member. In addition, some team members are involved in the co-supervision of students hosted in other labs.

• Marion Foare is involved in the co-supervision of the Ph.D. of Hoang Trieu Vy Le since 2021 (Laboratoire de Physique, Lyon).
• Elisa Riccietti is involved in the co-supervision of the Ph.D. of Valentin Mercier since 2021 (IRIT, Toulouse).

The following PhDs were defended in DANTE in 2021:

• Pierre Stock, Université de Lyon 40 (funded by ANRT and Facebook Artificial Intelligence Research; co-supervisors Rémi Gribonval and Hervé Jégou), Efficiency and Redundancy in Deep Learning Models : Theoretical Considerations and Practical Applications, April 2021
• Amélie Barbe, Université de Lyon (funded by ACADEMICS project, IdexLyon; co-supervisors Paulo Gonçalves, Pierre Borgnat and Marc Sebban), Diffusion-Wasserstein distances for attributed graphs, December 2021

11.2.3 Juries

Members of the DANTE team participated to the following juries

• PhD juries: Alexandre Araujo (Université Paris IX Dauphine, member); Marina Kremé (Aix-Marseille Université, chair); Pierre Humbert (University Paris-Saclay, chair); Raphaël Truffet (Université de Rennes I, chair); Vincent Schellekens (Université Catholique de Louvain, reviewer), PhD defence session at University of Florence (member)

International journals

• 11 articleC.Clément Bonet, T.Titouan Vayer, N.Nicolas Courty, F.François Septier and L.Lucas Drumetz. Subspace Detours Meet Gromov-Wasserstein.Algorithms14December 2021, 1-29
  HALDOIback to text
• 12 articleA.Amel Chadda, M.Marija Stojanova, T.Thomas Begin, A.Anthony Busson and I.Isabelle Guérin-Lassous. Assigning Channels in WLANs with Channel Bonding: A Fair and Robust Strategy.Computer NetworksJune 2021, 1-17
  HAL
• 13 articleA.Antoine Chatalic, V.Vincent Schellekens, F.Florimond Houssiau, Y.-A.Yves-Alexandre De Montjoye, L.Laurent Jacques and R.Rémi Gribonval. Compressive Learning with Privacy Guarantees.Information and Inference2021
  HALDOIback to text
• 14 articleC.Clément Gaultier, S.Srđan Kitić, R.Rémi Gribonval and N.Nancy Bertin. Sparsity-based audio declipping methods: selected overview, new algorithms, and large-scale evaluation.IEEE/ACM Transactions on Audio, Speech and Language Processing292021, 1174-1187
  HALDOI
• 15 articleR.Rémi Gribonval, G.Gilles Blanchard, N.Nicolas Keriven and Y.Yann Traonmilin. Compressive Statistical Learning with Random Feature Moments.Mathematical Statistics and Learning32August 2021, 113–164
  HALDOI
• 16 articleR.Rémi Gribonval, G.Gilles Blanchard, N.Nicolas Keriven and Y.Yann Traonmilin. Statistical Learning Guarantees for Compressive Clustering and Compressive Mixture Modeling.Mathematical Statistics and Learning32August 2021, 165–257
  HALDOIback to text
• 17 articleR.Rémi Gribonval, A.Antoine Chatalic, N.Nicolas Keriven, V.Vincent Schellekens, L.Laurent Jacques and P.Philip Schniter. Sketching Data Sets for Large-Scale Learning: Keeping only what you need.IEEE Signal Processing Magazine385September 2021, 12-36
  HALDOI
• 18 articleR.Rémi Gribonval, G.Gitta Kutyniok, M.Morten Nielsen and F.Felix Voigtlaender. Approximation spaces of deep neural networks.Constructive Approximation2021
  HALDOI
• 19 articleR.Rémy Grünblatt, I.Isabelle Guérin-Lassous and O.Olivier Simonin. A distributed antenna orientation solution for optimizing communications in a fleet of UAVs.Computer Communications181January 2022, 102-115
  HALDOI
• 20 articleB.Bo Jiang, P.Philippe Nain and D.Don Towsley. Covert Cycle Stealing in a Single FIFO Server.ACM Transactions on Modeling and Performance Evaluation of Computing Systems2021, 1-35
  HAL
• 21 articleP.Philippe Nain, N. K.Nitish K Panigrahy, P.Prithwish Basu and D.Don Towsley. One-dimensional Service Networks and Batch Service Queues.Queueing Systems2021
  HAL
• 22 articleP.Pierre Stock and R.Rémi Gribonval. An Embedding of ReLU Networks and an Analysis of their Identifiability.Constructive Approximation2022
  HALback to text
• 23 articleM.Marija Stojanova, T.Thomas Begin and A.Anthony Busson. A Markov Model for Performance Evaluation of Channel Bonding in IEEE 802.11.Ad Hoc Networks2021
  HAL
• 24 articleS.Samuel Unicomb, G.Gerardo Iñiguez, J.James Gleeson and M.Màrton Karsai. Dynamics of cascades on burstiness-controlled temporal networks.Nature Communications121December 2021, 1-9
  HALDOI
• 25 articleG.Gayane Vardoyan, S.Saikat Guha, P.Philippe Nain and D.Don Towsley. On the Stochastic Analysis of a Quantum Entanglement Distribution Switch.IEEE Transactions on Quantum EngineeringFebruary 2021
  HAL

International peer-reviewed conferences

• 26 inproceedingsL.Lafdal Abdelwedoud, A.Anthony Busson and I.Isabelle Guérin-Lassous. Use of a Weighted Conflict Graph in the Channel Selection Operation for Wi-Fi Networks.WONS 2021 - 16th Wireless On-demand Network systems and Services ConferenceVirtual Conference, FranceMarch 2021, 1-4
  HAL
• 27 inproceedingsA.Amélie Barbe, P.Paulo Gonçalves, M.Marc Sebban, P.Pierre Borgnat, R.Rémi Gribonval and T.Titouan Vayer. Optimization of the Diffusion Time in Graph Diffused-Wasserstein Distances: Application to Domain Adaptation.ICTAI 2021 - 33rd IEEE International Conference on Tools with Artificial IntelligenceVirtual conference, FranceIEEENovember 2021, 1-8
  HALback to text
• 28 inproceedingsA.Ayoub Belhadji. An analysis of Ermakov-Zolotukhin quadrature using kernels.NeurIPS 2021 - 35th Conference on Neural Information Processing SystemsVirtual-only Conference, AustraliaDecember 2021, 1-17
  HALback to text
• 29 inproceedingsQ.-T.Quoc-Tung Le and R.Rémi Gribonval. Structured Support Exploration For Multilayer Sparse Matrix Factorization.ICASSP 2021 - IEEE International Conference on Acoustics, Speech and Signal ProcessingToronto, Ontario, CanadaIEEEJune 2021, 1-5
  HALback to text
• 30 inproceedingsQ.-T.Quoc-Tung Le, L.Léon Zheng, E.Elisa Riccietti and R.Rémi Gribonval. Fast learning of fast transforms, with guarantees.ICASSP 2022 - IEEE International Conference on Acoustics, Speech and Signal ProcessingSingapore, SingaporeMay 2022
  HALback to text
• 31 inproceedingsS.Sibylle Marcotte, A.Amélie Barbe, R.Rémi Gribonval, T.Titouan Vayer, M.Marc Sebban, P.Pierre Borgnat and P.Paulo Gonçalves. Fast Multiscale Diffusion on Graphs.ICASSP, IEEE International Conference on Acoustics, Speech and Signal ProcessingSingapore, SingaporeMay 2022
  HALback to text
• 32 inproceedingsN.Nina Santi, R.Rémy Grünblatt, B.Brandon Foubert, A.Aroosa Hameed, J.John Violos, A.Aris Leivadeas and N.Nathalie Mitton. Automated and Reproducible Application Traces Generation for IoT Applications.Q2SWinet 2021 - 17th ACM Symposium on QoS and Security for Wireless and Mobile NetworksAlicante, SpainACMNovember 2021, 1-8
  HALDOI
• 33 inproceedingsP.Pierre Stock, A.Angela Fan, B.Benjamin Graham, E.Edouard Grave, R.Rémi Gribonval, H.Herve Jegou and A.Armand Joulin. Training with Quantization Noise for Extreme Model Compression.International Conference on Learning Representations 2021Vienna, AustriaMay 2021
  HALback to text
• 34 inproceedingsC.Cédric Vincent-Cuaz, T.Titouan Vayer, R.Rémi Flamary, M.Marco Corneli and N.Nicolas Courty. Online Graph Dictionary Learning.ICML 2021 - 38th International Conference on Machine LearningVirtual Conference, United States2021
  HALback to text

Conferences without proceedings

• 35 inproceedingsA.Anthony Bardou, T.Thomas Begin and A.Anthony Busson. Improving the Spatial Reuse in IEEE 802.11ax WLANs: A Multi-Armed Bandit Approach.MSWiM'21 - 24th ACM Conference on Modeling, Analysis and Simulation of Wireless and Mobile SystemsAlicante, SpainACMNovember 2021
  HAL
• 36 inproceedingsA.Alexandre Bonnefond, O.Olivier Simonin and I.Isabelle Guérin-Lassous. Extension des Modèles de Flocking aux Environnements avec Obstacles et Communications Dégradées.JFSMABordeaux, FranceJune 2021
  HAL
• 37 inproceedingsA.Alexandre Bonnefond, O.Olivier Simonin and I.Isabelle Guérin-Lassous. Extension of Flocking Models to Environments with Obstacles and Degraded Communications.IROS 2021 - IEEE/RSJ International Conference on Intelligent Robots and SystemsPrague / Virtual, Czech RepublicIEEEJuly 2021, 1-7
  HAL
• 38 inproceedingsE.Esther Guérin, T.Thomas Begin, A.Anthony Busson and I.Isabelle Guérin-Lassous. Towards a Throughput and Energy Efficient Association Strategy for Wi-Fi/LiFi Heterogeneous Networks.PE-WASUN 2021 - 18th ACM International Symposium on Performance Evaluation of Wireless Ad Hoc, Sensor, and Ubiquitous NetworksAlicante, SpainACMNovember 2021
  HALDOI

Doctoral dissertations and habilitation theses

• 39 thesisR.Rémy Grünblatt. From WiFi Performance Evaluation to Controlled Mobility in Drone Networks.Université Claude Bernard Lyon 1January 2021
  HAL
• 40 thesisP.Pierre Stock. Efficiency and Redundancy in Deep Learning Models : Theoretical Considerations and Practical Applications.Université de LyonApril 2021
  HALback to textback to textback to text

Reports & preprints

• 41 miscR.Rémi Gribonval, A.Antoine Chatalic, N.Nicolas Keriven, V.Vincent Schellekens, L.Laurent Jacques and P.Philip Schniter. Sketching Datasets for Large-Scale Learning (long version).January 2021
  HAL
• 42 miscB.Bo Jiang, P.Philippe Nain and D.Don Towsley. Covert Cycle Stealing in a Single FIFO Server (extended version).May 2021
  HAL
• 43 miscQ.-T.Quoc-Tung Le, E.Elisa Riccietti and R.Rémi Gribonval. Spurious Valleys, Spurious Minima and NP-hardness of Sparse Matrix Factorization With Fixed Support.May 2021
  HALback to text
• 44 miscP.Philippe Nain, G.Gayane Vardoyan, S.Saikat Guha and D.Don Towsley. Analysis of a Tripartite Entanglement Distribution Switch.January 2022
  HAL
• 45 miscB.Barbara Pascal, P.Patrice Abry, N.Nelly Pustelnik, S. g.Stéphane g. Roux, R.Rémi Gribonval and P.Patrick Flandrin. Nonsmooth convex optimization to estimate the Covid-19 reproduction number space-time evolution with robustness against low quality data.September 2021
  HALback to text
• 46 miscY.Yann Traonmilin, R.Rémi Gribonval and S.Samuel Vaiter. A theory of optimal convex regularization for low-dimensional recovery.December 2021
  HALback to text
• 47 miscT.Titouan Vayer and R.Rémi Gribonval. Controlling Wasserstein distances by Kernel norms with application to Compressive Statistical Learning.December 2021
  HALback to text
• 48 miscC.Cédric Vincent-Cuaz, R.Rémi Flamary, M.Marco Corneli, T.Titouan Vayer and N.Nicolas Courty. Semi-relaxed Gromov Wasserstein divergence with applications on graphs.October 2021
  HALback to text
• 49 miscL.Léon Zheng, E.Elisa Riccietti and R.Rémi Gribonval. Efficient Identification of Butterfly Sparse Matrix Factorizations.February 2022
  HALback to text
• 50 miscL.Léon Zheng, E.Elisa Riccietti and R.Rémi Gribonval. Identifiability in Two-Layer Sparse Matrix Factorization.November 2021
  HALback to text

Softwares

• 51 softwareQ.-T.Quoc-Tung Le and R.Rémi Gribonval. Code for the paper "Structured Support Exploration For Multilayer Sparse Matrix Factorization".February 2022BSD-3 Clause License
  HALSoftwage HeritageVCS
• 52 softwareS.Sibylle Marcotte, A.Amélie Barbe, R.Rémi Gribonval, T.Titouan Vayer, M.Marc Sebban, P.Pierre Borgnat and P.Paulo Gonçalves. Code for reproducible research - Fast Multiscale Diffusion on Graphs.February 2022BSD 3-Clause License
  HALVCS
• 53 softwareL.Léon Zheng, Q.-T.Quoc-Tung Le, E.Elisa Riccietti and R.Rémi Gribonval. Code for reproducible research - Fast learning of fast transforms, with guarantees.February 2022BSD 3-Clause License
  HALSoftwage HeritageVCS