6.1.1 Cyanure

• Name:
  Cyanure: An Open-Source Toolbox for Empirical Risk Minimization
• Keyword:
  Machine learning
• Functional Description:
  Cyanure is an open-source C++ software package with a Python interface. The goal of Arsenic is to provide state-of-the-art solvers for learning linear models, based on stochastic variance-reduced stochastic optimization with acceleration mechanisms and Quasi-Newton principles. Arsenic can handle a large variety of loss functions (logistic, square, squared hinge, multinomial logistic) and regularization functions (l2, l1, elastic-net, fused Lasso, multi-task group Lasso). It provides a simple Python API, which is very close to that of scikit-learn, which should be extended to other languages such as R or Matlab in a near future.
• Release Contributions:
  packaging on conda and pipy + various improvements
• URL:
  http://­thoth.­inrialpes.­fr/­people/­mairal/­arsenic/­welcome.­html
• Contact:
  Julien Mairal
• Participants:
  Julien Mairal, Thomas Ryckeboer

Self-Supervised Models are Continual Learners

Participants: Enrico Fini, Victor Turrisi da Costa, Xavier Alameda-Pineda, Elisa Ricci, Karteek Alahari, Julien Mairal.

fig.

LiDARTouch: Monocular metric depth estimation with a few-beam LiDAR

Participants: Florent Bartoccioni, Eloi Zablocki, Patrick Pérez, Matthieu Cord, Karteek Alahari.

In this paper 1, we address the task of monocular depth prediction, a key component of many autonomous systems, by self-supervised deep learning. Existing methods are either fully-supervised with an additional expensive LiDAR (32 or 64 beams) as input or self-supervised with camera-only methods, much cheaper, but suffering from scale ambiguity and infinite depth problems. In contrast, we introduce LiDARTouch, a novel method combining a monocular camera with a cheap minimal 4-beam LiDAR input, typical of laser scanners currently used in the automotive industry. We introduce a new self-supervision scheme to leverage this very sparse LiDAR input at three complementary levels. While being extremely sparse, we show that the use of a few-beam LiDAR alleviate the scaling ambiguity and infinite depth problems that camera-only methods suffer from. We also reach competitive performances with respect to fully-supervised depth completion methods while being significantly cheaper and more annotation friendly. Our method can be trained on any domain with no modification, and it can thus bring accurate and metric depth estimation at a vehicle fleet scale. In Figure 2, we present three examples along with selected close-ups highlighting the infinite-depth problem. For example, on the leftmost column, we observe a typical ‘hole’ in the depth map where previous `Depth Estimation` method estimates a vehicle three times as far as its true distance. In contrast, by leveraging small touches of LIDAR we disambiguate the prediction and can accurately and safely handle moving objects with no relative motion, typical of cars in fluid traffic.

fig.

LaRa: Latents and Rays for Multi-Camera Bird's-Eye-View Semantic Segmentation

Participants: Florent Bartoccioni, Eloi Zablocki, Andrei Bursuc, Patrick Pérez, Matthieu Cord, Karteek Alahari.

Recent works in autonomous driving have widely adopted the bird’s-eye-view (BEV) semantic map as an intermediate representation of the world. Online prediction of these BEV maps involves non-trivial operations such as multi-camera data extraction as well as fusion and projection into a common top-view grid. This is usually done with error-prone geometric operations (e.g., homography or back-projection from monocular depth estimation) or expensive direct dense mapping between image pixels and pixels in BEV (e.g., with MLP or attention). In this work 6, we present ‘LaRa’, an efficient encoder-decoder, transformer-based model for vehicle semantic segmentation from multiple cameras. Our approach uses a system of cross-attention to aggregate information over multiple sensors into a compact, yet rich, collection of latent representations. These latent representations, after being processed by a series of self-attention blocks, are then reprojected with a second cross-attention in the BEV space (an overview is provided in  3). We demonstrate that our model outperforms the best previous works using transformers on nuScenes. The code and trained models are available at https://github.com/valeoai/LaRa.

Masking Modalities for Cross-modal Video Retrieval

Participants: Valentin Gabeur, Arsha Nagrani, Chen Sun, Karteek Alahari, Cordelia Schmid.

Pre-training on large scale unlabelled datasets has shown impressive performance improvements in the fields of computer vision and natural language processing. Given the advent of large-scale instructional video datasets, a common strategy for pre-training video encoders is to use the accompanying speech as weak supervision. However, as speech is used to supervise the pre-training, it is never seen by the video encoder, which does not learn to process that modality. We address this drawback of current pre-training methods, which fail to exploit the rich cues in spoken language, in 20. Our proposal is to pre-train a video encoder using all the available video modalities as supervision, namely, appearance, sound, and transcribed speech. We mask an entire modality in the input and predict it using the other two modalities. This encourages each modality to collaborate with the others, and our video encoder learns to process appearance and audio as well as speech 4. We show the superior performance of our "modality masking" pre-training approach for video retrieval on the How2R, YouCook2 and Condensed Movies datasets.

Illustration of our approach

AVATAR: Unconstrained Audiovisual Speech Recognition

Participants: Valentin Gabeur, Paul Hongsuck Seo, Arsha Nagrani, Chen Sun, Karteek Alahari, Cordelia Schmid.

Audiovisual automatic speech recognition (AV-ASR) is an extension of ASR that incorporates visual cues, often from the movements of a speaker's mouth. Unlike works that simply focus on the lip motion, we investigate the contribution of entire visual frames (visual actions, objects, background etc.). This is particularly useful for unconstrained videos, where the speaker is not necessarily visible. To solve this task, we propose a new sequence-to-sequence AudioVisual ASR TrAnsformeR (AVATAR) 19 which is trained end-to-end from spectrograms and full-frame RGB (see Figure 5). To prevent the audio stream from dominating training, we propose different word-masking strategies, thereby encouraging our model to pay attention to the visual stream. We demonstrate the contribution of the visual modality on the How2 AV-ASR benchmark, especially in the presence of simulated noise, and show that our model outperforms all other prior work by a large margin. Finally, we also create a new, realworld test bed for AV-ASR called VisSpeech, which demonstrates the contribution of the visual modality under challenging audio conditions.

Illustration of our approach

A soft nearest-neighbor framework for continual semi-supervised learning

Participants: Zhiqi Kang, Enrico Fini, Moin Nabi, Elisa Ricci, Karteek Alahari.

Despite significant advances, the performance of state-of-the-art continual learning approaches hinges on the unrealistic scenario of fully labeled data. In 26, we tackle this challenge and propose an approach for continual semi-supervised learning—a setting where not all the data samples are labeled. An underlying issue in this scenario is the model forgetting representations of unlabeled data and overfitting the labeled ones. We leverage the power of nearest-neighbor classifiers to non-linearly partition the feature space and learn a strong representation for the current task, as well as distill relevant information from previous tasks, shown in Figure 6. We perform a thorough experimental evaluation and show that our method outperforms all the existing approaches by large margins, setting a strong state of the art on the continual semi-supervised learning paradigm. For example, on CIFAR100 we surpass several others even when using at least 30 times less supervision (0.8% vs. 25% of annotations).

fig.

Improving the Generalization of Supervised Models

Participants: Mert Bulent Sariyildiz, Yannis Kalantidis, Karteek Alahari, Diane Larlus.

We consider the problem of training a deep neural network on a given classification task, e.g., ImageNet-1K (IN-1K), so that it excels at that task as well as at other (future) transfer tasks. These two seemingly contradictory properties impose a trade-off between improving the model's generalization while maintaining its performance on the original task. Models trained with self-supervised learning (SSL) tend to generalize better than their supervised counterparts for transfer learning; yet, they still lag behind supervised models on IN-1K. In this work 31, we propose a supervised learning setup that leverages the best of both worlds. We enrich the common supervised training framework using two key components of recent SSL models: Multi-scale crops for data augmentation and the use of an expendable projector head. We replace the last layer of class weights with class prototypes computed on the fly using a memory bank. We show in our experiments (see Figure 7) that these three improvements lead to a more favorable trade-off between the IN-1K training task and 13 transfer tasks. Over all the explored configurations, we single out two models: t-ReX that achieves a new state of the art for transfer learning and outperforms top methods such as DINO and PAWS on IN-1K, and t-ReX* that matches the highly optimized RSB model on IN-1K while performing better on transfer tasks.

Fake it till you make it: Learning(s) from a synthetic ImageNet clone

Participants: Mert Bulent Sariyildiz, Karteek Alahari, Diane Larlus, Yannis Kalantidis.

Recent large-scale image generation models such as Stable Diffusion have exhibited an impressive ability to generate fairly realistic images starting from a very simple text prompt. Could such models render real images obsolete for training image prediction models? In this work 30, we answer part of this provocative question by questioning the need for real images when training models for ImageNet classification. More precisely, provided only with the class names that have been used to build the dataset, we explore the ability of Stable Diffusion to generate synthetic clones of ImageNet and measure how useful they are for training classification models from scratch. An overview of our experimental protocol is shown in Figure 8. We show that with minimal and class-agnostic prompt engineering those ImageNet clones we denote as ImageNet-SD are able to close a large part of the gap between models produced by synthetic images and models trained with real images for the several standard classification benchmarks that we consider in this study. More importantly, we show that models trained on synthetic images exhibit strong generalization properties and perform on par with models trained on real data.

Participants: Heeseung Kwon, Francisco M. Castro, Manuel J. Marin-Jimenez, Nicolas Guil, Karteek Alahari.

Vision Transformers (ViTs) have become a dominant paradigm for visual representation learning with self-attention operators. Although these operators provide flexibility to the model with their adjustable attention kernels, they suffer from inherent limitations: (1) the attention kernel is not discriminative enough, resulting in high redundancy of the ViT layers, and (2) the complexity in computation and memory is quadratic in the sequence length. In this paper 27, we propose a novel attention operator, called lightweight structure-aware attention (LiSA), which has a better representation power with log-linear complexity (see Figure 9). Our operator learns structural patterns by using a set of relative position embeddings (RPEs). To achieve log-linear complexity, the RPEs are approximated with fast Fourier transforms. Our experiments and ablation studies demonstrate that ViTs based on the proposed operator outperform self-attention and other existing operators, achieving state-of-the-art results on ImageNet, and competitive results on other visual understanding benchmarks such as COCO and Something-Something-V2. The source code of our approach will be released online.

On the Shift Invariance of Max Pooling Feature Maps in Convolutional Neural Networks

Participants: Hubert Leterme, Kévin Polisano, Valérie Perrier, Karteek Alahari.

In this paper 29, we aim to improve the mathematical interpretability of convolutional neural networks for image classification. When trained on natural image datasets, such networks tend to learn parameters in the first layer that closely resemble oriented Gabor filters. By leveraging the properties of discrete Gabor-like convolutions, we prove that, under specific conditions, feature maps computed by the subsequent max pooling operator tend to approximate the modulus of complex Gabor-like coefficients, and as such, are stable with respect to certain input shifts. We then compute a probabilistic measure of shift invariance for these layers. More precisely, we show that some filters, depending on their frequency and orientation, are more likely than others to produce stable image representations. We experimentally validate our theory by considering a deterministic feature extractor based on the dual-tree wavelet packet transform, a particular case of discrete Gabor-like decomposition. We demonstrate a strong correlation between shift invariance on the one hand and similarity with complex modulus on the other hand, as illustrated in Figure 10.

Top (a): expected discrepancy between complex modulus feature maps and max pooling feature maps (theoretical result). Input images are assumed to be filtered using oriented, band-pass kernels with characteristic frequencies ξ=(ξ1,ξ2)∈[-π,π]2\xi = (\xi _1,\, \xi _2) \in [-\pi ,\, \pi ]^2, and subsampled by a factor 2 (left) or 4 (right). Bottom (b): stability of max pooling outputs with respect to small shifts along the xx-axis, averaged over 50K images from the ImageNet dataset (experimental result). For the sake of visual comparison with (a), several band-pass convolution filters have been tested, with characteristic frequencies ξ\xi varying within [-π,π]2[-\pi ,\, \pi ]^2, and the same subsampling factors as above. We observe regular patterns of dark spots; more precisely, shift instabilities seem to occur when the filter's frequency is located in a dark region of (a).

From CNNs to Shift-Invariant Twin Wavelet Models

Participants: Hubert Leterme, Kévin Polisano, Valérie Perrier, Karteek Alahari.

In this paper 28, we propose a novel antialiasing method to increase shift invariance in convolutional neural networks (CNNs). More precisely, we replace the conventional combination “real-valued convolutions + max pooling” ($ℝ$Max) by “complex-valued convolutions + modulus” ($ℂ$Mod), which produce stable feature representations for band-pass filters with well-defined orientations. In a recent work 29, we proved that, for such filters, the two operators yield similar outputs. Therefore, $ℂ$Mod can be viewed as a stable alternative to $ℝ$Max. To separate band-pass filters from other freely-trained kernels, in this paper, we designed a “twin” architecture based on the dual-tree complex wavelet packet transform (DT-$ℂ$WPT), which generates similar outputs as standard CNNs with fewer trainable parameters. In addition to improving stability to small shifts, our experiments on AlexNet and ResNet showed increased prediction accuracy on natural image datasets such as ImageNet and CIFAR10. Furthermore, our approach outperformed recent antialiasing methods based on low-pass filtering by preserving high-frequency information, while reducing memory usage. Figure 11 compares the accuracy and shift invariance of the various methods.

The Right Spin: Learning Object Motion from Rotation-Compensated Flow Fields

Participants: Pia Bideau, Erik Learned-Miller, Cordelia Schmid, Karteek Alahari.

Both a good understanding of geometrical concepts and a broad familiarity with objects lead to our excellent perception of moving objects. The human ability to detect and segment moving objects works in the presence of multiple objects, complex background geometry, motion of the observer and even camouflage. How humans perceive moving objects so reliably is a longstanding research question in computer vision and borrows findings from related areas such as psychology, cognitive science and physics. One approach to the problem is to teach a deep network to model all of these effects. This contrasts with the strategy used by human vision, where cognitive processes and body design are tightly coupled and each is responsible for certain aspects of correctly identifying moving objects. Similarly from the computer vision perspective, there is evidence that classical, geometry-based techniques are better suited to the "motion-based" parts of the problem, while deep networks are more suitable for modeling appearance. In this work 21, we argue that the coupling of camera rotation and camera translation can create complex motion fields that are difficult for a deep network to untangle directly. We present a novel probabilistic model to estimate the camera's rotation given the motion field. We then rectify the flow field to obtain a rotation-compensated motion field for subsequent segmentation. This strategy of first estimating camera motion, and then allowing a network to learn the remaining parts of the problem, yields improved results on the widely used DAVIS benchmark as well as the recently published motion segmentation data set MoCA (Moving Camouflaged Animals).

Overcoming Label Noise for Source-free Unsupervised Video Domain Adaptation

Participants: Avijit Dasgupta, C. V. Jawahar, Karteek Alahari.

Despite the progress seen in classification methods, current approaches for handling videos with distribution shifts in source and target domains remain source-dependent as they require access to the source data during the adaptation stage. In this paper 10, we present a self-training based source-free video domain adaptation approach (without bells and whistles) to address this challenge by bridging the gap between the source and the target domains. We use the source pre-trained model to generate pseudo-labels for the target domain samples, which are inevitably noisy. We treat the problem of source-free video domain adaptation as learning from noisy labels and argue that the samples with correct pseudo-labels can help in the adaptation stage. To this end, we leverage the cross-entropy loss as an indicator of the correctness of pseudo-labels, and use the resulting small-loss samples from the target domain for fine-tuning the model. Extensive experimental evaluations show that our method termed as CleanAdapt achieves about 7% gain over the source-only model and outperforms the state-of-the-art approaches on various open datasets.

On the Benefits of Large Learning Rates for Kernel Methods

Participants: Gaspard Beugnot, Julien Mairal, Alessandro Rudi.

In this work 7, we study an intriguing phenomenon related to the good generalization performance of estimators obtained by using large learning rates within gradient descent algorithms. First observed in the deep learning literature, we show that such a phenomenon can be precisely characterized in the context of kernel methods, even though the resulting optimization problem is convex. Specifically, we consider the minimization of a quadratic objective in a separable Hilbert space, and show that with early stopping, the choice of learning rate influences the spectral decomposition of the obtained solution on the Hessian's eigenvectors, see Figure 12 for a piece of intuition. This extends previous work to realistic learning scenarios such as kernel ridge regression. While large learning rates may be proven beneficial as soon as there is a mismatch between the train and test objectives, we further explain why it already occurs in classification tasks without assuming any particular mismatch between train and test data distributions.

fig

Efficient Kernel UCB for Contextual Bandits

Participants: Houssam Zenati, Alberto Bietti, Eustache Diemert, Julien Mairal, Matthieu Martin, Pierre Gaillard.

figure

Nested Bandits

Participants: Matthieu Martin, Panagiotis Mertikopoulos, Thibaud Rahier, Houssam Zenati.

In many online decision processes, the optimizing agent is called to choose between large numbers of alternatives with many inherent similarities; in turn, these similarities imply closely correlated losses that may confound standard discrete choice models and bandit algorithms. We study this question in the context of nested bandits in 12, a class of adversarial multi-armed bandit problems where the learner seeks to minimize their regret in the presence of a large number of distinct alternatives with a hierarchy of embedded (non-combinatorial) similarities, as illustrated in Figure 14. In this setting, optimal algorithms based on the exponential weights blueprint (like Hedge, EXP3, and their variants) may incur significant regret because they tend to spend excessive amounts of time exploring irrelevant alternatives with similar, suboptimal costs. To account for this, we propose a nested exponential weights (NEW) algorithm that performs a layered exploration of the learner’s set of alternatives based on a nested, step-by-step selection method. In so doing, we obtain a series of tight bounds for the learner’s regret showing that online learning problems with a high degree of similarity between alternatives can be resolved efficiently, without a red bus / blue bus paradox occurring.

Figure

Efficient and Near-Optimal Online Portfolio Selection

Participants: Rémi Jézéquel, Dmitrii Ostrovskii, Pierre Gaillard.

In 25, we address the problem of online portfolio selection as formulated by Cover (1991), where a trader repeatedly distributes her capital over $d$ assets in each of $T>1$ rounds, with the goal of maximizing the total return. Cover proposed an algorithm, termed Universal Portfolios, that performs nearly as well as the best (in hindsight) static assignment of a portfolio, with an $O(dlog(T\left)\right)$ logarithmic regret. Without imposing any restrictions on the market this guarantee is known to be worst-case optimal, and no other algorithm attaining it has been discovered so far. Unfortunately, Cover's algorithm crucially relies on computing certain $d$-dimensional integral, which must be approximated in any implementation; this results in a prohibitive $\tilde{O}\left(d^4{(T+d)}^{14}\right)$ per-round runtime for the fastest known implementation due to Kalai and Vempala (2002). We propose an algorithm for online portfolio selection that admits essentially the same regret guarantee as Universal Portfolios—up to a constant factor and replacement of $log\left(T\right)$ with $log(T+d)$—yet has a drastically reduced runtime of $\tilde{O}\left(d^2(T+d)\right)$ per round. The selected portfolio minimizes the observed logarithmic loss regularized with the log-determinant of its Hessian—equivalently, the hybrid logarithmic-volumetric barrier of the polytope specified by the asset return vectors. As such, our work reveals surprising connections of online portfolio selection with two classical topics in optimization theory: cutting-plane and interior-point algorithms.

Portfolio selection results.

A Unified Framework for Achieving Optimal Regret Guarantees in Sleeping Bandits

Participants: Pierre Gaillard, Aadirupa Saha, Soham Dan.

Internal regret results.

Versatile Dueling Bandits: Best-of-both-World Analyses for Online Learning from Preferences

Participants: Pierre Gaillard, Aadirupa Saha.

We study the problem of $K$-armed dueling bandit for both stochastic and adversarial environments, where the goal of the learner is to aggregate information through relative preferences of pair of decisions points queried in an online sequential manner. We propose a novel reduction from any (general) dueling bandits to multi-armed bandits and despite the simplicity, it allows us to improve many existing results in dueling bandits. Moreover, our algorithm is also the first to achieve an optimal $O\left({\sum }_{i=1}^K\frac{logT}{{\Delta }_i}\right)$ regret bound against the Condorcet-winner benchmark, which scales optimally both in terms of the arm-size $K$ and the instance-specific suboptimality gaps ${\left\{{\Delta }_i\right\}}_{i=1}^K$. We further justify the robustness of our proposed algorithm by proving its optimal regret rate under adversarially corrupted preferences. In summary, we believe our reduction idea will find a broader scope in solving a diverse class of dueling bandits setting, which are otherwise studied separately from multi-armed bandits with often more complex solutions and worse guarantees.

Dueling regret results.

Amortized implicit differentiation for stochastic bilevel optimization

Participants: Michael Arbel, Julien Mairal.

AMIGO's results.

Non-Convex Bilevel Games with Critical Point Selection Maps

Participants: Michael Arbel, Julien Mairal.

Bilevel game with selection

Continual repeated annealed flow transport Monte Carlo

Participants: Alex Matthews, Michael Arbel, Danilo Rezende, Arnaud Doucet.

CRAFT

Maximum Likelihood Learning of Energy-Based Models for Simulation-Based Inference

Participants: Pierre Glaser, Michael Arbel, Arnaud Doucet, Arthur Gretton.

CRAFT

Towards an Understanding of Default Policies in Multitask Policy Optimization

Participants: Ted Moskovitz, Michael Arbel, Jack Parker-Holder.

DefaultPolicy's results.

The spectral bias of polynomial neural networks

Participants: Moulik Choraria, Leelo Dadi, Grigorios Chrysos, Julien Mairal, Volkan Cevher.

Memory-Augmented Reinforcement Learning for Image-Goal Navigation

Participants: Lina Mezghani, Sainbayar Sukhbaatar, Thibaut Lavril, Oleksandr Maksymets, Dhruv Batra, Piotr Bojanowski, Karteek Alahari.

In this work 15, we address the problem of image-goal navigation in the context of visually-realistic 3D environments. This task involves navigating to a location indicated by a target image in a previously unseen environment. Earlier attempts, including RL-based and SLAM-based approaches, have either shown poor generalization performance, or are heavily reliant on pose/depth sensors. We present a novel method, shown in Figure 23, that leverages a cross-episode memory to learn to navigate. We first train a state-embedding network in a self-supervised fashion, and then use it to embed previously-visited states into an agent’s memory. In order to avoid overfitting, we propose to use data augmentation on the RGB input during training. We validate our approach through extensive evaluations, showing that our data-augmented memory-based model establishes a new state of the art on the image-goal navigation task in the challenging Gibson dataset. We obtain this competitive performance from RGB input only, without access to additional sensors such as position or depth.

Illustration of our approach

Entropic Descent Archetypal Analysis for Blind Hyperspectral Unmixing

Participants: Alexandre Zouaoui, Gedeon Muhawenayo, Behnood Rasti, Jocelyn Chanussot, Julien Mairal.

In 32, we introduce a new algorithm based on archetypal analysis for blind hyperspectral unmixing, assuming linear mixing of endmembers. Archetypal analysis is a natural formulation for this task. This method does not require the presence of pure pixels (i.e., pixels containing a single material) but instead represents endmembers as convex combinations of a few pixels present in the original hyperspectral image. Our approach leverages an entropic gradient descent strategy, which (i) provides better solutions for hyperspectral unmixing than traditional archetypal analysis algorithms, and (ii) leads to efficient GPU implementations. Since running a single instance of our algorithm is fast, we also propose an ensembling mechanism along with an appropriate model selection procedure that make our method robust to hyper-parameter choices while keeping the computational complexity reasonable. By using six standard real datasets, we show that our approach outperforms state-of-the-art matrix factorization and recent deep learning methods.

figure

High Dynamic Range and Super-Resolution from Raw Image Bursts

Participants: Bruno Lecouat, Thomas Eboli, Jean Ponce, Julien Mairal.

Photographs captured by smartphones and mid-range cameras have limited spatial resolution and dynamic range, with noisy response in underexposed regions and color artefacts in saturated areas. This paper introduces the first approach (to the best of our knowledge) to the reconstruction of highresolution, high-dynamic range color images from raw photographic bursts captured by a handheld camera with exposure bracketing. This method uses a physically-accurate model of image formation to combine an iterative optimization algorithm for solving the corresponding inverse problem with a learned image representation for robust alignment and a learned natural image prior. The proposed algorithm is fast, with low memory requirements compared to state-of-the-art learning-based approaches to image restoration, and features that are learned end to end from synthetic yet realistic data. Extensive experiments demonstrate its excellent performance with super-resolution factors of up to ×4 on real photographs taken in the wild with hand-held cameras, and high robustness to low-light conditions, noise, camera shake, and moderate object motion.

figure

Memory-Augmented Reinforcement Learning for Image-Goal Navigation

Participants: Lina Mezghani, Sainbayar Sukhbaatar, Piotr Bojanowski, Alessandro Lazaric, Karteek Alahari.

Overview of our graph building approach

Evaluating the Label Efficiency of Contrastive Self-Supervised Learning for Multi-Resolution Satellite Imagery

Participants: Jules Bourcier, Gohar Dashyan, Jocelyn Chanussot, Karteek Alahari.

Self-Supervised Pretraining on Satellite Imagery: A Case Study on Label-Efficient Vehicle Detection

Participants: Jules Bourcier, Thomas Floquet, Gohar Dashyan, Tugdual Ceillier, Karteek Alahari, Jocelyn Chanussot.

9.1.1 Associate Teams in the framework of an Inria International Lab or in the framework of an Inria International Program

9.2.1 Visits of international scientists

9.3.1 H2020 projects

9.4.1 ANR Project AVENUE

Participants: Karteek Alahari.

9.4.2 MIAI chair: Towards More Data Efficiency in Machine Learning

Participants: Julien Mairal, Karteek Alahari, Massih-Reza Amini, Margot Selosse, Juliette Marrie, Romain Menegaux.

10.1.1 Scientific events: organisation

• Julien Mairal was tutorial chair for CVPR 2022.
• Julien Mairal was area chair for ICLR 2022, ICML 2022, NeurIPS 2022.
• Julien Mairal was a member of the organizing commmittee of the Continuous Time Perspectives in Machine Learning workshop at ICML 2022.
• Karteek Alahari was area chair for ECCV 2022, WACV 2022, and is currently area chair for CVPR 2023, ICCV 2023.
• Karteek Alahari is doctoral consortium co-chair at ICCV 2023.
• Pierre Gaillard was member of the organizing committee of the CFOL workshop at ICML 2022.
• Michael Arbel is an area chair for ICLR 2023.
• D. Khue Le was area chair for BMVC 2022.

10.1.2 Scientific events: selection

10.1.3 Journal

10.1.4 Invited talks

• J. Mairal: invited talk at SIAM Imaging Science, mini-symposium, online.
• J. Mairal: invited talk at Mathematics of machine learning workshop, Bilbao.
• J. Mairal: invited talk at the SICO conference, Autrans.
• J. Mairal: seminar at UCL.
• J. Mairal: seminar at Université Paul Sabatier, Toulouse.
• K. Alahari: seminar at University of Bristol
• K. Alahari: seminar at National University of Singapore
• K. Alahari: invited tutorial at Open Data Science Conference Europe.
• P. Gaillard: seminar at Université Grenoble Alpes, Grenoble.
• M. Arbel: seminar at Université Aix Marseille, Luminy.
• M. Arbel: seminar at Eurocom, Sophia-Antipolis.
• H. Leterme: seminar at Rutgers University

10.1.5 Scientific expertise

• In 2022, Julien Mairal was a reviewer for ERC, the Swiss national science foundation, the Czech science foundation, and for the PSL Junior Fellows call.
• In 2022, Karteek Alahari was a reviewer for ANR, National Fund for Scientific and Technology Development Chile.
• In 2022, Pierre Gaillard was a reviewer for ANR projects.

10.1.6 Research administration

• Julien Mairal is a member of the scientific committee (COS) of Inria's Grenoble research center.
• Karteek Alahari is a member of commission des emplois scientifiques at Inria Grenoble
• Karteek Alahari is a member of commission prospection postes at LJK.
• Karteek Alahari is responsible for the Mathematics and Computer Science specialist field at the MSTII doctoral school since September 2022.

10.2.1 Teaching

• Master: K. Alahari, Advanced Machine Learning, 11.25h eqTD, M2, UGA, Grenoble.
• Master: K. Alahari, Understanding Big Visual Data, 13.5h eqTD, M2, Grenoble INP.
• Master: K. Alahari, Graphical Models Inference and Learning, 13.5h eqTD, MVA, M2, CentraleSupelec, Paris.
• Master: K. Alahari, Introduction to computer vision, 4.5h eqTD, M1, ENS Paris.
• Doctorat: J. Mairal, Optimization for machine learning, 6H eqTD, Geilo winter school (online).
• Doctorat: J. Mairal, Optimization for machine learning, 6H eqTD, StatLearn, Cargese (online).
• Master: J. Mairal, Kernel methods for statistical learning, 3h eqTD, M2, African Master on Machine Intelligence (AMMI).
• Master: J. Mairal, Kernel methods for statistical learning, 12.5h eqTD, M2, Ecole Normale Supérieure, Cachan, France.
• Master: J. Mairal, Kernel methods for statistical learning, 13.5h eqTD, M2, UGA, Grenoble.
• Master: P. Gaillard, Sequential Learning, 13.5h eqTD, M2, Ecole Normale Supérieure, Cachan, France.
• Master: M. Arbel, Kernel methods for statistical learning, 13.5h eqTD, M2, UGA, Grenoble.
• Master: H. Zenati. Cours de visualisations de donnees (approximativement 10h avec TD inclus) IASD master at PSL Paris Dauphine.

10.2.2 Supervision

• Minttu Alakuijala defended her PhD in December 2022. She was co-advised by Jean Ponce, Julien Mairal and Cordelia Schmid.
• Ekaterina Iakovleva defended her PhD in December 2022. She was advised by Karteek Alahari.
• Valentin Gabeur defended his PhD in October 2022. He was co-advised by Karteek Alahari and Cordelia Schmid.
• Gregoire Mialon defended his PhD in February 2022. He was co-advised by Julien Mairal and Alexandre d'Aspremont.

10.2.3 Juries

• PhD: Julien Mairal was reviewer for the PhD thesis of Laurent Meunier, PSL Paris Dauphine.
• PhD: Julien Mairal was a member of the PhD thesis committee of Alfred Laugrois, UGA.
• PhD: Julien Mairal was a member of the CSI of Jules Bourcier, UGA.
• PhD: Karteek Alahari was reviewer for the PhD thesis of Arnaud Deleruyelle, Univ. Lille.
• PhD: Karteek Alahari was reviewer for the PhD thesis of Rui Dai, Univ. Côte d'Azur.
• PhD: Karteek Alahari was a member of the PhD thesis committee of Fabio Pizzati, Mines Paris.
• PhD: Karteek Alahari was a member of the PhD thesis committee of Arthur Douillard, Sorbonne Univ.
• PhD: Karteek Alahari was a member of the PhD thesis committee of Edward Beeching, INSA Lyon.
• PhD: Karteek Alahari was a member of the CSI of Abid Ali (Univ. Côte d'Azur), Alaaeldin Ali (ENS), Adrien Bardes (ENS), Hugo Cisneros (ENS), Yann Labbé (ENS), Guillaume Le Moing (ENS).

International journals

• 1 articleF.Florent Bartoccioni, É.Éloi Zablocki, P.Patrick Pérez, M.Matthieu Cord and K.Karteek Alahari. LiDARTouch: Monocular metric depth estimation with a few-beam LiDAR.Computer Vision and Image Understanding2022
  HALback to text
• 2 articleZ.Zhiqi Kang, M.Mostafa Sadeghi, R.Radu Horaud and X.Xavier Alameda-Pineda. Expression-preserving face frontalization improves visually assisted speech processing.International Journal of Computer VisionDecember 2022
  HAL
• 3 articleB.Bruno Lecouat, T.Thomas Eboli, J.Jean Ponce and J.Julien Mairal. High Dynamic Range and Super-Resolution from Raw Image Bursts.ACM Transactions on Graphics414July 2022, 1 - 21
  HALDOI

International peer-reviewed conferences

• 4 inproceedingsM.Michael Arbel and J.Julien Mairal. Amortized implicit differentiation for stochastic bilevel optimization.The Tenth International Conference on Learning RepresentationsOnline, France2022
  HALback to text
• 5 inproceedingsM.Michael Arbel and J.Julien Mairal. Non-Convex Bilevel Games with Critical Point Selection Maps.NeurIPS 2022 - 36th Conference on Neural Information Processing SystemsAdvances in Neural Information Processing Systems (NeurIPS) 2022New Orleans, United States2022, 1-34
  HALDOIback to text
• 6 inproceedingsF.Florent Bartoccioni, É.Éloi Zablocki, A.Andrei Bursuc, P.Patrick Pérez, M.Matthieu Cord and K.Karteek Alahari. LaRa: Latents and Rays for Multi-Camera Bird's-Eye-View Semantic Segmentation.Proceedings of CoRLCoRL 2022 - Conference on Robot LearningAuckland, New ZealandJune 2022, 1-21
  HALback to text
• 7 inproceedingsG.Gaspard Beugnot, A.Alessandro Rudi and J.Julien Mairal. On the Benefits of Large Learning Rates for Kernel Methods.COLT 2022 - 35th Annual Conference on Learning Theory178Proceedings of Thirty Fifth Conference on Learning TheoryLondon, United KingdomJuly 2022, 254--282
  HALback to text
• 8 inproceedingsJ.Jules Bourcier, G.Gohar Dashyan, J.Jocelyn Chanussot and K.Karteek Alahari. Evaluating the Label Efficiency of Contrastive Self-Supervised Learning for Multi-Resolution Satellite Imagery.Image and Signal Processing for Remote Sensing XXVIIIImage and Signal Processing for Remote Sensing XXVIII12267Berlin, GermanyOctober 2022, 122670K
  HALDOIback to text
• 9 inproceedingsM.Moulik Choraria, L.Leello Dadi, G. G.Grigorios G. Chrysos, J.Julien Mairal and V.Volkan Cevher. The Spectral Bias of Polynomial Neural Networks.ICLR 2022 - International Conference on Learning RepresentationsVirtual, FranceApril 2022, 1-30
  HALback to text
• 10 inproceedingsA.Avijit Dasgupta, C. V.C. V. Jawahar and K.Karteek Alahari. Overcoming Label Noise for Source-free Unsupervised Video Domain Adaptation.ICVGIP 2022 - Indian Conference on Computer Vision, Graphics and Image ProcessingGandhinagar, IndiaACM2022
  HALback to textback to text
• 11 inproceedingsE.Enrico Fini, V.Victor da Costa, X.Xavier Alameda-Pineda, E.Elisa Ricci, K.Karteek Alahari and J.Julien Mairal. Self-Supervised Models are Continual Learners.CVPR 2022 - IEEE/CVF Conference on Computer Vision and Pattern RecognitionNew Orleans, United StatesIEEEJune 2022, 1-13
  HALDOIback to text
• 12 inproceedingsM.Matthieu Martin, P.Panayotis Mertikopoulos, T.Thibaud Rahier and H.Houssam Zenati. Nested bandits.ICML 2022 - 39th International Conference on Machine LearningBaltimore, United StatesJuly 2022
  HALback to text
• 13 inproceedingsA. G.Alexander G D G Matthews, M.Michael Arbel, D. J.Danilo J Rezende and A.Arnaud Doucet. Continual Repeated Annealed Flow Transport Monte Carlo.International Conference on Machine Learning 2022Baltimore, United States2022
  HALback to text
• 14 inproceedingsL.Lina Mezghani, S.Sainbayar Sukhbaatar, P.Piotr Bojanowski, A.Alessandro Lazaric and K.Karteek Alahari. Learning Goal-Conditioned Policies Offline with Self-Supervised Reward Shaping.CoRL 2022- Conference on Robot LearningAuckland, New ZealandDecember 2022, 1-15
  HALback to text
• 15 inproceedingsL.Lina Mezghani, S.Sainbayar Sukhbaatar, T.Thibaut Lavril, O.Oleksandr Maksymets, D.Dhruv Batra, P.Piotr Bojanowski and K.Karteek Alahari. Memory-Augmented Reinforcement Learning for Image-Goal Navigation.IROS - IEEE/RSJ International Conference on Intelligent Robots and SystemsKyoto, JapanOctober 2022
  HALback to text
• 16 inproceedingsT.Ted Moskovitz, M.Michael Arbel, J.Jack Parker-Holder and A.Aldo Pacchiano. Towards an Understanding of Default Policies in Multitask Policy Optimization.25th International Conference on Artificial Intelligence and StatisticsVolume 130: International Conference on Artificial Intelligence and StatisticsOnline, France2022
  HALback to text
• 17 inproceedingsA.Aadirupa Saha and P.Pierre Gaillard. Versatile Dueling Bandits: Best-of-both-World Analyses for Online Learning from Preferences.ICML 2022 - 39th International Conference on Machine LearningBaltimore (MA), United StatesFebruary 2022
  HAL
• 18 inproceedingsH.Houssam Zenati, A.Alberto Bietti, E.Eustache Diemert, J.Julien Mairal, M.Matthieu Martin and P.Pierre Gaillard. Efficient Kernel UCB for Contextual Bandits.Proceedings of Machine Learning ResearchInternational Conference on Artificial Intelligence and Statistics151Proceedings of the 25th International Conference on Artificial Intelligence and Statistics (AISTATS) 2022Valencia, SpainPMLRMay 2022, 5689-5720
  HALback to text

Conferences without proceedings

• 19 inproceedingsV.Valentin Gabeur, P.Paul Hongsuck Seo, A.Arsha Nagrani, C.Chen Sun, K.Karteek Alahari and C.Cordelia Schmid. AVATAR: Unconstrained Audiovisual Speech Recognition.INTERSPEECH 2022 - Conference of the International Speech Communication AssociationIncheon, South KoreaSeptember 2022, 1-6
  HALback to text
• 20 inproceedingsV.Valentin Gabeur, A.Arsha Nagrani, C.Chen Sun, K.Karteek Alahari and C.Cordelia Schmid. Masking Modalities for Cross-modal Video Retrieval.WACV 2022 - Winter Conference on Applications of Computer VisionWaikoloa, United StatesJanuary 2022, 1-10
  HALback to text

Reports & preprints

• 21 miscP.Pia Bideau, E.Erik Learned-Miller, C.Cordelia Schmid and K.Karteek Alahari. The Right Spin: Learning Object Motion from Rotation-Compensated Flow Fields.March 2022
  HALback to text
• 22 miscJ.Jules Bourcier, T.Thomas Floquet, G.Gohar Dashyan, T.Tugdual Ceillier, K.Karteek Alahari and J.Jocelyn Chanussot. Self-Supervised Pretraining on Satellite Imagery: a Case Study on Label-Efficient Vehicle Detection.May 2022
  HALback to text
• 23 miscP.Pierre Gaillard, A.Aadirupa Saha and S.Soham Dan. One Arrow, Two Kills: An Unified Framework for Achieving Optimal Regret Guarantees in Sleeping Bandits.October 2022
  HAL
• 24 miscP.Pierre Glaser, M.Michael Arbel, A.Arnaud Doucet and A.Arthur Gretton. Maximum Likelihood Learning of Energy-Based Models for Simulation-Based Inference.November 2022
  HALDOIback to text
• 25 miscR.Rémi Jézéquel, D. M.Dmitrii M. Ostrovskii and P.Pierre Gaillard. Efficient and Near-Optimal Online Portfolio Selection.September 2022
  HALback to text
• 26 miscZ.Zhiqi Kang, E.Enrico Fini, M.Moin Nabi, E.Elisa Ricci and K.Karteek Alahari. A soft nearest-neighbor framework for continual semi-supervised learning.December 2022
  HALback to text
• 27 miscH.Heeseung Kwon, F. M.Francisco M. Castro, M. J.Manuel J. Marin-Jimenez, N.Nicolas Guil and K.Karteek Alahari. Lightweight Structure-Aware Attention for Visual Understanding.December 2022
  HALback to text
• 28 miscH.Hubert Leterme, K.Kévin Polisano, V.Valérie Perrier and K.Karteek Alahari. From CNNs to Shift-Invariant Twin Wavelet Models.December 2022
  HALback to text
• 29 miscH.Hubert Leterme, K.Kévin Polisano, V.Valérie Perrier and K.Karteek Alahari. On the Shift Invariance of Max Pooling Feature Maps in Convolutional Neural Networks.September 2022
  HALback to textback to text
• 30 miscM. B.Mert Bulent Sariyildiz, K.Karteek Alahari, D.Diane Larlus and Y.Yannis Kalantidis. Fake it till you make it: Learning(s) from a synthetic ImageNet clone.December 2022
  HALback to text
• 31 miscM. B.Mert Bulent Sariyildiz, Y.Yannis Kalantidis, K.Karteek Alahari and D.Diane Larlus. Improving the Generalization of Supervised Models.June 2022
  HALback to text
• 32 miscA.Alexandre Zouaoui, G.Gedeon Muhawenayo, B.Behnood Rasti, J.Jocelyn Chanussot and J.Julien Mairal. Entropic Descent Archetypal Analysis for Blind Hyperspectral Unmixing.September 2022
  HALback to text