3.1 Vehicle guidance and autonomous navigation

Participants: Pranav Agarwal, Said Alexander Alvarado Marin, Sylia Baraka, Pierre de Beaucorps, Anh Quan Cao, Raoul de Charette, Farouk Ghallabi, Manuel Gonzalez, Maximilian Jaritz, Imane Mahtout, Kathia Melbouci, Kaouther Messaoud, Fawzi Nashashibi, Fabio Pizzati, Renaud Poncelet, Danut Ovidiu Pop, Luis Roldao, Anne Verroust-Blondet, Leonardo Ward, Itheri Yahiaoui.

There are three basic ways to improve the safety of road vehicles and these ways are all of interest to the project-team. The first way is to assist the driver by giving him better information and warning. The second way is to take over the control of the vehicle in case of mistakes such as inattention or wrong command. The third way is to completely remove the driver from the control loop.

All three approaches rely on information processing. Only the last two involve the control of the vehicle with actions on the actuators, which are the engine power, the brakes and the steering. The research proposed by the project-team is focused on the following elements:

• perception of the environment,
• planning of the actions,
• real-time control.

3.2 Perception of the road environment

Participants: Anh Quan Cao, Raoul de Charette, Maximilian Jaritz, Farouk Ghallabi, Kaouther Messaoud, Fawzi Nashashibi, Fabio Pizzati, Danut Ovidiu Pop, Luis Roldao, Anne Verroust-Blondet, Itheri Yahiaoui.

Either for driver assistance or for fully automated guided vehicle purposes, the first step of any robotic system is to perceive the environment in order to assess the situation around itself. Proprioceptive sensors (accelerometer, gyrometer,...) provide information about the vehicle by itself such as its velocity or lateral acceleration. On the other hand, exteroceptive sensors, such as video camera, laser or GPS devices, provide information about the environment surrounding the vehicle or its localization. Obviously, fusion of data with various other sensors is also a focus of the research.

The following topics are already validated or under development in our team:

• relative ego-localization with respect to the infrastructure, i.e. lateral positioning on the road can be obtained by mean of vision (lane markings) and the fusion with other devices (e.g. GPS);
• global ego-localization by considering GPS measurement and proprioceptive information, even in case of GPS outage;
• road detection by using lane marking detection and navigable free space;
• detection and localization of the surrounding obstacles (vehicles, pedestrians, animals, objects on roads, etc.) and determination of their behavior can be obtained by the fusion of vision, laser or radar based data processing;
• simultaneous localization and mapping as well as mobile object tracking using laser-based and stereovision-based (SLAMMOT) algorithms.

Scene understanding is a large perception problem. In this research axis we have decided to use only computer vision as cameras have evolved very quickly and can now provide much more precise sensing of the scene, and even depth information. Two types of hardware setups were used, namely: monocular vision or stereo vision to retrieve depth information which allow extracting geometry information.

We have initiated several works:

• estimation of the ego motion using monocular scene flow. Although in the state of the art most of the algorithms use a stereo setup, researches were conducted to estimate the ego-motion using a novel approach with a strong assumption.
• bad weather conditions evaluations. Most often all computer vision algorithms work under a transparent atmosphere assumption which assumption is incorrect in the case of bad weather (rain, snow, hail, fog, etc.). In these situations the light ray are disrupted by the particles in suspension, producing light attenuation, reflection, refraction that alter the image processing.
• deep learning for object recognition. New works are being initiated in our team to develop deep learning recognition in the context of heterogeneous data.
• deep learning for vehicle motion prediction.

3.3 Planning and executing vehicle actions

Participants: Pierre de Beaucorps, Imane Mahtout, Fawzi Nashashibi, Renaud Poncelet, Anne Verroust-Blondet.

From the understanding of the environment, thanks to augmented perception, we have either to warn the driver to help him in the control of his vehicle, or to take control in case of a driverless vehicle. In simple situations, the planning might also be quite simple, but in the most complex situations we want to explore, the planning must involve complex algorithms dealing with the trajectories of the vehicle and its surroundings (which might involve other vehicles and/or fixed or moving obstacles). In the case of fully automated vehicles, the perception will involve some map building of the environment and obstacles, and the planning will involve partial planning with periodical recomputation to reach the long term goal. In this case, with vehicle to vehicle communications, what we want to explore is the possibility to establish a negotiation protocol in order to coordinate nearby vehicles (what humans usually do by using driving rules, common sense and/or non verbal communication). Until now, we have been focusing on the generation of geometric trajectories as a result of a maneuver selection process using grid-based rating technique or fuzzy technique. For high speed vehicles, Partial Motion Planning techniques we tested, revealed their limitations because of the computational cost. The use of quintic polynomials we designed, allowed us to elaborate trajectories with different dynamics adapted to the driver profile. These trajectories have been implemented and validated in the JointSystem demonstrator of the German Aerospace Center (DLR) used in the European project HAVEit, as well as in RITS's electrical vehicle prototype used in the French project ABV. HAVEit was also the opportunity for RITS to take in charge the implementation of the Co-Pilot system which processes perception data in order to elaborate the high level command for the actuators. These trajectories were also validated on RITS's cybercars. However, for the low speed cybercars that have pre-defined itineraries and basic maneuvers, it was necessary to develop a more adapted planning and control system. Therefore, we have developed a nonlinear adaptive control for automated overtaking maneuver using quadratic polynomials and Lyapunov function candidate and taking into account the vehicles kinematics. For the global mobility systems we are developing, the control of the vehicles includes also advanced platooning, automated parking, automated docking, etc. For each functionality a dedicated control algorithm was designed (see publication of previous years).

In the past few years we initiated and conducted focused research on the so-called "`Plug & Play Control"' as well as on their application to the control of autonomous vehicles. Two PhD thesis (Francisco Navas Matos and Imane Mahtout) have centered their studies on the design of control architectures that are able to deal with changes in the operating conditions and assure vehicle performance and stability. Youla-Kucera parametrization has been used to design control structures able to recognize the driving situation changes, adapting the controller response to satisfy the required performance level and keeping the motion stability with a natural vehicle behavior. These architectures were validated through real implementations on RITS and Renault autonomous prototype vehicles; more specifically to develop longitudinal and lateral control of the vehicle in the presence of disturbances and context sudden modifications.

3.4 Cyberphysical constructs and mobile communications for fully automated networked vehicles

Participants: Gérard Le Lann.

Safety, privacy, efficiency, and cybersecurity (SPEC) properties are key to the advent of self-forming and self-healing networks of fully automated driverless terrestrial vehicles.

• Safety (S): Ratios of fatalities and non-lethal accidents should be much smaller than those achieved with non-fully automated systems that rely on human supervision or interventions.
• Privacy (P): No personal data should be inferred or extracted from cyber information, wireless communications, or from physical information such as paths and routes followed by vehicles.
• Efficiency (E): Road capacity and vehicular densities should be much higher than those achieved with non-fully automated systems for identical velocities. For example, inter-vehicular gaps are much smaller, which is antagonistic with the safety property.
• Cybersecurity (C): Safety should not be compromised by internal or external cyberattacks such as spoofing, Sybil attacks, Man-in-the-Middle attacks, message falsification or suppression, injection of bogus data, replay attacks, Denial-of-Service.

Vehicles under study are referred to as Next-Gen Vehicles (NGVs), in order to avoid confusion with Connected Autonomous Vehicles (CAVs).

CAVs match SAE levels 3 and 4. Human factors are serious impediments to achieving desired safety properties with SAE L3 vehicles. This is due to large latencies involved with human reactivity, ranging between 2 and 7 seconds, as shown in numerous experiments. SAE L4 vehicles are radically different from SAE L5 vehicles, since L4 vehicles are bound to circulate in specific lanes and neighborhoods, at rather low velocities.

NGVs are SAE L5 vehicles: No human driving or supervision, any time, everywhere. Quoting Anthony Levandowski, formerly in charge of Google cars, before Waymo was created, we believe that ”The human is the bug”. Furthermore, solving L5 problems provides leads to solutions for every other SAE level, whereas solving L3 or L4 problems does not help much in inferring solutions for L5 driving.

CAVs and NGVs rest on robotics capabilities (radars, lidars, cameras, motion control laws, learning processes, GNSS devices, actuators, etc.). CAVs are equipped with V2X (vehicle-to-everything) functionalities based on medium range WiFi DSRC communications, and cellular C-V2X communications. They rely on periodic beaconing of messages that carry their GNSS coordinates, and on V2I (vehicle-to-infrastructure) communications for accessing remote cloud-based services via road-side units or static radio relays.

NGVs will be equipped with CMX (coordinated mobility for X) functionalities, X standing for S, P, E, and C, based on very short-range directional C-V2V communications (cellular radio and optics between vehicles proximate to each other), medium range WiFi communications, and explicit agreement algorithms aimed at handling safety-critical maneuvers. NGVs do not rely at all on V2I communications relayed via terrestrial infrastructures and nodes. The CMX framework matches mobile edge computing models, whereby everything that is needed for SPEC properties is available from vehicles onboard systems.

Examples of innovative concepts at the core of the CMX framework are vehicle cyberphysical levels (which complement SAE levels), unfalsifiable vehicle profiles, and proactive security modules (extension of hardware security modules). Periodic beaconing is banished. In addition to being useless regarding safety and efficiency properties, it enables tracking and illegitimate cyber-espionage.

3.5 Probabilistic modeling for large transportation systems

Participants: Guy Fayolle, Jean-Marc Lasgouttes.

This activity concerns the modeling of random systems related to ITS, through the identification and development of solutions based on probabilistic methods and more specifically through the exploration of links between large random systems and statistical physics. Traffic modeling is a very fertile area of application for this approach, both for macroscopic (fleet management  39, traffic prediction) and for microscopic (movement of each vehicle, formation of traffic jams) analysis. When the size or volume of structures grows (leading to the so-called “thermodynamic limit”), we study the quantitative and qualitative (performance, speed, stability, phase transitions, complexity, etc.) features of the system.

In the recent years, several directions have been explored.

4.1 Introduction

While the preceding section focused on methodology, in connection with automated guided vehicles, it should be stressed that the evolution of the problems which we deal with remains often guided by the technological developments. We enumerate three fields of application whose relative importance varies with time and which have strong mutual dependencies: driving assistance, cars available in self-service mode and fully automated vehicles (cybercars).

4.2 Driving assistance

Several techniques will soon help drivers. One of the first immediate goal is to improve security by alerting the driver when some potentially dangerous or dangerous situations arise, i.e. collision warning systems or lane tracking could help a bus driver and surrounding vehicle drivers to more efficiently operate their vehicles. Human factors issues could be addressed to control the driver workload based on additional information processing requirements. Another issue is to optimize individual journeys. This means developing software for calculating optimal (for the user or for the community) paths. Nowadays, path planning software is based on a static view of the traffic: efforts have to be done to take the dynamic component in account.

4.3 New transportation systems

The problems related to the abusive use of the individual car in large cities led the populations and the political leaders to support the development of public transport. A demand exists for a transport of people and goods which associates quality of service, environmental protection and access to the greatest number. Thus the tram and the light subways of VAL type recently introduced into several cities in France conquered the populations, in spite of high financial costs. However, these means of mass transportation are only possible on lines on which there is a keen demand. As soon as one moves away from these “lines of desire” or when one deviates from the rush hours, these modes become expensive and offer can thus only be limited in space and time. To give a more flexible offer, it is necessary to plan more individual modes which approach the car as we know it. However, if one wants to enjoy the benefits of the individual car without suffering from their disadvantages, it is necessary to try to match several criteria: availability anywhere and anytime to all, lower air and soils pollution as well as sound levels, reduced ground space occupation, security, low cost. Electric or gas vehicles available in self-service, as in the Praxitèle system, bring a first response to these criteria. To be able to still better meet the needs, it is however necessary to re-examine the design of the vehicles on the following points:

• ease empty car moves to better distribute them;
• better use of information systems inboard and on ground;
• better integrate this system in the global transportation system.

These systems are now operating. The challenge is to bring them to an industrial phase by transferring technologies to these still experimental projects.

4.4 Automated vehicles

The long term effort of the project is to put automatically guided vehicles (cybercars) on the road. It seems too early to mix cybercars and traditional vehicles, but data processing and automation now make it possible to consider in the relatively short term the development of such vehicles and the adapted infrastructures. RITS aims at using these technologies on experimental platforms (vehicles and infrastructures) to accelerate the technology transfer and to innovate in this field. Other application can be precision docking systems that will allow buses to be automatically maneuvered into a loading zone or maintenance area, allowing easier access for passengers, or more efficient maintenance operations. Transit operating costs will also be reduced through decreased maintenance costs and less damage to the braking and steering systems. Regarding technical topics, several aspects of Cybercars have been developed at RITS this year. First, we have stabilized a generic Cycab architecture involving Inria SynDEx tool and CAN communications. The critical part of the vehicle is using a real-time SynDEx application controlling the actuators via two Motorola’s MPC555. Today, we have decided to migrate to the new dsPIC architecture for more efficiency and ease of use. This application has a second feature, it can receive commands from an external source (Asynchronously to this time) on a second CAN bus. This external source can be a PC or a dedicated CPU, we call it high level. To work on the high level, in the past years we have been developing a R&D framework called (Taxi) which used to take control of the vehicle (Cycab and Yamaha) and process data such as gyro, GPS, cameras, wireless communications and so on. In order to rely on a professional and maintained solution, we use the RTMaps SDK development platform for our developments and demonstrations. These demonstrations include: reliable SLAMMOT algorithm using 2 to 4 laser sensors simultaneously, automatic line/road following techniques, PDA remote control, multi sensors data fusion, collaborative perception via ad-hoc network. The second main topic is inter-vehicle communications using ad-hoc networks. We have worked with the EVA team for setting and tuning OLSR, a dynamic routing protocol for vehicles communications. Our goal is to develop a vehicle dedicated communication software suite, running on a specialized hardware. It can be linked also with the Taxi Framework for getting data such GPS information’s to help the routing algorithm.

6.1 New software

7.1 Physics/GAN rendering for improving robustness to bad weather

Participants: Raoul de Charette, Shirsendu Halder, Jean-Francois Lalonde, Fabio Pizzati, Maxime Tremblay.

In this axis, we conducted a thorough study on the effect of fog and rain, using a rendering pipeline that enables the systematic evaluation of common computer vision algorithms to controlled amounts of rain/fog. Our work has been published in journal IJCV 2020 19, as an extension of our ICCV'19 work 4.

We developed three different ways to add synthetic rain to existing images datasets: completely physic-based; completely data-driven; and a combination of both. The physic-based rain augmentation combines a physical particle simulator and accurate rain photometric modeling. The benefit of either methods is evaluated quantitatively, qualitatively, and we also compare against real rain images using performance metrics and by conducting a user study. The latter validated our rendering methods, demonstrating our rain is judged as much as $73\%$ more realistic than the state-of-the-art. Using our generated rain-augmented KITTI, Cityscapes, and nuScenes datasets, we conduct a thorough evaluation of object detection, semantic segmentation, and depth estimation algorithms and show that their performance decreases in degraded weather, on the order of $15\%$ for object detection, $60\%$ for semantic segmentation, and 6-fold increase in depth estimation error. Finetuning on our augmented synthetic data results in improvements of $21\%$ on object detection, $37\%$ on semantic segmentation, and $8\%$ on depth estimation.

The code of both works was published open source (https://github.com/cv-rits/rain-rendering) and we released a dataset with close to 400k augmented images (https://team.inria.fr/rits/computer-vision/weather-augment/). Altogether code and datasets were downloaded approx. 100+ times.

The research was conducted with our scientific partner Université Laval (Québec, Canada) in the context of the joint grant obtained by R. de Charette and J-F. Lalonde.

7.2 Semi-/Un- supervised Domain Adaptation

Participants: Raoul de Charette, Maximilian Jaritz, Fabio Pizzati.

In this axis, we proposed 2 new methods on UDA. In the first one published in CVPR'20 22 with PhD student M. Jaritz, we study the benefit of learning from cross-modalities, and propose a cross-modal learning called xMUDA. Our proposal disentangles two heads in a two-branches 2D/3D deep architecture where either expert branch tries to predict the other modality prediction in addition to perform as best as possible with its own modality. This work outperforms all methods on multiple UDA scenarios, With PhD student F. Pizzati, we published in WACV'20 25 the ability to produce efficient pseudo-labels from a single image in an unsupervised manner to transfer learning to the target dataset. The latter includes work on image-to-image translation and a new region growing strategy for pseudo-label thresholding. The code for xMUDA 22 has been released open source (https://github.com/valeoai/xmuda).

We extended xMUDA to new UDA scenarios and to Semi-Supervised DA (SSDA). In the new work 34, we extended our work to scenarios where a few targets samples are labeled, typically like the Waymo Open Dataset. We combined here the benefit of our cross-modal loss with supervised semantic loss to boost the result. Thorough ablation studies show the benefit of our proposal and demonstrate better performance than the literature on almost all tested scenarios.

7.3 Image-to-image translation

Participants: Raoul de Charette, Fabio Pizzati.

In the first, we investigate i2i where the target domain encompasses two sub domains and propose a disentanglement model-driven mechanism. This is typically the case when the target domain (e.g. rainy image) include occlusion on the lenses or in the scene (e.g. water drops on the windshield, fog, etc.). Our unsupervised model-based learning disentangles scene and occlusions, while benefiting from an adversarial pipeline to regress physical parameters of the occlusion model. The experiments demonstrate our method is able to handle varying types of occlusions and generate highly realistic translations, qualitatively and quantitatively outperforming the state-of-the-art on multiple datasets. The latter was published in ECCV'20 24. A follow up work has been initiated encompassing new types of disentanglement, using either supervised disentanglement, or non-differentiable models with genetic-driven deep regression. It is planned to be soon submitted to a journal.

We also investigate the case of continuous i2i, where instead of translating from a source to a target domain, we wish to produce continuous translation assuming unknown and unordered target data. For this research, we applied our work to day-to-time translation and shown that our model-guidance enables continuous translations discovering – in an unsupervised manner – a efficient and consistent walk along the manifold. Thus discovering for example dawn and dusk appearance from unlabeled target images. The benefit of our method has been assessed with GAN metrics, which demonstrate better performance on all scenarios. The work has been submitted to a conference and will be published soon.

7.4 Sparse deep reinforcement learning for end-to-end driving

Participants: Pierre de Beaucorps, Raoul de Charette, Pranav Agarwal.

In this axis, rather than rewarding the agent at each step, we only reward it at the end of the driving scenario as it reached its goal. While this is significantly harder since the supervision signal is very weak, this enables the agent to discover by itself new driving strategies. In our work, we studied the benefit of sparse reward in conjunction of curriculum learning, which allows the agent to address progressively harder driving tasks (e.g. driving first 5m, then 10m, etc.). Finally, we learn multiple policies (drive straight, left, right) switched at inference time. The work will be published soon.

7.5 Fast tracking with RGB-D-E

Participants: Raoul de Charette, Etienne Dubeau, Mathieu Garon, Jean-François Lalonde.

In this axis, we proposed in 20, for the first time, to use an event-based camera to increase the speed of 3D object tracking in 6 degrees of freedom. This application requires handling very high object speed to convey compelling AR experiences. To this end, we propose a new system which combines a recent RGB-D sensor (Kinect Azure) with an event camera (DAVIS346). We develop a deep learning approach, which combines an existing RGB-D network along with a novel event-based network in a cascade fashion, and demonstrate that our approach significantly improves the robustness of a state-of-the-art frame-based 6-DOF object tracker using our RGB-D-E pipeline.

The research was conducted with our scientific partner Université Laval (Québec, Canada) in the context of the joint grant obtained by R. de Charette and J-F. Lalonde.

7.6 Cooperative perception for assistant system and automated driving

Participants: Pierre de Beaucorps, Raoul de Charette, Manuel Gonzales.

The outcome of our work was successfully tested in closed-circuit conditions (both at Inria Rocquencourt and at Satory test tracks) and was experimented in real highway conditions thanks to our industrial partners collaboration. Demo were done in the context of the project deliverables and a full-day workshop took place in Paris to share the results with officials and the research/industrial community.

7.7 3D Semantic Scene Completion

Participants: Luis Roldão, Raoul de Charette, Anne Verroust-Blondet, Anh Quan Cao.

A comprehensive 3D sensing of the scene is crucial for applications like mobile robotics, and more especially for autonomous driving. Recent research on Semantic Scene Completion (SSC) enables to jointly complete and semantically label entire 3D scenes, including missing and occluded regions from raw input scans. In this work, we introduce a new approach for multiscale 3D SSC from voxelized sparse 3D LiDAR scans for autonomous driving applications. As opposed to the literature, we use a 2D UNet backbone with comprehensive multiscale skip connections to enhance feature flow and decrease computation overhead, along with 3D segmentation heads to retrieve spatial information. Our method achieves performance on par with other state of the art (SoA) methods while being significantly lighter and faster.

3D data causes representation challenges for learning-based algorithms, where data is commonly encoded using 3D voxel grids processed by 3D CNNs. While this achieves good results, it requires heavy computation and memory needs. Consequently, most of the literature limits output resolution, limiting the deployment of such methods to real-time applications. We tackle this by proposing a lightweight CNN that process a 3D voxel grid with considerably lighter 2D convolutions. This is achieved by convolving along one spatial axis while mapping to the third dimension with the use of 3D segmentation heads. In our proposal, multiscale completion is also possible given informative features map flow, preserving computation efficiency and enabling very fast inference at coarse levels.

The proposed method was evaluated in challenging SSC benchmark of SemanticKITTI. Results show that our method performs on par with SoA for semantic completion (-0.69 mIoU) and significantly improves occupancy completion (+4.72 IoU), while containing significantly fewer parameters (0.35M compared to 0.93M of second lightest network). Furthermore, our method provides significantly higher inference speeds, especially at coarse resolutions thanks to its multiscale capability, achieving inference times up to 372FPS at lower scale. This research opens the door to new real-time oriented SSC methods for automated driving.

More details can be found in 27. This research is partially funded by AKKA Technology.

A new PhD student, Anh Quan Cao, has also investigated new ways to tackle the problem of semantic scene completion, using fold-based strategies for point-wise scene completion. New research directions are still investigated.

7.8 Attention-based vehicle trajectory prediction

Participants: Kaouther Messaoud, Fawzi Nashashibi, Anne Verroust-Blondet, Itheri Yahiaoui.

• In 15 an extension of our previous work 8 for highways context has been done.
• In collaboration with Laboratory for Intelligent & Safe Automobiles (LISA) team of the university of California San Diego, we tackled the task of trajectory prediction of a target vehicle in an urban environment 35. For this purpose, we developed a deep learning-based solution exploiting information about the surrounding agents and the scene context. Our solution models the temporal evolution of the target agent using a recurrent neural network. We also model the influence of the dynamic agents and the static scene context (road geometry, lane structure…) using a muti-head attention network. In order to account for the uncertainty of the future, we generate multiple predicted plausible trajectories and the probability of occurrence of each one. We also ensure that the predicted trajectory is compliant with the scene structure which means that the predicted point lie in the drivable area using an auxiliary loss function. The developed solution is trained and evaluated using the nuScenes naturalistic driving dataset. We also made a submission for the nuScenes challenge and we earned the second price. Part of this work has been done during the stay of Kaouther Messaoud in USA.

7.9 LPG-SLAM: a Light-weight Probabilistic Graph-based SLAM

Participants: Kathia Melbouci, Fawzi Nashashibi.

To include the 3D information efficiently, and help the downstream modules of the autonomous driving system (prediction, motion planning) to benefit from the localization and the mapping. The 3D occupancy grid map is encoded in an intermediate representation, such that this representation could be compressed, stored and loaded from the hard disk with few bits of memory, but in the same time this representation retains all the information necessary to do the localization and the global map update, this representation has to be robust to the perception uncertainty as well.

7.10 LIDAR-Based perception For Vehicle Localization in an HD Map

Participants: Farouk Ghallabi, Fawzi Nashashibi.

7.11 Safe motion planning through occluded intersections

Participants: Renaud Poncelet, Anne Verroust-Blondet, Fawzi Nashashibi.

7.12 Youla-Kucera based multi-objective controllers: Application to autonomous vehicles

Participants: Imane Mahtout, Fawzi Nashashibi.

The validation of the proposed control structures is provided in both simulation and real-time experimentation using a Renault ZOE vehicle. The adaptability of the controllers to autonomous driving tasks is proved in different operating conditions.

7.13 Cyberphysical constructs and mobile communications for fully automated networked vehicles

Participants: Gérard Le Lann.

Personal work in 2020 has been devoted to solving the following open problems:

• Fully automated crossing of unsignalized intersections of arbitrary topologies (any number of arteries), with switch times (next virtual green light turned on after virtual red light is turned off) in the order of a few milliseconds.
• UPFs (Unambiguous Physical Functions), simple variations of PUFs (Physically Unclonable Functions), for instantaneous, unambiguous, and privacy-preserving designation of a specific neighboring vehicle (no IP or MAC address). This is needed for engaging reciprocal communications and striking explicit agreements in the case of safety-critical maneuvers (e.g. lane changes, lateral insertions in vehicle strings, cuts of multilane traffic, zipper merging, etc.).
• How to eliminate exhaustion of short-term certificates (for authentication) and the need for importing new short-term certificates. With that solution, short-term certificates can be reused, while making physical tracking (routes followed are collected) and cyber tracking (espionage of communications) unfeasible.

Work with foreign partners in 2020 has been devoted to exploiting the CMX framework introduced in 6.

With a North-American company (San Diego and Boston), work has been focused on immunity to cyberattacks in networks of cohorts (an instance of cyberphysical constructs for SAE L5) and how to implement a proactive security module (Section 5 in 6).

With a Swedish University (Stockholm), work has been focused on pseudonymization and privacy properties with NGVs (Section 3 in 6).

7.14 Belief propagation inference for traffic prediction

Participant: Jean-Marc Lasgouttes.

This work 41, in collaboration with Cyril Furtlehner (TAU, Inria), deals with real-time prediction of traffic conditions in a urban setting with incomplete data. The main focus is on finding a good way to encode available information (flow, speed, counts,...) in a Markov Random Field, and to decode it in the form of real-time traffic reconstruction and prediction. Our approach relies in particular on the Gaussian belief propagation algorithm.

Through our collaboration with PTV Sistema, we obtained extensive results on large-scale datasets containing 250 to 2000 detectors. The results show very good ability to predict flow variables and a reasonably good performance on speed or occupancy variables. Some element of understanding of the observed performance are given by a careful analysis of the model, allowing to some extent to disentangle modelling bias from intrinsic noise of the traffic phenomena and its measurement process.

This year, our research report 41 has been reorganised and submitted to IEEE Transactions on Intelligent Transportation Systems.

7.15 Stabilization of traffic through cooperative autonomous vehicles

Participant: Guy Fayolle, Jean-Marc Lasgouttes.

We investigate in 33 the transfer function emanating from the linearization of a car-following model, when taking into account a driver reaction time. This leads to stability conditions, which are explicitly given. We also show how this reaction time can introduce a partial string instability. This is a joint work with Carlos Flores from UC Berkeley.

This is intended as a foundation of a larger work on traffic stabilization by means of a fleet of cooperative automated vehicles. However, contrary to some earlier work in, our approach is based on a car-following model with reaction-time delay, rather than on a first order fluid model. While the focus of the present work is on human-driven vehicles, the next steps will be to tackle shockwave analysis and adapted traffic-stabilizing control strategies.

This work has been submitted to IEEE Transactions on Automatic Control.

7.16 Automated vehicle collection using an hedonic game

Participants: Jean-Marc Lasgouttes.

We consider a shared transportation system in an environment where human drivers collect vehicles that are no longer being used. Each driver, also called a platoon leader, is in charge of driving collected vehicles as a platoon to bring them back to some given location (e.g. an airport, a railway station). Platoon allocation and route planning for picking up and returning automated vehicles is one of the major issues of shared transportation systems that need to be addressed. We propose a coalition game approach to compute 1) the allocation of unused vehicles to a minimal number of platoons, 2) the optimized tour of each platoon and 3) the minimum energy consumed to collect all these vehicles.

In this coalition game, the players are the parked vehicles, and the coalitions are the platoons that are formed. This game, where each player joins the coalition that maximizes its payoff, converges to a stable solution. The quality of the solution obtained is evaluated with regard to three optimization criteria and its complexity is measured by the computation time required. Simulation experiments are carried out in various configurations. They show that this approach is very efficient to solve the multi-objective optimization problem considered, since it provides the optimal number of platoons in less than a second for 300 vehicles to be collected, and considerably outperforms other well-known optimization approaches like MOPSO (Multi-Objective Particle Swarm Optimization) and NSGA-II (Non dominated Sorting Genetic Algorithm).

This work has been accepted for publication in 2021 in Concurrency and Computation: Practice and Experience.

7.17 Genus and classification of Random walks in the quarter plane

Participants: Guy Fayolle.

7.18 Reflected brownian motion in a non convex cone

Participants: Guy Fayolle.

7.19 Random walks in orthants and lattice path combinatorics

Participants: Guy Fayolle.

In the second edition of the book 2, original methods were proposed to determine the invariant measure of random walks in the quarter plane with small jumps (size 1), the general solution being obtained via reduction to boundary value problems. Among other things, an important quantity, the so-called group of the walk, allows to deduce theoretical features about the nature of the solutions. In particular, when the order of the group is finite and the underlying algebraic curve is of genus 0 or 1, necessary and sufficient conditions have been given for the solution to be rational, algebraic or $D$-finite (i.e. satisfying a linear differential equation). In this framework, number of difficult open problems related to lattice path combinatorics are currently being explored, in collaboration with A. Bostan and F. Chyzak (project-team SPECFUN, Inria-Saclay), both from theoretical and computer algebra points of view: concrete computation of the criteria, utilization of differential Galois theory, genus greater than 1 (i.e. when some jumps are of size $\ge 2$), etc. A recent topic (mentioned in 2019) deals with the connections between simple product-form stochastic networks (so-called Jackson networks) and explicit solutions of functional equations for counting lattice walks, Some partial extensions of 32 are under development.

8.1 Bilateral contracts with industry

Valeo Group: a very strong partnership is under reinforcement between Valeo and Inria. Several bilateral contracts were signed to conduct joint works on Driving Assistance, some of which Valeo is funding. This joint research includes:

• A CIFRE like PhD thesis is ongoing between Valeo and Inria (Maximilian JARITZ), dealing with multisensor processing and learning techniques for free navigable road detection.
• Valeo is currently a major financing partner of the “GAT” international Chaire/JointLab in which Inria is a partner. The other partners are: UC Berkeley, Shanghai Jiao-Tong University, EPFL, IFSTTAR, MPSA (Peugeot-Citroën) and SAFRAN.
• Technology transfer is also a major collaboration topic between RITS and Valeo as well as the development of a road automated prototype.
• Finally, Inria and Valeo are partners of the PIA French project CAMPUS ( Connected Automated Mobility Platform for Urban Sustainability) including SAFRAN, Invia and Gemalto. The aim of the project is the development of autonomous vehicles and the realization of two canonical uses-cases on highways and urban like environments.

Renault Group: Collaboration between Renault and RITS re-started in 2016. Different research teams in Renault are now working separately with RITS on different topics.

• A CIFRE like PhD thesis is ongoing between Renault and Inria (Farouk GHALLABI) The thesis deals with the accurate localization of an autonomous vehicle on a highway using mainly on-board low-cost perception sensors.
• Another CIFRE PhD thesis is ongoing since November 2017 (Imane MAHTOUT).

AKKA Technologies: Collaboration with AKKA since 2012 (for the Link & Go prototype).

• Inria and AKKA Technologies are partners in the VALET projects (ANR projects).
• A CIFRE PhD thesis (Luis ROLDAO) dealing with 3D-environment modeling for autonomous vehicles begun in October 2017.

9.1 International initiatives

9.2 International research visitors

9.3 European initiatives

9.4 National initiatives

10.1 Promoting scientific activities

10.2 Teaching - Supervision - Juries

10.3 Popularization

11.1 Major publications

• 1 inproceedings ZayedZ. Alsayed, GuillaumeG. Bresson, AnneA. Verroust-Blondet and FawziF. Nashashibi. 2D SLAM Correction Prediction in Large Scale Urban Environments ICRA 2018 - International Conference on Robotics and Automation 2018 Brisbane, Australia May 2018
  HAL
• 2 bookGuyG. Fayolle, RoudolfR. Iasnogorodski and Vadim A.V. Malyshev. S. AsmussenP. GlynnY. JanRandom Walks in the Quarter Plane: Algebraic Methods, Boundary Value Problems, Applications to Queueing Systems and Analytic Combinatorics40Probability Theory and Stochastic ModellingThe first edition was published in 1999Springer International PublishingFebruary 2017, 255
  HALDOIback to textback to textback to textback to text
• 3 article DavidD. Gonzalez Bautista, JoshuéJ. Pérez, VicenteV. Milanés and FawziF. Nashashibi. A Review of Motion Planning Techniques for Automated Vehicles IEEE Transactions on Intelligent Transportation Systems April 2016
  HAL DOI
• 4 inproceedings Shirsendu SukantaS. Halder, Jean-FrançoisJ.-F. Lalonde and RaoulR. De Charette. Physics-Based Rendering for Improving Robustness to Rain ICCV 2019 - International Conference on Computer Vision ICCV 2019. Supplementary pdf / videos available on project page Seoul, South Korea October 2019
  HAL back to text
• 5 inproceedings MaximilianM. Jaritz, Tuan-HungT.-H. Vu, RaoulR. De Charette, EmilieE. Wirbel and PatrickP. Pérez. xMUDA: Cross-Modal Unsupervised Domain Adaptation for 3D Semantic Segmentation Conference on Computer Vision and Pattern Recognition (CVPR) For a demo video, see http://tiny.cc/xmuda Virtual, United States June 2020
  HAL
• 6 techreport GérardG. Le Lann. Cyberphysical Constructs and Concepts for Fully Automated Networked Vehicles RR-9297 INRIA Paris-Rocquencourt October 2019
  HAL back to text back to text back to text
• 7 article ImaneI. Mahtout, FranciscoF. Navas, VicenteV. Milanés and FawziF. Nashashibi. Advances in Youla-Kucera parametrization: A Review Annual Reviews in Control June 2020
  HAL DOI
• 8 inproceedings KaoutherK. Messaoud, ItheriI. Yahiaoui, AnneA. Verroust-Blondet and FawziF. Nashashibi. Non-local Social Pooling for Vehicle Trajectory Prediction Intelligent Vehicles Symposium (IV) Paris, France 2019
  HAL DOI back to text
• 9 inproceedings FabioF. Pizzati, PietroP. Cerri and RaoulR. De Charette. Model-based occlusion disentanglement for image-to-image translation ECCV 2020 - European Conference on Computer Vision ECCV 2020 ECCV 2020 Glasgow / Virtual, United Kingdom August 2020
  HAL
• 10 inproceedings LuisL. Roldão, RaoulR. De Charette and AnneA. Verroust-Blondet. LMSCNet: Lightweight Multiscale 3D Semantic Completion 3DV 2020 - International Virtual Conference on 3D Vision Accepted at 3DV 2020 (Oral). For a demo video, see http://tiny.cc/lmscnet. Code is available at https://github.com/cv-rits/LMSCNet Fukuoka / Virtual, Japan November 2020
  HAL
• 11 article MaximeM. Tremblay, Shirsendu SukantaS. Halder, RaoulR. De Charette and Jean-FrançoisJ.-F. Lalonde. Rain Rendering for Evaluating and Improving Robustness to Bad Weather International Journal of Computer Vision 19 pages, 19 figures, IJCV 2020. September 2020
  HAL DOI

11.2 Publications of the year

11.3 Cited publications

• 36 misc ThomasT. Dreyfus, CharlotteC. Hardouin, JulienJ. Roques and Michael F.M. Singer. On the Kernel curves associated with walks in the quarter plane 2020
  back to text
• 37 inproceedings GuyG. Fayolle and CyrilC. Furtlehner. About Hydrodynamic Limit of Some Exclusion Processes via Functional Integration Int. Math. Conf. ``50 Years of IPPI'' Proceedings on CD. ISBN 978-5-901158-15-9 Institute for Information Transmission Problems (Russian Academy of Sciences) Moscow July 2011
  HAL back to text
• 38 article GuyG. Fayolle and RoudolphR. Iasnogorodski. Random Walks in the Quarter-Plane: Advances in Explicit Criterions for the Finiteness of the Associated Group in the Genus 1 Case Markov Processes and Related Fields 21 4 Accepted for publication in the journal MPRF (Markov Processes and Related Fields) December 2015
  HAL back to text
• 39 articleGuyG. Fayolle and Jean-MarcJ.-M. Lasgouttes. Asymptotics and Scalings for Large Product-Form Networks via the Central Limit TheoremMarkov Processes and Related Fields221996, 317-348
  back to text
• 40 inproceedings CyrilC. Furtlehner, YufeiY. Han, Jean-MarcJ.-M. Lasgouttes, VictorinV. Martin, FabriceF. Marchal and FabienF. Moutarde. Spatial and Temporal Analysis of Traffic States on Large Scale Networks 13th International IEEE Conference on Intelligent Transportation Systems ITSC'2010 Madère, Portugal September 2010
  HAL back to text
• 41 techreport CyrilC. Furtlehner, Jean-MarcJ.-M. Lasgouttes, AlessandroA. Attanasi, LorenzoL. Meschini and MarcoM. Pezzulla. Spatio-temporal Probabilistic Short-term Forecasting on Urban Networks RR-9236 INRIA Saclay, équipe TAU ; INRIA de Paris, équipe RITS ; PTV-SISTeMA December 2018
  HAL back to text back to text
• 42 inproceedingsVictorinV. Martin, CyrilC. Furtlehner, YufeiY. Han and Jean-MarcJ.-M. Lasgouttes. GMRF Estimation under Topological and Spectral Constraints7th European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases8725Lecture Notes in Computer ScienceNancy, FranceSpringer Berlin HeidelbergSeptember 2014, 370-385
  HALDOIback to text
• 43 articleVictorinV. Martin, Jean-MarcJ.-M. Lasgouttes and CyrilC. Furtlehner. Latent binary MRF for online reconstruction of large scale systemsAnnals of Mathematics and Artificial Intelligence7712016, 123-154
  back to text