3.2.1 Axis A1 - NextGen Virtual Characters

3.2.2 Axis A2 - Populated Dynamic Worlds

3.2.3 Axis A3 - New Immersive Experiences

Final evaluation of the H2020 CrowdBot project

The CrowdBot project ended on 31 December 2021, and its final evaluation by the European Commission was carried out in 2022. The project was initiated and coordinated by Julien Pettré (implemented in the Inria Rainbow team). The evaluation, whose outcome was very positive, marks the end of an epic journey that has greatly influenced the research carried out within the team, in particular towards modelling and prediction techniques of pedestrian behaviour, as well as the use of virtual reality for the collection of data on human-robot interaction, as well as the evaluation of the safety level of autonomous systems during such interactions.

Kick-off of the ANR OLICOW project

The team is a partner in the ANR OLICOW project which started on 1 December 2022. In this project, the VirtUs team will address the topics of behaviour prediction and calibration of simulation models by assimilation of real data.

End of project H2020 INVICTUS

The project INVICTUS, coordinated by M. Christie has reached its official end December 31st, 2022. The final evaluation of the project will take place beginning of 2023.

Etoile de l'europe:

Julien Pettré was awarded an “Etoile de l'Europe” from the Ministère de l'enseignement supérieur et de la Recherche. Lien vers le site d'information du Ministère.

Eurographics - Best Paper Award, Honorable Mention:

Adèle Colas was awarded a Honorable Mention at Eurographics for the her work Interaction Fields: Intuitive Sketch-based Steering Behaviors for Crowd Simulation.

IEEE Virtual Reality - TVCG Best Journal Papers Nominees:

Tairan Yin was nominated for the Best Journal Award for his work The One-Man-Crowd: Single User Generation of Crowd Motions Using Virtual Reality.

Outstanding Reviewer Award:

Ludovic Hoyet obtained the Outstanding Reviewer Award at the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games 2022 conference.

6.1.1 AvatarReady

• Name:
  A unified platform for the next generation of our virtual selves in digital worlds
• Keywords:
  Avatars, Virtual reality, Augmented reality, Motion capture, 3D animation, Embodiment
• Scientific Description:
  AvatarReady is an open-source tool (AGPL) written in C#, providing a plugin for the Unity 3D software to facilitate the use of humanoid avatars for mixed reality applications. Due to the current complexity of semi-automatically configuring avatars coming from different origins, and using different interaction techniques and devices, AvatarReady aggregates several industrial solutions and results from the academic state of the art to propose a simple and fast way to use humanoid avatars in mixed reality in a seamless way. For example, it is possible to automatically configure avatars from different libraries (e.g., rocketbox, character creator, mixamo), as well as to easily use different avatar control methods (e.g., motion capture, inverse kinematics). AvatarReady is also organized in a modular way so that scientific advances can be progressively integrated into the framework. AvatarReady is furthermore accompanied by a utility to generate ready-to-use avatar packages that can be used on the fly, as well as a website to display them and offer them for download to users.
• Functional Description:
  AvatarReady is a Unity tool to facilitate the configuration and use of humanoid avatars for mixed reality applications. It comes with a utility to generate ready-to-use avatar packages and a website to display them and offer them for download.
• URL:
  https://­gitlab.­inria.­fr/­avatar/­avatarreadypackage
• Authors:
  Ludovic Hoyet, Fernando Argelaguet Sanz, Adrien Reuzeau
• Contact:
  Ludovic Hoyet

6.1.2 PyNimation

• Keywords:
  Moving bodies, 3D animation, Synthetic human
• Scientific Description:
  PyNimation is a python-based open-source (AGPL) software for editing motion capture data which was initiated because of a lack of open-source software enabling to process different types of motion capture data in a unified way, which typically forces animation pipelines to rely on several commercial software. For instance, motions are captured with a software, retargeted using another one, then edited using a third one, etc. The goal of Pynimation is therefore to bridge the gap in the animation pipeline between motion capture software and final game engines, by handling in a unified way different types of motion capture data, providing standard and novel motion editing solutions, and exporting motion capture data to be compatible with common 3D game engines (e.g., Unity, Unreal). Its goal is also simultaneously to provide support to our research efforts in this area, and it is therefore used, maintained, and extended to progressively include novel motion editing features, as well as to integrate the results of our research projects. At a short term, our goal is to further extend its capabilities and to share it more largely with the animation/research community.
• Functional Description:

  PyNimation is a framework for editing, visualizing and studying skeletal 3D animations, it was more particularly designed to process motion capture data. It stems from the wish to utilize Python’s data science capabilities and ease of use for human motion research.

  In its version 1.0, Pynimation offers the following functionalities, which aim to evolve with the development of the tool : - Import / Export of FBX, BVH, and MVNX animation file formats - Access and modification of skeletal joint transformations, as well as a certain number of functionalities to manipulate these transformations - Basic features for human motion animation (under development, but including e.g. different methods of inverse kinematics, editing filters, etc.). - Interactive visualisation in OpenGL for animations and objects, including the possibility to animate skinned meshes

• URL:
  https://­gitlab.­inria.­fr/­lhoyet/­pynimation
• Authors:
  Ludovic Hoyet, Robin Adili, Benjamin Niay, Alberto Jovane
• Contact:
  Ludovic Hoyet

7.1.1 Warping Character Animations using Visual Motion Features

Participants: Julien Pettré [contact], Marc Christie, Ludovic Hoyet, Alberto Jovane, Anne-Hélène Olivier, Pierre Raimbaud, Katja Zibrek.

Despite the wide availability of large motion databases, and recent advances in exploiting them to create tailored contents, artists still spend tremendous efforts in editing and adapting existing character animations to new contexts. This motion editing problem is a well-addressed research topic. Yet only a few approaches have considered the influence of the camera angle, and the resulting visual features it yields, as a mean to control and warp a character animation. This work 3 proposes the design of viewpoint-dependent motion warping units that perform subtle updates on animations through the specification of visual motion features such as visibility, or spatial extent. We express the problem as a specific case of visual servoing, where the warping of a given character motion is regulated by a number of visual features to enforce. Results demonstrate the relevance of the approach for different motion editing tasks and its potential to empower virtual characters with attention-aware communication abilities, as diplayed in Figure 3. This work was performed in collaboration with Claudio Pacchierotti from the Rainbow team.

7.1.2 UnderPressure: Deep Learning for Foot Contact Detection, Ground Reaction Force Estimation and Footskate Cleanup

Participants: Ludovic Hoyet [contact], Lucas Mourot.

In this work, we leverage UnderPressure, a novel publicly available dataset of motion capture synchronized with pressure insoles data, to estimate vertical Ground Reaction Forces (vGRFs) from motion data with a deep neural network and derive foot contact labels. We further clean up footskate artefacts through an optimisation-based inverse kinematics algorithm while enforcing vGRFs invariance.

Human motion synthesis and editing are essential to many applications like video games, virtual reality, and film post-production. However, they often introduce artefacts in motion capture data, which can be detrimental to the perceived realism. In particular, footskating is a frequent and disturbing artefact, which requires knowledge of foot contacts to be cleaned up. Current approaches to obtain foot contact labels rely either on unreliable threshold-based heuristics or on tedious manual annotation. In this work 5, we address automatic foot contact label detection from motion capture data with a deep learning based method. To this end, we first publicly release UnderPressure, a novel motion capture database labelled with pressure insoles data serving as reliable knowledge of foot contact with the ground, wich we used to design and train a deep neural network to estimate ground reaction forces exerted on the feet from motion data and derive accurate foot contact labels (Figure 4). The evaluation of our model shows that we significantly outperform heuristic approaches based on height and velocity thresholds and that our approach is much more robust when applied on motion sequences suffering from perturbations like noise or footskate. We further propose a fully automatic workflow for footskate cleanup: foot contact labels are first derived from estimated ground reaction forces. Then, footskate is removed by solving foot constraints through an optimisation-based inverse kinematics (IK) approach that ensures consistency with the estimated ground reaction forces. Beyond footskate cleanup, both the database and the method we propose could help to improve many approaches based on foot contact labels or ground reaction forces, including inverse dynamics problems like motion reconstruction and learning of deep motion models in motion synthesis or character animation. This work was performed in collaboration with Pierre Hellier and François Le Clerc from InterDigital.

7.1.3 A new framework for the evaluation of motion datasets through motion matching techniques

Participants: Julien Pettré [contact], Vicenzo Abichequer Sangalli, Marc Christie, Ludovic Hoyet.

The many different motions that compose one of the datasets used in this work . Each motion clip is represented in a different color and symbol. Such representations are exploited to give the user an idea of how motions are distributed in the dataset.

Analyzing motion data is a critical step when building meaningful locomotive motion datasets. This can be done by labeling motion capture data and inspecting it, through a planned motion capture session or by carefully selecting locomotion clips from a public dataset. These analyses, however, have no clear definition of coverage, making it harder to diagnose when something goes wrong, such as a virtual character not being able to perform an action or not moving at a given speed. This issue is compounded by the large amount of information present in motion capture data, which poses a challenge when trying to interpret it. This work 9 provides a visualization and an optimization method to streamline the process of crafting locomotive motion datasets (Figure 5). It provides a more grounded approach towards locomotive motion analysis by calculating different quality metrics, such as: demarcating coverage in terms of both linear and angular speeds, frame use frequency in each animation clip, deviation from the planned path, number of transitions, number of used vs. unused animations and transition cost. By using these metrics as a comparison mean for different motion datasets, our approach is able to provide a less subjective alternative to the modification and analysis of motion datasets, while improving interpretability.

7.1.4 Impact of Self-Contacts on Perceived Pose Equivalences

Participants: Ludovic Hoyet [contact].

Human characters with varying morphologies performing poses with several self-contacts. Characters with similar poses but different body shapes (columns) can have slightly different spatial relationships between body segments, in particular self-contacts (e.g., contact between the left arm and the flank appearing for the middle characters). In this work  we explore which self-contacts are important to the meaning of the pose independently from other parameters such as body shape.

Defining equivalences between poses of different human characters is an important problem for imitation research, human pose recognition and deformation transfer. However, pose equivalence is a subjective information that depends on context and on the morphology of the characters. A common hypothesis is that interactions between body surfaces, such as self-contacts, are important attributes of human poses, and are therefore consistently included in animation approaches aiming at retargeting human motions. However, some of these self-contacts are only present because of the morphology of the character and are not important to the pose, e.g. contacts between the upper arms and the torso during a standing A-pose. In this work 10, we conduct a first study towards the goal of understanding the impact of self-contacts between body surfaces on perceived pose equivalences. More specifically, we focus on contacts between the arms or hands and the upper body as displayed in Figure 6, which are frequent in everyday human poses. We conduct a study where we present to observers two models of a character mimicking the pose of a source character, one with the same self-contacts as the source, and one with one self-contact removed, and ask observers to select which model best mimics the source pose. We show that while poses with different self-contacts are considered different by observers in most cases, this effect is stronger for self-contacts involving the hands than for those involving the arms. This work was conducted in collaboration with Franck Multon of the MimeTIC team, and Jean Basset and Stefanie Wuhrer of the Morpheo team.

7.2.1 A Survey on Reinforcement Learning Methods in Character Animation

Participants: Julien Pettré [contact].

A diagram showing a taxonomy of the Reinforcement Learning algorithms described in this work. We focus on two divisions: single agent or multiagent, and policy-based or value-based. The colors of nodes correspond to whether the algorithm is on-policy (red), off-policy (blue), or in between (green). Algorithms marked with an asterisk (⁎) can only be used with discrete action spaces.

Reinforcement Learning is an area of Machine Learning focused on how agents can be trained to make sequential decisions, and achieve a particular goal within an arbitrary environment. While learning, they repeatedly take actions based on their observation of the environment, and receive appropriate rewards which define the objective. This experience is then used to progressively improve the policy controlling the agent's behavior, typically represented by a neural network. This trained module can then be reused for similar problems, which makes this approach promising for the animation of autonomous, yet reactive characters in simulators, video games or virtual reality environments. We provide the community with a comprehensible survey 4 of the modern Deep Reinforcement Learning methods and discusses their possible applications in Character Animation, from skeletal control of a single, physically-based character to navigation controllers for individual agents and virtual crowds. It also describes the practical side of training DRL systems, comparing the different frameworks available to build such agents (Fig. 7).

7.2.2 Analysis of emergent patterns in crossing flows of pedestrians reveals an invariant of ‘stripe’ formation in human data

Participants: Julien Pettré [contact], Pratick Mullick.

When two streams of pedestrians cross at an angle, striped patterns spontaneously emerge as a result of local pedestrian interactions. This clear case of self-organized pattern formation remains to be elucidated. In counterflows, with a crossing angle of 180 degrees, alternating lanes of traffic are commonly observed moving in opposite directions, whereas in crossing flows at an angle of 90 degrees, diagonal stripes have been reported. Naka (1977) hypothesized that stripe orientation is perpendicular to the bisector of the crossing angle. However, studies of crossing flows at acute and obtuse angles remain underdeveloped. We tested the bisector hypothesis in experiments on small groups (18-19 participants each) crossing at seven angles (30 degrees intervals), and analyzed the geometric properties of stripes 8. We present 6 two novel computational methods for analyzing striped patterns in pedestrian data: (i) an edge-cutting algorithm, which detects the dynamic formation of stripes and allows us to measure local properties of individual stripes; and (ii) a pattern-matching technique, based on the Gabor function, which allows us to estimate global properties (orientation and wavelength) of the striped pattern at a time $T$. We find an invariant property: stripes in the two groups are parallel and perpendicular to the bisector at all crossing angles. In contrast, other properties depend on the crossing angle: stripe spacing (wavelength), stripe size (number of pedestrians per stripe), and crossing time all decrease as the crossing angle increases from 30 degrees to 180 degrees, whereas the number of stripes increases with crossing angle. We also observe that the width of individual stripes is dynamically squeezed as the two groups cross each other. The findings thus support the bisector hypothesis at a wide range of crossing angles, although the theoretical reasons for this invariant remain unclear. The present results provide empirical constraints on theoretical studies and computational models of crossing flows.

7.2.3 Interaction Fields: Intuitive Sketch-based Steering Behaviors for Crowd Simulation

Participants: Julien Pettré [contact], Ludovic Hoyet, Adèle Colas, Katja Zibrek, Anne-Hélène Olivier.

Photograph of our experimental set-up to study crossing flows. Agents participating in our experiment are shown in this photograph for a typical trial with crossing angle 120 degrees. The three stages of the trial are shown here, viz. (left) before crossing (middle) during crossing and (right) after crossing.

The real-time simulation of human crowds has many applications. In a typical crowd simulation, each person (“agent”) in the crowd moves towards a goal while adhering to local constraints. Many algorithms exist for specific local `steering' tasks such as collision avoidance or group behavior. However, these do not easily extend to completely new types of behavior, such as circling around another agent or hiding behind an obstacle. They also tend to focus purely on an agent's velocity without explicitly controlling its orientation. Our paper 2 presents a novel sketch-based method for modelling and simulating many steering behaviors for agents in a crowd. Central to this is the concept of an interaction field (IF): a vector field that describes the velocities or orientations that agents should use around a given `source' agent or obstacle. An IF can also change dynamically according to parameters, such as the walking speed of the source agent. IFs can be easily combined with other aspects of crowd simulation, such as collision avoidance. Using an implementation of IFs in a real-time crowd simulation framework, we could demonstrate the capabilities of IFs in various scenarios (e.g., hide and seek scenarios as illustrated in Figure 9). This includes game-like scenarios where the crowd responds to a user-controlled avatar. We also present an interactive tool that computes an IF based on input sketches. This IF editor lets users intuitively and quickly design new types of behavior, without the need for programming extra behavioral rules. We thoroughly evaluate the efficacy of the IF editor through a user study, which demonstrates that our method enables non-expert users to easily enrich any agent-based crowd simulation with new agent interactions.

7.2.4 Dynamic Combination of Crowd Steering Policies Based on Context

Participants: Ludovic Hoyet [contact].

PL ($S_{QF}=67$)	RVO ($S_{QF}=84$)	PL and RVO ($S_{QF}=92$)

Simulating crowds requires controlling a very large number of trajectories of characters and is usually performed using crowd steering algorithms. The question of choosing the right algorithm with the right parameter values is of crucial importance given the large impact on the quality of results. In this work 1, we study the performance of a number of steering policies (i.e., simulation algorithm and its parameters) in a variety of contexts, resorting to an existing quality function able to automatically evaluate simulation results. This analysis allows us to map contexts to the performance of steering policies. Based on this mapping, we demonstrate that distributing the best performing policies among characters improves the resulting simulations. Furthermore, we also propose a solution to dynamically adjust the policies, for each agent independently and while the simulation is running, based on the local context each agent is currently in. We demonstrate significant improvements of simulation results compared to previous work that would optimize parameters once for the whole simulation, or pick an optimized, but unique and static, policy for a given global simulation context (Figure 10). This work was performed in collaboration with Julien Pettré from the Virtus team.

7.2.5 Towards new models of collision avoidance for specific populations

Participants: Anne-Hélène Olivier [contact].

In this research topic, we are interested in identifying the interaction behavior of specific populations such as athletes who have sustained a previous concussion 7, persons with a chronic low back pain 1516 as well as users of powered wheelchair 14, 17, 13. Understanding those specificity is interesting to adapt our crowd simulations models, providing more variety.

Left: Illustration of the real world image of two athletes interacting within the experiment to understand the effect of concussion on visuo-motor strategies. Right: Relative contributions to solve the collision depending on the group of athletes interacting. When 2 concussed athletes are involved in the interaction, the variability is increased .

Individuals who have sustained a concussion often display associated balance control deficits and visuomotor impairments despite being cleared by a physician to return to sport. Such visuomotor impairments can be highlighted in collision avoidance tasks that involves a mutual adaptation between two walkers. However, studies have yet to challenge athletes with a previous concussion during an everyday collision avoidance task, following return to sport. In this project performed in the frame of the Inria Bear Associate team, we investigated whether athletes with a previous concussion display associated behavioural changes during a 90 degree collision avoidance task with an approaching pedestrian (Cf. Figure 11, Left). To this end, thirteen athletes (ATH; 9 females, 23$\pm$4years) and 13 athletes with a previous concussion (CONC; 9 females, 22$\pm$3 years, concussion $<$6 months) walked at a comfortable walking speed along a 12.6m pathway while avoiding another athlete on a 90 degree collision course. Each participant randomly interacted with individuals from the same group 20 times and interacted with individuals from the opposite group 21 times. Minimum predicted distance (mpd) was used to examine collision avoidance behaviours between ATH and CONC groups. The overall progression of mpd(t) did not differ between groups (p>.05). During the collision avoidance task, previously concussed athletes contributed less when passing second compared to their peers (p<.001). When two previously concussed athletes were on a collision course, there was a greater amount of variability resulting in inappropriate adaptive behaviours (Cf. Figure 11 Right). Although successful at avoiding a collision with an approaching athlete, previously concussed athletes exhibit behavioural changes manifesting in riskier behaviours. The current findings suggest that previously concussed athletes possess behavioural changes even after being cleared to returned to sport, which may increase their risk of a subsequent injury when playing.

We have also conducted experiments with person suffering from chronic non specific low back pain (cNSLBP) (Cf. Figure 12 Left). cNSLBP is characterized by symptoms without clear patho-anatomical causes and has been identified as one of the leading global causes of disability. The majority of clinical trials assess cNSLBP using scales or questionnaires reporting an influence of cognitive, emotional and behavioral factors. Previous studies have evaluated the motor consequences of cNSLBP using biomechanical analysis, showing alterations in the kinematics and dynamics of locomotion. However, few studies have evaluated cNSLBP participants in ecological situations, such as walking with obstacles or avoidance, allowing a simultaneous evaluation of cognitive, perceptual and motor components in a situation closer to the one encountered in daily life. In this context, using a horizontal aperture crossing paradigm, we aimed at 1) understanding the effect of cNSLBP in action strategies and (2) identifying factors that may influence these decisions. Healthy adults (N=15) and cNSLBP adults (N=15) performed a walking protocol composed of 3 random blocks of 19 trials with aperture ranging from 0.9 to 1.8 x shoulder width. The dimensions of the obstacle causing an individual to change their actions (shoulders rotation, velocity, trunk stability) is referred to as the Critical Point (CP). Results showed that the average CP in the healthy group was significantly larger than the one of cNSLBP group. This analysis on shoulder data is a first attempt to investigate the influence of cNSLBP on motor behavior and decisions in an ecological paradigm. Results show a slightly more risky behavior of patients with smaller CP. Future work is required to study walking speed profile and to relate behavioral data with variables that can influence pain perception (intensity, duration, kinesiophobia, fear avoidance beliefs, catastrophism and psychological inflexibility).

Finally, we studied the interaction behaviour when a wheelchair user interacts with other pedestrians both in lab or in a museum (Musée des Beaux Arts and Musée de Bretagne of Rennes) (Cf. Figure 12 Middle and Right). In the museum study, we also investigated the efficacy of a driving assistance system on the users' experience. This work is performed in collaboration with Marie Babel and Emilie Leblong in the Rainbow team as well as with the Pôle Saint-Hélier. Preliminary results showed the positive effect of the driving assistance system on various dimensions such as usability, acceptability, learning, social influence. In addition, observation data showed an importance of the social context in the regulation of proximity.

As a short note, we took also interest in the effect of ageing on the body segment strategies to initiate walking motion during street-crossing 18.

7.3.1 The Stare-in-the-Crowd Effect in Virtual Reality

Participants: Anne-Hélène Olivier [contact], Marc Christie, Ludovic Hoyet, Alberto Jovane, Julien Pettré, Pierre Raimbaud, Katja Zibrek.

Nonverbal cues are paramount in real-world interactions. Among these cues, gaze has received much attention in the literature. In particular, previous work has shown a search asymmetry between directed and averted gaze towards the observer using photographic stimuli, with faster detection and longer fixation towards directed gaze by the observer. This is known as the stare-in-the-crowd effect. In this work 12, we investigate whether stare-in-the crowd effect is preserved in Virtual Reality (VR). To this end, we designed a within-subject experiment where 30 human users were immersed in a virtual environment in front of an audience of 11 virtual agents (Figure 13), following 4 different gaze behaviours . We analysed the user's gaze behaviour when observing the audience, computing fixations and dwell time. We also collected the users' social anxiety score using a post-experiment questionnaire to control for some potential influencing factors. Results show that the stare-in-the-crowd effect is preserved in VR, as demonstrated by the significant differences between gaze behaviours, similarly to what was found in previous studies using photographic stimuli. Additionally, we found a negative correlation between dwell time towards directed gazes and users’ social anxiety scores. Such results are encouraging for the development of expressive and reactive virtual humans, which can be animated to express natural interactive behaviour. This work was performed in collaboration with Julien Pettré from the Virtus team and Claudio Pacchierotti from the Rainbow team.

7.3.2 The One-Man-Crowd: Single User Generation of Crowd Motions Using Virtual Reality

Participants: Julien Pettré [contact], Marc Christie, Ludovic Hoyet, Tairan Yin.

Crowd motion data is fundamental for understanding and simulating realistic crowd behaviours. Such data is usually collected through controlled experiments to ensure that both desired individual interactions and collective behaviours can be observed. It is however scarce, due to ethical concerns and logistical difficulties involved in its gathering, and only covers a few typical crowd scenarios. In this work 8, we propose and evaluate a novel Virtual Reality based approach lifting the limitations of real-world experiments for the acquisition of crowd motion data. Our approach immerses a single user in virtual scenarios where he/she successively acts each crowd member. By recording the past trajectories and body movements of the user, and displaying them on virtual characters, the user progressively builds the overall crowd behaviour by him/herself. We validate the feasibility of our approach by replicating three real experiments (Figure 14), and compare both the resulting emergent phenomena and the individual interactions to existing real datasets. Our results suggest that realistic collective behaviours can naturally emerge from virtual crowd data generated using our approach, even though the variety in behaviours is lower than in real situations. These results provide valuable insights to the building of virtual crowd experiences, and reveal key directions for further improvements. This work was performed in collaboration with Julien Pettré from the Virtus team and Marie-Paule Cani from LIX.

7.3.3 Proximity in VR: The Importance of Character Attractiveness and Participant Gender

Participants: Katja Zibrek [contact], Ludovic Hoyet, Anne-Hélène Olivier, Julien Pettré.

In this work, we expand upon recent evidence that the motion of virtual characters affects the proximity of users who are immersed with them in virtual reality (VR). Attractive motions decrease proximity, but no effect of character gender on proximity was found. We therefore designed a similar experiment where users observed walking motions in VR which were displayed on male and female virtual characters 19. Our results show similar patterns found in previous studies, while some differences related to the model of the character emerged.

7.3.4 Smart Motion Trails for Animating in VR

Participants: Marc Christie [contact], Robin Courant, Jean-Baptiste Bordier, Anthony Mirabile.

The artistic crafting of 3D animations by designers is a complex and iterative process. While classical animation tools have brought significant improvements in creating and manipulating shapes over time, most approaches rely on classical 2D input devices to create 3D contents. With the advent of virtual reality technologies and their ability to dive the users in their 3D worlds and to precisely track devices in 6 dimensions (position and orientation), a number of VR creative tools have emerged such as Quill, AnimVR, Tvori, Tiltbrush or MasterPieceVR. While these tools provide intuitive means to directly design in the 3D space by exploiting both the 6D tracking capacity of the hand devices and the stereoscopic perception by the user, the animation capacities or such tools remain strongly limited, and often reproduce classical 2D manipulators in VR. In this work, we propose the design of smart interactive manipulators which leverage on the specificity of VR to animate poly-articulated animations. We then perform a user study to evaluate the benefits of such manipulators over traditional 2D tools for three groups of users: beginner, intermediate, and professional artists. We build on this user to discuss how smart tools (e.g., using a variety of AI techniques) can be coupled with VR technologies to improve content creation. More details can be found in 11.

Cifre InterDigitial - Learning-Based Human Character Animation Synthesis for Content Production

Participants: Ludovic Hoyet [contact], Lucas Mourot.

The overall objective of the PhD thesis of Lucas Mourot, which started in June 2020, is to adapt and improve the state-of-art on video animation and human motion modelling to develop a semi-automated framework for human animation synthesis that brings real benefits to artists in the movie and advertising industry. In particular, one objective is to leverage the use of novel learning-based approaches, in order to propose skeleton-based animation representations, as well as editing tools, in order to improve the resolution and accuracy of the produced animations, so that automatically synthetized animations might become usable in an interactive way by animation artists.

Unreal MegaGrant

Participants: Marc Christie [contact], Anthony Mirabile.

The objective of the Unreal Megagrant (70k€) is to initiate the development of Augmented Reality techniques as animation authoring tools. While we have demonstrated the benefits of VR as a relevant authoring tool for artists, we would like to explore with this funding the animation capacities of augmented reality techniques. Underlying challenges are pertained to the capacity of estimating precisely hand poses for fine animation tasks, but also exploring means of high-level authoring tools such as gesturing techniques.

9.1.1 Inria associate team not involved in an IIL or an international program

9.2.1 Visits of international scientists

9.3.1 H2020 projects

ANR JCJC Per2

Participants: Ludovic Hoyet [contact], Anne-Hélène Olivier, Julien Pettré, Katja Zibrek.

The objective of this project is to focus on how viewers perceive motion variations to automatically produce natural motion personalisation accounting for inter-individual variations. In short, our goal is to automate the creation of motion variations to represent given individuals according to their own characteristics, and to produce natural variations that are perceived and identified as such by users. Challenges addressed in this project consist in (i) understanding and quantifying what makes motions of individuals perceptually different, (ii) synthesising motion variations based on these identified relevant perceptual features, according to given individual characteristics, and (iii) leveraging even further the synthesis of motion variations and to explore their creation for interactive large-scale scenarios where both performance and realism are critical.

Défi Avatar

Participants: Ludovic Hoyet [contact], Maé Mavromatis.

Défi Ys.AI

Participants: Ludovic Hoyet.

With the recent annoucements about massive investments on the Metaverses, which are seen as the future of the social and professional immersive communication for the emergent AI-based e-society, there is a need for the development of dedicated metaverse technologies and associated representation formats. In this context, the objective of this joint project between Inria and InterDigital is to focus on the representation formats of digital avatars and their behavior in a digital and responsive environment. In particular, the primary challenge tackled in this project consists in solving the uncanny valley effect to provide users with a natural and lifelike social interaction between real and virtual actors, leading to full engagement in those future metaverse experiences.

Financement SAD Reactive

The SAD REACTIVE funding allowed the recruitment of post-doctoral student Pierre Raimbaud, for the period from 1 October 2020 to 31 August 2022. During his post-doctoral stay at INRIA in Rennes, his research work focused on the study of user behaviour with virtual agents in immersive environments (virtual reality, via headsets). The origin of the project comes from the need in virtual reality for expressive and reactive behaviours in virtual agents, so that the user perceives a social feeling adapted to his own actions, in interaction with these agents.

Financement SAD Attract

The SAD ATTRACT project was able to run from 1 November 2020 to 30 April 2022, and was able to fund a postdoctoral stay for Mr Pratik Mullick, a PhD physicist of Indian origin. This stay has been extended by 6 months with our own funds, so Pratik is still present in our laboratory, and has been able to obtain a position of Senior Lecturer in Poland, which he has taken in October 2022. It is therefore a very satisfactory conclusion to this project.

10.1.1 Scientific events: organisation

10.1.2 Scientific events: selection

10.1.3 Journal

10.1.4 Invited talks

• Anne-Hélène Olivier: Interactions between pedestrians: from real to virtual studies. Wilfrid Laurier Graduate Seminar, Waterloo, Canada, November 2022.
• Anne-Hélène Olivier: VR to study interactions between pedestrians: validation, applications and future directions. 3rd Training Workshop of the H2020 ITN CLIPE project, Current trends in Virtual Humans, Barcelona, Spain, September 2022.
• Julien Pettré. Colloque des Sciences Appliquées au Sapeur Pompier “Simulation de Foules”
• Julien Pettré. Transmusicales /  French Touch “Musique live et mouvements de foule lors du Hellfest”
• Julien Pettré. Journées de la matière condensée 2022 “Crowd simulation: scales, algorithms and data”
• Julien Pettré. Graphyz 2 “Keynote in Duet: Crowds”

10.1.5 Scientific expertise

• Julien Pettré: ANRT (dossier CIFRE), NWO (agence de financement de la recherche néerlandaise)
• Marc Christie: Member of the CE33 for the ANR
• Anne-Hélène Olivier: ESR Program, Région Nouvelle Aquitaine 2022, Projets ANR AAPG Interfaces:digital sciences humanities and social sciences 2022

10.1.6 Research administration

• Ludovic Hoyet is the coordinator of the Inria Research Challenge Avatar.
• Ludovic Hoyet is the coordinator of the ANR JCJC Per${}^2$.
• Marc Christie is the coordinator of the H2020 Project INVICTUS.
• Julien Pettré is the coordinator of the H2020 Project CrowdDNA.

10.2.1 Teaching

• Master : Ludovic Hoyet, Réalité Virtuelle pour l’Analyse Ergonomique, Master Ingénierie et Ergonomie des Activités Physique, 21h, University Rennes 2, France
• Master: Marc Christie, Head of Master 2 Ingénierie Logicielle (45 students), University of Rennes 1, France
• Master: Marc Christie, "Multimedia Mobile", Master 2, leader of the module, 32h (IL) + 32h (Miage), Computer Science, University of Rennes 1, France
• Master: Marc Christie, "Projet Industriel Transverse", Master 2, 32h, leader of the module, Computer Science, University of Rennes 1, France
• Master: Marc Christie, "Modelistion Animation Rendu", Master 2, 16h, leader of the module, Computer Science, University of Rennes 1, France
• Master: Marc Christie, "Web Engineering", Master 1, 16h, leader of the module, Computer Science, University of Rennes 1, France
• Master: Marc Christie, "Advanced Computer Graphics", Master 1, 10h, leader of the module, Computer Science, ENS, France
• Master: Marc Christie, "Motion for Animation and Robotics", Master 2 SIF, Computer Science, France

10.2.2 Supervision

• PhD defended (beginning Nov. 2019, defended Nov. 2022): Adèle Colas, Modélisation de comportements collectifs réactifs et expressifs pour la réalité virtuelle, Ludovic Hoyet, Anne-Hélène Olivier, Claudio Pacchierotti (Rainbow team), Julien Pettré.
• PhD in progress (beginning Sept. 2019): Alberto Jovane, Modélisation de mouvements réactifs et comportements non verbaux pour la création d’acteurs digitaux pour la réalité virtuelle, Marc Christie, Ludovic Hoyet, Claudio Pacchierotti (Rainbow team), Julien Pettré.
• PhD in progress (beginning Jun. 2020): Lucas Mourot, Learning-Based Human Character Animation Synthesis for Content Production, Pierre Hellier (InterDigital), Ludovic Hoyet, François Le Clerc (InterDigital).
• PhD in progress (beginning Nov. 2020): Thomas Chatagnon, Micro-to-macro energy-based interaction models for dense crowds behavioral simulations, Ecole normale supérieure de Rennes, Ludovic Hoyet, Anne-Hélène Olivier, Charles Pontonnier, Julien Pettré.
• PhD in progress (beginning Nov. 2020): Vicenzo Abichequer-Sangalli, Humains virtuels expressifs et réactifs pour la réalité virtuelle, Marc Christie, Ludovic Hoyet, Carol O'Sullivan (TCD, Ireland), Julien Pettré.
• PhD in progress (beginning Nov. 2020): Tairan Yin, Création de scènes peuplées dynamiques pour la réalité virtuelle, Marc Christie, Marie-Paule Cani (LIX), Ludovic Hoyet, Julien Pettré.
• PhD in progress (beginning Oct. 2021): Maé Mavromatis, Towards “Avatar-Friendly” Characterization of Virtual Reality Interaction Methods, Ferran Argelaguet (Hybrid team), Ludovic Hoyet, Anatole Lécuyer (Hybrid team).
• PhD in progress (beginning Jan. 2022): Yulia Patotskaya, Appealing Character Design for Embodied Virtual Reality, Ludovic Hoyet, Julien Pettré, Katja Zibrek.
• PhD in progress (beginning Oct. 2022): Jordan Martin, Simulation immersive de foule pour l’étude et l’aménagement de lieux destinés à accueillir du public, Ludovic Hoyet, Jean-Luc Paillat, Julien Pettré, Etienne Pinsard.
• PhD in progress (beginning Oct 2021): Xiaoyuan Wang, Realistic planning of hand motions, with Adnane Boukhayma
• PhD in progress (beginning Oct 2020): Jean-Baptiste Bordier, Innovative interfaces for the creation of character animations, with Julien Pettré
• PhD in progress (beginning Oct 2019): Ludovic Burg, Real-time virtual cinematography for target tracking, with Franck Multon.

10.2.3 Juries

• Julien Pettré: PhD jury of Mr. David Julian GONON, EPFL, Switzerland
• Julien Pettré: PhD jury of Mr. Anjara Nobby RAKOTOARIVELO, Université Gustave Eiffel
• Julien Pettré: PhD jury of Mr. Jason CHEMIN, INSA Toulouse

10.3.1 Articles and contents

The activity of the VirtUs team has attracted the attention of several medias:

• BFMTV: article on Korea crowd crush events (links with CrowdDNA)
• Radio France - France Inter - Les Savanturiers: interview on CrowdDNA Hellfest experiments
• La Voix du Nord: article on Virtus experiments at Hellfest 2022
• Le Monde: article on Virtus experiments at Hellfest 2022
• Le Point: article on Stade de France events (crowd management) and relations with H2020 CrowdDNA
• Marianne: article on Stade de France events (crowd management) and relations with H2020 CrowdDNA
• ActuIA: article about H2020 CrowdDNA
• l'Usine Nouvelle : artcile about VirtUs' demos at Laval Virtual 2022
• l'Usine Nouvelle : artcile about VirtUs' demos at Laval Virtual 2022
• Radio Canada - Les Années Lumière : interview about VirtUs experiments at Hellfest

10.3.2 Interventions

• Ludovic Hoyet - Séminaire des Services du Centre Inria Rennes (Sept. 2022) - Présentation de la nouvelle Equipe Projet Inria Virtus.
• Ludovic Hoyet - Séminaire scientifique Sci-Rennes #4 (Nov. 2022) - Présentation du défi Inria “Avatar” : vers des représentations du `moi' virtuel plus incarnées et interactives.
• VirtUs participated the 2022 edition of the Laval Virtual forum where several demos were showcased. See newspaper article about our presence there.
• VirtUs participated the 2022 edition of the “Fête de la Science” in Rennes, where several demos were showcased. See newspaper article about our presence there.
• Julien Pettré - Pint of Science 2022 - Presentation of the VirtUs research acitvities on VR and crowd simulation to the general public (link).

International journals

• 1 articleB.Beatriz Cabrero-Daniel, R.Ricardo Marques, L.Ludovic Hoyet, J.Julien Pettré and J.Joseph Blat. Dynamic Combination of Crowd Steering Policies Based on Context.Computer Graphics Forum412May 2022, 209-219
  HALDOIback to text
• 2 articleA.Adèle Colas, W.Wouter van Toll, K.Katja Zibrek, L.Ludovic Hoyet, A.-H.Anne-Hélène Olivier and J.Julien Pettré. Interaction Fields: Intuitive Sketch-based Steering Behaviors for Crowd Simulation.Computer Graphics ForumApril 2022, 1-14
  HALDOIback to text
• 3 articleA.Alberto Jovane, P.Pierre Raimbaud, K.Katja Zibrek, C.Claudio Pacchierotti, M.Marc Christie, L.Ludovic Hoyet, A.-H.Anne-Hélène Olivier and J.Julien Pettré. Warping character animations using visual motion features.Computers and GraphicsNovember 2022
  HALDOIback to text
• 4 articleA.Ariel Kwiatkowski, E.Eduardo Alvarado, V.Vicky Kalogeiton, C. K.C. Karen Liu, J.Julien Pettré, M.Michiel van de Panne and M.-P.Marie-Paule Cani. A Survey on Reinforcement Learning Methods in Character Animation.Computer Graphics Forum2022, 1-27
  HALDOIback to textback to text
• 5 articleL.Lucas Mourot, L.Ludovic Hoyet, F.François Le Clerc and P.Pierre Hellier. UnderPressure: Deep Learning for Foot Contact Detection, Ground Reaction Force Estimation and Footskate Cleanup.Computer Graphics ForumSeptember 2022
  HALDOIback to text
• 6 articleP.Pratik Mullick, S.Sylvain Fontaine, C.Cécile Appert-Rolland, A.-H.Anne-Hélène Olivier, W. H.William H. Warren and J.Julien Pettré. Analysis of emergent patterns in crossing flows of pedestrians reveals an invariant of 'stripe' formation in human data.PLoS Computational Biology1862022, e1010210
  HALDOIback to text
• 7 articleN.Natalie Snyder, M.Michael Cinelli, V.Victoria Rapos, A.Armel Crétual and A.-H.Anne-Hélène Olivier. Collision avoidance strategies between two athlete walkers: Understanding impaired avoidance behaviours in athletes with a previous concussion.Gait & Posture922022, 24-29
  HALDOIback to textback to text
• 8 articleT.Tairan Yin, L.Ludovic Hoyet, M.Marc Christie, M.-P.Marie-Paule Cani and J.Julien Pettré. The One-Man-Crowd: Single User Generation of Crowd Motions Using Virtual Reality.IEEE Transactions on Visualization and Computer Graphics2022, 1-11
  HALDOIback to text

International peer-reviewed conferences

• 9 inproceedingsV.Vicenzo Abichequer Sangalli, L.Ludovic Hoyet, M.Marc Christie and J.Julien Pettré. A new framework for the evaluation of locomotive motion datasets through motion matching techniques.MIG 2022 - ACM SIGGRAPH Conference on Motion, Interaction and Games4Guanajuato Mexico, MexicoACMNovember 2022, 1-10
  HALDOIback to textback to text
• 10 inproceedingsJ.Jean Basset, B.Badr Ouannas, L.Ludovic Hoyet, F.Franck Multon and S.Stefanie Wuhrer. Impact of Self-Contacts on Perceived Pose Equivalences.MIG 2022 - ACM SIGGRAPH Conference on Motion, Interaction and GamesGuanajuato Mexico, MexicoACMNovember 2022, 1-10
  HALDOIback to textback to text
• 11 inproceedingsJ.-B.Jean-Baptiste Bordier, A.Anthony Mirabile, R.Robin Courant and M.Marc Christie. Smart Motion Trails for Animating in VR.5th IEEE International Conference on Artificial Intelligence and Virtual RealityVirtual, FranceDecember 2022
  HALback to text
• 12 inproceedingsP.Pierre Raimbaud, A.Alberto Jovane, K.Katja Zibrek, C.Claudio Pacchierotti, M.Marc Christie, L.Ludovic Hoyet, J.Julien Pettre and A.-H.Anne-Hélène Olivier. The Stare-in-the-Crowd Effect in Virtual Reality.VR 2022 - IEEE Conference on Virtual Reality and 3D User InterfacesChristchurch, New ZealandIEEEMarch 2022, 281-290
  HALDOIback to text

Conferences without proceedings

• 13 inproceedingsE.Emilie Leblong, P.Patrice Piette, M.Marie Dandois, E.Estelle Ceze, L.Louise Devigne, F.François Pasteau, M.Marie Babel, B.Bastien Fraudet, B. J.Bradford J. Mcfadyen and A.-H.Anne-Hélène Olivier. Navigation dans les musées de la ville de Rennes : approche écologique pour l’évaluation d’un module d’assistance à la conduite lors des interactions des usagers de fauteuil roulant électrique dans un environnement complexe..SOFPEL 2022 - 28ème congrès de la Société Francophone Posture Équilibre et LocomotionMarseille, FranceDecember 2022
  HALback to text
• 14 inproceedingsE.Emilie Leblong, P.Patrice Piette, M.Marie Dandois, E.Estelle Ceze, L.Louise Devigne, F.François Pasteau, M.Marie Babel and A.-H.Anne-Hélène Olivier. Evaluation de la confiance en soi lors de la visite d’un musée grâce à un module d’assistance à la conduite de fauteuil roulant électrique : Etude observationnelle prospective dans les musées de la ville de Rennes.SOFPEL 2022 - 28ème congrès de la Société Francophone Posture Équilibre et LocomotionRennes, FranceDecember 2022
  HALback to text

Other scientific publications

• 15 inproceedingsA.Agathe Bilhaut, A.Armel Crétual, M.Mathieu Ménard and A.-H.Anne-Hélène Olivier. Locomotion behavior of chronic non-specific low back pain (cNSLBP) patient while walking through apertures.ISPGR 2022 - Congres on International Society of Posture & Gait ResearchMontréal, CanadaJuly 2022
  HALback to text
• 16 inproceedingsA.Agathe Bilhaut, M.Mathieu Ménard, O.Olivier Roze, A.Armel Crétual and A.-H.Anne-Hélène Olivier. Etude du paradigme de passage à travers une ouverture horizontale dans une population présentant une lombalgie chronique commune.SOFPELMarseille, FranceDecember 2022
  HALback to text
• 17 inproceedingsA.-H.Anne-Hélène Olivier, M.Marie Babel, A.Armel Crétual, L.Leborgne Nicolas, L.Louise Devigne, F.François Pasteau, B.Bastien Fraudet, E.Emilie Leblong and J.Julien Pettré. Comment rejoindre un groupe quand on se déplace en fauteuil ? Etude observationnelle biomécanique des interactions piétons-usagers de fauteuil roulant.SOFMER 2022 - 37e congrès de la Société Française de Médecine Physique et de RéadaptationRennes, FranceDecember 2022
  HALback to text
• 18 inproceedingsA.-H.Anne-Hélène Olivier, A.Antoine Marin, J.Joris Boulo, A.Aurélie Dommes, G.Gaetan Merlhiot, N.-T.Nguyen-Thong Dang, F.Fabrice Vienne, R.Régis Lobjois, V.Viola Cavallo and A.Armel Crétual. Body movement strategies to initiate the crossing of a street in front of traditional and self-driving cars in young and older adults.ISPGR 2022 - Congres on International Society of Posture & Gait ResearchMontréal, CanadaJuly 2022, 1-1
  HALback to text
• 19 inproceedingsK.Katja Zibrek, B.Benjamin Niay, A.-H.Anne-Hélène Olivier, L.Ludovic Hoyet, J.Julien Pettré and R.Rachel Mcdonnell. Proximity in VR: The Importance of Character Attractiveness and Participant Gender.VR 2022 - IEEE Conference on Virtual Reality and 3D User InterfacesChristchurch, New ZealandMarch 2022, 1-2
  HALback to text