Keywords
Computer Science and Digital Science
- A5.2. Data visualization
- A5.5. Computer graphics
- A5.5.1. Geometrical modeling
- A5.5.2. Rendering
- A5.5.3. Computational photography
- A5.5.4. Animation
Other Research Topics and Application Domains
- B5.5. Materials
- B5.7. 3D printing
- B9.2.2. Cinema, Television
- B9.2.3. Video games
- B9.2.4. Theater
- B9.6.6. Archeology, History
1 Team members, visitors, external collaborators
Research Scientists
- Nicolas Holzschuch [Team leader, Inria, Senior Researcher, HDR]
- Fabrice Neyret [CNRS, Senior Researcher, HDR]
- Cyril Soler [Inria, Advanced Research Position, HDR]
Faculty Members
- Georges-Pierre Bonneau [Univ Grenoble Alpes, Professor, HDR]
- Joëlle Thollot [Institut polytechnique de Grenoble, Professor, HDR]
- Romain Vergne [Univ Grenoble Alpes, Associate Professor]
Post-Doctoral Fellow
- Maxime Garcia [Inria, until Jul 2021]
PhD Students
- Morgane Gerardin [Univ Grenoble Alpes]
- Ana Granizo Hidalgo [Univ Grenoble Alpes, from Oct 2021]
- Nicolas Guichard [KDAB, CIFRE, from Jul 2021]
- Nolan Mestres [Univ Grenoble Alpes]
- Ronak Molazem [Inria]
- Vincent Tavernier [Univ Grenoble Alpes, until Aug 2021]
- Sunrise Wang [Inria, until Nov 2021]
Technical Staff
- Julien Paparone [Univ Grenoble Alpes, Engineer]
Interns and Apprentices
- Mohamed Amine Farhat [Inria, from Feb 2021 until Jul 2021]
- Ana Granizo Hidalgo [Inria, from Feb 2021 until Jun 2021]
- Thien Huynh [Inria, from Mar 2021 until Jul 2021]
Administrative Assistant
- Diane Courtiol [Inria]
External Collaborators
- Mohamed Amine Farhat [Univ Grenoble Alpes, Jan 2021]
- Ana Granizo Hidalgo [Univ Grenoble Alpes, Sep 2021]
2 Overall objectives
Computer-generated pictures and videos are now ubiquitous: both for leisure activities, such as special effects in motion pictures, feature movies and video games, or for more serious activities, such as visualization and simulation.
Maverick was created as a research team in January 2012 and upgraded as a research project in January 2014. We deal with image synthesis methods. We place ourselves at the end of the image production pipeline, when the pictures are generated and displayed (see figure 1). We take many possible inputs: datasets, video flows, pictures and photographs, (animated) geometry from a virtual world... We produce as output pictures and videos.
These pictures will be viewed by humans, and we consider this fact as an important point of our research strategy, as it provides the benchmarks for evaluating our results: the pictures and animations produced must be able to convey the message to the viewer. The actual message depends on the specific application: data visualization, exploring virtual worlds, designing paintings and drawings... Our vision is that all these applications share common research problems: ensuring that the important features are perceived, avoiding cluttering or aliasing, efficient internal data representation, etc.
Computer Graphics, and especially Maverick is at the crossroad between fundamental research and industrial applications. We are both looking at the constraints and needs of applicative users and targeting long term research issues such as sampling and filtering.
Position of the team in the graphics pipeline
The Maverick project-team aims at producing representations and algorithms for efficient, high-quality computer generation of pictures and animations through the study of four Research problems:
- Computer Visualization, where we take as input a large localized dataset and represent it in a way that will let an observer understand its key properties,
- Expressive Rendering, where we create an artistic representation of a virtual world,
- Illumination Simulation, where our focus is modelling the interaction of light with the objects in the scene.
- Complex Scenes, where our focus is rendering and modelling highly complex scenes.
The heart of Maverick is understanding what makes a picture useful, powerful and interesting for the user, and designing algorithms to create these pictures.
We will address these research problems through three interconnected approaches:
- working on the impact of pictures, by conducting perceptual studies, measuring and removing artefacts and discontinuities, evaluating the user response to pictures and algorithms,
- developing representations for data, through abstraction, stylization and simplification,
- developing new methods for predicting the properties of a picture (e.g. frequency content, variations) and adapting our image-generation algorithm to these properties.
A fundamental element of the Maverick project-team is that the research problems and the scientific approaches are all cross-connected. Research on the impact of pictures is of interest in three different research problems: Computer Visualization, Expressive rendering and Illumination Simulation. Similarly, our research on Illumination simulation will gather contributions from all three scientific approaches: impact, representations and prediction.
3 Research program
The Maverick project-team aims at producing representations and algorithms for efficient, high-quality computer generation of pictures and animations through the study of four research problems:
- Computer Visualization where we take as input a large localized dataset and represent it in a way that will let an observer understand its key properties. Visualization can be used for data analysis, for the results of a simulation, for medical imaging data...
- Expressive Rendering, where we create an artistic representation of a virtual world. Expressive rendering corresponds to the generation of drawings or paintings of a virtual scene, but also to some areas of computational photography, where the picture is simplified in specific areas to focus the attention.
- Illumination Simulation, where we model the interaction of light with the objects in the scene, resulting in a photorealistic picture of the scene. Research include improving the quality and photorealism of pictures, including more complex effects such as depth-of-field or motion-blur. We are also working on accelerating the computations, both for real-time photorealistic rendering and offline, high-quality rendering.
- Complex Scenes, where we generate, manage, animate and render highly complex scenes, such as natural scenes with forests, rivers and oceans, but also large datasets for visualization. We are especially interested in interactive visualization of complex scenes, with all the associated challenges in terms of processing and memory bandwidth.
The fundamental research interest of Maverick is first, understanding what makes a picture useful, powerful and interesting for the user, and second designing algorithms to create and improve these pictures.
3.1 Research approaches
We will address these research problems through three interconnected research approaches:
Picture Impact
Our first research axis deals with the impact pictures have on the viewer, and how we can improve this impact. Our research here will target:
- evaluating user response: we need to evaluate how the viewers respond to the pictures and animations generated by our algorithms, through user studies, either asking the viewer about what he perceives in a picture or measuring how his body reacts (eye tracking, position tracking).
- removing artefacts and discontinuities: temporal and spatial discontinuities perturb viewer attention, distracting the viewer from the main message. These discontinuities occur during the picture creation process; finding and removing them is a difficult process.
Data Representation
The data we receive as input for picture generation is often unsuitable for interactive high-quality rendering: too many details, no spatial organisation... Similarly the pictures we produce or get as input for other algorithms can contain superfluous details.
One of our goals is to develop new data representations, adapted to our requirements for rendering. This includes fast access to the relevant information, but also access to the specific hierarchical level of information needed: we want to organize the data in hierarchical levels, pre-filter it so that sampling at a given level also gives information about the underlying levels. Our research for this axis include filtering, data abstraction, simplification and stylization.
The input data can be of any kind: geometric data, such as the model of an object, scientific data before visualization, pictures and photographs. It can be time-dependent or not; time-dependent data bring an additional level of challenge on the algorithm for fast updates.
Prediction and simulation
Our algorithms for generating pictures require computations: sampling, integration, simulation... These computations can be optimized if we already know the characteristics of the final picture. Our recent research has shown that it is possible to predict the local characteristics of a picture by studying the phenomena involved: the local complexity, the spatial variations, their direction...
Our goal is to develop new techniques for predicting the properties of a picture, and to adapt our image-generation algorithms to these properties, for example by sampling less in areas of low variation.
Our research problems and approaches are all cross-connected. Research on the impact of pictures is of interest in three different research problems: Computer Visualization, Expressive rendering and Illumination Simulation. Similarly, our research on Illumination simulation will use all three research approaches: impact, representations and prediction.
3.2 Cross-cutting research issues
Beyond the connections between our problems and research approaches, we are interested in several issues, which are present throughout all our research:
-
Sampling:
is an ubiquitous process occurring in all our application domains, whether photorealistic rendering (e.g. photon mapping), expressive rendering (e.g. brush strokes), texturing, fluid simulation (Lagrangian methods), etc. When sampling and reconstructing a signal for picture generation, we have to ensure both coherence and homogeneity. By coherence, we mean not introducing spatial or temporal discontinuities in the reconstructed signal. By homogeneity, we mean that samples should be placed regularly in space and time. For a time-dependent signal, these requirements are conflicting with each other, opening new areas of research.
-
Filtering:
is another ubiquitous process, occuring in all our application domains, whether in realistic rendering (e.g. for integrating height fields, normals, material properties), expressive rendering (e.g. for simplifying strokes), textures (through non-linearity and discontinuities). It is especially relevant when we are replacing a signal or data with a lower resolution (for hierarchical representation); this involves filtering the data with a reconstruction kernel, representing the transition between levels.
-
Performance and scalability:
are also a common requirement for all our applications. We want our algorithms to be usable, which implies that they can be used on large and complex scenes, placing a great importance on scalability. For some applications, we target interactive and real-time applications, with an update frequency between 10 Hz and 120 Hz.
-
Coherence and continuity:
in space and time is also a common requirement of realistic as well as expressive models which must be ensured despite contradictory requirements. We want to avoid flickering and aliasing.
-
Animation:
our input data is likely to be time-varying (e.g. animated geometry, physical simulation, time-dependent dataset). A common requirement for all our algorithms and data representation is that they must be compatible with animated data (fast updates for data structures, low latency algorithms...).
3.3 Methodology
Our research is guided by several methodological principles:
-
Experimentation:
to find solutions and phenomenological models, we use experimentation, performing statistical measurements of how a system behaves. We then extract a model from the experimental data.
-
Validation:
for each algorithm we develop, we look for experimental validation: measuring the behavior of the algorithm, how it scales, how it improves over the state-of-the-art... We also compare our algorithms to the exact solution. Validation is harder for some of our research domains, but it remains a key principle for us.
-
Reducing the complexity of the problem:
the equations describing certain behaviors in image synthesis can have a large degree of complexity, precluding computations, especially in real time. This is true for physical simulation of fluids, tree growth, illumination simulation... We are looking for emerging phenomena and phenomenological models to describe them (see framed box “Emerging phenomena”). Using these, we simplify the theoretical models in a controlled way, to improve user interaction and accelerate the computations.
-
Transferring ideas from other domains:
Computer Graphics is, by nature, at the interface of many research domains: physics for the behavior of light, applied mathematics for numerical simulation, biology, algorithmics... We import tools from all these domains, and keep looking for new tools and ideas.
-
Develop new fondamental tools:
In situations where specific tools are required for a problem, we will proceed from a theoretical framework to develop them. These tools may in return have applications in other domains, and we are ready to disseminate them.
-
Collaborate with industrial partners:
we have a long experience of collaboration with industrial partners. These collaborations bring us new problems to solve, with short-term or medium-term transfer opportunities. When we cooperate with these partners, we have to find what they need, which can be very different from what they want, their expressed need.
4 Application domains
The natural application domain for our research is the production of digital images, for example for movies and special effects, virtual prototyping, video games... Our research have also been applied to tools for generating and editing images and textures, for example generating textures for maps. Our current application domains are:
- Offline and real-time rendering in movie special effects and video games;
- Virtual prototyping;
- Scientific visualization;
- Content modeling and generation (e.g. generating texture for video games, capturing reflectance properties, etc);
- Image creation and manipulation.
5 Social and environmental responsibility
While research in the Maverick team generaly involves significantly greedy computer hardware (e.g. 1 Tflp graphics cards) and heavy computations, the objective of most work in the Maverick team is the improvement of the performance of algorithms or to create new methods to obtain results at a lower computation cost. The team can in some way be seen as having a favorable impact in the long term on the overall energy consumption in computer graphics activities such as movie productions, video games, etc.
6 Highlights of the year
This year has seen the publication of some long-term research on perception of materials 6, as well as the usage of neural networks in the process of generating lighting simulations 4.
6.1 Awards
The software/project "Panoramas Stylisés" has been a laureat of the INRIA Out of Labs Challenge 2021.
7 New software and platforms
7.1 New software
7.1.1 GRATIN
-
Keywords:
GLSL Shaders, Vector graphics, Texture Synthesis
-
Functional Description:
Gratin is a node-based compositing software for creating, manipulating and animating 2D and 3D data. It uses an internal direct acyclic multi-graph and provides an intuitive user interface that allows to quickly design complex prototypes. Gratin has several properties that make it useful for researchers and students. (1) it works in real-time: everything is executed on the GPU, using OpenGL, GLSL and/or Cuda. (2) it is easily programmable: users can directly write GLSL scripts inside the interface, or create new C++ plugins that will be loaded as new nodes in the software. (3) all the parameters can be animated using keyframe curves to generate videos and demos. (4) the system allows to easily exchange nodes, group of nodes or full pipelines between people.
- URL:
-
Contact:
Romain Vergne
-
Participants:
Pascal Barla, Romain Vergne
-
Partner:
UJF
7.1.2 HQR
-
Name:
High Quality Renderer
-
Keywords:
Lighting simulation, Materials, Plug-in
-
Functional Description:
HQR is a global lighting simulation platform. HQR software is based on the photon mapping method which is capable of solving the light balance equation and of giving a high quality solution. Through a graphical user interface, it reads X3D scenes using the X3DToolKit package developed at Maverick, it allows the user to tune several parameters, computes photon maps, and reconstructs information to obtain a high quality solution. HQR also accepts plugins which considerably eases the developpement of new algorithms for global illumination, those benefiting from the existing algorithms for handling materials, geometry and light sources.
- URL:
-
Contact:
Cyril Soler
-
Participant:
Cyril Soler
7.1.3 libylm
-
Name:
LibYLM
-
Keyword:
Spherical harmonics
-
Functional Description:
This library implements spherical and zonal harmonics. It provides the means to perform decompositions, manipulate spherical harmonic distributions and provides its own viewer to visualize spherical harmonic distributions.
- URL:
-
Author:
Cyril Soler
-
Contact:
Cyril Soler
7.1.4 ShwarpIt
-
Name:
ShwarpIt
-
Keyword:
Warping
-
Functional Description:
ShwarpIt is a simple mobile app that allows you to manipulate the perception of shapes in images. Slide the ShwarpIt slider to the right to make shapes appear rounder. Slide it to the left to make shapes appear more flat. The Scale slider gives you control on the scale of the warping deformation.
- URL:
-
Contact:
Georges-Pierre Bonneau
7.1.5 iOS_system
-
Keyword:
IOS
-
Functional Description:
From a programmer point of view, iOS behaves almost as a BSD Unix. Most existing OpenSource programs can be compiled and run on iPhones and iPads. One key exception is that there is no way to call another program (system() or fork()/exec()). This library fills the gap, providing an emulation of system() and exec() through dynamic libraries and an emulation of fork() using threads.
While threads can not provide a perfect replacement for fork(), the result is good enough for most usage, and open-source programs can easily be ported to iOS with minimal efforts. Examples of softwares ported using this library include TeX, Python, Lua and llvm/clang.
-
Release Contributions:
This version makes iOS_system available as Swift Packages, making the integration in other projects easier.
- URL:
-
Contact:
Nicolas Holzschuch
7.1.6 Carnets for Jupyter
-
Keywords:
IOS, Python
-
Functional Description:
Jupyter notebooks are a very convenient tool for prototyping, teaching and research. Combining text, code snippets and the result of code execution, they allow users to write down ideas, test them, share them. Jupyter notebooks usually require connection to a distant server, and thus a stable network connection, which is not always possible (e.g. for field trips, or during transport). Carnets runs both the server and the client locally on the iPhone or iPad, allowing users to create, edit and run Jupyter notebooks locally.
- URL:
-
Contact:
Nicolas Holzschuch
7.1.7 a-Shell
-
Keywords:
IOS, Smartphone
-
Functional Description:
a-Shell is a terminal emulator for iOS. It behaves like a Unix terminal and lets the user run commands. All these commands are executed locally, on the iPhone or iPad. Commands available include standard terminal commands (ls, cp, rm, mkdir, tar, nslookup...) but also programming languages such as Python, Lua, C and C++. TeX is also available. Users familiar with Unix tools can run their favorite commands on their mobile device, on the go, without the need for a network connection.
- URL:
-
Contact:
Nicolas Holzschuch
7.1.8 X3D TOOLKIT
-
Name:
X3D Development pateform
-
Keywords:
X3D, Geometric modeling
-
Functional Description:
X3DToolkit is a library to parse and write X3D files, that supports plugins and extensions.
- URL:
-
Contact:
Cyril Soler
-
Participants:
Gilles Debunne, Yannick Le Goc
7.1.9 GigaVoxels
-
Functional Description:
Gigavoxel is a software platform which goal is the real-time quality rendering of very large and very detailed scenes which couldn't fit memory. Performances permit showing details over deep zooms and walk through very crowdy scenes (which are rigid, for the moment). The principle is to represent data on the GPU as a Sparse Voxel Octree which multiscale voxels bricks are produced on demand only when necessary and only at the required resolution, and kept in a LRU cache. User defined producer lays accross CPU and GPU and can load, transform, or procedurally create the data. Another user defined function is called to shade each voxel according to the user-defined voxel content, so that it is user choice to distribute the appearance-making at creation (for faster rendering) or on the fly (for storageless thin procedural details). The efficient rendering is done using a GPU differential cone-tracing using the scale corresponding to the 3D-MIPmapping LOD, allowing quality rendering with one single ray per pixel. Data is produced in case of cache miss, and thus only whenever visible (accounting for view frustum and occlusion). Soft-shadows and depth-of-field is easily obtained using larger cones, and are indeed cheaper than unblurred rendering. Beside the representation, data management and base rendering algorithm themself, we also worked on realtime light transport, and on quality prefiltering of complex data. Ongoing researches are addressing animation. GigaVoxels is currently used for the quality real-time exploration of the detailed galaxy in ANR RTIGE. Most of the work published by Cyril Crassin (and al.) during his PhD (see http://maverick.inria.fr/Members/Cyril.Crassin/ ) is related to GigaVoxels. GigaVoxels is available for Windows and Linux under the BSD-3 licence.
- URL:
-
Contact:
Fabrice Neyret
-
Participants:
Cyril Crassin, Eric Heitz, Fabrice Neyret, Jérémy Sinoir, Pascal Guehl, Prashant Goswami
7.1.10 MobiNet
-
Keywords:
Co-simulation, Education, Programmation
-
Functional Description:
The MobiNet software allows for the creation of simple applications such as video games, virtual physics experiments or pedagogical math illustrations. It relies on an intuitive graphical interface and language which allows the user to program a set of mobile objects (possibly through a network). It is available in public domain for Linux,Windows and MacOS.
- URL:
-
Contact:
Fabrice Neyret
-
Participants:
Fabrice Neyret, Franck Hétroy, Joelle Thollot, Samuel Hornus, Sylvain Lefebvre
-
Partners:
CNRS, LJK, INP Grenoble, Inria, IREM, Cies, GRAVIR
7.1.11 PROLAND
-
Name:
PROcedural LANDscape
-
Keywords:
Real time, 3D, Realistic rendering, Masses of data, Atmosphere, Ocean
-
Functional Description:
The goal of this platform is the real-time quality rendering and editing of large landscapes. All features can work with planet-sized terrains, for all viewpoints from ground to space. Most of the work published by Eric Bruneton and Fabrice Neyret (see http://evasion.inrialpes.fr/Membres/Eric.Bruneton/ ) has been done within Proland and integrated in the main branch. Proland is available under the BSD-3 licence.
- URL:
-
Contact:
Fabrice Neyret
-
Participants:
Antoine Begault, Eric Bruneton, Fabrice Neyret, Guillaume Piolet
8 New results
8.1 Light Transport Simulation
8.1.1 Spectral Analysis of the Light Transport Operator
Participants: Ronak Molazem, Cyril Soler.
In this work we study the spectral properties of the light transport operator. We proved in particular that this operator is not compact and therefore the light transport equation does not benefit from the guaranties that normally come with Fredholm equations. This contradicts several previous work that took the compactness of the light transport operator for granted, and it explains why methods strive to bound errors next to scene edges where the integral kernel is not bounded.
This study also brought us to devise new methods to compute eigenvalues from the light transport operator in Lambertian scenes (where in some situations is locally behaves like a compact operator despite not being compact).These methods combine results from the fields of pure and abstract mathematics: resolvent theory, Monte-Carlo factorisation of large matrices and Fredholm determinents. We proved in particular that it is possible to compute the eigenvalues of the light transport operator by integrating "circular" light paths of various lengths accross the scene, and that eigenvalues are real in lambertian scenes despite the operator not being normal.
This work is part of the PhD of Ronak Molazem and is funded by the ANR project "CaLiTrOp". One part of this work has been submitted to Siggraph 2022. The other part (eigenvalue calculation methods) will be submitted to TOG in the spring of 2022.
|
|
|
|
8.1.2 Low Dimension Approximations of Light Scattering
Participants: Hugo Rens, Cyril Soler.
Light transport is known to be a low rank linear operator: the vector space formed by solutions of a light transport problem for different initial conditions is of low dimension. This is especially true when participating media are involved. Approximating this space using appropriate bases is therefore of paramount help to efficiently compute solutions to light transport problems in these environments.
We propose to compute a low dimensional approximation of the light transport operator in participating taking inspiration from techniques in Monte-Carlo approximation techniques for large matrices, and use the result in a real-time rendering engine, where, after some precomputation, a new image can be computed while changing the illumination and view point.
This work is an ongoing collaboration with the IRIT laboratory (Toulouse, France), and is funded by the ANR project "CaLiTrOp".
8.1.3 Real-Time Rendering of Self Indirect Illumination for Hero Assets using a Data-Driven Radiance Transfer Approach
Participants: Laurent Belcour, Thomas Deliot, Cyril Soler, Wilhem Barbier, Erik Heitz.
We introduce a new method to render self indirect illumination on key objects in real-time. Core to our method is the idea to reduce the genericity commonly found in Precomputed Radiance Transfer methods and to tailor the set of achievable indirect illuminations to the set of observable lighting conditions at runtime. Using a data-driven approach, we pre-compute and store the effects of multiple scattering of light from an object onto itself under specific lighting conditions. From this dataset, we extract: a basis decomposition to reconstruct the achievable indirect illumination and the transfer function from few measurement of direct illumination to the coefficients that reconstruct indirect illumination in such a basis. Thanks to this, at runtime, our method requires only a few texture fetches in the fragment shader. We show how our method enables to render interreflection or transluscency on objects efficiently with limited computing ressources.
|
|
This work is an ongoing collaboration with the Unity Research Grenoble (Laurent Belcour, Eric Heitz, Thomas Deliot). It has been submitted to Siggraph 2022 in January 2022.
8.2 Modelisation of material appearance
8.2.1 A Low-Dimension Perceptual Space for Intuitive Material Appearance Editing
Participants: Weiqi Qi, Holly Rushmeier, Cyril Soler.
Understanding human visual perception to materials is challenging. Real-world material appearance functions are usually described by high-dimensional tabulated measurements, and their connections to the human visual system are not explicitly defined. In this work, we present a method to build an intuitive low-dimensional perceptual space for real-worldmeasured material data based on its underlying non-linear reflectance manifold. Unlike many previous works that address individual perceptual attributes (such as gloss), we focus on building a complete perceptual space with multiple attributes and understanding how they influence perception of materials.
We first use crowdsourced data on the perceived similarity of measured materials to build a multi-dimensional scaling model. Then we use the output as a low-dimensional manifold to construct a perceptual space that can be interpolated and extrapolated with Gaussian process regression. Given with the perceptual space, we propose a material design interface using the manifold, allowing users to edit material appearance using its perceptual dimensions with visual variations and instant feedback. To evaluate the interface, we conduct a user study to compare the accuracy of editing and matching material appearances using perceptual and physical dimensions.
A new perceptual space for materials
This work is an ongoing collaboration with Yale University (With Prof. Holly Rushmeier and PhD. Weiqi Shi). It was published and presented at the Eurographics Symposium on rendering 2021 8.
8.2.2 Perceptual quality of BRDF approximations: dataset and metrics
Participants: Guillaume Lavoué, Jean-Philippe Farrugia, Nicolas Bonneel, Cyril Soler.
Bidirectional Reflectance Distribution Functions (BRDFs) are pivotal to the perceived realism in image synthesis. While measured BRDF datasets are available, reflectance functions are most of the time approximated by analytical formulas for storage efficiency reasons. These approximations are often obtained by minimizing metrics such as —or weighted quadratic—distances, but these metrics do not usually correlate well with perceptual quality when the BRDF is used in a rendering context, which motivates a perceptual study.
Perceptual study of BRDF approximation errors
The contributions of this paper are threefold. First, we perform a large-scale user study to assess the perceptual quality of 2026 BRDF approximations, resulting in 84138 judgments across 1005 unique participants. We explore this dataset and analyze perceptual scores based on material type and illumination. Second, we assess nine analytical BRDF models in their ability to approximate tabulated BRDFs. Third, we assess several image-based and BRDF-based (, optimal transport and kernel distance) metrics in their ability to approximate perceptual similarity judgments.
This work is a collaboration with the LIRIS laboratory (Lyon) that involves J.P.Farrugia, G.Lavoue and N.Bonneel. It has been published in Computer Graphics forum, proceedings of the Eurographics 2021 conference 6.
8.2.3 Correlation between micro-structural features and color of nanocrystallized powders of hematite
Participants: Morgane Gerardin, Pauline Martinetto, Nicolas Holzschuch.
Pigments are quite complex materials whose appearance involves many optical phenomena. In a recent paper 5, we establish a comparison between the size and shape of grains of powdered hematite and the pigment color. We focused on hematite as it is a traditional pigment, whose origin of coloration has been well discussed in the literature. Pure nanocrystallized hematite powders have been synthesized using different synthesis routes. These powders have been characterized by X–ray powder diffraction and scanning electronic microscopy. The color of the samples has been studied by visible–NIR spectrophotometry. We obtained hematite with both various grain morphologies and noticeably different shades going from orange–red to purple. Colorimetric parameters in CIE L*a*b* color space and diffuse reflectance spectra properties were studied against the structural parameters. For small nanocrystals, hue is increasing with the grain size until a critical diameter of about 80 nm, where the trend is reversed. We believe a combination of multiple physical phenomena occurring at this scale may explain this trend.
This problem is particularly interesting because all these pigments have the same chemical composition; changes in color are only due to physical phenomena, which are not described by the models currently used in Computer Graphics.
8.2.4 Participating media
Participants: Beibei Wang, Nicolas Holzschuch.
Rendering participating media is important to the creation of photorealistic images. Participating media have a translucent aspect that comes from light being scattered inside the material. Several effects are at play, depending on the characteristic of the material: with a large albedo or a small mean-free-path, higher-order scattering effects dominate. With low opacity materials, single scattering effects dominate. With a refractive boundary, we have concentration of the incoming light inside the medium, resulting in volume caustics.
Multiple scattering effects are mostly low frequency and can be precomputed for fast simulations. In a recent paper 4, we show that precomputed multiple scattering effects can be encoded using a neural network, resulting in a significant gain in memory: double and multiple scattering effects for the entire participating media space are encoded using only 23.6 KB of memory. This algorithm is implemented on GPU, and achieves 50 ms per frame in typical scenes and provides results almost identical to the reference.
8.3 Expressive rendering
8.3.1 Local Light Alignment for Multi-Scale Shape Depiction
Participants: Nolan Mestre, Romain Vergne, Joëlle Thollot.
Motivated by recent findings in the field of visual perception, we present in 7 a novel approach for enhancing shape depiction and perception of surface details. We propose a shading-based technique that relies on locally adjusting the direction of light to account for the different components of materials. Our approach ensures congruence between shape and shading flows, leading to an effective enhancement of the perception of shape and details while impairing neither the lighting nor the appearance of materials. It is formulated in a general way allowing its use for multiple scales enhancement in real-time on the GPU, as well as in global illumination contexts. We also provide artists with fine control over the enhancement at each scale.
8.3.2 Coherent Mark-based Stylization of 3D Scenes at the Compositing Stage
Participants: Maxime Garcia, Mohamed Amine Farhat, Romain Vergne, Joëlle Thollot.
We present in 3 a novel temporally coherent stylized rendering technique working entirely at the compositing stage. We first generate a distribution of 3D anchor points using an implicit grid based on the local object positions stored in a G-buffer, hence following object motion. We then draw splats in screen space anchored to these points so as to be motion coherent. To increase the perceived flatness of the style, we adjust the anchor points density using a fractalization mechanism. Sudden changes are prevented by controlling the anchor points opacity and introducing a new order-independent blending function. We demonstrate the versatility of our method by showing a large variety of styles thanks to the freedom offered by the splats content and their attributes that can be controlled by any G-buffer.
Wide range of styles using different control maps and splats produced by our real-time and temporally coherent anchor point generation process.
8.4 Synthetizing new materials
8.4.1 Geometric construction of auxetic metamaterials
Participants: Georges-Pierre Bonneau, Stefanie Hahmann, Johana Marku.
This work is devoted to a category of metamaterials called auxetics, identified by their negative Poisson's ratio. Our work consists in exploring geometrical strategies to generate irregular auxetic structures. More precisely we seek to reduce the Poisson's ratio, by pruning an irregular network based solely on geometric criteria. We introduce a strategy combining a pure geometric pruning algorithm followed by a physics-based testing phase to determine the resulting Poisson's ratio of our structures. We propose an algorithm that generates sets of irregular auxetic networks. Our contributions include geometrical characterization of auxetic networks, development of a pruning strategy, generation of auxetic networks with low Poisson's ratio, as well as validation of our approach. We provide statistical validation of our approach on large sets of irregular networks, and we additionally laser-cut auxetic networks in sheets of rubber. The findings reported here show that it is possible to reduce the Poisson's ratio by geometric pruning, and that we can generate irregular auxetic networks at lower processing times than a physics-based approach. This work has been published in the journal Computer Graphics Forum 2, and presented at the conference Eurographics'2021.
This work is part of the FET Open European Project ADAM2
Experimental realization of such an auxetic network, obtained by laser-cutting a sheet of natural rubber.
9 Bilateral contracts and grants with industry
9.1 Bilateral contracts with industry
Participants: Nicolas Holzschuch, Nicolas Guichard.
We have a contract with KDAB France, connected with the PhD thesis of Nicolas Guichard (CIFRE).
10 Partnerships and cooperations
10.1 International initiatives
10.1.1 Informal international partners
Maverick has informal relation with many companies related to special effects, gaming and image editing ( Weta Digital, Allegorithmics, Unity, Ubisoft, nVidia, Adobe, ... ), former contract partners, companies of ex-students ( Artineering ), and more. In particular, we have ongoing discussions with UbiSoft aim at defining common interests to create new collaborations, and with RSA Cosmos to renew collaborations.
10.1.2 International collaborations
Nicolas Holzschuch collaborated with Pr B.Wang from Nanjing University of Science and Technology, China.
Cyril Soler collaborated with Pr. Kartic Subr (University of Edinburgh), and Pr. Holly Rushmeier (Yale University).
10.2 European initiatives
10.2.1 FP7 & H2020 projects
Georges-Pierre Bonneau is part of the FET Open European Project "ANALYSIS, DESIGN And MANUFACTURING using MICROSTRUCTURES" (ADAM2, Grant agreement ID: 862025) , together with Stefanie Hahmann and Mélina Skouras from INRIA-Team ANIMA. This project is devoted to the geometric design, numerical analysis and fabrication of objects made of microstructures. The INRIA team participates to the research workpackage entitled "Modeling of curved geometries using microstructures and auxetic metamaterials", and leads the workpackage "Dissemination". A paper was published in 2021 by G-P. Bonneau in project ADAM2 2.
10.3 National initiatives
10.3.1 ANR: Materials
Participants: Nicolas Holzschuch [contact], Romain Vergne.
We are funded by the ANR for a joint research project on acquisition and restitution of micro-facet based materials. This project is in cooperation with Océ Print Logic technologies, the Museum of Ethnography at the University of Bordeaux and the Manao team at Inria Bordeaux. The grant started in October 2015, for 60 months.10.3.2 CDP: Patrimalp 2.0
Participants: Nicolas Holzschuch [contact], Romain Vergne.
The main objective and challenge of Patrimalp 2.0 is to develop a cross-disciplinary approach in order to get a better knowledge of the material cultural heritage in order to ensure its sustainability, valorization and diffusion in society. Carried out by members of UGA laboratories, combining skills in human sciences, geosciences, digital engineering, material sciences, in close connection with stakeholders of heritage and cultural life, curators and restorers, Patrimalp 2.0 intends to develop of a new interdisciplinary science: Cultural Heritage Science. The grant starts in January 2018, for a period of 48 months.10.3.3 ANR: CaLiTrOp
Participants: Cyril Soler [contact].
Computing photorealistic images relies on the simulation of light transfer in a 3D scene, typically modeled using geometric primitives and a collection of reflectance properties that represent the way objects interact with light. Estimating the color of a pixel traditionally consists in integrating contributions from light paths connecting the light sources to the camera sensor at that pixel.In this ANR we explore a transversal view of examining light transport operators from the point of view of infinite dimensional function spaces of light fields (imagine, e.g., reflectance as an operator that transforms a distribution of incident light into a distribution of reflected light). Not only are these operators all linear in these spaces but they are also very sparse. As a side effect, the sub-spaces of light distributions that are actually relevant during the computation of a solution always boil down to a low dimensional manifold embedded in the full space of light distributions.
Studying the structure of high dimensional objects from a low dimensional set of observables is a problem that becomes ubiquitous nowadays: Compressive sensing, Gaussian processes, harmonic analysis and differential analysis, are typical examples of mathematical tools which will be of great relevance to study the light transport operators.
Expected results of the fundamental-research project CALiTrOp, are a theoretical understanding of the dimensionality and structure of light transport operators, bringing new efficient lighting simulation methods, and efficient approximations of light transport with applications to real time global illumination for video games.
10.4 Regional initiatives
One publication was a collaboration with the LIRIS laboratory in Lyon 6.
11 Dissemination
Participants: Georges-Pierre Bonneau, Nicolas Holzschuch, Fabrice Neyret, Cyril Soler, Joëlle Thollot, Romain Vergne.
11.1 Promoting scientific activities
11.1.1 Scientific events: organisation
General chair, scientific chair
- Romain Vergne was paper chair at JFIG 2021.
Member of the organizing committees
- Georges-Pierre Bonneau was Session Chair at EuroVis'2021
- Nicolas Holzschuch is a member of the Eurographics Symposium on Rendering Steering Committee.
11.1.2 Scientific events: selection
Chair of conference program committees
- Georges-Pierre Bonneau was co-Chair of Solid and Physical Modeling 2021.
Member of the conference program committees
- Georges-Pierre Bonneau was member of the IPC of Shape Modeling International 2021
- Nicolas Holzschuch was a member of the program committee for: Siggraph 2021 technical papers, Siggraph 2021 general program, Eurographics Symposium on Rendering (EGSR) 2021, SIBGRAPI 2021.
- Cyril Soler was member of the program committee of Pacific Graphics 2021 and Gaphics Interface 2021.
Reviewer
-
Maverick faculties are regular reviewers of most of the major journals and conferences of the domain.
11.1.3 Journal
Member of the editorial boards
- Georges-Pierre Bonneau has been appointed to the Editorial Board of the journal Computer Aided Design, Elsevier.
Reviewer - reviewing activities
- Georges-Pierre Bonneau has reviewed papers submitted to IEEE Computer Graphics & Applications, Graphicals Models (Elsevier) and EnvirVis'2021.
- Cyril Soler was reviewer for Siggraph 2021, CGF 2021, TVCG, Graphics Interface 2021 and Pacific Graphics 2021.
11.1.4 Invited talks
Romain Vergne - at JFIG 2021
11.1.5 Leadership within the scientific community
Romain Vergne received the Young Researcher Fellow (EGFR)
11.1.6 Research administration
Nicolas Holzschuch is a member of the Conseil National de l'Enseignement Supérieur et de la Recherche (CNESER). He is co-responsible (with Anne-Marie Kermarrec) of the Inria International Lab (IIL) “Inria-EPFL”.
11.2 Teaching - Supervision - Juries
11.2.1 Teaching
Joëlle Thollot and Georges-Pierre Bonneau are both full Professor of Computer Science. Romain Vergne is an associate professor in Computer Science. They teach general computer science topics at basic and intermediate levels, and advanced courses in computer graphics and visualization at the master levels. Nicolas Holzschuch teaches advanced courses in computer graphics at the master level.
Joëlle Thollot is in charge of the Erasmus+ program ASICIAO: a EU program to support six schools from Senegal and Togo in their pursuit of autonomy by helping them to develop their own method of improving quality in order to obtain the CTI accreditation.
- Licence: Joëlle Thollot, Théorie des langages, 45h, L3, ENSIMAG, France
- Licence: Joëlle Thollot, Module d'accompagnement professionnel, 10h, L3, ENSIMAG, France
- Master : Joelle Thollot, English courses using theater, 18h, M1, ENSIMAG, France.
- Master : Joelle Thollot, Analyse et conception objet de logiciels, 24h, M1, ENSIMAG, France.
- Licence : Romain Vergne, Introduction to algorithms, 64h, L1, UGA, France.
- Licence : Romain Vergne, WebGL, 29h, L3, IUT2 Grenoble, France.
- Licence : Romain Vergne, Programmation, 68h. L1, UGA, France.
- Master : Romain Vergne, Image synthesis, 27h, M1, UGA, France.
- Master : Romain Vergne, 3D graphics, 15h, M1, UGA, France.
- Master : Nicolas Holzschuch, Computer Graphics II, 18h, M2 MoSIG, France.
- Master : Nicolas Holzschuch, Synthèse d’Images et Animation, 32h, M2, ENSIMAG, France.
- Licence: Georges-Pierre Bonneau, Algorithmique et Programmation Impérative, 23h, L3, Polytech-Grenoble, France.
- Master: Georges-Pierre Bonneau, responsable de la 4ième année du département INFO, 32h, M1, Polytech-Grenoble, France
- Master: Georges-Pierre Bonneau, Image Synthesis, 23h, M1, Polytech-Grenoble, France
- Master: Georges-Pierre Bonneau, Data Visualization, 40h, M2, Polytech-Grenoble, France
- Master: Georges-Pierre Bonneau, Digital Geometry, 23h, M1, UGA
- Master: Georges-Pierre Bonneau, Information Visualization, 22h, Mastere, ENSIMAG, France.
- Master: Georges-Pierre Bonneau, Scientific Visualization, M2, ENSIMAG, France.
- Master: Georges-Pierre Bonneau, Computer Graphics II, 18h, M2 MoSiG, UGA, France.
11.2.2 Supervision
- Georges-Pierre Bonneau co-supervised the PhD of Emmanuel Rodriguez in the ANIMA team.
- Joelle Thollot and Romain Vergne co-supervised the PhD of Nolan Mestres.
- Cyril Soler supervised the PhD of Ronak Molazem, and the internship of Wilhem Barbier.
- Fabrice Neyret, Joelle Thollot and Romain Vergne co-supervised the PhD of Vincent Tavernier.
- Nicolas Holzschuch supervised the PhD of Morgane Gérardin (4A), Sunrise Wang (4A), Nicolas Guichard (1A) and Anita Granizo-Hidalgo (1A).
11.2.3 Juries
- Georges-Pierre Bonneau was President of the Jury for the PhD Thesis of Matthieu Armando at Université Grenoble-Alpes on October 21, 2021
- Nicolas Holzschuch was a reviewer for the PhD thesis of Basile Fraboni (Université de Lyon) and Mégane Bati (Université de Bordeaux).
- Romain Vergne was part of the jury for the PhD Thesis of Manuel Lagunas at University of Zaragoza (Spain) on December 17, 2021
- Joelle Thollot was a reviewer for the PhD thesis of Ondřej Texler at the faculty of Electrical Engineering, Czech Technical University in Prague on April 8, 2021.
- Joelle Thollot was President of the Jury for the PhD Thesis of François Desrichard at Université de Toulouse on December 6, 2021.
11.3 Popularization
11.3.1 Internal or external Inria responsibilities
Romain Vergne was co-responsible of the Groupe de Travail Rendu et Visualisation of the French association for Computer Graphics (AFIG).
11.3.2 Articles and contents
Fabrice Neyret maintains the blog shadertoy-Unofficial and various shaders examples on Shadertoy site to popularize GPU technologies as well as disseminates academic models within computer graphics, computer science, applied math and physics fields. About 24k pages viewed and 11k unique visitors (93% out of France) in 2021.
Fabrice Neyret maintains the the blog desmosGraph-Unofficial to popularize the use of interactive grapher DesmosGraph for research, communication and pedagogy. For this year, about 10k pages viewed and 6k unique visitors (99% out of France) in 2021.
In August 2020 Fabrice Neyret launched the blog chiffres-et-paradoxes (in French) to popularize common errors, misunderstandings and paradoxes about statistics and numerical reasoning. About 9k pages viewed and 4.5k unique visitors since then (15% out of France, knowing the blog is in French) on the blog, plus the viewers via the Facebook and Twitter pages.
12 Scientific production
12.1 Major publications
- 1 inproceedingsA Low-Dimensional Perceptual Space for Intuitive BRDF Editing.EGSR 2021 - Eurographics Symposium on Rendering - DL-only TrackSaarbrücken, GermanyJune 2021, 1-13
12.2 Publications of the year
International journals
International peer-reviewed conferences