2025Activity reportProject-TeamGRAPHDECO
RNSR: 201521163T- Research center Inria Centre at Université Côte d'Azur
- Team name: GRAPHics and DEsign with hEterogeneous COntent
Creation of the Project-Team: 2015 July 01
Each year, Inria research teams publish an Activity Report presenting their work and results over the reporting period. These reports follow a common structure, with some optional sections depending on the specific team. They typically begin by outlining the overall objectives and research programme, including the main research themes, goals, and methodological approaches. They also describe the application domains targeted by the team, highlighting the scientific or societal contexts in which their work is situated.
The reports then present the highlights of the year, covering major scientific achievements, software developments, or teaching contributions. When relevant, they include sections on software, platforms, and open data, detailing the tools developed and how they are shared. A substantial part is dedicated to new results, where scientific contributions are described in detail, often with subsections specifying participants and associated keywords.
Finally, the Activity Report addresses funding, contracts, partnerships, and collaborations at various levels, from industrial agreements to international cooperations. It also covers dissemination and teaching activities, such as participation in scientific events, outreach, and supervision. The document concludes with a presentation of scientific production, including major publications and those produced during the year.
Keywords
Computer Science and Digital Science
- A3.1.4. Uncertain data
- A3.1.10. Heterogeneous data
- A5.1. Human-Computer Interaction
- A5.1.1. Engineering of interactive systems
- A5.1.2. Evaluation of interactive systems
- A5.1.5. Body-based interfaces
- A5.1.8. 3D User Interfaces
- A5.1.9. User and perceptual studies
- A5.2. Data visualization
- A5.3.5. Computational photography
- A5.5. Computer graphics
- A5.5.1. Geometrical modeling
- A5.5.2. Rendering
- A5.5.3. Computational photography
- A5.5.4. Animation
- A5.6. Virtual reality, augmented reality
- A5.6.1. Virtual reality
- A5.6.2. Augmented reality
- A5.6.3. Avatar simulation and embodiment
- A5.9.1. Sampling, acquisition
- A5.9.3. Reconstruction, enhancement
- A6.1. Methods in mathematical modeling
- A6.1.4. Multiscale modeling
- A6.1.5. Multiphysics modeling
- A6.2. Scientific computing, Numerical Analysis & Optimization
- A6.2.6. Optimization
- A6.2.8. Computational geometry and meshes
- A6.3.1. Inverse problems
- A6.3.2. Data assimilation
- A6.3.5. Uncertainty Quantification
- A6.5.2. Fluid mechanics
- A6.5.3. Transport
- A8.3. Geometry, Topology
- A9.2. Machine learning
- A9.2.1. Supervised learning
- A9.2.2. Unsupervised learning
- A9.2.3. Reinforcement learning
- A9.2.4. Optimization and learning
- A9.2.5. Bayesian methods
- A9.2.6. Neural networks
- A9.2.8. Deep learning
- A9.3. Signal processing
- A9.10. Hybrid approaches for AI
- A9.12.4. 3D and spatio-temporal reconstruction
- A9.12.5. Object tracking and motion analysis
Other Research Topics and Application Domains
- B3.2. Climate and meteorology
- B3.3.1. Earth and subsoil
- B3.3.2. Water: sea & ocean, lake & river
- B3.3.3. Nearshore
- B3.4.1. Natural risks
- B5. Industry of the future
- B5.2. Design and manufacturing
- B5.5. Materials
- B5.7. 3D printing
- B5.8. Learning and training
- B8. Smart Cities and Territories
- B8.3. Urbanism and urban planning
- B9. Society and Knowledge
- B9.1.2. Serious games
- B9.2. Art
- B9.2.2. Cinema, Television
- B9.2.3. Video games
- B9.3. Medias
- B9.5.1. Computer science
- B9.5.2. Mathematics
- B9.5.3. Physics
- B9.5.5. Mechanics
- B9.5.6. Data science
- B9.6. Humanities
- B9.6.6. Archeology, History
- B9.8. Reproducibility
- B9.11.1. Environmental risks
1 Team members, visitors, external collaborators
Research Scientists
- George Drettakis [Team leader, INRIA, Senior Researcher, HDR]
- Adrien Bousseau [INRIA, Senior Researcher, HDR]
- Guillaume Cordonnier [INRIA, Researcher]
- Andreas Meuleman [INRIA, Starting Research Position, until Sep 2025]
Post-Doctoral Fellows
- Melike Aydinlilar [INRIA, Post-Doctoral Fellow]
- Linus Franke [INRIA, Post-Doctoral Fellow, from Jul 2025]
- Alban Gauthier [INRIA, from Apr 2025 until Jun 2025]
- Alban Gauthier [INRIA, Post-Doctoral Fellow, until Feb 2025]
- Simon Lucas [INRIA, Post-Doctoral Fellow, from Mar 2025]
- Simon Lucas [INRIA, until Feb 2025]
- Anran Qi [INRIA, Post-Doctoral Fellow, until Nov 2025]
- Marzia Riso [INRIA, Post-Doctoral Fellow]
PhD Students
- Berend Baas [INRIA]
- Aryamaan Jain [INRIA]
- Henro Kriel [INRIA]
- Alexandre Lanvin [INRIA]
- Panagiotis Papantonakis [INRIA]
- Yohan Poirier-Ginter [UNIV LAVAL QUEBEC]
- Petros Tzathas [INRIA]
- Nicolas Violante [INRIA]
Technical Staff
- Loic Gaillard [INRIA, Engineer, from Oct 2025]
- Jeffrey Hu [INRIA, Engineer, from Feb 2025 until Sep 2025]
- Ishaan Shah [INRIA, Engineer, until Nov 2025]
- Jiayi Wei [INRIA, Engineer, until May 2025]
Interns and Apprentices
- Oren Amsalhem [INRIA, Intern, from Feb 2025 until Aug 2025]
- Margot Catherinot [INRIA, Intern, from Jun 2025 until Aug 2025]
- Jackson James Cooper [INRIA, Intern, from Sep 2025]
- Nischay Diwan [INRIA, Intern, from Feb 2025 until May 2025]
- Boris Zhestiankin [INRIA, Intern, from Oct 2025]
Administrative Assistant
- Sophie Honnorat [INRIA]
Visiting Scientists
- Eugene Fiume [Simon Fraser University]
- Alexander Mai [UCSD, from Sep 2025]
- Gilda Manfredi [UNIV SALENTO, until Jun 2025]
- Eric Paquette [ETS MONTREAL, until May 2025]
2 Overall objectives
In traditional Computer Graphics (CG), input is accurately modeled by artists. Artists first create the 3D geometry – i.e., the surfaces used to represent the 3D scene. This task can be achieved using tools akin to sculpting for human-made objects, or using physical simulation for objects formed by natural phenomena. Artists then need to assign colors, textures and more generally material properties to each piece of geometry in the scene. Finally, they also define the position, type and intensity of the lights.
Creating all this 3D content by hand is a notoriously tedious process, both for novice users who do not have the skills to use complex modeling software, and for creative professionals who are primarily interested in obtaining a diversity of imagery and prototypes rather than in accurately specifying all the ingredients listed above. While physical simulation can alleviate some of this work for certain classes of objects (landscapes, fluids, plants), simulation algorithms are often costly and difficult to control.
Once all 3D elements of a scene are in place, a rendering algorithm is employed to generate a shaded, realistic image. Rendering algorithms typically involve the accurate simulation of light transport, accounting for the complex interactions between light and materials as light bounces over the surfaces of the scene to reach the camera. Similarly to the simulation of natural phenomena, the simulation of light transport is computationally expensive, and only provides meaningful results if the input is accurate and complete.
A major recent development is that many alternative sources of 3D content are becoming available. Cheap depth sensors but also video and photos allow anyone to capture real objects. However, the resulting 3D models are often inaccurate and incomplete due to limitations of these sensors and acquisition setups. There have also been significant advances in casual content creation, e.g., sketch-based modeling tools. But the resulting models are often approximate since people rarely draw accurate perspective and proportions, nor fine details. Unfortunately, the traditional Computer Graphics pipeline outlined above is unable to directly handle the uncertainty present in cheap sources of 3D content. The abundance and ease of access to inaccurate, incomplete and heterogeneous 3D content imposes the need to rethink the foundations of 3D computer graphics to allow uncertainty to be treated in inherent manner in Computer Graphics, from design and simulation all the way to rendering and prototyping.
The technological shifts we mention above, together with developments in computer vision and machine learning, and the availability of large repositories of images, videos and 3D models represent a great opportunity for new imaging methods. In GraphDeco, we have identified three major scientific challenges that we strive to address to make such visual content widely accessible:
- First, the design pipeline needs to be revisited to explicitly account for the variability and uncertainty of a concept and its representations, from early sketches to 3D models and prototypes. Professional practice also needs to be adapted to be accessible to all.
- Second, a new approach is required to develop computer graphics models and rendering algorithms capable of handling uncertain and heterogeneous data as well as traditional synthetic content.
- Third, physical simulation needs to be combined with approximate user inputs to produce content that is realistic and controllable.
We have developed a common thread that unifies these three axes: the combination of machine learning with optimization and simulation, allowing the treatment of uncertain data for the synthesis of visual content. This common methodology – which falls under the umbrella term of machine learning for visual computing – provides a shared language and toolbox for the three research axes in our group, allowing frequent and in-depth collaborations between all three permanent researchers of the group, and a strong cohesive dynamic for Ph.D. students and postdocs.
As a result of this approach, GRAPHDECO is one of the few groups worldwide with in-depth expertise of both computer graphics techniques and deep learning approaches, in all three “traditional pillars” of CG: modeling, animation and rendering.
3 Research program
3.1 Introduction
Our research program is oriented around three main axes: 1) Computer-Assisted Design with Heterogeneous Representations, 2) Graphics with Uncertainty and Heterogeneous Content, and 3) Physical Simulation of Natural Phenomena. These three axes are governed by a set of common fundamental goals, share many common methodological tools and are deeply intertwined in the development of applications.
3.2 Computer-Assisted Design with Heterogeneous Representations
Designers use a variety of visual representations to explore and communicate about a concept. Figure 1 illustrates some typical representations, including sketches, hand-made prototypes, 3D models, 3D printed prototypes or instructions.
|
|
|
|
| (a) Ideation sketch | (b) Presentation sketch | (c) Coarse prototype | (d) 3D model |
|
|
|
|
| (f) Simulation | (e) 3D Printing | (g) Technical diagram | (h) Instructions |
Various design sketches used to inspire our research.
The early representations of a concept, such as rough sketches and hand-made prototypes, help designers formulate their ideas and test the form and function of multiple design alternatives. These low-fidelity representations are meant to be cheap and fast to produce, to allow quick exploration of the design space of the concept. These representations are also often approximate to leave room for subjective interpretation and to stimulate imagination; in this sense, these representations can be considered uncertain. As the concept gets more finalized, time and effort are invested in the production of more detailed and accurate representations, such as high-fidelity 3D models suitable for simulation and fabrication. These detailed models can also be used to create didactic instructions for assembly and usage.
Producing these different representations of a concept requires specific skills in sketching, modeling, manufacturing and visual communication. For these reasons, professional studios often employ different experts to produce the different representations of the same concept, at the cost of extensive discussions and numerous iterations between the actors of this process. The complexity of the multi-disciplinary skills involved in the design process also hinders their adoption by laymen.
Existing solutions to facilitate design have focused on a subset of the representations used by designers. However, no solution considers all representations at once, for instance to directly convert a series of sketches into a set of physical prototypes. In addition, all existing methods assume that the concept is unique rather than ambiguous. As a result, rich information about the variability of the concept is lost during each conversion step.
We plan to facilitate design for professionals and laymen by addressing the following objectives:
- We want to assist designers in the exploration of the design space that captures the possible variations of a concept. By considering a concept as a distribution of shapes and functionalities rather than a single object, our goal is to help designers consider multiple design alternatives more quickly and effectively. Such a representation should also allow designers to preserve multiple alternatives along all steps of the design process rather than committing to a single solution early on and pay the price of this decision for all subsequent steps. We expect that preserving alternatives will facilitate communication with engineers, managers and clients, accelerate design iterations and even allow mass personalization by the end consumers.
- We want to support the various representations used by designers during concept development. While drawings and 3D models have received significant attention in past Computer Graphics research, we will also account for the various forms of rough physical prototypes made to evaluate the shape and functionality of a concept. Depending on the task at hand, our algorithms will either analyze these prototypes to generate a virtual concept, or assist the creation of these prototypes from a virtual model. We also want to develop methods capable of adapting to the different drawing and manufacturing techniques used to create sketches and prototypes. We envision design tools that conform to the habits of users rather than impose specific techniques to them.
- We want to make professional design techniques available to novices. Affordable software, hardware and online instructions are democratizing technology and design, allowing small businesses and individuals to compete with large companies. New manufacturing processes and online interfaces also allow customers to participate in the design of an object via mass personalization. However, similarly to what happened for desktop publishing thirty years ago, desktop manufacturing tools need to be simplified to account for the needs and skills of novice designers. We hope to support this trend by adapting the techniques of professionals and by automating the tasks that require significant expertise.
3.3 Graphics with Uncertainty and Heterogeneous Content
Our research is motivated by the observation that traditional CG algorithms have not been designed to account for uncertain data. For example, global illumination rendering assumes accurate virtual models of geometry, light and materials to simulate light transport. While these algorithms produce images of high realism, capturing effects such as shadows, reflections and interreflections, they are not applicable to the growing mass of uncertain data available nowadays.
The need to handle uncertainty in CG is timely and pressing, given the large number of heterogeneous sources of 3D content that have become available in recent years. These include data from cheap depth+image sensors (e.g., Kinect or the Tango), 3D reconstructions from image/video data, but also data from large 3D geometry databases, or casual 3D models created using simplified sketch-based modeling tools. Such alternate content has varying levels of uncertainty about the scene or objects being modeled. This includes uncertainty in geometry, but also in materials and/or lights – which are often not even available with such content. Since CG algorithms cannot be applied directly, visual effects artists spend hundreds of hours correcting inaccuracies and completing the captured data to make them useable in film and advertising.
|
|
Image-Based Rendering (IBR) techniques use input photographs and approximate 3D to produce new synthetic views.
Image-Based Rendering (IBR) techniques use input photographs and approximate 3D to produce new synthetic views.
We identify a major scientific bottleneck which is the need to treat heterogeneous content, i.e., containing both (mostly captured) uncertain and perfect, traditional content. Our goal is to provide solutions to this bottleneck, by explicitly and formally modeling uncertainty in CG, and to develop new algorithms that are capable of mixed rendering for this content.
We strive to develop methods in which heterogeneous – and often uncertain – data can be handled automatically in CG with a principled methodology. Our main focus is on rendering in CG, including dynamic scenes (video/animations) (see Fig. 2).
Given the above, we need to address the following challenges:
- Develop a theoretical model to handle uncertainty in computer graphics. We must define a new formalism that inherently incorporates uncertainty, and must be able to express traditional CG rendering, both physically accurate and approximate approaches. Most importantly, the new formulation must elegantly handle mixed rendering of perfect synthetic data and captured uncertain content. An important element of this goal is to incorporate cost in the choice of algorithm and the optimizations used to obtain results, e.g., preferring solutions which may be slightly less accurate, but cheaper in computation or memory.
- The development of rendering algorithms for heterogeneous content often requires preprocessing of image and video data, which sometimes also includes depth information. An example is the decomposition of images into intrinsic layers of reflectance and lighting, which is required to perform relighting. Such solutions are also useful as image-manipulation or computational photography techniques. The challenge will be to develop such “intermediate” algorithms for the uncertain and heterogeneous data we target.
- Develop efficient rendering algorithms for uncertain and heterogeneous content, reformulating rendering in a probabilistic setting where appropriate. Such methods should allow us to develop approximate rendering algorithms using our formulation in a well-grounded manner. The formalism should include probabilistic models of how the scene, the image and the data interact. These models should be data-driven, e.g., building on the abundance of online geometry and image databases, domain-driven, e.g., based on requirements of the rendering algorithms or perceptually guided, leading to plausible solutions based on limitations of perception.
3.4 Physical Simulation of Natural Phenomena
Our world emerged from the conjunction of natural phenomena at different scales, from the orogenesis of mountains to the evolution of ecosystems or the daily changes in weather conditions.
Understanding and modeling these phenomena is key to visually synthesizing our environments, reducing the uncertainty inherent to the capture of natural sceneries, and anticipating the impacts of natural hazards on our societies. For all these applications, the ability of a user to efficiently direct the simulation is preeminent, which provides us with two key constraints: first, the models should be fast to enable interactive interactions between the user and the simulation. Second, the models have to exhibit efficient control mechanisms.
The previous work on natural phenomena is as diverse as the number of scientific fields specialized in environmental and Earth sciences but with a main focus on predictability. In contrast, computer graphics has a long history of models focused on efficiency, robustness, and controllability, although originally explored for dynamic visual effects (smoke, explosions) and less so for natural phenomena that are more considered from a procedural or phenomenon-based perspective.
We benefit from computer graphics expertise in efficient and controllable physically-based simulations and extend it to natural phenomena. We explore new methods in machine learning and optimization, that enable us to enhance the efficiency of our models and reach a new space of forward and inverse control mechanisms. Coupling these models with physics provides guarantees on the quality of the results and a physical interpretation of the controls.
4 Application domains
Our research on design, simulation and computer graphics with heterogeneous data has the potential to change many different application domains. Such applications include:
Product design will be significantly accelerated and facilitated. Current industrial workflows separate 2D illustrators, 3D modelers and engineers who create physical prototypes, which results in a slow and complex process with frequent misunderstandings and corrective iterations between different people and different media. Our unified approach based on design principles could allow all processes to be done within a single framework, avoiding unnecessary iterations. This could significantly accelerate the design process (from months to weeks), result in much better communication between the different experts, or even create new types of experts who cross boundaries of disciplines today.
Mass customization will allow end customers to participate in the design of a product before buying it. In this context of “cloud-based design”, users of an e-commerce website will be provided with controls on the main variations of a product created by a professional designer. Intuitive modeling tools will also allow users to personalize the shape and appearance of the object while remaining within the bounds of the pre-defined design space.
Digital instructions for creating and repairing objects, in collaboration with other groups working in 3D fabrication, could have significant impact in sustainable development and allow anyone to be a creator of things, not just consumers, the motto of the makers movement.
Gaming experience individualization is an important emerging trend; using our results players will also be able to integrate personal objects or environments (e.g., their homes, neighborhoods) into any realistic 3D game. The success of creative games where the player constructs their world illustrates the potential of such solutions. This approach also applies to serious gaming, with applications in medicine, education/learning, training etc. Such interactive experiences with high-quality images of heterogeneous 3D content will be also applicable to archeology (e.g., realistic presentation of different reconstruction hypotheses), urban planning and renovation where new elements can be realistically used with captured imagery.
Virtual training, which today is restricted to pre-defined virtual environment(s) that are expensive and hard to create; with our solutions on-site data can be seamlessly and realistically used together with the actual virtual training environment. With our results, any real site can be captured, and the synthetic elements for the interventions rendered with high levels of realism, thus greatly enhancing the quality of the training experience.
Earth and environmental sciences use simulations to understand and characterize natural processes. One of the common scientific methodologies requires testing several simulations with different sets of parameters to observe emergent behavior or to match observed data. Our fast simulation models accelerate this workflow, while our focus on control gives new tools to efficiently reduce the misfit between simulations and observations.
Natural hazard prevention is becoming ever more critical now that several climatic tipping points are crossed or about to be. Fast and controllable simulations of natural phenomena could allow public authorities to quickly assert different scenarios on the verge of imminent hazards, informing them of the probable impacts of their decisions.
Another interesting novel use of heterogeneous graphics could be for news reports. Using our interactive tool, a news reporter can take on-site footage, and combine it with 3D mapping data. The reporter can design the 3D presentation allowing the reader to zoom from a map or satellite imagery and better situate the geographic location of a news event. Subsequently, the reader will be able to zoom into a pre-existing street-level 3D online map to see the newly added footage presented in a highly realistic manner. A key aspect of these presentation is the ability of the reader to interact with the scene and the data, while maintaining a fully realistic and immersive experience. The realism of the presentation and the interactivity will greatly enhance the readers experience and improve comprehension of the news. The same advantages apply to enhanced personal photography/videography, resulting in much more engaging and lively memories. Such interactive experiences with high-quality images of heterogeneous 3D content will be also applicable to archeology (e.g., realistic presentation of different reconstruction hypotheses), urban planning and renovation where new elements can be realistically used with captured imagery.
Other applications may include scientific domains which use photogrammetric data (captured with various 3D scanners), such as geophysics and seismology. Note however that our goal is not to produce 3D data suitable for numerical simulations; our approaches can help in combining captured data with presentations and visualization of scientific information.
5 Social and environmental responsibility
5.1 Footprint of research activities
Deep learning algorithms use a significant amount of computing resources. We are attentive to this issue and plan to implement a more detailed policy for monitoring overall resource usage.
5.2 Impact of research results
G. Cordonnier collaborates with geologists and glaciologists on various projects, developing computationally efficient models that can have direct impact in climate related research. A. Bousseau regularly collaborates with designers; their needs serve as an inspiration for some of his research projects, including the developement of innovative digital tools for circular design. Finally, the work in FUNGRAPH (G. Drettakis) has advanced research in visualization for reconstruction of real scenes. The recent 3D Gaussian Splatting (3DGS) 36 work has resulted in extensive technology transfer with many commercial licenses of the code already completed. These involve diverse industrial domains, including e-commerce, casual capture for 3D reconstruction, film and television production, virtual and extended reality, real-estate visualization and others. The 3DGS technology significantly reduces the computation time and thus computational resources required compared to the previous state of the art in 3D reconstruction/novel view synthesis that was the Neural Radiance Fields method (for typical scenes, 40 min instead of 48 hours of GPU time).
6 Highlights of the year
6.1 Awards
- Guillaume Cordonnier received the Young Investigator Award from the conference Shape Modeling international.
- George Drettakis received the ACM SIGGRAPH Computer Graphics Achievement award, the first for a researcher from a lab in Europe, and was inducted to the ACM SIGGRAPH Academy.
- An Inria Startup Studio project (OnTheFly) was accepted and started in October, led by Andreas Meuleman (full-time, now at Inria Rennes), and with the participation of George Drettakis.
7 Latest software developments, platforms, open data
7.1 Latest software developments
7.1.1 3DGaussianSplats
-
Name:
3D Gaussian Splatting for Real-Time Radiance Field Rendering
-
Keywords:
3D, View synthesis, Graphics
-
Scientific Description:
Implementation of the method 3D Gaussian Splatting for Real-Time Radiance Field Rendering, see https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/
-
Functional Description:
3D Gaussian Splatting is a method that achieves real-time rendering of captured scenes with quality that equals the previous method with the best quality, while only requiring optimization times competitive with the fastest previous methods
3D Gaussian Splatting represents 3D scenes with 3D Gaussians that preserve desirable properties of continuous volumetric radiance fields for scene optimization while avoiding unnecessary computation in empty space. The method performs interleaved optimization/density control of the 3D Gaussians, notably optimizing anisotropic covariance to achieve an accurate representation of the scene. We provide a fast visibility-aware rendering algorithm that supports anisotropic splatting and both accelerates training and allows realtime rendering.
We have provided several updates to the software this year, most importantly integrating new features for better quality and speed of optimization.
- URL:
-
Contact:
George Drettakis
-
Participant:
3 anonymous participants
7.1.2 3DLayers
-
Name:
VR painting system with layers
-
Keywords:
Virtual reality, 3D, Painting
-
Functional Description:
This is the source code for the prototype implementation of the research paper: "3D-Layers: Bringing Layer-Based Color Editing to VR Painting", Emilie Yu, Fanny Chevalier, Karan Singh and Adrien Bousseau, ACM Transactions on Graphics (SIGGRAPH) - 2024
This is a Unity project that implements a simple VR application compatible with Quest 2/3/Pro headsets. The project features:
- A VR app with basic 3D painting features (painting tube strokes, stroke deletion and transformation, color palette, undo/redo). - A UI in the VR app to create, paint in, and edit shape and appearance layers, as described in the 3DLayers paper. We have a basic menu UI for users to visualize and navigate in the layer hierarchy. - A basic in-Unity visualizer for paintings created with our system. It enables users to view and render still frames or simple camera path animations. We used it to create all results in the paper/video.
- URL:
-
Contact:
Emilie Yu
7.1.3 VideoDoodles
-
Name:
VideoDoodles: Hand-Drawn Animations on Videos with Scene-Aware Canvases
-
Keywords:
3D web, 3D, 2D animation, 3D animation, Visual tracking
-
Scientific Description:
Implementation for Siggraph 2023 paper VideoDoodles: Hand-Drawn Animations on Videos with Scene-Aware Canvases
-
Functional Description:
We present an interactive system to ease the creation of so-called video doodles – videos on which artists insert hand-drawn animations for entertainment or educational purposes. Video doodles are challenging to create because to be convincing, the inserted drawings must appear as if they were part of the captured scene. In particular, the drawings should undergo tracking, perspective deformations and occlusions as they move with respect to the camera and to other objects in the scene – visual effects that are difficult to reproduce with existing 2D video editing software. Our system supports these effects by relying on planar canvases that users position in a 3D scene reconstructed from the video. Furthermore, we present a custom tracking algorithm that allows users to anchor canvases to static or dynamic objects in the scene, such that the canvases move and rotate to follow the position and direction of these objects.
Our system is composed of the following elements: * A preprocessing library to convert depth, camera and motion data into data formats compatible with our system * A collection of Python scripts that can be used either offline to execute 3D point tracking (with the possibility to specify 1 to N keyframes) , or as a backend server to our interactive frontend UI * A web UI to author video doodles. It features a renderer capable of displaying the composited video doodles , an editing interface to keyframe canvases , a sketching interface to create frame-by-frame animations on canvases , basic export capabilities.
- URL:
-
Contact:
Emilie Yu
7.1.4 pySBM
-
Keywords:
3D modeling, Vector-based drawing
-
Scientific Description:
This project is the official implementation of our paper Symmetry-driven 3D Reconstruction from Concept Sketches, published at SIGGRAPH 2022 https://ns.inria.fr/d3/SymmetrySketch/
The software includes a Blender plugin developed in the context of our ERC PoC grant DLift.
-
Functional Description:
This is a sketch-based modeling library. It proposes an interface to input a vector sketch, process it and reconstruct it in 3D.
-
Release Contributions:
This release includes a Blender add-on that communicates with the 3D reconstruction code using a client-server architecture. This add-on also provides a user interface for inspecting the reconstruction and making corrections.
- URL:
- Publication:
-
Contact:
Adrien Bousseau
-
Participants:
Jiayi Wei, Felix Hahnlein, Yulia Gryaditskaya, Alla Sheffer, Adrien Bousseau
-
Partner:
University of British Columbia
7.1.5 pyLowStroke
-
Keywords:
Vector-based drawing, 3D modeling
-
Scientific Description:
This library contains several functionalities to process line drawings, including loading and saving in svg format, detecting vanishing points, calibrating a camera, detecting intersections. It has been used to develop our reconstruction algorithm Symmetry-driven 3D Reconstruction from Concept Sketches published at SIGGRAPH 2022 https://ns.inria.fr/d3/SymmetrySketch/
-
Functional Description:
This is a library for low-level processing of freehand sketches. Example applications include reading and writing vector drawings, removing hooks, classifying lines into straight lines and curves and the calibration of a perspective camera model.
The software is currently part of an ERC PoC development cycle for the creation of a Blender plugin.
- URL:
-
Contact:
Adrien Bousseau
-
Participants:
Felix Hahnlein, Yulia Gryaditskaya, Bastien Wailly, Adrien Bousseau
7.1.6 Fastflow
-
Name:
GPU Acceleration of Flow and Depression Routing for Landscape Simulation
-
Keywords:
Flow routing, Landscape, GPU
-
Functional Description:
Library for the fast computation of flow and depression routing on the GPU for numerical simulations of landscapes in hydrology and geomorphology. This code enables the computation of flow-related properties such as the discharge over large Digital Elevation Models (flow routing). It solves the problem that local minima in topography interrupt the flow path (depression routing). The code is optimized for the GPU, resulting in fast execution time for large domains.
- URL:
-
Contact:
Guillaume Cordonnier
7.1.7 graphdecoviewer
-
Name:
A flexible tool for viewing graphics/novel view synthesis content
-
Keywords:
3D visualisation, Graphics
-
Functional Description:
This is a python library that allows to very rapidly prototype an interactive viewer for novel view synthesis algorithms such as 3D Gaussian Splatting, but also for traditional graphics. Several important functionalities are provided, such as remote viewing of a training process (possibly on a cluster), local viewing on a workstation with a fast GPU, as well as typical functionalities required for such research projects.
- URL:
-
Contact:
George Drettakis
7.1.8 H3DGS
-
Name:
Hierarchical 3D Gaussian Splatting
-
Keywords:
3D modeling, 3D rendering, Differentiable Rendering
-
Scientific Description:
Implementation of the SIGGRAPH 2024 paper A Hierarchical 3D Gaussian Representation for Real-Time Rendering of Very Large Datasets, project page https://repo-sam.inria.fr/fungraph/hierarchical-3d-gaussians/
-
Functional Description:
Implementation of the SIGGRAPH 2024 paper A Hierarchical 3D Gaussian Representation for Real-Time Rendering of Very Large Datasets, project page https://repo-sam.inria.fr/fungraph/hierarchical-3d-gaussians/
- URL:
-
Contact:
George Drettakis
-
Participant:
6 anonymous participants
-
Partner:
Technische Universität Wien
7.1.9 DiffRelightGS
-
Name:
A Diffusion Approach to Radiance Field Relighting using Multi-Illumination Synthesis
-
Keywords:
3D, 3D reconstruction, 3D rendering, Artificial intelligence, Machine learning
-
Functional Description:
A method to create relightable 3D radiance fields from single-illumination data by exploiting priors extracted from 2D image diffusion models.
- URL:
- Publication:
-
Contact:
George Drettakis
-
Participant:
5 anonymous participants
-
Partner:
Université Laval
7.1.10 On-The-Fly
-
Name:
On-the-fly Reconstruction for Large-Scale Novel View Synthesis from Unposed Images
-
Keywords:
3D, 3D rendering, Differentiable Rendering, 3D reconstruction
-
Functional Description:
This codebase contains the implementation of the paper https://repo-sam.inria.fr/nerphys/on-the-fly-nvs/. The code provides python/pytorch online training for joint pose estimation and 3D Gaussian Splatting reconstruction as well as visualization tools both for training and post-training visualization, using the graphdecoviewer codebase. The method allows online training with on-the-fly feedback, including a prototype version for phone-based capture.
- URL:
-
Contact:
George Drettakis
7.1.11 Splat and Replace
-
Name:
Splat and Replace: 3D Reconstruction with Repetitive Elements
-
Keyword:
3D reconstruction
-
Functional Description:
Official Implementation of "Splat and Replace: 3D Reconstruction with Repetitive Elements" SIGGRAPH Conference Papers 2025, which can be found here https://repo-sam.inria.fr/nerphys/splat-and-replace/. The method allows improved reconstruction of 3D Gaussian Splatting scenes when the capture contains repetitive elements. The code base contains python/pytorch code for training and C++ code for viewing.
- URL:
- Publication:
-
Contact:
George Drettakis
-
Participant:
6 anonymous participants
-
Partner:
Adobe
7.1.12 EditGaussReflection
-
Name:
Editable Physically-based Reflections in Raytraced Gaussian Radiance Fields
-
Keywords:
Graphics, 3D reconstruction
-
Functional Description:
Implementation of the paper https://repo-sam.inria.fr/nerphys/editable-gaussian-reflections/. The codebase contains pytorch/python and cuda/Optix code for ray-traced Gaussian reflections. The codebase includes code for training and for visualization using the graphdeco viewer. The method also includes code for processing of synthetic scenes where ground truth maps for diffuse and non-diffuse layers are provided for input images, and code that uses deep learning to extract the layers for real images.
- URL:
-
Contact:
George Drettakis
7.1.13 ContextTextureGS
-
Name:
Content-Aware Texturing for Gaussian Splatting
-
Keywords:
Graphics, 3D reconstruction
-
Functional Description:
This codebase is an implementation of paper https://repo-sam.inria.fr/nerphys/gs-texturing/. The repository provides python/pytorch code for training and visualization as well as a C++ viewer based on SIBR. The method allows context aware texturing of 3D Gaussians, demonstrating superior performance to other 3DGS texturing methods.
- URL:
-
Contact:
George Drettakis
7.1.14 sibr-core
-
Name:
System for Image-Based Rendering
-
Keyword:
Graphics
-
Scientific Description:
Core functionality to support Image-Based Rendering research. The core provides basic support for camera calibration, multi-view stereo meshes and basic image-based rendering functionality. Separate dependent repositories interface with the core for each research project. This library is an evolution of the previous SIBR software, but now is much more modular.
sibr-core has been released as open source software, as well as the code for several of our research papers, as well as papers from other authors for comparisons and benchmark purposes.
The corresponding gitlab is: https://gitlab.inria.fr/sibr/sibr_core
The full documentation is at: https://sibr.gitlabpages.inria.fr
This year several improvements were added as part of the 3D Gaussian Splatting support.
-
Functional Description:
sibr-core is a framework containing libraries and tools used internally for research projects based on Image-Base Rendering. It includes both preprocessing tools (computing data used for rendering) and rendering utilities and serves as the basis for many research projects in the group.
-
Contact:
George Drettakis
7.1.15 CAD2Sketch
-
Keywords:
Non-photorealistic rendering, CAD
-
Scientific Description:
This is a program to render CAD models as design drawings. The method generates lines for each part of the CAD model, and selects a subset of these lines to produce a drawing that is well constructed with as little clutter as possible. The resulting drawings look like drawings created by product designers.
This code is the result of the following research project: https://ns.inria.fr/d3/cad2sketch/
-
Functional Description:
This is a program to render CAD models as design drawings. The method generates lines for each part of the CAD model, and selects a subset of these lines to produce a drawing that is well constructed with as little clutter as possible. The resulting drawings look like drawings created by product designers.
- URL:
-
Contact:
Adrien Bousseau
-
Participants:
Adrien Bousseau, Felix Hahnlein
7.2 Open data
OpenSketch
-
Contributors:
Adrien Bousseau
-
Description:
A dataset of around 400 design drawings
-
Project link:
https://repo-sam.inria.fr/d3/OpenSketch/
-
Publications:
OpenSketch: A Richly-Annotated Dataset of Product Design Sketches, SIGGRAPH Asia 2019 https://ns.inria.fr/d3/OpenSketch/
-
Contact:
Adrien Bousseau
8 New results
8.1 Computer-Assisted Design with Heterogeneous Representations
8.1.1 Rags2Riches: Computational Garment Reuse
Participants: Anran Qi, Adrien Bousseau.
We present the first algorithm to automatically compute sewing patterns for upcycling existing garments into new designs (Fig. 3). Our algorithm takes as input two garment designs along with their corresponding sewing patterns and determines how to cut one of them to match the other by following garment reuse principles. Specifically, our algorithm favors the reuse of seams and hems present in the existing garment, thereby preserving the embedded value of these structural components and simplifying the fabrication of the new garment (Fig. 3). Finding optimal reused patterns is computationally challenging because it involves both discrete and continuous quantities. Discrete decisions include the choice of existing panels to cut from and the choice of seams and hems to reuse. Continuous variables include the precise placement of the new panels along seams and hems, as well as potential deformations of these panels to maximize reuse. Our key idea for making this optimization tractable is to quantize the shape of garment panels. This allows us to frame the search for an optimal reused pattern as a discrete assignment problem, which we solve efficiently with an ILP solver. We showcase our proposed pipeline on several reuse examples, including comparisons with reused patterns crafted by a professional garment designer. Additionally, we manufacture a physical reused garment to demonstrate the practical effectiveness of our approach.
Virtual and real examples of reusing a pullover to create a dog coat.
This work is a collaboration with Nico Pietroni from University of Technology Sydney and Maria Korosteleva and Olga Sorkine-Hornung from ETH Zurich. It was published in ACM SIGGRAPH Conference Proceedings 27.
8.1.2 Sketch2Data: Recovering Data from Hand-Drawn Infographics
Participants: Anran Qi, Adrien Bousseau.
Data collection and visualization have traditionally been seen as activities reserved for experts. However, by drawing simple geometric figures – known as glyphs – anyone can visually record their own data. Still, the resulting hand-drawn infographics do not provide direct access to the underlying data, hindering digital editing of both the glyphs and their values. We introduce a method to recover data values from glyph-based hand-drawn infographics (Fig. 4). Given a visualization in a bitmap format and a user-defined parametric template of its glyphs, we leverage deep neural networks to detect and localize the visualization glyphs, and estimate the data values they represent. We also provide a user interface to review and correct these estimates, informed by a measure of uncertainty of the neural network predictions. Our reverse-engineering procedure effectively disentangles the depicted data from its visual representation, enabling various visualization authoring applications, such as visualizing new data values or experimenting with alternative visualizations of the same data.
Illustration of a hand-drawn infographics and the editing features enabled by recovering its data.
This work is a collaboration with Ariel Shamir from Reichman University and Theophanis Tsandilas from the ExSitu group in the context of the ANR project GLACIS. It was published in the Computer and Graphics journal 21, and presented in the ACM/EG Expressive Symposium.
8.1.3 GarmentImage: Raster Encoding of Garment Sewing Patterns with Diverse Topologies
Participants: Anran Qi.
Garment sewing patterns are the design language behind clothing, yet their current vector-based digital representations were not built with machine learning in mind. Vector-based representation encodes a sewing pattern as a discrete set of panels, each defined as a sequence of lines and curves, stitching information between panels and the placement of each panel around a body. However, this representation causes two major challenges for neural networks: discontinuity in latent space between patterns with different topologies and limited generalization to garments with unseen topologies in the training data. In this work, we introduce GarmentImage, a unified raster-based sewing pattern representation that addresses these challenges (Fig. 5). GarmentImage encodes a garment sewing pattern’s geometry, topology and placement into multi-channel regular grids. Machine learning models trained on GarmentImage achieve seamless transitions between patterns with different topologies and show better generalization capabilities compared to models trained on vector-based representation. We demonstrate the effectiveness of GarmentImage across three applications: pattern exploration in latent space, text-based pattern editing, and image-to-pattern prediction. The results show that GarmentImage achieves superior performance on these applications using only simple convolutional networks.
Illustration of how a GarmentImage represents a garment and enable various applications.
This work is a collaboration with Yuki Tatsukawa, I-Chao Shen and Takeo Igarashi from The University of Tokyo and was initiated while Anran Qi was a postdoc in their group. It was published in ACM SIGGRAPH Conference Proceedings 29.
8.1.4 FontCraft: Multimodal Font Design Using Interactive Bayesian Optimization
Participants: Anran Qi.
Creating new fonts requires a lot of human effort and professional typographic knowledge. Despite the rapid advancements of automatic font generation models, existing methods require users to prepare pre-designed characters with target styles using font-editing software, which poses a problem for non-expert users. To address this limitation, we propose FontCraft, a system that enables font generation without relying on pre-designed characters (Fig. 6). Our approach integrates the exploration of a font-style latent space with human-in-the-loop preferential Bayesian optimization and multimodal references, facilitating efficient exploration and enhancing user control. Moreover, FontCraft allows users to revisit previous designs, retracting their earlier choices in the preferential Bayesian optimization process. Once users finish editing the style of a selected character, they can propagate it to the remaining characters and further refine them as needed. The system then generates a complete outline font in OpenType format. We evaluated the effectiveness of FontCraft through a user study comparing it to a baseline interface. Results from both quantitative and qualitative evaluations demonstrate that FontCraft enables non-expert users to design fonts efficiently.
Illustration of how users create a font in FontCraft.
This work is a collaboration with Yuki Tatsukawa, I-Chao Shen, Yuki Koyama, and Takeo Igarashi from The University of Tokyo, Mustafa Doga Dogan from Adobe Research, and Ariel Shamir from Reichman University. The project was initiated while Anran Qi was a postdoc at The University of Tokyo. It was published at the ACM conference on Human Factors in Computing Systems (CHI) 29.
8.1.5 Shape Approximation by Surface Reuse
Participants: Berend Baas, Adrien Bousseau.
The manufacturing industry faces an urgent need to transition from the linear “make-take-use-dispose” production model towards more sustainable circular models that retain resources in the production chain. Motivated by this need, we introduce the new problem of approximating 3D surfaces by reusing panels from other surfaces. We present an algorithm that takes as input one or several existing shapes and relies on partial shape registration to identify a small set of simple panels that, once cut from the existing shapes and transformed rigidly, approximate a target shape within a user-defined distance threshold. As a proof of concept, we demonstrate our algorithm in the context of rapid prototyping, where we harvest curved panels from plastic bottles and assemble them with custom connectors to fabricate medium-size freeform structures. See Fig. 7.
Example of reusing surface panels from a plastic bottle to approximate a saddle shape.
This work is a collaboration with David Bommes from University of Bern, Switzerland and was published SGP 16.
8.1.6 Computational Material Reuse with fabrication Constraints
Participants: Berend Baas, Adrien Bousseau, Marzia Riso.
Material reuse focuses on the design and fabrication of objects from reclaimed parts of old stock material. Due to the potential impacts on the environment and increasing the value of waste materials, recent research has focussed on the domain of computational material reuse: The development of computational design tools that aid in identifying and optimizing the reuse of source elements for new targets.
In this domain, several topics have been explored in recent years: Ranging from reuse of fabric patches or one-dimensional beam/truss structures, to more recently the reuse of two-dimensional elements.
Practical reuse scenarios however generally come with many fabrication constraints: One such example is surface continuity, where we aim to reuse elements while minimizing gaps (which have to be filled with virgin material). We explore how to combine material reuse and practical fabrication constraints in a single optimization framework, with a focus on the reuse of rigid elements.
8.1.7 From blades to tracks: a case study in structural reuse of curved surfaces for circular design
Participants: Marzia Riso, Adrien Bousseau.
We explore the fabrication of curved surfaces by reusing panels extracted from decommissioned wind turbine blades, using cycling pumptracks as a case study. We first analyze real-world prototypes of pumptrack modules that we manufactured to evaluate the practicality of this reuse scenario and to define the boundary conditions for harvesting blade panels and assembling a track. We then propose an algorithm to optimize the segmentation of a wind turbine blade into quadrilateral panels whose sides fall within a small set of compatible boundaries. These panels form a library of modules that designers can connect side by side to create pumptracks of various lengths and curvatures. Together, these contributions provide a proof-of-concept of how computer-aided design and manufacturing can support circular design through the reuse of curved surfaces. See Fig. 8.
From blades to tracks
From blades to tracks
This work is a collaboration with Jesse Pupping, Mariana Popescu, Jelle Joustra from TU Delft - Industrial Design Engineering. It was published in the proceedings of the 10th ACM Symposium on Computational Fabrication 26, where it was presented.
8.1.8 Interactive Optimization of Scaffolded Procedural Patterns
Participants: Marzia Riso.
A procedural program is the representation of a family of assets that share the same structural or semantic properties, whose final appearance is determined by different parameter assignments. Identifying the parameter values that define a desired asset is usually a time-consuming operation, since it requires manually tuning parameters separately and in a non-intuitive manner. In the domain of procedural patterns, recent works focused on estimating parameter values to match a target render or sketch, using parameter optimization or inference via neural networks. However, these approaches are neither fast enough for interactive design nor precise enough to give direct control. We propose an interactive method for procedural parameter estimation based on the idea of scaffolded procedural patterns. A scaffolded procedural pattern is a sequence of procedural programs that model a pattern in a coarse-to-fine manner, in which the desired pattern appearance is reached step-by-step by inheriting previously optimized parameters. Through scaffolding, patterns are more straightforward to sketch for users and easier to optimize for most algorithms. In our implementation, patterns are represented as procedural signed distance functions whose parameters are estimated with a gradient-free optimization method that runs in real-time on the GPU. We show that scaffolded patterns can be created with a node-based interface familiar to artists. We validate our approach by creating and interactively editing several scaffolded patterns. We show the effectiveness of scaffolding through a user study, where scaffolding enhances both the output quality and the editing experience with respect to approaches that optimize the procedural parameters all at once. We also perform a comparison with previous strategies and provide several recordings of real-time editing sessions in the accompanying materials. See Fig. 9.
Scaffolded Optimization
This work is a collaboration with Davide Sforza, Filppo Muzzini, Nicola Capodieci and Fabio Pellacini from Sapienza University of Rome and University of Modena and Reggio Emilia. It was published in ACM SIGGRAPH 2025 Conference Track 28.
8.1.9 Sketched CAD Sequences Dataset
Participants: Marzia Riso, Loïc Gaillard, Adrien Bousseau.
Existing datasets of sketched Computer-Aided Design (CAD) models have been collected by asking designers to reproduce only the final model image, without considering the full construction sequence, and are typically limited in scale. Motivated by the observation that sketching CAD models naturally follows a coarse-to-fine process, we aim to collect a new dataset of sketches that includes a larger and more diverse set of shapes while providing designers with the complete construction sequence as a reference. Target models are selected from the Fusion 360 Reconstruction Dataset and enriched with additional details through operations such as chamfers and fillets. To facilitate data collection, we developed a web-based sketching interface implemented using React and Node.js, which is connected to a backend database for storing sketches and associated metadata.
8.1.10 CADrawer : Autoregressive CAD Generation from 3D Sketches
Participants: Henro Kriel, Adrien Bousseau.
We present CADrawer, a system that translates 3D sketches into CAD programs using an autoregressive approach, leveraging construction lines as a rich source of information for recovering intermediate CAD operations. This work is a collaboration with Gilda Manfredi from the University of Basilicata, Yuanbo Li and Daniel Ritchie from Brown University. We submitted this work to Eurographics.
8.1.11 ImplicitDrag: Drag-Based Editing of Differentiable Implicits
Participants: Henro Kriel, Marzia Riso, Adrien Bousseau.
We present a method to perform drag-based editing of parametric shape models that represent classes of shapes as implicit surfaces. This work is a collaboration with Siddhartha Chaudhuri from Adobe, Aamen Muharram and Daniel Ritchie from Brown University.
8.1.12 DancingBox: A Lightweight MoCap System for Character Animation from Physical Proxies
Participants: Adrien Bousseau.
Creating compelling 3D character animations typically requires either expert use of professional software or expensive motion capture systems operated by skilled actors. We are working on a lightweight, vision-based system that makes motion capture accessible to novices by reimagining the process as digital puppetry. Instead of tracking precise human motions, our method captures the approximate movements of everyday objects manipulated by users with a single webcam. These coarse proxy motions are then refined into realistic character animations by conditioning a generative motion model on bounding-box representations, enriched with human motion priors learned from large-scale datasets. To overcome the lack of paired proxy–animation data, we synthesize training pairs by converting existing motion capture sequences into proxy representations. Users of our system can create character animation using diverse proxies, from plush toys to bananas, lowering the barrier to entry for novice animators.
This ongoing work is a collaboration with Haocheng Yuan and Changjian Li from The University of Edinburgh and Hao Pan from Tsinghua University. This project was initiated during Adrien Bousseau's visit at The University of Edinburgh in June-July 2025.
8.2 Graphics with Uncertainty and Heterogeneous Content
8.2.1 On-the-fly Reconstruction for Large-Scale Novel View Synthesis from Unposed Images.
Participants: Andreas Meuleman, Ishaan Shah, Alexandre Lanvin, George Drettakis.
Radiance field methods such as 3D Gaussian Splatting (3DGS) allow easy reconstruction from photos, enabling free-viewpoint navigation. Nonetheless, pose estimation using Structure from Motion and 3DGS optimization can still each take between minutes and hours of computation after capture is complete. Simultaneous Localization and Mapping (SLAM) methods combined with 3DGS are fast but struggle with wide camera baselines and large scenes. We present an on-the-fly method to produce camera poses and a trained 3DGS immediately after capture. Our method can handle dense and wide-baseline captures of ordered photo sequences and large-scale scenes (Fig. 10). To do this, we first introduce fast initial pose estimation, exploiting learned features and a GPU-friendly mini bundle adjustment. We then introduce direct sampling of Gaussian primitive positions and shapes, incrementally spawning primitives where required, significantly accelerating training. These two efficient steps allow fast and robust joint optimization of poses and Gaussian primitives. Our incremental approach handles large-scale scenes by introducing scalable radiance field construction, progressively clustering 3DGS primitives, storing them in anchors, and offloading them from the GPU. Clustered primitives are progressively merged, keeping the required scale of 3DGS at any viewpoint. We evaluate our solution on a variety of datasets and show that it can provide on-the-fly processing of all the capture scenarios and scene sizes we target. At the same time, our method remains competitive – in speed, image quality, or both – with other methods that only handle specific capture styles or scene sizes.
Illustration of the on-the-fly 3DGS method for joint pose and radiance field estimation
This work was in collaboration with B. Kerbl from TU Wien, was published in ACM Transactions on Graphics and presented at ACM SIGGRAPH 20.
8.2.2 Splat and Replace: 3D Reconstruction with Repetitive Elements.
Participants: Nicolas Violante, Andreas Meuleman, Alban Gauthier, George Drettakis.
We leverage repetitive elements in 3D scenes to improve novel view synthesis. Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have greatly improved novel view synthesis but renderings of unseen and occluded parts remain low-quality if the training views are not exhaustive enough. Our key observation is that our environment is often full of repetitive elements. We propose to leverage those repetitions to improve the reconstruction of low-quality parts of the scene due to poor coverage and occlusions. We propose a method that segments each repeated instance in a 3DGS reconstruction, registers them together, and allows information to be shared among instances (Fig. 11). Our method improves the geometry while also accounting for appearance variations across instances. We demonstrate our method on a variety of synthetic and real scenes with typical repetitive elements, leading to a substantial improvement in the quality of novel view synthesis.
Illustration of our method to exploit repetitive elements for 3DGS reconstruction
This work is a collaboration with Fredo Durand from MIT and Thibault Groueix from Adobe Research. It was published in SIGGRAPH 2025 Conference Track 31.
8.2.3 Editable Gaussian Reflections
Participants: Yohan Poirier-Ginter, Jeffrey Hu, George Drettakis.
Radiance fields such as 3D Gaussian Splatting allow real-time rendering of scenes captured from photos. They also reconstruct most specular reflections with high visual quality, but typically model them with “fake” reflected geometry, using primitives behind the reflector. Our goal is to correctly reconstruct the reflector and the reflected objects, such as to make specular reflections editable. We present a proof of concept that exploits promising learning-based methods to extract diffuse and specular buffers from photos, as well as geometry and bidirectional Reflectance Distribution Function (BRDF) buffers. Our method builds on three key components. First, by using diffuse and specular buffers of input training views, we optimize a diffuse version of the scene and use path tracing to efficiently generate physically based specular reflections. Second, we present a specialized training method that allows this process to converge. Finally, we present a fast ray tracing algorithm for 3D Gaussian primitives that enables efficient multi-bounce reflections. Our method reconstructs reflectors and reflected objects—including those not seen in the input images — in a unique scene representation. Our solution allows real-time, consistent editing of captured scenes with specular reflections, including multi-bounce effects, changing roughness, and more. We mainly show results using ground truth buffers from synthetic scenes, and also preliminary results in real scenes with currently imperfect learning-based buffers. See Fig. 12.
Illustration for Editable Physically-based Reflections in Raytraced Gaussian Radiance Fields
This work is in collaboration with Jean-François Lalonde Université Laval, Quebec, Canada, in the context of the co-tutelle for Yohan Poirier-Ginter. This work was published at SIGGRAPH Asia 2025 as a Conference Paper 25.
8.2.4 An evaluation of Spatially-Varying BRDF (SVBRDF) Prediction from Generative Image Models for Appearance Modeling of 3D Scenes
Participants: Alban Gauthier, Alexandre Lanvin, Adrien Bousseau, George Drettakis.
Digital content creation is experiencing a profound change with the advent of deep generative models. For texturing, conditional image generators now allow the synthesis of realistic RGB images of a 3D scene that aligns with the geometry of that scene. For appearance modeling, SVBRDF prediction networks recover material parameters from RGB images. Combining these technologies allows us to quickly generate SVBRDF maps for multiple views of a 3D scene, which can be merged to form an SVBRDF texture atlas of that scene (see Fig. 13). In this work, we analyze the challenges and opportunities for SVBRDF prediction in the context of such a fast appearance modeling pipeline. On the one hand, single-view SVBRDF predictions might suffer from multiview incoherence and yield inconsistent texture atlases. On the other hand, generated RGB images, and the different modalities on which they are conditioned, can provide additional information for SVBRDF estimation compared to photographs. We compare neural architectures and conditions to identify designs that achieve high accuracy and coherence. We find that, surprisingly, a standard UNet is competitive with more complex designs.
An AI method to extract materials
This work is a collaboration with Valentin Deschaintre from Adobe Research and Fredo Durand from MIT. It was published as a Symposium-Track paper at EGSR 2025, and presented at the in-person conference in July 2025 23.
8.2.5 Content-Aware Texturing for Gaussian Splatting
Participants: Panagiotis Papantonakis, George Drettakis.
Gaussian Splatting has become the method of choice for 3D reconstruction and real-time rendering of captured real scenes. However, fine appearance details need to be represented as a large number of small Gaussian primitives, which can be wasteful when geometry and appearance exhibit different frequency characteristics.
Inspired by the long tradition of texture mapping, we propose to use texture to represent detailed appearance where possible. Our main focus is to incorporate per-primitive texture maps that adapt to the scene in a principled manner during Gaussian Splatting optimization. We do this by proposing a new appearance representation for 2D Gaussian primitives with textures where the size of a texel is bounded by the image sampling frequency and adapted to the content of the input images. We achieve this by adaptively upscaling or downscaling the texture resolution during optimization. In addition, our approach enables control of the number of primitives during optimization based on texture resolution. We show that our approach performs favorably in image quality and total number of parameters used compared to alternative solutions for textured Gaussian primitives. See Fig. 14.
Illustration of Context Aware Texturing for Gaussian Splatting.
This work is a collaboration with Georgios Kopanas from Google DeepMind and Frédo Durand from MIT CSAIL and was presented at the 2025 Eurographics Symposium on Rendering (EGSR) 24.
8.2.6 Does 3D Gaussian Splatting Need Accurate Volumetric Rendering?
Participants: George Drettakis.
Since its introduction, 3D Gaussian Splatting (3DGS) has become an important reference method for learning 3D representations of a captured scene, allowing real-time novel-view synthesis with high visual quality and fast training times. Neural Radiance Fields (NeRFs), which preceded 3DGS, are based on a principled ray-marching approach for volumetric rendering. In contrast, while sharing a similar image formation model with NeRF, 3DGS uses a hybrid rendering solution that builds on the strengths of volume rendering and primitive rasterization. A crucial benefit of 3DGS is its performance, achieved through a set of approximations, in many cases with respect to volumetric rendering theory. A naturally arising question is whether replacing these approximations with more principled volumetric rendering solutions can improve the quality of 3DGS. In this project, we present an in-depth analysis of the various approximations and assumptions used by the original 3DGS solution. We demonstrate that, while more accurate volumetric rendering can help for low numbers of primitives, the power of efficient optimization and the large number of Gaussians allows 3DGS to outperform volumetric rendering despite its approximations (see Fig. 15)
Image illustrating that approximate volumetric rendering only matters for 3D Gaussian Splatting for low primitive counts.
This project is a collaboration with Adam Celarek, Michael Wimmer and Bernhard Kerbl from the TU Wien (Autria) and Georgios Kopanas (Google UK), and was published at Eurographics 2025 and Computer Graphics Forum 17.
8.2.7 MILo: Mesh-In-the-Loop Gaussian Splatting for Detailed and Efficient Surface Reconstruction
Participants: George Drettakis.
While recent advances in Gaussian Splatting have enabled fast reconstruction of high-quality 3D scenes from images, extracting accurate surface meshes remains a challenge. Current approaches extract the surface through costly post-processing steps, resulting in the loss of fine geometric details or requiring significant time and leading to very dense meshes with millions of vertices. More fundamentally, the a posteriori conversion from a volumetric to a surface representation limits the ability of the final mesh to preserve all geometric structures captured during training. We present MILo, a novel Gaussian Splatting framework that bridges the gap between volumetric and surface representations by differentiably extracting a mesh from the 3D Gaussians (Fig. 16). We design a fully differentiable procedure that constructs the mesh—including both vertex locations and connectivity—at every iteration directly from the parameters of the Gaussians, which are the only quantities optimized during training.
Our method introduces three key technical contributions: (1) a bidirectional consistency framework ensuring both representations—Gaussians and the extracted mesh—capture the same underlying geometry during training; (2) an adaptive mesh extraction process performed at each training iteration, which uses Gaussians as differentiable pivots for Delaunay triangulation; (3) a novel method for computing signed distance values from the 3D Gaussians that enables precise surface extraction while avoiding geometric erosion.
Our approach can reconstruct complete scenes, including backgrounds, with state-of-the-art quality while requiring an order of magnitude fewer mesh vertices than previous methods.
Due to their light weight and empty interior, our meshes are well suited for downstream applications such as physics simulations and animation.
This project is in collaboration with Antoine Guedon, Diego Gomez, Bingchen Gog and Maks Oksjanikov from the École Polytechnique in Paris and Nissim Maruani from the TITANE team and was published in ACM Transactions on Graphics and presented at ACM SIGGRAPH Asia 18.
Illustration of the Mesh-in-the-Loop Gaussian Splatting approach.
8.2.8 Importance Sampling of the Micrograin Visible NDF
Participants: Simon Lucas, Romain Pacanowski (Inria Manao), Pascal Barla (Inria Manao).
Importance sampling of visible normal distribution functions (vNDF) is a required ingredient for the efficient rendering of microfacet-based materials. We explain how to sample the vNDF for a micrograin material model, which has been recently improved to handle height-normal correlations through a new Geometric Attenuation Factor, leading to a stronger impact on appearance compared to the earlier Smith approximation. We first derived an analytic expression for the marginal and conditional cumulative distribution functions (CDFs) of the vNDF. Then we provide efficient methods for inverting these CDFs based respectively on a 2D lookup table and on the triangle-cut method. Our method provided better efficiency for equal time renderings as illustrated in Fig. 17.
|
|
|
| (a) NDF sampling | (b) vNDF sampling | (c) Reference. |
Rendering of a living-room scene showing the benefit of our method.
This work was published in Computer Graphics Forum (CGF) journal and was also presented at Eurographics Symposium On Rendering (EGSR) 2025 19. We collaborated with Pascal Barla and Romain Pacanowski from Inria Bordeaux.
8.2.9 A Discrete Polydisperse Porous BSDF Model based on the Micrograin Framework
Participants: Kewei Xu (Université de Poitiers), Simon Lucas, Pascal Barla (Inria Manao), Benjamin Bringier (Université de Poitiers), Mickaël Ribardière (Université de Poitiers).
While previous work on micrograin models assumes a distribution of micrograins of same shapes and colors, our work extend this formalism to polydispersity. The new BSDF model support varying micrograin height and anisotropy giving additional control to simulate phenomena like retro-reflection from mixed materials, color mixture depending on lighting and observation directions, multiple directions of anisotropy.
This work is a collaboration with Kewei Xu, Benjamin Bringier, Mickaël Ribardière from Université de Poitiers and Pascal Barla from Inria Bordeaux.
8.2.10 Spectral Prefiltering of Neural Fields
Participants: Andreas Meuleman.
Neural fields excel at representing continuous visual signals but typically operate at a single, fixed resolution. We present a simple yet powerful method to optimize neural fields that can be prefiltered in a single forward pass (Fig. 18). Key innovations and features include: (1) We perform convolutional filtering in the input domain by analytically scaling Fourier feature embeddings with the filter’s frequency response. (2) This closed-form modulation generalizes beyond Gaussian filtering and supports other parametric filters (Box and Lanczos) that are unseen at training time. (3) We train the neural field using single-sample Monte Carlo estimates of the filtered signal. Our method is fast during both training and inference, and imposes no additional constraints on the network architecture. We show quantitative and qualitative improvements over existing methods for neural-field filtering.
SPNF
This project was a collaboration with Mustafa B. Yaldiz, Ishit Mehta, Nithin Raghavan, Tzu-Mao Li and Ravi Ramamoorthi from the University of California San Diego and was published at ACM SIGGRAPH Asia 2025 Conference Track 32.
8.2.11 Importance sampling of light field
Participants: Simon Lucas, George Drettakis.
To realistically light a virtual object using a real-world scene, the scene must be captured in high-dynamic range (HDR) to enable accurate virtual reconstruction. In this project, we investigate the integration of virtual objects into HDR scene reconstructions based on 3D Gaussian Splatting. However, the use of HDR Gaussians introduces challenges for efficient rendering, particularly in path-tracing–based renderers, where high dynamic range values can lead to increased noise, similar to issues encountered when using HDR environment maps. To address this issue, we will develop an importance sampling method tailored to HDR Gaussian representations in order to improve rendering efficiency and reduce noise.
8.2.12 Editable Gaussian Shading
Participants: Yohan Poirier-Ginter, George Drettakis.
Path tracing diffuse illumination in 3D Gaussians is too slow for interactive use. We are thus investigating different approaches to accelerate diffuse illumination in 3D Gaussian scenes, including cone tracing, with the goal of allowing the extension of our previous work to real-time, editable diffuse illumination. This work is in collaboration with Jean-François Lalonde Université Laval, Quebec, Canada, in the context of the co-tutelle for Yohan Poirier-Ginter.
8.2.13 Fast Gaussian Raytracing
Participants: Yohan Poirier-Ginter, George Drettakis.
3D Gaussian Splatting is a popular representation for radiance field reconstruction, distinguished by the rendering speed of its rasterization-based renderer. While 3D Gaussians can also be raytraced, this approach has so far been slower, with 3D Gaussian Ray Tracing (3DGRT) taking nearly one order of magnitude longer to optimize. To address this, we propose a fast ray tracer for 3D Gaussians designed to close this performance gap and match 3DGS's speed while preversing quality near 3DGRT's. This work is in collaboration with Jean-François Lalonde Université Laval, Quebec, Canada, in the context of the co-tutelle for Yohan Poirier-Ginter.
8.2.14 Adaptive Spatio-Temporal 3D Gaussian Splatting for Scenes with Oscillatory Motion
Participants: Petros Tzathas, Andreas Meuleman, Jeffrey Hu, Guillaume Cordonnier, George Drettakis.
We propose a method for the 3D reconstruction of dynamic scenes with incoherent motion, such as leaves moving in the wind, that can be particularly challenging because of small objects with similar appearance that move independently. The method comprises a 3D Gaussian Splatting representation with per primitive keyframes for the motion, which is adapted via a space-time densification algorithm based on error moments. Our approach has higher quality than previous solution, and has significantly higher framerate for rendering.
8.2.15 Lighting-Consistent Object Transfer Across Radiance Fields.
Participants: Nicolas Violante, Linus Franke, George Drettakis.
3D Gaussian Splatting (3DGS) is widely used to capture and render real scenes. Taking objects from one capture and inserting them into another has applications in many domains, such as Visual Effects (VFX), architecture and interior design or marketing. However, naively extracting the Gaussians from a source scene and inserting into a target will fail, because the lighting is inconsistent. We present a solution to this problem. We first allow interactive selection and segmentation to extract an object from the source scene. We then naively composite the source object into the input images of the target scene, resulting in inconsistent lighting. To harmonize lighting, we create a heterogeneous dataset of image pairs, combining synthetic, generated and real data, allowing us to fine-tune a diffusion model allowing harmonization of lighting for each input image. We then propose a consolidation step that combines our fine-tuned diffusion model with 3DGS optimization to provide a consolidated scene which has consistent lighting, including shadows and reflections. Our method provides visually compelling results, making object transfer between 3DGS easy to use and significantly improving quality compared to previous methods.
This work is a collaboration with George Kopanas from Runway ML and Google DeepMind, and Julien Philip from Netflix Eyeline Studios.
8.2.16 Extracting Material-informed Meshes from Primitive-based Radiance Fields
Participants: Panagiotis Papantonakis, Linus Franke, Alexander Mai, George Drettakis.
In this ongoing project, we aim to extract material-informed meshes from a primitive-based radiance field reconstruction of a scene. Unlike previous techniques we aim to use the reconstructed lighting of the scene instead of an distance light approximation of it. To that end, we extend an existing technique (MILo) to support physically-based rendering and material learning.
8.2.17 Immediate 3D Gaussian Splat Reconstruction of Unordered Input with Global Consistency
Participants: Andreas Meuleman, Linus Franke, Boris Zhestiankin, George Drettakis.
3D Gaussian Splatting (3DGS) has become the method of choice for reconstructing and real-time rendering of captured scenes. To capture a scene with good visual quality, continuous images sequences are usually combined with out-of-order shots for better scene coverage. Structure from motion can reconstruct such captures, but only after they are all available and often with high computational cost. Incremental reconstruction methods – often derived from SLAM solutions – provide immediate feedback, but cannot handle the out-of-order capture we require. We provide the first immediate feedback solution for such radiance field capture that provides global consistency. We first introduce a method for fast matching in out-of-order sequences, by repurposing visual place recognition models and a covisibility graph, and provide an efficient way to find highly connected keyframes, improving quality even for ordered sequences. We show how these steps – together with GPU optimization and careful Gaussian primitive placement – provide fast local reconstruction, in our challenging radiance field reconstruction case. We then introduce a novel cluster-based method, again using the covisibility graph, to provide efficient loop closure that does not require sequential input. Finally, to handle large scenes in our context, we introduce a progressive hierarchy that allows our method to scale to large environments, without compromising efficiency. Our results show that we provide immediate feedback 3DGS reconstruction with good visual quality in several datasets, with up to thousands of input images.
This project is a collaboration with Camille Montemagni from Inria Rennes (ISS OnTheFly).
8.2.18 Multi-spectral Gaussian Splatting
Participants: Linus Franke.
MS-Splatting is a project that develops a flexible framework for generating consistent 3D images from multiple cameras with different spectral ranges. The approach does not require cross-modal calibration and works with a variety of spectral data. It improves rendering quality and has applications in fields like agriculture.
This work is in collaboration with L. Meyer, J. Grün, M. Weiherer, B. Egger and M. Stamminger from the Friedrich-Alexander-Universität Erlangen Nürnberg (FAU Erlangen-Nürnberg), Germany.
8.2.19 Accelerated Gaussian Splatting
Participants: Linus Franke.
This project is focused on optimizing 3D Gaussian Splatting (3DGS) to accelerate training without sacrificing visual quality. It consolidates effective strategies from prior research and adds new optimizations to improve numerical stability, Gaussian truncation, and gradient approximation. The system offers faster training and sets a new benchmark for cost-effective, resource-efficient 3DGS optimization, with applications in non-rigid scene reconstruction.
This work is in collaboration with F. Hahlbohm, M. Eisemann and M. Magnor from the Technische Universität Braunschweig (TU Braunschweig), Germany.
8.3 Physical Simulation for Graphics
8.3.1 Arenite: A Physics Based Sandstone Simulator
Participants: Aryamaan Jain, Guillaume Cordonnier.
We introduced Arenite, a novel physics-based approach for modeling sandstone structures. The key insight of our work is that simulating a combination of stress and multi-factor erosion enables the generation of a wide variety of sandstone structures observed in nature. We isolate the key shape-forming phenomena: multi-physics fabric interlocking, wind and fluvial erosion, and particle-based deposition processes. Complex 3D structures such as arches, alcoves, hoodoos, or buttes can be achieved by creating simple 3D structures with user-painted erodable areas and vegetation and running the simulation. We demonstrate the algorithm on a wide variety of structures, and our GPU-based implementation achieves the simulation in less than 5 minutes on a desktop computer for our most complex example. See Fig. 19.
The image depicts a desert landscape with tall rock formations illustrating the structure generated by Arenite.
This work is a collaboration with Zhanyu Yang, Zhaopeng Wang and Bedrich Benes from Purdue University, USA and Marie-Paule Cani from Ecole Polytechnique, CNRS (LIX), IP Paris, France. It was published in ACM Transactions on Graphics, and presented at SIGGRAPH 2025 22.
8.3.2 Authoring Terrestrial Planets with Generative AI
Participants: Guillaume Cordonnier.
To support the design and subsequent generation of terrestrial planets for use in the creative media, we propose a solution that employs a generative model trained on satellite data from planetary bodies with a defined solid surface, such as the Earth and Mars. A user sketches coarse elevation, landcover, temperature, and precipitation directly onto a globe. Our model then infers high-resolution heightmap and surface appearance layers at planetary scales, with sufficient detail to enable animated flyovers within the exosphere at a distance of a few thousand kilometers from the planet's surface. We address the issue of distortion in the mapping from atlas to globe using a quadsphere representation, and the consistency of large-scale geomorphological features by extracting a global river network from the sketch inputs and providing this as conditioning to the diffusion. As our results demonstrate, our generative model provides a balance between: authoring control through a multi-layer painting interface with a satellite image pre-visualization; computation times proportional to the surface area being generated; landscape diversity, displaying, without repetition artifacts, the full range of elevation and landcover features drawn from multiple source planets, and geomorphological plausibility through the provision of a consistent uninterrupted exorheic global river network, where permitted by the input sketches.
This is ongoing work with Oliver Borg and James Gain from the University of Cape Town, Eric Galin, Eric Guérin, Adrien Peytavie from LIRIS / Université de Lyon, and Marie-Paule Cani from Ecole polytechynique.
8.3.3 GlacierGAN: Using Generative Adversarial Networks (GANs) to generate synthetic satellite imagery given physical simulations
Participants: Guillaume Cordonnier.
Despite the abundance of studies predicting profound changes in mountainous landscapes as a result of global warming, the general public lacks concrete and intuitive visualizations to clearly grasp the consequences of climate scenarios. This calls for efforts to translate high-level scientific results into accessible visual products, such as satellite imagery. To tackle this challenge, we use a Generative Adversarial Network (GAN) to reconstruct synthetic satellite images from simulation data, showcasing what the European Alps could look like ice-free in the future, as well as what they could have looked like during the last ice age for comparison. We integrated our visualization model into a physics-based glacier evolution model to allow glaciologists present their results directly in the form of plausible satellite images, maximizing their impact on the public. While scientific communication is imperative in the domain of global warming, this approach can be generalized to other domains in an effort to aid scientific clarity.
This is ongoing work with Guillaume Jouvet and Brandon Finley from the University of Lausanne.
8.3.4 Particle-based geomorphological transport for terrain erosion simulation
Participants: Guillaume Cordonnier.
Mountainous terrains evolve over geological timescales through erosion processes driven by the complex interplay of transported quantities such as water, sediment, and rockfall. A key challenge in erosion modeling is the simultaneous simulation of transport and erosive processes, which differ in temporal scales by several orders of magnitude. We address this challenge with a novel, parallel, stochastic particle-based method capable of simulating transport over geological timescales. Our approach relaxes the strong assumptions on velocity required by prior works (e.g., based on the Stream Power Law), enabling a new erosion model grounded in a more general form of momentum conservation. We demonstrate that our scheme accurately solves the underlying conservation laws and avoids artifacts common in previous works. Furthermore, we show that our new erosion model captures multiscale geomorphological features, producing coherent basin structures and dynamic phenomena such as braided rivers, meanders, and deltas.
This is ongoing work with Nicholas McDonald from erosiv Studio (Austria).
8.3.5 Terrain Synthesis as a Boundary Value Problem
Participants: Guillaume Cordonnier.
The digital synthesis of terrains is often expressed as an initial value problem, by which some initial heightmap is eroded and uplifted until convergence, with research focusing on ways to make more efficient or larger time-steps. In this project, terrain synthesis is instead formulated as a boundary value problem, where the recovered solutions correctly satisfy the erosion laws at steady-state. Our algorithm is fully implementable on the GPU and yields state-of-the-art performance by requiring significantly fewer iterations to achieve convergence. Beyond the performance benefits, the key conceptual benefit of our method is from a user-control perspective. Instead of constructing a solution that is dependent on an initial state, this method will synthesize any terrain that satisfies both the erosion law and the user-specified boundary conditions, which are imposed on the final state.
This is ongoing work with Nicholas McDonald from erosiv Studio (Austria).
8.3.6 Pixels2Peaks: Converting Terrain Images To Heightmaps
Participants: Aryamaan Jain, Guillaume Cordonnier.
We develop a novel AI-based method to automatically reconstruct a full 3D terrain model from a single 2D photograph. This inverse problem is a longstanding challenge in computer graphics. Our approach uses diffusion models guided by physical consistency to ensure the generated terrain is not only visually plausible but also hydrologically and geomorphologically correct.
This work is a collaboration with James Gain from University of Cape Town, South Africa.
9 Partnerships and cooperations
9.1 International research visitors
9.1.1 Visits of international scientists
- Angela Dai from TU Munich visited the group as part of her participation in the Morgenstern Colloquium in March.
- Alyosha Efros from UC Berkeley visited in January the group as part of his sabbatical in France.
- Eric Paquette from ETS Montreal visited the group in May for four weeks.
- Marc Stamminger (FAU Erlangen), Miika Aittala (NVIDIA Research Helsinki) and Peter Hedman (Google UK) visited the group and participated with talks in a mini-workshop with the opportunity of the Ph.D. defense of Nicolas Violante.
- Eric Tabellion from Google US visited the group and gave a talk in September.
9.1.2 Visits to international teams
Research stays abroad
Adrien Bousseau
-
Visited institution:
University of Edinburgh
-
Country:
UK
-
Dates:
June 2nd to July 11th (6 weeks)
-
Context of the visit:
Collaboration with the GraphViX Group let by Changjian Li in the School of Informatics, supported by the Huawei Strategic Talent Programme of University of Edinburgh
-
Mobility program/type of mobility:
Research stay
Marzia Riso
-
Visited institution:
Brown University
-
Country:
USA
-
Dates:
November 13th to November 18th
-
Context of the visit:
Collaboration with Daniel Ritchie supported by the ANR-NSF project NaturalCAD
-
Mobility program/type of mobility:
Research stay
9.2 European initiatives
9.2.1 Horizon Europe
NERPHYS
NERPHYS project on cordis.europa.eu
-
Title:
Empowering Neural Rendering Methods with Physically-Based Capabilities
-
Duration:
From December 1, 2024 to November 30, 2029
-
Partners:
- INSTITUT NATIONAL DE RECHERCHE EN INFORMATIQUE ET AUTOMATIQUE (INRIA), France
-
Inria contact:
Georges Drettakis
- Coordinator:
-
Summary:
While long restricted to an elite of expert digital artists, 3D content creation has recently been greatly simplified by deep learning. Neural representations of 3D objects have revolutionized real-world capture from photos, while generative models are starting to enable 3D object synthesis from text prompts. These methods use differentiable neural rendering that allows efficient optimization of the powerful and expressive “soft” neural representations, but ignores physically-based principles, and thus has no guarantees on accuracy, severely limiting the utility of the resulting content.
Differentiable physically-based rendering on the other hand can produce 3D assets with physics-based parameters, but depends on rigid traditional “hard” graphics representations required for light-transport computation, that make optimization much harder and is also costly, limiting applicability.
In NERPHYS, we will combine the strengths of both neural and physically-based rendering, lifting their respective limitations by introducing polymorphic 3D representations, i.e., capable of morphing between different states to accommodate both efficient gradient-based optimization and physically-based light transport. By augmenting these representations with corresponding polymorphic differentiable renderers, our methodology will unleash the potential of neural rendering to produce physically-based 3D assets with guarantees on accuracy.
NERPHYS will have ground-breaking impact on 3D content creation, moving beyond today's simplistic plausible imagery, to full physically-based rendering with guarantees on error, enabling the use of powerful neural rendering methods in any application requiring accuracy. Our polymorphic approach will fundamentally change how we reason about scene representations for geometry and appearance, while our rendering algorithms will provide a new methodology for image synthesis, e.g., for training data generation or visual effects.
DLift
Participants: Adrien Bousseau, Jiayi Wei, Loic Gaillard.
DLift project on cordis.europa.eu
-
Title:
Lifting Design Drawings to 3D
-
Duration:
From April 1, 2024 to September 30, 2025
-
Partners:
- INSTITUT NATIONAL DE RECHERCHE EN INFORMATIQUE ET AUTOMATIQUE (INRIA), France
-
Inria contact:
Adrien BOUSSEAU
-
Coordinator:
Adrien BOUSSEAU
-
Summary:
Building on the outcome of the ERC Starting Grant D3 (ERC-2016-STG 714221), the objective of DLift is to demonstrate how our technology for reconstructing drawings in 3D can streamline the Computer-Aided-Design (CAD) workflow, and to take the first steps in industrializing this technology.
Drawing is a fundamental tool of product design. However, while 2D drawings are easily understood by humans, they are currently not interpretable by computers. To confront their ideas with physical reality, designers have to separately create 3D models that form the necessary input for digital engineering tools. Skilled 3D modelers often need several hours to convert a drawing into a 3D model, at an hourly rate of 50 euros on average, making 3D modeling a major bottleneck that hinders rapid iterations between ideation and prototyping.
The ERC Starting Grant D3 aimed at bridging design exploration and design engineering by offering the first algorithm capable of automatically lifting 2D design drawings to 3D. The objective of DLift is to optimize our algorithm to integrate it within leading CAD software. This objective will be achieved through three iterative steps:
1. Feature development. Based on preliminary discussions with CAD users and software editors, we have identified a set of key features to unleash the potential of our technology.
2. User testing. We will hire professional designers and 3D modelers to stress test our tool, first to make it robust to the diversity and complexity of real-world design drawings, but also to assemble a portfolio of artworks that will illustrate diverse ways in which our technology can be used in practice.
3. Transfer. Within each iteration of feature development and testing, we will work hand-in-hand with software companies interested in our technology to assess how it addresses their specific needs. We aim at licensing our technology to one or several of these companies such that it can be integrated in their own solutions.
9.2.2 Other european programs/initiatives
Circularity 4.0
Participants: Adrien Bousseau.
Adrien Bousseau participates in a collaboration with TU Delft (The Netherlands) and École Nationale Supérieure d’Arts et Métiers (ENSAM) funded by TU Delft – France Initiative. The goal of this project is to develop novel circular design workflows and accompanying digital tools for reusing decomissioned aircrafts to create new objects.
9.3 National initiatives
9.3.1 ANR projects
ANR GLACIS
Participants: Adrien Bousseau, Anran Qi.
Project description on anr.fr
-
Title:
Graphical Languages for Creating Infographics
-
Duration:
From April 1, 2022 to March 31, 2026
-
Partners:
- INSTITUT NATIONAL DE RECHERCHE EN INFORMATIQUE ET AUTOMATIQUE (INRIA), France
- LIRIS
- University of Toronto
-
Inria contact:
Theophanis TSANDILAS
-
Coordinator:
Theophanis TSANDILAS
-
Summary:
Visualizations are commonly used to summarize complex data, illustrate problems and solutions, tell stories over data, or shape public attitudes. Unfortunately, dominant visualization systems largely target scientists and data-analysis tasks and thus fail to support communication purposes. This project looks at visualization design practices. It investigates tools and techniques that can help graphic designers, illustrators, data journalists, and infographic artists, produce creative and effective visualizations for communication. The project aims to address the more ambitious goal of computer-aided design tools, where visualization creation is driven by the graphics, starting from sketches, moving to flexible graphical structures that embed constraints, and ending to data and generative parametric instructions, which can then re-feed the designer’s sketches and graphics. The partners bring expertise from Human-Computer Interaction, Information Visualization, and Computer Graphics. In particular, GraphDeco will work on analysing sketches of data visualizations to translate them into parametric graphical objects that can be binded to data.
ANR INVTERRA
Participants: Guillaume Cordonnier, Aryamaan Jain, Melike Aydinlilar.
Project description on anr.fr
-
Title:
Inverse Control of Physically Consistent Terrains
-
Duration:
From February 1, 2023 to January 31, 2027
-
Partners:
- Inria Centre de Recherche Inria Sophia Antipolis - Méditerranée, France
-
Inria contact:
Guillaume Cordonnier
-
Coordinator:
Guillaume Cordonnier
-
Summary:
In a world where digital exchanges drive a pressing need for virtual environments, a challenge lies in the authoring of the root of these synthetic worlds: the mountains, plains, and other landforms concatenated and represented as terrains. This problem is notoriously difficult because terrains result from the interplay of physical events over geological time scales. This project aims to explore the inversion of simulation parameters as a novel paradigm for terrain generation in virtual worlds, combining geological consistency, natural diversity, and expressive user control for the first time.
ANR-NSF NaturalCAD
Participants: Adrien Bousseau, Henro Kriel, Marzia Riso.
Project description on anr.fr
-
Title:
Learning to Translate Freehand Design Drawings into Parametric CAD Programs
-
Duration:
From March 1, 2024 to October 31, 2027
-
Partners:
- INSTITUT NATIONAL DE RECHERCHE EN INFORMATIQUE ET AUTOMATIQUE (INRIA), France
- Brown University
-
Inria contact:
Adrien BOUSSEAU
-
Coordinator:
Adrien BOUSSEAU
-
Summary:
Computer Aided Design (CAD) is a multi-billion dollar industry responsible for the digital design of almost all manufactured goods. It leverages parametric modeling, which allows dimensions of a design to be changed, facilitating physically-based optimization and design re-mixing by non-experts. But CAD’s potential is diminished by the difficulty of creating parametric models: in addition to mastering design principles, professionals must learn complex CAD software interfaces. To promote effective modeling strategies and creative flow, design educators advocate freehand drawing as a preliminary step to parametric modeling. Unfortunately, CAD systems do not understand these drawings, so designers must re-create their entire design using complex CAD software. Can we automatically convert freehand drawings to parametric CAD models? Sketch-based modeling techniques do not produce parametric CAD programs; classic CAD reverse-engineering techniques cannot handle drawings as input; the newer field of visual program induction is promising but has been demonstrated only on simple shapes and programs. By leveraging the visual vocabulary shared by drawing and CAD modeling, we will develop a system to translate from the natural language of drawing to the formal language of CAD.
9.3.2 PEPER IRiMa
Participants: Guillaume Cordonnier.
Co-led by BRGM, CNRS, and Grenoble Alpes University, the Priority Research Program and Equipment - PEPR Risks (IRiMa) aims to formalize a science of risks to contribute to the development of a new national strategy for risk and disaster management in an era of global change. With a budget of €51.9 million over eight years, it brings together nearly 30 partner organizations and laboratories.
Our team is involved in the Intelligent Mapping project of the IRiMa PEPR, in a research axis aiming to integate physical principles in the mapping of risks, espcially targeted to underwater instabilities and collapses.
9.3.3 AEx NeuralGeoFlow
Participants: Guillaume Cordonnier.
NeuralGeoFlow explores a new neural solver for fast geophysical flow simulations. By training the neural network to minimize a physical energy, this data-free approach will add a memory effect to geophysical solvers, bringing significant accelerations at a marginal cost in numerical precision. I expect this methodology, inspired by methods from visual computing, to allow geophysical simulations at unprecedented spatial and temporal scales.
9.3.4 AEx CircleD
Participants: Adrien Bousseau.
Circular design aims to extend the lifespan of industrial products through repair and reuse. However, the main computer-aided design tools have been developed to support the creation of new products rather than the reuse of existing ones. The objective of this project is to revisit one of the central tasks of computer-aided design—the creation of shapes—under the new constraint of reusing parts of existing shapes. We will formulate shape reuse as a combinatorial optimization problem and explore geometry- and data-driven strategies to address the resulting computational complexity.
10 Dissemination
Participants: George Drettakis, Adrien Bousseau, Guillaume Cordonnier, Aryamaan Jain, Berend Baas, Alban Gauthier, Andreas Meuleman, Yohan Poirier-Ginter, Panagiotis Papantonakis, Marzia Riso.
10.1 Promoting scientific activities
10.1.1 Scientific events: selection
Chair of conference program committees
- Adrien Bousseau co-chaired the full paper program for the Eurographics 2025 conference. This duty involved assembling a program committee of a hundred members and overseeing the reviewing process for more than 200 submissions.
Member of the conference program committees
- Guillaume Cordonnier was member of the programm committee of CASA, SGP, Eurographics and Pacific Graphics.
- George Drettakis was a member of the program committee of EGSR and I3D.
Reviewer
- Adrien Bousseau reviewed submissions for ACM SIGGRAPH, SIGGRAPH Asia, ACM UIST, ACM CHI.
- Guillaume Cordonnier reviewed submissions for ACM SIGGRAPH.
- Aryamaan Jain reviewed submissions for ICVGIP 2025.
- Berend Baas reviewed submissions for tje Symposium on Computational Fabrication 2025.
- Yohan Poirier-Ginter reviewed submissions for Eurographics.
- Marzia Riso reviewed submissions for ACM SIGGRAPH Asia and Pacific Graphics.
- Alban Gauthier reviewed papers for the conferences Eurographics and SIGGRAPH 2025.
- Andreas Meuleman was a reviewer for ACM Transactions on Graphics (SIGGRAPH).
- George Drettakis reviewed for ACM SIGGRAPH and SIGGRAPH Asia.
10.1.2 Journal
Member of the editorial boards
- George Drettakis is an associate editor for the journal Computer Graphics Forum
Reviewer - reviewing activities
- Adrien Bousseau reviewed submissions for ACM Transactions on Graphics and IEEE Transactions on Visualization and Computer Graphics.
- Guillaume Cordonnier reviewed submissions for Computer Graphics Forum.
- Nicolas Violante reviewed papers for the journal Computer Graphics Forum
- Panagiotis Papantonakis reviewed papers Transactions on Pattern Analysis and Machine Intelligence (TPAMI) and the Computer Graphics Forum (CGF).
10.1.3 Invited talks
- Adrien Bousseau gave an invited talk at MIT and at Brown University.
- Nicolas Violante gave an invited talk at the Centre Borelli at ENS Paris-Saclay.
- George Drettakis gave invited talks at TU Munich, at the Max Planck Institut Tuebingen, Germany, at the French Academy of Sciences and gave the keynote presentations at IEEE VR 2025 conference (Saint Mellado) and Driving Simulation Conference 2025 (Stuttgart).
10.1.4 Presentations
- Berend Baas presented "Shape Approximation by Surface Reuse" at the Symposium on Geometry Processing 2025 in Bilbao.
- Nicolas Violante presented the paper "Splat and Replace: 3D Reconstruction with Repetitive Elements" at ACM SIGGRAPH 2025.
- Alban Gauthier attended EGSR 2025 where he presented the article titled "An evaluation of SVBRDF Prediction from Generative Image Models for Appearance Modeling of 3D Scenes".
- Panagiotis Papantonakis attended the Eurographics Symposium on Rendering (EGSR) 2025 in Copenhagen to present "Content-Aware Texturing for Gaussian-Splatting".
- Yohan Poirier-Ginter attended SIGGRAPH Asia 2025 where he presented "Editable Physically-based Reflections in Raytraced Gaussian Radiance Fields."
- Marzia Riso attended ACM Siggraph and ACM Symposium on Computiational Fabrication where she presented her work.
- Andreas Meuleman attended SIGGRAPH 2025 and presented "On-the-fly Reconstruction for Large-Scale Novel View Synthesis from Unposed Images"
10.1.5 Leadership within the scientific community
- George Drettakis chairs the ACM SIGGRAPH Papers Advisory Group and the Eurographics Working Group on Rendering (which is also the Steering Committee for EGSR).
10.1.6 Scientific expertise
- Adrien Bousseau reviewed a grant proposal for ANR.
10.1.7 Research administration
- Guillaume Cordonnier is a member of the organization committee of the thematic semester "machine learning and simulation" from Université Côte d'Azur, managed by Regis Duvigneau (Acumes).
- Guillaume Cordonnier is a member of the direction committee of the working group on computer graphics, geometry, virtual reality and visualization (GT CNRS IGRV) and chairs the Ph.D. award given by this working group.
- Guillaume Cordonnier is a member of the Comité du centre of the Centre Inria d’Université Côte d’Azur.
- Adrien Bousseau co-chairs the committee in charge of electing a young research fellow each year for the French Chapter of Eurographics.
- Adrien Bousseau is a member of the committee for doctoral studies (CSD) of the Centre Inria d’Université Côte d’Azur.
- George Drettakis is an elected member of the Inria Scientific Board and in charge of the Morgenstern Colloquium in Sophia-Antipolis.
- George Drettakis co-leads the Working Group on Rendering of the GT IGRV with R. Vergne and organized the GT workshop in Paris in September 2025. He is member of the Administrative Council of the Eurographics French Chapter.
10.2 Teaching - Supervision - Juries - Educational and pedagogical outreach
10.2.1 Supervision
- Capucine Nghiem, co-supervised by Adrien Bousseau with Theophanis Tsandilas, defended in July 33.
- Berend Baas, supervised by Adrien Bousseau since December 2023.
- Henro Kriel, supervised by Adrien Bousseau since September 2024.
- Alexandre Lanvin, co-supervised by Adrien Bousseau and George Drettakis since December 2024.
- Aryamaan Jain, supervised by Guillaume Cordonnier since October 2023.
- Nicolás Violante, supervised by George Drettakis since October 2022, defended December 2025 34.
- Yohan Poirier-Ginter, co-supervised by George Drettakis with Jean Francois Lalonde from Univ. Laval as part of a "co-tutelle" Ph.D., since September 2023.
- Petros Tzathas, co-supervised by George Drettakis and Guillaume Cordonnier since November 2023.
- Panagiotis Papantonakis, supervised by George Drettakis since September 2023.
10.2.2 Juries
- Adrien Bousseau was reviewer and jury president for the Ph.D. thesis of Ivan Lopes (Inria Paris), reviewer of the Ph.D. thesis of Kirill Brodt (Univ. Montreal), reviewer of the HDR thesis of Nicolas Mellado (CNRS) and examiner of the Ph.D. thesis of Xuejiao Luo (TU Delft).
- George Drettakis was a member of the Ph.D. committee of Karlis Briedis at ETH Zurich and of Briac Toussaint at INRIA Grenoble.
10.2.3 Educational and pedagogical outreach
- Adrien Bousseau presented his work to high school students twice in the context of “1 scientifique, 1 classe: chiche!”.
- George Drettakis presented his work to high school students twice in the context of “1 scientifique, 1 classe: chiche!”.
10.3 Popularization
10.3.1 Productions (articles, videos, podcasts, serious games, ...)
- George Drettakis wrote a review article on 3D Gaussian splatting for the magazine "Industrie et Technologies"
10.3.2 Participation in Live events
- Guillaume Cordonnier gave a talk on Geosciences and computers at the academic consortium for scientific mediation Terra Numerica.
- Panagiotis Papantonakis and Alexandre Lanvin, presented the lab's work for students of L3 ENS-Lyon.
- Andreas Meuleman gave a talk at "Rencontres Animation Développement Innovation" and "Back From SIGGRAPH 2025".
11 Scientific production
11.1 Major publications
- 1 articleForming Terrains by Glacial Erosion.ACM Transactions on Graphics424July 2023, 1-14HALDOI
- 2 articleSingle-Image SVBRDF Capture with a Rendering-Aware Deep Network.ACM Transactions on Graphics372018, 128 - 143HALDOI
- 3 articleActive Exploration for Neural Global Illumination of Variable Scenes.ACM Transactions on Graphics2022HALDOI
- 4 inproceedingsSymmetry-driven 3D Reconstruction from Concept Sketches.SIGGRAPH 2022 - International Conference & Exhibition On Computer Graphics & Interactive Techniques19Vancouver, CanadaACMAugust 2022, 1-8HALDOI
- 5 articleDeep Blending for Free-Viewpoint Image-Based Rendering.ACM Transactions on Graphics (SIGGRAPH Asia Conference Proceedings)376November 2018, URL: http://www-sop.inria.fr/reves/Basilic/2018/HPPFDB18
- 6 articleScalable Inside-Out Image-Based Rendering.ACM Transactions on Graphics (SIGGRAPH Asia Conference Proceedings)356December 2016, URL: http://www-sop.inria.fr/reves/Basilic/2016/HRDB16
- 7 articleEfficient Debris-flow Simulation for Steep Terrain Erosion.ACM Transactions on Graphics4342024, 58HALDOI
- 8 articleFastFlow: GPU Acceleration of Flow and Depression Routing for Landscape Simulation.Computer Graphics Forum4372024HAL
- 9 articleDeep learning speeds up ice flow modelling by several orders of magnitude.Journal of GlaciologyDecember 2021, 1-14HALDOI
- 10 article3D Gaussian Splatting for Real-Time Radiance Field Rendering.ACM Transactions on Graphics424July 2023, 1–14HALDOI
- 11 articleAccommodation and Comfort in Head-Mounted Displays.ACM Transactions on Graphics (SIGGRAPH Conference Proceedings)364July 2017, 11URL: http://www-sop.inria.fr/reves/Basilic/2017/KBBD17
- 12 articleSketch2CAD: Sequential CAD Modeling by Sketching in Context.ACM Transactions on Graphics2020HAL
- 13 articleMulti-view Relighting using a Geometry-Aware Network.ACM Transactions on Graphics382019HALDOI
- 14 articleFree-viewpoint Indoor Neural Relighting from Multi-view Stereo.ACM Transactions on Graphics2021HALDOI
- 15 article3D-Layers: Bringing Layer-Based Color Editing to VR Painting.ACM Transactions on Graphics434July 2024HALDOI
11.2 Publications of the year
International journals
International peer-reviewed conferences
Doctoral dissertations and habilitation theses
Other scientific publications
11.3 Cited publications
- 36 article3D Gaussian Splatting for Real-Time Radiance Field Rendering.ACM Trans. Graph.424July 2023DOIback to text