Keywords
Computer Science and Digital Science
- A2.5. Software engineering
- A3.1.1. Modeling, representation
- A3.1.4. Uncertain data
- A3.2.2. Knowledge extraction, cleaning
- A5.1. Human-Computer Interaction
- A5.3.4. Registration
- A5.4.4. 3D and spatio-temporal reconstruction
- A5.4.5. Object tracking and motion analysis
- A5.6. Virtual reality, augmented reality
- A6.1.1. Continuous Modeling (PDE, ODE)
- A6.1.2. Stochastic Modeling
- A6.1.5. Multiphysics modeling
- A6.2.3. Probabilistic methods
- A6.2.4. Statistical methods
- A6.2.5. Numerical Linear Algebra
- A6.2.8. Computational geometry and meshes
- A6.3.1. Inverse problems
- A6.3.2. Data assimilation
- A6.3.3. Data processing
- A6.3.4. Model reduction
- A9.2. Machine learning
- A9.10. Hybrid approaches for AI
Other Research Topics and Application Domains
- B1.2. Neuroscience and cognitive science
- B2.2.6. Neurodegenerative diseases
- B2.4. Therapies
- B2.4.3. Surgery
- B2.6. Biological and medical imaging
- B2.7. Medical devices
- B2.7.1. Surgical devices
1 Team members, visitors, external collaborators
Research Scientists
- Stéphane Cotin [Team leader, Inria, Senior Researcher, HDR]
- Hadrien Courtecuisse [CNRS, Researcher]
- Michel Duprez [Inria, Researcher]
- Axel Hutt [Inria, Senior Researcher, HDR]
Post-Doctoral Fellow
- Andréa Mendizabal [Assistance publique/Hôpitaux de Paris, until Feb 2021]
PhD Students
- Paul Baksic [Univ de Strasbourg]
- Francois Lecomte [Inria, from Feb 2021]
- Guillaume Mestdagh [Univ de Strasbourg]
- Sergei Nikolaev [European Union, until Sep 2021]
- Alban Odot [Inria]
- Joséphine Riedinger [Inria]
- Valentina Scarponi [Univ de Strasbourg, from Oct 2021]
- Thomas Wahl [Inria, from Oct 2021]
- Ziqiu Zeng [Univ de Strasbourg]
Technical Staff
- Rémi Bessard Duparc [Inria, Engineer]
- Jean-Nicolas Brunet [Inria, Engineer, until Jul 2021]
- Sidaty El Hadramy [Institut Hospitalo-Universitaire de Strasbourg, Engineer, from May 2021 until Oct 2021]
- Robin Enjalbert [Inria, Engineer, from Feb 2021]
- Anass Maourid [Inria, Engineer, from May 2021]
- Pedro Suruagy Perrusi [Univ de Strasbourg, Engineer, until Sep 2021]
Interns and Apprentices
- Anne Sarah Debly [Inria, from Mar 2021 until Aug 2021]
- Vincent Italiano [Inria, from Jun 2021 until Jul 2021]
- Virginie Marec [Univ de Strasbourg, from Mar 2021 until Aug 2021]
- Philippe Pincon [Inria, from Mar 2021 until Aug 2021]
- Killian Vuillemot [Inria, from Mar 2021 until Jul 2021]
- Thomas Wahl [Inria, from Mar 2021 until Jul 2021]
Administrative Assistant
- Ouiza Herbi [Inria]
Visiting Scientist
- Andréa Mendizabal [Univ de Verone - Italie, from Sep 2021]
External Collaborators
- Nazim Haouchine [Brigham and Women’s Hospital - Boston USA]
- Didier Pinault [INSERM, HDR]
- Nava Schulmann [Ecole polytechnique de Milan, until Jul 2021]
2 Overall objectives
2.1 Team Overview
The MIMESIS team develops numerical tools for advanced numerical simulations in the context of surgical training, planning and per-operative guidance (see Fig. 1). The underlying objectives include patient-specific biophysical and electrophysiological modeling, novel numerical techniques for real-time computation, data assimilation using Bayesian methods and more generally data-driven simulation. This last topic is a transverse research theme which raises several open problems, related to the field of machine learning. To pursue these directions we have assembled a team with a multidisciplinary background, and have established close collaborations with academic and clinical partners, in particular the IHU institute in Strasbourg. We also continue the development of the SOFA framework through the creation of a consortium, to better support the increasingly large community of users.
General pipeline for generating a digital twin of an organ.
2.2 Challenges
Image- and signal-guided therapy has revolutionized medicine, in its ability to provide care that is both efficient and effective. However, images and multivariate signals acquired during an intervention are either incomplete, under-exploited or can induce adverse outcomes. This can be due, for instance, to the lack of dimensionality of X ray images and the associated radiation exposure for the patient or the spatial sub-sampling in multivariate signals by a too low number of measurement electrodes. We believe that by combining our expertise in real-time numerical simulation (of soft tissues, flexible medical devices, and complex interactions) with data extracted from intra-operative images and experimental multi-electrode signals, we could provide efficient per-operative guidance. To reach these objectives we need to solve challenges that lie at the intersection of several scientific domains. They include the development of novel numerical strategies (to enable real-time computation even with the increase in complexity of future models), and data-driven simulation (to link simulation with real world data).
3 Research program
3.1 Real-time computational models for interactive applications
The principal objective of this challenge is to improve, at the numerical level, the efficiency, robustness, and quality of the simulations (see Fig. 2). An important part of our research is dedicated to the development of computational models that remain compatible with real-time computation, i.e., which allow immediate visual or haptic feedback. This typically requires computation times below 50 ms and in some cases around 1 ms when multiple iterations of the model deformation are needed within a single time step, for instance, to solve control problems or to perform data assimilation. Such advanced models can not only increase the realism of future training systems, but also act as a bridge toward the development of patient-specific solutions for computer-aided interventions. Additionally, such simulations should run on (high-end) consumer level computers (i.e. with a single multi-core CPU and a dedicated GPU).
To reach these goals, in a first topic we are investigating novel finite element techniques able to cope with complex, potentially ill-defined input data. After developing Smoothed FEM for real-time simulations, we are developing meshless techniques and immersed boundary methods. The first one is well suited for topological changes, which we sometimes need to account for in our simulations. The second is expected to lead to more stable, and numerically efficient, formulations of the finite element method. We are also developing numerical techniques to compute the complex interactions that can take place between anatomical structures or between medical devices and organs. Boundary conditions are known to also play an important role in the solution of such problems. Therefore we are investigating solutions to both identify and model the interactions that take place between the structure of interest and its anatomical environment.
In a different research topic, the team develops neuronal network models that describe the interactions between brain areas under neural stimulation. Each neuron exhibits stochastic spiking activity at the millisecond scale and each brain area comprises thousands of neurons. To achieve numerical real-time integration of the dynamical network models, we aim to reduce the dimensionality of the network models and to this end, derive analytically dimension-reduced mean-field models. These models capture the major dynamics of the network models. Typically, such mean-field models are stochastic delayed differential equation systems. For evalution, the network models are simulated on multi-core CPU computers.
3.2 Data-driven simulations
Data-driven simulation has been a recent area of research in our team (see Fig. 3). We have demonstrated that it has the potential to bridge the gap between medical imaging and clinical routine by adapting pre-operative data to the time of the procedure. In the areas of non-rigid registration and augmented reality during surgery, we have demonstrated the benefit of our physics-based approaches with several key publications in major conferences (MICCAI, CVPR, IPCAI, ISMAR).
We have continued this work with an emphasis on robustness to uncertainty and outliers in the information extracted in real-time from image data, as well as real-time parameter estimation. This is currently done by combining Bayesian methods with advanced physics-based methods to handle uncertainties in image-driven simulations.
Finally, Bayesian or similar methods require to perform a large amount of simulations to sample the domain space, even when using efficient methods such as Reduced Order Unscented Kalman Filters. For this reason, we are investigating the use of neural networks to perform predictions instead of using full numerical simulations. In one of our recent paper, we show that it is possible to teach a neural network from numerical simulations and predict, with good accuracy, the deformation of an organ.
4 Application domains
4.1 Surgical training
Virtual training helps medical students to get familiar with surgical procedures before manipulation of real patients. The development of simulation used for medical training usually requires important computational power, since realistic behaviors are key to deliver a high-fidelity experience to the trainee. Further, the quality of interaction with the simulator (usually via visual and haptic rendering) is also of paramount importance. All these constraints make the development of training systems time-consuming, thus limiting the deployment of virtual simulators in standard medical curriculum.
4.2 Pre-operative planning
Beyond training, clinicians ask for innovative tools that can assist them in the pre-operative planning of an intervention. Using the patient information acquired before the operation, physics-based simulations allow to simulate the effect of therapy with no risk to the patient. The clinicians can thus virtually assess different strategies and select the optimal procedure. Compared to a training simulation, a planning system requires a high accuracy to ensure reliability. Constrained by the time elapsed between the preoperative acquisition and the intervention, the computation must also be efficient.
4.3 Intra-operative guidance
Besides the surgery training and planning, another major need from clinicians is surgical guidance. While the clinician is performing the operation, a guidance system provides enriched visual feedback. This is especially useful with the emergence of minimally invasive surgery (MIS) where the visual information is often strongly limited. It can be used for example to avoid critical areas such as vessels or to highlight the position of a tumor during its resection. In the MIS technique, the clinician does not interact with organs directly as in the open surgery, but manipulates instruments inserted through trocars placed in small incisions in the wall of the abdominal cavity. The surgeon can observe these instruments on a display showing a video stream captured by an endoscopic camera inserted through the navel. The main advantage of the method resides in reducing pain and time recovery, in addition to reducing bleeding and risks of infection. However, from a surgical standpoint, the procedure is quite complex since the field of view is considerably reduced and the direct manipulation of organs is not possible.
5 Highlights of the year
5.1 Awards
- Thesis Award - 3rd place : Andrea Mendizabal
- Award for best innovative technology from the Academy of Surgery: Nicolas Golse (MD) and Stéphane Cotin
- Publication in Nature Communications 33
6 New software and platforms
In this section, we present the new software and platforms of the team.
6.1 New software
6.1.1 Caribou
-
Name:
Advanced and modular FEM library
-
Keywords:
Sofa, Multiphysics modelling, Biomechanics, Geometric algorithms, Plugin SOFA
-
Functional Description:
The Caribou project is aimed at multiphysics computation. This plugin complements the FEM-based models existing in SOFA with a more modularized approach. For continuum solid mechanics, the mechanical law, the type of finite element and the quadrature method can be selected separately. This method therefore eases the reading and understanding of the code and the associated theory. The Caribou project also provides generic C++ utilities, and SOFA components such as solvers.
The project is composed of two modules: (i) Caribou library brings multiple geometric, linear analysis and topological tools that are designed to be as independent as possible from external projects. (ii) SofaCaribou library is built on top of the caribou library, but brings new components to the SOFA project as a plugin.
- Publication:
-
Authors:
Jean-Nicolas Brunet, Stephane Cotin
-
Contact:
Jean-Nicolas Brunet
6.1.2 SOFA Optimus
-
Name:
Optimization methods in SOFA: stochastic filtering and data assimilation.
-
Keywords:
Data assimilation, Kalman filter, Stochastic optimization
-
Functional Description:
Optimus is a plugin to work with advanced methods of state estimation and parameter identification. It was created to provide a testing environment for data-driven physics-based modeling (typically finite elements). While currently the plugin implements only stochastic methods based on Kalman filtering, its architecture allows for the implementation of generic prediction–correction schemes where the model is employed as a predictor and correction is performed using given observation data.
-
News of the Year:
Late 2020, the repository has been cleaned and prepared for its open-source release.
- URL:
-
Contact:
Stephane Cotin
-
Participant:
Sergei Nikolaev
6.1.3 DeepPhysX
-
Name:
DeepPhysX: interfacing AI with multi-physics simulation
-
Keywords:
Numerical simulations, Deep learning, Neural networks, Python
-
Functional Description:
The purpose of DeepPhysX framework is to provide an interface between deep learning algorithms and numerical simulations. It is a full Python project with two main pipelines, allowing both to train artificial neural networks with simulated data and to use trained neural networks as components of numerical simulations. DeepPhysX manages not only the production of synthetic data with multiple numerical simulations in multiprocessing but also the storage of the produced dataset. Additional tools are provided to visualize numerical simulations and to follow the evolution of training sessions.
-
News of the Year:
Started in February 2021, now the project is providing an interface with SOFA and TensorFlow, respectively, for the numerical simulation compatibility and for the deep learning compatibility.
- URL:
-
Authors:
Robin Enjalbert, Alban Odot, Stephane Cotin
-
Contact:
Robin Enjalbert
6.2 New platforms
SOFA
Participants: Hugo Talbot, Frédérick Roy, Guillaume Paran, Christian Duriez, Stéphane Cotin.
-
Name:
Simulation Open Framework Architecture (SOFA)
-
Keywords:
Real time, Multi-physics simulation, Medical applications Functional
-
Description:
SOFA is an Open Source framework primarily targeted at real-time simulation of soft bodies, with an emphasis on medical simulation and robotics. It is mostly intended for the research community to help develop new algorithms, but can also be used by companies as an efficient library on which product can be developed. Marion Surgical, InSimo, Antoscope, Siemens R&D, Naviworks, and many others have adopted SOFA in their development process.
Based on an advanced software architecture, it allows the creation of complex and evolving simulations by combining new algorithms with algorithms already included in SOFA through Python scripting. This makes it possible to modify most parameters of the simulation (deformable behavior, surface representation, solver, constraints, collision algorithm, etc. ) but also to build complex models from simpler ones using a scene-graph description and an efficient simulation of the dynamics of interacting objects using abstract equation solvers.
Members of the team participate to the technical committee and contribute to new design choices as well as new plugins derived from our research. The main evolution of SOFA this year has been the simplification of the core of the framework, to allow for a more modular and flexible software.
- URL:
-
Contacts:
Hugo Talbot, Stéphane Cotin
-
Partner:
MLMS
-
Creation:
2004
7 New results
7.1 Real-time prediction of hyperelastic deformation using Deep Learning
Participants: Alban Odot, Andrea Mendizabal, Jean-Nicolas Brunet, Stéphane Cotin.
Real-time simulation of elastic structures is essential in many applications, from computer-guided surgical interventions to interactive design in mechanical engineering. The Finite Element Method is often used as the numerical method of reference for solving the partial differential equations associated with these problems. Yet, deep learning methods have recently shown that they could represent an alternative strategy to solve physics-based problems. In 40, we propose a solution to simulate hyper-elastic materials using a data-driven approach, where a neural network is trained to learn the non-linear relationship between boundary conditions and the resulting displacement field. We also introduce a method to guarantee the validity of the solution. In total, we present three contributions: an optimized data set generation algorithm based on modal analysis, a physics-informed loss function, and a Hybrid Newton-Raphson algorithm. The method is applied to two benchmarks: a cantilever beam and a propeller. The results show that our network architecture trained with a limited amount of data can predict the displacement field in less than a millisecond. The predictions on various geometries, topologies, mesh resolutions, and boundary conditions are accurate to a few micrometers for non-linear deformations of several centimeters of amplitude.7.2 Methods to interweave observations and models
Participants: Axel Hutt, Nava Schulmann, Stéphane Cotin.
Observations may improve theoretical models and several techniques are known to merge both in an optimal way. Data assimilation combines both observations and models in one technique. We have employed the Localized Ensemble Transform Kalman Filter (LETKF) to estimate short-term predictions in the presence of nonlocal observations and how they should be assimilated to achieve optimum predictions. Moreover, we have employed a conventional LETKF in a new adaptive feedback loop to control the power of a neural system in two frequency bands. To this end, an observer model estimates a stimulus by a LETKF whose observed data represents frequency-filtered data from the system under study. This theoretical study represents a theoretical pre-study to an application in experimental neurostimulation rodent experiment.7.3 The impact of additive noise on neural systems
Participants: Axel Hutt, Thomas Wahl.
Neural populations in the brain are embedded in network of various brain areas. Experimental studies have shown that intrinsic noise plays an important role in the brain information processing. By virtue of previous theoretical studies on general high-dimensional nonlinear stochastic systems, we know that additive noise tunes the stability of nonlinear neural systems. To understand better the underlying mechanism and to investigate whether such a stability modulation by additive noise may explain experimental results, we have studied randomly-connected stochastic networks [ref: hal-03244053] and could show that additive noise tunes gamma-activity. This effect may explain experimental features in cortical activity. The additive-noise induced neural activity can also explain the switch between experimentally observed occipital gamma- and alpha- EEG activity while closing and opening eyes 14.
7.4 New Finite Element Method for biomechanical models
Participants: Michel Duprez, Stéphane Cotin.
One of the major issues in the computational mechanics is to take into account the geometrical complexity. To overcome this difficulty and to avoid the expensive mesh generation, geometrically unfitted methods, i.e. the numerical methods using the simple computational meshes that do not fit the boundary of the domain, and/or the internal interfaces, have been widely developed. In a preview work, we have developed for the Poisson Equation with Dirichlet boundary conditions, a new unfitted method called φ-FEM that converges optimally and uses classical finite element spaces so that it can be easily implemented using general FEM libraries. The main idea is to take into account the geometry thanks to a level set function describing the boundary or the interface. In 8, we have established and analysed a φ-FEM scheme for Neumann boundary conditions. More recently, in 5, we have demonstrated the applicability of φ-FEM to some more sophisticated governing equations arising in the computational mechanics. We have considered the linear elasticity equations accompanied by either pure Dirichlet boundary conditions or by the mixed ones (Dirichlet and Neumann boundary conditios co-existing on parts of the boundary), an interface problem (linear elasticity with material coefficients abruptly changing over an internal interface), a model of elastic structures with cracks, and finally the heat equation. In all these settings, we derive an appropriate variant of φ-FEM and then illustrate it by numerical tests on manufactured solutions. We also compare the accuracy and efficiency of φ-FEM with those of the standard fitted FEM on the meshes of similar size, revealing the substantial gains that can be achieved by φ-FEM in both the accuracy and the computational time.
7.5 Adaptive neurofeedback control in tDCS with application to psychosis
Participants: Thomas Wahl, Joséphine Riedinger, Michel Duprez, Axel Hutt, Didier Pinault.
Schizophrenia is a devastating illness that affects about 1 of the population worldwide. It shortens the lifespan of people affected by the illness by at least 20 years. The cause of schizophrenia is not known and current treatments do not reduce disability. Hence research on treatments that reduce disability is critical. Transcranial electric stimulation promises to alleviate patients from the disease symptoms. Although transcranial Direct Current Stimulation (tDCS) is widely applied in psychosis patients suffering from schizophrenia, it is still inefficient in its impact since too less is known about optimal stimulation parameters specific for each patient. It is known from experimental studies of electroencephalographic data (EEG) that psychosis diminishes alpha-activity (oscillations about a frequency of 10Hz) and enhances gamma-activity (oscillations about 40Hz). One objective of optimal tDCS is to enhance alpha- and diminish gamma-activity to remedy patients from symptoms. To this end, we have been working on three approaches to improve tDCS :
-
(i)
we have developed a mathematical neural population model that describes observable electroencephalographic data in psychosis (J. Riedinger, D. Pinault and A. Hutt).
-
(ii)
we have developed a new adaptive neurostimulation method (utilizing the model in approach (i)) based on an Ensemble Kalman Filter that allows to enhance and/or diminish spectral power in two frequency bands independently (J. Riedinger and A. Hutt).
-
(iii)
we have analysed mathematically several different linear adaptive feedback control methods to find the optimal control technique (T. Wahl, M. Duprez and A. Hutt).
7.6 Shared control strategy for needle insertion into soft tissues
Participants: Hadrien Courtecuisse, Paul Baksic.
This work (see 34) deals with the problem of needle steering in deformable tissues subject to physiological motions. A novel shared control method is proposed, which combines an automatic needle steering algorithm with the motions applied by the radiologist, in order to place the needle tip at the desired location. The core motivation is to leave potentially dangerous decisions and actions to the practitioner, whereas complex nonintuitive manipulations of the needle are performed automatically, in particular to compensate for breathing motions. The most original part of the present work lies in the method used to combine user inputs with a closed-loop automatic needle steering control method based on inverse Finite Element simulations. The method is evaluated with a realistic virtual environment using 2D X-ray projection images. The results are compared with those obtained with a fully teleoperated system, on the one hand, and with a fully automatic solution, on the other hand. These experiments show that the shared control solution allows for a better needle tip placement when only projection imaging is available.
8 Bilateral contracts and grants with industry
8.1 Bilateral contracts with industry
Infiny tech 3D
Participants: Stéphane Cotin, Hadrien Courtecuisse.
-
Company:
Infiny tech 3D (France)
-
Duration:
3 years
-
Description:
Founded in 2018, InfinyTech3D develops and markets a software solution that enables the realistic simulation of complex physical complex physical behaviors, such as surgical operations in Virtual Reality and Augmented Reality, using haptic interfaces and 3D visualization devices such as VR headsets. The partnership between MIMESIS and InfinyTech3D has led to the development of a prototype of training system for percutaneous procedures. This work is based on the work of the team on needle interactions with soft tissues. The Canadian company Marion Surgical is also involved in this cooperation.
9 Partnerships and cooperations
Participants: Stéphane Cotin, Axel Hutt, Michel Duprez, Hadrien Courtecuisse.
9.1 International initiatives
9.1.1 STIC/MATH/CLIMAT AmSud project
STIC AmSud project: NEMBICA
-
Title:
NEw Methods for BIological Control of the Arboviruses
-
Duration:
2018 - 2021
-
Coordinator:
Pierre-Alexandre Bliman
-
Partners:
- Inria Strasbourg / Nancy - Grand Est, MIMESIS team (France)
- Cirad (France)
- Inra (France)
- Universidad de Buenos Aires (Argentina)
- Fundação Getulio Vargas (Brazil)
- Universidade Federal Fluminense (Brazil)
- Fiocruz, Rio de Janeiro (Brazil)
- Universidade Estadual Paulista “Júlio de Mesquita Filho” (Brazil)
- Universidad de Quindio (Colombia)
- Universidad Autónoma de Occidente (Colombia)
- Universidad del Valle (Paraguay)
- National University of Asuncion (Paraguay)
- Universidad Nacional Mayor de San Marcos (Peru)
- Université Paris 13 (France)
-
Inria contact (in MIEMSIS):
michel.duprez@inria.fr
-
Summary:
The present project is concerned with new strategies of biological control of the spread of many and important diseases transmitted by mosquitoes of the genus Aedes, which put at risk considerable portions of the human population and infect millions of people every year. The aim is to elaborate and analyze pertinent models, and to evaluate qualitatively and quantitatively different control strategies, with the goal of understanding the key aspects and parameters of the dynamics, testing concepts, estimating feasibility, identifying risks and reducing cost.
9.2 International research visitors
9.2.1 Visits of international scientists
Dr Eligiusz Postek
-
Status
(researcher)
-
Institution of origin:
Department of Information and Computational Science, Institute of Fundamental Technological, Research of the Polish Academy of Sciences
-
Country:
Poland
-
Dates:
Oct. - Dec. 2021
-
Context of the visit:
The physical environment of living cells and tissues, and more specifically their mechanical interaction with it, plays a crucial regulatory role in their biological behavior such as cell differentiation, apoptosis, proliferation, tissue growth, remodeling, wound healing, etc. During this visit, we worked on the idea of coupling agent-based modeling (ABM) and mechanical modeling to create a prototype of an agent-stress tissue growth simulator.
-
Mobility program/type of mobility:
(research stay and lecture)
Muhammad Sajjad
-
Status
(researcher)
-
Institution of origin:
Islamia College Peshawar
-
Country:
Pakistan
-
Dates:
November 2021
-
Context of the visit:
Prof. Muhammad Sajjad took place during his ERCIM Research fellowship. His work on using Artificial Intelligence in healthcare support systems within the ALAMEDA European project aims to provide personalized assessments for patients with Parkinson's disease, Multiple Sclerosis, and Stroke. The main objectives are to ensure that medical interventions are effective and that situations likely to aggravate can be predicted. During his visit, we discussed our different research results and methods to predict human kinematics or soft tissue deformation through the combined use of machine learning and physics-based modeling.
-
Mobility program/type of mobility:
(ERCIM research stay)
9.2.2 FP7 & H2020 projects
H2020 - HiPerNav
-
Title:
High Performance Soft-tissue Navigation
-
Duration:
Nov. 2016 - April 2021
-
Coordinator:
Prof. Ole Jakob Elle
-
Partners:
- Inria Strasbourg / Nancy - Grand Est, MIMESIS team (France)
- Cascination AG (Switzerland)
- Innovasjon Norge (Norway)
- Norges Teknisk-naturvitenskapelige Universitet Ntnu (Norway)
- Oslo Universitetssykehus Hf (Norway)
- Siemens Healthcare Gmbh (Germany)
- Sintef As (Norway)
- Technische Universiteit Delft (Netherlands)
- Universidad De Cordoba (Spain)
- Universite Paris 13 (France)
-
Inria contact:
stephane.cotin@inria.fr
-
Summary:
HiPerNav is an Innovative Training Network (ITN) funded through a Marie Skłodowska-Curie grant. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 722068. There are 14 fully funded and 2 partially funded PhD’s working on the project.
H2020 - LASER OPTIMAL
-
Title:
Laser Ablation: SElectivity and monitoRing for OPTImal tuMor removAL
-
Duration:
May 2018 - April 2023
-
Coordinator:
Paola Saccomandi
-
Partners:
- Inria Strasbourg / Nancy - Grand Est, MIMESIS team (France)
- IHU Strasbourg (France)
- Politecnico di Milano (Italy)
-
Inria contact:
stephane.cotin@inria.fr
-
Summary:
Laser Ablation (LA) was extensively investigated for its benefits as minimally invasive thermal therapy for tumor. Despite the LA pros as potential alternative to surgical resection (e.g. use of small fiber optics, echo-endoscope procedures and image-guidance without artifact), the lack of tools for safe and patient-specific treatment restrained its clinical use. LASER OPTIMAL offers a renaissance to LA for the practical management of challenging tumors (e.g. pancreatic cancer): it investigates and develops integrated solutions to achieve an effective and selective LA, that thermally destroys the whole tumor mass, while spearing the normal tissue around. The excellent ambition of LASER OPTIMAL is to achieve and merge: a) biocompatible nanoparticles (BNPs) injected in the tumor, to enhance the selective absorption of laser light; b) patient-specific anatomy of tumor and its surrounding, extracted from clinical images, to retrieve the optimal laser settings; c) accurate, fast and real-time heat-transfer model to simulate laser-tissue-BNPs interaction, predict and visualize the treatment dynamics; d) real-time temperature measurement system to monitor LA effects, account for unpredictable physiological events and tune the settings (closed-loop). The design of ex vivo and in vivo animal tests allows assessing the system performances and driving the possible workflow re-design. Finally, human trials are envisaged to prove the significant impact of the LASER OPTIMAL paradigm. The collaboration of researchers, engineers and clinicians will drive the use of this innovative strategy in clinical routine. The research on the patient-specific system for the mini-invasive tumors removal, and the ground-breaking insights on clinical use of BNPs will strongly impact on EU healthcare system and society, by creating a novel product. This paradigm is also embeddable in existing system of industrial partner, extendable to other procedures, thus able to encourage a dedicated market.
H2020 - DRIVEN
-
Title:
Increasing the scientific excellence and innovation capacity in Data-Driven Simulation of the University of Luxembourg
-
Duration:
Oct. 2018 - March 2022
-
Coordinator:
Stéphane Bordas
-
Partners:
- Inria Strasbourg / Nancy - Grand Est, MIMESIS team (France)
- University of Limerick (Ireland)
- University of Texas, Austin (USA)
- Université du Luxembourg
-
Inria contact:
stephane.cotin@inria.fr
-
Summary:
DRIVEN project is a 1 million euro project coordinated by the University of Luxembourg and aiming at boosting the scientific excellence and technology-transfer capacity in data-driven simulation of the University of Luxembourg. Funded by the EU’s Horizon 2020 programme for research and innovation, the University of Luxembourg will benefit from top international partners: Institut National de Recherche en Informatique et en Automatique (France), University of Limerick (Ireland) and University of Texas at Austin (USA).
9.3 National initiatives
ADT - DeepPhysX: Data-driven simulation
-
Coordinator:
Stéphane Cotin
-
Inria contact:
stephane.cotin@inria.fr
-
Summary:
This project aims to develop new tools for real-time navigation and registration in image-guided surgery. It aims to integrate learning methods with numerical simulations in order to obtain robust predictions adapted to the patient. At first, the project considers previous calculation codes, e.g. PyTorch, SOFA and its Python3 interface and the Caribou plugin of Sofa. Then the development of the Kromagon plugin of Sofa dedicated to deep learning (Python and C ++ development) will continue. Essentially, a new Newton algorithm will be developed that combines AI-based prediction and numerical computation. Performance and accuracy evaluation and documentation and provision of the plugin (binary or open) will close the project. This work will be performed in close collaboration to the “surgical guidance” project (Hopital Paul Brousse and / or IHU Strasbourg).
ADT - Flexible Endoscopy Training System
-
Coordinator:
Stéphane Cotin
-
Inria contact:
stephane.cotin@inria.fr
-
Summary:
The Mimesis team has contributed to the EVEREST (ANR) project which ended in March 2021. Its developments have allowed to take advantage of the SOFA physics engine and the Unity rendering engine in the creation of a playful simulation for flexible endoscopy training. Many improvements and new functionalities remainded to be implemented to reach the objective of making this simulator a realistic and clinically validated learning tool that is accessible to the largest number of people.
ADT - AI for Surgical Vision
-
Coordinator:
Stéphane Cotin
-
Inria contact:
stephane.cotin@inria.fr
-
Summary:
The objective of the project is to develop and integrate computer vision algorithms capable of processing images from the operating room in real time into a clinically usable augmented reality prototype. This project reinforces the work carried out in partnership with the "Augmented Operating Room" (BOPA) innovation chair at the APHP (www.chaire-bopa.fr). The project is part of the Blok-Viz axis with the video capture of the operating field and then the processing of these data in order to extract relevant information on which AI algorithms dedicated to attention analysis, registration and augmented reality can be based. The project includes in particular:
- the development of state-of-the-art algorithms to segment the liver (and other anatomical structures) in the image of the surgical field
- reconstruction of the visible surface of the liver from video streams
- optimization of the calculation codes in order to allow real-time processing
- integration of the algorithms in a numerical simulation code base
- taking into account other image modalities or sensors in order to improve the current results
ANR - SPERRY
-
Title:
SuPervisEd Robotic suRgerY - application to needle insertion.
-
Duration:
Dec. 2018 - Dec. 2022
-
Coordinator:
Hadrien Courtecuisse (MIMESIS)
-
Partners:
- Inria Strasbourg / Nancy - Grand Est, MIMESIS team (France)
- ICube Laboratoire des sciences de l'Ingénieur, de l'Informatique et de l'Imagerie (France)
-
Inria contact:
hadrien.courtecuisse@inria.fr
-
Summary:
Percutaneous medical procedures (using surgical needles) are among the least invasive approaches to accessing deep internal structures of organs without damaging surrounding tissues. Today, many surgical procedures rely on the use of needles allowing for complex interventions such as curie-therapies or thermo-ablations of tumors (cryoablation, radio frequencies). Unlike traditional open surgery, these approaches only affect a localized area around the needle, reducing trauma and risks of complications. These treatments also offer new solutions for tumors or for metastases for which traditional methods may be contraindicated due to the age of the patient and the extent or location of the disease. In this project, we want to develop new solutions for the control of medical robots interacting with soft tissues. This work is motivated by recent advances in the field of medical simulation achieving a sufficient level of realism to help surgeons during the operation. The maturity of these techniques now suggests the ability to use a simulation intra-operatively to control the motion of a robotic system for needle insertion. This is really a challenge, because in general, little information can be extracted in real time from images during an intervention. We believe that even minimal knowledge of the mechanical behavior of structures, associated with the use of images can make it possible and allow a robot to reach a pre-identified target during a planning stage, without human intervention.
ANR - SPECULAR
-
Title:
Simulation of needle insertion with virtual reality and haptics.
-
Duration:
Oct. 2021 - Jan. 2025
-
Coordinator:
Stephane Cotin (MIMESIS)
-
Partners:
- Inria Strasbourg / Nancy - Grand Est, MIMESIS team (France)
- Inria Lille - Nord Europe, DEFROST team (France)
- InfinyTech 3D (France)
-
Inria contact:
stephane.cotin@inria.fr
-
Summary:
Needle-based interventions are among the least invasive surgical approaches to access deep internal structures into organs' volumes without damaging surrounding tissues. Percutaneous image- guided interventions have become gold standard for numerous interventions such as biopsy, vertebroplasty, or radiofrequency ablation (RFA) that uses heat at the tip of a needle to destroy cancer cells. Unlike traditional invasive approaches, needles only affect a localized area around the needle shaft, reducing this way the occurrence of traumas and risks of postoperative complications [CGCF11]. Yet, the possibility for patients to benefit from these alternative treatments depends first, on the medical recommendations (concerns about the age, the extent or localization of the disease), but also on the practitioner’s experience and preferences with respect to needle-based approaches. The project targets the development of a platform for virtual-reality based surgical simulation, with focus on hepatic surgery. The overall goal is to create a reliable, powerful and complete platform able to train clinicians at different level of complications in a way to improve their learning curve and skills. The system will comprise realistic physical, haptics and visual components for representing the behavior of various tissues, and the interaction of surgical tools in a way to model complex intervention scenarios able to increase the realism up to the maximum level.
ANR - VATSOP
-
Title:
Images and models for computer guidance during Video Assisted Thoracic Surgery (VATS).
-
Duration:
Jan. 2021 - Jan. 2025
-
Coordinator:
Jean-Louis Dillenseger (LTIS Rennes)
-
Partners:
- TIMC-IMAG Techniques de l'Ingénierie Médicale et de la Complexité - Informatique, Mathématiques et Applications, Grenoble (France)
- Inria Strasbourg / Nancy - Grand Est, MIMESIS team (France)
- LTSI Laboratoire Traitement du Signal et de l'Image (France)
-
Inria contact:
stephane.cotin@inria.fr
-
Summary:
In France, lung cancer is the most common cause of death by cancer before breast and colorectal cancers. Video-Assisted Thoracic Surgery (VATS) is a minimally-invasive trans-thoracic procedure that has become prominent for the treatment of lung cancer. However, localizing the nodules during VATS is extremely challenging due to the technical difficulties inherent to endoscopic surgery, and due to the collapse of the lung during the procedure. The main objective of our project is to develop and integrate, in an operating room, an image-based guidance solution which would: 1) estimate the position of the patient’s nodule after pneumothorax from CBCT images; 2) propose an augmented reality-based guidance to this location on the endoscopic view; 3) allow to follow this location in real-time on the endoscopic view, during manipulation of the lung by the surgeon. Partners of the project are TIMC Grenoble and LTSI Rennes.
ANR - TRECOS
-
Title:
New Trends in Control and Stabilization: Constraints and a Non-local Terms.
-
Duration:
Dec. 2020 - Nov. 2024
-
Coordinator:
Sylvain Ervedoza (Institut de mathématiques de Bordeaux) (Laboratoire Traitement du Signal et de l'Image)
-
Partners:
- IMT Institut de Mathématiques de Toulouse (France)
- Inria Strasbourg / Nancy - Grand Est, MIMESIS team (France)
- IMB Institut de mathématiques de Bordeaux (France)
- LJLL Laboratoire Jacques-Louis Lions, Paris (France)
-
Inria contact:
michel.duprez@inria.fr
-
Summary:
The goal of this project is to contribute to develop new trends in control theory for partial differential i equations, motivated by models arising in ecology and biology. We will in particular focus on two aspects. The first one is related to the constraints required on the controls or on the controlled trajectories, for instance positivity constraints, which appear S naturally when the state models a temperature. The second aspect concerns the questions of controllability and stabilization d of problems involving non-local operators, such as integral operators in space, in order to take into account phenomena depending on the total mass of the population for instance, or delay and memory terms, such as in visco-elastic fluids, often used to a model blood flows, or more generally models described by systems coupling hyperbolic and parabolic effects.
ANR - S-KELOID
-
Title:
Understanding Keloid Disorders: A multi-scale in vitro/in vivo/in silico approach towards digital twins of skin organoids on the chip.
-
Duration:
Oct. 2021 - Oct. 2025
-
Coordinator:
Eftimie RALUCA Stéphane BORDAS
-
Partners:
- Laboratoire de Mathématiques de Besançon (France)
- CHU de Besançon (France)
- FEMTO-ST, Besançon (France)
- Institut Mathématiques de Bourgogne, Dijon (France)
- Univertsité du Luxembourg (Luxembourg)
- Inria Strasbourg / Nancy - Grand Est, MIMESIS team (France)
-
Inria contact:
michel.duprez@inria.fr
-
Summary:
Mathematical and numerical modelling approaches allow us to integrate pathological processes that occur across different scales: cell, cells assembly and tissue. S-Keloid project aims to investigate the role of mechanical and inflammatory environmental factors on cells associated with keloid disorders. To address the multiscale aspects of these disorders, we plan to combine and integrate 3D in vitro cell models cultured under mechanical and biological stresses in order to mimic in vivo situation, and mathematical modelling of such systems. From applying experimental tests at tissue-scale and using a multiscale approach, the mechanical stress fields will be integrated into the 3D mathematical model. Parameter identification, optimization and their use across multiple scales will ensure the realism of the models and the quantitative and qualitative predictions of the keloid disorder.
Consortium - SOFA
-
Title:
Simulation Open Framework Architecture
-
Duration:
Nov. 2015 - ...
-
Summary:
MIMESIS is closely connected to the SOFA Consortium, created by Inria in November 2015 with the objective to support the SOFA community and encourage contributions from new SOFA users. The consortium should also be a way to better answer to the needs of academic or industrial partners. MIMESIS actively participates at the development of SOFA and contributes to the evolution of the framework. Moreover, MIMESIS also participates in an initiative aiming at verification and validation of codes and algorithms of SOFA.
National Exploratory Action - A/D Drugs
-
Title:
A/D Drugs
-
Duration:
April 2020 - December 2023
-
Coordinator:
Axel Hutt
-
Partners:
Didier Pinault (INSERM 1114)
-
Participants:
Axel Hutt, Joséphine Riedinger, Didier Pinault
-
Summary:
When it comes to treating mental disorders, the emergence of resistance to medication is a major problem. Replacing chemical medicine with digital medicine (neural stimulation) could be one way of getting around the problem. This is the concept behind "A/D Drugs" (PI : Axel Hutt) which will deploy a process of data assimilation and control in order to adapt stimulation to each patient. But this research is not without its risks - very little is known about the links between the effect of the chemical molecules and neurostimulation (www.inria.fr/en/ad-drugs-exploratory-action-aimed-optimising-neurostimulation).
10 Dissemination
Participants: Stéphane Cotin, Axel Hutt, Michel Duprez, Hadrien Courtecuisse.
10.1 Promoting scientific activities
10.1.1 Scientific events: selection
Chair of conference program committees
- Stephane Cotin program committee co-chair of MICCAI 2021.
- Axel Hutt was member of the Program Committee of the Organization for Computational Neuroscience (OCNS) 2021-2023.
Reviewer
- Stephane Cotin was area chair for IPCAI 2021. IPCAI is the international conference on Information Processing in Computer-Assisted Interventions.
- Axel Hutt was reviewer of abstracts submitted to BIOMESIP 2021.
10.1.2 Journal
Member of the editorial boards
- Axel Hutt was member of the editorial boards of “Frontiers in Systems Neuroscience”, “Frontiers in Applied Mathematics and Statistics”, “Journal of Clinical Medicine”
- Axel Hutt is chief Section Editor of “Frontiers in Applied Mathematics and Statistics - Dynamical Systems” since 2016.
Reviewer - reviewing activities
- Michel Duprez has reviewed manuscripts for "Nonlinear Analysis: Real World Applications", "biomath" and "MCRF"
- Axel Hutt has reviewed manuscripts for “NeuroImage” and “Cognitive Neurodynamics”
10.1.3 Invited talks
- Michel Duprez gave the talk “Séminaire MOCO”, Institut de Recherche Mathématique Avancée in Strasbourg (Feb. 2021)
- Michel Duprez gave the talk “Café virtuel, Inria Nancy Grand Est” in virtual (March 2021)
- Michel Duprez gave the talk “14th WCCM & ECCOMAS Congress” in virtual (June 2021)
- Michel Duprez gave the talk Workshop “New methods for biological control of the arboviruses” in Fréjus (Sept. 2021)
- Michel Duprez gave the talk “Modélisation TIS” in La Réunion (Nov. 2021)
- Stéphane Cotin gave a talk in the keynote lecture, “Technological innovations for the precision medicine of tomorrow”, Politecnico di Milano (June 2021)
- Stéphane Cotin gave a talk in the “NAFEMS World Congress 2021” (Oct. 2021)
- Axel Hutt gave a talk in the INRIA CaféIn: “A/D Drugs” (February 2021)
- Axel Hutt gave a talk in the conference “Cognitive Neuroscience 2021” (February 2021)
- Axel Hutt gave a talk in the conference “Dynamics Days 2021” (August 2021)
- Axel Hutt gave a talk in the workshop “EnKF workshop 2021” (June 2021)
- Axel Hutt gave a talk in the conference “Modern Stochastics” (May 2021)
- Pedro Henrique Suruagy Perrusi gave a talk in the conference “Eurographics 2021” (May 2021)
- Paul Baksic gave a talk in “Sofa week 2021” (Nov. 2021)
- Pedro Henrique Suruagy Perrusi gave a talk in the conference “ICRA 2021” (June 2021)
10.1.4 Scientific expertise
- Stéphane Cotin serves as scientific advisor for the company InSimo (www.insimo.com)
- Axel Hutt was reviewer of abstracts sent to conference “BIOMESISP 2021”
- Axel Hutt was reviewer of ANR grant proposal
10.1.5 Research administration
- Stephane Cotin has organized a summer scool on AI.
- Axel Hutt has managed the Action Exploratoire “A/D Drugs”
10.2 Teaching - Supervision - Juries
10.2.1 Teaching
- License: Michel Duprez, Numerical analysis techniques, 17h+15h, L3, University of Strasbourg
- Master: Michel Duprez, Incertitude quantification, 14h, M2, University of Strasbourg
- License: Axel Hutt, Anglais, 2h, L3, University of Strasbourg
10.2.2 Supervision
- Master 1: Killian Vuillemot, "Développement de Méthodes Éléments Finis non-conformes adaptées à la chirurgie assistée par ordinateur", supervised by Michel Duprez and Stéphane Cotin (Summer 2021)
- Master 1: Vincent Italiano, Stratégie de contrôle d'épidémies", supervised by Michel Duprez and Yannick Privat (Summer 2021)
- Master 2: Philippe Pincon, "Mise à jour de factorisations de Cholesky creuses dans le contexte de la simulation chirurgicale interactive", supervised by Michel Duprez and François Jourdes (Summer 2021)
- PhD starting: Francois Lecomte, "Development of a CT Scan to Radiography real-time registration method using Deep Learning.", started 01/10/2021, supervized by Stephane Cotin.
- PhD starting: Valentina Scarponi, "Catheter insertion", started 01/10/2021, supervized by Stéphane Cotin, Michel Duprez and Florent Nageotte.
- PhD starting: Thomas Wahl, "Model-based closed-loop neurostimulation with application to schizophrenia", started 01/10/2021, supervized by Axel Hutt and Michel Duprez.
- PhD in progress: Guillaume Mestdagh, "Real-Time Tumor Tracking as an Optimization Problem", started 01/09/2019,co-supervised by Stéphane Cotin.
- PhD in progress: Ziqiu Zeng, "SPERRY - Supervised Robotic Surgery - Application to needle inser- tion", started 01/07/2019, supervised by Hadrien Courtecuisse.
- PhD in progress: Joséphine Riedinger, "Closed-Loop Transcranial Electric Stimulation of Neural Networks in a Rodent Model of Psychosis Transition", co-supervized by Axel Hutt
- PhD in progress: Paul Baksic, "Robotic assistance for percutaneous surgical interventions in deformable structures – Application to radiofrequency ablation", started 01/10/2018, supervised by Hadrien Courtecuisse.
- PhD in progress: Alban Odot, "speeding up / improving Finite Element methods using Deep learning", started 01/10/2019, supervised by Stephane Cotin.
- PhD ended: Sergei Nikolaev, "Identification and characterization of boundary conditions for patient specific biomechanical simulation", started 01/05/2017, supervised by Stephane Cotin, co-supervised by Igor Pe- terlik and Hadrien Courtecuisse, defended 15/09/2021.
- PhD ended: Nicolas Golse, "Modélisation anatomique, hémodynamique et physiologique en chirurgie hépatique", supervised by Eric Vibert, Irene Vignon Clementel and Stephane Cotin, defended 20/09/2021.
10.2.3 Juries
- Stephane Cotin was member of the jury for the HDR thesis of Hadrien COURTECUISSE (Univ Strasbourg)
10.3 Popularization
10.3.1 Articles and contents
- Article HiPerNav : www.inria.fr/en/surgery-liver-project-hipernav-modeling-simulation
10.3.2 Interventions
- Jan. 2021, Irene Vignon-Clementel and Stéphane Cotin - Le numérique, appui au contrôle de risque en chirurgie : www.youtube.com/watch-Vignon-Clementel-cotin
11 Scientific production
11.1 Major publications
- 1 miscOptimal control strategies for the sterile mosquitoes technique.2020
- 2 articleHow Best Can Finite-Time Social Distancing Reduce Epidemic Final Size?Journal of Theoretical BiologyFebruary 2021, 110557
- 3 articleOptimal Immunity Control and Final Size Minimization by Social Distancing for the SIR Epidemic Model.Journal of Optimization Theory and Applications18922021, 408--436
- 4 articleControlling the error on target motion through real-time mesh adaptation: applications to Deep Brain Stimulation.International Journal for Numerical Methods in Biomedical EngineeringDecember 2017, 1-26
- 5 miscφ-FEM: an efficient simulation tool using simple meshes for problems in structure mechanics and heat transfer.October 2021
- 6 articleReal-time simulation of contact and cutting of heterogeneous soft-tissues.Medical Image AnalysisFebruary 2014, 20
- 7 bookMedical Image Computing and Computer Assisted Intervention – MICCAI 2021 - Parts I to VIII.12901 to 12908Springer2021
-
8
articleA new
-FEM approach for problems with natural boundary conditions.Numerical Methods for Partial Differential Equations2022 - 9 articleBiomechanics-based graph matching for augmented CT-CBCT.IJCARSApr 2018, URL: https://doi.org/10.1007/s11548-018-1755-1
- 10 articleElastic Registration Based on Compliance Analysis and Biomechanical Graph Matching.Annals of Biomedical Engineering2019, 1--16
- 11 thesisAnatomical, Hemodynamic and Physiological Modeling in Liver Surgery.Université Paris-SaclaySeptember 2021
- 12 articleAugmented Reality during Open Liver Surgery Using a Markerless Non-rigid Registration System.Journal of Gastrointestinal SurgeryJanuary 2020
- 13 articleVision-based Force Feedback Estimation for Robot-assisted Surgery using Instrument-constrained Biomechanical 3D Maps.IEEE Robotics and Automation LettersProceedings of International Conference on Robotics and Automation ICRA 2018February 2018
- 14 articleArousal fluctuations govern oscillatory transitions between dominant gamma and alpha occipital activity during eyes open/closed conditions.Brain Topography: a Journal of Cerebral Function and Dynamics2021
- 15 articleBlood vessel modeling for interactive simulation of interventional neuroradiology procedures.Medical Image Analysis35January 2017, 685 - 698
- 16 inproceedingsA Combined Simulation & Machine Learning Approach for Image-based Force Classification during Robotized Intravitreal Injections.MICCAI 2018 - 21st International Conference on Medical Image Computing and Computer Assisted InterventionGranada, SpainSeptember 2018
- 17 articleSimulation of hyperelastic materials in real-time using deep learning.Medical Image Analysis592019, 101569
- 18 inbookData-driven simulation for augmented surgery.132Developments and Novel Approaches in Biomechanics and MetamaterialsJuly 2020, 71-96
- 19 articleBrain-shift compensation using intraoperative ultrasound and constraint-based biomechanical simulation.Medical Image Analysis402017, 133 - 153
- 20 miscDeepPhysics: a physics aware deep learning framework for real-time simulation.September 2021
- 21 articleFast Elastic Registration of Soft Tissues under Large Deformations.Medical Image Analysis2017
- 22 inproceedingsCapturing Deformations of Interacting Non-rigid Objects Using RGB-D Data.IROS 2018 - IEEE/RSJ International Conference on Intelligent Robots and SystemsMadrid, SpainOctober 2018
- 23 articleFiloGen: A Model-Based Generator of Synthetic 3D Time-Lapse Sequences of Single Motile Cells with Growing and Branching Filopodia.IEEE Transactions on Medical ImagingJune 2018, 1-12
- 24 inproceedingsIntra-operative Update of Boundary Conditions for Patient-specific Surgical Simulation.MICCAI - 24th International Conference on Medical Image Computing and Computer Assisted InterventionStrasbourg, FranceSeptember 2021
- 25 articleInteractive training system for interventional electrocardiology procedures.Medical image analysis352017, 225--237
11.2 Publications of the year
International journals
International peer-reviewed conferences
Conferences without proceedings
Doctoral dissertations and habilitation theses
Reports & preprints