Section: Research Program

Data-driven simulations

Data-driven simulation has been a recent area of research in our team (see Fig. 3). We have demonstrated that it has the potential to bridge the gap between medical imaging and clinical routine by adapting pre-operative data to the time of the procedure. In the areas of non-rigid registration and augmented reality during surgery, we have demonstrated the benefit of our physics-based approaches with several key publications in major conferences (MICCAI, CVPR, IPCAI, ISMAR).

We have continued this work with an emphasis on robustness to uncertainty and outliers in the information extracted in real-time from image data, as well as real-time parameter estimation. This is currently done by combining Bayesian methods with advanced physics-based methods to handle uncertainties in image-driven simulations (MICCAI 2017, CVCS 2018).

Finally, Bayesian or similar methods require to perform a large amount of simulations to sample the domain space, even when using efficient methods such as Reduced Order Unscented Kalman Filters. For this reason, we are investigating the use of neural networks to perform predictions instead of using full numerical simulations. Our latest paper [22] at MICCAI 2019 shows it is possible to teach a neural network from numerical simulations and predict, with good accuracy, the deformation of an organ.

Figure 3. Real-time deformation of a virtual liver according to tissue motion tracked in laparoscopic images.