2025Activity reportProject-Team‌DATAMOVE

4.1.1‌ Algorithms

The most common‌​‌ batch scheduling policy is​​ to consider the jobs​​​‌ according to the First‌ Come First Served order‌​‌ (FCFS) with backfilling (BF).​​ BF is the most​​​‌ widely used policy due‌ to its easy and‌​‌ robust implementation and known​​ benefits such as high​​​‌ system utilization. It is‌ well-known that this strategy‌​‌ does not optimize any​​ sophisticated function, but it​​​‌ is simple to implement‌ and it guarantees that‌​‌ there is no starvation​​ (i.e. every job will​​​‌ be scheduled at some‌ moment).

More advanced algorithms‌​‌ are seldom used on​​ production platforms due to​​​‌ both the gap between‌ theoretical models and practical‌​‌ systems and speed constraints.​​ When looking at theoretical​​​‌ scheduling problems, the generally‌ accepted goal is to‌​‌ provide polynomial algorithms (in​​ the number of submitted​​​‌ jobs and the number‌ of involved computing units).‌​‌ However, with millions of​​ processing cores where every​​​‌ process and data transfer‌ have to be individually‌​‌ scheduled, polynomial algorithms are​​ prohibitive as soon as​​​‌ the polynomial degree is‌ too large. The model‌​‌ of parallel tasks simplifies​​ this problem by bundling​​​‌ many threads and communications‌ into single boxes, either‌​‌ rigid, rectangular or malleable.​​​‌ Especially malleable tasks capture​ the dynamicity of the​‌ execution. Yet these models​​ are ill-adapted to heterogeneous​​​‌ platforms, as the running​ time depends on more​‌ than simply the number​​ of allotted resources, and​​​‌ some of the common​ underlying assumptions on the​‌ speed-up functions (such as​​ monotony or concavity) are​​​‌ most often only partially​ verified.

In practice, the​‌ job execution times depend​​ on their allocation (due​​​‌ to communication interferences and​ heterogeneity in both computation​‌ and communication), while theoretical​​ models of parallel jobs​​​‌ usually consider jobs as​ black boxes with a​‌ fixed (maximum) execution time.​​ Though interesting and powerful,​​​‌ the classical models (namely,​ synchronous PRAM model, delay,​‌ LogP) and their variants​​ (such as hierarchical delay),​​​‌ are not well-suited to​ large scale parallelism on​‌ platforms where the cost​​ of moving data is​​​‌ significant, non uniform and​ may change over time.​‌ Recent studies are still​​ refining such models in​​​‌ order to take into​ account communication contentions more​‌ accurately while remaining tractable​​ enough to provide a​​​‌ useful tool for algorithm​ design.

Today, all algorithms​‌ in use in production​​ systems are oblivious to​​​‌ communications. One of our​ main goals is to​‌ design a new generation​​ of scheduling algorithms fitting​​​‌ more closely job schedules​ according to platform topologies​‌.

4.1.2 Locality Aware​​ Allocations

Recently, we developed​​​‌ modifications of the standard​ back-filling algorithm taking into​‌ account platform topologies. The​​ proposed algorithms take into​​​‌ account locality and contiguity​ in order to hide​‌ communication patterns within parallel​​ tasks. The main result​​​‌ here is to establish​ good lower bounds and​‌ small approximation ratios for​​ policies respecting the locality​​​‌ constraints. The algorithms work​ in an online fashion,​‌ improving the global behavior​​ of the system while​​​‌ still keeping a low​ running time. These improvements​‌ rely mainly on our​​ past experience in designing​​​‌ approximation algorithms. Instead of​ relying on complex networking​‌ models and communication patterns​​ for estimating execution times,​​​‌ the communications are disconnected​ from the execution time.​‌ Then, the scheduling problem​​ leads to a trade-off:​​​‌ optimizing locality of communications​ on one side and​‌ a performance objective (like​​ the makespan or stretch)​​​‌ on the other side.​

In the perspective of​‌ taking care of locality,​​ other ongoing works include​​​‌ the study of schedulers​ for platforms whose interconnection​‌ network is a static​​ structured topology (like the​​​‌ 3D-torus of the BlueWaters​ platform we work on​‌ in collaboration with the​​ Argonne National Laboratory). One​​​‌ main characteristic of this​ 3D-torus platform is to​‌ provide I/O nodes at​​ specific locations in the​​​‌ topology. Applications generate and​ access specific data and​‌ are thus bounded to​​ specific I/O nodes. Resource​​​‌ allocations are constrained in​ a strong and unusual​‌ way. This problem is​​ close for actual hierarchical​​​‌ platforms. The scheduler needs​ to compute a schedule​‌ such that I/O nodes​​ requirements are filled for​​​‌ each application while at​ the same time avoiding​‌ communication interferences. Moreover, extra​​ constraints can arise for​​​‌ applications requiring accelerators that​ are gathered on the​‌ nodes at the edge​​ of the network topology.​​

While current results are​​​‌ encouraging, they are however‌ limited in performance by‌​‌ the low amount of​​ information available to the​​​‌ scheduler. We look forward‌ to extend ongoing work‌​‌ by progressively increasing application​​ and network knowledge (by​​​‌ technical mechanisms like profiling‌ or monitoring or by‌​‌ more sophisticated methods like​​ learning). It is also​​​‌ important to anticipate on‌ application resource usage in‌​‌ terms of compute units,​​ memory as well as​​​‌ network and I/Os to‌ efficiently schedule a mix‌​‌ of applications with different​​ profiles. For instance, a​​​‌ simple solution is to‌ partition the jobs as‌​‌ "communication intensive" or "low​​ communications". Such a tag​​​‌ could be achieved by‌ the users them selves‌​‌ or obtained by learning​​ techniques. We could then​​​‌ schedule low communications jobs‌ using leftover spaces while‌​‌ taking care of high​​ communication jobs. More sophisticated​​​‌ options are possible, for‌ instance those that use‌​‌ more detailed communication patterns​​ and networking models. Such​​​‌ options would leverage the‌ work proposed in Section‌​‌ 4.2 for gathering application​​ traces.

4.1.3 Data-Centric Processing​​​‌

Exascale computing is shifting‌ away from the traditional‌​‌ compute-centric models to a​​ more data-centric one. This​​​‌ is driven by the‌ evolving nature of large‌​‌ scale distributed computing, no​​ longer dominated by pure​​​‌ computations but also by‌ the need to handle‌​‌ and analyze large volumes​​ of data. These data​​​‌ can be large databases‌ of results, data streamed‌​‌ from a running application​​ or another scientific instrument​​​‌ (collider for instance). These‌ new workloads call for‌​‌ specific resource allocation strategies.​​

Data movements and storage​​​‌ are expected to be‌ a major energy and‌​‌ performance bottleneck on next​​ generation platforms. Storage architectures​​​‌ are also evolving, the‌ standard centralized parallel file‌​‌ system being complemented with​​ local persistent storage (Burst​​​‌ Buffers, NVRAM). Thus, one‌ data producer can stage‌​‌ data on some nodes'​​ local storage, requiring to​​​‌ schedule close by the‌ associated analytics tasks to‌​‌ limit data movements. This​​ kind of configuration, often​​​‌ referred as in-situ analytics‌, is expected to‌​‌ become common as it​​ enables to switch from​​​‌ the traditional I/O intensive‌ workflow (batch-processing followed by‌​‌ post mortem analysis and​​ visualization) to a more​​​‌ storage conscious approach where‌ data are processed as‌​‌ closely as possible to​​ where and when they​​​‌ are produced (in-situ processing‌ is addressed in details‌​‌ in section 4.3).​​ By reducing data movements​​​‌ and scheduling the extra‌ processing on resources not‌​‌ fully exploited yet, in-situ​​ processing is expected to​​​‌ have also a significant‌ positive energetic impact. Analytics‌​‌ codes can be executed​​ in the same nodes​​​‌ than the application, often‌ on dedicated cores commonly‌​‌ called helper cores, or​​ on dedicated nodes called​​​‌ stagging nodes. The results‌ are either forwarded to‌​‌ the users for visualization​​ or saved to disk​​​‌ through I/O nodes. In-situ‌ analytics can also take‌​‌ benefit of node local​​ disks or burst buffers​​​‌ to reduce data movements.‌ Future job scheduling strategies‌​‌ should take into account​​ in-situ processes in addition​​​‌ to the job allocation‌ to optimize both energy‌​‌ consumption and execution time.​​​‌ On the one hand,​ this problem can be​‌ reduced to an allocation​​ problem of extra asynchronous​​​‌ tasks to idle computing​ units. But on the​‌ other hand, embedding analytics​​ in applications brings extra​​​‌ difficulties by making the​ application more heterogeneous and​‌ imposing more constraints (data​​ affinity) on the required​​​‌ resources. Thus, the main​ point here is to​‌ develop efficient algorithms for​​ dealing with heterogeneity without​​​‌ increasing the global computational​ cost.

4.1.4 Learning

Another​‌ important issue is to​​ adapt the job management​​​‌ system to deal with​ the bad effects of​‌ uncertainties, which may be​​ catastrophic in large scale​​​‌ heterogeneous HPC platforms (jobs​ delayed arbitrarly far or​‌ jobs killed). A natural​​ question is then: is​​​‌ it possible to have​ a good estimation of​‌ the job and platform​​ parameters in order to​​​‌ be able to obtain​ a better scheduling ?​‌ Many important parameters (like​​ the number or type​​​‌ of required resources or​ the estimated running time​‌ of the jobs) are​​ asked to the users​​​‌ when they submit their​ jobs. However, some of​‌ these values are not​​ accurate and in many​​​‌ cases, they are not​ even provided by the​‌ end-users. In DataMove, we​​ propose to study new​​​‌ methods for a better​ prediction of the characteristics​‌ of the jobs and​​ their execution in order​​​‌ to improve the optimization​ process. In particular, the​‌ methods well-studied in the​​ field of big data​​​‌ (in supervised Machine Learning,​ like classical regression methods,​‌ Support Vector Methods, random​​ forests, learning to rank​​​‌ techniques or deep learning)​ could and must be​‌ used to improve job​​ scheduling in large scale​​​‌ HPC platforms. This topic​ received a great attention​‌ recently in the field​​ of parallel and distributed​​​‌ processing. A preliminary study​ has been done recently​‌ by our team with​​ the target of predicting​​​‌ the job running times​ (called wall times). We​‌ succeeded to improve significantly​​ in average the reference​​​‌ EASY Back Filling algorithm​ by estimating the wall​‌ time of the jobs,​​ however, this method leads​​​‌ to big delay for​ the stretch of few​‌ jobs. Even if we​​ succeed in determining more​​​‌ precisely hidden parameters, like​ the wall time of​‌ the jobs, this is​​ not enough to determine​​​‌ an optimized solution. The​ shift is not only​‌ to learn on dedicated​​ parameters but also on​​​‌ the scheduling policy. The​ data collected from the​‌ accounting and profiling of​​ jobs can be used​​​‌ to better understand the​ needs of the jobs​‌ and through learning to​​ propose adaptations for future​​​‌ submissions. The goal is​ to propose extensions to​‌ further improve the job​​ scheduling and improve the​​​‌ performance and energy efficiency​ of the application. For​‌ instance preference learning may​​ enable to compute on-line​​​‌ new priorities to back-fill​ the ready jobs.

4.1.5​‌ Multi-objective Optimization

Several optimization​​ questions that arise in​​​‌ allocation and scheduling problems​ lead to the study​‌ of several objectives at​​ the same time. The​​​‌ goal is then not​ a single optimal solution,​‌ but a more complicated​​ mathematical object that captures​​ the notion of trade-off.​​​‌ In broader terms, the‌ goal of multi-objective optimization‌​‌ is not to externally​​ arbitrate on disputes between​​​‌ entities with different goals,‌ but rather to explore‌​‌ the possible solutions to​​ highlight the whole range​​​‌ of interesting compromises. A‌ classical tool for studying‌​‌ such multi-objective optimization problems​​ is to use Pareto​​​‌ curves. However, the‌ full description of the‌​‌ Pareto curve can be​​ very hard because of​​​‌ both the number of‌ solutions and the hardness‌​‌ of computing each point.​​ Addressing this problem will​​​‌ opens new methodologies for‌ the analysis of algorithms.‌​‌

To further illustrate this​​ point here are three​​​‌ possible case studies with‌ emphasis on conflicting interests‌​‌ measured with different objectives.​​ While these cases are​​​‌ good representatives of our‌ HPC context, there are‌​‌ other pertinent trade-offs we​​ may investigate depending on​​​‌ the technology evolution in‌ the coming years. This‌​‌ enumeration is certainly not​​ limitative.

Energy versus Performance​​​‌. The classical scheduling‌ algorithms designed for the‌​‌ purpose of performance can​​ no longer be used​​​‌ because performance and energy‌ are contradictory objectives to‌​‌ some extent. The scheduling​​ problem with energy becomes​​​‌ a multi-objective problem in‌ nature since the energy‌​‌ consumption should be considered​​ as equally important as​​​‌ performance at exascale. A‌ global constraint on energy‌​‌ could be a first​​ idea for determining trade-offs​​​‌ but the knowledge of‌ the Pareto set (or‌​‌ an approximation of it)​​ is also very useful.​​​‌

Administrators versus application developers‌. Both are naturally‌​‌ interested in different objectives:​​ In current algorithms, the​​​‌ performance is mainly computed‌ from the point of‌​‌ view of administrators, but​​ the users should be​​​‌ in the loop since‌ they can give useful‌​‌ information and help to​​ the construction of better​​​‌ schedules. Hence, we face‌ again a multi-objective problem‌​‌ where, as in the​​ above case, the approximation​​​‌ of the Pareto set‌ provides the trade-off between‌​‌ the administrator view and​​ user demands. Moreover, the​​​‌ objectives are usually of‌ the same nature. For‌​‌ example, max stretch and​​ average stretch are two​​​‌ objectives based on the‌ slowdown factor that can‌​‌ interest administrators and users,​​ respectively. In this case​​​‌ the study of the‌ norm of stretch can‌​‌ be also used to​​ describe the trade-off (recall​​​‌ that the $L_{\mathrm{1‌}}$-norm corresponds to the‌​‌ average objective while the​​ $L_{\infty }$-norm to​​​‌ the max objective). Ideally,‌ we would like to‌​‌ design an algorithm that​​ gives good approximate solutions​​​‌ at the same time‌ for all norms. The‌​‌ $L_2$ or ${\mathrm{L}}_3$-norm are useful​​​‌ since they describe the‌ performance of the whole‌​‌ schedule from the administrator​​ point of view as​​​‌ well as they provide‌ a fairness indication to‌​‌ the users. The hard​​ point here is to​​​‌ derive theoretical analysis for‌ such complicated tools.

In‌​‌ general, resource augmentation can​​ explain the intuitive good​​​‌ behavior of some greedy‌ algorithms while, more interestingly,‌​‌ it can give ideas​​ for new algorithms. For​​​‌ example, in the rejection‌ context we could dedicate‌​‌ a small number of​​​‌ nodes for the usually​ problematic rejected jobs. Some​‌ initial experiments show that​​ this can lead to​​​‌ a schedule for the​ remaining jobs that is​‌ very close to the​​ optimal one.

4.2.1 Workload Traces​​​‌ with Resource Consumption

Workload​ traces are the base​‌ element to capture the​​ behavior of complete systems​​​‌ composed of submitted jobs,​ running applications, and operating​‌ tools. These traces must​​ be obtained on production​​​‌ platforms to provide relevant​ and representative data. To​‌ get a better understanding​​ of the use of​​​‌ such systems, we need​ to look at both,​‌ how the jobs interact​​ with the job management​​​‌ system, and how they​ use the allocated resources.​‌ We propose a general​​ workload trace format that​​​‌ adds jobs resource consumption​ to the commonly used​‌ Standard Workload Format workload​​ trace format. This requires​​​‌ to instrument the platforms,​ in particular to trace​‌ resource consumptions like CPU,​​ data movements at memory,​​​‌ network and I/O levels,​ with an acceptable performance​‌ impact. In a previous​​ work we studied and​​​‌ proposed a dedicated job​ monitoring tool whose impact​‌ on the system has​​ been measured as lightweight​​​‌ (0.35$\%$ speed-down) with​ a 1 minute sampling​‌ rate. Other tools also​​ explore job monitoring, like​​​‌ TACC Stats. A unique​ feature from our tool​‌ is its ability to​​ monitor distinctly jobs sharing​​​‌ common nodes.

Collected workload​ traces with jobs resource​‌ consumption will be publicly​​ released and serve to​​​‌ provide data for works​ presented in Section 4.1​‌. The trace analysis​​ is expected to give​​​‌ valuable insights to define​ models encompassing complex behaviours​‌ like network topology sensitivity,​​ network congestion and resource​​​‌ interferences.

4.2.2 Simulation

Simulations​ of large scale systems​‌ are faster by multiple​​ orders of magnitude than​​ real experiments. Unfortunately, replacing​​​‌ experiments with simulations is‌ not as easy as‌​‌ it may sound, as​​ it brings a host​​​‌ of new problems to‌ address in order to‌​‌ ensure that the simulations​​ are closely approximating the​​​‌ execution of typical workloads‌ on real production clusters.‌​‌ Most of these problems​​ are actually not directly​​​‌ related to scheduling algorithms‌ assessment, in the sense‌​‌ that the workload and​​ platform models should be​​​‌ defined independently from the‌ algorithm evaluations, in order‌​‌ to ensure a fair​​ assessment of the algorithms'​​​‌ strengths and weaknesses. These‌ research topics (namely platform‌​‌ modeling, job models and​​ simulator calibration) are addressed​​​‌ in the other subsections.‌

We developed an open‌​‌ source platform simulator within​​ DataMove (in conjunction with​​​‌ the OAR development team)‌ to provide a widely‌​‌ distributable test bed for​​ reproducible scheduling algorithm evaluation.​​​‌ Our simulator, named Batsim,‌ allows to simulate the‌​‌ behavior of a computational​​ platform executing a workload​​​‌ scheduled by any given‌ scheduling algorithm. To obtain‌​‌ sound simulation results and​​ to broaden the scope​​​‌ of the experiments that‌ can be done thanks‌​‌ to Batsim, we did​​ not chose to create​​​‌ a (necessarily limited) simulator‌ from scratch, but instead‌​‌ to build on top​​ of the SimGrid simulation​​​‌ framework.

To be open‌ to as many batch‌​‌ schedulers as possible, Batsim​​ decouples the platform simulation​​​‌ and the scheduling decisions‌ in two clearly-separated software‌​‌ components communicating through a​​ complete and documented protocol.​​​‌ The Batsim component is‌ in charge of simulating‌​‌ the computational resources behaviour​​ whereas the scheduler component​​​‌ is in charge of‌ taking scheduling decisions. The‌​‌ scheduler component may be​​ both a resource and​​​‌ a job management system.‌ For jobs, scheduling decisions‌​‌ can be to execute​​ a job, to delay​​​‌ its execution or simply‌ to reject it. For‌​‌ resources, other decisions can​​ be taken, for example​​​‌ to change the power‌ state of a machine‌​‌ i.e. to change its​​ speed (in order to​​​‌ lower its energy consumption)‌ or to switch it‌​‌ on or off. This​​ separation of concerns also​​​‌ enables interfacing with potentially‌ any commercial RJMS, as‌​‌ long as the communication​​ protocol with Batsim is​​​‌ implemented. A proof of‌ concept is already available‌​‌ with the OAR RJMS.​​

Using this test bed​​​‌ opens new research perspectives.‌ It allows to test‌​‌ a large range of​​ platforms and workloads to​​​‌ better understand the real‌ behavior of our algorithms‌​‌ in a production setting.​​ In turn, this opens​​​‌ the possibility to tailor‌ algorithms for a particular‌​‌ platform or application, and​​ to precisely identify the​​​‌ possible shortcomings of the‌ theoretical models used.

4.2.3‌​‌ Job and Platform Models​​

The central purpose of​​​‌ the Batsim simulator is‌ to simulate job behaviors‌​‌ on a given target​​ platform under a given​​​‌ resource allocation policy. Depending‌ on the workload, a‌​‌ significant number of jobs​​ are parallel applications with​​​‌ communications and file system‌ accesses. It is not‌​‌ conceivable to simulate individually​​ all these operations for​​​‌ each job on large‌ plaforms with their associated‌​‌ workload due to implied​​​‌ simulation complexity. The challenge​ is to define a​‌ coarse grain job model​​ accurate enough to reproduce​​​‌ parallel application behavior according​ to the target platform​‌ characteristics. We will explore​​ models similar to the​​​‌ BSP (Bulk Synchronous Program)​ approach that decomposes an​‌ application in local computation​​ supersteps ended by global​​​‌ communications and a global​ synchronization. The model parameters​‌ will be established by​​ means of trace analysis​​​‌ as discussed previously, but​ also by instrumenting some​‌ parallel applications to capture​​ communication patterns. This instrumentation​​​‌ will have a significant​ impact on the concerned​‌ application performance, restricting its​​ use to a few​​​‌ applications only. There are​ a lot of recurrent​‌ applications executed on HPC​​ platform, this fact will​​​‌ help to reduce the​ required number of instrumentations​‌ and captures. To assign​​ each job a model,​​​‌ we are considering to​ adapt the concept of​‌ application signatures as proposed​​ in. Platform models and​​​‌ their calibration are also​ required. Large parts of​‌ these models, like those​​ related to network, are​​​‌ provided by Simgrid. Other​ parts as the filesystem​‌ and energy models are​​ comparatively recent and will​​​‌ need to be enhanced​ or reworked to reflect​‌ the HPC platform evolutions.​​ These models are then​​​‌ generally calibrated by running​ suitable benchmarks.

4.2.4 Emulation​‌ and Reproducibility

The use​​ of coarse models in​​​‌ simulation implies to set​ aside some details. This​‌ simplification may hide system​​ behaviors that could impact​​​‌ significantly and negatively the​ metrics we try to​‌ enhance. This issue is​​ particularly relevant when large​​​‌ scale platforms are considered​ due to the impossibility​‌ to run tests at​​ nominal scale on these​​​‌ real platforms. A common​ approach to circumvent this​‌ issue is the use​​ of emulation techniques to​​​‌ reproduce, under certain conditions,​ the behavior of large​‌ platforms on smaller ones.​​ Emulation represents a natural​​​‌ complement to simulation by​ allowing to execute directly​‌ large parts of the​​ actual evaluated software and​​​‌ system, but at the​ price of larger compute​‌ times and a need​​ for more resources. The​​​‌ emulation approach was chosen​ in to compare two​‌ job management systems from​​ workload traces of the​​​‌ CURIE supercomputer (80000 cores).​ The challenge is to​‌ design methods and tools​​ to emulate with sufficient​​​‌ accuracy the platform and​ the workload (data movement,​‌ I/O transfers, communication, applications​​ interference). We will also​​​‌ intend to leverage emulation​ tools like Distem from​‌ the MADYNES team. It​​ is also important to​​​‌ note that the Batsim​ simulator also uses emulation​‌ techniques to support the​​ core scheduling module from​​​‌ actual RJMS. But the​ integration level is not​‌ the same when considering​​ emulation for larger parts​​​‌ of the system (RJMS,​ compute node, network and​‌ filesystem).

Replaying traces implies​​ to prepare and manage​​​‌ complex software stacks including​ the OS, the resource​‌ management system, the distributed​​ filesystem and the applications​​​‌ as well as the​ tools required to conduct​‌ experiments. Preparing these stacks​​ generate specific issues, one​​​‌ of the major one​ being the support for​‌ reproducibility. We propose to​​ further develop the concept​​ of reconstructability to improve​​​‌ experiment reproducibility by capturing‌ the build process of‌​‌ the complete software stack.​​ This approach ensures reproducibility​​​‌ over time better than‌ other ways by keeping‌​‌ all data (original packages,​​ build recipe and Kameleon​​​‌ engine) needed to build‌ the software stack.

In‌​‌ this context, the Grid'5000​​ (see Sec. 7.2)​​​‌ experimentation infrastructure that gives‌ users the control on‌​‌ the complete software stack​​ is a crucial tool​​​‌ for our research goals.‌ We will pursue our‌​‌ strong implication in this​​ infrastructure.

4.3.1 Programming Model and​​ Software Architecture

In situ​​​‌ creates a tighter loop‌ between the scientist and‌​‌ her/his simulation. As such,​​ an in situ framework​​​‌ needs to be flexible‌ to let the user‌​‌ define and deploy its​​ own set of analysis.​​​‌ A manageable flexibility requires‌ to favor simplicity and‌​‌ understandability, while still enabling​​ an efficient use of​​​‌ parallel resources. Visualization libraries‌ like VTK or Visit,‌​‌ as well as domain​​ specific environments like VMD​​​‌ have initially been developed‌ for traditional post-mortem data‌​‌ analysis. They have been​​ extended to support in​​​‌ situ processing with some‌ simple resource allocation strategies‌​‌ but the level of​​ performance, flexibility and ease​​​‌ of use that is‌ expected requires to rethink‌​‌ new environments. There is​​ a need to develop​​​‌ a middleware and programming‌ environment taking into account‌​‌ in its fundations this​​ specific context of high​​​‌ performance scientific analytics.

Similar‌ needs for new data‌​‌ processing architectures occurred for​​​‌ the emerging area of​ Big Data Analytics, mainly​‌ targeted to web data​​ on cloud-based infrastructures. Google​​​‌ Map/Reduce and its successors​ like Spark or Stratosphere/Flink​‌ have been designed to​​ match the specific context​​​‌ of efficient analytics for​ large volumes of data​‌ produced on the web,​​ on social networks, or​​​‌ generated by business applications.​ These systems have mainly​‌ been developed for cloud​​ infrastructures based on commodity​​​‌ architectures. They do not​ leverage the specifics of​‌ HPC infrastructures. Some preliminary​​ adaptations have been proposed​​​‌ for handling scientific data​ in a HPC context.​‌ However, these approaches do​​ not support in situ​​​‌ processing.

Following the initial​ development of FlowVR, our​‌ middleware for in situ​​ processing, we will pursue​​​‌ our effort to develop​ a programming environment and​‌ software architecture for high​​ performance scientific data analytics.​​​‌ Like FlowVR, the map/reduce​ tools, as well as​‌ the machine learning frameworks​​ like TensorFlow, adopted a​​​‌ dataflow graph for expressing​ analytics pipe-lines. We are​‌ convinced that this dataflow​​ approach is both easy​​​‌ to understand and yet​ expresses enough concurrency to​‌ enable efficient executions. The​​ graph description can be​​​‌ compiled towards lower level​ representations, a mechanism that​‌ is intensively used by​​ Stratosphere/Flink for instance. Existing​​​‌ in situ frameworks inherit​ from the HPC way​‌ of programming with a​​ thiner software stack and​​​‌ a programming model close​ to the machine. Though​‌ this approach enables to​​ program high performance applications,​​​‌ this is usually too​ low level to enable​‌ the scientist to write​​ its analysis pipe-line in​​​‌ a short amount of​ time. The data model,​‌ i.e. the data semantics​​ level accessible at the​​​‌ framework level for error​ check and optimizations, is​‌ also a fundamental aspect​​ of such environments. The​​​‌ key/value store has been​ adopted by all map/reduce​‌ tools. Except in some​​ situations, it cannot be​​​‌ adopted as such for​ scientific data. Results from​‌ numerical simulations are often​​ more structured than web​​​‌ data, associated with acceleration​ data structures to be​‌ processed efficiently. We will​​ investigate data models for​​​‌ scientific data building on​ existing approaches like Adios​‌ or DataSpaces.

4.3.2 Resource​​ Sharing

To alleviate the​​​‌ I/O bottleneck, the in​ situ paradigm proposes to​‌ start processing data as​​ soon as made available​​​‌ by the simulation, while​ still residing in the​‌ memory of the compute​​ node. In situ processings​​​‌ include data compression, indexing,​ computation of various types​‌ of descriptors (1D, 2D,​​ images, etc.). Per se,​​​‌ reducing data output to​ limit I/O related performance​‌ drops or keep the​​ output data size manageable​​​‌ is not new. Scientists​ have relied on solutions​‌ as simple as decreasing​​ the frequency of result​​​‌ savings. In situ processing​ proposes to move one​‌ step further, by providing​​ a full fledged processing​​​‌ framework enabling scientists to​ more easily and thoroughly​‌ manage the available I/O​​ budget.

The most direct​​​‌ way to perform in​ situ analytics is to​‌ inline computations directly in​​ the simulation code. In​​​‌ this case, in situ​ processing is executed in​‌ sequence with the simulation​​ that is suspended meanwhile.​​ Though this approach is​​​‌ direct to implement and‌ does not require complex‌​‌ framework environments, it does​​ not enable to overlap​​​‌ analytics related computations and‌ data movements with the‌​‌ simulation execution, preventing to​​ efficiently use the available​​​‌ resources. Instead of relying‌ on this simple time‌​‌ sharing approach, several works​​ propose to rely on​​​‌ space sharing where one‌ or several cores per‌​‌ node, called helper cores​​, are dedicated to​​​‌ analytics. The simulation responsibility‌ is simply to handle‌​‌ a copy of the​​ relevant data to the​​​‌ node-local in situ processes,‌ both codes being executed‌​‌ concurrently. This approach often​​ lead to significantly beter​​​‌ performance than in-simulation analytics.‌

For a better isolation‌​‌ of the simulation and​​ in situ processes, one​​​‌ solution consists in offloading‌ in situ tasks from‌​‌ the simulation nodes towards​​ extra dedicated nodes, usually​​​‌ called staging nodes.‌ These computations are said‌​‌ to be performed in-transit​​. But this approach​​​‌ may not always be‌ beneficial compared to processing‌​‌ on simulation nodes due​​ to the costs of​​​‌ moving the data from‌ the simulation nodes to‌​‌ the staging nodes.

But​​ today the choice of​​​‌ the resource allocation strategy‌ is mostly ad-hoc and‌​‌ defined by the programmer.​​ We will investigate solutions​​​‌ that enable a cooperative‌ use of the resource‌​‌ between the analytics and​​ the simulation with minimal​​​‌ hints from the programmer.‌ In situ processings inherit‌​‌ from the parallelization scale​​ and data distribution adopted​​​‌ by the simulation, and‌ must execute with minimal‌​‌ perturbations on the simulation​​ execution (whose actual resource​​​‌ usage is difficult to‌ know a priori). We‌​‌ need to develop adapted​​ scheduling strategies that operate​​​‌ at compile and run‌ time. Because analysis are‌​‌ often data intensive, such​​ solutions must take into​​​‌ consideration data movements, a‌ point that classical scheduling‌​‌ strategies designed first for​​ compute intensive applications often​​​‌ overlook. We expect to‌ develop new scheduling strategies‌​‌ relying on the methodologies​​ developed in Sec. 4.1.5​​​‌. Simulations as well‌ as analysis are iterative‌​‌ processes exposing a strong​​ spatial and temporal coherency​​​‌ that we can take‌ benefit of to anticipate‌​‌ their behavior and then​​ take more relevant resources​​​‌ allocation strategies, possibly based‌ on advanced learning algorithms‌​‌ or as developed in​​ Section 4.1.

In​​​‌ situ analytics represent a‌ specific workload that needs‌​‌ to be scheduled very​​ closely to the simulation,​​​‌ but not necessarily active‌ during the full extent‌​‌ of the simulation execution​​ and that may also​​​‌ require to access data‌ from previous runs (stored‌​‌ in the file system​​ or on specific burst-buffers).​​​‌ Several users may also‌ need to run concurrent‌​‌ analytics pipe-lines on shared​​ data. This departs significantly​​​‌ from the traditional batch‌ scheduling model, motivating the‌​‌ need for a more​​ elastic approach to resource​​​‌ provisioning. These issues will‌ be conjointly addressed with‌​‌ research on batch scheduling​​ policies (Sec. 4.1).​​​‌

4.3.3 Co-Design with Data‌ Scientists

Given the importance‌​‌ of users in this​​ context, it is of​​​‌ primary importance that in‌ situ tools be co-designed‌​‌ with advanced users, even​​​‌ if such multidisciplinary collaborations​ are challenging and require​‌ constant long term investments​​ to learn and understand​​​‌ the specific practices and​ expectations of the other​‌ domain.

We will tightly​​ collaborate with scientists of​​​‌ some application domains, like​ molecular dynamics or fluid​‌ simulation, to design, develop,​​ deploy and assess in​​​‌ situ analytics scenarios.

7.1.1​‌ OAR

• Keywords:
  HPC, Cloud,​​ Clusters, Resource manager, Light​​​‌ grid
• Scientific Description:
  This​ batch system is based​‌ on a database (PostgreSQL​​ (preferred) or MySQL), a​​​‌ script language (Perl) and​ an optional scalable administrative​‌ tool (e.g. Taktuk). It​​ is composed of modules​​​‌ which interact mainly via​ the database and are​‌ executed as independent programs.​​ Therefore, formally, there is​​​‌ no API, the system​ interaction is completely defined​‌ by the database schema.​​ This approach eases the​​​‌ development of specific modules.​ Indeed, each module (such​‌ as schedulers) may be​​ developed in any language​​​‌ having a database access​ library.
• Functional Description:
  OAR​‌ is a versatile resource​​ and task manager (also​​​‌ called a batch scheduler)​ for HPC clusters, and​‌ other computing infrastructures (like​​ distributed computing experimental testbeds​​​‌ where versatility is a​ key).
• URL:
  http://­oar.­imag.­fr
• Contact:​‌
  Olivier Richard
• Participant:
  3​​ anonymous participants
• Partners:
  LIG,​​​‌ CNRS, Grid'5000, CIMENT, UAR​ GRICAD

7.1.2 MELISSA

• Keywords:​‌
  Sensitivity Analysis, HPC, Data​​ assimilation, Exascale, AI4Science
• Functional​​​‌ Description:
  Melissa is a​ middleware framework for on-line​‌ processing of data produced​​ from large scale ensemble​​​‌ runs (parameter sweep data​ analysis) for sensibility analysis,​‌ data assimilation and deep​​ surrogate training. Largest runs​​​‌ so far involved up​ to 30k core, executed​‌ 80 000 parallel simulations,​​ and generated 288 TB​​​‌ of intermediate data that​ did not need to​‌ be stored on the​​ file system. For deep​​ surrogate training Melissa demonstrated​​​‌ it can significantly speed-up‌ training on multiple GPUs‌​‌ by maintaining a very​​ high GPU usage.
• URL:​​​‌
  https://­gitlab.­inria.­fr/­melissa
• Publications:
  hal-04145897,‌ hal-04213978, hal-04102400,‌​‌ hal-01383860, hal-01607479,​​ hal-03017033, hal-03927612,​​​‌ hal-03842106
• Contact:
  Bruno Raffin‌
• Partner:
  Edf

7.1.3 NixOS-Compose‌​‌

• Keywords:
  Infrastructure software, Deployment,​​ High performance computing, Distributed​​​‌ computing
• Functional Description:
  NixOS-Compose‌ simplifies the process of‌​‌ setting up ephemeral distributed​​ systems by utilizing Nix's​​​‌ functional package management and‌ NixOS's declarative configuration management.‌​‌ The tool facilitates testing,​​ development, infrastructure prototyping, benchmarking,​​​‌ and advanced experiments in‌ high-performance computing by providing‌​‌ easy and reproducible software​​ stack deployment.
• URL:
  https://­gitlab.­inria.­fr/­nixos-compose/­nixos-compose​​​‌
• Publication:
  hal-03723771
• Contact:
  Olivier‌ Richard
• Partners:
  LIG, CNRS,‌​‌ UGA

7.1.4 Batsim

• Functional​​ Description:
  BatSim is a​​​‌ Resource and Job Management‌ System (RJMS) framework simulator‌​‌ based on SimGrid. It​​ aims at taking into​​​‌ account platform's hardware capabilities‌ and impacts in simulations.‌​‌ Also, schedulers parts are​​ plugable through a comprehensive​​​‌ API and they are‌ seen as external component‌​‌ of the framework.
• Release​​ Contributions:
  see https://batsim.readthedocs.io/en/latest/changelog.html
• URL:​​​‌
  https://­batsim.­readthedocs.­io/­en/­latest/
• Contact:
  Olivier Richard‌

7.1.5 Kameleon

• Keyword:
  Engineering‌​‌ software systems
• Functional Description:​​
  Kameleon is a simple​​​‌ but powerful tool to‌ generate customized appliances. With‌​‌ Kameleon, you make your​​ recipe that describes how​​​‌ to create step by‌ step your own distribution.‌​‌ At start Kameleon is​​ used to create custom​​​‌ kvm, docker, VirtualBox, ...,‌ but as it is‌​‌ designed to be very​​ generic you can probably​​​‌ do a lot more‌ than that.
• URL:
  http://­kameleon.­imag.­fr/‌​‌
• Contact:
  Olivier Richard
• Participant:​​
  an anonymous participant
• Partner:​​​‌
  Grid'5000

7.1.6 alumet

• Name:‌
  ALUMET: unified measurement software‌​‌
• Keywords:
  Energy, Rust, Power​​ monitoring, High performance computing,​​​‌ Performance measure
• Functional Description:‌
  Alumet provides a generic‌​‌ measurement pipeline with three​​ steps: poll measurement sources,​​​‌ transform the data, and‌ write the result. It‌​‌ is designed to be​​ able to ingest metrics​​​‌ from various sources without‌ redundant work. Supported sources‌​‌ include RAPL domains, Nvidia's​​ NVML, and Jetson INA​​​‌ sensors. The list of‌ supported devices will quickly‌​‌ grow over time, thanks​​ to the next feature​​​‌ of Alumet.
• URL:
  https://­alumet.­dev/‌
• Contact:
  Guillaume Raffin
• Partner:‌​‌
  Bull - Atos Technologies​​

7.2.1​​​‌ Slices-fr/Grid'5000 and Meso Center‌ Gricad

We are very‌​‌ active in promoting the​​ factorization of compute resources​​​‌ at a regional and‌ national level. We have‌​‌ a three level implication,​​ locally to maintain a​​​‌ pool of very flexible‌ experimental machines (hundreds of‌​‌ cores), regionally through the​​ GRICAD meso center,​​​‌ and nationally by contributing‌ to the Slices-fr/Grid'5000 platform‌​‌, our local resources​​ being included in this​​​‌ platform. Olivier Richard is‌ member of Slices-fr/Grid'5000 scientific‌​‌ committee. The OAR scheduler​​ in particular is deployed​​​‌ on both infrastructures. DataMove‌ is hosting several engineers‌​‌ dedicated to Grid'5000 support.​​

8.2.1 Carbon​​​‌ Footprint of High-Performance Computing​

A central research theme​‌ concerns the environmental impact​​ of high-performance computing (HPC),​​​‌ ranging from system-level carbon​ emissions to device lifetimes​‌ and power-aware scheduling. One​​ study analyzes the evolution​​​‌ of the carbon footprint​ of large-scale HPC systems​‌ by combining performance data​​ with information on energy​​​‌ mixes and projected trajectories​ toward 2030 20.​‌ Moving beyond the traditional​​ Top500 and Green500 perspectives,​​​‌ the work considers the​ entire life span of​‌ several major systems and​​ derives a predictive model​​​‌ to estimate the contribution​ of HPC to global​‌ carbon emissions over the​​ next five years. By​​​‌ incorporating the carbon intensity​ of electricity and long-term​‌ deployment patterns, this analysis​​ provides a forward-looking view​​​‌ on how the HPC​ community may need to​‌ adapt architectures, locations, and​​ operational practices.

The environmental​​​‌ footprint is further refined​ at smaller scales, for​‌ example in the context​​ of networked sensor infrastructures​​​‌ embedded in electrical distribution​ boards 19. In​‌ this line of work,​​ an empirical study compares​​​‌ three scenarios: a baseline​ board without energy measurements,​‌ a board with wired​​ Modbus RS485-based metering, and​​​‌ a board with IEEE​ 802.15.4 wireless metering. Using​‌ Product Environmental Profiles and​​ comparative life-cycle assessment, the​​​‌ authors show that instrumented​ boards inevitably increase carbon​‌ emissions compared to the​​ non-instrumented baseline, but also​​​‌ that the wireless solution​ can reduce the environmental​‌ impact by nearly 45%​​ relative to the wired​​​‌ configuration. The analysis also​ underlines that current models​‌ for wireless devices may​​ overestimate operational consumption by​​​‌ not fully accounting for​ duty-cycling capabilities, thereby motivating​‌ more accurate modeling of​​ connected devices in future​​​‌ work.

Another contribution addresses​ the lifetime of processors​‌ and accelerators and its​​ relationship with the environmental​​​‌ footprint of supercomputers 21​. The work emphasizes​‌ that the increasing demand​​ for GPUs, particularly from​​​‌ AI workloads, has created​ strong pressure on hardware​‌ availability and replacement cycles.​​ By modeling aging as​​​‌ a function of operating​ temperature and frequency, the​‌ authors propose node frequency​​ reconfiguration and dedicated scheduling​​​‌ algorithms that aim to​ increase the total number​‌ of floating-point operations delivered​​ by a machine before​​​‌ component failure. Simulation results​ indicate that appropriate frequency​‌ decisions can substantially raise​​ the cumulative computational output​​​‌ of a system, at​ the cost of controlled​‌ performance trade-offs on individual​​ jobs, and that such​​​‌ strategies remain effective under​ different, imperfect aging models.​‌

Complementing these system-level approaches,​​ a separate study introduces​​​‌ Alumet, a modular framework​ that standardizes the measurement​‌ of energy consumption across​​ hardware and software stacks​​​‌ 16. Alumet provides​ a generic pipeline to​‌ collect, transform, and export​​ a wide variety of​​ measurements, and is designed​​​‌ with a plugin system‌ to support new environments‌​‌ and energy models without​​ requiring major changes to​​​‌ the core framework. Experimental‌ deployments on heterogeneous platforms‌​‌ show that Alumet can​​ operate at higher acquisition​​​‌ frequencies while limiting monitoring‌ overhead, and that it‌​‌ facilitates the development of​​ energy estimation models in​​​‌ diverse contexts. By improving‌ the accuracy and extensibility‌​‌ of energy measurements, Alumet​​ underpins the broader objective​​​‌ of making energy-aware decisions‌ in HPC and distributed‌​‌ systems.

8.2.2 Power-Aware Scheduling​​ and Dynamic Resource Management​​​‌

Energy and power constraints‌ are also addressed from‌​‌ a scheduling and runtime​​ management perspective. One article​​​‌ focuses on power-constrained HPC‌ platforms and investigates how‌​‌ to predict workload power​​ consumption and exploit these​​​‌ predictions in power-aware scheduling‌ algorithms 10. The‌​‌ proposed method combines lightweight,​​ history-based prediction schemes with​​​‌ a scheduler inspired by‌ EASY backfilling, and models‌​‌ power capping as a​​ greedy knapsack-like optimization problem.​​​‌ Using logs from Marconi‌ 100, a 980-node supercomputer,‌​‌ simulation results show that​​ relatively simple prediction models​​​‌ can achieve sufficiently accurate‌ workload power forecasts to‌​‌ reduce overall power consumption​​ without degrading scheduling performance​​​‌ or quality of service.‌

Dynamic Resource Management (DRM)‌​‌ forms another major axis​​ of research. One study​​​‌ examines how to bridge‌ genericity and programmability for‌​‌ dynamic resources in HPC​​ by interfacing the Dynamic​​​‌ Management of Resources API‌ (DMR-API) with the Dynamic‌​‌ Processes with PSets (DPP)​​ design principles 15.​​​‌ The DMR-API provides an‌ application-level abstraction that simplifies‌​‌ the integration of dynamic​​ resources into iterative HPC​​​‌ applications, while DPP offers‌ a generic, programming-model-agnostic approach‌​‌ to resource control at​​ the system level. By​​​‌ combining both, the authors‌ propose a methodology that‌​‌ retains the flexibility of​​ DPP while reducing the​​​‌ coding effort required to‌ exploit dynamic resource allocation.‌​‌ Experimental evaluations indicate that​​ DRM can be effectively​​​‌ leveraged in realistic HPC‌ environments with limited software‌​‌ changes, improving job throughput​​ and system utilization.

8.2.3​​​‌ Environmental Impact of Generative‌ AI

Beyond HPC in‌​‌ a narrow sense, the​​ DataMove team also investigates​​​‌ the sustainability of emerging‌ digital services such as‌​‌ generative AI (Gen-AI). One​​ article addresses the environmental​​​‌ impact of Gen-AI services‌ through a life-cycle and‌​‌ measurement-based study of a​​ Stable Diffusion image generation​​​‌ service 9. The‌ methodology explicitly differentiates between‌​‌ embodied impacts, related to​​ the manufacturing and deployment​​​‌ of models and hardware,‌ and operational impacts, associated‌​‌ with runtime energy use​​ across data centers, networks,​​​‌ and user devices. The‌ analysis demonstrates that, when‌​‌ Gen-AI is offered as​​ a service, the cumulative​​​‌ impact of numerous terminals‌ and communication networks becomes‌​‌ a significant component of​​ its footprint, and that​​​‌ decarbonizing electricity alone is‌ insufficient to render such‌​‌ services sustainable in the​​ long term. By emphasizing​​​‌ constraints related to energy‌ consumption and rare metals‌​‌ in a finite-resource world,​​ the study argues for​​​‌ early and comprehensive impact‌ assessments of Gen-AI solutions‌​‌ and provides tools to​​ support more informed decisions​​​‌ about their deployment.

8.3.1​​​‌ Edge–Cloud and Serverless Scheduling​

Several 2025 publications focus​‌ on the computing continuum,​​ spanning edge to cloud​​​‌ resources and embracing serverless​ and container-based paradigms. One​‌ contribution, FOA-Energy, proposes a​​ multi-objective scheduling policy for​​​‌ serverless platforms deployed across​ an edge–cloud continuum 13​‌. Recognizing that data-centric​​ applications often require large​​​‌ software environments and handle​ massive data volumes, the​‌ authors extend an existing​​ methodology to study serverless​​​‌ infrastructures via simulation. They​ design a scheduling algorithm​‌ that simultaneously considers platform​​ heterogeneity, cold start delays,​​​‌ energy consumption, data transfers,​ makespan, and resource utilization.​‌ Using a standard greedy​​ Kubernetes-inspired algorithm as a​​​‌ baseline, the study shows​ that the proposed multi-objective​‌ scheduler can outperform the​​ baseline by up to​​​‌ three orders of magnitude​ on several metrics, highlighting​‌ the importance of tailored​​ scheduling strategies in heterogeneous,​​​‌ serverless environments.

8.3.2 Operational​ Technology Platforms and Orchestration​‌

Another strand of research​​ addresses the integration of​​​‌ operational technology (OT) with​ platform-as-a-service (PaaS) models in​‌ the continuum. The OTPaaS​​ initiative is presented as​​​‌ a structured framework for​ managing and storing industrial​‌ data with strong requirements​​ on response times, security,​​​‌ reliability, technological and data​ sovereignty, robustness, and energy​‌ efficiency 31. The​​ associated publication discusses successful​​​‌ deployments, adaptive application management,​ and integration components for​‌ both edge and cloud​​ environments, emphasizing how a​​​‌ PaaS-style abstraction can encapsulate​ complexity while preserving stringent​‌ industrial constraints.

Complementing this,​​ two closely related publications​​​‌ introduce the concept of​ User-Friendly Orchestration Management (UFOM)​‌ for containerized services in​​ the computing continuum 18​​​‌, 17. UFOM​ targets non-expert users by​‌ offering an intuitive interface,​​ automated workflows, and contextual​​​‌ assistance to simplify the​ deployment, monitoring, and maintenance​‌ of distributed applications. The​​ approach integrates with osmotic​​​‌ computing principles to support​ seamless interactions between edge​‌ and cloud resources, and​​ evaluates the impact on​​​‌ user-perceived Quality of Experience.​ A smart home automation​‌ case study illustrates how​​ UFOM can democratize orchestration​​​‌ by reducing technical barriers​ while maintaining system reliability​‌ and efficiency in real-world​​ scenarios.

8.3.3 Testbeds and​​​‌ Complexity Analysis for the​ Continuum

To fully exploit​‌ the computing continuum, appropriate​​ research testbeds are necessary.​​​‌ One paper proposes a​ conceptual testbed for network​‌ operating systems in continuum​​ environments, aiming to improve​​​‌ the replicability, scalability, and​ robustness of experiments 12​‌. The testbed allows​​ experimenters to modify the​​​‌ operating systems of network​ devices and dynamically reconfigure​‌ network topologies, thereby supporting​​ studies spanning multi-operator settings​​​‌ and internal architectures of​ telecommunication providers. The authors​‌ also investigate mechanisms for​​ virtual topology management, OS​​​‌ deployment, and service orchestration,​ underlining the need for​‌ flexible and programmable infrastructures.​​

Beyond testbed design, another​​​‌ publication proposes a holistic​ multidimensional analysis framework to​‌ manage complexity in computing​​ continuum systems 11.​​​‌ The framework combines Quality​ of Service, Service Level​‌ Agreement specifications, and Quality​​ of Experience metrics across​​​‌ multiple levels of the​ continuum, enabling the characterization​‌ of system behavior along​​ several axes simultaneously. By​​​‌ applying this approach to​ two tiers of a​‌ continuum system, the authors​​ show how interlinked metrics​​ can reveal critical properties​​​‌ and bottlenecks that might‌ be missed by single-dimension‌​‌ analyses. This multidimensional perspective​​ supports more informed design​​​‌ and optimization of continuum‌ services, particularly when performance,‌​‌ energy, and user satisfaction​​ must be co-optimized.

10.1.1 Horizon Europe​​

10.2.1 PEPR NUMPEX

• Goals:​
  The main objective of​‌ the NumPEx (Numeric for​​ Exascale) program in France​​ is to develop state-of-the-art​​​‌ skills and infrastructures in‌ the field of exascale‌​‌ computing.
• Duration:
  From 2023​​ to 2030
• Web site:​​​‌
  NUMPEX
• DataMove implication:
  • Exa-DoST‌ (Data-oriented Software and Tools‌​‌ for the Exascale): Co-lead​​ WP3.
  • Exa-AToW (Architectures and​​​‌ Tools for Large-Scale Workflows):‌ Co-lead WP5.
  • Exa-DI (Development‌​‌ and integration): CO-lead WP3.​​
• DataMove budget:
  1.295 M​​​‌ euros.

10.2.2 ANR

• PPR‌ Océan et Climat MEDIATION‌​‌ (2022-2030). Methodological developments​​ for a robust and​​​‌ efficient digital twin of‌ the ocean. Pi: INRIA‌​‌ team AIRSEA. Partners: INRIA,​​ CNRS, IFREMER, IRD, Université​​​‌ Aix-Marseille, Institut National Polytechnique‌ de Toulouse, Ecole Nationale‌​‌ Supérieure Mines-Télécom Atlantique Bretagne​​ Pays de la Loire,​​​‌ Service Hygrodgraphique et Océanographique‌ de la Marine, Université‌​‌ Grenoble Alpes, Météo-France-DESR-Centre National​​ de Recherches Météorologiques. Total​​​‌ budget: 2,4 Meuros. DataMove‌ Budget: 110 Keuros. CO-lead‌​‌ of the WP Leveraging​​ AI and HPC for​​​‌ Digital Twins of the‌ Ocean.
• AAPG2023 PREDICTIONS (2024-2027)‌​‌. This project aims​​ to substantially strengthen and​​​‌ expand the foundations of‌ the nascent, but fast-growing‌​‌ area of algorithms with​​ predictions, in a global​​​‌ framework that addresses all‌ aspects of algorithm development:‌​‌ modeling, design, framework of​​ analysis, and performance evaluation.​​​‌ Specifically, we put forward‌ three main objectives. Pi:‌​‌ LIP6. Partners: LIG/DataMove, IRIF,CC-IN2P3,​​ LIRIS. Total budget: 358k​​​‌ euros.Datamova Budget: 128k euros.‌
• AAPG2025 SOCLOUD (20252029).‌​‌ The aim of the​​ project is to study​​​‌ the human and technical‌ conditions for implementing sobriety‌​‌ in the cloud, and​​ to identify the levers​​​‌ and their consequences. Partenaires:‌ Univ. Besançon, Univ Toulouse,‌​‌ INRIA, Eaton Industries.

International journals‌

• 9 articleA.Adrien‌​‌ Berthelot, E.Eddy​​ Caron, M.Mathilde​​​‌ Jay and L.Laurent‌ Lefèvre. Understanding the‌​‌ environmental impact of generative​​ AI services.Communications​​​‌ of the ACMSpecial‌ Issue on Sustainability and‌​‌ Computing 6872025​​, 46-53HALDOI​​​‌back to text
• 10‌ articleD.Danilo Carastan-Santos‌​‌, G.Georges da​​ Costa, I.Igor​​​‌ Fontana de Nardin,‌ M.Millian Poquet,‌​‌ K.Krzysztof Rzadca,​​ P.Patricia Stolf and​​​‌ D.Denis Trystram.‌ Scheduling with lightweight predictions‌​‌ in power-constrained HPC platforms​​.IEEE Transactions on​​​‌ Parallel and Distributed Systems‌July 2025, 1-12‌​‌HALDOIback to​​ text

International peer-reviewed conferences​​​‌

• 11 inproceedingsC. J.‌Carlos J Barrios,‌​‌ Y.Yves Denneulin and​​ F.Frédéric Le Mouël​​​‌. A Holistic Approach‌ to Complexity Management and‌​‌ Multidimensional Analysis in Computing​​ Continuum.WSCC 2025​​​‌ - 3rd International Workshop‌ on Scalable Compute Continuum‌​‌Dresde (Germany), Germany2025​​, 1-12HALback​​​‌ to text
• 12 inproceedings‌J.Julien Caposiena,‌​‌ O.Oscar Carrillo,​​​‌ F.Frédéric Le Mouël​, B.Baptiste Jonglez​‌, P.Pierre Neyron​​ and T.Thierry Arrabal​​​‌. Towards a flexible​ Network Operating System Testbed​‌ for the Computing Continuum​​.CCGridW 2025 -​​​‌ 25th IEEE International Symposium​ on Cluster, Cloud and​‌ Internet Computing Workshops2025​​ IEEE 25th International Symposium​​​‌ on Cluster, Cloud and​ Internet Computing Workshops (CCGridW)​‌Tromsø Norway, Norway2025​​, 148-155HALDOI​​​‌back to text
• 13​ inproceedingsA. A.Anderson​‌ Andrei Da Silva,​​ Y.Yiannis Georgiou,​​​‌ M.Michael Mercier,​ G.Gregory Mounié and​‌ D.Denis Trystram.​​ FOA-Energy: A Multi-objective Energy-Aware​​​‌ Scheduling Policy for Serverless-based​ Edge-Cloud Continuum.SAC​‌ 2025 - 40th ACM/SIGAPP​​ Symposium on Applied Computing​​​‌Catania International Airport Catania​ Italy, ItalyACMMarch​‌ 2025, 225-232HAL​​DOIback to text​​​‌
• 14 inproceedingsY.Yoann​ Dupas, O.Olivier​‌ Hotel, G.Grégoire​​ Lefebvre and C.Christophe​​​‌ Cerin. MEFA: Multimodal​ Image Early Fusion with​‌ Attention Module for Pedestrian​​ and Vehicle Detection.​​​‌VISAPP 2025 - 20th​ International Conference on Computer​‌ Vision Theory and Applications​​Porto, PortugalSCITEPRESS -​​​‌ Science and Technology Publications​2025, 610-617HAL​‌DOIback to text​​
• 15 inproceedingsD.Dominik​​​‌ Huber, S.Sergio​ Iserte, M.Martin​‌ Schreiber, A. J.​​Antonio J. Peña and​​​‌ M.Martin Schulz.​ Bridging the Gap Between​‌ Genericity and Programmability of​​ Dynamic Resources in HPC​​​‌.ISC High Performance​ 2025 - 40th ISC​‌ High Performance International Conference​​Hamburg, Germany2025,​​​‌ 1-11HALback to​ text
• 16 inproceedingsG.​‌Guillaume Raffin, D.​​Denis Trystram and O.​​​‌Olivier Richard. Alumet:​ a Modular Framework to​‌ Standardize the Measurement of​​ Energy Consumption.PECS​​​‌ 2025 - Workshop on​ Performance and Energy Efficiency​‌ in Concurrent and Distributed​​ SystemsDresden, GermanySpringer​​​‌2025, 1-12HAL​back to text
• 17​‌ inproceedingsP. J.Pablo​​ Josue Rojas Yepes,​​​‌ C. J.Carlos Jaime​ Barrios Hernández, O.​‌Oscar Carrillo and F.​​Frédéric Le Mouël.​​​‌ User-Friendly Orchestration Management Proposal​.CCGRID 2025 -​‌ 25th IEEE International Symposium​​ on Cluster, Cloud, and​​​‌ Internet ComputingTromso, Norway,​ NorwayIEEE2025,​‌ 1-4HALback to​​ text
• 18 inproceedingsP.​​​‌ J.Pablo Josue Rojas​ Yepes, C. J.​‌Carlos Jaime Barrios Hernández​​, O.Oscar Carrillo​​​‌ and F.Frédéric Le​ Mouël. User-Friendly Orchestration​‌ Management Proposal.2025​​ - 5th Workshop CATAÏ​​​‌Chalon sur Saône, France​2025, 1-4HAL​‌back to text

National​​ peer-reviewed Conferences

• 19 inproceedings​​​‌M.Marina Gradvohl,​ E.Elodie Chargy,​‌ E.Emmanuel Dreina,​​ D.Danilo Carastan-Santos and​​​‌ F.Franck Rousseau.​ Étude de l'empreinte carbone​‌ d'un réseau de capteurs​​ dans un tableau de​​​‌ distribution électrique.ALGOTEL​ 2025 – 27èmes Rencontres​‌ Francophones sur les Aspects​​ Algorithmiques des TélécommunicationsALGOTEL​​​‌ 2025 – 27èmes Rencontres​ Francophones sur les Aspects​‌ Algorithmiques des TélécommunicationsSaint​​ Valery-sur-Somme, France2025,​​​‌ 1-5HALback to​ text

Conferences without proceedings​‌

• 20 inproceedingsA.Abdessalam​​ Benhari, Y.Yves​​ Denneulin, F.Frédéric​​​‌ Desprez, F.Fanny‌ Dufossé and D.Denis‌​‌ Trystram. Analysis of​​ the carbon footprint of​​​‌ HPC.PECS 2025‌ - International Workshop on‌​‌ Performance and Energy Efficiency​​ in Concurrent and Distributed​​​‌ SystemsDresden, Germany2025‌, 1-13HALback‌​‌ to text
• 21 inproceedings​​R.Robin Boëzennec,​​​‌ F.Fanny Dufossé,‌ G.Guillaume Pallez and‌​‌ A.Alix Tremodeux.​​ Improving Supercomputer Usage with​​​‌ Aging Awareness.Sustainable‌ Supercomputing (Workshop of SC25)‌​‌St. Louis, Missouri, United​​ StatesNovember 2025HAL​​​‌back to text
• 22‌ inproceedingsJ.Julien Caposiena‌​‌, O.Oscar Carrillo​​, B.Baptiste Jonglez​​​‌, P.Pierre Neyron‌ and T.Thierry Arrabal‌​‌. Vers un banc​​ d'essai flexible pour les​​​‌ systèmes d'exploitation réseau dans‌ le Computing Continuum.‌​‌ComPASBordeaux, FranceJune​​ 2025HAL
• 23 inproceedings​​​‌Y.Yoann Dupas,‌ O.Olivier Hotel,‌​‌ G.Grégoire Lefebvre and​​ C.Christophe Cérin.​​​‌ MEFA-MS: Attention-Based U-Net for‌ Pedestrian and Vehicle Detection‌​‌.ICMV 2025 -​​ Eighteenth International Conference on​​​‌ Machine VisionParis, France‌2025, 1-8HAL‌​‌back to text
• 24​​ inproceedingsE.Ernest Foussard​​​‌, M.Margaux Nattaf‌, M.-L.Marie-Laure Espinouse‌​‌ and G.Grégory Mounié​​. Minimisation du délai​​​‌ moyen en présence d'une‌ contrainte de santé de‌​‌ l'équipement : propriétés structurelles​​.ROADEF 2025 -​​​‌ 26ème édition du congrès‌ annuel de la Société‌​‌ Française de Recherche Opérationnelle​​ et d'Aide à la​​​‌ DécisionChamps-sur-Marne, France2025‌, 1-2HAL

Reports‌​‌ & preprints

• 25 misc​​M.Mathieu Bacou,​​​‌ D.David Beserra,‌ E.Eugen Dedu,‌​‌ L.Loïc Desgeorges,​​ D.Didier Donsez,​​​‌ A.Alexandre Guitton,‌ B.Baptiste Jonglez,‌​‌ A.Arnaud Legrand,​​ G.Georgios Papadopoulos,​​​‌ O.Olivier Richard,‌ S.Samir Si-Mohammed,‌​‌ N.Nina Tamdrari and​​ F.Fabrice Theoleyre.​​​‌ Journée thématique du GDR‌ RSD : pratiques expérimentales‌​‌ de la communauté systèmes​​ et réseaux.January​​​‌ 2025HAL
• 26 report‌S.Sylvain Bouveret,‌​‌ A.Aurélie Bugeau,​​ F.Frenoux Emmanuelle,​​​‌ J.Julien Lefevre,‌ L.Laurent Lefèvre,‌​‌ A.-L.Anne-Laure Ligozat,​​ P.Philippe Marquet,​​​‌ A.-C.Anne-Cécile Orgerie and‌ D.Denis Trystram.‌​‌ Quiz sur les impacts​​ environnementaux du numérique.​​​‌EcoInfoFebruary 2025,‌ 1-5HAL
• 27 misc‌​‌E.Ernest Foussard,​​ J.John Martinovic,​​​‌ M.-L.Marie-Laure Espinouse,‌ G.Grégory Mounié and‌​‌ M.Margaux Nattaf.​​ Bin Packing with Thresholds:​​​‌ Mathematical Models and Theoretical‌ Results.2025HAL‌​‌
• 28 reportA.Andrea​​ Letizia, C.Christophe​​​‌ Cérin and D.Didier‌ Donsez. WildCount: Embedded‌​‌ Deep Learning for Wildlife​​ Recognition.LIG :​​​‌ Laboratoire d'informatique de Grenoble;‌ Grenoble INP - UGA‌​‌September 2025, 1-17​​HAL
• 29 miscG.​​​‌Gabriella Saraiva, M.‌Miguel Vasconcelos, S.‌​‌Sarita Mazzini Bruschi,​​ D.Danilo Carastan-Santos and​​​‌ D.Daniel Cordeiro.‌ Estimating CO2 emissions of‌​‌ distributed applications and platforms​​ with SimGrid/Batsim.2025​​​‌HAL
• 30 miscN.‌ K.Nguyễn Kim Thắng‌​‌. Price of Anarchy​​​‌ in Resource Allocation and​ Auto-Bidding Advertising via Duality​‌ in Linear/Convex Programming.​​2025HAL

Scientific popularization​​​‌

• 31 inproceedingsC. J.​Carlos J Barrios and​‌ Y.Yves Denneulin.​​ Bridding OT and PaaS​​​‌ in Edge-to-Cloud Continuum: The​ OTPaaS Concept.COMPAS​‌ 2025 - Conférence Francophone​​ d'Informatique en Parallélisme, Architecture​​​‌ et SystèmeBORDEAUX, France​2025, 1-13HAL​‌back to text

Software​​

• 32 softwareG.Guillaume​​​‌ Raffin. Alumet.​April 2025 lic: European​‌ Union Public License 1.2​​.HALSoftware Heritage​​​‌VCS