2024Activity reportTeamFLOWERS

7.1.1 SocialAI

• Name:
  SocialAI: Benchmarking Socio-Cognitive Abilities in Deep Reinforcement Learning Agents
• Keywords:
  Artificial intelligence, Deep learning, Reinforcement learning, Large Language Models
• Functional Description:

  Source code for the paper https://arxiv.org/abs/2107.00956.

  A suite of environments for testing socio-cognitive abilities of artificial agents. Environments can be used in the multimodal setting (suitable for RL agents) and in the pure text setting (suitable for Large Language Model-based agents). Also contains RL and LLM baselines.

• URL:
  https://­gitlab.­inria.­fr/­gkovac/­act-and-speak
• Contact:
  Grgur Kovac

7.1.2 AutoDisc

• Keyword:
  Complex Systems
• Functional Description:
  AutoDisc is a software built for automated scientific discoveries in complex systems (e.g. self-organizing systems). It can be used as a tool to experiment automated discovery of various systems using exploration algorithms (e.g. curiosity-driven). Our software is fully Open Source and allows user to add their own systems, exploration algorithms or visualization methods.
• URL:
  https://­gitlab.­inria.­fr/­cromac/­AutomatedDiscoveryTool
• Contact:
  Clément Romac

7.1.3 ADTool

• Keywords:
  Machine learning, Python, Cellular automaton, Physical simulation, Pattern discovery, Exploration
• Functional Description:

  ADTool is a versatile and open-source Python framework designed to explore complex parametric systems using IMGEP algorithms (Intrinsic Motivation for Goal Exploration Processes) as described in https://arxiv.org/pdf/1708.02190. This curiosity-driven approach enables automatic exploration and the discovery of new behaviors across a wide range of domains, offering a novel way to study complex systems.

  With ADTool, users can explore cellular automata such as Lenia, Particle Lenia, and Flowlenia to uncover patterns and emergent behaviors. Its capabilities extend to drug discovery, exploring chemical spaces to identify promising protein-ligand affinity profiles. The framework also ventures into physics, with applications such as searching for trajectories in the N-body problem, simulating the Kuramoto model, exploring the Gray-Scott reaction-diffusion system, and studying hypergraph rewriting systems for Wolfram physics. In digital art, ADTool fosters creativity by exploring processes like subtractive sound synthesis and other artistic methods.

  The framework is designed to be flexible and extensible, allowing users to define their own systems and integrate custom exploration strategies. It includes mechanisms for saving discoveries to disk, making it easier to resume experiments or share results with collaborators. Additionally, an integrated visualization tool provides a user-friendly interface to track exploration progress, enhancing the understanding and analysis of results.

  The scientific foundation of ADTool lies in "curiosity-search" algorithms, which autonomously explore behavioral spaces to identify interesting phenomena without predefined objectives. These algorithms, initially developed for robotic learning, are now applied to the study of emergent behaviors in various systems.

  Whether you are a physicist, chemist, biologist, or digital artist, ADTool can help you explore and understand complex systems.

  Reproducibility is guarantied with a predifined Python environment and experiments can be launched with a simple command line: python3 run.py –config_file examples/grayscott/gray_scott.json

• Contact:
  Zacharie Bugaud

7.1.4 Kids Ask

• Keywords:
  Human Computer Interaction, Cognitive sciences
• Functional Description:
  Kids Ask is a web-based educational platform that involves an interaction between a child and a conversational agent. The platform is designed to teach children how to generate curiosity-based questions and use them in their learning in order to gain new knowledge in an autonomous way.
• URL:
  https://­github.­com/­RaniaAbdelghani/­KidAsk
• Contact:
  Rania Abdelghani

7.1.5 ToGather

• Keywords:
  Education, Handicap, Environment perception
• Scientific Description:
  With participatory design methods, we have designed an interactive website application for educational purposes. This application aims to provide interactive services with continuously updated content for the stakeholders of school inclusion of children with specific educational needs.
• Functional Description:
  Website gathering information on middle school students with neurodevelopmental disorders. Authentication is required to access the site's content. Each user can only access the student file(s) of the young person(s) they are accompanying. A student file contains 6 tabs, in which each type of user can add, edit or delete information: 1. Profile: to quickly get to know the student 2. Skills: evaluation at a given moment and evolution over time 3. Compendium of tips: includes psycho-educational tips 4. Meetings: manager and reports 5. News: share information over time 6. Contacts: contact information for stakeholders The student only has the right to view information about him/her.
• Publication:
  hal-03898867
• Contact:
  Cécile Mazon
• Participants:
  Isabeau Saint-Supery, Cécile Mazon, Eric Meyer, Hélène Sauzéon

7.1.6 mc_training

• Name:
  Platform for metacognitive training
• Keywords:
  Human Computer Interaction, Education
• Functional Description:

  This is a web platform for children between 9 and 11 years old, designed to help children practice 4 metacognitive skills that are thought to be involved in curiosity-driven learning: - the ability to identify uncertainties - the ability to generate informed hypotheses - the ability to ask questions - the ability to evaluate the value of a preconceived inference.

  Children work on a reading-comprehension tasks and, for each of these skills, the platform offers help through a "conversation" with conversational agents that give instructions to perform the task, with respect to every skill, and can give suggestions if the child asks for it.

• Contact:
  Rania Abdelghani

7.1.7 Evolution of adaptation mechanisms in complex environments

• Name:
  Plasticity and evolvability under environmental variability: the joint role of fitness-based selection and niche-limited competition
• Keywords:
  Evolution, Ecology, Dynamic adaptation
• Functional Description:

  This is the code accompannying our paper Plasticity and evolvability under environmental variability: the joint role of fitness-based selection and niche-limited competition" which is to be presented at the Gecco 2022 conference.

  In this work we have studied the evolution of a population of agents in a world where the fitness landscape changes with generations based on climate function and a latitudinal model that divides the world in different niches. We have implemented different selection mechanisms (fitness-based selection and niche-limited competition).

  The world is divided into niches that correspond to different latitudes and whose state evolves based on a common climate function.

  We model the plasticity of an individual using tolerance curves originally developed in ecology. Plasticity curves have the form of a Gaussian the capture the benefits and costs of plasticity when comparing a specialist (left) with a generalist (right) agent.

  The repo contains the following main elements :

  folder source contains the main functionality for running a simulation scripts/run/reproduce_gecco.py can be used to rerun all simulations in the paper scripts/evaluate contains scripts for reproducing figures. reproduce_figures.py will produce all figures (provided you have already run scripts/run/reproduce_gecco.py to generate the data) folder projects contains data generated from running a simulation How to run To install all package dependencies you can create a conda environment as:

  conda env create -f environment.yml

  All script executions need to be run from folder source. Once there, you can use simulate.py, the main interface of the codebase to run a simulation, For example:

  python simulate.py –project test_stable –env_type stable –num_gens 300 –capacity 1000 –num_niches 10 –trials 10 –selection_type NF –climate_mean_init 2

  will run a simulation with an environment with a climate function whose state is constantly 2 consisting of 100 niches for 300 generations and 10 independent trials. The maximum population size will be 1000*2 and selection will be fitness-based (higher fitness means higher chances of reproduction) and niche limited (individuals reproduce independently in each niche and compete only within a niche),

  You can also take a look at scripts/run/reproduce_gecco.py to see which flags were used for the simulations presented in the paper.

  Running all simulations requires some days. You can instead download the data produced by running scripts/run/reproduce_gecco.py from this google folder and unzip them under the projects directory.

• URL:
  https://­github.­com/­eleninisioti/­ClimateAndLearning
• Contact:
  Eleni Nisioti

7.1.8 SAPIENS

• Name:
  SAPIENS: Structuring multi-Agent toPology for Innovation through ExperieNce Sharing
• Keywords:
  Reinforcement learning, Multi-agent
• Functional Description:

  SAPIENS is a reinforcement learning algorithm where multiple off-policy agents solve the same task in parallel and exchange experiences on the go. The group is characterized by its topology, a graph that determines who communicates with whom.

  All agents are DQNs and exchange experiences have the form of transitions from their replay buffers.

  Using SAPIENS we can define groups of agents that are connected with others based on a a) fully-connected topology b) small-world topology c) ring topology or d) dynamic topology.

  Install required packages You can install all required python packages by creating a new conda environment containing the packages in environment.yml:

  conda env create -f environment.yml

  And then activating the environment:

  conda activate sapiens

  Example usages Under notebooks there is a Jupyter notebook that will guide you through setting up simulations with a fully-connected and a dynamic social network structure for solving Wordcraft tasks. It also explains how you can access visualizations of the metrics produced during th$

  Reproducing the paper results Scripts under the scripts directory are useful for reproducing results and figures appearing in the paper.

  With scripts/reproduce_runs.py you can run all simulations presented in the paper from scratch.

  This file is useful for looking at how the experiments were configured but better avoid running it: simulations will run locally and sequentially and will take months to complete.

  Instead, you can access the data files output by simulations on this online repo.

  Download this zip file and uncompress it under the projects directory. This should create a projects/paper_done sub-directory.

  You can now reproduce all visualization presented in the paper. Run:

  python scripts/reproduce_visuals.py

  This will save some general plots under visuals, while project-specific plots are saved under the corresponding project in projects/paper_done

• URL:
  https://­github.­com/­eleninisioti/­SAPIENS
• Contact:
  Eleni Nisioti

7.1.9 architect-builder-abig

• Name:
  Architect-Builder Iterated Guiding
• Keyword:
  Artificial intelligence
• Functional Description:

  Codebase for the paper Learning to guide and to be guided in the Architect-Builder Problem

  ABIG stands for Architect-Builder Iterated Guiding and is an algorithmic solution to the Architect-Builder Problem. The algorithm leverages a learned model of the builder to guide it while the builder uses self-imitation learning to reinforce its guided behavior.

• URL:
  https://­github.­com/­flowersteam/­architect-builder-abig
• Contact:
  Tristan Karch

7.1.10 EAGER

• Name:
  Exploit question-Answering Grounding for effective Exploration in language-conditioned Reinforcement learning
• Keywords:
  Reinforcement learning, Language, Question Generation Question Answering, Reward shaping
• Functional Description:
  A novel QG/QA framework for RL called EAGER In EAGER, an agent reuses the initial language goal sentence to generate a set of questions (QG): each of these self-generated questions defines an auxiliary objective. Here, generating a question consists in masking a word of the initial language goal. Then the agent tries to answer these questions (guess the missing word) only by observing its trajectory so far. When it manages to answer a question correctly (QA) it obtains an intrinsic reward proportional to its confidence in the answer. The QA module is trained using a set of successful example trajectories. If the agent follows a path too different from correct ones at some point in its trajectory, the QA module will not answer the question correctly, resulting in zero intrinsic reward. The sum of all the intrinsic rewards measures the quality of a trajectory in relation to the given goal. In other words, maximizing this intrinsic reward incentivizes the agent to produce behaviour that unambiguously explains various aspects of the given goal.
• URL:
  https://­github.­com/­flowersteam/­EAGER
• Contact:
  Thomas Carta

7.1.11 Flow-Lenia

• Name:
  Flow Lenia: Mass conservation for the study of virtual creatures in continuous cellular automata
• Keywords:
  Cellular automaton, Self-organization
• Functional Description:

  This repo contains the code to run the Flow Lenia system which is a continuous parametrized cellular automaton with mass conservation. This work extends the classic Lenia system with mass conservation and allows to implement new feature like local parameter, environment components etc

  Several declination of the system (1 or several channels etc ) are available

  Please refer to the associated paper for the details of the system

  Implemented in JAX

• URL:
  https://­github.­com/­erwanplantec/­FlowLenia
• Contact:
  Gautier Hamon

7.1.12 Kidlearn: money game application

• Functional Description:
  The games is instantiated in a browser environment where students are proposed exercises in the form of money/token games (see Figure 1). For an exercise type, one object is presented with a given tagged price and the learner has to choose which combination of bank notes, coins or abstract tokens need to be taken from the wallet to buy the object, with various constraints depending on exercises parameters. The games have been developed using web technologies, HTML5, javascript and Django.

  

  

  

  

• URL:
  https://­flowers.­inria.­fr/­research/­kidlearn/
• Contact:
  Benjamin Clement

7.1.13 cognitive-testbattery

• Name:
  Cognitive test battery of human attention and memory
• Keywords:
  Open Access, Cognitive sciences
• Scientific Description:
  Cognitive test batteries are widely used in diverse research fields, such as cognitive training, cognitive disorder assessment, or brain mechanism understanding. Although they need flexibility according to the objectives of their usage, most of the test batteries are not be available as open-source software and not be tuned by researchers in detail. The present study introduces an open-source cognitive test battery to assess attention and memory, using a javascript library, p5.js. Because of the ubiquitous nature of dynamic attention in our daily lives, it is crucial to have tools for its assessment or training. For that purpose, our test battery includes seven cognitive tasks (multiple-objects tracking, enumeration, go/no-go, load-induced blindness, task-switching, working memory, and memorability), common in cognitive science literature. By using the test battery, we conducted an online experiment to collect the benchmark data. Results conducted on two separate days showed the high cross-day reliability. Specifically, the task performance did not largely change with the different days. Besides, our test battery captures diverse individual differences and can evaluate them based on the cognitive factors extracted from latent factor analysis. Since we share our source code as open-source software, users can expand and manipulate experimental conditions flexibly. Our test battery is also flexible in terms of the experimental environment, i.e., it is possible to experiment either online or in a laboratory environment.
• Functional Description:
  The evaluation battery consists of 6 cognitive activities (serious games: multi-object tracking, enumeration, go/no-go, Corsi, load-induced blindness, taskswitching, memorability). Easily deployable as a web application, it can be re-used and modified for new experiments. The tool is documented in order to facilitate the deployment and the analysis of results.
• URL:
  https://­github.­com/­flowersteam/­cognitive-testbattery
• Publication:
  hal-03723887
• Contact:
  Maxime Adolphe
• Participants:
  Pierre-Yves Oudeyer, Hélène Sauzéon, Masataka Sawayama, Maxime Adolphe

7.1.14 Sensorimotor-lenia

• Keywords:
  Cellular automaton, Gradient descent, Curriculum Learning
• Functional Description:
  Source code for the search of sensorimotor agency in cellular automata associated to this blogpost https://developmentalsystems.org/sensorimotor-lenia/. The code allows to find rules in the cellular automata Lenia (through gradient descent, curriculum learning and diversity search) that lead to the self-organization of moving agents robust to perturbation by obstacles.
• URL:
  https://­github.­com/­flowersteam/­sensorimotor-lenia-search
• Contact:
  Gautier Hamon

7.1.15 Lamorel

• Keywords:
  Large Language Models, Reinforcement learning, Distributed computing
• Scientific Description:

  Lamorel allows for seamless scaling of LLMs when using embodied artificial agents such as Reinforcement Learning agents. One can use and modify the LLM in any part of such agents (policy, goal sampler, social peer...). Lamorel is particularly useful when performing large-scale experiments on clusters.

  It was already used in several papers, notably leading to the first paper performing online RL on an LLM-based agent in an embodied environment (Carta et. al, 2023).

• Functional Description:

  Lamorel was initially designed to easily use LLMs in interactive environments. It is especially made for high throughput using a distributed architecture. The philosophy of *Lamorel* is to be very permissive and allow as much as possible usage of LLMs while maintaining scaling: the application should run with 1 or N LLMs.

  For this reason, it is not specialised neither in RL nor in particular in RLHF. Our examples illustrate how *Lamorel* can be used for various applications including RLHF-like finetuning. However, one must understand that *Lamorel*'s philosophy means that users must implement themselves what they want to do with the LLM(s).

  This is why we advise users knowing in advance they want to do RLHF, especially without any modification of classic implementations, to use libs specialised in RLHF that already come with RL implementations (e.g. RL4LMs, TRL). On the other hand, users more inclined to experiment with implementations or looking for an LLM lib they can use in different projects may prefer Lamorel.

  Here are Lamorel's key features: 1. Abstracts the use of LLMs (e.g. tonekization, batches) into simple calls

  2. Provides a method to compute the log probability of token sequences (e.g. action commands) given a prompt 3. Is made for scaling up your experiments by deploying multiple instances of the LLM and dispatching the computation thanks to a simple configuration file 4. Provides access to open-sourced LLMs from the Hugging Face's hub along with Model Parallelism to use multiple GPUs for an LLM instance 5. Allows one to give their own PyTorch modules to compute custom operations (e.g. to add new heads on top of the LLM) 6. Allows one to train the LLM (or part of it) thanks to a Data Parallelism setup where the user provides its own update method

• URL:
  https://­github.­com/­flowersteam/­lamorel
• Publications:
  hal-03970122, hal-04844089, hal-04844077
• Contact:
  Clément Romac

7.1.16 GLAM

• Name:
  Grounding LAnguage Models
• Keywords:
  Large Language Models, Reinforcement learning
• Scientific Description:
  Recent works successfully leveraged Large Language Models' (LLM) abilities to capture abstract knowledge about world's physics to solve decision-making problems. Yet, the alignment between LLMs' knowledge and the environment can be wrong and limit functional competence due to lack of grounding. In this paper, we study an approach (named GLAM) to achieve this alignment through functional grounding: we consider an agent using an LLM as a policy that is progressively updated as the agent interacts with the environment, leveraging online Reinforcement Learning to improve its performance to solve goals. Using an interactive textual environment designed to study higher-level forms of functional grounding, and a set of spatial and navigation tasks, we study several scientific questions: 1) Can LLMs boost sample efficiency for online learning of various RL tasks? 2) How can it boost different forms of generalization? 3) What is the impact of online learning? We study these questions by functionally grounding several variants (size, architecture) of FLAN-T5.
• Functional Description:
  GLAM is a new approach to achieve alignment between a Large Language Model (LLM) and a considered environment/world through functional grounding: we consider an agent using an LLM as a policy that is progressively updated as the agent interacts with the environment, leveraging online Reinforcement Learning to improve its performance to solve goals.
• URL:
  https://­github.­com/­flowersteam/­Grounding_LLMs_with_online_RL
• Publication:
  hal-03970122
• Contact:
  Clément Romac

7.1.17 SBMLtoODEjax

• Keywords:
  SBML, JAX, Python, Numerical simulations, Numerical optimization, Automatic differentiation, Ordinary differential equations, Biomedical data
• Scientific Description:
  Advances in bioengineering and biomedicine demand a deep understanding of the dynamic behavior of biological systems, ranging from protein pathways to complex cellular processes. Biological networks like gene regulatory networks and protein pathways are key drivers of embryogenesis and physiological processes. Comprehending their diverse behaviors is essential for tackling diseases, including cancer, as well as for engineering novel biological constructs. Despite the availability of extensive mathematical models represented in Systems Biology Markup Language (SBML), researchers face significant challenges in exploring the full spectrum of behaviors and optimizing interventions to efficiently shape those behaviors. Existing tools designed for simulation of biological network models are not tailored to facilitate interventions on network dynamics nor to facilitate automated discovery. Leveraging recent developments in machine learning (ML), this paper introduces SBMLtoODEjax, a lightweight library designed to seamlessly integrate SBML models with ML-supported pipelines, powered by JAX. SBMLtoODEjax facilitates the reuse and customization of SBML-based models, harnessing JAX's capabilities for efficient parallel simulations and optimization, with the aim to accelerate research in biological network analysis.
• Functional Description:
  SBMLtoODEjax extends SBMLtoODEpy, a python library developed in 2019 for converting SBML files into python files written in Numpy/Scipy. The chosen conventions for the generated variables and modules are slightly different from the standard SBML conventions (used in the SBMLtoODEpy library) with the aim here to accommodate for more flexible manipulations while preserving JAX-like functional programming style.
• URL:
  https://­developmentalsystems.­org/­sbmltoodejax/
• Publication:
  hal-04317246
• Contact:
  Mayalen Etcheverry
• Partner:
  Tufts University

7.1.18 Vivarium

• Name:
  Large-scale simulator for research and teaching in Artificial Intelligence and Artificial Life
• Keywords:
  Simulation, Artificial intelligence, Artificial Life, Multi-Agents System, Teaching of programming, Research
• Functional Description:

  This project aims to seize these opportunities through the design and implementation of a software platform providing an integrated simulation environment for research, teaching, and dissemination in the fields of Artificial Intelligence (AI) and Artificial Life (AL). The project is titled The Vivarium, which reflects a fundamental aspect of the convergence between these two domains: the emergence of complex behaviors, whether in the natural or artificial world, necessarily relies on a need for adaptation to a complex environment in which many autonomous entities interact.

  It will be used as an educational software in a course from CISC Master at UPF-Barcelona in January 2025.

• Release Contributions:

  This release corresponds to the state of the repo after all fixes were made following the SDIC course at Universitat Pompeu Fabra of Barcelone (CSIM Master) in January 2025 .

  This version mostly focuses on educational purposes, with ready-to-use practical sessions in notebooks/sessions. Corentin Léger was the main contributor over the last year.

• News of the Year:
  Corentin Léger, ingénieur de recherche recruté sur l'ANR JCJC ECOCURL (porté par Clément Moulin-Frier) a mené un gros travail de développement du logiciel au cours de l'année 2024. Ses applications pour l'enseignement sont maintenant validées. Le logiciel a notamment été utilisé pendant 10 heures de travaux pratiques dans le Master CSIC de Universitat Pompeu Fabra à Barcelone, Espagne.
• URL:
  https://­github.­com/­flowersteam/­vivarium
• Contact:
  Clément Moulin-Frier
• Participants:
  Corentin Leger, Clément Moulin-Frier, Martial Marzloff

7.1.19 LLM_Culture

• Keywords:
  LLM, Multi-Agents System, Natural language processing
• Functional Description:

  Code for the 'Cultural evolution in populations of Large Language Models' paper. This repository provides a comprehensive framework for studying the cultural evolution of linguistic content in populations of Large Language Models (LLM).

  It allows organizing LLM agents into networks wherein each agent interacts with neighboring agents by exchanging texts. Each agent can be assigned specific personalities and transmission instructions, serving as prompts for generating new texts from their neighbors’ narratives. Once the network structure and agent characteristics are defined, you can simulate the cultural evolution of texts across generations of agents. We also provide built-in metrics and vizualizations to analyze the results.

• URL:
  https://­github.­com/­flowersteam/­LLM-Culture
• Contact:
  Jeremy Perez
• Participants:
  Jeremy Perez, Corentin Leger

7.1.20 TelephoneGameLLMs

• Keywords:
  Large Language Models, Multi-Agents System, Cultural Evolution
• Functional Description:
  Code for the paper "When LLMs Play the Telephone Game: Cumulative Changes and Attractors in Iterated Cultural Transmissions" https://arxiv.org/abs/2407.04503 In this paper, we introduce conceptual and methodological tools for evaluating Large Language Models in multi-turn settings. Those tools are inspired by cultural evolutionary theory, and in particular by the concepts of cultural attractors.
• URL:
  https://­github.­com/­flowersteam/­TelephoneGameLLM
• Publication:
  hal-04714994
• Contact:
  Jeremy Perez

7.1.21 styr

• Name:
  Stick To Your Role
• Keywords:
  LLM, Cognitive sciences
• Functional Description:

  Code for our paper https://arxiv.org/abs/2402.14846 and leaderboard https://huggingface.co/spaces/flowers-team/StickToYourRoleLeaderboard.

  Enables evaluating LLMs using personal value questionnaires (PVQ, SVS). More precisely, it instructs the LLM to simulated various personas and exposes it to different contexts (e.g. long reddit posts). Then it evaluates the value stability of the simulated population between those contexts. Additionally, it computes confirmatory factor analysis (CFI, SRMR, RMSEA), and the structure of expressed values (stress metric).

• URL:
  https://­github.­com/­flowersteam/­value_stability
• Contact:
  Grgur Kovac

7.1.22 transformerXL_PPO_JAX

• Keywords:
  Reinforcement learning, Transformer
• Functional Description:

  This repository provides a JAX implementation of TranformerXL with PPO in a RL setup following : "Stabilizing Transformers for Reinforcement Learning" from Parisotto et al. (https://arxiv.org/abs/1910.06764).

  The code uses the PureJaxRL template for PPO and copied some of the code from hugging face trasnformer XL repo transferring it to JAX. We also took inspiration from the pytorch code in https://github.com/MarcoMeter/episodic-transformer-memory-ppo, which has some simplification of gradient propagation and positional encoding compared to transformerXL as it is described in the original paper (https://arxiv.org/abs/1901.02860). The training handles [Gymnax](https://github.com/RobertTLange/gymnax) environment.

  We also tested it on Craftax, on which it beat the baseline presented in the paper (https://arxiv.org/abs/2402.16801) including PPO-RNN, training with unsupervised environment design and intrinsic motivation. Notably we reach the 3rd level (the sewer) and obtain several advanced advancements, which was not achieved by the methods presented in the paper. See Craftax Results for more informations.

  The training of a 5M transformer on craftax for 1e9 steps (with 1024 environments) takes about 6h30 on a single A100.

• Contact:
  Gautier Hamon

7.1.23 ER-MRL

• Keywords:
  Reinforcement learning, Evolutionary Algorithms, Recurrent network
• Functional Description:

  Code for the "Evolving-Reservoirs-for-Meta-Reinforcement-Learning" (ER-MRL) paper (https://arxiv.org/abs/2312.06695).

  We adopt a computational framework based on meta reinforcement learning, modeling the interplay between evolution and development. At the evolutionary scale, we evolve reservoirs, a family of recurrent neural networks generated from hyperparameters. These evolved reservoirs are then utilized to facilitate the learning of a behavioral policy through reinforcement learning. This is done by encoding the environment state through the reservoir before providing it to the agent's policy.

• Contact:
  Corentin Leger

7.2.1 ToGather application

Participants: Cécile Mazon, Hélène Sauzéon, Eric Meyer, Isabeau Saint-Supery.

• Name:
  Application for Specialized education
• Keywords:
  Parent-professional relationships; user-centered design; school inclusion; autism spectrum disorder; ecosystemic approach
• Participants:
  Isabeau Saint-supery, Cécile Mazon, Hélène Sauzéon, Agilonaute
• Scientific Description:
  With participatory design methods, we have designed an interactive website application for educational purposes. This application aims to provide interactive services with continuously updated content for the stakeholders of school inclusion of children with specific educational needs. Especially, the services provide: 1) the student's profile with strengths and weaknesses; 2) an evaluation and monitoring over time of the student's repertoire of acquired, emerging or targeted skills; 3) a shared notebook of effective psycho-educational solutions for the student ; 4) a shared messaging system for exchanging "news" about the student and his/her family and, 5) a meeting manager allowing updates of evaluations (student progress). This application is currently assessed with a field study. Then, it will be transferred to the Academy of Nouvelle-Aquitaine-Bordeaux of the National Education Ministery.
• URL:
  The website is not online yet, but all informations such as tutorials are here.
• Publication:
  hal-03436355

8.1.1 ACES: Generating a Diversity of Challenging Programming Puzzles with Autotelic Generative Models

Participants: Julien Pourcel [correspondant], Cédric Colas, Gaia Molinaro, Pierre-Yves Oudeyer, Laetitia Teodorescu.

Results. 

We report results of our runs in Figure 3. Overall, the methods based on semantic archives, ACES and ELM-Semantic, achieve the highest diversity in the semantic space. We report diversity measures inside the embedding spaces of various smaller language models in Figure 4. In these figures we see that overall ACES outperforms other methods in this measure of diversity. We additionally perform tests of the suitability of generated puzzles as finetuning data for smaller LMs. For all methods, we finetune a smaller model (OpenLlama-3b) on the generated set and we test the pass@k metric for different values of k on the P3 test set; we report the scores in Figure 5. From that figure we see that we encounter a tradeoff between how diverse the data is and how useful it is to get a high score on the P3 test set. Further work is needed to get data that is both diverse and useful.a

8.1.2 MAGELLAN: Metacognitive Generalization of Learning Progress for Online RL in LLM agents

Participants: Loris Gaven [correspondant], Thomas Carta, Clément Romac, Cédric Colas, Pierre-Yves Oudeyer, Olivier Sigaud [ISIR Sorbonne Université, Paris, France], Sylvain Lamprier [Univ Angers, LERIA].

We are developing MAGELLAN, a method designed to enable LLM-based reinforcement learning (RL) agents to estimate their own Learning Progress (LP) and use it to dynamically organize their training curriculum. By leveraging the LLM's rich semantic representations, MAGELLAN allows agents to generalize LP estimations to unseen, language-defined goals, overcoming limitations of classical methods that require direct evaluation of each goal.

MAGELLAN uses the LLM to generate latent representations of goals and tasks, capturing their semantic relationships. It continuously monitors the agent's performance over time, estimating LP as the change in success rates for specific goals. This approach enables the agent to identify goals where it is making progress and focus its training on those areas. MAGELLAN's integration ensures that the LLM-based agent can simultaneously refine its policy and competence estimations, adapting both to new tasks in real time.

Our experiments in the Little-Zoo environment, which features hierarchical and commonsense-driven tasks, demonstrate that MAGELLAN effectively prioritizes high-LP goals, even when faced with novel or unseen tasks. Unlike traditional LP estimation methods, which rely on direct evaluations and struggle with generalization, MAGELLAN enables the agent to quickly identify meaningful learning opportunities. This results in faster adaptation, improved sample efficiency, and more effective curriculum organization, paving the way for truly autonomous agents capable of navigating vast and complex goal spaces.

8.1.3 SAC-GLAM: Improving Online RL for LLM agents with Soft Actor-Critic and Hindsight Relabeling

Participants: Loris Gaven [correspondant], Thomas Carta, Clément Romac, Pierre-Yves Oudeyer, Olivier Sigaud [ISIR Sorbonne Université, Paris, France], Sylvain Lamprier [Univ Angers, LERIA].

We developed SAC-GLAM, a novel reinforcement learning approach designed for Large Language Model (LLM) agents. By adapting Soft Actor-Critic (SAC) with Hindsight Experience Replay (HER), SAC-GLAM addresses key limitations of on-policy methods like PPO-GLAM, enabling more efficient and effective learning in complex environments.

SAC-GLAM leverages HER to reframe failed attempts as learning opportunities by relabeling goals based on achieved outcomes, significantly improving sample efficiency. Additionally, the method integrates pre-trained LLM policies with a randomly initialized critic, incorporating a warm-up phase and architectural enhancements to ensure stable and effective training. We also used n-step returns to further enhance sample efficiency.

Our approach was evaluated in Playground-Text, a text-based environment featuring sequential and multi-goal tasks. Results showed that SAC-GLAM outperformed PPO-GLAM in sample efficiency while maintaining comparable time performance, even when extended with HER.

This work represents a step forward in creating autonomous, self-motivated LLM agents capable of setting and pursuing their own goals, leveraging language understanding to learn more effectively. It highlights the potential of combining off-policy RL techniques with LLMs to tackle complex, multi-goal tasks in text-based environments.

8.1.4 Learning when (not) to use tools with Memory-based In-context learning and online multi-objective RL in LLM agents

Participants: Clément Romac [correspondant], Pierre-Yves Oudeyer.

Augmenting Large Language Models (LLMs) with tools they can call is now seen as a natural way to enhance their abilities. In most prior work, very large LLMs were used and calling tools was not assumed costly in the light of the model's size. In this paper, we argue that using tools is particularly important for smaller-scale LLMs, and that it should be assumed as a costly decision regarding the lower compute needs from such small models. We propose to study, in such conditions, the central question of when should tools be used, which we show depends on 1) a model's ability to solve tasks without tools and 2) how costly tools are.

We take the perspective of an autonomous agent, where the LLM should explore and discover by itself when tools are helpful for it to minimize its answers' errors and tool usage (see Figure 7). We show that current methods fall short of finding the optimal strategy when tools are costly. Notably, we argue this comes both from an inability to properly explore (as identified by prior work) and an inability to reinforce good strategies quickly.

We introduce a new method that aims to solve these two issues. We take inspiration from Episodic Control and use an external memory to store and retrieve the best trajectories. When appended to the prompt, we show these trajectories, along with the in-context learning abilities of LLMs, allowing the model to both try new strategies and reproduce the best ones depending on the instruction. However, Episodic Control alone easily gets stuck in local optima using deceptive strategy. To overcome this, we use an off-policy online RL method named AWR. While necessitating as few changes on the LLM as current methods used (e.g. PPO), AWR's off-policy nature allows the agent to reinforce best strategies and escape from local optima efficiently.

8.1.5 WorldLLM: Improving LLMs' world modelling using Curiosity-driven Theory-making

Participants: Guillaume Levy, Clément Romac [correspondant], Thomas Carta, Cédric Colas, Pierre-Yves Oudeyer.

Large Language Models (LLMs) possess various knowledge about the world that allows them to answer questions or propose plans for solving tasks. Yet, this knowledge can be wrong or incomplete, especially when considering problems with specific properties (e.g. counterintuitive). To this day, aligning the world modelling abilities of LLMs remains a central and open challenge. Prior work notably showed that allowing LLMs to perform embodied experiences can improve their world model 96.

Learning a world model of an environment has been extensively studied in the Reinforcement Learning (RL) community. A large body of work focused on the forward model aspect, that is, predicting what will happen (i.e. the next observation) given an action and what happened before (i.e. current observation). While learning such a forward model can allow better planning or policy improvement without interacting with the environment, it has also been shown useful to foster exploration. Notably, ICM 161 showed how the prediction error of a learned forward model can be used as an intrinsic reward motivating the policy to explore and find transitions unknown by the world model.

Parallel to this, Theory-based RL 185 proposes to integrate human-inspired inductive reasoning to produce a world model able to predict causal relations based on intuitive theories. The use of intuitive theories notably allows humans to generalize their world model and quickly adapt it by improving their theories.

In this paper, we propose to study how to improve the world modelling abilities of an LLM. Inspired by Theory-based RL and following recent work, we propose to see the LLM as our causal model of the environment (i.e. the forward model) and to search for natural language theories that would improve its ability to predict the environment's dynamics. We argue that generating rules that help predict the world fosters abstract and general rules that humans can interpret. To obtain such rules, we rely on the inductive reasoning abilities of LLMs together with Bayesian Inference. Inspired by ICM, we also propose to actively collect the experiences used to induce rules by leveraging an RL agent solely trained on an intrinsic reward provided by the LLM-based forward model's error.

To summarize, we introduce WorldLLM, a method for improving an LLM's world modelling capabilities. It is constituted of 1) a forward model leveraging the LLM we want to improve, 2) another LLM incrementally inducing theories using a Bayesian Inference framework, and 3) an RL agent that looks for experiences the LLM-based forward model fails to predict given the current theories. Inspired by how humans and, in particular, scientists explain the world, our method iteratively proposes theories, designs experiments to challenge them and refines the theories according to the experiments' outcome. We show that, upon perfect data collectors replacing our RL agent, our framework generates rules that significantly improve an LLM's ability to predict the outcome of actions. But more importantly, we show that random data collection is insufficient to cover the space of possible outcomes. By inciting the RL agent to find transitions the forward model does not predict well, we show our LLM successfully model the space of possible outcomes.

8.1.6 Reinforcement Learning for Aligning Large Language Models Agents with Interactive Environments: Quantifying and Mitigating Prompt Overfitting

Participants: Mohamed Salim Aissi [ISIR Sorbonne Université, Paris, France], Clément Romac [correspondant], Thomas Carta, Sylvain Lamprier [Univ Angers, LERIA], Pierre-Yves Oudeyer, Olivier Sigaud [ISIR Sorbonne Université, Paris, France], Laure Soulier [ISIR Sorbonne Université, Paris, France], Nicholas Thome [ISIR Sorbonne Université, Paris, France].

In GLAM 96, we showed an environment's dynamics could be explicitly integrated into an LLM using Reinforcement Learning (RL) to fine-tune the LLM to solve sequential decision-making tasks. While this showed one could ground an LLM's abilities at manipulating words related to environments' dynamics, it remains unclear how RL impacts the inherent knowledge of the LLM and whether the model gains generalization capabilities.

In this paper, we study the sensitivity of LLMs to prompt formulations and its impact on knowledge acquisition. We define a set of different prompt formulations to evaluate LLM performance when prompts are changed, and also analyze how the LLM represents these prompts. Our findings reveal that the performance of LLMs is highly sensitive to prompt variations, suggesting that fine-tuning only induces superficial updates and fails to improve the acquisition of new knowledge about the environment. By analogy with observational overfitting in RL, we refer to this phenomenon as prompt overfitting. With this in mind, our paper proposes two main contributions:

• We design experiments to measure prompt overfitting issues of LLMs in interactive environments. The study reveals that fine-tuned LLMs heavily depend on the training prompts, exhibiting a significant drop in "zero-shot" performance when using new prompt formulations. To further analyze this behavior, we thoroughly analyze latent representations and salient tokens in LLMs, both showing a strong bias towards the prompt formulation.
• We design experiments to measure prompt overfitting issues of LLMs in interactive environments. The study reveals that fine-tuned LLMs heavily depend on the training prompts, exhibiting a significant drop in "zero-shot" performance when using new prompt formulations. To further analyze this behavior, we thoroughly analyze latent representations and salient tokens in LLMs, both showing a strong bias towards the prompt formulation.
• We propose a solution for mitigating prompt overfitting with a contrastive regularization loss that makes the latent representation of the LLM invariant to prompt formulations. This solution significantly improves the zero-shot performance and the robustness to prompt variations, as well as the acquisition of new knowledge about the environment. See Figure 8.

8.1.7 Jack of All Trades, Master of Some, a Multi-Purpose Transformer Agent

Participants: Clément Romac [correspondant], Quentin Gallouédec [Ecole Centrale de Lyon, CNRS, Universite Claude Bernard Lyon 1, INSA Lyon, Université Lumière Lyon 2, LIRIS, UMR5205, 69130 Ecully, France], Edward Beeching [Hugging Face], Emmanuel Dellandréa [Ecole Centrale de Lyon, CNRS, Universite Claude Bernard Lyon 1, INSA Lyon, Université Lumière Lyon 2, LIRIS, UMR5205, 69130 Ecully, France].

We present the Jack of All Trades (JAT) model, a transformer-based generalist agent capable of high performance on tasks ranging from Atari to MuJoCo, while being able to generate text and image captions. Inspired by GATO 165, we collect what is, to our knowledge, the largest open dataset for training generalist agents. This dataset features hundreds of thousands of expert trajectories.

We then trained a transformer model with Behavioral Cloning on these trajectories. To overcome the challenge of unifying the different observation and action spaces of our various environments, we designed a new approach, simpler than previous methods, using a single embedding space and one token per action or observation (see Figure 9). We also propose to add an auxiliary loss turning our JAT model into a forward model (i.e. predicting the next observation given the previous observation and action). We show this auxiliary task improves our model's abilities. Finally, we also train our model using a Language Modeling loss on text generation and image captioning datasets.

We show our model achieves strong performance on all the tasks it was trained with Behavioral Cloning. These results are above human-level performance on multiple tasks and on par with results reported by GATO. Our JAT model also exhibits rudimentary language modelling abilities. These results can be explained by the small-scale datasets used for text generation and image captioning.

We open-source our dataset along with the experts used to collect trajectories as well as the trained JAT model. We believe our work can act as a foundation model for future agents while opening up new avenues in designing generalist agents.

8.1.8 LLM-based goal generation for autotellic exploration with goal-conditioned RL

Participants: Guillaume Pourcel [correspondant], Grgur Kovač, Thomas Carta, Cédric Colas, Pierre-Yves Oudeyer.

Autotelic agents, capable of autonomously generating and pursuing their own goals, a represent promising approach to open-ended learning and skill acquisition in reinforcement learning. Such agents learn to set and pursue their own goals. This challenge is even more difficult in open worlds that require inventing new previously unobserved goals. In this work, we propose an architecture where a single generalist autotelic agent is trained on an automatic curriculum of goals. We leverage a large language models (LLMs) to generate goals as code for reward functions based on learnability and difficulty estimates. The goal-conditioned RL agent is trained on those goals sampled based on learning progress. We evaluate our method in terms of novely, difficuly, and learability of the generated goals. Our preliminary experiments imply that our method generates goals of increasing difficulty while maintaing their learnability and diversity.

8.2.1 The effect of social network structure on collective innovation

Participants: Eleni Nisioti [correspondant], Mateo Mahaut, Pierre-Yves Oudeyer, Ida Momennejad, Sebastian Risi, Pierre-Yves Oudeyer, Clément Moulin-Frier.

Innovations are a central component of open-ended skill acquisition: they denote the emergence of new solutions by the recombination of existing ones and their presence is necessary to ensure a continuous complexification of an agent's cultural repertoire. While we often tend to attribute discoveries to certain innovative individuals, if we shed a broad perspective at the history of our species we see that human innovation is primarily a collective process. Fields such as psychology and anthropology have been studying the ability of human groups to innovate for some time, with studies indicating that the social network structure has a significant impact: fully-connected structures are better suited for quick convergence in easy problems with clear global optima, while partially-connected structures perform best in difficult tasks where local optima may lure agents away from the globally optimal solution 111. At the same time a parallel story is unfolding in reinforcement learning (RL): distributed RL is a sub-field where multiple agents solve a task collectively 151. Compared to the single-agent paradigm, distributed RL algorithms converge quicker and often achieve superior performance. However, these algorithms have only considered full connectivity. In this inter-disciplinary project, we presented a novel learning framework that augments distributed RL with the notion of a social network structure and employed it to study the hypothesis from human studies that partial connectivity performs best in innovation tasks.

Cultural evolution in populations of RL agents. 

We implemented such innovation tasks using Wordcraft, a recently introduced RL playground inspired from the Little Alchemy 2 game (see left of figure 11 for an illustration of how this task works). We considered a wide diversity of social network structures: static structures that remain constant throughout learning (fully-connected, ring, small-world) and a dynamic structure where the group oscillates between phases of low and high connectivity (we illustrate this dynamic structure on the right of figure 11). Each agent in our implementation employs the DQN learning algorithm and exchanges experiences that have the form of sequences of state-action combinations with its neighbors.

A central conclusion of our empirical analysis was that the dynamic social network structure performs best. In addition to the performance groups achieve we measured behavioral and mnemonic metrics such as behavioral conformity and mnemonic diversity. Such metrics were inspired from human studies and helped us further analyze the behavior of groups. For example, one empirical observation was that sharing experiences did not help the group learn quicker in a very simple innovation task; instead the fully-connected group was the slowest. By looking at the diversity in the memories of the agents we observed that the fully-connected structure had the highest individual diversity (left of figure 12 ) and the lowest group diversity (right of figure 12): sharing experiences with others diversifies an individual's experiences but also homogenizes the group, which is bad for its performance.

We see the contribution of this project as two-fold. From the perspective of fields studying human intelligence, we have shown that using RL algorithms as computational tool is a promising direction towards increasing the verisimilitude of simulations and analyzing both behavior and memory. From the perspective of RL, we have shown that distributed RL algorithm should move beyond the fully-connected architecture and explore groups with dynamic topologies. This work is currently a preprint 154 and is about to be submitted in PNAS. We open-source the code at this link.

Cultural evolution in populations of LLM agents. 

In 2024, we have extended this framework with agents equipped with Large Language Models (LLMs) playing Little Alchemy 2, a creative video game originally developed for humans (figure 13). We, first, study an LLM in isolation and discover that it exhibits both useful skills and crucial limitations. We, then, study groups of LLMs that share information related to their behaviour and focus on the effect of social connectivity on collective performance. In agreement with previous human and computational studies (including the one described above), we observe that groups with dynamic connectivity out-compete fully-connected groups. Our work reveals opportunities and challenges for future studies of collective innovation that are becoming increasingly relevant as Generative Artificial Intelligence algorithms and humans innovate alongside each other. We published this work at the ALife 2024 conference 52.

8.2.2 The SocialAI School: Insights from Developmental Psychology Towards Artificial Socio-Cultural Agents

Participants: Grgur Kovač [correspondant], Remy Portelas, Peter Ford Dominey, Pierre-Yves Oudeyer.

This project aims to present a tool facilitating research into socio-cognitive AI agents. The project was finished in 2023 and published in 2024.

8.2.3 Stick to your Role! Stability of Personal Values Expressed in Large Language Models

Participants: Grgur Kovač [correspondant], Jeremy Perez, Remy Portelas, Masataka Sawayama, Peter Ford Dominey, Pierre-Yves Oudeyer.

Following our perspective paper 133 from last year we continued this project with an empricial paper 47, and then with a Leaderboard73. The standard way to study Large Language Models (LLMs) with benchmarks or psychology questionnaires is to provide many different queries from similar minimal contexts (e.g. multiple choice questions). However, due to LLMs' highly context-dependent nature, conclusions from such minimal-context evaluations may be little informative about the model's behavior in deployment (where it will be exposed to many new contexts). We argue that context-dependence (specifically, value stability) should be studied as a specific property of LLMs and used as another dimension of LLM comparison (alongside others such as cognitive abilities, knowledge, or model size). In the empricial paper 47, we present a case-study on the stability of value expression over different contexts (simulated conversations on different topics) as measured using a standard psychology questionnaire (PVQ) and on behavioral downstream tasks. Reusing methods from psychology, we study Rank-order stability on the population (interpersonal) level, and Ipsative stability on the individual (intrapersonal) level. We consider two settings (with and without instructing LLMs to simulate particular personas), two simulated populations, and three downstream tasks. We observe consistent trends in the stability of models and model families - Mixtral, Mistral, GPT-3.5 and Qwen families are more stable than LLaMa-2 and Phi. The consistency of these trends implies that some models exhibit higher value stability than others, and that stability can be estimated with the set of introduced methodological tools. When instructed to simulate particular personas, LLMs exhibit low Rank-order stability, which further diminishes with conversation length. This highlights the need for future research on LLMs that coherently simulate different personas. This paper provides a foundational step in that direction, and, to our knowledge, it is the first study of value stability in LLMs. For the leaderboard73, we extend the methodology with Confirmatory Factor Analysis and Stress metrics, which test for the structure of expressed values. Many new models were compared in the leaderboard, and it is regularly updated.

8.2.4 Cultural evolution in populations of Large Language Models

Participants: Jérémy Perez [correspondant], Corentin Léger, Marcela Ovando-Tellez, Chris Foulon, Joan Dussauld, Pierre-Yves Oudeyer, Clément Moulin-Frier.

Research in cultural evolution aims at providing causal explanations for the change of culture over time. Over the past decades, this field has generated an important body of knowledge, using experimental, historical, and computational methods. While computational models have been very successful at generating testable hypotheses about the effects of several factors, such as population structure or transmission biases, some phenomena have so far been more complex to capture using agent-based and formal models. This is in particular the case for the effect of the transformations of social information induced by evolved cognitive mechanisms. We here proposed that leveraging the capacity of Large Language Models (LLMs) to mimic human behavior may be fruitful to address this gap. On top of being an useful approximation of human cultural dynamics, multi-agents models featuring generative agents are also important to study for their own sake. Indeed, as artificial agents are bound to participate more and more to the evolution of culture, it is crucial to better understand the dynamics of machine-generated cultural evolution.

In this project, we presented a framework for simulating cultural evolution in populations of LLMs, allowing the manipulation of variables known to be important in cultural evolution, such as network structure, personality, and the way social information is aggregated and transformed. The software we developed for conducting these simulations is open-source and features an intuitive user-interface.

The model simulates the cultural evolution of linguistic content in a population LLMs. Each agent can be seen as an independent instance of a LLM. The agents are arranged according to a specified social network structure. At the first generation, all agents are prompted with an Initialization Prompt (e.g. “Tell me a story”). All agents then output an answer by passing the initialization prompt to their respective instance of the LLM. The agents then transmit stories to their neighbors (according to the specified network structure): each agent receives a new prompt, which is the concatenation of a Transformation Prompt (e.g. “Combine the stories you received”) and the list of stories produced by its neighbors at the previous generation. Agents can additionally be provided with a personality (e.g. “You are very imaginative”).

We also introduce several visualizations and measures that are useful for tracking the results of the simulations, namely the similarity matrix, the similarity graph, and the evolution of the words used at each generation.

We find some interesting dynamics, such as the fact that stories seem to evolve in a punctuated manner: there is an alternation of phases where stories are transmitted with no modifications, and phases where stories are modified. These dynamics recall previous observations from experiments and modelling work in cultural evolution [24] [46] [35] [34]. The effect of network structure also mirrors results found in humans, with more efficient networks (that is, networks with smaller average path length) leading to quicker diffusion of information and thus lower diversity. Finally, we found that manipulating personality appears to affect cultural dynamics; for instance, assigning a “creative” personality to agents led groups to generate more diversity.

Overall, this contribution shows how using generative agents to simulate cultural evolution may be promising, both for generating hypotheses about human cultural evolution and for better understanding the dynamics of machine-generated culture.

The article is available as a pre-print 68, and we open-source the code here.

8.2.5 When LLMs Play the Telephone Game: Cumulative Changes and Attractors in Iterated Cultural Transmissions

Participants: Jérémy Perez [correspondant], Grgur Kovač, Corentin Léger, Cédric Colas, Gaia Molinaro, Maxime Derex, Pierre-Yves Oudeyer, Clément Moulin-Frier.

As large language models (LLMs) start interacting with each other and generating an increasing amount of text online, it becomes crucial to better understand how information is transformed as it passes from one LLM to the next. While significant research has examined individual LLM behaviors, existing studies have largely overlooked the collective behaviors and information distortions arising from iterated LLM interactions. Small biases, negligible at the single output level, risk being amplified in iterated interactions, potentially leading the content to evolve towards attractor states.

In this project, we ran a series of telephone game experiments, applying a transmission chain design borrowed from the human cultural evolution literature: LLM agents iteratively receive, produce, and transmit texts from the previous to the next agent in the chain.

This figures depicts the method to estimate the strength and position of theoretical attractors. Each dot in this figure corresponds to one chain, for a total of 100 chains (20 initial texts * 5 seeds). The position of a dot on the x-axis corresponds to the value of the property (positivity in this example) in the initial text, while the position on the y-axis corresponds to the value of this property of the text produced after 50 generations. We then used these 100 data points to fit a linear regression predicting the relationship between the initial and final values of the property.

Our main contributions are:

• We propose that there might be a gap in current LLM evaluations methods (single-turn evaluations might not be suited to assess the properties of multi-turn interactions)
• We empirically confirm this hypothesis by showing that multi-turn interactions indeed often lead to distributions of text properties that are significantly different from what is observed after a single interaction.
• We introduce novel conceptual and methodological tools to fill this gap, grounded in research in cultural evolution, and in particular the concept of cultural attractor.
• We showcase the potential of this method by applying it to compare the effect of different tasks, of different models, of temperature, and of fine-tuning on the properties of multi-turn interactions.
• We find several robust effects, such as the fact that less constrained tasks lead to stronger attractors, that some properties posses stronger attractors than others, and that fine-tuning can shift the position and modify the strength of attractors.

The heigth of the bars represent the position (top row) and strength (bottom row) of theoretical attractors, for each property (columns), task, and model. Less constrained tasks, such as Continue, appear to produce stronger attractors than more constrained tasks, such as Rephrase. Attractors appear to be stronger for toxicity than for length. Finally, we can notice that the position of attractors appears to vary between models.

These findings highlight the importance of accounting for multi-step transmission dynamics and represent a first step towards a more comprehensive understanding of LLM cultural dynamics.

This work was presented during a 15-minutes talk given at the Cultural Evolution Society conference. The article is available as a pre-print 67, and we open-source the code here. We also created a website featuring a Data Explorer tool, allowing to directly inspect the texts generated during our experiments. An extended version of the paper has been accepted at the ICLR'2025 conference (one of the three main international conferences in machine learning).

8.2.6 Collapse of diveristy and quality in the cultural evolution of Large Language models

Participants: Grgur Kovač [correspondant], Jérémy Perez [correspondant], Remy Portelas, Peter Ford Dominey, Pierre-Yves Oudeyer.

LLMs are increasingly contributing to the creation of content on Internet. This creates a feedback loop as next generations of models will be trained on this generated, synthetic data. This phenomenon is receiving increasing interest, in particular because previous studies have shown that it may lead to drastic losses in performance and quality, an outcome commonly referred to as “model collapse” 175. While studying model collapse and mitigation strategies is under active investigation, existing studies have so far ignored how features of the data that the model is trained to learn impact collapse. Indeed, different online domains contain data that possess different properties. Some domains may be associated with a comparatively higher ratio of synthetic-to-human data (e.g. GitHub), while others may contain data of comparatively lower quality (e.g. Reddit). If future specialized models are trained on some specific internet domains, we may therefore expect different rates and forms of model collapse depending on the features of the data they are trained on. To empirically evaluate whether model collapse is influenced by dataset properties, we ran iterative fine-tuning experiments manipulating various features of the initial data. Specifically, we varied the ratio of synthetic-to-human data, as well as the quality and the diversity of the initial human data. To measure model collapse along several dimensions, we track how repeated fine-tuning influences the quality and diversity of the data generated by the LLMs. We find that while low ratios of synthetic-to-human data tend to prevent model collapse, the effect of this parameter is not linear. We also show that quality has a very noticeable effect on model collapse, leading to very different dynamics on different internet domains (e.g. Political Twitter vs Reddit). Diversity, on the other hand, does not appear to have such a strong effect. Finally, we find that the tendencies observed when running iterative chains are preserved when scaling the number of agents from 20 to 400, which suggests that using this simplified setting is indeed suited for making prediction about real-world dynamics. Overall, our work extends the existing literature on model collapse by showing that this phenomenon is highly dependent on the human data on which training occurs. This predicts that we might observe collapse at different paces and along different dimensions for different internet domains.

8.2.7 Inferring the Phylogeny of Large Language Models and Predicting their Performances in Benchmarks

Participants: Nicolas Yax [correspondant], Pierre-Yves Oudeyer, Stefano Palminteri.

In recent month the number of Large Language Models (LLMs) released has never been that high. On one hand, multiple private companies such as OPENAI, Claude, Google, Mistral, etc. are making cutting-edge models that have a lot of visibility in our modern society and science. However, as the number of LLMs is raising, the training methods are becoming more secretive making the field increasingly obscur to science. On the other hand, everyday, a few hundreds of open-access language models are uploaded on the hugging face hub which is far too much to keep track of the evolution of LLMs in the field. Knowing that not all of these open models are perfectly transparent about the training methods and only very few of them are benchmarked (due to the high cost of benchmarking) there is an increasing need for methods to help keep track of the progress and evolution of these models in the field.

We developped an algorithm inspired from phylogenetics to compute evolutionary trees in LLMs. We show this method efficient in reconstructing the evolutionary history of LLMs within families 20, in discriminating the different families, and also in finding similarities between these families. Additionaly, the genetic information can be used to predict LLM capabilities like benchmark scores showing a very significant correlation between predicted and true scores. These advances could be instrumental in our way to navigate the field of LLMs by making the world of LLM more transparent at a very low cost.

8.3.1 Research perspective: The Ecology of Open-Ended skill Acquisition

Participants: Clément Moulin-Frier [correspondant], Eleni Nisioti, Pierre-Yves Oudeyer.

An intriguing feature of the human species is our ability to continuously invent new problems and to proactively acquiring new skills in order to solve them: what is called Open-Ended Skill Acquisition (OESA). Understanding the mechanisms underlying OESA is an important scientific challenge in both cognitive science (e.g. by studying infant cognitive development) and in artificial intelligence (aiming at computational architectures capable of open-ended learning). Both fields, however, mostly focus on cognitive and social mechanisms at the scale of an individual’s life. It is rarely acknowledged that OESA, an ability that is fundamentally related to the characteristics of human intelligence, has been necessarily shaped by ecological, evolutionary and cultural mechanisms interacting at multiple spatiotemporal scales.

We have recently initiated a new research direction aiming at understanding, modeling and simulating the dynamics of OESA in artificial systems, grounded in theories studying its eco-evolutionary bases in the human species. For this aim, we have proposed a conceptual framework, called ORIGINS (illustrated Fig. 21 and developed in 149), expressing the complex interactions between environmental, adaptive, multi-agent and cultural dynamics. This framework raises three main research questions:

• What are the ecological conditions favoring the evolution of autotelic agents?
• How to bootstrap the formation of a cultural repertoire in populations of adaptive agents?
• What is the role of cultural feedback effects in the open-ended dynamics of human skill acquisition?

The contributions described below are addressing some aspects of these research questions. Note that there might be a thematic overlap between the two last research questions outlined above and the previous section on Models of Cultural Evolution 8.2, where we also present related results.

8.3.2 Eco-evolutionary Dynamics of Non-episodic Neuroevolution in Large Multi-agent Environments

Participants: Gautier Hamon [correspondant], Eleni Nisioti, Clément Moulin-Frier.

This work was published in 2023 but we keep it in this report as it introduces a general computational framework, called non-episodic neuroevolution, that forms the basis of the two next contributions.

This contribution focuses on eco-evolutionary dynamics where "organisms are not solely products but, by modifying their niche and therefore its associated fitness landscape, are also causes of evolution" 134. The main objective of this paper is to propose a method for studying large-scale eco-evolutionary dynamics in agent-based simulations with a reasonable level of biological and ecological plausibility. For this aim, we implement a system with the following properties (see Fig. 22 for illustration):

• Non-episodic simulation environment with complex intrinsic dynamics. We model our environment after common-pool resource (CPR) appropriation problems, where a group of agents competes for finite resources. We extend an existing environment of CPR appropriation 163 with the presence of multiple niches, where resources regrow proportionally to the density of nearby resources at different rates in different regions of the environment (Fig 22). We prevent any environment or population reset during a whole simulation run, enabling coupled environmental and population dynamics leading to complex eco-evolutionary feedback effects.
• Continuous neuroevolution in a large, size-varying agent population The environment contains thousands of agents, each controlled by a neural network whose weights are optimized using neuroevolution 179
• Physiology-driven death and reproduction There is no notion of rewards, agents are instead equipped with a physiological system modulating their energy level according to the resources they consume, in a non-linear way. At the evolutionary scale, agents reproduce as long as they are able to maintain their energy level within a reasonable range and die if this level goes below a minimum threshold. This is departure from the notion of fitness-based selection and more in line with a minimal criterion selection 92. Note that the population size can vary with time.

In addition to experiments conducted in the large environment presented, we also conduct experiments in "lab environment" (as opposed to the "natural environment") to isolate the study of certain behavior (which are often intertwined with a lot of dynamics in the natural environment).

One interesting results of these simulation is the emergence of sustainable foragers which as shown in lab environment Fig.23 tends to not overconsume when there is enough resource in their neighbourhood. This allows to keep a certain amount of resource to spread which is therefore beneficial for their future survival as well as the survival of their offspring. (as there is no reset of the environment)

This work was published at the Genetic and Evolutionary Computation Conference (GECCO) 2023. The computational framework it introduced led to the two next recent contributions.

8.3.3 Emergent kin selection of altruistic feeding via non-episodic neuroevolution

Participants: Max Taylor-Davies, Gautier Hamon, Timothe Boulet, Clément Moulin-Frier [correspondant].

This work extends the project presented in previous contribution Sec.8.3.2. It is the result from the visit in the team of Max Taylor-Davies doing his PhD at School of Informatics, University of Edinburgh, Scotland. It has been accepted at the EvoStar conference the International Conference on the Applications of Evolutionary Computation (Part of EvoStar) 54.

At first glance, it seems difficult to square the phenomenon of purely altruistic behaviour (acts which confer a benefit to the recipient at a cost to the actor) with the basic principle of natural selection: how can a gene be selected for when it decreases, rather than increases, the fitness of its host? One plausible account can be made through the theory of inclusive fitness. Key to this theory is the recognition that individual organisms within a social environment are not isolated from their conspecifics in terms of fitness. Whether a given gene is selected for is thus determined by its effect(s) on the fitness of any bearers of copies of that gene. Under this view, we can think of an altruistic act as an exchange of fitness from one agent to another. If the exchange is positive-sum and both sides are bearers of the gene in question, then from the gene's perspective the behaviour confers a fitness benefit–even while it decreases the fitness of the acting individual.

Kin selection theory 178 has proven to be a popular and widely accepted account of how altruistic behaviour can evolve under natural selection. Hamilton's rule, first published in 1964 118, 119, has since been experimentally validated across a range of different species and social behaviours. In contrast to this large body of work in natural populations, however, there has been relatively little study of kin selection in silico. In the current work, we offer what is to our knowledge the first demonstration of kin selection emerging naturally within a population of agents undergoing continuous neuroevolution. Specifically, we find that zero-sum transfer of resources from parents to their infant offspring evolves through kin selection in environments where it is hard for offspring to survive alone. In an additional experiment, we show that kin selection in our simulations relies on a combination of kin recognition and population viscosity. We believe that our work may contribute to the understanding of kin selection in minimal evolutionary systems, without explicit notions of genes and fitness maximisation.

This paper was accepted at The International Conference on the Applications of Evolutionary Computation (EvoAPPS) 2025 (part of EvoStar).

8.3.4 Evolving large populations of adaptive neural agents in ecologically plausible environments

Participants: Timothé Boulet [correspondant], Gautier Hamon, Clément Moulin-Frier.

This work continues the project presented in the previous paragraph, with a focus on the ability of agents to develop adaptability behaviors. Specifically, we extend the framework by adding fruits, a spatially variable ressource, and a memory of the values of each type of fruits for the agents. The goal is to observe whether the agents manage to exploit the knowledge of the fruits values to decide which fruit exploit.

Results : the agent were able to exploit the fruit value information to optimize their behavior. There were also some results that we were not necessarily expecting and that comes from our choice of model. Notably, it seems the agents choice for exploiting a cluster is heavily influenced by social criteria (the number of agents already exploiting it) and cultural criteria (whether the cluster is empty or full of fruits). This effect exceeds the adapatability effect in the latest stages of the simulations.

8.3.5 Cooperative control of environmental extremes by artificial intelligent agents

Participants: Martí Sànchez Fibla, Clément Moulin-Frier [correspondant], Ricard Solé.

This contribution is the result of a collaboration between Ricard Solé (University Pompeu Fabra (Barcelona, Spain) and Martí Sànchez-Fibla (CSIC, Barcelona, Spain) and Clément Moulin-Frier (Flowers, Inria). It has been published in the Journal of the Royal Society Interface in 2024 39.

Humans have been able to tackle biosphere complexities by acting as ecosystem engineers, profoundly changing the flows of matter, energy and information. This includes major innovations that allowed to reduce and control the impact of extreme events. Modelling the evolution of such adaptive dynamics can be challenging given the potentially large number of individual and environmental variables involved. This paper shows how to address this problem by using fire as the source of external, bursting and wide fluctuations. Fire propagates on a spatial landscape where a group of agents harvest and exploit trees while avoiding the damaging effects of fire spreading. The agents need to solve a conflict to reach a group-level optimal state: while tree harvesting reduces the propagation of fires, it also reduces the availability of resources provided by trees. It is shown that the system displays two major evolutionary innovations that end up in an ecological engineering strategy that favours high biomass along with the suppression of large fires. The implications for potential A.I. management of complex ecosystems are discussed.

The computational model is illustrated Fig. 25 and the main results in Fig. 26.

8.3.6 Emergence of agriculture in an artificial society of reinforcement learning agents

Participants: Gautier Hamon [correspondant], Clément Moulin-Frier, Marti Sanchez-Fibla, Ricard Solé.

This work is still a work in progress. It is the result of a collaboration with Ricard Solé (Complex System Lab, Universitat Pompeu Fabra, Barcelona, Spain) and Martí Sànchez-Fibla (CSIC, Barcelona, Spain), with a 3 months visit of Gautier Hamon in the Complex system lab.

This project aims at exploring the emergence of agriculture in simulations of artificial agents. In particular, we study how group of agents trained independently with reinforcement learning can learn to collectively engineer their environment to improve the growth of a specific beneficial plant that they can eat. We explore the environmental and agentic conditions that favor the emergence of agriculture in the simulations.

Our simulated environment consists in a 2D map with discrete locations (a lattice). We simulate the growth of different types of competing plants on this map, with only one being beneificial to the agent. We simulate plants lifecycle with the growth of the plant and the local spread of seeds. The environment also contains sources of water. The presence of water at the same location of a plant speed up its growing, i.e. water acts as a fertilizer.

This dynamic allows the agent to potentially eco-engineer their environment through the removal of unwanted plants, spreading seeds or watering the plants. We observe that in some environmental and agentic conditions, agents effectively learn to collectively improve the growth of the beneficial plant, allowing for an increase in resources to eat. Interestingly the agents learn a collective strategy to not overconsume to allow further spreading of the beneficial plant. In addition, we observe specialization, with some agents planting more seeds while other bring water or remove the unwanted plants.

Also, in a setup where agent can reproduce if they have enough resources, we show that the emergence of "agriculture" (eco-engineering the environment to have more abundant sustained resources), comes with a seemingly increase in population size. These new inviduals also adopt an "agriculture" behavior and allow to grow even more beneficial plant through their workforce, allowing again to grow the population.

These simulations therefore could help understand some of the dynamics underlying the emergence of agriculture. Further work need to be done to confirm and analyse more thoroughly the results that we obtained in these simulations. This work can also benefit the multi-agent RL community with the emergence of strong collective capabilities.

8.3.7 Open-ended recipe crafting through meta reinforcement learning

Participants: Gautier Hamon [correspondant], Eleni Nisioti, Clément Moulin-Frier.

As a first step towards studying the evolution of open-ended skill acquisition in artificial agents, we studied the environmental conditions favoring the systematic exploration of combinatorial recipes involving various objects. By combinatorial recipe, we mean the ability of agents to combine objects in the environment in order to create new ones (in the spirit of the Minecraft video game), some of these crafted objects being associated with a reward. In this work, the training of an agent uses meta reinforcement learning where an outer loop, equivalent to an evolutionary mechanism, meta-learns the parameters of an inner loop which can be seen as a developmental mechanism (where the agent acquire skills during its lifetime by interacting with the environment).

In the current setup we use $R{L}^2$ as our meta-learning algorithm 112, 188 which has already been used for acquiring behaviors efficiently balancing exploration and exploitation in a simple navigation task. Other work studied how different conditions in a bandit favor the evolution of innate vs learned behaviors 135.

Our experiments with recipe crafting are inspired by the little alchemy game. The difference with previous works in similar environments (e.g. 8.2.1) is that at every episode the structure of the recipe is randomly chosen. The agent therefore cannot predict what recipes will be rewarding and have to explore different combinations of objects in order to find the rewarding ones. The agent should also memorize the successful and unsuccessful combinations in order to explore and exploit efficiently.

Our preliminary results use both in a vectorized version of the game (where the agents action are only to choose the 2 objects to combine) and an embodied gridworld version (where the agent has to move, grab objects and put them on top of others in order to craft new ones). In both of these cases, the training efficiently meta learns an exploration/exploitation strategy which is to try new recipes (most of the time it does not try non working recipes more than once) until it finds the rewarding ones and then simply exploits them by making them over and over.

In preliminary results, by introducing drastic changes of recipe during an episode we observed the meta-learning of curious exploration just for the sake of information. Agents despite not being rewarded for it, explore the possible recipe as a way to get information that might be useful later in case of the change.

Further work will explore how increasing the variability (potentially through a curriculum) could lead to never ending exploration of the environment. In addition, adding other types of environmental dynamic to explore might lead to general curious behavior.

During the project, we switched from evolutionary algorithm and Recurrent neural networks to reinforcement learning and transformers. This allowed for more complex environments with more possibilities. We also obtained preliminary results with agents exploring the environment to gain information for the future.

This work uses the JAX python library for both the model/machine learning part and the environment simulation. JAX allows easy parallelization and fast GPU computation and so learning it through this project will be useful for later projects.

8.3.8 Evolving Reservoirs for Meta Reinforcement Learning

Participants: Corentin Leger [correspondant], Gautier Hamon, Eleni Nisioti, Xavier Hinaut, Clément Moulin-Frier.

This contribution was realized in the context of the Master internship of Corentin Léger in 2023, as a collaboration between C. Moulin-Frier from the Flowers team and Xavier Hinaut from the Mnemosyne team. It led to a paper which has been published in the International Conference on the Applications of Evolutionary Computation (Part of EvoStar) 48 in April 2024.

Animals demonstrate remarkable adaptability to their environments, a trait honed through the evolution of their morphological and neural structures 184162. Animals are born equipped with both hard-wired behavior routines (e.g. breathing, motor babbling) and learning capabilities to adapt from experience. The costs and benefits of evolving hard-wired behaviors vs. learning capabilities depends on different factors, a central one being the level of unpredictability of environmental conditions across generations 181130. Phenotypic traits addressing environmental challenges that are shared across many generations are more likely to evolve hard-wired (e.g. breathing), while traits whose utility can hardly be predicted from its utility in previous generations are likely to be learned through individual development (e.g. learning a specific language).

This prompts an intriguing question: How can neural structures, optimized at an evolutionary scale, enhance the capabilities of agents to learn complex tasks at a developmental scale? To address this question, we propose to model the interplay between evolution and development as two nested adaptive loops: evolution optimizes the generation of neural structures through natural selection over generations, shaping developmental learning during an agent’s lifetime (Fig. 27).

More precisely, at the evolutionary scale (the outer loop), we use an evolutionary algorithm to optimize a genome specifying Hyper Parameters of Reservoirs 173. At a developmental scale (the inner loop), a RL agent equipped with a generated reservoir learns an action policy to maximize cumulative reward in a simulated environment. Thus, the objective of the outer evolutionary loop is to optimize macro properties of reservoirs in order to facilitate the learning of an action policy in the inner developmental loop. See Fig.28 for an overview of the method.

Using this computational model, we run experiments in diverse simulated environments, e.g. 2D environments where the agent learns how to balance a pendulum and 3D environments where the agent learns how to control complex morphologies. These experiments provide support to three main hypotheses for how evolved reservoirs can affect intralife learning. First, they can facilitate solving partially-observable tasks, where the agent lacks access to all the information necessary to solve the task. In this case, we test the hypothesis that the recurrent nature of the reservoir will enable learning to infer the unobservable information. Second, it can generate oscillatory dynamics useful for solving locomotion tasks. In this case, the reservoir acts as a meta-learned CPG. Third, it can facilitate the generalization of learned behaviors to new tasks unknown during the evolution phase.

8.5.1 Fostering curiosity and meta-cognition in children using LLM-based conversational agents

Participants: Pierre-Yves Oudeyer, Hélène Sauzéon [correspondant], Rania Abdelghani, Didier Roy, Edith Law, Chloé Desvaux.

For empowering our conversational agents embedded into the curiosity-driven intervention we built, we leverage from LLMs, as verbal incentives either to curious behaviors (question asking and information search) or to metacognitive states triggering curious behaviors.

Matching conversational agents and LLM as lever for scaling up field applications. 

At the end of 2020, we started an industrial collaboration project with EvidenceB on this topic (CIFRE, contract of Rania Abdelghani validated by the ANRT). The overall objective of the thesis is to propose new educational technologies driven by epistemic curiosity, and allowing childre,n to express themselves more and learn better. To this end, a central question of the work will be to specify the impact of self-questioning aroused by states of curiosity about student performance. Another objective will be to create and study the pedagogical impact of new educational technologies in real situations (schools) promoting an active education of students based on their curiosity. To this end, a web platform called 'Kids Ask' has been designed, developed and tested in three primary schools. The tool offers an interaction with a conversational agent that trains children's abilities to generate curiosity-driven questions and use these questions to explore a learning environment and acquire new knowledge. The results suggest that the configuration helped enhance children's questioning and exploratory behaviors; they also show that learning progress differences in children can be explained by the differences in their curiosity-driven behaviors 81.

Despite showing pedagogical efficiency, the method used in the first study of this PhD is still very limited since it relies on generating curiosity-prompting cues by hand for each educational resource in order to feed the "discussion" with the agent, which can be a very long and costly process. For this reason, a logical follow-up to scale-up and generalize this study was to explore ways to automate the said conversational agents' behaviors in order to facilitate their implementation on a larger scale and for different learning tasks. More particularly, we move towards the natural language processing (NLP) field and the large language models (LLMs) that showed an impressive ability in generating text that resembles the way people write.

In this context, we study using the recent LLM GPT-3 to implement conversational agents that can prompt children's curiosity about a given text-based educational content, by proposing some specific cues. We investigate the validity of this automation method by comparing its impact on children's divergent question-asking skills with respect to the hand-crafted condition we had in our previous work.

In a second step, we explore using GPT-3 to propose a new curiosity-prompting behavior for our agent that aims to better support the children's needs of competence, autonomy and relatedness during the question-asking training. The study was conducted in two primary schools with 75 children aged between 9 and 1, and assigned to one of the three experimental conditions of agent’s design: 1. incentive hand-crafted agent ; 2. the incentive GPT-3-driven agent and 3. the open GPT-3 agent (see agent description hereafter).

Our first results suggest the validity of using GPT-3 to facilitate the implementation of curiosity-stimulating learning technologies. Indeed, children's performance was similar between the conditions where they had hand-generated or GPT-3-gene,rated cues. In a second step, we also found that GPT-3 can be efficient in proposing the relevant cues that leave children with more autonomy to express their curiosity 83 (publication in process).

​​​In evaluating the cues generated, we noted no offensive output from GPT3 and the three conditions were similar in terms of both the divergence level and the semantic relatedness to the text with what we had before by hand (as evaluated by human annotators) (as illustrated in the figure below).

Importantly, the results in terms of learner experience (elicited motivation and cognitive load, as shown below) as well as of curiosity behaviors have been conclusive by revealing equivalent results in children with GPT3-generated incentives, or even a superiority effect of the “open” condition of GPT-3.

8.5.2 LLMs usages and curiosity-related metacognitive skills in educational setting

Thanks to a collaboration with Pr. Celeste Kidd (Kidd Lab, Berkeley University of California, US), on another subject, we also started investigating the importance of curiosity-related metacognitive skills on students' use of the GenAI (Generative AI) tools during learning. Indeed, in  82, we argue about the importance of developing children's sense of critical thinking, epistemic vigilence, etc in order to allow a more active and informed use of these tools during learning. Such skills can help have a more realistic expectations of such tools and evaluate their outputs before integrating them in one's beliefs.

Identify-Guess-Seek-Assess (IGSA), a framework for modeling and training children’s curiosity-driven learning through metacognition. 

Leveraging the hypotheses about the roles of metacognition in organizing curiosity-driven learning, we recently introduced a training paradigm aiming to teach curiosity-driven learning via training of metacognitive skills  80. This training paradigm is based on considering the four basic metacognitive skills of Murayama’s framework  150 : identify (I) a knowledge gap, guess (G) possible answers, seek (S) information to fill the gap, and assess (A) the quality of the information (as shown below).

Design. 

In the training paradigm, the relevant metacognitive skills are personified as animated verbal metacognitive characters as follows: 1. The first character is the referrer who enacts the “identify” skill: when learning new content, it observes the task, reflects on its previous knowledge and chooses which uncertainty or missing information to pursue. 2. The second character is the detective, reflecting the “guess” skill: it formulates educated guesses and makes predictions about the missing information to pursue. 3. The third character is the explorer reflecting the “seek” skill: it pursues uncertainty by asking the relevant questions or by exploring the relevant resources. 4. The final character reflects the “assess” skill: it evaluates whether the inquiry resulted in learning progress with respect to the initial uncertainty.

The training paradigm is divided into two steps. • The first step aims to help children gain declarative knowledge about the said metacognitive skills. It consists of animated videos explaining key concepts related to curiosity and/or metacognition as well as the four metacognitive skills. To facilitate the explaining of these skills, the videos model them with 2D characters that correspond to their role in the curiosity-driven cycle: a first controller for IDENTIFY skill, a detective for the GUESS skill, an explorer for the SEEK skill and a second for the ASSESS skill (see figure below).

• The second step of training aims to help children gain procedural knowledge about the four Metacognitive skills they saw during the videos. For this we create a web-based platform we name “Kids meta-think”, where they are prompted to use these skills appropriately during a reading-comprehension task, using the help of conversational agents that have the same appearance and roles as the 2D characters presented during the videos. The interaction with the agents is designed to be like the following: once participants finish reading a text, the first referee appears (representing the IDENTIFY skill) and guides them into identifying and formulating a knowledge gap. After validating this step, the detective agent appears to guide them into formulating an educated hypothesis about it (GUESS skill). Then the explorer appears (SEEK skill) to help formulate the appropriate divergent question. And finally, once the divergent answer submitted, we display a set of 3 pieces of information that may or may not contain the answer to the question. The second referee thus appears (ASSESS skill) to lead children into reflecting on these pieces of information and decide whether or not they bring them closer to closing their knowledge gap. The agents had predefined scripted behaviors and did not give any feedback regarding children’s inputs. See following figure for a concrete example of the agents’ dialogue for a given text.

Technical implementation of “Kids meta-think”. 

The behaviors of the four agents in terms of selection of the appropriate prompts as well as proposing a list of options (if requested) are entirely predefined and hand-scripted. Indeed, the agents are connected to a database containing the text resources and, for each one, a list of corresponding prompts relating to the four metacognitive skills, i.e., the utterances for the conversational agents for each skill. These utterances consist of sentences to remind the definition of the skill, its importance and how to use it. Each text is also associated with a list of 3 propositions for each skill. All of these resources have been previously hand-generated by the research team and validated by two teachers as to their pedagogical relevance.

During the interaction, the agent’s automaton composes the dialogue utterances in order to include the appropriate cuing strategies (see figure below. We changed the utterances between the texts to avoid repetition in the different agents’ dialogue. This implementation has no natural language processing or generative artificial intelligence methods to manipulate the behaviors for the different agents. All data entered by participants was saved in a local database and was only evaluated post-experimentation, during the data

Intervention Effectiveness. 

As a follow-up direction to this line of work, a first study has been conducted with two classrooms to evaluate the accessibility of this new training and the impact on metacognitive efficiency, curiosity-driven question-asking and learning. The first results being rather positive, a larger sample size was recruited to validate them 80.

To go further in the support of effectiveness of the "learning to learn through curiosity", (PhD project Marie-Sarah Desvaux - started in October 2023 University of Bordeaux), we’re currently conducted a field study in a larger scale in collaboration with the Académie de Bordeaux and five elementary school teachers of Bordeaux Métropole, for a total of 100 students aged 9 to 11 years and according to the methodological rules of RCT in field setting; this approach is expected in the educational sciences. The objective is twofold : first the intervention was tested when led by teachers, as this reflects the classical school setting. Teachers underwent short training sessions to familiarize themselves with the format and content of the intervention, enabling them to autonomously implement it in their classrooms. Experiments were conducted during schooltime in the classrooms and lasted from February to June 2024.

Preliminary findings indicate good efficiency of the teacher-led training. Secondly, other than the efficiency of the intervention on students' performance, we needed to assess teachers’ willingness to adopt such an intervention. At the end of the experiments, interviews were conducted with teachers to evaluate their perception of the acceptability of the intervention and usefulness of the training. In a relevant way, we demonstrated that the workshop effect is robust in ecological setting of classrooms where the teachers managed themselves the intervention. Indeed, the workshop effects obtained with teachers are similar to those obtained by experimenters in terms of child’s curiosity perceptions, metacognitive sensibility and question-asking performances (figure below).

In related-way, we started in October 2024, experiments to test the relationships the creative thinking-based process and the curious thinking-based processe, with a second field study conducted in three schools within Bordeaux Métropole, involving a total of 130 children aged 9 to 11 years. These experiments aim to further assess the interplay between curiosity and creativity in question-asking behaviors. Drawing from creativity literature, we examine the process of question formulation through associative thinking involved in creativity. To do so, the conversational agent’s behavior in “Kids Ask” was modified to prompt children to identify important keywords from a text, then generate free associations based on their prior knowledge. Given the intricate interplay between curiosity and creativity, it is hypothesized that this associative guidance will further enhance children’s ability to formulate divergent, curiosity-driven questions.

Illustration of the conversational agent's strategy based on associative thinking in the question-asking workspace of the "Kids Ask" platform

8.5.3 Supporting Qualitative Analysis with Large Language Models: Combining Codebook with GPT-3 for Deductive Coding

Qualitative analysis of textual contents unpacks rich and valuable information by assigning labels to the data. However, this process is often labor-intensive, particularly when working with large datasets. While recent AI-based tools demonstrate utility, researchers may not have readily available AI resources and expertise, let alone be challenged by the limited generalizability of those task-specific models. In this study, we explored the use of large language models (LLMs) in supporting deductive coding, a major category of qualitative analysis where researchers use pre-determined codebooks to label the data into a fixed set of codes. Instead of training task-specific models, a pre-trained LLM could be used directly for various tasks without fine-tuning through prompt learning. Using a curiosity-driven questions coding task as a case study, we found, by combining GPT-3 with expert-drafted codebooks, our proposed approach achieved fair to substantial agreements with expert-coded results. We lay out challenges and opportunities in using LLMs to support qualitative coding and beyond. This work was published in 190 and involved a collaboration with Z. Xiao, V. Liao, E. Yuan from Microsoft Research Montreal.

8.6.1 New digital approaches for studying curiosity-driven learning

Participants: Hélène Sauzeon [correspondant], Pierre-Yves Oudeyer [correspondant], Rania Abdelghani, Mehdi Alaimi, Fabien Lotte, Aurélien appriou, Myra Fernandes, Edith Law, Yadurshana Sivashankar.

As curiosity is a recent research topic, we studied some basic mechanisms of curiosity-based learning, thanks to three studies have been completed.

The first one regards a new interactive educational application to foster curiosity-driven question-asking in children. Determined to improve children’s curiosity, we developed a new interactive system aiming to foster curiosity-related question-asking from texts and their perception of curiosity. To assess its efficiency, we conducted a study with 95 fifth grade students of Bordeaux elementary schools. Two types of interventions were designed, one trying to focus children on the construction of low-level question (i.e. convergent) and one focusing them on high-level questions (i.e. divergent) with the help of prompts or questions starters models. We observed that both interventions increased the number of divergent questions, the question fluency performance, while they did not significantly improve the curiosity perception despite high intrinsic motivation scores they have elicited in children. The curiosity-trait score positively impacted the divergent question score under divergent condition, but not under convergent condition. The overall results supported the efficiency and usefulness of digital applications for fostering children’s curiosity that we need to explore further. The overall results are published in CHI'20 85. In parallel to these first experimental works, we wrote this year a review of the existing works on the subject 97.

The second study investigates the neurophysiological underpinnings of curiosity and the opportunities of their use for Brain-computer interactions 86. Understanding the neurophysiological mechanisms underlying curiosity and therefore being able to identify the curiosity level of a person, would provide useful information for researchers and designers in numerous fields such as neuroscience, psychology, and computer science. A first step to uncovering the neural correlates of curiosity is to collect neurophysiological signals during states of curiosity, in order to develop signal processing and machine learning (ML) tools to recognize the curious states from the non-curious ones. Thus, we ran an experiment in which we used electroencephalography (EEG) to measure the brain activity of participants as they were induced into states of curiosity, using trivia question and answer chains. We used two ML algorithms, i.e. Filter Bank Common Spatial Pattern (FBCSP) coupled with a Linear Discriminant Algorithm (LDA), as well as a Filter Bank Tangent Space Classifier (FBTSC), to classify the curious EEG signals from the non-curious ones. Global results indicate that both algorithms obtained better performances in the 3-to-5s time windows, suggesting an optimal time window length of 4 seconds to go towards curiosity states estimation based on EEG signals. These results have been published 86.

Thanks to Virtual reality device, a third study investigates the role of intrinsic motivation in spatial learning in children 177. In this study, the state curiosity is manipulated as a preference for a level of uncertainty during the exploration of new virtual environments. To this end, a series of virtual environments have been created and is presented to children. During encoding, participants explore routes in environments according the three levels of uncertainty (low, medium, and high), thanks to a virtual reality headset and controllers and, are later asked to retrace their travelled routes. The exploration area and the wayfinding. ie the route overlap between encoding and retrieval phase, (an indicator of spatial memory accuracy) are measured. Neuropsychological tests are also performed. The results showed that there are better performances under the medium uncertainty condition in terms of exploration area and wayfinding score. These first results supports the idea that curiosity states are a learning booster. In Sivashankar et al. study, 10-year-old children (20 females; 22 males) with low to high trait curiosity actively explored virtual environments 39 containing varying levels of uncertainty (low, medium, high) (Fig. 40), after which memory for the route travelled was assessed 177.

From left to right: Condition 1 with Low Uncertainty (1 character); Condition 2 with Medium Uncertainty (3 characters); and Condition 3 with High uncertainty (7 characters)

As trait curiosity increased (41), so did memory performance in the high uncertainty condition, suggesting that children with high levels of curiosity can better recruit cognitive resources within such environments. Children with high compared to low curiosity also had higher feelings of presence during the immersive experience. Importantly, in environments with medium uncertainty, children with low trait curiosity were able to perform as well as those with high curiosity. Results suggest that individual differences in trait curiosity influences route memory in environments with varying levels of uncertainty.

8.6.2 Machine Learning for Adaptive Personalization in Intelligent Tutoring Systems

Participants: Pierre-Yves Oudeyer [correspondant], Hélène Sauzeon [correspondant], Benjamin Clément, Didier Roy, Cécile Mazon.

8.6.3 Machine learning for adaptive cognitive training

Participants: Pierre-Yves Oudeyer, Hélène Sauzéon [correspondant], Masataka Sawayama, Benjamin Clément, Maxime Adolphe, Marion Pech, Olivier Clerc.

Because of its cross-cutting nature to all cognitive activities such as learning tasks, attention is a hallmark of good cognitive health throughout life and more particularly in the current context of societal crisis of attention. Recent works have shown the great potential of computerized attention training for an example of attention training, with efficient training transfers to other cognitive activities, and this, over a wide spectrum of individuals (children, elderly, individuals with cognitive pathology such as Attention Deficit and Hyperactivity Disorders). Despite this promising result, a major hurdle is challenging: the high inter-individual variability in responding to such interventions. Some individuals are good responders (significant improvement) to the intervention, others respond variably, and finally some respond poorly, not at all, or occasionally. A central limitation of computerized attention training systems is that the training sequences operate in a linear, non-personalized manner: difficulty increases in the same way and along the same dimensions for all subjects. However, different subjects require in principle a progression at a different, personalized pace according to the different dimensions that characterize attentional training exercises.

To tackle the issue of inter-individual variability, the present project proposes to apply some principles from intelligent tutoring systems (ITS) to the field of attention training. In this context, we have already developed automatic curriculum learning algorithms such as those developed in the KidLearn project, which allow to customize the learner's path according to his/her progress and thus optimize his/her learning trajectory while stimulating his/her motivation by the progress made. ITS are widely identified in intervention research as a successful way to address the challenge of personalization, but no studies to date have actually been conducted for attention training. Thus, whether ITS, and in particular personalization algorithms, can optimize the number of respondents to an attention training program remains an open question.

Grounded state-of-the-art. 

To investigate this question, we first conducted a systematic review aiming at exploring existing methods in computerized CT and analyzing their outcomes in terms of learning mechanics (intra-training performance) and effectiveness (near, far and everyday life transfer effects of CT) 84. A search up to June 2023 with multiple databases selecting 19 computerized CT studies revealed that only two studies emphasized the favorable influence of individualization on CT effectiveness, while five underscored its capacity to enhance the training experience by boosting motivation, engagement, and offering diverse learning pathways. In sum, despite promising results in this new research avenue, more research is needed to fully understand and empirically support individualized techniques in cognitive training.

Complementing the study of adaptive methods applied to cognitive training, we have attempted through a review of the subjective literature to gain a better understanding of the Multiple Object Tracking (MOT) task, which seems to have the best results in terms of attentional training efficiency in young and older adults. Our investigation pursues three main objectives: (1) identifying the cognitive processes influenced by each adjustable parameter of the MOT task; (2) determining which parameters, when progressively adapted during repeated MOT practice, produce the greatest enhancements in task performance; and (3) evaluating how improvements in MOT performance translate into effective transfer effects, including practical, real-world outcomes. The evidence suggests that the MOT task involves a nuanced interplay of visual processing, attentional resources, and working memory, shaped by the intrinsic properties of the objects and the task conditions. The results of this work highlight that: (1) Multiple cognitive mechanisms are identified as active in the task (divided and sustained attention; foveal and peripheric attention ; automatic and controlled inhibition, etc. ); (2) a limited number of studies have actually implemented the MOT task in computer-assisted cognitive training; and (3) tIt's the near (attention tasks) and far (other cognitive tasks) effects that are well documented as positive outcomes of MOT-based training while there is a scarcity of research that has thoroughly analyzed the ecological effects of attentional training, namely the potential transfer effects in everyday life (paper in progress).

Randomized and controlled Trial in Young and Olders adults : Predifined vs. ZPDES condition. 

Utilizing these tools, a pilot study campaign was conducted to evaluate the impact of our AI-based personalized cognitive training program. The first pilot experiment involved n=27 participants and aimed to compare the effectiveness of a cognitive training program using a linear difficulty management procedure (staircase procedure) to a program using an ITS for difficulty manipulation. The online training lasted for 10 hours over a period of 2 weeks. The results indicated that the ITS-based intervention produced diverse learning trajectories compared to the linear procedure 43, leading to broader improvements in pre-post cognitive assessment. However, no significant differences were observed in subjective measures of motivation and engagement between the two groups. Subsequent to this initial experiment, two pilot studies (n=11 and n=10, respectively) were conducted with the goal of enhancing motivation and engagement in the game. The first study implemented gamified components such as scores and feedback, while the second study examined hyperparameter updates to the ITS. The analysis of learning trajectories, learning outcomes, and subjective measures yielded promising results in favor of the AI-based personalized procedure.

Building on the preliminary findings, we expanded our research scope with a more comprehensive experimental setup involving two distinct studies. The first study encompassed 64 young adults, sourced through the Prolific platform, while the second study consisted of 50 older adults, recruited from the "Université du temps libre". Our experimental methodology mirrored that of our initial pilot studies, with a notable enhancement: the integration of new gamified elements (including mini-story creation and new visual content) aimed at boosting participant motivation and engagement.

The data analysis encompassed three primary dimensions: initially, an exploratory phase to delineate learning trajectories between control and intervention groups; subsequently, a comparative analysis of pre- and post-test performance on the cognitive battery; and lastly, an examination of participants' self-reported experiences during training, providing insights into their subjective perceptions of the experiment.

The pilot studies' preliminary outcomes were corroborated in these larger sample groups. Notably, learning trajectories exhibited greater diversity in the group undergoing the intervention procedure. This group also demonstrated a more pronounced improvement across a wider range of cognitive assessment tasks. Although participants engaging in the personalized cognitive training reported a higher cognitive load via questionnaires, the levels of engagement and frustration did not significantly differ between the two groups.

The results showed that ZPDES could be more effective than a control condition, with improved performance on trained tasks in both studies, underlining the benefits of individualized training paths. However, motivation and engagement were lower in the groups using ZPDES, probably due to cognitive load and metacognitive factors. Overall, individualizing cognitive training through systems like ZPDES provides a promising direction for future research by providing automatic methods for taking individual differences into account in CT programs while respecting methodological standards for evaluating the effectiveness of CT. As a result, our work contributes to the growing body of knowledge in both ITS and CT domains while stressing the crucial role of challenges related to motivation and engagement to optimize the effectiveness of these individualized approaches for cognitive and educational outcomes.

As part of the creation of the new University Hospital Institute (UHI) VBHI (VASCULAR BRAIN HEALTH INSTITUTE), we aim to develop and test a personalized, multimodal digital therapeutic approach to slow down the functional consequences of small vessel disease. More specifically:

• Evaluate the impact of personalized cognitive training compared to non-personalized conditions (comparative efficacy).
• Identify potential ElectroEncephaloGraphic (EEG) biomarkers that reflect cognitive activity impacted by small vessel disease and could later (in a subsequent study) be used as targets for exploratory EEG neurofeedback therapy.
• Identify brain areas to target for delivering non-invasive HD-tACS electrical stimulation, using previously acquired MRI data.
• Evaluate the impact of this stimulation on brain activity, neural synchronization, and cognitive performance.

To achieve this, 60 participants from the SHIVA cohort (n=60) will be divided into two subgroups according to the severity of the disease:

• Severe group: presenting multiple lesions on MRI
• Non-severe group: presenting a few lesions

These groups will then be further divided based on the type of training: personalized tests (ZPDES) versus standard tests.

justification=centering,margin=1cm

During the pre- and post-training sessions, participants will perform cognitive tests on a computer. Participants will be equipped with an EEG headset, which, combined with a tACS stimulator, will allow for both brain activity recording and stimulation.

We are carrying out an ancillary study with Myra Fernandez's laboratory in Canada, thanks to my participation with the Inria Curiositytech international associate team. We have proposed to collaboratively analyze certain data and dimensions of interest in our respective laboratories (e.g. physical activities) associated with the cognitive training proposed in the SHIVA-DTX-COG project.

8.6.4 ToGather : Interactive website to foster collaboration among stakeholders of school inclusion for pupils with neurodevelopmental disorders

Participants: Hélène Sauzéon [correspondant], Cécile Mazon, Eric Meyer, Isabeau Saint-Supery, Christelle Maillart [Uni. Liège, Belgium], Kamélia Belassel, Mathieu Périé, Valentin Strahm.

Sustain and support the follow-up of the school inclusion of children with neurodevelopmental disorders (e.g., autism, attention disorders, intellectual deficiencies) has become an emergency : the higher is the school level, the lower is the amount of schooled pupils with cognitive disabilities.

Technology-based interventions to improve school inclusion of children with neurodevelopmental disorders have mostly been individual centered, focusing on their socio-adaptive, and cognitive impairments and implying they have to adapt themselves in order to fit in our society's expectations. Although this approach centered on the normalization of the person has some advantages (reduction of clinical, symptoms), it carries social stereotypes and misconceptions of cognitive disability that are not respectful of the cognitive diversity and intrinsic motivations of the person, and in particular of the student's wishes in terms of school curriculum to achieve his or her future life project 146.

The "ToGather" project aims at enlightening the field of educational technologies for special education by proposing an approach centered on the educational needs of the students and bringing a concerted and informed answer between all the stakeholders including the student and all their support spheres (family, school, medico-social care). To this end, ToGather project that emanates from participatory design methods, primarily consists of having developed a pragmatic tool (interactive website) to help students with cognitive disability and their caregivers to formalize and to visualize the repertoire of academic skills of the student and to make it evolve according to his or her proximal zone of development (in the sense of Vygotsky) on the one hand, and to the intrinsic motivations of the student (his or her own educational and life project) on the other 143.

This project is in partnership with the School Academy of Bordeaux of the French Education Minestery, the ARI association, the Centre of Autism of Aquitaine. It is funded by the FIRAH (foundation) and the Nouvelle-Aquitaine Region (see the dedicated webpages).

First, usability studies have been conducted for evaluating ergonomic qualities of the ToGather website, yielding positive resultats in French and Belgian contexts. Then, we conducted a large field-study to assess the effectiveness of the tool in helping stakeholders to support children with neurodevelopmental disorders (NDD)  169  167  168.

The study protocol consisted in a longitudinal non-randomized controlled trial, with baseline, 3-months, and 6-months fllow-up assessments. The recruitment was conducted across the entire French territory. Our local partners facilitated the dissemination of the call for participation in Gironde and provided us with contacts to extend it to other regions. Additionally, a recruitment campaign through social media was carried out to communicate about the study and encourage participants to test the ToGather tool.

As the tool was designed to support co-educational process between parents and professionals, a support team had to consist of at least two stakeholders, including at least one of the parents. Initially, 157 participants were recruited in 37 support teams, but 30 individuals did not answer to baseline questionnaire, leading to the exclusion of 11 support teams. After baseline assessment, 13 support teams were allocated to the experimental condition (ToGather app) and 11 to the control condition (usual follow-up).

Primary outcomes measures covered stakeholders’ relationships, self-efficacy, and attitudes towards inclusive education, while secondary outcomes measures were related to stakeholders’ burden and quality of life, as well as children’s school well-being and quality of life.

As the study ended recently, data analysis is still ongoing. Preliminary results after 3 months of use showed encouraging results with an improvement in communication between stakeholders and their respective quality of life (paper in progress)

8.6.5 Curious and therefore not overloaded : Study of the links between curiosity and cognitive load in learning mediated by immersive technologies

Participants: Hélène Sauzéon [correspondant], Matisse Poupard, André Tricot [Cosupervisor - Univ. Montpellier], Florian Larrue [Industrialist - Le Catie].

In collaboration with CATIE (industrial partner) and the EPSYLON laboratory at the University of Montpellier (led by Prof. André Tricot), this research program, initiated in April 2022, aims to achieve two main objectives:

• To establish theoretical connections between cognitive load theory and models of curiosity-driven learning.
• To experimentally evaluate how the choice of educational technology influences the relationship between pedagogical approaches (guided instruction vs. exploration) and learner expertise.

To this end, the program is divided into three main phases of study:

Experimentation. 

Two experiments were conducted in 2023 with 131 second-year medical students, and replicated in 2024 with 164 medical students ranging from their second to fifth years.

In the first experiment, we used spatial augmented reality (SAR) and mixed reality (MR, HoloLens 2) to investigate whether support students' drawings during lectures could reduce cognitive load and enhance their motivation to learn. Participants followed a 20-minute neuroanatomy video lecture and simultaneously reproduced drawings with the instructor, following four experimental conditions :

• Spatial Augmented Reality (SAR) Condition: A digital overlay of the structure to be drawn was projected, allowing the learner to trace the structure onto their paper using a projector and a tracking system.
• Mixed Reality (MR) Condition: The digital overlay was projected using a HoloLens2 headset.
• Mixed Reality with 3D Model (MR+3D) Condition: In addition to the digital trace, this condition allowed for the display and manipulation of a 3D model of the anatomical structure.
• Control Condition: This condition served as the reference and did not include a digital overlay.

Preliminary results from the 2023 data showed that AR-supported drawing led to higher intrinsic motivation, lower extraneous cognitive load, and improved drawing accuracy, particularly in the SAR and MR+3D conditions. However, no significant differences in learning outcomes were observed between conditions. These findings were presented at the 4th International Conference on Science and Technology Education in Porto in October 2024 75. The role of students' prior knowledge in anatomy on the effectiveness of different systems was also discussed, and the 2024 data is expected to provide further insights into how expertise influences AR-assisted learning.

In the second experiment, we shifted the learning paradigm by using virtual reality (VR) with varying levels of interaction and guidance. This allowed us to examine how exploration and embodied interaction with a 3D model impact learning, cognitive load, and curiosity.

The analysis of 2023 data, presented in a paper currently under review in *Computers & Education* 69, revealed that VR conditions yielded better learning performance, especially in the passive and active conditions. These were associated with higher intrinsic motivation and optimized cognitive load. Furthermore, intrinsic motivation was positively correlated with germane cognitive load (relevant for learning) and negatively correlated with extraneous load (irrelevant for learning). In essence, highly motivated students found the task less cognitively demanding in terms of irrelevant information, enabling them to allocate more cognitive resources to meaningful learning. These findings on the interplay between cognitive load and intrinsic motivation were discussed in a presentation at the International Cognitive Load Theory Conference in Sydney in November 2024 58.

The replication of these two experiments in 2024 aims to deepen our understanding of the relationships between cognitive load and curiosity in learning mediated by immersive technologies, using structural equation modeling."

8.6.6 Self-determination-driven digital services for supporting aging-in place and well-being: a study of relationships between longitudinal data from smart home and clinical data

Participants: Hélène Sauzéon, Juliette Deyts, Lucile Dupuy, Rafik Belloum.

This work is based on data already collected from elderly people with frailty syndrome, who used an ambient assistant-living (HomeAssist) for 12 months, designed to support self-determination and cover 3 domains of need (daily activities, home safety and social participation).

The aim is to analyze various clinical (cognitive assessments, frailty, autonomy, SDT, etc.) and use (user experience questionnaires, diaries, actimetrics based on environmental sensors) data, in order :

• to relate the benefits regarding care with the use of assistive and monitoring services provided by HomeAssist and
• to explore the predictive value of data provided by HomeAssist-based sensors for explaining clinical data

These activities aim to design a user-centered tool that will enable clinicians (psychologists or physicians) to visualize and understand these links effectively. This tool will meet the needs of clinicians who are currently seeking for tools capable of displaying the evolution of older adults over a long period, in order to detect “weak signals” indicating the first signs of decline. This intuitive visualization will facilitate early intervention, as clinicians will be able to identify trends over time, such as reduced sleep duration, which is a key predictor of health in the elderly.

8.7.1 Design of an Interactive Software for Automated Discovery in Complex Systems

Participants: Clément Romac [correspondant], Zacharie Bugaud, Clément Moulin-Frier, Pierre-Yves Oudeyer.

We further developed our Automated Discovery software and particularly focused on adding and experimenting with new systems.

Our public software now features more than ten examples ranging from artificial life, to physics or protein docking. The software was publicly released in 2024: presentation thread.

8.7.2 Exploration of Gene Regulatory Network Behaviors Using Automated Discovery Tools

Participants: Mayalen Etcheverry [correspondent], Clément Moulin-Frier, Pierre-Yves Oudeyer, Michael Levin.

In the context of project "Automated Discovery in Self-Organizing Systems", it has been demonstrated that modern tools leveraging computational models of curiosity developed in the Flowers team can be transposed to form efficient AI-driven "discovery assistants." These tools can assist scientists in mapping and navigating the space of possible outcomes in complex systems 29, 16, 120. In 2022, we initiated a collaboration with Dr. Michael Levin, a renowned biologist at Tufts University, through a 5-month academic exchange with Mayalen Etcheverry in his lab in Boston. This collaboration laid the foundation for continued collaboration throughout 2023, resulting in the submission of one paper 33 (currently under review) and another accepted at the NeurIPS 2023 AI for Science workshop 115.

The primary focus of this collaboration was to leverage curiosity-driven exploration algorithms as tools to empower scientific exploration and analysis of basal cognition in biological systems, specifically numerical models of gene regulatory networks (GRNs). Understanding, mapping, predicting, and controlling the complex behavior of these networks is crucial for applications in biomedicine and synthetic bioengineering. However, there are few quantitative tools that facilitate exploration of these networks, especially when their complexity makes unguided exploration infeasible.

To address these challenges in practice, we proposed an experimental framework summarized in Figure 50. In this framework, we formalized and investigated a view of gene regulatory networks as agents navigating a problem space. We developed automated tools to efficiently map the repertoire of robust goal states that GRNs can reach despite perturbations. These tools rely on two main contributions that we made in this work: (1) The use of curiosity-driven exploration algorithms, originating from the AI community, to explore the range of behavioral abilities of a given system, which we adapted and leveraged to automatically discover the range of reachable goal states of GRNs, and (2) The use of a battery of empirical tests inspired by implementation-agnostic behaviorist approaches that we leveraged to assess the navigation competencies of GRNs.

Our data revealed that models inferred from real biological data can reach a surprisingly wide spectrum of steady states, showcasing various competencies that living agents often exhibit in physiological network dynamics and that do not require structural changes to network properties or connectivity. Furthermore, we investigated the applicability of the discovered “behavioral catalogs” for comparing the evolved competencies across classes of evolved biological networks, as well as for the design of drug interventions in biomedical contexts or for the design of synthetic gene networks in bioengineering. Altogether, these automated tools and the resulting emphasis on behavior-shaping and exploitation of innate competencies can open the path to better interrogation platforms for exploring the complex behavior of biological networks in an efficient and cost-effective manner.

To encourage broader adoption and development of the tools and algorithms, we have released two software packages: SBMLtoODEJax7.1.17 and AutoDiscJax. SBMLtoODEJax converts Systems Biology Markup Language (SBML) models into Python classes written in JAX, enabling easy simulation and manipulation. AutoDiscJax, built upon JAX and SBMLtoODEJax, facilitates automated discovery and exploration of complex systems, specifically organizing the exploration of computational models of biological GRNs.

This work has been published in 2024 in the eLife journal 33.

8.7.3 Discovering Sensorimotor Agency in Cellular Automata using Diversity Search

Participants: Gautier Hamon [correspondant], Mayalen Etcheverry, Bert Chan, Clément Moulin-Frier, Pierre-Yves Oudeyer.

As a continuation of the previous projects in Automated Discovery in Self-Organizing Systems, we have been working on expanding the set of discoveries of possible structures in continuous CAs such as Lenia 99, 98, and in particular we have been interested to search for emerging agents with sensorimotor capabilities. Understanding what has led to the emergence of life and sensorimotor agency as we observe in living organisms is a fundamental question. In our work, we initially only assume environments made of low-level elements of matter (called atoms, molecules or cells) locally interacting via physics-like rules. There is no predefined notion of agent embodiment and yet we aim to answer the following scientific question: is it possible to find environments in which there exists/emerge a subpart that could be called a sensorimotor agent?

We use Lenia continuous cellular automaton as our artificial "world" 98. We introduce a novel method based on gradient descent and curriculum learning combined within an intrinsically-motivated goal exploration process (IMGEP) to automatically search parameters of the CA rule that can self-organize spatially localized 1 and moving patterns 2 within Lenia. The IMGEP defines an outer exploratory loop (generation of training goal/loss) and an inner optimization loop (goal-conditioned). We use a population-based version of IMGEP 17,106 but introduce two novel elements compared to previous papers in the IMGEP literature. First, whereas previous work in 29 and 16 used a very basic nearest-neighbor goal-achievement strategy, our work relies on gradient descent for the local optimization of the (sensitive) parameters of the complex system, which has shown to be very powerful. To do so we made a differentiable version of the Lenia framework, which is also a contribution of this work. Secondly, we propose to control subparts of the environmental dynamics with functional constraints (through predefined channels and kernels in Lenia) to build a curriculum of tasks; and to integrate this stochasticity in the inner optimization loop. This has shown central to train the system to emerge sensorimotor agents that are robust to stochastic perturbations in the environment. In particular, we focus on modeling obstacles in the environment physics and propose to probe the agent sensorimotor capability as its performance to move forward under a variety of obstacle configurations. We also provide in this work tests and metrics to measure the robustness of the obtained agents.

While many complex behaviors have already been observed in Lenia, among which some could qualify as sensorimotor behaviors, they have so far been discovered "by chance" as the result of time-consuming manual search or with simple evolutionary algorithms. Our method provides a more systematic way to automatically learn the CA rules leading to the emergence of basic sensorimotor structures, as shown in Figure 53. Moreover, we investigated and provided ways to measure the (zero-shot) generalization of the discovered sensorimotor agents to several out-of-distribution perturbations that were not encountered during training. Impressively, even though the agents still fail to preserve their integrity in certain configurations, they show very strong robustness to most of the tested variations. The agents are able to navigate in unseen and harder environmental configurations while self-maintaining their individuality (Figure 51). Not only the agents are able to recover their individuality when subjected to external perturbations but also when subjected to internal perturbations: they resist variations of the morphogenetic processes such that less frequent cell updates, quite drastic changes of scales as well as changes of initialization (Figure 52). Furthermore, when tested in a multi-entity initialization and despite hav,ing been trained alone, not only the agents are able to preserve their individuality but they show forms of coordinated interactions (attractiveness and reproduction). Our results sug,gest that, contrary to the (still predominant) mechanistic view on embodiment, biologically-inspired embodiment could pave the way toward agents with strong coherence and generalization to out-of-distribution changes, mimicking the remarkable robustness of living systems to maintain specific functions despite environmental and body perturbations 131. Searching for rules at the cell-level in order to give rise to higher-level cognitive processes at the level of the organism and at the level of the group of organisms opens many exciting opportunities to the development of embodied approaches in AI in general.

The work has been released in 2022 as a distill-like article which is currently hosted at this link. This article contains an interactive demo in webGL and javascript, as well as many videos and animations of the results. A colab notebook with the source code of the work is publicly available at.

In 2024, additional quantitative experiments were conducted as well as ablations. This work has been submitted to the Science Advances journal.

8.7.4 Flow lenia: Mass conservation for the study of virtual creatures in continuous cellular automata

Participants: Erwan Plantec, Gautier Hamon [correspondant], Mayalen Etcheverry, Pierre-Yves Oudeyer, Clément Moulin-Frier, Bert Chan.

Following our work on trying to find sensorimotor capabilities in cellular automata such as Lenia 99, 98, we kept exploring the search for low level cognition in continuous cellular automata. This led to preliminary search on trying to emerge memory in self-organizing agents as well as work on trying to implement other environmental constraints in the CA in order to emerge interesting behavior. To implement more easily those environmental constraints as well as to ease the emergence of spatially localized patterns (and thus have the optimization/search to focus more on the cognitive ability, removing the need to optimize to prevent uncontrollable growth/explosion of the pattern), we worked on adding mass conservation to the Lenia system.

We propose in this work a mass-conservative (i.e the sum of the CA’s activations remains constant over time) extension to Lenia called Flow Lenia 164. We hypothesize that such conservation laws will help in the search for artificial life-forms by constraining emerging patterns to spatially localized ones. It ,also allows to implement more easily environmental constraints on the self-organizing agents such as a need for food to grow, etc.

Furthermore, we show that this new model allows for the integration of the update rule parameters within the CA dynamics enabling the emergence of creatures with different parameters and so different properties in the same environment/grid. This leads to multi-species simulation where the grid is filled with agents with different behaviors and properties 54. Such a feature opens up research perspectives towards the achievement of open-ended intrinsic evolution inside continuous CAs, which means that all the evolutionary part would be a result of the dynamic of the CA (without any external loop/system). We hypothesize that this open-ended instrinsic evolution could, through the competition/cooperation, lead to the emergence of interesting low level cognition in those system.

Simple evolutionary strategy (with an evolutionary loop outside the system) was also used to optimized for pattern with directional and rotational movement.

You can find some examples of the system and pattern in this companion website, including the ones trained for movement, random parameters, food in flow Lenia, and multi species s,imulations: see. Notebook with the system can be found : here.

This work led to an oral presentation to the WIVACE 2022, 15th International Workshop on Artificial Life and Evolutionary Computation.

In 2023, final quantitative experiments on optimizing the parameters with evolutionary strategies and writing was conducted, as well as some additional exploratory experiments on large simulations for open ended evolution.

This work got the best paper award at the ALIFE 2023 conference where it got an oral presentation.

In 2024, an extended version of the paper was accepted in the Alife Journal (submitted upon their demand), with additional experiments focusing on the ermegent evolutionary dynamics of the simulation.

8.7.5 Semantic Open-Endedness in Flow-Lenia using Vision Language Models and IMGEP

Participants: Sina Khajehabdollahi [correspondent], Gautier Hamon, Cédric Colas, Pierre-Yves Oudeyer, Clément Moulin-Frier.

The growing capabilities of large language models (and their extension into vision language models) has opened the possibility to integrate these foundation models within the framework of automated discovery. Using Flow-Lenia as the substrate for a complex and emergent system, we integrate vision language models into an IMGEP exploration algorithm in order to discover behavioural patterns in goal spaces that are semantically meaningful and interpretable. Previous work on exploration without these foundation models required expert knowledge and human intervention in order to hand-craft the goal spaces and/or targets for exploration and optimization. In cases where goal spaces were not hand-crafted, the semantic interpretation of automatically clustered behaviours remained difficult to translate to human perception. By utilizing vision language models to label patterns generated by FlowLenia, the output behaviour of the model can be summarized using keywords or sentences which are then readily interpretable by humans. Setting behavioural goals can then also be done at a high level with language, where we use these higher-level abstractions to push for more biologically rich behaviours that would otherwise be difficult or impossible to quantify in a generalized way. In this framework, we can search for open-endedness in these models of complexity in a much broader and human-relevant way, leveraging human expectations of novelty and higher-level descriptors through the eyes of these emerging foundation models.

Research on lifelong Deep Reinforcement Learning of multiple tasks (Microsoft

Participants: Pierre-Yves Oudeyer [correspondant], Laetitia Teodorescu.

Financing of the PhD grant of Laetitia Teodorescu.

Automated Discovery of Self-Organized Structures (Poïetis)

Participants: Pierre-Yves Oudeyer [correspondant], Mayalen Etcheverry.

Financing of the CIFRE PhD grant of Mayalen Etcheverry by Poietis.

Machine learning for adaptive cognitive training (OnePoint)

Participants: Hélène Sauzéon [correspondant], Pierre-Yves Oudeyer, Maxime Adolphe.

Financing of the CIFRE PhD grant of Maxime Adolphe by Onepoint.

Curiosity-driven interaction system for learning (evidenceB)

Participants: Hélène Sauzéon [correspondant], Pierre-Yves Oudeyer, Rania Abdelghani.

Financing of the CIFRE PhD grant of Rania Abdelghani by EvidenceB.

Curious and therefore not overloaded : Study of the links between curiosity and cognitive load in learning mediated by immersive technologies (CATIE)

Participants: Hélène Sauzéon [correspondant], Matisse Poupard, André Tricot [Cosupervisor - Univ. Montpellier], Florian Larrue [Industrialist - Le Catie].

Financing of a PhD grant of Matisse Poupard with CATIE and EPSYLON Lab (Univ. Montpellier).

Augmenting curiosity-driven exploration with very large language models in deep reinforcement learning agents (Hugging Face)

Participants: Pierre-Yves Oudeyer [correspondant], Clément Romac.

Financing of the PhD grant of Clément Romac by Hugging Face.

Autonomous Driving Commuter Car (Renault)

Participants: David Filliat [correspondant], Emmanuel Battesti.

We developed planning algorithms for a autonomous electric car for Renault SAS in the continuation of the previous ADCC project. We improved our planning algorithm in order to go toward navigation on open roads, in particular with the ability to reach higher speed than previously possible, deal with more road intersection case (roundabouts), and with multiple lane roads (overtake, insertion...).

We received a 30keuros grant from Google Brain, as well as 30keuros Google cloud credit, for developing projects on automated exploration of continuous cellular automata.

School+ /ToGather project (FIRAH and Region Nouvelle-Aquitaine)

Participants: Hélène Sauzéon [correspondant], Cécile Mazon, Isabeau Saint-supery, Eric Meyer.

Financing of one year-postdoctoral position and the app. development by the International Foundation for Applied Research on Disability (FIRAH). The School+ project consists of a set of educational technologies to promote inclusion for children with Autism Spectrum Disorder (ASD). School+ primary aims at encouraging the acquisition of socio-adaptive behaviours at school while promoting self-determination (intrinsic motivation), and has been created according to the methods of the User-Centered Design (UCD). Requested by the stakeholders (child, parent, teachers, and clinicians) of school inclusion, Flowers team works to the adding of an interactive tool for a collaborative and shared monitoring of school inclusion of each child with ASD. This new app will be assessed in terms of user experience (usability and elicited intrinsic motivation), self-efficacy of each stakeholder and educational benefit for child. This project includes the Academie de Bordeaux –Nouvelle Aquitaine, the CRA (Health Center for ASD in Aquitania), and the ARI association.

CLEMENCE Cohort (Fondation de France and Théa Pharma)

Participants: Hélène Sauzéon [correspondant], Cécile Mazon, Cécile Delcourt.

The project "Cohorte LongitudinalE sur la Myopie et le développement oculaire dans l’ENfanCE(CLEMENCE) is led by C. Delcourt from the lab of Bordeaux Populational Health (2M€). Hélène Sauzéon and Cécile Mazon participate to the research program with the study of developemental changes due to Myopa in visual attention.

10.1.1 Associate Teams in the framework of an Inria International Lab or in the framework of an Inria International Program

10.2.1 Visits of international scientists

Gaia Molinaro, PhD student at University of Berkeley, visited the team for 6 months (Feb-July), working on several projects including: developing the concept of latent learning progress to account for human curiosity-driven exploration, and testing it in a novel experimental paradigm with humans (this led to a publications at Neurips 2024, 50), collaborating on a project about autotelic generative models (publications at Neurips 2024 38), and on a project modeling the dynamics of iterated transmission of stories in chains of LLMs (publications at ICLR 2025, 67). This visit was partially funded by a grant from the Chateaubriand fellowship program. We also welcomed Anne Collins, associate professor at University of Berkeley.

Max Taylor-Davis, PhD student at the University of Edimburg, made a 3-month visit during the summer 2024. This led to the publication of a paper accepted at the EVOSTAR 2025 conference (see New Results).

We also welcomed several researchers for short visits and invited seminars: Yves Barral (ETH Zurich, feb. 2024), Edith Law (Univ. Waterloo, Canada, feb. 2024), Mathieu Lefort (Univ. Lyon, May 2024), Louise Goupil (CNRS, Grenoble, France).

10.2.2 Visits to international teams

10.3.1 Horizon Europe

10.3.2 Other european programs/initiatives

ORA project - Open Research Area (ORA) for the Social Sciences 8th call for proposals

• Title:
  How curiosity enhances learning across childhood and adolescence: The role of metacognition and agency
• Duration:
  From January 5, 2024 to December 31, 2027
• Partners:
  • INSTITUT NATIONAL DE RECHERCHE EN INFORMATIQUE ET AUTOMATIQUE (INRIA), France
  • Cardiff University, UK
  • MaxPlanck institute, Berlin, Germany
• Inria contact:
  Pierre-Yves Oudeyer
• Coordinator:
  Mathias Grüber, Brain and Imagery centre, Cardiff University, UK ; Funds : 1,177 k€
• Summary:
  In a constantly changing world full of uncertainties, one way to adapt to unforeseen circumstances is by harnessing lifelong learning driven by curiosity - the desire to acquire information about the world. Initial research suggests that curiosity not only enhances learning but also metacognitive awareness of one’s own learning progress strengthens curiosity. However, the reciprocal influences between curiosity-based learning and metacognition, particularly during late childhood and adolescence when both abilities continue to develop, remain poorly understood. To address this gap in the existing literature, our interdisciplinary approach aims to explore the bidirectional connections between curiosity-based learning and metacognition during this crucial developmental period. Additionally, late childhood and adolescence is marked by increasing needs of agency, which can boost the curiosity effects on learning because the learner is in control of the learning material. We will therefore seek to study how the protracted development of metacognition enhances the efficiency of curiosity-based learning, especially in situations of high agency. To this end, we will conduct a series of five experiments, leveraging complementary skills and approaches of world-leading research teams around curiosity and development. We will combine different methodological approaches, including behavioral experiments and functional neuroimaging, training studies, and longitudinal assessments to investigate how across-person differences and within-person changes in metacognition and agency contribute to curiosity-based learning across late childhood and adolescence. We will leverage established and recently developed experimental and naturalistic paradigms from our labs to understand how metacognition affects curiosity-based learning across development. Furthermore, we will translate the lab-based findings from the proposed experiments to design pedagogical interventions that stimulate curiosity and metacognition in the classroom. This is a timely and innovative project as recent research provides all the building blocks needed for a step change in our understanding of the mechanisms of curiosity development. The complementary perspectives of the three pioneering teams (Cardiff, Bordeaux, Trier) provide a unique collaborative opportunity to combine parallel research domains to generate important discoveries on the development of curiosity-based learning with broad scientific and societal impact.

Clément Moulin-Frier is continuing his collaboration with Ricard Solé (Universitat Pompeu Fabra, ICREA, Spain) and Marti Sanchez-Fibla (CSIC, Spain) on using Multi-Agent Reinforcement Learning to study fundamental questions on the origins of eco-engineering strategies in the human species. In 2024, this collaboration led to the 3-months visit of Gautier Hamon (Flowers PhD student) in the Complex Systems Lab of Ricard Solé, with a paper in preparation on the emergence of agricultural practices in RL agents. It also led to the publication of a journal article in the Journal of the Royal Society Interface on the cooperative control of environmental extremes 39.

ANR Chaire Individuelle Deep Curiosity

- PY Oudeyer continued to work on the research program of this Chaire, funding 2 PhDs and 3 postdocs for five years (until 2025).

ANR JCJC ECOCURL

- C. Moulin-Frier obtained an ANR JCJC grant. The project is entitled "ECOCURL: Emergent communication through curiosity-driven multi-agent reinforcement learning". The project starts in Feb 2021 for a duration of 48 months. It will fund a PhD student (36 months) and a Research Engineer (18 months) as well as 4 Master internships (one per year).

Projet AIxIA: "Analyse d’Interférences par Intelligence Artificielle".

Pierre-Yves Oudeyer and Clément Moulin-Frier obtained a grant from the call for project AIRSTRIP "L'intelligence Artificielle au service de l'IngénieRie des SysTèmes aéRonautIques et sPatiaux", in collaboration with the IRT Saint Exupery. The project was accepted in 2023 and will fund 18 months of a research engineer position starting in 2024.

Inria Exploratory Action AIDE

- Didier Roy is collaborator of the Inria Exploratory Action AIDE "Artificial Intelligence Devoted to Education", ported by Frédéric Alexandre (Inria Mnemosyne Project-Team), Margarida Romero (LINE Lab) and Thierry Viéville (Inria Mnemosyne Project-Team, LINE Lab). The aim of this Exploratory Action consists to explore to what extent approaches or methods from cognitive neuroscience, linked to machine learning and knowledge representation, could help to better formalize human learning as studied in educational sciences. AIDE is a four year project started middle 2020 until 2024 see.

Inria Exploratory Action I'AM

- Hélène Sauzéon is co-PI with P. Dragicevic of the Inria Exploratory Action I'AM "Impact of Augmented Reality on Autobiographical Memory: Examining Involuntary Memories and False Memories" (174,5k€). Starting in last september, the aim of this Exploratory Action consists to explore to what extent augmented reality based devices can produce erroneous autobiographical memories, and more particularly in vulnerable people (Children and older adults or yound adults with low memory abilities of source monitoring).

New collaboration with Maxime Derex from IAST Toulouse

for the co-direction of the PhD thesis of Jeremy Perez with Clément Moulin-Frier and Pierre-Yves Oudeyer on "Interactions between intrinsically motivated goal-exploration processes and cummulative cultural evolution" (see section 8.2.5).

France 2030 - PPR AUTONOMIE : Vieillissement Et Situations De Handicap - Projet INNOVCare (Lechevalier S., 3,5M€) (2023-26)

- Hélène Sauzéon and AS Rigaud will supervize the WP5 dedicated to two care-led innovation experiments with assistive technologies (400k € for Bordeaux). - Hélène Sauzéon is responsible of the WP3 « Digital technology for aging in place » (470k€/3,5M€), Défi 4 - Numérique, Innovcare (PPR Autonomie PIA2030, 2023-28).

VBHI project(Vascular Brain Health Institute -IHU, led by S. Debette, 5M€)) (2023-26)

- Hélène Sauzéon will supervize the WP4.3 dedicated to "Explore Digital Therapeutics To Slow Down Cognitive Decline In Covert Csvd" (150k€)

10.4.1 Adaptiv'Math

• Adaptiv'Math
• Program: PIA
• Duration: 2019 - 2020
• Coordinator: EvidenceB
• Partners:
  • EvidenceB
  • Nathan
  • APMEP
  • LIP6
  • INRIA
  • ISOGRAD
  • Daesign
  • Schoolab
  • BlueFrog

The solution Adaptiv'Math comes from an innovation partnership for the development of a pedagogical assistant based on artificial intelligence. This partnership is realized in the context of a call for projects from the Ministry of Education to develop a pedagogical plateform to propose and manage mathematical activities intended for teachers and students of cycle 2. The role of Flowers team is to work on the AI of the proposed solution to personalize the pedagogical content to each student. This contribution is based on the work done during the Kidlearn Project and the thesis of Benjamin Clement 101, in which algorithms have been developed to manage and personalize sequence of pedagogical activities. One of the main goal of the team here is to transfer technologies developed in the team in a project with the perspective of industrial scaling.

Member of the organizing committees

• Cédric Colas, Laetitia Teodorescu and Gaia Molinaro co-organized the Intrinsically Motivated Open-ended Learning workshop at Neurips 2024.
• Hélène Sauzeon was a member of the organization committees of 3 scientific events:
  1. International Summer School of Disability Alliance between FEDRHA & REPAR (Canadian Researcher Network for Disability) about « Handicap, mobility and XR: Beyond the real? », May 27-30th, 2024, Marseille (Institut du Mouvement, Université Aix-Marseille);
  2. Annual Conference of FEDHRA, November 7th & 8th, 2024, at the Centre of Inria of the University of Bordeaux, Talence (Journées scientifiques de la FEDRHA: “Le handicap mental à tous les âges de la vie: vivre avec et l'accompagner”);
  3. 2nd Annual Conference of Gérontopôle Nouvelle Aquitaine, April 16th, 2024, Cité internationale de la BD et de l'image, Angoulême.
• Cécile Mazon was a member of the organization committes of 2 scientific evvents
  1. Annual Conference of FEDHRA, November 7th & 8th, 2024, at the Centre of Inria of the University of Bordeaux, Talence (Journées scientifiques de la FEDRHA: “Le handicap mental à tous les âges de la vie: vivre avec et l'accompagner”);
  2. Inria Scientific days on Disability, November 19th & 20th, 2024, at the Hôtel-Dieu, Paris.
• PY Oudeyer co-organized the Life, Structure and Cognition symposium 2024 at IHES, France, this year on the topic "A Multitude of Times", link.
• PY Oudeyer and H. Sauzéon co-organized the Workshop on Children learning at school through curiosity and creativity at Inria Bordeaux.

Member of the conference program committees

• Hélène Sauzéon was a member of the conference program committees of the International Conference on Advances in Computer-Human Interactions (since 2022).
• Hélène Sauzéon attended the program committees of 2 scientific events:
  1. Annual Conference of FEDHRA, November 7th & 8th, 2024, at the Centre of Inria of the University of Bordeaux, Talence (Journées scientifiques de la FEDRHA: “Le handicap mental à tous les âges de la vie: vivre avec et l'accompagner”);
  2. 2nd Annual Conference of Gérontopôle Nouvelle Aquitaine, April 16th, 2024, Cité internationale de la BD et de l'image, Angoulême.
• PY Oudeyer was senior area chair of the Reinforcement Learning Conference 2024.

Reviewer

Clément Romac has reviewed for ICML workshops, EWRL, NeurIPS conference and workshops. Jérémy Perez has reviewed for Evolutionary Human Sciences. Grgur Kovač has reviewed for Artificial Intelligence Review. Nicolas Yax has reviewed for PLOS Computational Biology and Nature Communications Psychology. Sina Khajehabdollahi has reviewed for the ICLR 2024 conference. Matisse Poupard has reviewed for Metaverse journal and NeurIPS workshop. Helene Sauzéon reviewed 3 conference papers for International Conference on Advances in Computer-Human Interactions and ACM Interaction Design & Children. Cécile Mazon reviewed articles for Nature Scientific Reports, Education and Information Technology, and International Journal of Developmental Disabilities

Member of the editorial boards

PY Oudeyer was member of the editorial board of: Frontiers in Neurorobotics.

11.1.1 Invited talks

Hélène Sauzéon gave 2 invited talks:

1. Les technologies numériques au service d’un vieillissement en bonne santé, 13ième Édition Conférence Handicap (IFRATH), 5–7 juin, Paris;
2. CuriousTECH associate team: From the study of curiosity-driven learning across life-span to the design of new interactive curiosity-driven systems for learning in real-life settings. Workshop Inria–University of Waterloo–Université de Bordeaux, 22–23rd February, Bordeaux, France.

PY Oudeyer gave 9 invited talks:

• (mars 2024) Curiosité et métacognition: sciences cognitives, IA et éducation, Groupe de travail sur la métacognition du Conseil Scientifique de l'Education Nationale.
• (mars 2024) Grounding Language and LLMs through Autotelic Curiosity-driven Learning, Seminar in Cognitive Informatics, DIC, UQÀM, Canada.
• (June 2024) Les enjeux de la recherche en IA pour l'éducation, Conseil Scientifique de l'Education Nationale
• (June 2024) Open-ended learning and curiosity in infants and machines: learning progress, autotelism and language, Workshop on Emotions and Learning in infants, University Paris Nanterre
• (July 2024) Generating a Diversity of Challenging Programming Puzzles with Autotelic Generative Models, Workshop LLM4Code, Inria Paris.
• (Oct 2024) The learning progress hypothesis: a theory of curiosity-driven learning and applications in educational technologies, Workshop on Leveraging educational games to study motivation, University of Amsterdam.
• (Nov 2024) Open-ended generation of diverse solvable coding problems with autotelic generative models, Workshop on Program Synthesis, Labri, University of Bordeaux.
• (Nov 2024) Les enjeux sociétaux de l'éducation à l'IA générative, Journée "Enfance et numérique: pour un numérique éducatif au service de la réussite de tous les élèves", associée à la chaire Neuroéducation et créativité (G. Borst), Sorbonne Université.
• (Jan 2025) Curiosity-driven learning in humans: learning progress, autotelic exploration and open-ended development, Keynote lecture at the Budapest Conference on Cognitive Development.

11.1.2 Leadership within the scientific community

Hélène Sauzéon was member of scientific committe of Gérontopôle Nouvelle-Aquitaine, since 2021

Helene Sauzéon supported the structuration of inria research network on Disability and the valorization research outcomes for the internal road map for Inria as socio-organization since 2023

Helene sauzéon is member of GIS Institut de la Longévité, des Vieillesses et du Vieillissement (ILVV), since 2021 and of RTP CNRS Éducation network since 2022.

Hélène Sauzéon and PY Oudeyer are members of two local NA networks related to digital research and its applications (R3IA et R3NumEd) since 2022

Cécile Mazon is co-responsible of the WP on technological tool in the PIA AtypieFriendly

PY Oudeyer collaborated with J. Gottlieb, R. Wilson and E. Bonawitz on the design of new conference series in the Gordon Research Conference program, dedicated to the Sciences of Curiosity.

PY Oudeyer continued to be part of the steering committee of the Life, Structure and Cognition conference series at IHES, which is a yearly interdisciplinary symposium organized at IHES, and where every year he participates to define the main scientific topics and to select and invite speakers from various disciplines.

PY Oudeyer was part of the advisory committee of the Intrinsically Motivated Open-ended Learning workshop at Neurips 2024.

PY Oudeyer led the design of the AI dimensions of the research and development program of major P2IA (ministry of education)and BPI projects in educational technologies, in collaboration with the EvidenceB company.

11.1.3 Scientific expertise

Hélène Sauzéon was member of HCERES Evaluation Committee of the ChArT Lab (Paris 8) in 2024.

Hélène Sauzéon was member of international committee of “Fonds Rech. Québec” for assessing the Canadian multidisciplinary research network on ageing, 2024

Hélène Sauzéon was a reviewer for the ANRT (2 PhD projects)

Hélène Sauzéon is member of the ANR Committee for “interface” Topic (CE 38 pluridisciplinary committee: Computer, Humaties, and social sciences) since 2023

Hélène Sauzéon participated to a working group for instructing the creation of an Inria Project Team (new-STARS, for the center of Nice -Sophia Antipollis) in 2024

Hélène Sauzéon is scientific expert of the Research Project Call by Fondation Clément Fayat (since 2022).

Clément Moulin-Frier reviewed a project for the ANR.

PY Oudeyer reviewed projects for the Templeton Foundation and the European Commission, and was a reviewer of the CIFAR research program on Learning in Brains and Machines (Nov. 2024).

11.1.4 Research administration

PY Oudeyer has been head of the Flowers project team, and coordinated the design and writing of a new research program for the renewal of the project-team at Inria.

Hélène Sauzéon is local referent at the Inria Centre of UB for PIQ since 2024

Hélène Sauzéon is responsible of the Research axis "Interventions innovantes" in ACTIVE team of the BPH Lab (UB-Inserm) since 2022

Hélène Sauzéon is the head of the Innovations and Transfer Committee of the BIND Center of Excellence in Bordeaux and member of the BIND directory Committee since 2018

Hélène Sauzéon is member of Scientific Committee of extented committee of Project-team of the centre of Inria of university of Bordeaux, since 2020.

Hélène Sauzéon is member of directory committee of IFHR which is a national institute on disability funded by Inserm aiming the researcher networking and dissemination of knowledge on multidisciplinary research on disability; since 2018.

Helene Sauzéon was member of scientific committee of MAVIE-II-Calyxis.

11.2.1 Teaching

Founded Teaching Innovation projects

• Hélène Sauzéon is responsible of a task within the i UB2030-CAP santé numérique - France 2030 -AMI 2022 Compétences et Métiers d'avenir, Volet santé numérique (€3.1m) (Coord., R. Thibault). This will involve applying and testing algorithms for personalizing learning paths on certain e-learning courses in health and digital technology for the Bordeaux University (€120k).
• Hélène Sauzéon is member of the "Jeu, Inclusivité & Hanpathie (JIH)" project (Coord., E. Dugas, in charge of Mission Handicap, Bordeaux University) - AAP Région NA 2023- "Favoriser la réussite étudiante" (Promoting student success). The aim of this educational project is to design a digital game to develop empathy towards students with disabilities and thus contribute to a better quality of social life for these students (interactions with their peers and University staff).

Teaching Responsibilities in University structures:

• Clément Moulin-Frier is responsible professor of the "System Design, Integration and Control" course at the University Pompeu Fabra in Barcelona, Spain.
• Cécile Mazon is responsible of the second year of the master curriculum in Technology, Ergonomics, Cognition and Handicap (TECH, Cognitive Sciences - University of Bordeaux) since sept. 2021.
• Cécile Mazon is responsible of the master curriculum in Technology, Ergonomics, Cognition and Handicap (TECH, Cognitive Sciences - University of Bordeaux) since sept. 2022.
• Cécile Mazon is responsible of the licence curriculum in Cognitive sciences (Licence MIASHS) since sept. 2024.

Teaching Involvement in Computer / Engineer science or in cognitive science for Bachelors and master Degrees:

• BS & Master: Cognitive Science, Univ. of Bordeaux- , 104,5h, Marion Pech
• BS & Master: Neuropsychology, Univ. of Caen-,4H, Marion Pech
• Licence MIASHS,Univ. of Bordeaux: Supervision for research, (6h) Marion Pech
• ENSC/ENSEIRB Presentation of developmental artificial intelligence and the Flowers Lab, 2h, Option Robot (Laetitia Teodorescu)
• ENSC Introduction to bayesian analysis, 8h, Option AI (Adolphe Maxime)
• ENSC Transformers and Large Language Models, 16h, Option AI (Clément Romac)
• ENSC Reinforcement Learning, 8h, Option AI (Clément Romac)
• BS & Master: Cognitive Science, Univ. of Bordeaux- , 240h, Cécile Mazon
• Master: Navigation for Robotics, 21 h, M2, ENSTA Paris, David Filliat
• Master: Navigation for Robotics, 24 h, M2 DataAI, IP Paris - Paris, David Filliat
• Université de Bordeaux - MIASHS Bachelor : Models and measures of high-level cognitive functions - knowledge and representation, 22h, (Poupard Matisse)
• Université de Bordeaux - TECH Master: Virtual reality, interaction and healthcare applications, 7h, (Poupard Matisse)
• Université de Bordeaux - TECH Master: Scientific basis, 2h, (Poupard Matisse)
• 2nd year : Deep Learning, 12h, IMT Atlantique (Sao Mai Nguyen).
• Université de Bordeaux - TECH Master: IT Project Management, 18h (Isabeau Saint-Supery)
• Université de Bordeaux - TECH Master: IT Project Management, 14h (Marie-Sarah Desvaux)
• Master UPF-Barcelona: Robotics and AI, 10h (Clément Moulin-Frier)
• PY Oudeyer gave a course on developmental reinforcement learning at ENSEIRB (2h), dec. 2024.
• PY Oudeyer gave a course on developmental reinforcement learning at Cogmaster at Sorbonne Univ./ENS (2h+4h project jury), dec. 2024.
• Master: Cognitive Science, 24h (Eric Meyer)
• Teacher training at the Rectorat of Bordeaux : digital technologies for students with special educational needs , 6h (Eric Meyer)
• 2nd year Master in cognitive science : Assistive technologies (20h), Rania Abdelghani.
• University Degree in Neuropsychological Sciences : Cognitive aging and Digital technologies (3h), Helene Sauzéon
• Clément Moulin-Frier and Corentin Léger gave a course on System Design, Integration and Control at the University Pompeu Fabra in Barcelona, Spain (7.5h). This year, we used for the first time the new Vivarium simulator (see New Software).

Teaching Involvement in Computer / Engineer science or in cognitive science for lifelong curriculum

• Hélène Sauzéon and Marie-Sarah Desvaux gave talks and animated workshops for 'École Académique de la Formation Continue' of Rectorat of Paris, on the subject of Curiosity-driven learning in schools
• Hélène Sauzéon and Matisse Poupard gave a talk and animated a workshops on "User-centered design of XR applications" for 'École d'été 2024 - Mobilité et réalité virtuelle : au delà du réel' (FEDRHA/REPAR)
• University Degree in Neuropsychological Sciences : Cognitive aging and Digital technologies (3h), Helene Sauzéon
• University Degree in Neuropsychological Sciences : Neuropsychological assessment in normal aging (3h), Cécile Mazon

11.2.2 Supervision

• Rania ABDELGHANI (50%, codirection avec P.Y. Oudeyer, DR Inria) Guider les esprits de Demain : Agents Conversationnels pour Entraîner la Curiosité et la Métacognition chez les Jeunes Apprenants, depuis Octobre 2020 (CIFRE avec EvidenceB), defended, Sept. 27th, 2024.. (supervisors: H. Sauzéon & PY. Oudeyer)
• Maxime ADOLPHE (50%, codirection avec P.Y. Oudeyer, DR Inria) Design and evaluation of new personalisation algorithms embedded in an intelligent tutorial system for attention training / Conception et évaluation de nouveaux algorithmes de personnalisation embarqués dans un système tutoriel intelligent d’entraînement de l’attention” beg. in sept. 2020 (CIFRE contract with Onepoint), defended, Sept. 27th, 2024. (supervisors: H. Sauzéon & PY. Oudeyer)
• PhD in progress: Grgur Kovac, "Developmental training of socio-cognitive abilities in AI systems", (supervisors:PF. Dominey and PY. Oudeyer)
• PhD in progress: Gauthier Hamon, "Open-endedness in artificial life and articial intelligence: an eco-evo-devo perspective" (supervisor: C. Moulin-Frier)
• PhD in progress: Nicolas Yax, "Studying cognitive and metacognitive skills in foundation models" (supervisors: S. Palminteri, PY. Oudeyer)
• PhD in progress: Julien Pourcel, "Autotelic LLMs that learn how to code", (supervisors: C. Moulin-Frier and PY. Oudeyer)
• PhD in progress: Thomas Carta, "LLM-based Autotelic deep reinforcement learning agents", (supervisors: O. Sigaud, S. Lamprier and PY. Oudeyer)
• PhD in progress: Clément Romac, "Grounding LLMs with online RL", (supervisors: T. Wolf and PY. Oudeyer)
• PhD in progress: Jeremy Perez, "Studying mechanisms and roles of curiosity in socio-cultural contexts" (supervisors: C. Moulin-Frier, M. Derex, PY. Oudeyer)
• PhD in progress: Timothé Boulet, "Controller synthesis for artificial agents in simulated environments using generative AI" (supervisors C. Moulin-Frier, X. Hinault, N. Fijalkow)
• PhD in progress: Marko Cvjetko, "Autotelic exploration algorithms for automated search of open-endedness in artificial life" (supervisors: C. Moulin-Frier, PY. Oudeyer) item PhD in progress: Loris Gaven, "Metacognitive prediction of learning progress for guiding autotelic agents" (supervisors: PY. Oudeyer and C. Moulin-Frier)
• PhD in progress: Isabeau SAINT-SUPERY "Designing and Assessing a new interactive tool fostering stakeholders' cooperation for school inclusion", beg. in sept. 2021, supervised by H. Sauzéon and C. Mazon.
• PhD in progress : Matisse POUPARD "Optimize learning in a digital environment according to learners' level of expertise, epistemic curiosity and mode of instruction" ", beg. in April. 2022 (CIFRE Contract with Catie), supervised by H. Sauzéon and A. Tricot (Univ. Montpellier).
• PhD in progress : Léana PETIOT (50% codirection avec P. Dragicevic, CR Inria) Impact of augmented reality on involuntary autobiographical memory /Impact de la réalité augmentée sur la mémoire autobiographique involontaire, beg. in. Oct. 2023 (Allocation Inria AEx IAM, 2023-25), supervised by H. Sauzéon and P. Dragicevic (Potioc Team, CR Inria)
• PhD in progress : Chloé (Marie-Sarah) DESVAUX (100%) Designing and testing digital metacognitive training workshops to encourage curiosity and creativity in a school context Concevoir et expérimenter des ateliers numériques d’entraînement métacognitifs pour favoriser la curiosité et la créativité en contexte scolaire, Beg. Oct. 2023 (Alloc. MESRI, ED SP2 -Univ. Bordeaux), supervised by H. Sauzéon and PY Oudeyer
• PhD in progress : Juliette DEYTS “HomeAssist : study of actimetric and clinical data from 131 equiped participants” (Alloc. Sur Projet ANR Innovcare, ED SP2 -Univ. Bordeaux), début Nov. 2024, supervised by H. Sauzéon and Lucile Dupuy, MCF (BPH lab, Active team)

11.2.3 Juries

• PY Oudeyer was a member of the selection committee of the Inria Prize from Académie des Sciences.
• PY Oudeyer was reviewer in the PhD of Jerome Tinker (OIST, Japan), James Daly (Univ. Texax, US) and Jean-Baptiste Gaya (Sorbonne Université).
• PY Oudeyer was in the PhD "comité de suivi" of Yanis Bendi-Ouis (Univ. Bordeaux), Paul-Antoine Le Tolguennec (Univ. Toulouse), Matthis Poupard (Univ. Bordeaux), Marie Martin (Université Interdisciplinaire de Paris), Marc Welter (Univ. Bordeaux),
• Clément Moulin-Frier was reviewer of the PhD thesis of Timothée Anne (thesis director: Jean-Baptiste Mouret) at the Université of Lorraine (France), defended on June 6, 2024.
• Clément Moulin-Frier was reviewer of the PhD thesis of Maxime Toquebiau (thesis director: Nicolas Brédèche) at Sorbonne Université (ISIR lab), defended on December 13, 2024.
• Clément Moulin-Frier was examinator of the PhD thesis of Loïc Goasguen (thesis director: Sylvain Argentieri) at Sorbonne Université (ISIR lab), defended on December 18, 2024.
• Clément Moulin-Frier was examinator of the PhD thesis of Noami Chaix-Eichel (thesis director: Nicolas Rougier) at the Université of Bordeaux, defended on November 22, 2024.
• H. Sauzéon is member of 6 PhD juries (2 president, 3 reviewer ; 1 examiner).
  • Jianling Zou Laboratoire Cognitions humaine et artificielle (EA4004)/ Uni. Paris 8 President
  • Christelle Nahas Equipe TheMAS - laboratoire TIMC (UMR 5525) / Uni. Grenoble Reviewer
  • Hugo Fournier Equipe Santé - LaB ; de Psychologie / UB President
  • Elen Sargzyan Equipe ELIPSE – Lab. IRIT / University Paul Sabatier Toulouse Reviewer
  • Edwige Chauvergne Equipe Bivouac – Centre Inria de l’UB Examiner
  • Julie Restout UR 20217 HAVAE -Institut Recherche Omegahealth / Univ. Limoges Reviewer
• H. Sauzéon and C. Mazon are permanent members of jury of Master degree in cognitive science at the university of Bordeaux.

11.3.1 Specific official responsibilities in science outreach structures

PY Oudeyer participated several time to the piloting committee of the research network on education and digital sciences (R3NumEd, Nouvelle Aquitaine)

11.3.2 Productions (articles, videos, podcasts, serious games, ...)

D. Roy and P-Y. Oudeyer wrote a popular science book to introduce generative AI (mechanisms, applications, societal dimensions) to adolescents, as well as to their teachers and families. It is entitled "C'est (pas) moi, c'est l'IA", and was published in september 2024 by Nathan. It was reviewed in widely distributed magazines (e.g. Magazine de l'APEL) and radios (e.g. France Culture, RFI). The web page of the book is here: http://­www.­pyoudeyer.­com/­clia/.

A. Torres-Leguet, C. Romac, T. Carta and PY. Oudeyer produced the pedagogical video series "ChatGPT explained in 5 mn", aimed at training generative AI literacy in a wide diversity of students (e.g. high school), available here: https://­developmentalsystems.­org/­chatgpt_en_5_minutes/. They are under a Creative Commons licence, CC-BY, enabling open and free reuse. They were already integrated in the MOOC AI4T (https://­www.­ai4t.­eu), as well as in an internal training platform of "Académie du Numérique du Ministère de la défense", in a mobile app made by Inria with educational materials related to AI (https://­epoc.­inria.­fr/­epocs/­E009LL/), and are being adapted and integrated in a training platform for the whole population of civil servants in France, coordinated by DINUM.

Clément Romac, Thomas Carta and Pierre-Yves Oudeyer wrote down the article "Les algorithmes des IA peuvent-ils comprendre notre monde ?" published in Pour La Science n°557 (February 2024).

Clément Moulin-Frier, Gautier Hamon and Pierre-Yves Oudeyer wrote down the article "Quand l'IA explore les prémisses d'une vie artificielle" published in Pour La Science n°562 (August 2024).

PY Oudeyer wrote a note for the French educational institutions on "IA générative, société et éducation: En quoi l’IA générative représente-elle un enjeu dans la formation des citoyens ?", in the context of the Conférence de Consensus on Nouveaux Savoirs et Nouvelles Compétences des Jeunes of Cnesco, (Nov. 2024)

Hélène Sauzéon participated to the editing of 2 Popularization articles for the communication services of the centre Inria of University of UB : IA et apprentissage ; réinventer l’éducation).

Hélène Sauzéon participated to the “1 minute with…” video program by the popularization service : 1 Minute avec H. Sauzéon

11.3.3 Participation in Live events

Marion Pech & Ikram Chraibi Kaadoud (2024) « ChatGPT sera-t-il votre psychologue 2.0 ? ». Oral communication for the French Tech, November 21, Bordeaux. France.

Marion Pech, Florence Amardeilh, Gala Leibar, Trang Pham [DigitEco] (2024) Placéco program produced by Digital Aquitaine on the TV7 plateau in Sud-Ouest. La place des femmes dans la tech : le compte n'y est pas, comment faire ? Hosted by Gaëlle Richard. November 6, Bordeaux. France

Hélène Sauzéon took part in a round-table discussion; TABLE RONDE - Communautés, espaces sociaux, impact de l’IA sur la restructuration des sociétés Forum NAIA -Robocup, 2024, Bordeaux.

Hélène Sauzéon took part in a round-table discussion about the Education and generative AI, Colloque Infiné "Éducation et I.A 11 octobre 2024, Futuroscope, Poitiers.

Hélène Sauzéon took part in the science popularization event for secondary schools Têtes chercheuses 2024.

Marion Pech, Florence Amardeilh, Gala Leibar, Trang Pham [DigitEco] (2024) Placéco program produced by Digital Aquitaine on the TV7 plateau in Sud-Ouest. La place des femmes dans la tech : le compte n'y est pas, comment faire ? Hosted by Gaëlle Richard. November 6, Bordeaux. France

Marion Pech (2024) « ChatGPT sera-t-il votre psychologue 2.0 ? ». Oral communication for the French Tech, November 21, Bordeaux. France.

PY Oudeyer participated to several live events:

• (Sept. 2024) Presentation and interview about the popular science book "C'est (pas) moi c'est l'IA" at an event organized by Nathan Jeunesse, Paris.
• (Oct. 2024) Conceptualizing education and training alongside generative AI, BIG Conference, organized by BPI.
• (Dec 2024) Presentation and debate about the topic "Comment éduquer à un usage responsable des IA ?", organized by CCIC-Unesco, Apprentis d'Auteuils.

11.3.4 Others science outreach relevant activities

Clément Moulin-Frier co-organized the 3rd edition of Hack1robo with Xavier Hinaut (Mnemosyne), David Trocellier (Potioc) and Anne-Lise Pernel (SAER). The event took place at Le Node in Bordeaux in November 2024 (15-17) and received financial support from Inria, Inria Startup Studio, R3NumEd, R4 and GDR Robotique. https://­sites.­google.­com/­view/­hack1robo/­accueil. There was more than 40 participants, from teenagers to seniors (but mostly University students), working on 7 collaborative projects. Two of these projects were led by students of the Flowers team. The first project (led by Guillaume Pourcel, Julien Pourcel, Loris Gaven and Corentin Léger from Flowers) explored whether making Large Language Models more socially persuasive (rather than purely truth-seeking) could improve their reasoning abilities, by having them engage in group debates and testing if enhanced persuasiveness leads to better argument generation and evaluation. This project received the 1st prize of Hack1robo. The second project (led by Sina Khajehabdollahi, Jeremy Perez and Marko Cjvetko) merged music with visually representable dynamical systems to create music-driven animations. The core idea was to extract musical features—such as tempo, frequency, and other parameters—and use them to manipulate a dynamical system in real-time. For this purpose, the team selected Lenia, a class of continuous cellular automata renowned for generating diverse, life-like patterns. By mapping musical features to Lenia’s parameters, they created a visually engaging, ever-evolving display of music-inspired animation. The implementation was built using TouchDesigner, a visual programming language for creating interactive multimedia content. This framework allows users to easily modify the interaction between music and Lenia, enabling efficient exploration of possible mappings. An example of how the audio signal and Lenia interact is shown in 55.

Hélène Sauzéon gave a talk on the sciences of autonomy and disability to the National Assembly's "Longevity and adaptation of society to ageing" study group, 2024;

Cécile Mazon participated to debates about teaching Cognitive Sciences during the CogGames (November 18th & 19th, Bordeaux).

Hélène Sauzéon gave an interview for the Sud-Ouest » news papers Sud-Ouest "Assistants vocaux : faut-il laisser les enfants utiliser seuls une enceinte connectée ? ;

Hélène Sauzéon provides the Scientific Council with support for innovation in companies with 2 actions in 2024 : for TelecomDesign and produit Vibby (contract being set up with STIP Bdx) and REVLIM via the European DIHNAMIC Nvelle Aquitaine - STIP Bdx project.

PY. Oudeyer was interviewed, or the work of the team was discussed, in various newspapers, magazines and radios/podcasts:

• Scientific American, Dec. 2024, Hitting the curiosity sweep spot speeds up learning.
• Famille et Education, Le magazine de l'APEL, Déc. 2024, IA, un vrai ou faux ami?
• Lemonde.fr, Oct. 2024, MIA Seconde : à l’heure de ChatGPT, le tuteur sous intelligence artificielle de l’éducation nationale est-il déjà obsolète ?
• Les Échos, Nov. 2024, La France face au défi de la formation à l'IA.
• RFI, Sept. 2024, Comment se familiariser avec l’IA ? Emission 8 milliards de voisins
• France Culture, Sept. 2024, "C'est pas moi, c'est l'IA", un manuel indispensable pour comprendre l'intelligence artificielle
• Le Café Pédagogique, 2024, "Il est urgent de former les profs à l'IA"
• Le Monde Numérique" (Jérôme Colombain): "Apprivoiser l'IA à l'école"
• Blog Binaire (Lemonde.fr) : Mon enfant apprivoise l'IA !
• Sqool TV, Sept. 2024, L'école du futur: comment apprivoiser l'IA?
• La Nouvelle République, 2024, L'IA, une aide aux devoirs?
• Geek Junior: C'est par moi c'est l'IA: Voici un guide malin et bienveillant pour découvrir comment fonctionne l'IA et à quoi elle sert

International journals

• 33 articleM.Mayalen Etcheverry, C.Clément Moulin-Frier, P.-Y.Pierre-Yves Oudeyer and M.Michael Levin. AI-driven Automated Discovery Tools Reveal Diverse Behavioral Competencies of Biological Networks.eLifeAugust 2024HALDOIback to textback to textback to textback to text
• 34 articleH.Hugo Fournier, M.Marie Hasdenteufel, C.Constance Garrouteigt, M.Mathieu Perie, A.Antoine Gliksohn, B.Beatrice Jouanne, S.Smail Hadj-Rabia, B.Benoit Arveiler, F.Fanny Morice-Picard and B.Bruno Quintard. The experience of albinism in France: a qualitative study on dyads of parents and their adult child with albinism.BMC Medicine221January 2024, 40HALDOI
• 35 articleG.Grgur Kovač, R.Rémy Portelas, P. F.Peter Ford Dominey and P.-Y.Pierre-Yves Oudeyer. The SocialAI school: a framework leveraging developmental psychology toward artificial socio-cultural agents.Frontiers in Neurorobotics18October 2024HALDOIback to text
• 36 articleG.Grgur Kovač, R.Rémy Portelas, M.Masataka Sawayama, P. F.Peter Ford Dominey and P.-Y.Pierre-Yves Oudeyer. Stick to your role! Stability of personal values expressed in large language models.PLoS ONE198August 2024, e0309114HALDOIback to text
• 37 articleM.Matisse Poupard, F.Florian Larrue, H.Hélène Sauzéon and A.André Tricot. A systematic review of immersive technologies for education: effects of cognitive load and curiosity state on learning performance.British Journal of Educational Technology5612025, 5-41HALDOIback to text
• 38 articleJ.Julien Pourcel, C.Cédric Colas, G.Gaia Molinaro, P.-Y.Pierre-Yves Oudeyer and L.Laetitia Teodorescu. ACES: Generating diverse programming puzzles with autotelic language models and semantic descriptors.Neurips2024HALback to text
• 39 articleM.Martí Sánchez-Fibla, C.Clément Moulin-Frier and R.Ricard Solé. Cooperative control of environmental extremes by artificial intelligent agents.Journal of the Royal Society Interface212202024HALDOIback to textback to text
• 40 articleY.Yadurshana Sivashankar, M.Myra Fernandes, P.-Y.Pierre-Yves Oudeyer and H.Hélène Sauzéon. The beneficial role of curiosity on route memory in children.Frontiers in Cognition3March 2024HALDOI
• 41 articleY.Y. Sivashankar, P. L.P. L. He, H.Helene Sauzeon and M. A.M. A. Fernandes. Motoric engagement, but not decision-making, during encoding influences route memory.Spatial Cognition and ComputationMay 2024, 1-26HALDOI

International peer-reviewed conferences

• 42 inproceedingsL.Louis Annabi, Z.Ziqi Ma and S. M.Sao Mai Nguyen. Unsupervised Motion Retargeting for Human-Robot Imitation.HRI 2024 - Companion of the 2024 ACM/IEEE International Conference on Human-Robot InteractionBoulder (CO), United StatesACMMarch 2024HALDOI
• 43 inproceedingsD.Damien Bouchabou and S. M.Sao Mai Nguyen. Generative Pretrained Embedding and Hierarchical Irregular Time Series Representation for Daily Living Activity Recognition.Frontiers in Artificial Intelligence and ApplicationsECAI 2024Frontiers in Artificial Intelligence and ApplicationsSantiago de Compostela, SpainIOS PressOctober 2024, 4764 - 4771HALDOI
• 44 inproceedingsD.Damien Bouchabou, S. M.Sao Mai Nguyen, C.Christophe Lohr, B.Benoit Leduc and I.Ioannis Kanellos. Representation of Irregularly Sampled Time Series with Generative Language Models for Classification and Transfer Learning: a Case Study in Activities of Daily Living Recognition.ML4ITS 2023 - Machine Learning for Irregular Time Series Workshop in ECML PKDDTurin (IT), ItalyFebruary 2024HAL
• 45 inproceedingsD.David Brellmann, E.Eloïse Berthier, D.David Filliat and G.Goran Frehse. On double descent in reinforcement learning with LSTD and random features.Proceedings of the Twelfth International Conference on Learning Representations (ICLR)ICLR 2024 - The Twelfth International Conference on Learning RepresentationsVienna, AustriaMay 2024HAL
• 46 inproceedingsG.Gianni Franchi, M.Marwane Hariat, X.Xuanlong Yu, N.Nacim Belkhir, A.Antoine Manzanera and D.David Filliat. InfraParis: A multi-modal and multi-task autonomous driving dataset.WACV 2024 - IEEE/CVF Winter Conference on Applications of Computer VisionWaikoloa, United States2024HAL
• 47 inproceedingsG.Grgur Kovač, R.Rémy Portelas, M.Masataka Sawayama, P.Peter Ford Dominey and P.-Y.Pierre-Yves Oudeyer. Stick to your Role! Stability of Personal Values Expressed in Large Language Models.Proceedings of the Annual Meeting of the Cognitive Science Society Volume 46CogSci 2024Rotterdam, NetherlandsJuly 2024HALback to textback to text
• 48 inproceedingsC.Corentin Léger, G.Gautier Hamon, E.Eleni Nisioti, X.Xavier Hinaut and C.Clément Moulin-Frier. Evolving Reservoirs for Meta Reinforcement Learning.Applications of Evolutionary ComputationInternational Conference on the Applications of Evolutionary Computation (Part of EvoStar)14635Lecture Notes in Computer ScienceAberystwyth, United KingdomSpringer Nature Switzerland2024, 36-60HALDOIback to text
• 49 inproceedingsA.Aleksa Marusic, S. M.Sao Mai Nguyen and A.Adriana Tapus. PhysioFormer: A Spatio-Temporal Transformer for Physical Rehabilitation Assessment.ICSR 2024HRI '23: Companion of the 2023 ACM/IEEE International Conference on Human-Robot InteractionOdense, DenmarkLecture Notes in Artificial Intelligence series (LNAI)October 2024HALDOI
• 50 inproceedingsG.Gaia Molinaro, C.Cédric Colas, P.-Y.Pierre-Yves Oudeyer and A.Anne Collins. Latent Learning Progress Drives Autonomous Goal Selection in Human Reinforcement Learning.NeurIPS 2024 - The Thirty-Eighth Annual Conference on Neural Information Processing SystemsVancouver, CanadaJune 2024HALDOIback to textback to text
• 51 inproceedingsS. M.Sao Mai Nguyen, M.Maxime Devanne, O.Olivier Remy-Neris, M.Mathieu Lempereur and A.Andre Thepaut. A Medical Low-Back Pain Physical Rehabilitation Dataset for Human Body Movement Analysis.IJCNN 2024 - International Joint Conference on Neural NetworksYokohama, Japan2024HAL
• 52 inproceedingsE.Eleni Nisioti, S.Sebastian Risi, I.Ida Momennejad, P.-Y.Pierre-Yves Oudeyer and C.Clément Moulin-Frier. Collective Innovation in Groups of Large Language Models.ALIFE 2024 - The Conference on Artificial LifeCopenhagen, DenmarkMIT Press2024HALDOIback to textback to text
• 53 inproceedingsG.Gabriel Quere, F.Freek Stulp, D.David Filliat and J.Joao Silverio. A probabilistic approach for learning and adapting shared control skills with the human in the loop.Proceedings of the 2024 IEEE International Conference on Robotics and AutomationICRA 2024 - IEEE International Conference on Robotics and AutomationYOKOHAMA, JapanMay 2024HAL
• 54 inproceedingsM.Max Taylor-Davies, G.Gautier Hamon, T.Timothé Boulet and C.Clément Moulin-Frier. Emergent kin selection of altruistic feeding via non-episodic neuroevolution.Applications of Evolutionary ComputationInternational Conference on the Applications of Evolutionary Computation (Part of EvoStar)Trieste, Italy2025HALback to text
• 55 inproceedingsM.Mehdi Zadem, S.Sergio Mover, S.Sergio Mover and S. M.Sao Mai Nguyen. Reconciling Spatial and Temporal Abstractions for Goal Representation.ICLR 2024 - The Twelfth International Conference on Learning RepresentationsVienna, AustriaMay 2024HAL

Conferences without proceedings

• 56 inproceedingsQ.Quentin Gallouédec, E.Edward Beeching, C.Clément Romac and E.Emmanuel Dellandréa. Jack of All Trades, Master of Some, a Multi-Purpose Transformer Agent.ARLET 2024 - Aligning Reinforcement Learning Experimentalists and Theorists (Workshop at the International Conference on Machine Learning)Vienne, AustriaJuly 2024HAL
• 57 inproceedingsS. M.Sao Mai Nguyen. Online Learning a Symbolic Abstraction of Actions in Hierarchical RL with Formal Methods.ActSynt@ECAI2024Santiago de Compostela, SpainOctober 2024HAL
• 58 inproceedingsM.Matisse Poupard, F.Florian Larrue, H.Hélène Sauzéon and A.André Tricot. The Interplay of Cognitive Load and Intrinsic Motivation in Neuroanatomy Learning through Immersive Technologies.CLT 2024 - The 16th International Cognitive Load Theory ConferenceSydney, AustraliaNovember 2024HALback to text
• 59 inproceedingsG.Guillaume Pourcel, T.Thomas Carta, G.Grgur Kovač and P.-Y.Pierre-Yves Oudeyer. Autotelic LLM-based exploration for goal-conditioned RL.Intrinsically Motivated Open-ended Learning Workshop at NeurIPS 2024Vancouver, CanadaDecember 2024HAL

Edition (books, proceedings, special issue of a journal)

• 60 proceedingsUstensiles et espaces culinaires de la Protohistoire au début du XXe siècle, pré-actes du colloque international Corpus, Dijon, 4-7 juin 2024.Ustensiles et espaces culinaires de la Protohistoire au début du XXe siècleCORPUSJune 2024, 79 p.HAL

Doctoral dissertations and habilitation theses

• 61 thesisM.Maxime Adolphe. Development and evaluation of AI-based personalization algorithms for attention training.Université de BordeauxSeptember 2024HAL
• 62 thesisS. M.Sao Mai Nguyen. The intrinsic motivation of reinforcement and imitation learning for sequential tasks: Body movement analysis, activity of daily living analysis and intrinsically motivated robot for hierarchical reinforcement learning.Sorbone UniversitéSeptember 2024HAL

Reports & preprints

• 63 miscM. S.Mohamed Salim Aissi, C.Clément Romac, T.Thomas Carta, S.Sylvain Lamprier, P.-Y.Pierre-Yves Oudeyer, O.Olivier Sigaud, L.Laure Soulier and N.Nicolas Thome. Reinforcement Learning for Aligning Large Language Models Agents with Interactive Environments: Quantifying and Mitigating Prompt Overfitting.December 2024HAL
• 64 reportB.Benjamin Clément, H.Hélène Sauzéon, D.Didier Roy and P.-Y.Pierre-Yves Oudeyer. Improved Performances and Motivation in Intelligent Tutoring Systems: Combining Machine Learning and Learner Choice.Inria Bordeaux Sud-OuestFebruary 2024HAL
• 65 miscB.Benjamin Clément, H.Hélène Sauzéon, D.Didier Roy and P.-Y.Pierre-Yves Oudeyer. Improved performances and motivation in intelligent tutoring systems: combining machine learning and learner choice.January 2024HAL
• 66 miscG.Gautier Hamon, M.Mayalen Etcheverry, B.-C. W.Bert Wang-Chak Chan, C.Clément Moulin-Frier and P.-Y.Pierre-Yves Oudeyer. Discovering Sensorimotor Agency in Cellular Automata using Diversity Search.February 2024HAL
• 67 miscJ.Jérémy Perez, G.Grgur Kovač, C.Corentin Léger, C.Cédric Colas, G.Gaia Molinaro, M.Maxime Derex, P.-Y.Pierre-Yves Oudeyer and C.Clément Moulin-Frier. When LLMs Play the Telephone Game: Cumulative Changes and Attractors in Iterated Cultural Transmissions.2024HALback to textback to textback to text
• 68 miscJ.Jérémy Perez, C.Corentin Léger, M.Marcela Ovando-Tellez, C.Chris Foulon, J.Joan Dussauld, P.-Y.Pierre-Yves Oudeyer and C.Clément Moulin-Frier. Cultural evolution in populations of Large Language Models.2024HALDOIback to text
• 69 miscM.Matisse Poupard, F.Florian Larrue, M.Martin Bertrand, D.Dominique Liguoro, A.André Tricot and H.Hélène Sauzéon. Using virtual reality for enhancing neuroanatomy learning by optimizing cognitive load and intrinsic motivation.October 2024HALback to text
• 70 miscM.Megha Srivastava, C.Cédric Colas, D.Dorsa Sadigh and J.Jacob Andreas. Policy Learning with a Language Bottleneck.2024HALDOI
• 71 miscN.Nicolas Yax, P.-Y.Pierre-Yves Oudeyer and S.Stefano Palminteri. PhyloLM : Inferring the Phylogeny of Large Language Models and Predicting their Performances in Benchmarks.January 2025HAL

Other scientific publications

• 72 inproceedingsL.Loris Gaven, C.Clément Romac, T.Thomas Carta, S.Sylvain Lamprier, O.Olivier Sigaud and P.-Y.Pierre-Yves Oudeyer. SAC-GLAM: Improving Online RL for LLM agents with Soft Actor-Critic and Hindsight Relabeling.IMOL 2024 - Intrinsically Motivated Open-ended Learning (Workshop at Neurips)Vancouver, CanadaOctober 2024HAL
• 73 miscG.Grgur Kovač, J.Jérémy Perez, R.Rémy Portelas, M.Masataka Sawayama, P. F.Peter Ford Dominey and P.-Y.Pierre-Yves Oudeyer. StickToYourRoleLeaderboard.July 2024HALback to textback to text
• 74 inproceedingsM.Marion Pech, M.Maxime Adolphe, P.-Y.Pierre-Yves Oudeyer and H.Hélène Sauzéon. Benefits of Individualized Computerized Cognitive Training on older adults.La Fédération pour la Recherche sur le Handicap et l’Autonomie (Fedrha)Bordeaux, FranceNovember 2024HAL
• 75 inproceedingsM.Matisse Poupard, F.Florian Larrue, M.Martin Bertrand, D.Dominique Liguoro, A.André Tricot and H.Hélène Sauzéon. Guided Drawing in Medical Education: Evaluating AR and MR Systems for Anatomy Learning.STE 2024 - 4th International Conference on Science and Technology EducationPorto, PortugalOctober 2024HALback to text

Scientific popularization

• 76 articleC.Clément Moulin-Frier, G.Gautier Hamon and P.-Y.Pierre-Yves Oudeyer. Quand l'IA explore les prémices d'une vie artificielle.Pour la science5622024HAL
• 77 bookA.Anne Nicole and M.Marylène Sbaraglia. D.Didier Roy, eds. Digital Education Manual - Cycle 1 (3-4°) - Collection Décodage 2024.January 2024HAL
• 78 bookD.Didier Roy, eds. Education numérique - Manuel Cycle 2 (7-8°) - Collection Décodage 2024.January 2024HAL

Software

• 79 softwareG.Gautier Hamon. transformerXL_PPO_JAX.July 2024 lic: MIT License.HALVCS