2.1 Summary

At the frontier between integrative and computational neuroscience, we propose to model the brain as a system of active memories in synergy and in interaction with the internal and external world and to simulate it as a whole and in situation.

In integrative and cognitive neuroscience (cf. § 3.1), on the basis of current knowledge and experimental data, we develop models of the main cerebral structures, taking a specific care of the kind of mnemonic function they implement and of their interface with other cerebral and external structures. Then, in a systemic approach, we build the main behavioral loops involving these cerebral structures, connecting a wide spectrum of actions to various kinds of sensations. We observe at the behavioral level the properties emerging from the interaction between these loops.

We claim that this approach is particularly fruitful for investigating cerebral structures like the basal ganglia and the prefrontal cortex, difficult to comprehend today because of the rich and multimodal information flows they integrate. We expect to cope with the high complexity of such systems, inspired by behavioral and developmental sciences, explaining how behavioral loops gradually incorporate in the system various kinds of information and associated mnesic representations. As a consequence, the underlying cognitive architecture, emerging from the interplay between these sensations-actions loops, results from a mnemonic synergy.

In computational neuroscience (cf. § 3.2), we concentrate on the efficiency of local mechanisms and on the effectiveness of the distributed computations at the level of the system. We also take care of the analysis of their dynamic properties, at different time scales. These fundamental properties are of high importance to allow the deployment of very large systems and their simulation in a framework of high performance computing

Running simulations at a large scale is particularly interesting to evaluate over a long period a consistent and relatively complete network of cerebral structures in realistic interaction with the external and internal world. We face this problem in the domain of autonomous robotics (cf. § 3.4) and ensure a real autonomy by the design of an artificial physiology and convenient learning protocoles.

We are convinced that this original approach also permits to revisit and enrich algorithms and methodologies in machine learning (cf. § 3.3) and in autonomous robotics (cf. § 3.4), in addition to elaborate hypotheses to be tested in neuroscience and medicine, while offering to these latter domains a new ground of experimentation similar to their daily experimental studies.

3.1 Integrative and Cognitive Neuroscience

The human brain is often considered as the most complex system dedicated to information processing. This multi-scale complexity, described from the metabolic to the network level, is particularly studied in integrative neuroscience, the goal of which is to explain how cognitive functions (ranging from sensorimotor coordination to executive functions) emerge from (are the result of the interaction of) distributed and adaptive computations of processing units, displayed along neural structures and information flows. Indeed, beyond the astounding complexity reported in physiological studies, integrative neuroscience aims at extracting, in simplifying models, regularities at various levels of description. From a mesoscopic point of view, most neuronal structures (and particularly some of primary importance like the cortex, cerebellum, striatum, hippocampus) can be described through a regular organization of information flows and homogenous learning rules, whatever the nature of the processed information. From a macroscopic point of view, the arrangement in space of neuronal structures within the cerebral architecture also obeys a functional logic, the sketch of which is captured in models describing the main information flows in the brain, the corresponding loops built in interaction with the external and internal (bodily and hormonal) world and the developmental steps leading to the acquisition of elementary sensorimotor skills up to the most complex executive functions.

In summary, integrative neuroscience builds, on an overwhelming quantity of data, a simplifying and interpretative grid suggesting homogenous local computations and a structured and logical plan for the development of cognitive functions. They arise from interactions and information exchange between neuronal structures and the external and internal world and also within the network of structures.

This domain is today very active and stimulating because it proposes, of course at the price of simplifications, global views of cerebral functioning and more local hypotheses on the role of subsets of neuronal structures in cognition. In the global approaches, the integration of data from experimental psychology and clinical studies leads to an overview of the brain as a set of interacting memories, each devoted to a specific kind of information processing 70. It results also in longstanding and very ambitious studies for the design of cognitive architectures aiming at embracing the whole cognition. With the notable exception of works initiated by 64, most of these frameworks (e.g. Soar, ACT-R), though sometimes justified on biological grounds, do not go up to a connectionist neuronal implementation. Furthermore, because of the complexity of the resulting frameworks, they are restricted to simple symbolic interfaces with the internal and external world and to (relatively) small-sized internal structures. Our main research objective is undoubtly to build such a general purpose cognitive architecture (to model the brain as a whole in a systemic way), using a connectionist implementation and able to cope with a realistic environment.

3.2 Computational Neuroscience

From a general point of view, computational neuroscience can be defined as the development of methods from computer science and applied mathematics, to explore more technically and theoretically the relations between structures and functions in the brain 73, 63. During the recent years this domain has gained an increasing interest in neuroscience and has become an essential tool for scientific developments in most fields in neuroscience, from the molecule to the system. In this view, all the objectives of our team can be described as possible progresses in computational neuroscience. Accordingly, it can be underlined that the systemic view that we promote can offer original contributions in the sense that, whereas most classical models in computational neuroscience focus on the better understanding of the structure/function relationship for isolated specific structures, we aim at exploring synergies between structures. Consequently, we target interfaces and interplay between heterogenous modes of computing, which is rarely addressed in classical computational neuroscience.

We also insist on another aspect of computational neuroscience which is, in our opinion, at the core of the involvement of computer scientists and mathematicians in the domain and on which we think we could particularly contribute. Indeed, we think that our primary abilities in numerical sciences imply that our developments are characterized above all by the effectiveness of the corresponding computations: we provide biologically inspired architectures with effective computational properties, such as robustness to noise, self-organization, on-line learning. We more generally underline the requirement that our models must also mimick biology through its most general law of homeostasis and self-adaptability in an unknown and changing environment. This means that we propose to numerically experiment such models and thus provide effective methods to falsify them.

Here, computational neuroscience means mimicking original computations made by the neuronal substratum and mastering their corresponding properties: computations are distributed and adaptive; they are performed without an homonculus or any central clock. Numerical schemes developed for distributed dynamical systems and algorithms elaborated for distributed computations are of central interest here 58, 59 and were the basis for several contributions in our group 72, 67, 74. Ensuring such a rigor in the computations associated to our systemic and large scale approach is of central importance.

Equally important is the choice for the formalism of computation, extensively discussed in the connectionist domain. Spiking neurons are today widely recognized of central interest to study synchronization mechanisms and neuronal coupling at the microscopic level 65; the associated formalism 75 can be possibly considered for local studies or for relating our results with this important domain in connectionism. Nevertheless, we remain mainly at the mesoscopic level of modeling, the level of the neuronal population, and consequently interested in the formalism developed for dynamic neural fields 68, that demonstrated a richness of behavior 69 adapted to the kind of phenomena we wish to manipulate at this level of description. Our group has a long experience in the study and adaptation of the properties of neural fields 67, 66 and their use for observing the emergence of typical cortical properties 61. In the envisioned development of more complex architectures and interplay between structures, the exploration of mathematical properties such as stability and boundedness and the observation of emerging phenomena is one important objective. This objective is also associated with that of capitalizing our experience and promoting good practices in our software production.

In summary, we think that this systemic approach also brings to computational neuroscience new case studies where heterogenous and adaptive models with various time scales and parameters have to be considered jointly to obtain a mastered substratum of computation. This is particularly critical for large scale deployments.

3.3 Machine Learning

The adaptive properties of the nervous system are certainly among its most fascinating characteristics, with a high impact on our cognitive functions. Accordingly, machine learning is a domain 62 that aims at giving such characteristics to artificial systems, using a mathematical framework (probabilities, statistics, data analysis, etc.). Some of its most famous algorithms are directly inspired from neuroscience, at different levels. Connectionist learning algorithms implement, in various neuronal architectures, weight update rules, generally derived from the hebbian rule, performing non supervised (e.g. Kohonen self-organizing maps), supervised (e.g. layered perceptrons) or associative (e.g. Hopfield recurrent network) learning. Other algorithms, not necessarily connectionist, perform other kinds of learning, like reinforcement learning. Machine learning is a very mature domain today and all these algorithms have been extensively studied, at both the theoretical and practical levels, with much success. They have also been related to many functions (in the living and artificial domains) like discrimination, categorisation, sensorimotor coordination, planning, etc. and several neuronal structures have been proposed as the substratum for these kinds of learning 60, 56. Nevertheless, we believe that, as for previous models, machine learning algorithms remain isolated tools, whereas our systemic approach can bring original views on these problems.

At the cognitive level, most of the problems we face do not rely on only one kind of learning and require instead skills that have to be learned in preliminary steps. That is the reason why cognitive architectures are often referred to as systems of memory, communicating and sharing information for problem solving. Instead of the classical view in machine learning of a flat architecture, a more complex network of modules must be considered here, as it is the case in the domain of deep learning. In addition, our systemic approach brings the question of incrementally building such a system, with a clear inspiration from developmental sciences. In this perspective, modules can generate internal signals corresponding to internal goals, predictions, error signals, able to supervise the learning of other modules (possibly endowed with a different learning rule), supposed to become autonomous after an instructing period. A typical example is that of episodic learning (in the hippocampus), storing declarative memory about a collection of past episods and supervising the training of a procedural memory in the cortex.

At the behavioral level, as mentionned above, our systemic approach underlines the fundamental links between the adaptive system and the internal and external world. The internal world includes proprioception and interoception, giving information about the body and its needs for integrity and other fundamental programs. The external world includes physical laws that have to be learned and possibly intelligent agents for more complex interactions. Both involve sensors and actuators that are the interfaces with these worlds and close the loops. Within this rich picture, machine learning generally selects one situation that defines useful sensors and actuators and a corpus with properly segmented data and time, and builds a specific architecture and its corresponding criteria to be satisfied. In our approach however, the first question to be raised is to discover what is the goal, where attention must be focused on and which previous skills must be exploited, with the help of a dynamic architecture and possibly other partners. In this domain, the behavioral and the developmental sciences, observing how and along which stages an agent learns, are of great help to bring some structure to this high dimensional problem.

At the implementation level, this analysis opens many fundamental challenges, hardly considered in machine learning : stability must be preserved despite on-line continuous learning; criteria to be satisfied often refer to behavioral and global measurements but they must be translated to control the local circuit level; in an incremental or developmental approach, how will the development of new functions preserve the integrity and stability of others? In addition, this continous re-arrangement is supposed to involve several kinds of learning, at different time scales (from msec to years in humans) and to interfer with other phenomena like variability and meta-plasticity.

In summary, our main objective in machine learning is to propose on-line learning systems, where several modes of learning have to collaborate and where the protocoles of training are realistic. We promote here a really autonomous learning, where the agent must select by itself internal resources (and build them if not available) to evolve at the best in an unknown world, without the help of any deus-ex-machina to define parameters, build corpus and define training sessions, as it is generally the case in machine learning. To that end, autonomous robotics (cf. § 3.4) is a perfect testbed.

3.4 Autonomous Robotics

Autonomous robots are not only convenient platforms to implement our algorithms; the choice of such platforms is also motivated by theories in cognitive science and neuroscience indicating that cognition emerges from interactions of the body in direct loops with the world (embodiment of cognition57). In addition to real robotic platforms, software implementations of autonomous robotic systems including components dedicated to their body and their environment will be also possibly exploited, considering that they are also a tool for studying conditions for a real autonomous learning.

A real autonomy can be obtained only if the robot is able to define its goal by itself, without the specification of any high level and abstract cost function or rewarding state. To ensure such a capability, we propose to endow the robot with an artificial physiology, corresponding to perceive some kind of pain and pleasure. It may consequently discriminate internal and external goals (or situations to be avoided). This will mimick circuits related to fundamental needs (e.g. hunger and thirst) and to the preservation of bodily integrity. An important objective is to show that more abstract planning capabilities can arise from these basic goals.

A real autonomy with an on-line continuous learning as described in  § 3.3 will be made possible by the elaboration of protocols of learning, as it is the case, in animal conditioning, for experimental studies where performance on a task can be obtained only after a shaping in increasingly complex tasks. Similarly, developmental sciences can teach us about the ordered elaboration of skills and their association in more complex schemes. An important challenge here is to translate these hints at the level of the cerebral architecture.

As a whole, autonomous robotics permits to assess the consistency of our models in realistic condition of use and offers to our colleagues in behavioral sciences an object of study and comparison, regarding behavioral dynamics emerging from interactions with the environment, also observable at the neuronal level.

In summary, our main contribution in autonomous robotics is to make autonomy possible, by various means corresponding to endow robots with an artificial physiology, to give instructions in a natural and incremental way and to prioritize the synergy between reactive and robust schemes over complex planning structures.

4.1 Overview

Modeling the brain to emulate cognitive functions offers direct and indirect application domains. Our models are designed to be confronted to the reality of life sciences and to make predictions in neuroscience and in the medical domain. Our models also have an impact in digital sciences; their performances can be questioned in informatics, their algorithms can be compared with models in machine learning and artificial intelligence, their behavior can be analysed in human-robot interaction. But since what they produce is related to human thinking and behavior, applications will be also possible in various domains of social sciences and humanities.

4.2 Applications in life sciences

One of the most original specificity of our team is that it is part of a laboratory in Neuroscience (with a large spectrum of activity from the molecule to the behavior), focused on neurodegenerative diseases and consequently working in tight collaboration with the medical domain. Beyond data and signal analysis where our expertise in machine learning may be possibly useful, our interactions are mainly centered on the exploitation of our models. They will be classically regarded as a way to validate biological assumptions and to generate new hypotheses to be investigated in the living. Our macroscopic models and their implementation in autonomous robots will allow an analysis at the behavioral level and will propose a systemic framework, the interpretation of which will meet aetiological analysis in the medical domain and interpretation of intelligent behavior in cognitive neuroscience and related domains like for example educational science.

The study of neurodegenerative diseases is targeted because they match the phenomena we model. Particularly, the Parkinson disease results from the death of dopaminergic cells in the basal ganglia, one of the main systems that we are modeling. The Alzheimer disease also results from the loss of neurons, in several cortical and extracortical regions. The variety of these regions, together with large mnesic and cognitive deficits, require a systemic view of the cerebral architecture and associated functions, very consistent with our approach.

4.3 Application in digital sciences

Of course, digital sciences are also impacted by our researches, at several levels. At a global level, we will propose new control architectures aimed at providing a higher degree of autonomy to robots, as well as machine learning algorithms working in more realistic environment. More specifically, our focus on some cognitive functions in closed loop with a real environment will address currently open problems. This is obviously the case for planning and decision making; this is particularly the case for the domain of affective computing, since motivational characteristics arising from the design of an artificial physiology allow to consider not only cold rational cognition but also hot emotional cognition. The association of both kinds of cognition is undoubtly an innovative way to create more realistic intelligent systems but also to elaborate more natural interfaces between these systems and human users.

At last, we think that our activities in well-founded distributed computations and high performance computing are not just intended to help us design large scale systems. We also think that we are working here at the core of informatics and, accordingly, that we could transfer some fundamental results in this domain.

4.4 Applications in human sciences

Because we model specific aspects of cognition such as learning, language and decision, our models could be directly analysed from the perspective of educational sciences, linguistics and economy. Because we manipulate such concepts as embodiment of cognition, theory of mind and emotions, we could also participate in debates in philosophy and ethics.

Futhermore, our implication in science outreach actions, including computer science teaching in secondary and primary school, with the will to analyse and evaluate the outcomes of these actions, is at the origin of building a link between our research in computational learning and human learning, providing not only tools but also new modeling paradigms.

5.1 Footprint of research activities

As part of the Institute of Neurodegenerative Diseases that developed a strong commitment to the environment, we take our share in the reduction of our carbon footprint by deciding to reduce our commuting footprint and the number of yearly travels to conference. In 2020, this collided with the sanitary crisis that de facto reduced our travel capacity. But beyond the crisis, we aim at enforcing this policy.

5.2 Impact of research results

We're engaged in the EcoMob regional project in collaboration with the University of Bordeaux and the University of La Rochelle to study and model the behavior of individuals during their daily trips to and from work places. In this context and based on our previous work on decision making, our team is interested in elucidating how habits are formed and more importantly, how can they be changed. This is especially important in order to convince people to change their daily habits and consider alternative when they move from point A to point B.

7.1 New software

8.1 Overview

This year we have addressed several important questions. We have studied and modeled higher cognitive functions (cf. § 8.2) and the brain circuitry that emulate them. We have also studied how they are made possible, by interaction with the environment (cf. § 8.3), embodiment in a robotic platform and temporal organization of the behavior. We have also been active in Machine Learning (cf. § 8.4), and have studied relations of our work to classical Artificial Intelligence and to the symbolic and numerical aspects of cognition (cf. § 8.5), raising the question of using knowledge, ontologies and aspects of logics.

8.2 Modeling higher cognitive functions

This year, we have studied how the contribution of working memory and episodic memory can contribute to going beyond reactive immediate behavior, toward cognitive control. We have also proposed that these processes can play a central role in creativity 14.

As part of the Philosophical Transactions of the Royal Society B themed issue on the Existence and prevalence of economic behaviours among non-human primates, we published a collaborative work 7 on the evolutionary orign of the prospect theory. In humans, the attitude toward risk is not neutral and is dissimilar between bets involving gains and bets involving losses. The existence and prevalence of these decision features in non-human primates are unclear. In addition, only a few studies have tried to simulate the evolution of agents based on their attitude toward risk. Therefore, we still ignore to what extent Prospect theory’s claims are evolutionarily rooted. To shed light on this issue, we collected data from nine macaques that performed bets involving gains or losses. We confirmed that their overall behaviour is coherent with Prospect theory’s claims. In parallel, we used a genetic algorithm to simulate the evolution of a population of agents across several generations. We showed that the algorithm selects progressively agents that exhibit risk-seeking, and has an inverted S-shape distorted perception of probability. We compared these two results and found that monkeys’ attitude toward risk is only congruent with the simulation when they are facing losses. This result is consistent with the idea that gambling in the loss domain is analogous to deciding in a context of life-threatening challenges where a certain level of risk-seeking behaviour and probability distortion may be adaptive.

Gated working memory is defined as the capacity of holding arbitrary information at any time in order to be used at a later time. Based on electrophysiological recordings, several computational models have tackled the problem using dedicated and explicit mechanisms. We have proposed instead to consider an implicit mechanism based on a random recurrent neural network 71. We have previously introduced a robust yet simple reservoir model of gated working memory with instantaneous updates. The model is able to store an arbitrary real value at random time over an extended period of time. We rencently introduced a method based on conceptors 12 that allows us to manipulate information stored in the dynamics (latent space) of this previously developped gated working memory model. The memorized information results in complex dynamics inside the reservoir that can be faithfully captured by a conceptor. Such conceptors allow us to explicitly manipulate this information in order to perform various, but not arbitrary, operations. In this work 29, we show how working memory can be stabilized or discretized using such conceptors, how such conceptors can be linearly combined to form new memories, and how these conceptors can be extended to a functional role. These preliminary results suggest that conceptors can be used to manipulate the latent space of the working memory even though several results we introduce are not as intuitive as one would expect.

8.3 Interactions with the environment

Understanding the mechanisms enabling children to learn rapidly word-to-meaning mapping through cross-situational learning in uncertain conditions is still a matter of debate. In particular, many models simply look at the word level, and not at the full sentence comprehension level. We present a model of language acquisition, applying cross-situational learning on Recurrent Neural Networks with the Reservoir Computing paradigm 16. Using the co-occurrences between words and visual perceptions, the model learns to ground a complex sentence, describing a scene involving different objects, into a perceptual representation space. The model processes sentences describing scenes it perceives simultaneously via a simulated vision module: sentences are inputs and simulated vision are target outputs of the RNN. Evaluations of the model show its capacity to extract the semantics of virtually hundred of thousands possible combinations of sentences (based on a context-free grammar); remarkably the model generalises only after a few hundred of partially described scenes via cross-situational learning. Furthermore, it handles polysemous and synonymous words, and deals with complex sentences where word order is crucial for understanding. Finally, further improvements of the model are discussed in order to reach proper reinforced and self-supervised learning schemes, with the goal to enable robots to acquire and ground language by themselves (with no oracle supervision). We futher investigated the developmental plausibility of such model in 15: (i) if it can learn to generalize from single-object sentence to double-object sentence; (ii) if it can use more plausible representations: (ii.a) inputs as sequence of phonemes (instead of words) and (ii.b) outputs fully independent from sentence structure (in order to enable purely unsupervised cross-situational learning). Interestingly, tasks (i) and (ii.a) are solved in a straightforward fashion, whereas task (ii.b) suggests that learning with tensor representations is a more difficult task.

More studies on the modelling of language acquisition and language processing are described in section 8.4 "Machine Learning" because of their potential impact in this field. S. Oota started his PhD in November 2020 in the team: he will continue these studies on language.

More generally, sensorimotor learning (e.g. language learning) represents a challenging problem for artificial and natural systems. Several computational models try to explain the neural mechanisms at play in the brain to implement such learning. These models have several common components: a motor control model, a sensory system and a learning architecture. In order to make complete sensorimotor models, we also investigate at "simpler" learning problems such as song developmental learning by birds. In S. Pagliarini’s PhD, our challenge is to build a biologically plausible model for song learning in birds including neuro-anatomical and developmental constraints. We made a major comparative review of existing vocal sensorimotor models in order to give an overall view useful to this multidisciplinary community 8. We grouped the methodologies in order to clearly identify the learning algorithms and representations used for the sensory, perceptual and motor spaces. We expect this review to be beneficial to the community by facilitating the comparison of newly proposed models with already existing ones.

This year, a particular fruitful collaboration between S. Pagliarini and N. Trouvain (M2 intern during the first half of 2020) was made. Indeed, N. Trouvain developped a "canary decoder" which makes the transcription of canary songs based on the sounds recordings from Orsay (see below). S. Pagliarini has trained a low-dimensional Generative Adversarial Network (GAN) with canary sounds: this GAN is able to reproduce faithfully canary syllables, which are most of the time indistinguishable from real canary sounds. The derived version of N. Trouvain's decoder enables to make advanced quantitative and qualitative analyses of sounds generated by the GAN, which would have been more limited otherwise.

On this songbird topic, X. Hinaut is also collaborating with Catherine del Negro’s team (CNRS, NeuroPSI, Orsay) on the representation of syntax in songbird brains. In particular, the project aims at (1) linking the neural activity of a sensorimotor area (HVC) to syntax elements in the songs of domestic canaries ; (2) analysing the audio files and transcripts of canary songs in order to find syntax cues and higher order representations (graph properties of songs, evaluate Markovian forward and backward transition probabilities of various orders). The newly developped canary decoder by N. Trouvain will enable to transcribe more raw data of canary songs already available (indeed this process was time consuming previously because manually made). Additionally, N. Trouvain developped a graphical user interface in order to simplify the cleaning and class merging operations on manually transcribed data. Indeed, prior human classification is still needed to create the dictionary of syllables which is specific to each canary. This new tool will speed-up our collaborative studies with Orsay on syntax analysis while improving the quality and statistical significance of the results.

8.4 Machine learning

The basic problem in Parkinson’s disease is loss of dopamine-producing nerve cells in a region of the brain called the substantia nigra pars compacta. Everybody has a gradual loss of these dopamine-producing nerve cells as they age, but patients with Parkinson’s disease have lost more of them than other people. Why these cells die in Parkinson’s disease is unclear, and the focus of much research. To answer this question, research requires the use of a variety of animal models to study different aspects of the disease. With colleagues, we published an article in Science Advances 5 showing that dopaminergic neurodegeneration can be induced in non-human primates by both, small and large aggregates of alpha-synuclein. In contrast, experiments in rodents, used in 85% of studies, show that small alpha-synuclein aggregates do not induce neurodegeneration.

In 6, we provide theoretical conditions guaranteeing that a self-organizing map efficiently develops representations of the input space. The study relies on a neural field model of spatiotemporal activity in area 3b of the primary somatosensory cortex. We rely on Lyapunov’s theory for neural fields to derive theoretical conditions for stability. We verify the theoretical conditions by numerical experiments. The analysis highlights the key role played by the balance between excitation and inhibition of lateral synaptic coupling and the strength of synaptic gains in the formation and maintenance of self-organizing maps.

We extended this work in 11 where we propose a variation of the self organizing map algorithm by considering the random placement of neurons on a two-dimensional manifold, following a blue noise distribution from which various topologies can be derived. These topologies possess random (but controlable) discontinuities that allow for a more flexible self-organization, especially with high-dimensional data. The proposed algorithm has been tested on one-, two- and three-dimensions tasks as well as on the MNIST handwritten digits dataset and validated using spectral analysis and topological data analysis tools. We also demonstrate the ability of the randomized self-organizing map to gracefully reorganize itself in case of neural lesion and/or neurogenesis

We propose a novel architecture 18, 36 called Hierarchical-Task Reservoir (HTR) suitable for real-time applications for which different levels of abstraction are available. We apply it to semantic role labelling based on continuous speech recognition. Taking inspiration from the brain, that demonstrates hierarchies of representations from perceptive to integrative areas, we consider a hierarchy of four sub-tasks with increasing levels of abstraction (phoneme, word, part-of-speech and semantic role tags). These tasks are progressively learned by the layers of the HTR architecture. Interestingly, quantitative and qualitative results show that the hierarchical-task approach provides an advantage to improve the prediction. In particular, the qualitative results show that a shallow or a hierarchical reservoir considered as baselines do not produce a quality of estimation as the HTR model. Moreover, we show that it is possible to further improve the accuracy of the model by designing skip connections and by considering word embedding in the internal representations. Overall, the HTR outperformed the other state-of-the-art reservoir-based approaches. The HTR architecture is proposed as a step toward the modeling of online and hierarchical processes at work in the brain during language comprehension. It is also developed for real-time and efficient Human-Robot Interaction (HRI) for which the availability of different levels of abstraction would provide more robustness.

Echo States Networks (ESN) and Long-Short Term Memory networks (LSTM) are two popular architectures of Recurrent Neural Networks (RNN) to solve machine learning task involving sequential data. However, little has been done to compare their performances and their internal mechanisms on a common task. We trained ESNs and LSTMs on a Cross-Situationnal Learning (CSL) task 39. This task aims at modelling how infants learn language: they create associations between words and visual stimuli in order to extract meaning from words and sentences. The results are of three kinds: performance comparison, internal dynamics analyses and visualization of latent space. (1) We found that both models were able to successfully learn the task: the LSTM reached the lowest error for the basic corpus, but the ESN was quicker to train. Furthermore, the ESN was able to outperform LSTMs on datasets more challenging without any further tuning needed. (2) We also conducted an analysis of the internal units activations of LSTMs and ESNs. Despite the deep differences between both models (trained or fixed internal weights), we were able to uncover similar inner mechanisms: both put emphasis on the units encoding aspects of the sentence structure. (3) Moreover, we present Recurrent States Space Visualisations (RSSviz), a method to visualize the structure of latent state space of RNNs, based on dimension reduction (using UMAP). This technique enables us to observe a fractal embedding of sequences in the LSTM. RSSviz is also useful for the analysis of ESNs (i) to spot difficult examples and (ii) to generate animated plots showing the evolution of activations across learning stages. Finally, we explore qualitatively how the RSSviz could provide an intuitive visualisation to understand the influence of hyperparameters on the reservoir dynamics prior to ESN training.

In 19 we present a simple user-friendly library called ReservoirPy based on Python scientific modules. It provides a flexible interface to implement efficient Reservoir Computing (RC) architectures with a particular focus on Echo State Networks (ESN). Advanced features of ReservoirPy allow to improve up to 87.9 percent of computation time efficiency on a simple laptop compared to basic Python implementation. Overall, we provide tutorials for hyperparameters tuning, offline and online training, fast spectral initialization, parallel and sparse matrix computation on various tasks (MackeyGlass and audio recognition tasks). In particular, we provide graphical tools to easily explore hyperparameters using random search with the help of the hyperopt library.

Several people participated in the development of the ReservoirPy library, among which N. Trouvain during his internship. An Inria ADT grant ("Aide au Dévelopment Technologique") was obtained to recruit an engineer for 20 months to pursue the development of ReservoirPy. Thanks to this grant, N. Trouvain is continuing this development in the team.

8.5 Symbolic and numerical cognition

This year, we have contributed several book chapters 23, 24, 25, dealing about interactions between neuroscience and artificial intelligence. In order to explore new ways of building direct and inverse models of sensorimotor transition functions, we have studied, in our collaboration with CEA (cf. § 9.1.1), an original way to learn both at the same time 17.

The on-going work on an original prototype based approach of deep-learning considering standard learning and meta-learning paradigms, on not so big data sets, including a fine study on metaparameter adjustment in this context, has been finalized 28, offering an important and deeply analyzed bibliography on recurrent deep learning networks and not so big data set management. The capability to easily explain the "how it works" mechanism to non specialist of the field is an important outcome of this work.

Co-leaded by Margarida Romero scientific director of the LINE laboratory of the UCA and researchers of our team, an Inria AEx action has been launched, regarding artificial intelligence devoted to education (AIDE) (cf. § 10.3.4). We want to explore to what extent approaches or methods from cognitive neuroscience, linked to machine learning and knowledge representation, could help to better formalize human learning as studied in educational sciences. In other words: we are taking advantage of our better understanding of how our brains work to help us better understand how our children learn. The focus here is on learning computational thinking, i.e., what to share in terms of the skills needed to master the digital world, not just consume or endure it, considering specific learning tasks modeling. Preliminary results include positioning papers 13 and multi-disciplinary scientific preliminary outcomes 14, continuing contribution to educational science in order to raise our level of competence in this companion field 26, 22. We also have started experimental setup realization 33, 32 and started the main work on modeling the brain of the learner and the context of the learning activity 38, 44, including large audience publication 54, 55, 50.

While language and birdsongs both involve symbols and sub-symbolic representations, related studies of the team are reported in section 8.3 "Interaction with the environment" because of their links with sensorimotor learning during the processing and production of symbols.

9.1 Bilateral contracts with industry

10.1 International initiatives

10.2 European initiatives

10.3 National initiatives

10.4 Regional initiatives

11.1 Promoting scientific activities

11.2 Teaching - Supervision - Juries

11.3 Popularization

F. Alexandre gave a conference for TeDx Bordeaux about decision making https://www.tedxbordeaux.com/edition-2019/tedxbordeauxsalon-3-je-decide-donc-je-suis/ and gave a conference about AI to the high school of management of Bordeaux, ISG. Other large live conferences were canceled. Two articles by F. Alexandre in the blog Binaire of the French newspaper Le Monde: https://www.lemonde.fr/blog/binaire/2020/01/16/de-quelles-facons-lintelligence-artificielle-se-sert-elle-des-neurosciences/ and https://www.lemonde.fr/blog/binaire/2020/04/26/les-modeles-mathematiques-miracle-ou-supercherie/, the latter one being about the covid pandemics 46. One article in Interstices https://interstices.info/les-relations-difficiles-entre-lintelligence-artificielle-et-lesneurosciences/ about AI and neuroscience 47;

Thierry Viéville gave several large audience online conferences about Artificial Intelligence popularization, within the scope of https://classcode.fr/iai, and participated in formation sessions regarding computer science teaching and continue contributing to computer science teaching in secondary school https://hal.inria.fr/hal-02445562.

The team has been involved and has succeeded in producing the https://neurosmart.inria.fr science outreach resource allowing everyone to better understand the functional aspects of the brain processing 31, with a good dissemination, e.g. http://www.scilogs.fr/intelligence-mecanique/neurosmart-une-histoire-de-cerveau-et-de-passionnes/, https://www.lemonde.fr/blog/binaire/2020/11/27/neurosmart-une-histoire-de-cerveau-et-de-passionnes/ and https://www.fondation-blaise-pascal.org/neurosmart-le-logiciel-pour-comprendre-le-cerveau/.

Thierry Viéville and other colleagues have published several large audience papers regarding our own research 54, 55, 50, which is very important due top the societal impact of our work, and also beyond 51, 49.

Nicolas Rougier co-created the Hypermondes association whose goal is to support a yearly festival around science and fiction open to the general public. The annual budget of the festival is around 75,000 euros. First edition will be held in September 2021 (initially date was June 2020).

12.1 Major publications

• 1 articleMathieuM. Bourdenx, AurélienA. Nioche, SandraS. Dovero, Marie-LaureM.-L. Arotcarena, Sandrine M. J.S. Camus, GregoryG. Porras, Marie-LaureM.-L. Thiolat, Nicolas P.N. Rougier, AliceA. Prigent, PhilippeP. Aubert, SylvainS. Bohic, ChristopheC. Sandt, FlorentF. Laferrière, EvelyneE. Doudnikoff, NielsN. Kruse, BritB. Mollenhauer, SalvatoreS. Novello, MicheleM. Morari, ThierryT. Leste-Lasserre, InesI. Trigo Damas, MichelM. Goillandeau, CelineC. Perier, CristinaC. Estrada, NuriaN. García Carrillo, AriadnaA. Recasens, Nishant NarayananN. Vaikath, OmarO. El Agnaf, Maria TrinidadM. Herrero, PascalP. Derkinderen, MiquelM. Vila, Jose AJ. Obeso, BenjaminB. Dehay and ErwanE. Bezard. Identification of distinct pathological signatures induced by patient-derived $$$$ -synuclein structures in nonhuman primates'Science Advances 620May 2020, eaaz9165
  HALDOI
• 2 article AurélienA. Nioche, Nicolas P.N. Rougier, MarcM. Deffains, SachaS. Bourgeois-Gironde, SébastienS. Ballesta and ThomasT. Boraud. The adaptive value of probability distortion and risk-seeking in macaques' decision-making Philosophical Transactions of the Royal Society B: Biological Sciences January 2021
  HAL DOI
• 3 article SilviaS. Pagliarini, ArthurA. Leblois and XavierX. Hinaut. Vocal Imitation in Sensorimotor Learning Models: a Comparative Review IEEE Transactions on Cognitive and Developmental Systems November 2020
  HAL DOI

12.2 Publications of the year

12.3 Other

12.4 Cited publications

• 56 inproceedings FrédéricF. Alexandre. Biological Inspiration for Multiple Memories Implementation and Cooperation International Conference on Computational Intelligence 2000
  back to text
• 57 articleDana H.D. Ballard, Mary M.M. Hayhoe, Polly K.P. Pook and Rajesh P. N.R. Rao. Deictic codes for the embodiment of cognitionBehavioral and Brain Sciences20041997, 723--742URL: http://dx.doi.org/10.1017/S0140525X97001611
  back to text
• 58 book D.P.~Bertsekas and J.N.~Tsitsiklis. Parallel and Distributed Computation: Numerical Methods Athena Scientific 1997
  back to text
• 59 articleD.~Mitra. Asynchronous relaxations for the numerical solution of differential equations by parallel processorsSIAM J. Sci. Stat. Comput.811987, 43--58
  back to text
• 60 articleKenjiK. Doya. What are the computations of the cerebellum, the basal ganglia and the cerebral cortex?Neural Networks121999, 961--974
  back to text
• 61 articleJérémyJ. Fix, Nicolas P.N. Rougier and FrédéricF. Alexandre. A dynamic neural field approach to the covert and overt deployment of spatial attentionCognitive Computation312011, 279-293URL: http://hal.inria.fr/inria-00536374/en
  DOIback to text
• 62 book TomT. Mitchell. Machine Learning Mac Graw-Hill Press 1997
  back to text
• 63 book P.~Dayan and L.F.~Abbott. Theoretical Neuroscience : Computational and Mathematical Modeling of Neural Systems MIT Press 2001
  back to text
• 64 book R.C.~O'Reilly and Y.~Munakata. Computational Explorations in Cognitive Neuroscience: Understanding the Mind by Simulating the Brain Cambridge, MA, USA MIT Press 2000
  back to text
• 65 articleR.~Brette, M.~Rudolph, T.~Carnevale, M.~Hines, D.~Beeman, J.M.~J. Bower, M.~Diesmann, A. Morrison, P. H.P. Goodman, F. C. Jr.F. Harris, M. Zirpe, T.~Natschläger, D.~Pecevski, B.~Ermentrout, M.~Djurfeldt, A.~Lansner, O.~Rochel, T.~Viéville, E.~Muller, A.P.~Davison, S.~ElS. Boustani and A.~Destexhe. Simulation of networks of spiking neurons: a review of tools and strategiesJournal of Computational Neuroscience2332007, 349--398
  back to text
• 66 articleNicolas P.N. Rougier. Dynamic Neural Field with Local InhibitionBiological Cybernetics9432006, 169-179
  back to text
• 67 article Nicolas P.N. Rougier and AxelA. Hutt. Synchronous and Asynchronous Evaluation of Dynamic Neural Fields J. Diff. Eq. Appl. 2009
  back to text back to text
• 68 articleS.~Amari. Dynamic of pattern formation in lateral-inhibition type neural fieldsBiological Cybernetics271977, 77--88
  back to text
• 69 articleS.~Coombes. Waves, bumps and patterns in neural field theoriesBiol. Cybern.932005, 91-108
  back to text
• 70 articleL.R.L. Squire. Memory systems of the brain: a brief history and current perspectiveNeurobiol. Learn. Mem.822004, 171-177
  back to text
• 71 articleAnthonyA. Strock, XavierX. Hinaut and Nicolas P.N. Rougier. A Robust Model of Gated Working MemoryNeural Computation321January 2020, 153--181URL: https://doi.org/10.1162/neco_a_01249
  DOIback to text
• 72 articleWahibaW. Taouali, ThierryT. Viéville, Nicolas P.N. Rougier and FrédéricF. Alexandre. No clock to rule them allJournal of Physiology1051-32011, 83-90
  back to text
• 73 book T.P.T. Trappenberg. Fundamentals of Computational Neuroscience Oxford University Press 2002
  back to text
• 74 articleThierryT. Viéville. An unbiased implementation of regularization mechanismsImage and Vision Computing23112005, 981--998URL: http://authors.elsevier.com/sd/article/S0262885605000909
  back to text
• 75 book W.~Gerstner and W.M.~Kistler. Spiking Neuron Models: Single Neurons, Populations, Plasticity Cambridge University Press Cambridge University Press 2002
  back to text