2025Activity reportProject-Team‌RANDOPT

2.1​ Scientific Context

Critical problems​‌ of the 21st century​​ like the search for​​​‌ highly energy efficient or​ even carbon-neutral, and cost-efficient​‌ systems, or the design​​ of new molecules against​​​‌ extensively drug-resistant bacteria crucially​ rely on the resolution​‌ of challenging numerical optimization​​ problems. Such problems typically​​​‌ depend on noisy experimental​ data or involve complex​‌ numerical simulations where derivatives​​ are not useful or​​​‌ not available and the​ function is considered as​‌ a black-box.

Many​​ of those optimization problems​​​‌ are in essence multiobjective​—one needs to optimize​‌ simultaneously several conflicting objectives​​ like minimizing the cost​​​‌ of an energy network​ and maximizing its reliability—and​‌ most of the challenging​​ black-box problems are non-convex​​​‌ and non-smooth and they​ combine difficulties related to​‌ ill-conditioning, non-separability, and ruggedness​​ (a term that characterizes​​​‌ functions that can be​ non-smooth but also noisy​‌ or multi-modal). Additionally, the​​ objective function can be​​​‌ expensive to evaluate, that​ is, one function evaluation​‌ can take several minutes​​ to hours (it can​​​‌ involve for instance a​ CFD simulation).

In this​‌ context, the use of​​ randomness combined with proper​​​‌ adaptive mechanisms that notably​ satisfy certain invariance properties​‌ (affine invariance, invariance to​​ monotonic transformations) has proven​​​‌ to be one key​ component for the design​‌ of robust global numerical​​ optimization algorithms 52,​​​‌ 37.

The field​ of adaptive stochastic optimization​‌ algorithms has witnessed some​​ important progress over the​​​‌ past 15 years. On​ the one hand, subdomains​‌ like medium-scale unconstrained optimization​​ may be considered as​​​‌ “solved” (particularly, the CMA-ES​ algorithm, an instance of​‌ Evolution Strategy (ES) algorithms,​​ stands out as state-of-the-art​​​‌ method) and considerably better​ standards have been established​‌ in the way benchmarking​​ and experimentation are performed.​​​‌ On the other hand,​ multiobjective population-based stochastic algorithms​‌ became the method of​​ choice to address multiobjective​​​‌ problems when a set​ of some best possible​‌ compromises is sought after.​​ In all cases, the​​ resulting algorithms have been​​​‌ naturally transferred to industry‌ (the CMA-ES algorithm is‌​‌ now regularly used in​​ companies such as Bosch,​​​‌ Total, ALSTOM, ...) or‌ to other academic domains‌​‌ where difficult problems need​​ to be solved such​​​‌ as physics, biology 56‌, geoscience 45,‌​‌ or robotics 48).​​

ES algorithms also attracted​​​‌ quite some attention in‌ Machine Learning with the‌​‌ OpenAI article Evolution Strategies​​ as a Scalable Alternative​​​‌ to Reinforcement Learning.‌ It is shown that‌​‌ the training time for​​ difficult reinforcement learning benchmarks​​​‌ could be reduced from‌ 1 day (with standard‌​‌ RL approaches) to 1​​ hour using ES 54​​​‌.1 Already ten‌ years ago, another impressive‌​‌ application of CMA-ES, how​​ “Computer Sim Teaches Itself​​​‌ To Walk Upright” (published‌ at the conference SIGGRAPH‌​‌ Asia 2013) was presented​​ in the press in​​​‌ the UK.

Several‌ of these important advances‌​‌ around adaptive stochastic optimization​​ algorithms rely to a​​​‌ great extent on works‌ initiated or achieved by‌​‌ the founding members of​​ RandOpt, particularly related to​​​‌ the CMA-ES algorithm and‌ to the Comparing Continuous‌​‌ Optimizer (COCO) benchmarking platform.​​

Yet, the field of​​​‌ adaptive stochastic algorithms for‌ black-box optimization is relatively‌​‌ young compared to the​​ “classical optimization” field that​​​‌ includes convex and gradient-based‌ optimization. For instance, the‌​‌ state-of-the art algorithms for​​ unconstrained gradient based optimization​​​‌ like quasi-Newton methods (e.g.‌ the BFGS method) date‌​‌ from the 1970s 36​​ while the stochastic derivative-free​​​‌ counterpart, CMA-ES dates from‌ the early 2000s 38‌​‌. Consequently, in some​​ subdomains with important practical​​​‌ demands, not even‌ the most fundamental and‌​‌ basic questions are answered:​​

• This is the case​​​‌ of constrained optimization where‌ one needs to find‌​‌ a solution $x^{*}\in {ℝ}^n$ minimizing​​​‌ a numerical function ${min‌}_{x\in {ℝ}^{\mathrm{n‌‌}}}f(x)$ while respecting a number​​​‌ of constraints $m$ typically‌ formulated as $g_{\mathrm{i‌‌}}(x^{*})\le 0$ for $\mathrm{i‌}=1,...‌,m$. Only‌​‌ somewhat recently, the fundamental​​ requirement of linear convergence​​​‌2, as in‌ the unconstrained case, has‌​‌ been clearly stated 27​​.
• In multiobjective optimization,​​​‌ most of the research‌ so far has been‌​‌ focusing on how to​​ select candidate solutions from​​​‌ one iteration to the‌ next one. The‌​‌ difficult question of how​​ to generate effectively new​​​‌ solutions is not yet‌ answered in a proper‌​‌ way and we know​​ today that simply applying​​​‌ operators from single-objective optimization‌ may not be effective‌​‌ with the current best​​ selection strategies. As a​​​‌ comparison, in the single-objective‌ case, the question of‌​‌ selection of candidate solutions​​ was already solved in​​​‌ the 1980s and 15‌ more years were needed‌​‌ to solve the trickier​​ question of an effective​​​‌ adaptive strategy to generate‌ new solutions.
• With the‌​‌ current demand to solve​​ larger and larger optimization​​​‌ problems (e.g. in the‌ domain of deep learning),‌​‌ optimization algorithms that scale​​ linearly (in terms of​​​‌ internal complexity, memory and‌ number of function evaluations‌​‌ to reach an $\mathrm{ϵ‌}$-ball around the optimum)​ with the problem dimension​‌ are nowadays in increasing​​ demand. Not long ago,​​​‌ first proposals of how​ to reduce the quadratic​‌ scaling of CMA-ES have​​ been made without a​​​‌ clear view of what​ can be achieved in​‌ the best case in​​ practice. These later​​​‌ variants apply to optimization​ problems with thousands of​‌ variables. The question of​​ designing randomized algorithms capable​​​‌ to handle problems with​ one or two orders​‌ of magnitude more variables​​ effectively and efficiently is​​​‌ still largely open.
• For​ expensive optimization, standard methods​‌ are so called Bayesian​​ optimization (BO) algorithms often​​​‌ based on Gaussian processes.​ Commonly used examples of​‌ BO algorithms are EGO​​ 42, SMAC 40​​​‌, Spearmint 55,​ or TPE 30 which​‌ are implemented in different​​ libraries. Yet, our experience​​​‌ with a popular method​ like EGO is that​‌ many important aspects to​​ come up with a​​​‌ good implementation rely on​ insider knowledge and are​‌ not standard across implementations.​​ Two EGO implementations can​​​‌ differ for example in​ how they perform the​‌ initial design, which bandwidth​​ for the Gaussian kernel​​​‌ is used, or which​ strategy is taken to​‌ optimize the expected improvement.​​

Additionally, the development of​​​‌ stochastic adaptive methods for​ black-box optimization has been​‌ mainly driven by heuristics​​ and practice—rather than​​​‌ a general theoretical framework—validated​ by intensive computational simulations.​‌ Undoubtedly, this has been​​ an asset as the​​​‌ scope of possibilities for​ design was not restricted​‌ by mathematical frameworks for​​ proving convergence. In effect,​​​‌ powerful stochastic adaptive algorithms​ for unconstrained optimization like​‌ the CMA-ES algorithm emerged​​ from this approach. At​​​‌ the same time, naturally,​ theory strongly lags behind​‌ practice. For instance,​​ the striking performances of​​​‌ CMA-ES empirically observed contrast​ with how little is​‌ theoretically proven on the​​ method. This situation is​​​‌ clearly not satisfactory. On​ the one hand, theory​‌ generally lifts performance assessment​​ from an empirical level​​​‌ to a conceptual one,​ rendering results independent from​‌ the problem instances where​​ they have been obtained.​​​‌ On the other hand,​ theory typically provides insights​‌ that change perspectives on​​ some algorithm components. Also​​​‌ theoretical guarantees generally increase​ the trust in the​‌ reliability of a method​​ and facilitate the task​​​‌ to make it accepted​ by wider communities.

Finally,​‌ as discussed above, the​​ development of novel black-box​​​‌ algorithms strongly relies on​ scientific experimentation, and it​‌ is quite difficult to​​ conduct proper and meaningful​​​‌ experimental analysis. This is​ well known for more​‌ than two decades now​​ and summarized in this​​​‌ quote from Johnson in​ 1996

Since then, quite some​​​‌ progress has been made​ to set better standards​‌ in conducting scientific experiments​​ and benchmarking. Yet, some​​​‌ domains still suffer from​ poor benchmarking standards and​‌ from the generic problem​​ of the lack of​​ reproducibility of results. For​​​‌ instance, in multiobjective optimization,‌ it is (still) not‌​‌ rare to see comparisons​​ between algorithms made by​​​‌ solely visually inspecting Pareto‌ fronts after a fixed‌​‌ budget. In Bayesian optimization,​​ good performance seems often​​​‌ to be due to‌ insider knowledge not always‌​‌ well described in papers.​​

In the context of​​​‌ black-box numerical optimization previously‌ described, the scientific positioning‌​‌ of the RandOpt ream​​ is at the intersection​​​‌ between theory, algorithm design,‌ and applications. Our vision‌​‌ is that the field​​ of stochastic black-box optimization​​​‌ should reach the same‌ level of maturity than‌​‌ gradient-based convex mathematical optimization.​​ This entails major algorithmic​​​‌ developments for constrained, multiobjective‌ and large-scale black-box optimization‌​‌ and major theoretical developments​​ for analyzing current methods​​​‌ including the state-of-the-art CMA-ES.‌

The specificity in black-box‌​‌ optimization is that methods​​ are intended to solve​​​‌ problems characterized by "‌non-properties"—non-linear,‌​‌ non-convex, non-smooth,​​ non-Lipschitz. This contrasts​​​‌ with gradient-based optimization and‌ poses on the one‌​‌ hand some challenges when​​ developing theoretical frameworks but​​​‌ also makes it compulsory‌ to complement theory with‌​‌ empirical investigations.

On the​​ practical side, our ultimate​​​‌ goal is to provide‌ software that is suitable‌​‌ for researchers and industry​​ that need to solve​​​‌ practical optimization problems. We‌ see theory also as‌​‌ a means for this​​ end (rather than only​​​‌ an end in itself)‌ and we also firmly‌​‌ belief that parameter tuning​​ is part of the​​​‌ algorithm designer's task.

This‌ shapes, on the one‌​‌ hand, four main scientific​​ objectives for our team:​​​‌

1. develop novel theoretical frameworks‌ for guiding (a) the‌​‌ design of novel black-box​​ methods and (b) their​​​‌ analysis, allowing to
2. provide‌ proofs of key features‌​‌ of stochastic adaptive algorithms​​ including the state-of-the-art method​​​‌ CMA-ES: linear convergence and‌ learning of second order‌​‌ information.
3. develop stochastic numerical​​ black-box algorithms following a​​​‌ principled design in domains‌ with a strong practical‌​‌ need for much better​​ methods namely constrained, multiobjective,​​​‌ large-scale and expensive optimization‌. Implement the methods‌​‌ such that they are​​ easy to use. And​​​‌ finally, to
4. set new‌ standards in scientific experimentation,‌​‌ performance assessment and benchmarking​​ both for optimization on​​​‌ continuous or combinatorial search‌ spaces. This should allow‌​‌ in particular to advance​​ the state of reproducibility​​​‌ of results of scientific‌ papers in optimization.

On‌​‌ the other hand, the​​ above motivates our objectives​​​‌ with respect to dissemination‌ and transfer:

1. develop software‌​‌ packages that people can​​ directly use to solve​​​‌ their problems. This means‌ having carefully thought out‌​‌ interfaces, generically applicable setting​​ of parameters and termination​​​‌ conditions, proper treatment of‌ numerical errors, catching properly‌​‌ various exceptions, etc.;
2. have​​ direct collaborations with industrials;​​​‌
3. publish our results both‌ in applied mathematics and‌​‌ computer science bridging the​​ gap between very often​​​‌ disjoint communities.

3.1 Developing Novel Theoretical‌​‌ Frameworks for Analyzing and​​​‌ Designing Adaptive Stochastic Algorithms​

Stochastic black-box algorithms typically​‌ optimize non-convex, non-smooth functions​​. This is possible​​​‌ because the algorithms rely​ on weak mathematical properties​‌ of the underlying functions:​​ the algorithms do not​​​‌ use the derivatives—hence the​ function does not need​‌ to be differentiable—and, additionally,​​ often do not use​​​‌ the exact function value​ but instead how the​‌ objective function ranks candidate​​ solutions (such methods are​​​‌ sometimes called function-value-free). (To​ illustrate a comparison-based update,​‌ consider an algorithm that​​ samples $\lambda$ (with $\mathrm{\lambda ‌}$ an even integer) candidate​ solutions from a multivariate​‌ normal distribution. Let ${\mathrm{x}}_1,...,‌x_{\lambda }$ in ${\mathrm{ℝ}}^n$ denote those $\mathrm{\lambda ‌}$ candidate solutions at a​​ given iteration. The solutions​​​‌ are evaluated on the​ function $f$ to be​‌ minimized and ranked from​​ the best to the​​​‌ worse:

In​ the previous equation $\mathrm{i‌}:\lambda$ denotes the​​ index of the sampled​​​‌ solution associated to the​ $i$-th best solution.​‌ The new mean of​​ the Gaussian vector from​​​‌ which new solutions will​ be sampled at the​‌ next iteration can be​​ updated as

The previous​​​‌ update moves the mean​ towards the $\lambda /‌2$ best solutions. Yet​​ the update is only​​​‌ based on the ranking​ of the candidate solutions​‌ such that the update​​ is the same if​​​‌ $f$ is optimized or​ $g\circ f$ where​‌ $g:\mathrm{Im}(f)\to \mathrm{ℝ‌}$ is strictly increasing. Consequently,​ such algorithms are invariant​‌ with respect to strictly​​ increasing transformations of the​​​‌ objective function. This entails​ that they are robust​‌ and their performances generalize​​ well.)

Additionally, adaptive stochastic​​​‌ optimization algorithms typically have​ a complex state space​‌ which encodes the parameters​​ of a probability distribution​​​‌ (e.g. mean and covariance​ matrix of a Gaussian​‌ vector) and other state​​ vectors. This state-space is​​​‌ a manifold. While​ the algorithms are Markov​‌ chains, the complexity of​​ the state-space makes that​​​‌ standard Markov chain theory​ tools do not directly​‌ apply. The same​​ holds with tools stemming​​​‌ from stochastic approximation theory​ or Ordinary Differential Equation​‌ (ODE) theory where it​​ is usually assumed that​​​‌ the underlying ODE (obtained​ by proper averaging and​‌ limit for learning rate​​ to zero) has its​​​‌ critical points inside the​ search space. In contrast,​‌ in the cases we​​ are interested in, the​​​‌ critical points of the​ ODEs are at the​‌ boundary of the domain​​.

Last, since we​​​‌ aim at developing theory​ that on the one​‌ hand allows to analyze​​ the main properties of​​​‌ state-of-the-art methods and on​ the other hand is​‌ useful for algorithm design,​​ we need to be​​​‌ careful not to use​ simplifications that would allow​‌ a proof to be​​ done but would not​​​‌ capture the important properties​ of the algorithms. With​‌ that respect one tricky​​ point is to develop​​ theory that accounts for​​​‌ invariance properties.

To‌ face those specific challenges,‌​‌ we need to develop​​ novel theoretical frameworks exploiting​​​‌ invariance properties and accounting‌ for peculiar state-spaces. Those‌​‌ frameworks should allow researchers​​ to analyze one of​​​‌ the core properties of‌ adaptive stochastic methods, namely‌​‌ linear convergence on the​​ widest possible class of​​​‌ functions.

We are planning‌ to approach the question‌​‌ of linear convergence from​​ three different complementary angles,​​​‌ using three different frameworks:‌

• the Markov chain framework‌​‌ where the convergence derives​​ from the analysis of​​​‌ the stability of a‌ normalized Markov chain existing‌​‌ on scaling-invariant functions for​​ translation and scale-invariant algorithms​​​‌ 29. This framework‌ allows for a fine‌​‌ analysis where the exact​​ convergence rate can be​​​‌ given as an implicit‌ function of the invariant‌​‌ measure of the normalized​​ Markov chain. Yet it​​​‌ requires the objective function‌ to be scaling-invariant. The‌​‌ stability analysis can be​​ particularly tricky as the​​​‌ Markov chain that needs‌ to be studied writes‌​‌ as ${\Phi }_{t+1}=F(‌{\Phi }_t,{\mathrm{W‌}}_{t+1})‌‌$ where $\{W_{\mathrm{t}}:t>\mathrm{0‌}\}$ are independent identically‌ distributed and $F$ is‌​‌ typically discontinuous because the​​ algorithms studied are comparison-based.​​​‌ This implies that practical‌ tools for analyzing a‌​‌ standard property like irreducibility,​​ that rely on investigating​​​‌ the stability of underlying‌ deterministic control models 49‌​‌, cannot be used.​​ Additionally, the construction of​​​‌ a drift to prove‌ ergodicity is particularly delicate‌​‌ when the state space​​ includes a (normalized) covariance​​​‌ matrix as it is‌ the case for analyzing‌​‌ the CMA-ES algorithm.
• The​​ stochastic approximation or ODE​​​‌ framework. Those are standard‌ techniques to prove the‌​‌ convergence of stochastic algorithms​​ when an algorithm can​​​‌ be expressed as a‌ stochastic approximation of the‌​‌ solution of a mean​​ field ODE 33,​​​‌ 32, 46.‌ What is specific and‌​‌ induces difficulties for the​​ algorithms we aim at​​​‌ analyzing is the non-standard‌ state-space since the ODE‌​‌ variables correspond to the​​ state-variables of the algorithm​​​‌ (e.g. ${ℝ}^n\times ‌{ℝ}_{>0}$ for‌​‌ step-size adaptive algorithms, ${\mathrm{ℝ}}^n\times {ℝ}_{>‌0}\times S_{+‌+}^n$ where ${\mathrm{S‌‌}}_{++}^n$ denotes​​ the set of positive​​​‌ definite matrices if a‌ covariance matrix is additionally‌​‌ adapted). Consequently, the ODE​​ can have many critical​​​‌ points at the boundary‌ of its definition domain‌​‌ (e.g. all points corresponding​​ to ${\sigma }_t=‌0$ are critical points‌ of the ODE) which‌​‌ is not typical. Also​​ we aim at proving​​​‌ linear convergence, for‌ that it is crucial‌​‌ that the learning rate​​ does not decrease to​​​‌ zero which is non-standard‌ in ODE method.
• The‌​‌ direct framework where we​​ construct a global Lyapunov​​​‌ function for the original‌ algorithm from which we‌​‌ deduce bounds on the​​ hitting time to reach​​​‌ an $ϵ$-ball of‌ the optimum. For this‌​‌ framework as for the​​ ODE framework, we expect​​​‌ that the class of‌ functions where we can‌​‌ prove linear convergence are​​​‌ composite of $g\circ f$ where $f$ is​‌ differentiable and $g:\mathrm{Im}(f)‌\to ℝ$ is strictly​ increasing and that we​‌ can show convergence to​​ a local minimum.

We​​​‌ expect those frameworks to​ be complementary in the​‌ sense that the assumptions​​ required are different. Typically,​​​‌ the ODE framework should​ allow for proofs under​‌ the assumptions that learning​​ rates are small enough​​​‌ while it is not​ needed for the Markov​‌ chain framework. Hence this​​ latter framework captures better​​​‌ the real dynamics of​ the algorithm, yet under​‌ the assumption of scaling-invariance​​ of the objective functions.​​​‌ Also, we expect some​ overlap in terms of​‌ function classes that can​​ be studied by the​​​‌ different frameworks (typically convex-quadratic​ functions should be encompassed​‌ in the three frameworks).​​ By studying the different​​​‌ frameworks in parallel, we​ expect to gain synergies​‌ and possibly understand what​​ is the most promising​​​‌ approach for solving the​ holy grail question of​‌ the linear convergence of​​ CMA-ES. We foresee for​​​‌ instance that similar approaches​ like the use of​‌ Foster-Lyapunov drift conditions are​​ needed in all the​​​‌ frameworks and that intuition​ can be gained on​‌ how to establish the​​ conditions from one framework​​​‌ to another one.

3.2​ Algorithmic developments

We are​‌ planning on developing algorithms​​ in the subdomains with​​​‌ strong practical demand for​ better methods of constrained,​‌ multiobjective, large-scale and expensive​​ optimization.

Many of the​​​‌ algorithm developments, we propose,​ rely on the CMA-ES​‌ method. While this seems​​ to restrict our possibilities,​​​‌ we want to emphasize​ that CMA-ES became a​‌ family of methods over​​ the years that nowadays​​​‌ include various techniques and​ developments from the literature​‌ to handle non-standard optimization​​ problems (noisy, large-scale, ...).​​​‌ The core idea of​ all CMA-ES variants—namely the​‌ mechanism to adapt a​​ Gaussian distribution—has furthermore been​​​‌ shown to derive naturally​ from first principles with​‌ only minimal assumptions in​​ the context of derivative-free​​​‌ black-box stochastic optimization 52​, 37. This​‌ is a strong justification​​ for relying on the​​​‌ CMA-ES premises while new​ developments naturally include new​‌ techniques typically borrowed from​​ other fields. While CMA-ES​​​‌ is now a full​ family of methods, for​‌ visibility reasons, we continue​​ to refer often to​​​‌ “the CMA-ES algorithm”.

3.3‌ Setting novel standards in‌​‌ scientific experimentation and benchmarking​​

Numerical experimentation is needed​​​‌ as a complement to‌ theory to test novel‌​‌ ideas, hypotheses, the stability​​ of an algorithm, and/or​​​‌ to obtain quantitative estimates.‌ Optimally, theory and experimentation‌​‌ go hand in hand,​​ jointly guiding the understanding​​​‌ of the mechanisms underlying‌ optimization algorithms. Though performing‌​‌ numerical experimentation on optimization​​ algorithms is crucial and​​​‌ a common task, it‌ is non-trivial and easy‌​‌ to fall in (common)​​ pitfalls as stated by​​​‌ J. N. Hooker in‌ his seminal paper 39‌​‌.

In the RandOpt​​ team we aim at​​​‌ raising the standards for‌ both scientific experimentation and‌​‌ benchmarking.

On the experimentation​​ aspect, we are convinced​​​‌ that there is common‌ ground over how scientific‌​‌ experimentation should be done​​ across many (sub-)domains of​​​‌ optimization, in particular with‌ respect to the visualization‌​‌ of results, testing extreme​​ scenarios (parameter settings, initial​​​‌ conditions, etc.), how to‌ conduct understandable and small‌​‌ experiments, how to account​​ for invariance properties, performing​​​‌ scaling up experiments and‌ so forth. We therefore‌​‌ want to formalize and​​ generalize these ideas in​​​‌ order to make them‌ known to the entire‌​‌ optimization community with the​​ final aim that they​​​‌ become standards for experimental‌ research.

Extensive numerical benchmarking,‌​‌ on the other hand,​​ is a compulsory task​​​‌ for evaluating and comparing‌ the performance of algorithms.‌​‌ It puts algorithms to​​ a standardized test and​​​‌ allows to make recommendations‌ which algorithms should be‌​‌ used preferably in practice.​​ To ease this part​​​‌ of optimization research, we‌ have been developing the‌​‌ Comparing Continuous Optimizers platform​​ (COCO) since 2007 which​​​‌ allows to automatize the‌ tedious task of benchmarking.‌​‌ It is a game​​ changer in the sense​​​‌ that the freed time‌ can now be spent‌​‌ on the scientific part​​ of algorithm design (instead​​​‌ of implementing the experiments,‌ visualization, etc.) and it‌​‌ opened novel perspectives in​​ algorithm testing. COCO implements​​​‌ a thorough, well-documented methodology‌ that is based on‌​‌ the above mentioned general​​ principles for scientific experimentation.​​​‌

Also due to the‌ freely available data from‌​‌ 350+ algorithms benchmarked with​​ the platform, COCO became​​​‌ a quasi-standard for single-objective,‌ noiseless optimization benchmarking. It‌​‌ is therefore natural to​​ extend the reach of​​​‌ COCO towards other subdomains‌ (particularly constrained optimization, many-objective‌​‌ optimization) which can benefit​​ greatly from an automated​​​‌ benchmarking methodology and standardized‌ tests without (much) effort.‌​‌ This entails particularly the​​ design of novel test​​​‌ suites and rethinking the‌ methodology for measuring performance‌​‌ and more generally evaluating​​ the algorithms. Particularly challenging​​​‌ is the design of‌ scalable non-trivial testbeds for‌​‌ constrained optimization where one​​​‌ can still control where​ the solutions lies. Other​‌ optimization problem types, we​​ are targeting are expensive​​​‌ problems (and the Bayesian​ optimization community in particular),​‌ optimization problems in machine​​ learning (for example parameter​​​‌ tuning in reinforcement learning),​ and the collection of​‌ real-world problems from industry.​​

Another aspect of our​​​‌ future research on benchmarking​ is to investigate the​‌ large amounts of benchmarking​​ data, we collected with​​​‌ COCO during the years.​ Extracting information about the​‌ influence of algorithms on​​ the best performing portfolio,​​​‌ clustering algorithms of similar​ performance, or the automated​‌ detection of anomalies in​​ terms of good/bad behavior​​​‌ of algorithms on a​ subset of the functions​‌ or dimensions are some​​ of the ideas here.​​​‌

Last, we want to​ expand the focus of​‌ COCO from automatized (large)​​ benchmarking experiments towards everyday​​​‌ experimentation, for example by​ allowing the user to​‌ visually investigate algorithm internals​​ on the fly or​​​‌ by simplifying the set​ up of algorithm parameter​‌ influence studies.

5.1 Footprint of research​​​‌ activities

We are concerned​ about CO${}_2$ footprint​‌ and discourage oversea conferences​​ when far away. Since​​​‌ the situation with respect​ to Covid went back​‌ to normal with respect​​ to travelling, we have​​​‌ been dedicated to travel​ less than in the​‌ past and attend some​​ conferences online. Travel by​​​‌ train is preferred over​ planes whenever possible. Our​‌ location in central Europe​​ gives us a clear​​ advantage here.

5.2 Impact​​​‌ of research results

We‌ develop general purpose optimization‌​‌ methods that apply in​​ difficult optimization contexts where​​​‌ little is required on‌ the function to be‌​‌ optimized. Application domains include​​ optimization and design of​​​‌ renewable systems and climate‌ change.

Our main method‌​‌ CMA-ES is transferred and​​ widely used. The code​​​‌ stemming from the team‌ is frequently downloaded (see‌​‌ Section 7). Among​​ the usage of our​​​‌ method and our code,‌ we find naturally problems‌​‌ in the domain of​​ energy to capture carbon​​​‌ dioxide 50, 47‌, 51, solar‌​‌ energy 43, 44​​, or wind-thermal power​​​‌ systems 53.

Those‌ publications witness the impact‌​‌ of our research results​​ with respect to research​​​‌ questions and engineering design‌ related to climate change‌​‌ and renewable energy.

6.1 Awards

• Armand Gissler:‌ PGMO PhD Award, awarded‌​‌ during the PGMO Days,​​ November 2025
• Armand Gissler:​​​‌ honorable mention for the‌ SIGEVO best dissertation award‌​‌ (awarded at ACM-GECCO, July​​ 2025)
• The `pycma` Python​​​‌ package acquired the 1‌ million downloads/month badge.

7.1 Latest software​​ developments

7.2 New platforms

Participants:​​ Anne Auger, Dimo​​​‌ Brockhoff, Nikolaus Hansen‌, Olaf Mersmann (HS‌​‌ Bund, Germany), Tea​​ Tušar (JSI, Slovenia).​​​‌

7.3 Open data

We​​ maintain an open data​​​‌ archive for optimization benchmarking‌ data acquired within the‌​‌ COCO platform. Data​​ submissions can be requested​​​‌ by opening an issue‌. The archive is‌​‌ open but heavily curated​​ and contains 392 data​​​‌ sets. The data can‌ be displayed with our‌​‌ own `coco-postprocess` Python module​​ or, for example, with​​​‌ the IOH profiler or‌ the IOH analizer.‌​‌

8.1​​ On the robustness to​​​‌ noise

Participants: Anne Auger‌, Dimo Brockhoff,‌​‌ Oskar Girardin, Nikolaus​​ Hansen, Alexander Jungeilges​​​‌.

We have initiated‌ a project to more‌​‌ systematically assess the performance​​ to noise of numerical​​​‌ optimizers. This project is‌ connected to the benchmarking‌​‌ section. We have developped​​ a module within COCO​​​‌ that can add different‌ types of noise to‌​‌ the different existing test​​​‌ suites and allows for​ systematic testing of robustness​‌ to noise. A first​​ step has been the​​​‌ testing of a few​ classical optimizers 18,​‌ 17, 16,​​ 15, 14 while​​​‌ ongoing research is about​ a comparative study.

Additionnally,​‌ in the context of​​ the visit of Alexander​​​‌ Jungeilges we have studied​ convergence rates in the​‌ case of multiplicative noise.​​ The work is currently​​​‌ in the process to​ be submitted for publication.​‌

8.2 Algorithm Design and​​ application for Single-Ojective Optimization​​​‌

Participants: Anne Auger,​ Dimo Brockhoff, Oskar​‌ Girardin, Nikolaus Hansen​​, Mohamed Gharafi,​​​‌ Tristan Marty.

One​ main goal we are​‌ pursuing in the RandOpt​​ team concerns the developement​​​‌ of randomized algorithms that​ aim at enriching the​‌ class of problems that​​ can be handled robustly​​​‌ with randomized search methods​ like CMA-ES. We have​‌ this year progressed with​​ respect to constrained optimization,​​​‌ mixed-integer optimization and surrogate​ methods.

In 19,​‌ we developed a surrogate-based​​ approach for handling inequality​​​‌ constraints in constrained black-box​ optimization with the Augmented​‌ Lagrangian CMA-ES. The method​​ relies on linear surrogate​​​‌ functions derived from a​ binary classifier that predicts​‌ only the sign of​​ each constraint violation. As​​​‌ a result, the classifier—and​ the overall algorithm—is invariant​‌ to sign-preserving transformations of​​ the constraint values, enabling​​​‌ robust handling of binary,​ flat, and deceptive constraints.​‌ Interestingly, we observe that​​ modeling constraints solely through​​​‌ sign-based classification allows the​ algorithm to solve certain​‌ classes of constrained problems​​ that cannot be addressed​​​‌ by the original Augmented​ Lagrangian method when using​‌ the exact constraint values.​​

In 20, the​​​‌ AL-CMA-ES algorithm is applied​ for resolving a complex​‌ zoom lens optical design​​ problem. The handling of​​​‌ constraints via augmented Lagrangian​ is key for the​‌ resolution.

In 23,​​ we introduced a rank-based​​​‌ surrogate-assisted variant of CMA-ES.​ Unlike previous methods that​‌ employ rank information as​​ constraints to train an​​​‌ SVM classifier, our approach​ employs a linear-quadratic regression​‌ on the ranks. While​​ this algorithm outperforms CMA-ES​​​‌ with a few exceptions,​ it falls short to​‌ entirely meet the lq-CMA-ES​​ performance levels. To address​​​‌ this, we proposed an​ enhanced variant that handles​‌ together two alternative surrogates,​​ one based on the​​​‌ ranks and one based​ on the original function​‌ values. Although this variant​​ sacrifices strict invariance, it​​​‌ gains in robustness and​ achieves performance comparable to,​‌ or even exceeding, lq-CMA-ES​​ on transformed problems.

8.3​​​‌ Multi-objective optimization

Participants: Anne​ Auger, Dimo Brockhoff​‌, Mohamed Gharafi,​​ Nikolaus Hansen, external​​​‌ collaborators: Tea Tušar, Jordan​ Cork (Jozef Stefan Institute)​‌.

A central theme​​ for the team is​​​‌ the design, analysis, and​ benchmarking of multiobjective optimization​‌ algorithms. In 2025, we​​ have progressed on the​​​‌ following aspects.

With the​ goal to extend our​‌ COCO platform towards constrained​​ multiobjective problems, we investigated​​​‌ theoretically the Pareto sets​ of $m$-objective problems​‌ with spherical objective functions​​ and convex feasible sets​​​‌ 13. Our main​ result shows that the​‌ Pareto set of the​​ constrained problem can be​​ constructed by projecting the​​​‌ Pareto set of the‌ unconstrained problem onto the‌​‌ convex feasible set. Extensions​​ towards more general objective​​​‌ and constraint functions are‌ currently ongoing.

Two other‌​‌ contributions stem from Bachelor​​ student projects. With Nguyen​​​‌ Vu, we proposed a‌ new so-called continuation method‌​‌ for bi-objective problems, which​​ aims at following the​​​‌ Pareto set (in case‌ it is continuous) from‌​‌ one objective function's optimum​​ to the optimum of​​​‌ the second objective function‌ (and back). The implementation‌​‌ with the default CMA-ES​​ as single-objective optimizer, shows​​​‌ good performance on the‌ bbob-biobj test suite when‌​‌ the budget is relatively​​ small (smaller than 1000​​​‌ times dimension) 24.‌ With Luca Vernhes, we‌​‌ worked on a simple​​ bi-objective algorithm (based on​​​‌ COMO-CMA-ES 12) to‌ tackle single-objective constrained problems‌​‌ where the single objective​​ and the sum of​​​‌ all constraint violations are‌ handled as objectives. This‌​‌ work has not been​​ published yet.

8.4 Benchmarking:​​​‌ methodology and the Comparing‌ Continuous Optimziers Platform (COCO)‌​‌

Participants: Anne Auger,​​ Dimo Brockhoff, Alexandre​​​‌ Chotard, Oskar Girardin‌, Nikolaus Hansen,‌​‌ Tanguy Villain, external​​ collaborators: Tobias Glasmachers (Ruhr-Universität​​​‌ Bochumm, Germany), Olaf Mersmann‌ (HS Bund, Germany), Tea‌​‌ Tušar (Jozef Stefan Institute,​​ Slovenia).

Benchmarking is​​​‌ an important task in‌ optimization to compare the‌​‌ performance of algorithms, recommend​​ the best-performing ones to​​​‌ practitioners, and to motivate‌ the design of better‌​‌ solvers. We are leading​​ the benchmarking of derivative​​​‌ free solvers in the‌ context of difficult problems:‌​‌ we have been developing​​ methodologies and testbeds as​​​‌ well as assembled this‌ into a platform automatizing‌​‌ the benchmarking process. This​​ is a continuing effort​​​‌ that we are pursuing‌ in the team.

The‌​‌ COCO platform, developed at​​ Inria since 2007, aims​​​‌ at automatizing numerical benchmarking‌ experiments and the visual‌​‌ presentation of their results.​​ The platform consists of​​​‌ an experimental part to‌ generate benchmarking data (in‌​‌ various programming languages) and​​ a postprocessing module (in​​​‌ Python), see Figure 2‌. At the interface‌​‌ between the two, we​​ provide data sets from​​​‌ numerical experiments of 350+‌ algorithms and algorithm variants‌​‌ from various fields (quasi-Newton,​​ derivative-free optimization, evolutionary computing,​​​‌ Bayesian optimization) and for‌ various problem characteristics (noiseless/noisy‌​‌ optimization, single-/multi-objective optimization, continuous/mixed-integer,​​ ...).

Visual overview of​​​‌ the COCO platform

We have been using​​​‌ the platform in the‌ past to initiate workshop‌​‌ papers during the ACM-GECCO​​ conference as well as​​​‌ to collect algorithm data‌ sets from the entire‌​‌ optimization community. In 2025,​​ the BBOB workshop at​​​‌ the ACM-GECCO conference in‌ Malaga, Spain saw 8‌​‌ regular talks in two​​​‌ sessions corresponding to 10​ scientific publications, see also​‌ the workshop webpage.​​ Besides the co-organization of​​​‌ the workshop, we contributed​ five-fold: In 14,​‌ we compared five classical​​ algorithms from Michael D.​​​‌ Powell on the bbob​ test suite. For the​‌ newly introduced problems with​​ outlier noise, we investigated​​​‌ the performance of Powell's​ UOBYQA 15, the​‌ classical quasi-Newton BFGS 16​​, the Nelder-Mead simplex​​​‌ method 17 as well​ as of CMA-ES 18​‌.

9.1 Bilateral contracts with​ industry

• Contract with the​‌ company Storengy funding the​​ PhD thesis of Mohamed​​​‌ Gharafi in the context​ of the CIROQUO project​‌ (2021–2024), for the latest​​ report, see 31
• Contract​​​‌ with Thales for the​ CIFRE PhD thesis of​‌ Tristan Marty (2023–2026)

10.1​​​‌ International research visitors

10.2 European initiatives

10.3 National​​ initiatives

11.1 Promoting scientific​ activities

11.2 Teaching -‌​‌ Supervision - Juries -​​ Educational and pedagogical outreach​​​‌

11.3 Popularization

12.1 Major publications​

• 1 articleY.Youhei​‌ Akimoto, A.Anne​​ Auger, T.Tobias​​​‌ Glasmachers and D.Daiki​ Morinaga. Global Linear​‌ Convergence of Evolution Strategies​​ on More Than Smooth​​​‌ Strongly Convex Functions.​SIAM Journal on Optimization​‌June 2022HAL
• 2​​ articleY.Youhei Akimoto​​​‌, A.Anne Auger​ and N.Nikolaus Hansen​‌. An ODE Method​​ to Prove the Geometric​​​‌ Convergence of Adaptive Stochastic​ Algorithms.Stochastic Processes​‌ and their Applications145​​2022, 269-307HAL​​​‌DOI
• 3 articleY.​Youhei Akimoto and N.​‌Nikolaus Hansen. Diagonal​​ Acceleration for Covariance Matrix​​​‌ Adaptation Evolution Strategies.​Evolutionary Computation283​‌2020, 405-435HAL​​DOI
• 4 articleA.​​​‌Anne Auger and N.​Nikolaus Hansen. A​‌ SIGEVO impact award for​​ a paper arising from​​​‌ the COCO platform.​ACM SIGEVOlution134​‌January 2021, 1-11​​HALDOI
• 5 article​​​‌D.Dimo Brockhoff,​ A.Anne Auger,​‌ N.Nikolaus Hansen and​​ T.Tea Tušar.​​​‌ Using Well-Understood Single-Objective Functions​ in Multiobjective Black-Box Optimization​‌ Test Suites.Evolutionary​​ Computation2022HALDOI​​​‌
• 6 articleA.Alexandre​ Chotard and A.Anne​‌ Auger. Verifiable Conditions​​ for the Irreducibility and​​​‌ Aperiodicity of Markov Chains​ by Analyzing Underlying Deterministic​‌ Models.Bernoulli25​​1December 2018,​​​‌ 112-147HALDOI
• 7​ inproceedingsN.Nikolaus Hansen​‌. A Global Surrogate​​ Assisted CMA-ES.GECCO​​​‌ 2019 - The Genetic​ and Evolutionary Computation Conference​‌ACMPrague, Czech Republic​​July 2019, 664-672​​​‌HALDOI
• 8 article​N.Nikolaus Hansen,​‌ A.Anne Auger,​​ R.Raymond Ros,​​​‌ O.Olaf Mersmann,​ T.Tea Tušar and​‌ D.Dimo Brockhoff.​​ COCO: A Platform for​​​‌ Comparing Continuous Optimizers in​ a Black-Box Setting.​‌Optimization Methods and Software​​361ArXiv e-prints,​​​‌ arXiv:1603.087852020, 114-144​HALDOI
• 9 article​‌Y.Yann Ollivier,​​ L.Ludovic Arnold,​​​‌ A.Anne Auger and​ N.Nikolaus Hansen.​‌ Information-Geometric Optimization Algorithms: A​​ Unifying Picture via Invariance​​​‌ Principles.Journal of​ Machine Learning Research18​‌182017, 1-65​​HAL
• 10 articleC.​​​‌Cheikh Touré, A.​Anne Auger and N.​‌Nikolaus Hansen. Global​​ linear convergence of Evolution​​​‌ Strategies with recombination on​ scaling-invariant functions.Journal​‌ of Global Optimization2022​​HALDOI
• 11 article​​​‌C.Cheikh Touré,​ A.Armand Gissler,​‌ A.Anne Auger and​​ N.Nikolaus Hansen.​​ Scaling-invariant functions versus positively​​​‌ homogeneous functions.Journal‌ of Optimization Theory and‌​‌ ApplicationsSeptember 2021HAL​​
• 12 inproceedingsC.Cheikh​​​‌ Touré, N.Nikolaus‌ Hansen, A.Anne‌​‌ Auger and D.Dimo​​ Brockhoff. Uncrowded Hypervolume​​​‌ Improvement: COMO-CMA-ES and the‌ Sofomore framework.GECCO‌​‌ 2019 - The Genetic​​ and Evolutionary Computation Conference​​​‌Part of this research‌ has been conducted in‌​‌ the context of a​​ research collaboration between Storengy​​​‌ and InriaPrague, Czech‌ RepublicJuly 2019HAL‌​‌DOIback to text​​

12.2 Publications of the​​​‌ year

12.3​‌ Cited publications

• 25 inproceedings​​Y.Youhei Akimoto and​​​‌ N.Nikolaus Hansen.​ Online model selection for​‌ restricted covariance matrix adaptation​​.International Conference on​​​‌ Parallel Problem Solving from​ NatureSpringer2016,​‌ 3--13back to text​​
• 26 inproceedingsY.Youhei​​​‌ Akimoto and N.Nikolaus​ Hansen. Projection-based restricted​‌ covariance matrix adaptation for​​ high dimension.Proceedings​​​‌ of the 2016 on​ Genetic and Evolutionary Computation​‌ ConferenceACM2016,​​ 197--204back to text​​​‌
• 27 inproceedingsD. V.​D. V. Arnold and​‌ J.J. Porter.​​ Towards au Augmented Lagrangian​​​‌ Constraint Handling Approach for​ the $(1+‌1)$-ES.​​Genetic and Evolutionary Computation​​​‌ ConferenceACM Press2015​, 249-256back to​‌ textback to text​​back to text
• 28​​​‌ inproceedingsA.Asma Atamna​, A.Anne Auger​‌ and N.Nikolaus Hansen​​. Linearly Convergent Evolution​​​‌ Strategies via Augmented Lagrangian​ Constraint Handling.Foundation​‌ of Genetic Algorithms (FOGA)​​2017back to text​​​‌
• 29 articleA.Anne​ Auger and N.Nikolaus​‌ Hansen. Linear Convergence​​ of Comparison-based Step-size Adaptive​​​‌ Randomized Search via Stability​ of Markov Chains.​‌SIAM Journal on Optimization​​2632016,​​​‌ 1589-1624back to text​
• 30 inproceedingsJ.J.​‌ Bergstra, R.R.​​ Bardenet, Y.Y.​​​‌ Bengio and B.B.​ Kégl. Algorithms for​‌ Hyper-Parameter Optimization.Neural​​ Information Processing Systems (NIPS​​​‌ 2011)2011HALback​ to text
• 31 techreport​‌C.Christophette Blanchet-Scalliet,​​ C.Céline Helbert,​​​‌ D.Delphine Sinoquet,​ M. M.Miguel Munoz​‌ Munoz Zuniga, R.​​Rodolphe Le Riche,​​ D.Didier Rullière,​​​‌ C.Clémentine Prieur,‌ O.Olivier Zahm,‌​‌ J.Josselin Garnier,​​ D.Dimo Brockhoff,​​​‌ O.Olivier Roustant,‌ F.François Bachoc,‌​‌ L.Luc Pronzato,​​ Y.Yann Richet,​​​‌ J.Jérémy Rohmer,‌ F.Frédéric Huguet,‌​‌ D.David Gaudrie,​​ A.Amandine Marrel,​​​‌ B.Baptiste Kerleguer,‌ T.Thomas Perrillat-Bottonet and‌​‌ C.Cedric Durantin.​​ Activity report ciroquo research​​​‌ & industry consortium.‌Ecole Centrale de Lyon‌​‌ ; Mines Saint-Etienne ;​​ Université Toulouse 3 (Paul​​​‌ Sabatier) ; Stellantis France‌ ; BRGM (Bureau de‌​‌ recherches géologiques et minières)​​ ; CEA ; IFP​​​‌ Energies Nouvelles ; Institut‌ de Radioprotection et de‌​‌ Sûreté Nucléaire ; Storengy​​ ; INRIA ; CNRS​​​‌2024, 1-11HAL‌back to text
• 32‌​‌ articleV.V.S. Borkar​​ and S.S.P. Meyn​​​‌. The O.D.E. Method‌ for Convergence of Stochastic‌​‌ Approximation and Reinforcement Learning​​.SIAM Journal on​​​‌ Control and Optimization38‌2January 2000back‌​‌ to text
• 33 booklet​​V. S.Vivek S​​​‌ Borkar. Stochastic approximation:‌ a dynamical systems viewpoint‌​‌.Cambridge University Press​​2008back to text​​​‌
• 34 inproceedingsC. A.‌Carlos A Coello Coello‌​‌. Constraint-handling techniques used​​ with evolutionary algorithms.​​​‌Proceedings of the 2008‌ Genetic and Evolutionary Computation‌​‌ ConferenceACM2008,​​ 2445--2466back to text​​​‌
• 35 inproceedingsG.Guillaume‌ Collange, S.Stéphane‌​‌ Reynaud and N.Nikolaus​​ Hansen. Covariance Matrix​​​‌ Adaptation Evolution Strategy for‌ Multidisciplinary Optimization of Expendable‌​‌ Launcher Families.13th​​ AIAA/ISSMO Multidisciplinary Analysis Optimization​​​‌ Conference, Proceedings2010back‌ to textback to‌​‌ text
• 36 bookJ.​​ E.J. E. Dennis​​​‌ and R. B.R.‌ B. Schnabel. Numerical‌​‌ Methods for Unconstrained Optimization​​ and Nonlinear Equations.​​​‌Englewood Cliffs, NJPrentice-Hall‌1983back to text‌​‌
• 37 incollectionN.Nikolaus​​ Hansen and A.Anne​​​‌ Auger. Principled design‌ of continuous stochastic search:‌​‌ From theory to practice​​.Theory and principled​​​‌ methods for the design‌ of metaheuristicsSpringer2014‌​‌, 145--180back to​​ textback to text​​​‌
• 38 articleN.N.‌ Hansen and A.A.‌​‌ Ostermeier. Completely Derandomized​​ Self-Adaptation in Evolution Strategies​​​‌.Evolutionary Computation9‌22001, 159--195‌​‌back to text
• 39​​ articleJ. N.John​​​‌ N Hooker. Testing‌ heuristics: We have it‌​‌ all wrong.Journal​​ of heuristics11​​​‌1995, 33--42back‌ to text
• 40 inproceedings‌​‌F.F. Hutter,​​ H.H. Hoos and​​​‌ K.K. Leyton-Brown.‌ An Evaluation of Sequential‌​‌ Model-based Optimization for Expensive​​ Blackbox Functions.GECCO​​​‌ (Companion) 2013Amsterdam, The‌ NetherlandsACM2013,‌​‌ 1209--1216back to text​​
• 41 articleD. S.​​​‌David S Johnson.‌ A theoretician’s guide to‌​‌ the experimental analysis of​​ algorithms.Data structures,​​​‌ near neighbor searches, and‌ methodology: fifth and sixth‌​‌ DIMACS implementation challenges59​​2002, 215--250back​​​‌ to text
• 42 article‌D. R.Donald R‌​‌ Jones, M.Matthias​​ Schonlau and W. J.​​​‌William J Welch.‌ Efficient global optimization of‌​‌ expensive black-box functions.​​​‌Journal of Global optimization​1341998,​‌ 455--492back to text​​
• 43 articleJ. H.​​​‌Jérôme Henri Kämpf and​ D.Darren Robinson.​‌ A hybrid CMA-ES and​​ HDE optimisation algorithm with​​​‌ application to solar energy​ potential.Applied Soft​‌ Computing922009​​, 738--745back to​​​‌ text
• 44 articleJ.​ H.Jérôme Henri Kämpf​‌ and D.Darren Robinson​​. Optimisation of building​​​‌ form for solar energy​ utilisation using constrained evolutionary​‌ algorithms.Energy and​​ Buildings4262010​​​‌, 807--814back to​ text
• 45 articleI.​‌Iris Kriest, V.​​Volkmar Sauerland, S.​​​‌Samar Khatiwala, A.​Anand Srivastav and A.​‌Andreas Oschlies. Calibrating​​ a global three-dimensional biogeochemical​​​‌ ocean model (MOPS-1.0).​Geoscientific Model Development10​‌12017, 127​​back to text
• 46​​​‌ bookH. J.Harold​ Joseph Kushner and G.​‌George Yin. Stochastic​​ approximation and recursive algorithms​​​‌ and applications.Applications​ of mathematicsNew York​‌Springer2003back to​​ text
• 47 inproceedingsX.​​​‌Xueying Lu, B.​Benjamin Ganis and M.​‌ F.Mary F Wheeler​​. Optimal Design of​​​‌ CO2 Sequestration with Three-Way​ Coupling of Flow-Geomechanics Simulations​‌ and Evolution Strategy.​​SPE Reservoir Simulation Conference​​​‌OnePetro2019back to​ text
• 48 inproceedingsP.​‌Patrick MacAlpine, S.​​Samuel Barrett, D.​​​‌Daniel Urieli, V.​Victor Vu and P.​‌Peter Stone. Design​​ and Optimization of an​​​‌ Omnidirectional Humanoid Walk: A​ Winning Approach at the​‌ RoboCup 2011 3D Simulation​​ Competition.Proceedings of​​​‌ the Twenty-Sixth AAAI Conference​ on Artificial Intelligence (AAAI)​‌Toronto, Ontario, CanadaJuly​​ 2012back to text​​​‌
• 49 bookS.S.P.​ Meyn and R.R.L.​‌ Tweedie. Markov Chains​​ and Stochastic Stability.​​​‌New YorkSpringer-Verlag1993​back to text
• 50​‌ inproceedingsA.Atsuhiro Miyagi​​, Y.Youhei Akimoto​​​‌ and H.Hajime Yamamoto​. Well placement optimization​‌ for carbon dioxide capture​​ and storage via CMA-ES​​​‌ with mixed integer support​.Proceedings of the​‌ Genetic and Evolutionary Computation​​ Conference Companion2018,​​​‌ 1696--1703back to text​
• 51 inproceedingsA.Atsuhiro​‌ Miyagi, H.Hajime​​ Yamamoto, Y.Youhei​​​‌ Akimoto and Z.Ziqiu​ Xue. Development of​‌ a high speed optimization​​ tool for well placement​​​‌ in Geological Carbon dioxide​ Sequestration.5th ISRM​‌ Young Scholars' Symposium on​​ Rock Mechanics and International​​​‌ Symposium on Rock Engineering​ for Innovative FutureOnePetro​‌2019back to text​​
• 52 articleY.Y.​​​‌ Ollivier, L.L.​ Arnold, A.A.​‌ Auger and N.N.​​ Hansen. The Journal​​​‌ of Machine Learning Research​1812017,​‌ 564--628back to text​​back to text
• 53​​​‌ articleS. S.S​ Surender Reddy, B.​‌BK Panigrahi, R.​​Rupam Kundu, R.​​​‌Rohan Mukherjee and S.​Shantanab Debchoudhury. Energy​‌ and spinning reserve scheduling​​ for a wind-thermal power​​​‌ system using CMA-ES with​ mean learning technique.​‌International Journal of Electrical​​ Power & Energy Systems​​​‌532013, 113--122​back to text
• 54​‌ articleT.Tim Salimans​​, J.Jonathan Ho​​, X.Xi Chen​​​‌ and I.Ilya Sutskever‌. Evolution strategies as‌​‌ a scalable alternative to​​ reinforcement learning.arXiv​​​‌ preprint arXiv:1703.038642017back‌ to textback to‌​‌ text
• 55 inproceedingsJ.​​J. Snoek, H.​​​‌H. Larochelle and R.‌ ..R .P. Adams‌​‌. Practical bayesian optimization​​ of machine learning algorithms​​​‌.Neural Information Processing‌ Systems (NIPS 2012)2012‌​‌, 2951--2959back to​​ text
• 56 articleJ.​​​‌Jannis Uhlendorf, A.‌Agnès Miermont, T.‌​‌Thierry Delaveau, G.​​Gilles Charvin, F.​​​‌François Fages, S.‌Samuel Bottani, G.‌​‌Gregory Batt and P.​​Pascal Hersen. Long-term​​​‌ model predictive control of‌ gene expression at the‌​‌ population and single-cell levels​​.Proceedings of the​​​‌ National Academy of Sciences‌109352012,‌​‌ 14271--14276back to text​​