Section: New Results

Sparse regression estimation

Participants : Gérard Biau, Olivier Catoni, Sébastien Gerchinovitz, Vincent Rivoirard, Gilles Stoltz, Jia Yuan Yu.

Sébastien Gerchinovitz and Jia Yuan Yu continued the work initiated by the former in the above-mentioned conference paper [25] ; they derived from the sparsity results in individual sequences presented therein the minimax optimal rates of aggregation for individual sequences on ${\ell }^1$ balls. In particular, they exhibited, in certain cases, a phase transition between the $lnT$ and the $\sqrt{T}$ behavior of the minimax regret, where $T$ denotes the number of instances. These results and all previous ones are summarized in the PhD thesis [10] .

Other results were obtained in a stochastic framework, where input–output pairs are given by i.i.d. variables; they are described in the technical report [30] . Let $(𝐗,Y)$ be a random pair taking values in ${ℝ}^p\times ℝ$. In the so-called single-index model, one has $Y=f^{*}\left({\theta }^{*T}𝐗\right)+𝐖$, where $f^{*}$ is an unknown univariate measurable function, ${\theta }^{*}$ is an unknown vector in ${ℝ}^d$, and $W$ denotes a random noise satisfying $𝔼\left[𝐖\right|𝐗]=0$. The single-index model is known to offer a flexible way to model a variety of high-dimensional real-world phenomena. However, despite its relative simplicity, this dimension reduction scheme faces severe complications as soon as the underlying dimension becomes larger than the number of observations and this is why this estimation problem was considered from a sparsity perspective using a PAC-Bayesian approach.

Last but not least, we mention the edited book [29] , which provides a modern overview on high-dimensional estimation.