Sparse recovery in convex hulls via entropy penalization
| dc.creator | Koltchinskii, Vladimir | |
| dc.date | 2009-05-13 | |
| dc.date.accessioned | 2026-07-07T13:14:28Z | |
| dc.date.available | 2026-07-07T13:14:28Z | |
| dc.description | Let $(X,Y)$ be a random couple in $S\times T$ with unknown distribution $P$ and $(X_1,Y_1),...,(X_n,Y_n)$ be i.i.d. copies of $(X,Y).$ Denote $P_n$ the empirical distribution of $(X_1,Y_1),...,(X_n,Y_n).$ Let $h_1,...,h_N:S\mapsto [-1,1]$ be a dictionary that consists of $N$ functions. For $λ\in {\mathbb{R}}^N,$ denote $f_λ:=\sum_{j=1}^Nλ_jh_j.$ Let $\ell:T\times {\mathbb{R}}\mapsto {\mathbb{R}}$ be a given loss function and suppose it is convex with respect to the second variable. Let $(\ell \bullet f)(x,y):=\ell(y;f(x)).$ Finally, let $Λ\subset {\mathbb{R}}^N$ be the simplex of all probability distributions on $\{1,...,N\}.$ Consider the following penalized empirical risk minimization problem \begin{eqnarray*}\hatλ^{\varepsilon}:={\mathop {argmin}_{λ\in Λ}}\Biggl[P_n(\ell \bullet f_λ)+\varepsilon \sum_{j=1}^Nλ_j\log λ_j\Biggr]\end{eqnarray*} along with its distribution dependent version \begin{eqnarray*}λ^{\varepsilon}:={\mathop {argmin}_{λ\in Λ}}\Biggl[P(\ell \bullet f_λ)+\varepsilon \sum_{j=1}^Nλ_j\log λ_j\Biggr],\end{eqnarray*} where $\varepsilon\geq 0$ is a regularization parameter. It is proved that the ``approximate sparsity'' of $λ^{\varepsilon}$ implies the ``approximate sparsity'' of $\hatλ^{\varepsilon}$ and the impact of ``sparsity'' on bounding the excess risk of the empirical solution is explored. Similar results are also discussed in the case of entropy penalized density estimation. | |
| dc.description | Published in at http://dx.doi.org/10.1214/08-AOS621 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org) | |
| dc.identifier | https://arxiv.org/abs/0905.2078 | |
| dc.identifier | http://arxiv.org/abs/0905.2078 | |
| dc.identifier | Annals of Statistics 2009, Vol. 37, No. 3, 1332-1359 | |
| dc.identifier | doi:10.1214/08-AOS621 | |
| dc.identifier.uri | http://salesiana.dossiersoluciones.com/handle/123456789/230221 | |
| dc.subject | Statistics Theory | |
| dc.subject | 62G07, 62G08, 62H30 (Primary) | |
| dc.title | Sparse recovery in convex hulls via entropy penalization | |
| dc.type | text |