High-dimensional covariance estimation by minimizing $\ell_1$-penalized log-determinant divergence
| dc.creator | Ravikumar, Pradeep | |
| dc.creator | Wainwright, Martin J. | |
| dc.creator | Raskutti, Garvesh | |
| dc.creator | Yu, Bin | |
| dc.date | 2008-11-21 | |
| dc.date.accessioned | 2026-07-07T10:20:12Z | |
| dc.date.available | 2026-07-07T10:20:12Z | |
| dc.description | Given i.i.d. observations of a random vector $X \in \mathbb{R}^p$, we study the problem of estimating both its covariance matrix $Σ^*$, and its inverse covariance or concentration matrix {$Θ^* = (Σ^*)^{-1}$.} We estimate $Θ^*$ by minimizing an $\ell_1$-penalized log-determinant Bregman divergence; in the multivariate Gaussian case, this approach corresponds to $\ell_1$-penalized maximum likelihood, and the structure of $Θ^*$ is specified by the graph of an associated Gaussian Markov random field. We analyze the performance of this estimator under high-dimensional scaling, in which the number of nodes in the graph $p$, the number of edges $s$ and the maximum node degree $d$, are allowed to grow as a function of the sample size $n$. In addition to the parameters $(p,s,d)$, our analysis identifies other key quantities covariance matrix $Σ^*$; and (b) the $\ell_\infty$ operator norm of the sub-matrix $Γ^*_{S S}$, where $S$ indexes the graph edges, and $Γ^* = (Θ^*)^{-1} \otimes (Θ^*)^{-1}$; and (c) a mutual incoherence or irrepresentability measure on the matrix $Γ^*$ and (d) the rate of decay $1/f(n,δ)$ on the probabilities $ \{|\hatΣ^n_{ij}- Σ^*_{ij}| > δ\}$, where $\hatΣ^n$ is the sample covariance based on $n$ samples. Our first result establishes consistency of our estimate $\hatΘ$ in the elementwise maximum-norm. This in turn allows us to derive convergence rates in Frobenius and spectral norms, with improvements upon existing results for graphs with maximum node degrees $d = o(\sqrt{s})$. In our second result, we show that with probability converging to one, the estimate $\hatΘ$ correctly specifies the zero pattern of the concentration matrix $Θ^*$. | |
| dc.description | 35 pages, 9 figures | |
| dc.identifier | https://arxiv.org/abs/0811.3628 | |
| dc.identifier | http://arxiv.org/abs/0811.3628 | |
| dc.identifier.uri | http://salesiana.dossiersoluciones.com/handle/123456789/174771 | |
| dc.subject | Machine Learning | |
| dc.subject | Statistics Theory | |
| dc.title | High-dimensional covariance estimation by minimizing $\ell_1$-penalized log-determinant divergence | |
| dc.type | text |