Finite sample approximation results for principal component analysis: a matrix perturbation approach
| dc.creator | Nadler, Boaz | |
| dc.date | 2009-01-21 | |
| dc.date.accessioned | 2026-07-07T12:32:33Z | |
| dc.date.available | 2026-07-07T12:32:33Z | |
| dc.description | Principal component analysis (PCA) is a standard tool for dimensional reduction of a set of $n$ observations (samples), each with $p$ variables. In this paper, using a matrix perturbation approach, we study the nonasymptotic relation between the eigenvalues and eigenvectors of PCA computed on a finite sample of size $n$, and those of the limiting population PCA as $n\to\infty$. As in machine learning, we present a finite sample theorem which holds with high probability for the closeness between the leading eigenvalue and eigenvector of sample PCA and population PCA under a spiked covariance model. In addition, we also consider the relation between finite sample PCA and the asymptotic results in the joint limit $p,n\to\infty$, with $p/n=c$. We present a matrix perturbation view of the "phase transition phenomenon," and a simple linear-algebra based derivation of the eigenvalue and eigenvector overlap in this asymptotic limit. Moreover, our analysis also applies for finite $p,n$ where we show that although there is no sharp phase transition as in the infinite case, either as a function of noise level or as a function of sample size $n$, the eigenvector of sample PCA may exhibit a sharp "loss of tracking," suddenly losing its relation to the (true) eigenvector of the population PCA matrix. This occurs due to a crossover between the eigenvalue due to the signal and the largest eigenvalue due to noise, whose eigenvector points in a random direction. | |
| dc.description | Published in at http://dx.doi.org/10.1214/08-AOS618 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org) | |
| dc.identifier | https://arxiv.org/abs/0901.3245 | |
| dc.identifier | http://arxiv.org/abs/0901.3245 | |
| dc.identifier | Annals of Statistics 2008, Vol. 36, No. 6, 2791-2817 | |
| dc.identifier | doi:10.1214/08-AOS618 | |
| dc.identifier.uri | http://salesiana.dossiersoluciones.com/handle/123456789/216816 | |
| dc.subject | Statistics Theory | |
| dc.subject | 62H25, 62E17 (Primary) 15A42 (Secondary) | |
| dc.title | Finite sample approximation results for principal component analysis: a matrix perturbation approach | |
| dc.type | text |