One of the main differences between factor analysis (FA) and principal components analysis (PCA) is that FA attempts to find “unobserved variables” (factors) in a set of original variables, whereas PCA transforms the existing variables into several new variables (components) (Field, 2013, sec. 17.3.0-17.3.2; Warner, 2013, p. 830). Therefore, in FA, it is assumed that there are some “latent variables,” and it is attempted to find them; however, PCA makes no such assumption, only reducing the number of dimensions.
It is emphasized that components in PCA are simply calculated from the original variables as the linear combinations of these variables weighed by their loadings for that component (Field, 2013). On the contrary, in FA, the measured variables are predicted from the latent factors (Field, 2013). Also, the FA calculation formula contains the error term, whereas PCA does not (Field, 2013, sec. 17.3.2).
Importantly, both methods use (factor/component) loadings as weights and seek variables that have large correlations with a group of other variables and very small correlations with variables not included in that group (Field, 2013, sec. 17.3.0).
When choosing between FA and PCA for a study, one should consider the goal of that analysis. If it is simply needed to reduce dimensions, and it is desired that the new dimensions explain the maximum amount of variance in the data (Warner, 2013), it is possible to use PCA to simply determine the orthogonal components in the data and the contributions of each original variable to those components. However, if one assumes that there are “latent variables” in the data and wishes to uncover them, PCA is inappropriate, and FA should be used (Field, 2013, sec. 17.4.3).
As for differences between PCA and FA, PCA attempts to maximize the total variance explained by the components, whereas FA tries to maximize the amount of common variance in factors. Also, it was found that when there are ≥30 variables and the commonalities between all the variables are larger than 0.7, the results of PCA and FA are improbable to differ; however, if there are <20 variables and there exist low communalities (<0.4), the results might be likely to differ (Field, 2013, sec. 17.4.3).
References
Field, A. (2013). Discovering statistics using IBM SPSS Statistics (4th ed.). Thousand Oaks, CA: SAGE Publications.
Warner, R. M. (2013). Applied statistics: From bivariate through multivariate techniques (2nd ed.). Thousand Oaks, CA: SAGE Publications.