Abstract
In applications of covariance structure modeling in which an initial model does not fit sample data well, it has become common practice to modify that model to improve its fit. Because this process is data driven, it is inherently susceptible to capitalization on chance characteristics of the data, thus raising the question of whether model modifications generalize to other samples or to the population. This issue is discussed in detail and is explored empirically through sampling studies using 2 large sets of data. Results demonstrate that over repeated samples, model modifications may be very inconsistent and cross-validation results may behave erratically. These findings lead to skepticism about generalizability of models resulting from data-driven modifications of an initial model. The use of alternative a priori models is recommended as a preferred strategy.
Keywords
Generalizability theoryCovarianceEconometricsA priori and a posterioriSample (material)Structural equation modelingSampling (signal processing)PopulationComputer scienceSample size determinationStatisticsPsychologyMathematics
Related Publications
REBUS‐PLS: A response‐based procedure for detecting unit segments in PLS path modelling
Abstract Structural equation models (SEMs) make it possible to estimate the causal relationships, defined according to a theoretical model, linking two or more latent complex co...
Critical comments on applying covariance structure modeling
Abstract Covariance structure modeling, also known as structural equation modeling or causal modeling, appears increasingly popular. Such techniques can be used to conduct tests...
CODA: convergence diagnosis and output analysis for MCMC
[1st paragraph] At first sight, Bayesian inference with Markov Chain Monte Carlo (MCMC) appears to be straightforward. The user defines a full probability model, perhaps using o...
Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification.
This study evaluated the sensitivity of maximum likelihood (ML)-, generalized least squares (GLS)-, and asymptotic distribution-free (ADF)-based fit indices to model misspecific...
Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives
This article examines the adequacy of the "rules of thumb" conventional cutoff criteria and several new alternatives for various fit indexes used to evaluate model fit in practi...
Publication Info
- Year
- 1992
- Type
- article
- Volume
- 111
- Issue
- 3
- Pages
- 490-504
- Citations
- 1515
- Access
- Closed
External Links
Social Impact
Altmetric
PlumX Metrics
Social media, news, blog, policy document mentions
Citation Metrics
1515
OpenAlex
Cite This
Robert C. MacCallum,
Mary Roznowski,
Lawrence B. Necowitz
(1992).
Model modifications in covariance structure analysis: The problem of capitalization on chance..
Psychological Bulletin
, 111
(3)
, 490-504.
https://doi.org/10.1037/0033-2909.111.3.490
Identifiers
- DOI
- 10.1037/0033-2909.111.3.490