01 Measurement Invariance

Measurement invariance assesses the (psychometric) equivalence of a construct across groups or measurement occasions and demonstrates that a construct has the same meaning to those groups or across repeated measurements

Hence, noninvariance of a construct across groups or measurements can lead to erroneous conclusions about the effectiveness of a trial

methodologists increasingly directed attention to the significance of measurement invariance, especially within a structural equation modeling framework

• in an item-response theory (IRT) framework or

• a structural equation modeling (SEM) framework

=> we focus exclusively on the SEM framework using confirmatory factor analysis (CFA) because SEM is more commonly used than IRT.

In CFA, items that make up a construct (e.g., questionnaire items that form a scale) load on a latent or unobserved factor representing the construct.

1. Configural

2. Metric

3. Scalar

4. Strict / residual

1. Configural invariance: equivalence of the model form

2. Metric invariance: equivalence of the factor loadings

3. Scalar invariance: equivalence of intercepts 

=> In practice, meeting the first three levels of invariance, or achieving partial invariance, is considered enough to guarantee appropriate cross-group comparisons of the latent constructs (Van de Schoot et al., 2012).

• least stringent

• to test whether the constructs (in this case, latent factors of parental warmth and control) have the same pattern of free and fixed loadings

• means that the basic organization of the constructs (i.e., 5 loadings on each latent factor) is supported in the two cultures

Noninvariance:

Configural noninvariance means that the pattern of loadings of items on the latent factors differs in the two cultures (e.g., in one culture only, at least one item loads on a different factor, cross-loads on both factors, etc).

Finding configural noninvariance leaves two options:

(1) redefine the construct (e.g., omit some items and retest the model) or

(2) assume that the construct is noninvariant and discontinue invariance and group difference testing

• Metric invariance means that each item contributes to the latent construct to a similar degree across groups

• Metric invariance is tested by constraining factor loadings (i.e., the loadings of the items on the constructs) to be equivalent in the two groups.

The model with constrained factor loadings (Fig. 1C) is then compared to the configural invariance model (Fig. 1A) to determine fit. If the overall model fit is significantly worse in the metric invariance model compared to the configural invariance model (model fit is discussed below), it indicates that at least one loading is not equivalent across the groups, and metric invariance is not supported

e.g. noninvariance of a loading related to kissing a child on the warmth factor would indicate that this item is more closely related to parental warmth in one culture than in the other (assume Fig. 1C applies to one group and Fig. 1D applies to the other).

Finding metric noninvariance leaves three options:

(1) investigate the source of noninvariance by sequentially releasing (in a backward approach) or adding (in a forward approach; see e.g., Jung & Yoon, 2016) factor loading constraints and retesting the model until a partially invariant model is achieved (partial invariance is discussed below),

(2) omit items with noninvariant loadings and retest the configural and metric invariance models, or

(3) assume that the construct is noninvariant and discontinue invariance and group difference testing.

=> If full or partial metric invariance is supported, the next step is to test for scalar invariance, or equivalence of item intercepts, for metric invariant items

• Scalar invariance means that mean differences in the latent construct capture all mean differences in the shared variance of the items.

• Scalar invariance is tested by constraining the item intercepts to be equivalent in the two groups.

• If the overall model fit is significantly worse in the scalar invariance model compared to the metric invariance model, it indicates that at least one item intercept differs across the two groups, and scalar invariance is not supported.

=> For example, noninvariance of an item intercept for kissing a child would mean that parents in one culture kiss their children more, but that increased kissing is not related to increased levels of parental warmth in that culture

Finding scalar noninvariance leaves three options:

(1) investigate the source of noninvariance by sequentially releasing (in a backward approach) or adding (in a forward approach) item intercept constraints and retesting the model until a partially invariant model is achieved,

(2) omit items with noninvariant intercepts and retest the configural, metric, and scalar invariance models, or

(3) assume that the construct is noninvariant and discontinue invariance and group difference testing.

• Residual invariance means that the sum of specific variance (variance of the item that is not shared with the factor) and error variance (measurement error) is similar across groups.

• It should be noted that there could be more measurement error and less specific variance in one group than another, and residual invariance could still be supported if the totals of these two components were similar.

• Although a required component for full factorial invariance (Meredith, 1993), testing for residual invariance is not a prerequisite for testing mean differences because the residuals are not part of the latent factor, so invariance of the item residuals is inconsequential to interpretation of latent mean differences (Vandenberg & Lance, 2000).