Cronbachs Alpha

• Test is short (see Figure 1, slide 3)

• Average item correlation is low (see Figure 1, slide 3)

• Low variability in your total scores or small range of ability in the sample you are testing. (Recall that correlational or covariance-based measures, respectively, are variance dependent)

• Test only contains either very easy items or very hard dichotomous items

  • This is because item variance is a function of item difficulty p (i.e., proportion of people who answered the item correctly): Var(X) = pq, where q = 1-p

A high alpha may mean that the items in the test are highly correlated. However, α is also sensitive to the number of items in a test: A larger number of items can result in a larger α, a smaller number of items in a smaller α

=> If you have (even weakly) positively correlated items but many of them, you will get a high Cronbach’s alpha

A common misunderstanding: Alpha is a measure of unidimensionality: High Alpha —> test is unidimensional (i.e., measures primarily one thing)

If you have (even weakly) positively correlated items but many of them, you will get a high Cronbach’s alpha

• But this doesn’t necessarily imply that your test is one- dimensional!

• If you have a one-dimensional test, Cronbach’s Alpha can be interpreted unambiguously and indicates the amount of systematic variance in your data

• If you have a multidimensional test, an “overall” Cronbach Alpha doesn’t make sense

• Cronbach’s Alpha for the entire Big Five inventory?

Internal consistency: Usually assessed by split-half reliability

• Randomly divide all items into two sets of equal size

• Calculate total scores for each half and correlate the scores

Problem: This split-half reliability coefficient is based on alternate forms that have only one-half the number of items that the full test has

• Reliability is also a function of the number of items but we have effectively halved the number of itemsàAdjust the calculated correlation to estimate the reliability of a scale that is twice the length, using the Spearman Brown formula:

• You might see a problem here because you just picked two halves at random —> Likely get different reliabilities from different random splits

• Random variation in split-half reliabilities stems from the fact that any estimate of split-half reliability that one gets depends, to some extent, on the particular manner in which one chooses to split the test: First half vs. second half; easy vs. harder items; odd- even split-half method (dividing the test into odd-even numbered items, with odd-numbered items (1, 3, 5 etc.) forming the first group while even-numbered items (2, 4, 6 etc.) forms the second group); etc.)

• Wouldn’t it be better to take all possible split-halves into account and to calculate an average (i.e., typical) split-half reliability?

• That’s exactly the idea of Cronbach’s Alpha: Average split-half reliability of all possible split-halves

  – The formal proof of Cronbach’s Alpha as the average split-half reliability of all possible split-halves can be found in Cronbach (1951, p. 305) or in Lord and Novic (1968, p. 93)

• What we need to determine: Number of ways of picking k unordered outcomes from n possibilities

• This is known as the binomial coefficient or choice number and read "n choose k“

Example: With 8 items, there are 8!/(4!(8 - 4 )!) = 70 possible split-half combinations

– To avoid possible confusion: No, it‘s not 8!/(2!(8-2)!) because we need split-halves, hence 8/2 = 4

Reliability is also a function of the number of items but we have effectively halved the number of items —> Adjust the calculated correlation to estimate the reliability of a scale that is twice the length, using the Spearman Brown formula:

As Warrens (2015) showed, alpha is approximately identical to the mean of all split-half reliabilities, if a test consists of an odd number of items and has at least eleven items