Running an Intraclass Correlation Coefficient in SPSS

Question

I need to figure out how to run an ICC on a data set before moving forward with factor analysis. In my data, I have 78 teachers who have responded to questionnaire data for 5 students each, leading to a total of 390 students. I need to find out if there is a teacher effect in the data to determine whether my factor analysis model needs to be nested. Most things for ICC that I am finding are about the same raters rating the same subjects...however, none of my subjects were rated by multiple raters. My first 5 subjects share 1 unique rater, then the next 5 have the same 1 unique rater, etc. through all 390 subjects and 78 raters. Does anyone know how to run this in SPSS? Also, how to organize the data set in SPSS?

Answer 1

It appears that you want to calculate the Intraclass Correlation Coefficient (ICC) to assess the degree of variation in your data that can be attributed to differences between teachers (raters). In your case, you have multiple students rated by the same teacher, which can be treated as a random effect. You can calculate the ICC using a one-way random-effects ANOVA model.

Here's how to organize and run the ICC analysis in SPSS:

Data Organization:

1. Data Structure: Your data should be structured with one row per student and include variables for the student's responses, the teacher (rater) ID, and the student's ID.
2. Variables: You should have at least the following variables:
3. Student ID: A unique identifier for each student.
4. Teacher ID: A unique identifier for each teacher (rater).
5. Dependent Variable(s): The responses from the students, which you want to assess for teacher effects.

Running ICC Analysis in SPSS:

1. Open your dataset in SPSS.
2. Analyze > General Linear Model > Univariate:
3. In the "Univariate" dialog box, move your dependent variable(s) into the "Dependent Variable(s)" box.
4. Move the "Teacher ID" variable into the "Fixed Factors" box.
5. Click the "Options" button.
6. Options:
7. In the "Options" dialog box, check the "Descriptives for the factors" option.
8. Click "Continue" to return to the "Univariate" dialog.
9. Univariate:
10. In the "Univariate" dialog, click the "Plots" button.
11. In the "Plots" dialog, check the "Homogeneity tests" option.
12. Click "Continue" to return to the "Univariate" dialog.
13. Univariate (cont.):
14. Click "OK" to run the analysis.

SPSS will generate output that includes the ICC value under "Intraclass Correlation Coefficients" and "Teacher ID" as the source of variation.

The ICC value represents the proportion of total variance in the responses that can be attributed to differences between teachers. If the ICC is significant (i.e., not close to zero), it suggests that there is a teacher effect, indicating that students' responses are influenced by the teacher who rated them.

Remember to interpret the ICC in the context of your research question and consider whether this teacher effect is practically significant for your analysis. If the ICC is substantial, it may support the need for a nested factor analysis model that accounts for the teacher effect.