Which statistical analysis should I run?

Question

Thank you! I am struggling with settling on what the best approach to my test design to choose. My initial inclination is to run a two-way repeated ANOVA. I have two dependent variables measured at three points in time. The variables are independent of one another and are self-report. I have a control and treatment group. However, I am stuck at this point, both the control and experimental group have 3 additional groups (cancer, diabetes, and epilepsy diagnoses) that are all independent of one another (they are the same 3 subgroups in the control and experimental group). I am wanting to know whether my intervention will lead to a reduction in DV scores between the control and experimental group and if there are significant differences between the scores of patients with each disease.

Answer 1

Without really diving into your project with you, it is hard to say what exactly would be the best analysis to run (there are all kinds of caveats like the appropriate distribution to model the error term, what kind of variables you have, specific aspects of your project that may influence how exactly you should set up the model). If you want to get more specific, please feel free to reach out for a consult! I love to help with this kind of thing.

Given that, assuming a normal error distribution and constant error variance (typical ANOVA assumptions) prove to be okay modeling assumptions, it sounds like a repeated measures ANOVA might be an okay way to go. A potentially 'easier' way to set your problem up though, would be to use a mixed effect linear model where you treat the subject/patient as a random intercept, and then include fixed effects for treatment (control vs experimental), one binary variable for each of the three sub-categories, and for the two-way interactions between the treatment and those diagnosis variables. If the interactions prove to not be significant, you could remove those in a step-wise way. Hope this helps some!

Answer 2

Based on the description of the study design, the best statistical approach would likely be a Mixed-Effects Repeated Measures ANOVA (Mixed ANOVA or Linear Mixed Model) rather than a standard Two-Way Repeated Measures ANOVA. Here’s why:

Key Factors in the Study Design:

1. Repeated Measures Factor (Time)
2. Each participant is measured at three points in time.
3. Between-Subjects Factor (Group)
4. Two groups: Control and Experimental.
5. Additional Between-Subjects Factor (Disease Type)
6. Three subgroups: Cancer, Diabetes, and Epilepsy (within both control and experimental groups).
7. Multiple Dependent Variables (DV)
8. Two self-reported dependent variables.

Best Approach: Mixed-Effects Repeated Measures ANOVA

A Mixed-ANOVA (Linear Mixed Model, LMM) allows for:

1. Time (Repeated Measures Factor): Within-subjects effect.
2. Group (Control vs. Experimental): Between-subjects effect.
3. Disease Type (Cancer, Diabetes, Epilepsy): Additional between-subjects factor.
4. Interaction Effects: Examining if the effect of time on DV varies by group or disease type.
5. Flexibility with Missing Data: Linear mixed models handle missing data better than traditional repeated-measures ANOVA.

Steps to Implement in Software:

In SPSS:

1. Go to: Analyze → General Linear Model → Repeated Measures (for traditional RM-ANOVA)
2. OR
3. Analyze → Mixed Models → Linear (for more flexibility with LMM)
4. Define Factors:
5. Within-Subjects Factor: Time (3 levels)
6. Between-Subjects Factors: Group (2 levels) and Disease Type (3 levels)
7. Dependent Variables: Your two DVs
8. Check for Interaction Effects:
9. Time × Group interaction → Does the intervention affect scores over time?
10. Time × Disease interaction → Do different diseases respond differently over time?
11. Group × Disease interaction → Does the intervention work differently for different diseases?
12. Examine Pairwise Comparisons: To determine where significant differences occur.

Alternative Approach: MANOVA

If you are also interested in analyzing multiple dependent variables simultaneously, a Multivariate ANOVA (MANOVA) could be an option. However, given that you have repeated measures, a Mixed-ANOVA or Linear Mixed Model (LMM) is a better fit.

Final Recommendation:

Use Mixed-Effects Repeated Measures ANOVA (Linear Mixed Model) to assess:

1. The effect of time on your dependent variables.
2. Differences between control vs. experimental groups.
3. Variations across disease subgroups.
4. Interaction effects (e.g., does the intervention work differently for different diseases over time?).

This method provides greater flexibility and handles missing data better than a traditional Two-Way Repeated Measures ANOVA. I hope this helps. If you have any additional questions, please let me know.

Take care!

Dr. Christal-Joy Turner

Answer 3

It sounds like you have a complex experimental design with multiple factors and dependent variables, as well as subgroups within both the control and experimental groups. To address your research questions effectively, you can consider the following steps:

1. Initial Data Exploration: Begin by conducting an initial exploration of your data. This may include looking at descriptive statistics, distributions, and visualizations of your dependent variables across the various groups and time points. This can help you better understand the nature of your data.
2. Statistical Analysis Plan:
3. a. Two-Way Repeated Measures ANOVA: A two-way repeated measures ANOVA can be a suitable choice to analyze the effects of your intervention (control vs. experimental) and the time points (pre, post, and follow-up) on your dependent variables. This would address the primary question of whether your intervention had a significant effect over time.
4. b. Multivariate Analysis: Given that you have two dependent variables, you might consider a multivariate analysis of variance (MANOVA) instead of separate ANOVAs for each dependent variable. MANOVA allows you to examine whether there are significant differences across groups while accounting for the intercorrelations between the dependent variables.
5. c. Post Hoc Tests: After conducting your initial analysis, you may need to perform post hoc tests or pairwise comparisons to identify specific differences between groups or time points. This can help you pinpoint where significant effects are occurring.
6. Accounting for Subgroups: To address the subgroups within both control and experimental groups (cancer, diabetes, and epilepsy diagnoses), you can use them as additional categorical factors in your analysis. This would involve extending your two-way repeated measures ANOVA or MANOVA to include these subgroup factors as independent variables. You can then assess whether there are significant differences in the dependent variables between these subgroups.
7. Interactions: Pay attention to potential interactions between factors. For example, you might examine whether the effect of your intervention varies across different disease diagnoses (interaction between intervention and diagnosis).
8. Effect Sizes and Clinical Significance: While statistical significance is essential, also consider effect sizes to assess the practical or clinical significance of your findings.