The Mediating Role of Student Session Engagement in the Relationship between School-Based Behavioral Health Interventions and Outcomes

Yijing Lin presented the results of a mediation analysis examining mediating role of student engagement in the relationship between treatment type (modular vs. TAU) and student outcomes on a teacher-rated measure of conduct problems. Given the inability to model clustering, results should be interpreted as preliminary and associational, with emphasis on confidence intervals rather than statistical significance.

Introduction

Behavioral problems in youth are associated with poor academic performance and impaired peer relationships (Kremer et al., 2016). The Modular Approach to Therapy for Children with Anxiety, Depression, Trauma, or Conduct Problems (MATCH-ADTC) has demonstrated efficacy in reducing these difficulties (Weisz et al., 2012), but treatment success may be strongly influenced by the therapeutic alliance and student engagement (Baier et al., 2020).

Aim: We examined student engagement in the MATCH-ADTC to clarify the role of this key mechanism driving implementation quality and outcomes in school-based settings.

Method

The present study was conducted in 16 schools in North Carolina and South Carolina as part of the Early Supports for Study Success study. Schools were randomly assigned to either the MATCH-ADTC or treatment as usual (TAU) conditions. Participants were 126 fourth and fifth grade students referred for intensive, one-to-one behavioral health support between 2022 and 2024. Six community-based therapists provided services across both conditions, with therapists in the MATCH-ADTC receiving a brief training in modular therapy followed by monthly supervision. Student engagement was assessed using a custom therapist-rated scale from 0 (The client responded poorly) to 4 (The client responded positively). Teachers rated the students’ conduct on the Behavioral Intervention Monitoring Assessment System (BIMAS; McDougal et al., 2011); higher scores indicate greater concerns.

Analysis & Results

Statistical mediation was used to examine the extent to which student engagement explains the relationship between treatment condition (MATCH-ADTC vs. TAU) and behavioral outcomes. The indirect effect was examined through a series of regression models using the lavaan.mi package in R (R Core Team, 2024), and Monte Carlo confidence intervals were estimated using 50,000 replications. There were no missing data on condition status and engagement, but 28.6% of cases had missing behavioral outcomes. Multiple imputation using the mice package (van Buuren & Groothuis-Oudshoorn, 2011) was used to recover these data. Due to the missingness of the dataset, listwise deletion was used to check multivariate outliers and multicollinearity assumptions for the models. No assumptions were violated.

Individuals who received MATCH-ADTC intervention had significantly fewer conduct problems compared to individuals who were in the TAU group. However, student engagement did not significantly mediate this relationship, a*b = -0.12, Monte Carlo CI [ -2.05, 1.74]. Contrary to expectations, the MATCH-ADTC appeared to result in lower student engagement than TAU, which ultimately failed to predict conduct problems. The model R² explained a small percentage of variance in the engagement (R² = 0.08) and total effect model (R² = 0.07). The total R² in the direct model explained 8% of the variance in conduct problems.

Discussion

The MATCH-ADTC has shown promising results in reducing behavior problems in school, and outperformed treatment-as-usual in the present study. But contrary to our expectations, student engagement failed to explain the relationship between treatment and improved behavior outcomes, which is inconsistent with some of the existing research and further investigation is needed.

It is possible that our custom session rating scale data were unreliable or have a complex relationship to treatment outcomes, and future research might explore these possibilities. We also note that there is a potential nesting issue caused by the one-to-many relationship between therapists and participants. Unfortunately, the nesting of students within therapists was inseparable from preexisting school and district assignment structures, which were not under the researchers’ control. For this reason, the results presented here must be interpreted as preliminary and associational, with emphasis on confidence intervals rather than statistical significance.

References

Baier, A. L., Kline, A. C., & Feeny, N. C. (2020). Therapeutic alliance as a mediator of change: A systematic review and evaluation of research. Clinical psychology review, 82, 101921. https://doi.org/10.1016/j.cpr.2020.101921

Kremer, K. P., Flower, A., Huang, J., & Vaughn, M. G. (2016). Behavior problems and children's academic achievement: A test of growth-curve models with gender and racial differences. Children and youth services review, 67, 95–104. https://doi.org/10.1016/j.childyouth.2016.06.003

McDougal, J., Bardos, A., & Meier, S. (2011). Behavior intervention monitoring assessment system technical manual. Multi Health Systems.

Stef van Buuren, Karin Groothuis-Oudshoorn (2011). mice: Multivariate Imputation by Chained Equations in R. Journal of Statistical Software, 45(3), 1-67. https://doi.org/10.18637/jss.v045.i03

Weisz, J. R., Chorpita, B. F., Palinkas, L. A., Schoenwald, S. K., Miranda, J., Bearman, S. K., Daleiden, E. L., Ugueto, A. M., Ho, A., Martin, J., Rodriguez, J., & Gray, J. (2012). Testing standard and modular designs for psychotherapy in youth with anxiety, depression, or conduct problems: A randomized effectiveness trial. Archives of General Psychiatry, 69(3), 274–282. https://doi.org/10.1001/archgenpsychiatry.2011.147

Acknowledgements

The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R324A210179 to East Carolina University and the University of South Carolina. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education.