Dr Adam Shoesmith

Background: There is a need for valid and reliable measures of determinants of sustainability of public health interventions in early childhood education and care (ECEC) settings. This study aimed to develop and evaluate the psychometric and pragmatic properties of such a measure ¿ the Integrated Measure of PRogram Element SuStainability in Childcare Settings (IMPRESS-C). Methods: We undertook a two-phase process guided by the COnsensus-based Standards for the selection of health status Measurement INstruments checklist (COSMIN) and Psychometric and Pragmatic Evidence Rating Scale (PAPERS). Phase 1 involved measure development; i.e., determining items and scales through an iterative process and assessment of face and content validity. Phase 2 involved the evaluation of psychometric and pragmatic properties. The 29-item measure completed by service executives (directors and nominated supervisors) was embedded in a larger survey from a national sample of Australian ECEC services assessing their implementation of nutrition and physical activity programs. Structural validity, concurrent validity, known groups validity, internal consistency, floor and ceiling effects, norms, and pragmatic qualities of the measure were assessed according to the PAPERS criteria. Results: The final measure contained 26 items, with respondents reporting how strongly they agreed or disagreed on a five-point Likert scale. Phase 1 assessments confirmed the relevance, and face and content validity of the scale. In Phase 2, we obtained 482 completed surveys, of which 84% (n = 405) completed the entire measure across 405 ECEC settings (one executive¿per service). Three of the four fit indices for the confirmatory factor analysis met the pre-specified criteria (SRMR = 0.056, CFI = 0.993, RMSEA = 0.067) indicating 'good' structural validity. The IMPRESS-C illustrated: 'good' internal consistency, with Cronbach's alpha values from 0.53 to 0.92; 'emerging' concurrent validity; 'poor' known groups validity; 'good' norms; and 'good' overall pragmatic qualities (cost, readability, length, and assessor burden). Conclusions: The IMPRESS-C possesses strong psychometric and pragmatic qualities for assessing service executive-level perceptions of determinants influencing sustainment of public health interventions within ECEC settings. To achieve a full range of perspectives in this setting, future work should be directed to also develop and test measures of sustainability determinants at the implementer level (e.g., among individual educators and staff).

Objectives: Assess the impact of an implementation intervention on student's physical activity, health-related quality of life (HRQoL) and on-task behaviour. Design: A cluster-randomised controlled trial. Methods: Following baseline 61 eligible schools were randomised to a 12-month, implementation intervention to increase teacher scheduling of physical activity, or a waitlist control. Whole school-day and class-time physical activity of students from grades 2 and 3 (~ages 7 to 9) were measured via wrist-worn accelerometers and included: moderate-to-vigorous physical activity, light physical activity, sedentary behaviour and activity counts per minute. Children's health related quality of life (HRQoL) and out-of-school-hours physical activity was measured via parent-proxy surveys. Class level on-task behaviour was measured via teacher self-report surveys. Student and teacher obtained outcomes were measured at baseline and 12-month follow-up. Parent reported outcomes were measured at 12-month follow-up. Linear mixed models compared between group differences in outcomes. Differential effects by sex were explored for student and parent reported outcomes. Results: Data from 2485 students, 1220 parents and >500 teachers were analysed. There was no statistically significant between group differences in any of the outcomes, including accelerometer measured physical activity, out-of-school-hours physical activity, HRQoL, and on-task behaviour. A statistically significant differential effect by sex was found for sedentary behaviour across the whole school day (3.16 min, 95% CI: 0.19, 6.13; p = 0.028), with females illustrating a greater difference between groups than males. Conclusions: Only negligible effects on student physical activity were found. Additional strategies including improving the quality of teacher's delivery of physical activity may be required to enhance effects.

There is a lack of valid and reliable measures of determinants of sustainability specific to public health interventions in the elementary school setting. This study aimed to adapt and evaluate the Program Sustainability Assessment Tool (PSAT) for use in this setting. An expert reference group adapted the PSAT to ensure face validity. Elementary school teachers participating in a multi-component implementation intervention to increase their scheduling of physical activity completed the adapted PSAT. Structural validity was assessed via confirmatory factor analysis. Convergent validity was assessed using linear mixed regression evaluating the associations between scheduling of physical activity and adapted PSAT scores. Cronbach's alpha was used to evaluate internal consistency and intracluster correlation coefficients for interrater reliability. Floor and ceiling effects were also evaluated. Following adaptation and psychometric evaluation, the final measure contained 26 items. Domain Cronbach's alpha ranged from 0.77 to 0.92. Only one domain illustrated acceptable interrater reliability. Evidence for structural validity was mixed and was lacking for convergent validity. There were no floor and ceiling effects. Efforts to adapt and validate the PSAT for the elementary school setting were mixed. Future work to develop and improve measures specific to public health program sustainment that are relevant and psychometrically robust for elementary school settings are needed.