Social Impact Assessment: Engaging in Participatory Evaluation - Implications and Practice

This social marketing evaluation project explores how a research and industry partnership can develop an approach to evaluation to understand the social impact of health promotion intervention. This thesis proposes the need for a participatory approach to Social Impact Assessment (SIA). Specifically, this research project sought to understand: first, how social impact is considered and measured with health promotion interventions in four comparable Commonwealth contexts (Australia, Canada, New Zealand, and the United Kingdom); and second, how an Australian NGO may (or may not be) delivering social impact within health promotion and how an SIA assessment may add insight into social impact measurement within health promotion. This research implemented the SIA methodology of Social Return on Investment (SROI) to evaluate the social impact of an NGO Health Promotion program aiming to help children make safe and healthy life choices (Lane & Devin, 2018; McHugh, Domegan, & Duane, 2018). While a Theory of Change (ToC) or program logic is essential for effective measurement of the outcomes or effects of interventions (Breuer, Lee, De Silva, & Lund, 2016; Hamdani et al., 2021; Meiksin et al., 2021), as a standalone process it is insufficient to understand the intricacies of social change. To understand intervention mechanisms of behaviour change—where and how social change (has or has not occurred)—there is a need to implement a measurement process that captures the broader social and economic determinants of health. These wider determinants extend beyond the current linear outcome focus of ToC, which shows a sequential approach to behaviour change along a social impact chain or program logic (Mayne, 2015; McHugh et al., 2018). This thesis proposes a participatory approach to SIA enabling a reflexive approach to evaluation that allows a deeper understanding of the complexities of how children make healthy life choices. Furthermore, a participatory approach enables program outcomes and evaluation to accurately attribute long-term behaviour change to interventions (Arkesteijn, Van Mierlo, & Leeuwis, 2015). Achieving reflexivity in SIA requires a relational approach to stakeholder engagement, which is often non-linear and iterative, throughout the program or intervention (Esponda et al., 2021; Lane & Devin, 2018; Missonier, 2014). Accordingly, this research highlighted the need to close the gap between theoretical and practitioner knowledge and application. Consequently, two processes should be embedded into intervention design, implementation, and measurement for social impact to be effectively measured in behaviour change interventions. The first is theory use, since theories may extend program outcomes by focusing on constructs known to cause specific behaviours (Abraham et al., 2015; Hardeman et al., 2005; Michie et al., 2011). The second process is deepening essential stakeholder involvement throughout interventions (Dabravolskaj et al., 2020; Hurley, Dietrich, & Rundle-Thiele, 2019) to understand the value of improved health behaviours and outcomes for school community stakeholders such as students, schools, and parental school communities. Subsequently, the fundamental motivation of this thesis was: To examine social impact within primary school health promotion interventions from design to evaluation and understand the role of Social Impact Assessment in contributing to more effective behaviour change interventions within health promotion. A systematic review was conducted, which sought to examine the role of social impact assessment within primary school health promotion interventions. This review considered four comparable Commonwealth countries, Australia, Canada, New Zealand, and the United Kingdom, and aimed to answer the following research question: To what extent has the social impact been considered and evaluated in health promotion interventions in primary school contexts in Australia, Canada, New Zealand, and the United Kingdom? The review found 77 studies representing 55 health promotion interventions delivered in primary school settings. Only eight (15%) measured or attempted to measure the social impact of these interventions, while another eight (15%) alluded to social impact. The prevailing theories reported were social-based theories (theories which examine the social influences on people, environments, and behaviours) (n = 17, 59%), and almost a third of interventions were not informed by an overt health promotion framework or model (n = 34, 59%). A systematic rating system was applied to identify stakeholder engagement (n = 30, 53%). The review highlighted the lack of social impact measurement within health promotion and indicated that social impact is poorly understood in the reviewed academic literature. Study 2 was a Social Return on Investment (SROI) Baseline, which measured the social impact of two NGO Health Promotion modules. This study examined the strengths and limitations of applying social impact measurement within a primary school health promotion intervention. The second research question asked how the application of an SIA, specifically an SROI, contributes insights into the strengths and limitations of social impact measurement within a specific primary school health promotion context. This question explored the theoretical and practical insights social impact assessment (SIA) contributes to estimating social impact within a particular context—specifically, two health promotion modules delivered by [redacted] ([redacted]). The purpose was to apply SIA to understand what community benefits (if any) can be derived from health promotion program delivery, and assess what outcomes and social impact could reasonably be expected from a 90-minute health education module. Although study 2’s findings indicated the strengths of applying an SIA to health promotion, specifically, Social Return on Investments (SROI), there is still a long way to go in both theory and application. The SROI conducted on the two modules demonstrated a positive social impact for the two 90-minute modules. However, this effect dropped off over time. Overall, this research contributes to bridging the divide between academic and practitioner-informed behaviour change interventions. Notwithstanding its predominant health promotion focus, the theoretical parallels and convergences with social marketing are considered, suggesting implications for more robust theory-informed health promotion interventions. This thesis proposes an iterative participatory process involving key stakeholders as salient to the provision of evidence beyond intervention outputs and outcomes. To measure the social impact of an intervention beyond nominal indicators requires a shift in what is measured within behaviour change interventions—individual determinants—to the broader social and economic determinants of health. From a practitioner perspective, key stakeholders must be involved in longer-duration interventions from design through implementation and evaluation. This thesis advocates for a better understanding of social impact relevant to applied settings to more clearly evidence the value of behaviour change interventions demonstrating beyond doubt how ‘wicked problems’ are reduced.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.