Strengthening health systems through networks: the need for measurement and feedback

Health policies result from a complex interplay between social, political, physical, ecological, biological, cultural, technical and economic factors (Atkins et al. 2005; Clancy and Cronin 2005; Sterman 2006; McCaughey and Bruning 2010). Negotiating the inputs from these various sources often is done under temporal and resource pressures, complicated by political and social upheaval. Populations therefore feel the effects of health policies unevenly: some may benefit, some may suffer, while others are simply excluded. Conceptualizing the effects a policy decision may have requires ways of thinking, working and monitoring that acknowledge the importance of relationships. The World Health Organization’s (WHO) model of health systems strengthening is an important step in this direction (de Savigny and Adam 2009). In this model, understanding and strengthening the relationships between health system ‘building blocks’ is seen as critical for mitigating unintended effects such as negative behaviours and exclusion, identifying opportunities for synergy, and generating health policies that are ‘system ready’ (de Savigny and Adam 2009). In support of this vision, we contend that network-centric approaches that foster integration, innovation and local creativity hold much promise for strengthening health systems and health policy development in low- and middle-income countries (LMIC). However, we suggest that effective monitoring and evaluation techniques for learning about the performance of networks are lacking or underutilized. We therefore call for greater investments in developing robust measurement and accountability strategies for ensuring that network modes of collaboration lead to improved system performance and enhanced health outcomes. It is perhaps little surprise that many well-intentioned policies fail to address the problems they are designed to solve, and in attempting to do so, actually generate new ones. As illustrated by Agyepong et al. in this supplement, policy decisions are too often built on ‘crisis driven linear decision making’ (Agyepong et al. 2012). The tendency for these policies to be defeated by the response of the system to the policy itself is known as ‘policy resistance’, and results in part from ‘a narrow, reductionist worldview’ built on linear, mechanistic thinking that reduces problems to their component parts (Sterman 2006; Leischow et al. 2008). However, when problems cannot be broken down, when they are not just the parts but how they are put together, when behaviours are unpredictable and when non-linear ‘effects’ are felt in remote parts of the system, designing effective policy interventions becomes difficult. What is needed is a way of thinking that recognizes that parts are not disconnected from the whole and that dynamic relationships exist which shape, and are shaped, by the environments in which they are embedded. Systems thinking, while not a panacea to poor decision-making, offers a lens of enquiry through which a system may be viewed as more than the sum of its parts. Sterman (2006) outlines systems thinking ‘ … as an iterative learning process in which we replace a reductionist, narrow, short-run, static view of the world with a holistic, broad, long-term, dynamic view, reinventing our policies and institutions accordingly’ (Sterman 2006). The dynamic characteristics of systems thinking afford insights into the nature of policy decisions, including their potential positive, negative, short-term and long-term effects. Systems thinking emphasizes the importance of relationships and the unpredictable behaviours that arise from interactions between system components (Plsek and Greenhalgh 2001; Trochim et al. 2006; National Cancer Institute 2007). Planning grounded in systems thinking promotes the importance of multilevel and multi-actor methods and the use of tools (e.g. system dynamics modelling, social network analyses) to better understand system behaviours and the ill-defined boundaries of health systems (Plsek and Greenhalgh 2001). However, thinking in systems challenges the way we traditionally do business in health systems. New strategies and approaches to organizing efforts threaten long established norms, habits and assumptions (Senge and Sterman, 1992). The siloed structures of the past, built in part to preserve professional autonomy, are difficult to align with the interprofessional approaches demanded by systems thinking. Creating agile, responsive, intelligent systems that learn from feedback, reduce unnecessary duplication and harness the creativity of those within and beyond the traditional borders of health systems, therefore, demands new ways of working. These new modes of working must seek to integrate across disciplines, hierarchies, departments and specialties. They must identify appropriate partners, grow relationships and build local capacity. And they must learn from feedback, adjust, adapt, dissolve and regenerate to meet the changing needs of health systems. Interorganizational networks have emerged at the global level, as well as at national and subnational levels, as important strategies for organizing human effort in a systems-thinking mode of operation. The rise in network popularity has come largely from the recognition that money alone cannot sufficiently improve the quality of health systems, and that the major health problems facing societies are unlikely to be successfully addressed by individual organizations acting in isolation (CHSRF 2011; Provan et al. 2011). Only through pooling of resources, talents and strategies from across a range of actors and organizations, may population health be improved (Woulfe et al. 2010). Multiple definitions of the network concept exist, leaving some to question whether or not networks genuinely represent a unique working structure (Borgatti and Foster 2003). Brass et al. provide a generalized perspective that is helpful at a broad conceptual level: that networks are ‘a set of nodes and the set of ties representing some relationship, or lack of relationship, between the nodes’ (Brass et al. 2004). Relationships between nodes are typically non-hierarchical, and may be founded on many and varied factors, including formal or informal flows of resources, information, people or ideas (Huerta et al. 2006). Over time, the distinction between networks and other relationship-based structures, such as partnerships, alliances, coalitions and collaborations, has become blurred (Riley and Best, in press). While it is not our intention with this commentary to delineate between these concepts, it is important to note that each is founded on the importance of ‘social interaction (of individuals acting on behalf of their organizations), relationships, connectedness, collaboration, collective action, trust, and cooperation’ (Provan et al. 2007). The study of networks has provided important learning for how people and groups might work together either in spontaneously evolving networks, or through formal establishment of network partnerships. The structure of both naturally evolving and mandated networks may be classified using a range of approaches, including that proposed by Huerta et al. (2006) that considers networks to lie on a continuum between those that are purely exploratory (concerned with creating knowledge) and those that are purely exploitative (concerned with using resources to achieve a given purpose) (Huerta et al. 2006). Furthermore, networks may be thought of in terms of being conceptually focused (creating ‘conceptual or methodological discourse’) or implementation focused (primarily aiming to deliver services) (Huerta et al. 2006). Yet as noted by Provan and Milward, networks are rarely ‘one or the other’, with different networks taking on different roles over time and in response to changing circumstances (Provan and Milward 2006). Interorganizational networks are well aligned with the World Health Organization’s (WHO) conceptualization of health systems, which places people at the centre of health systems, supported by multiple health system ‘building blocks’ (de Savigny and Adam 2009). Networks will differ in the emphasis they place on the co-involvement of these building blocks. Network purpose is key to deciding on which structures to involve and to what extent. In public health settings, Woulfe et al. (2010) describe three general network classifications: (1) public health agencies partnering with others to extend programme reach; (2) health service delivery networks built on provider collaborations; and (3) multi-sector networks involving a range of actors/agencies each aiming to influence population health (Woulfe et al. 2010). Within each classification, network formation may cross disciplinary and geopolitical boundaries to form linkages (such as between researchers, practitioners, governments and non-government organizations), and may involve: integration of activities and funding between public and private sources; links between research and development teams; and/or training partnerships between educational providers. Regardless of purpose, network viability depends on the ability to demonstrate that these modes of working are indeed legitimate, that they enhance the effectiveness and efficiency of system performance, and that they ultimately improve the health status of populations. There has been considerable growth in the number and scope of interorganizational network activities in LMIC settings, with both global and in-country level impacts. For example, the Health Systems Action Network (HSAN) was established to act as a global health systems strengthening network, employing diverse stakeholder collaborations as a way of improving the use of evidence, avoiding duplication of effort and fostering in-country capacity for building ‘equitable, accountable and sustainable health systems for improved health outcomes’ (Health Systems Action Network 2012). WHO’s Evidence Informed Policy Network (EVIPNet) has similar intentions, aiming to build capacity at local levels to synthesize evidence into policy briefs, facilitate shared learning at national level forums, and to mobilize global support from funders, researchers and knowledge translation experts [Evidence Informed Policy Network (EVIPNet), 2012]. EVIPNet represents one of four initiatives in the ‘Health Information For All’ (HIFA), which now has 5000 members from more than 2000 organizations and 158 countries (Smith and Koehlmoos 2011). Yet, funding for network activities such as HIFA remains elusive, with some suggesting a greater role for public–private partnerships to provide support and sustainability for knowledge-based collaborations. Learning from these partnerships in intervention-specific networks such as the Global Alliance for Vaccines and Immunizations (GAVI Alliance), therefore, may be useful in understanding how intervention-focused networks may be broadened to incorporate health system strengthening activities, and contribute to improved use of knowledge, more integrated providers, and greater reach of public health services (Naimoli 2009). Critical for this success is the development of funding mechanisms (such as the International Finance Facility for Immunization) that ensure on-going access (with accountability) to long-term resource streams (Clemens et al. 2010). In each example, careful consideration of the policy transfer dynamics from global network activities to national level networks is required to ensure sensitivity to local contexts and the creation of enabling environments that optimize the impact of systems interventions. Appropriate governance structures therefore are needed at global, national and subnational levels to foster commitment among network members and promote accountability through shared measures of network performance. Despite the growth and diversity of network experiments in LMIC and high-income country (HIC) settings, our understanding of network performance remains limited, potentially undermining investments being made in network initiatives. Recently, identifying the optimal approach to developing and supporting interorganizational collaborations in LMICs has been highlighted as a primary challenge for health systems strengthening activities (Sundewall et al. 2011). With only modest evidence that collaborative networks are effective, research is needed urgently to ensure current and future network efforts do not become ‘yet another ineffective talking shop that [do] more harm than good’ (The Lancet 2006; Woulfe et al. 2010). To guide these efforts, a systems thinking process structured around a systems intervention framework may be useful [such as that described by Meadows (2008) and adapted by Finegood (2011)]. In Finegood’s Intervention Level Framework, system behaviour is thought to be shifted through interventions on five levels: paradigm, goals, system structure, feedback loops and structural elements (Finegood 2011). In complex problems, such as obesity prevention, relatively little consideration is given to interventions aiming to modify feedback structures. Yet, these interventions—grounded in the importance of measurement and information use—may have significant and lasting effects on system performance (Finegood 2011). Intervening at the level of feedback therefore might provide powerful leverage for strengthening the role of interorganizational networks in health systems. Through a systems thinking lens, measures of network performance and accompanying feedback loops provide opportunities for learning. They also build accountability. However, developing strategies for measuring network performance has been hampered by multiple factors, including the often short evaluative periods requested by funders, difficulties in measuring the degree of ‘exposure’ among participants, and the reliance on broad population-based indicators (Woulfe et al. 2010). Moreover, despite growing consensus around the elements that are essential for understanding network performance, agreed on strategies for actually measuring these elements and then using these data are lacking (Woulfe et al. 2010). To increase the role and value of networks in health systems, a clearer understanding of two issues is required: firstly, how should network performance be measured in ways that promote broad stakeholder learning; and secondly, how can these measures be built into appropriate accountability structures in order to strengthen network and health system performance? A systematic, transparent and rigorous approach to examining the effects of networks on national or subnational health system performance and population health status is urgently needed. This is not a simple task, given the variety of network structures that exist, their often non-hierarchical natures, the contextual factors at play, the political sensitivity in developing networks, their purposes, boundaries and organizational members. As such, an evaluative approach built on systems thinking concepts is required for recognizing the complex factors influencing network performance and the dynamic, non-linear and interrelated nature of network activities. Measures of structure (such as gained through social network analyses) may be important for giving insights into network development, relationship strength and member involvement. Longitudinal network analyses provide rich insights into how measures of network structure (such as centrality, density and clique sub-structure) may change and evolve over time in response to both internal and external pressures (Friedman et al. 2007). As important as these measures are, they do not sufficiently describe network functions, processes or outcomes. Provan and Milward (2001) outline an evaluative framework for assessing network outcomes based on three levels: community, network and organization (Provan and Milward 2001). For advancing the measurement of network performance, this framework is a logical start. Community level outcomes may be measured through the contributions made by the network to communities, including costs incurred, public perceptions of network performance, and aggregate indicators of health status in the population. Network level effectiveness is more concerned with the legitimacy of the network itself and may be measured by growth in member organizations, services or activities provided, integration/co-ordination between activities (and with that reduced duplication), and member commitment to network goals. Organizational effectiveness is primarily concerned with the survival and continued success of member organizations, the outcomes of individual clients they serve, their ability to acquire new resources, and costs of network participation (Provan and Milward 2001). A comprehensive network performance measurement strategy will therefore need to assess each level of ‘outcome’ in addition to measures of network structure and process. This approach is likely to be highly context-sensitive with different measures gaining different prominence in different network settings. Therefore, determining a set of common measures or categories of measures will require a collaborative approach that is cognisant of the needs and incentives of a diverse range of stakeholders (Riley et al. 2012). One approach to doing so might first generate the ‘simple rules’ for evaluating network performance, thereby providing options that allow local flexibility and creativity, while generating meaningful and actionable measures for performance improvement. In both LMIC and HIC settings, generating feedback mechanisms that facilitate access to timely, feasible, cost-effective and actionable performance data will be essential for informing network improvements and ensuring clear accountability structures. A second challenge in measuring and improving network performance relates to how data are used; specifically how performance metrics are incorporated into accountability structures that promote the pursuit of high level network performance. Critically, accountability ‘does not show lack of trust, but is evidence that what one is doing really matters’ (Ganz 2010). As such, accountability structures may be best seen as useful sources of feedback for informing strategy and action. In networks involving global partners as well as those at national and subnational levels, accountability structures are difficult to conceive and operate. Framed by systems thinking, the feedback mechanisms through which accountability may be generated require consideration of: how to use what data, by whom and when; what proportion of data to use; how to direct this use; and how to ensure data are used appropriately and effectively (Ottoson and Wilson 2003). To generate these feedback systems, how data are used just needs to be shifted from being an to being a (Ottoson and Wilson 2003). the development of common it is essential for accountability and feedback structures to be built so that decisions the and of feedback structures are with from The need to involve stakeholders potential data is important in monitoring the performance of networks, collaborative and often non-hierarchical structures challenges for traditional performance measurement by whom and when metrics should be used to performance requires collaborative to the linkages between data, decisions and (Ottoson and Wilson 2003). In global networks as well as those at the country level, determining appropriate structures is For network performance metrics may be and to the network, and are therefore of use network members for learning and network for determining internal resource flows or network or policy for informing other network efforts, evaluating network or the of continued network These varied require a clear understanding of how network accountability processes with (and different sources of data, different on and modes of network factors will likely lead to feedback structures in networks by shared lead organization or those with organizations (Provan and 2007). feedback structures need to ensure data are not only and but that they are and actionable for in different network settings. between performance measurement feedback and accountability and governance structures is therefore important future work for advancing the value gained from collaborative health systems for improved policy in LMIC requires efforts from many working across multiple The of health systems approaches that are grounded in systems thinking at the global, national and subnational levels, and that the need for ways of working that promote integration than Network structures represent a broad set of collaborative approaches that are useful for stakeholders Health systems in LMIC and HIC will be we are to and systems thinking concepts to the monitoring and evaluation of Critical challenges exist in developing measurement tools and feedback mechanisms that not only provide opportunities for but that also build accountability into the the measures of network performance and the feedback systems to their use need to be with and for those will use these data, to meaningful measures that are of improving network and system performance. The of this is supported by the Alliance for Health Policy and Systems World Health The alone are for the in this and they do not represent the decisions or policies of the World Health was supported by funding from the National Health and of the ‘Health in The and a

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.