MétaCan
Menu
Back to cohort
Record W2963572936

Give me the place to stand: Leverage analysis in systemic design

2018· other· en· W2963572936 on OpenAlexaff
Ryan J. Murphy, Peter Jones

Bibliographic record

VenueOCAD University Open Research Repository (OCAD University) · 2018
Typeother
Languageen
FieldDecision Sciences
TopicComplex Systems and Decision Making
Canadian institutionsOntario College of Art and Design
Fundersnot available
KeywordsLeverage (statistics)Computer scienceBusinessArtificial intelligence
DOInot available

Abstract

fetched live from OpenAlex

A key component of many systemic design processes is the development and analysis of systems models that represent the issue(s) at hand. A system is a collection of interdependent social, technological, and environmental phenomena. Models of systems often take the form of Causal Loop Diagrams (CLDs—sometimes referred to as influence diagrams) in which phenomena are graphed as nodes with connections between them indicating an influencing relationship. These visual modelling techniques provide systemic designers with a mechanism for stakeholder collaboration, problem finding, and generative insight (i.e., sticky note ideation makes everyone feel heard, appears democratic, and often results in emergent themes and ideas). These functions are valorized in design thinking, and they provide real value in garnering momentum and achieving common mental models in complex problems. They give systemic designers powerful resources for use in visual argument.
\n
\nHowever, while we believe these tools are useful, we also believe their true potential is unfulfilled. The properties of complex systems (and of how people engage with them) present a number of issues that introduce bias and chance into this process (Norman & Stappers, 2015). Given a model, systemic designers work through what they observe and interpret, engage in dialogue about what is important, and look for patterns (one category of which is archetypes, in which phenomena following certain patterns tend to produce similar emergent behaviours; Braun, 2002). While some principles and processes exist (see Jones, 2014), identifying leverage points and designing solutions tends to happen by “muddling through” a problem. This means solutions are developed and implemented in opportunistic form, through satisficing rather than optimizing (Norman & Stappers, 2015; see also Simon, 2008, chapter 2). Thus, we find a critical value gap: models are used in visual argument, but they could be used to augment those very arguments founded on evidence and logical relationship analysis.
\n
\nWe propose the application of semi-quantitative analytics to systemic design models to go beyond visual argument, offering a powerful toolkit for:
\nComprehensive system mapping for complex sociotechnical systems (including the development of reference models that can inform synthesis/Gigamaps, or that can be used as their own arguments);
\nNetwork-based analysis to uncover key structures, relationships, and latent leverage positions of modelled phenomena;
\nAnalytical mapping of problem systems and sorting out multicausality; A toolkit for cross-impact analysis between problematiques; and A “reality check” on strategic foresight proposals (by mapping temporal changes in networks and problematiques, we can better predict signal -> trend outcomes).
\n
\nWith these analytics, models may be rethought in terms of the logics of leverage to reconcile this value gap.
\nWe introduce (or at least renew emphasis) on centrality analysis (metrics derived from social network analysis, evaluating the relative importance of mapped phenomena through measuring the structure of the directed graph made by the phenomena) and decomposition heuristics (algorithms derived from systems dynamics that analyze the directed graph structure to reveal the causal and loop hierarchy of modelled systems) in systemic design.
\n
\nTo demonstrate the application of centrality analysis, we map the interconnectivity of the Sustainable Development Goals (SDGs) and their targets based on the work of Le Blanc (2015). By using metrics adopted from social network analysis, we are able to differentiate between goals and targets of differing levels of importance based on the structure of the map. Phenomena closeness (how proximate a given element is to the rest of the map) provides a ranked list of key indicators of change in the mapped system. Eigenvector (how well-connected an element is to other well-connected elements) analysis provide a ranked list of highly connective forces in the system: potential leverage points. These metrics therefore help identify which goals and targets to watch and which to intervene on the process of creating systemic change in the SDGs.
\n
\nTo demonstrate the application of decomposition heuristics, we create a level partition (a hierarchy of causal structure of a map) and a loop inclusion graph (a hierarchy of feedback loop subsystems nested within one another) from feedback loops modelled in previous work on education systems change (Murphy, 2016). The level partition only decomposes the system into two levels, showing the strongly connected nature of the modelled phenomena in the system at hand. The loop inclusion graph, however, shows that certain feedback loops dominate the feedback loops they are contained within. Understanding—and intervening upon—these dominant loops should take precedence over their subsidiaries.
\n
\nThe potential value in combining these tools should be clear. Decomposition heuristics can be used to break down the structure of modelled systems, making clear hierarchies and isolated systems within systems that sometimes disappear in the hairball complexity of these models. Likewise, centrality analytics can indicate key indicators, leverage points, bottlenecks, and other useful phenomena in the system. Taken together, isolated, dominant subsystems with high rankings on centrality measures tell systemic designers exactly where to stand in order to move their systems.
\n
\nThe resolution of this value gap is particularly important as we see growth in the use of systemic design—and the technologies that support its practice. In order to develop models of systems that accurately represent the many stakeholders involved in the system, systemic designers must draw on diverse sources to collect and organize as much data as possible (Jones, 2014; Stroh, 2015). Fortunately, thanks to the development of recent technologies and practices such as crowdsourcing (the development of participatory systems that involve publics in a collaborative project, usually directed by a project owner; Lukyanenko & Parsons, 2012) and data science (a set of techniques and theories that help distill insight from data; Šćepanović, 2018), the collection and organization of large amounts of data will become ever easier.
\n
\nThis brings us to an important paradox. Larger, more complex, data-driven models are likely more representative, as they capture more perspectives and nuances than simpler models. At the same time, larger, more complex models are harder to learn and understand (Rossi & Brinkkemper, 1996), and therefore they are also harder to use in the development of solutions. Thus, the tools we propose come at a crucial moment for leverage analysis in systemic design. Their advancement and provisioning could elevate the potential of the tools at the core of the discipline. With this careful rethinking of the logics of leverage, we might make better arguments for change, finding the place to stand from which to move the world.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.

How this classification was reachedexpand

Full frame distilled prediction

Teacher imitation

Not calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.

metaresearch head score (Codex)0.010
metaresearch head score (Gemma)0.001
Version: codex-gemma-dda1882f352aValidation status: machine_predicted_unvalidated
Candidate categoriesMeta-epidemiology (narrow), Science and technology studies, Scholarly communication, Open science, Insufficient payload (model declined to judge)
Consensus categoriesnone
DomainCandidate signal: none · Consensus signal: none
Study designCandidate signal: Not applicable · Consensus signal: Not applicable
GenreCandidate signal: Other · Consensus signal: Other
Teacher disagreement score0.436
Threshold uncertainty score1.000

Codex and Gemma teacher scores by category

CategoryCodexGemma
Metaresearch0.0100.001
Meta-epidemiology (narrow)0.0010.000
Meta-epidemiology (broad)0.0010.000
Bibliometrics0.0080.014
Science and technology studies0.0020.001
Scholarly communication0.0020.001
Open science0.0110.003
Research integrity0.0010.001
Insufficient payload (model declined to judge)0.0000.001

Machine scores (provisional)

The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.

Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.

Opus teacher head0.268
GPT teacher head0.413
Teacher spread0.145 · how far apart the two teachers sit on this one work
Validation statusscore_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it

Classification

machine, unvalidated

Machine predicted; a candidate call from one teacher head, not a consensus.

Study designNot applicable
Domainnot available
GenreOther

How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".

Quick stats

Citations0
Published2018
Admission routes1
Has abstractyes

Explore more

Same venueOCAD University Open Research Repository (OCAD University)Same topicComplex Systems and Decision MakingFrench-language works237,207