Understanding policy workers’ policy innovation capacity: An exploratory and qualitative mixed methods evaluation study of a policy hackathon program in Prince Edward Island, Canada
Bibliographic record
Abstract
<p><strong>Background: </strong>In 2018, the Government of PEI, Veterans Affairs Canada, Atlantic Canada Opportunities Agency and the Start-Up Zone brought together 49 individuals from the public and private sector to participate in a Policy Hackathon Program. A series of learning sessions were delivered while participants moved through a public policy case competition. This paper evaluates and studies this program and makes design recommendations for future policy hackathon programs. In the process, the paper draws attention to not only the relevance, performance and impact of the Program, but also larger discussions related to the unique attributes of the islandness of public policy, policy innovation, and austerity on an island. </p> <p><strong>Methods: </strong>The evaluation study adopted a social-constructivist worldview, whereby the perceptions of participants and the interpretation of the researcher were used to understand the Program. A qualitative mixed methods design was employed which involved generating qualitative and quantitative data through a pre-program survey (N=48), post-program survey (N=38), interviews with a random sample of participants (N=6), and interviews with a purposive sample of key informants (N=2). Bason’s (2014) design for policy theory and the OECD’s (2017) core skills for public sector innovation framework were operationalized to understand the results in relation to theory and best practice. Quantitative and qualitative results were interpreted by the researcher to understand the Program and also to connect the results to public policy theory and constructs. Results: Relevance The Program responded to a need in PEI’s policy environment. There was clear indication that participants believed that PEI needs new micro-and meso-level policy tools to develop public policy. Participants indicated that having opportunities to learn about policy innovation was important to them. The Program’s emphasis on mentorship was relevant, given that participants believed that such multidisciplinary connections were important for policy development. </p> <p><strong>Performance:</strong> The Program performed well in terms of increasing participants’ individual policy capacity as well as that of the entire group, meeting participants’ expectations to receive valuable learning, and allowing participants to meaningfully connect with a broad range of individuals. The Program performed less optimally in the areas of providing participants with new policy tools, mentorship, and connecting with citizens. </p> <p><strong>Impact:</strong> Participants perceived the Program to have had a positive impact on their skill development in a wide range of areas and in increasing their comfort level with on-the-spot decision-making. Participants indicated that they would seek to integrate similar learning opportunities into their professional development plans in the future. Participants also reported that they believed the Program had a positive impact on the group’s policy capacity and capacity to undertake innovative policy work. Policy Innovation The policy workers involved in the Program (i.e., participants) have cognitively established the positive connection between mentorship and innovation. Participants reported an increase in their confidence to apply human-centered design concepts. In terms of Bason’s (2014) theory and the OECD’s (2017) framework, the Program exposed participants to important policy innovation concepts. Given that participants indicated they thought that individuals who participated in the Program were better prepared to conduct innovative policy work in the future, it is assumed that the Program had a positive impact, to some degree, on increasing the policy innovation capacity of policy workers. Conclusion The study concludes by reiterating that the value of a policy hackathon program is as much related to process as new policies. In other words, in order for policy hackathon programs to be successful, they do PEI Policy Hackathon Program not necessarily need to result in the development of a new policy. Rather, as shown in this study, there can be positive impacts to participants’ policy innovation capacity which can occur during the program. Policy hackathon programs therefore should not be judged entirely on the intervention’s outputs. The study also concludes with a discussion in relation to the islandness of public policy, policy innovation, policy hackathons, and evaluation heuristics. </p> <p>Finally, the paper offers some thoughts on findings which pointed to the existence of austerity and the need for greater citizen-focus in public policy.</p>
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.007 | 0.003 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.003 | 0.007 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.001 |
| Open science | 0.000 | 0.001 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".