MétaCan
Menu
Back to cohort
Record W6968230646 · doi:10.5281/zenodo.1292849

Hier Stehe Ich! Operationalising Conviction In The Scholarly Commons

2018· article· en· W6968230646 on OpenAlexaffabout

Bibliographic record

VenueZenodo (CERN European Organization for Nuclear Research) · 2018
Typearticle
Languageen
FieldComputer Science
TopicResearch Data Management Practices
Canadian institutionsUniversity of Lethbridge
Fundersnot available
KeywordsScholarshipConvictionCommonsFace (sociological concept)Set (abstract data type)Foundation (evidence)

Abstract

fetched live from OpenAlex

Hier stehe ich! Operationalising conviction in the Scholarly Commons Michael Bar-Sinai, Ian Jeroen Bosman, Utrecht University Ian Bruno, Cambridge Crystallographic Data Centre Chris Chapman, Pentandra Research Solutions Bastian Greshake Tzovaras, Lawrence Berkeley National Laboratory Stephanie Hagstrom, University of California San Diego Nate Jacobs, University of California, Irvine Bianca Kramer, Utrecht University Maryann Martone, University of California San Diego Fiona Murphy, University of Reading Daniel Paul O'Donnell, University of Lethbridge Introduction Open Scholarship is well supplied with manifestos, guidelines, and statements [1–10]. Together, these define a set of practices that comprise an alternative to our current system: content is FAIR (Findable, Accessible, Interoperable, and Reusable), citable, and open; format is determined by the results rather than the demands of tradition or reward systems; credit is based on participation rather than “authorship”; there are no systemic barriers to participation by qualified researchers. The problem is how we develop this alternative coherently and sustainably. Progress over the last thirty years has been uneven across disciplines, regions, and sectors. There remains a sometimes strong disconnect between efforts towards Open Science in the Global South and North. Equity-seeking groups face structural barriers. The Scholarly Commons is an attempt to address this problem. It sets forth a set of principles and rules that stakeholders can use to guide their broad practice as developers, implementers, and users of Open Scholarship and Science. With funding from the Helmsley foundation, the FORCE11 Scholarly Commons Working Group conducted a series of workshops that led to the distillation of almost 90 Open Science charters and manifestos into seven high-level principles that describe actionable behaviour individuals can take to make research open and participatory for anyone anywhere [11]. Our goal is not to replace other statements and guidelines. Rather, it is to provide a means by which participants can navigate, state their allegiance to, and measure their compliance with this broader project---to make explicit the currently often implicit agreement we make when we promote Open Science. By, in essence, badging themselves as members of the Scholarly Commons, researchers commit themselves to open communication and participation across their research activity. This provides an accountability that they can appeal to in addressing legacy systems, and leverage in promoting sustainable Open Scholarship. What if we were to redo things from start? The impetus for this project was Sarah Callaghan’s “1K Challenge” from the Force2015 conference: Redo from start - what would research communication look like after a clean start? Research communication carries with it the weight of 350 years of tradition, still using workarounds for technological limitations from many centuries ago. What would research communication look like if we threw everything out and started again, given current technologies? [12] With the support of the Helmsley foundation and FORCE11, the Scholarly Commons Working Group organised workshops in Madrid and San Diego to look at the question of system wide innovation. We also then tested these proposals and principles against recognised innovations [13] and developed tools and protocols to help in their implementation. The result was a set of 18 “principles” that closely reflected our participants’ sense of inchoate but uncoordinated revolution [14–16]. Like the charters and manifestos they reflect, these statements all promote “openness,” but in ways that compete, contradict, or overlap. Some represent very broad concepts; others focus on specific disciplinary, cultural, or economic contexts. There was a broad desire to effect change; but we missed a coherent, top-level description of what this desire entailed. The Working Group then developed what we argue is the conceptual framework that that underlies these 18 statements. Core is the concept of agreement--that a revolution in scholarly practice requires an agreement within the community as to overall goals and aspirations. These could then be summarised in a small number of rules and principles: P1. The Scholarly Commons is an agreement among knowledge producers and users that P2. Research and knowledge should be freely available to all who wish to use or reuse it, and P3. Participation in the production and use of knowledge should be open to all who wish to participate. In order to effect these principles, participants in the commons agree R1. The rewards for participating in the commons are access, opportunity, and attribution; R2. The commons is agnostic regarding form and technology; R3. (Use of) External systems or technology including reward systems must not harm the commons. These principles portray a system with a different set of dynamics---a new system that works according to “a different logic” [ref]. We then started probing this new system to illuminate these dynamics and to discover how these rules and principles could be particularized for various contexts and stakeholders in ways that build conviction in the Scholarly Commons. 7. An overview of state of the art in the field We are not aware of any similar approach to navigating the Open Science movement as a whole. Most charters and manifestos we have reviewed have their origins in specific disciplines, regions, or industries and focus on one or two aspects of the problem. There are also hundreds of tools that focus on individual aspects of systemic change [13]. This is not evidence of failure. The desire for change we see in the statements from the participants in our workshops and the manifestos requires the kind of focussed, detailed work that characterises this space. Open Science needs open licences, tools, metrics, and processes. We are providing an intellectual infrastructure that allows the community as a whole a way of operationalising their allegiance to these goals. 8. Outcomes thus far or expected outcomes The Scholarly Commons represents an ambitious intellectual infrastructure that provides a context for other initiatives in Open Scholarship. It describes what Open Science advocates are striving for. But it is not confining. Individuals and organizations can differ in the practices they adopt and the speed and completeness of their adoption. Perhaps most importantly, the principles are non-exclusive. While they define a goal and represent a specific attitude, they do not preclude participation in other forms of scholarly communication or by commercial players--as long as this participation does not harm the commons. For example, while the commons itself does not include rewards other than participation, it does not forbid the use of external reward systems (such as the use of publications in promotion decisions at universities) provided these do not reduce the open use and publication of research. At this stage in our work, we believe the principles can provide a broad intellectual framework for understanding the energy and innovation that characterises Open Scholarship. What we are working on now is how they can be used to foster system-wide change. 9. A bibliography 1. The Scholarly Commons Working Group. SC-SCWG Charters - draft list [Internet]. Google Docs. Available: https://docs.google.com/spreadsheets/d/1-aRXFiRg-VL9hpLpxoJqX6-OC-A0R2oCogHfIx52Nug/edit 2. Bosman J, Bruno I, Chapman C, Tzovaras BG, Jacobs N, Kramer B, et al. The Scholarly Commons - principles and practices to guide research communication [Internet]. OSF Preprints. 2017. doi:10.17605/OSF.IO/6C2XT 3. Open Source Initiative. The Open Source Definition (Annotated). In: Open Source Initiative [Internet]. 2007 [cited 27 Jan 2018]. Available: https://opensource.org/osd-annotated 4. Max-Planck Gesellschaft. Berlin Declaration. In: Max Planck Open Access [Internet]. 22 Oct 2003 [cited 27 Jan 2018]. Available: https://openaccess.mpg.de/Berlin-Declaration 5. Priem J, Taraborelli D, Goth P, Neylon C. Altmetrics: A manifesto [Internet]. Altmetrics.org; 2010 Oct. Available: http://altmetrics.org/manifesto/ 6. OANA working group Open Access. Vienna Principles a vision for scholarly communication [Internet]. Jun 2016 [cited 27 Jan 2018]. Available: http://viennaprinciples.org/ 7. Force11. Guiding Principles for Findable, Accessible, Interoperable and Re-usable Data Publishing version b1.0. In: FORCE11 [Internet]. 10 Sep 2014 [cited 27 Jan 2018]. Available: https://www.force11.org/fairprinciples 8. Hicks D, Wouters P, Waltman L, de Rijcke S, Rafols I. Bibliometrics: The Leiden Manifesto for research metrics. Nature. 2015;520: 429–431. doi:10.1038/520429a 9. Open Knowledge Open Definition Group. Open Definition 2.1 - Open Definition - Defining Open in Open Data, Open Content and Open Knowledge [Internet]. [cited 27 Jan 2018]. Available: http://opendefinition.org/od/2.1/en/ 10. Adie E, Ahmad S, Alberts B, Antin P, Bergfeld E, Bertuzzi S, et al. San Francisco Declar

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.

How this classification was reachedexpand

Full frame machine prediction

Teacher imitation

Not calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.

metaresearch head score (Codex)0.039
metaresearch head score (Gemma)0.144
Version: metacan-v3-hybrid-931329e0061cValidation status: machine_predicted_unvalidated
Candidate categoriesMetaresearch, Scholarly communication
Consensus categoriesnone
DomainCandidate signal: Reproducibility · Consensus signal: none
Study designCandidate signal: Qualitative · Consensus signal: none
GenreCandidate signal: Empirical · Consensus signal: none
Teacher disagreement score0.973
Threshold uncertainty score0.209

Distilled classifier scores by category (both heads)

CategoryCodexGemma
Metaresearch0.0390.144
Meta-epidemiology (narrow)0.0000.000
Meta-epidemiology (broad)0.0010.001
Bibliometrics0.0060.006
Science and technology studies0.0120.031
Scholarly communication0.0270.024
Open science0.0020.026
Research integrity0.0040.007
Insufficient payload (model declined to judge)0.0440.013

Machine scores (provisional)

The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.

Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.

Opus teacher head0.108
GPT teacher head0.317
Teacher spread0.209 · how far apart the two teachers sit on this one work
Validation statusscore_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it

Classification

machine, unvalidated

Machine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.

Study designQualitative
DomainReproducibility
GenreEmpirical

How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".

Quick stats

Citations0
Published2018
Admission routes2
Has abstractyes

Explore more

Same venueZenodo (CERN European Organization for Nuclear Research)Same topicResearch Data Management PracticesFrench-language works237,207