Bibliographic record
Abstract
PSE policies Canada (July 2024) Project Team Ray Huang - [Github] [https://orcid.org/0009-0008-1699-6267] Tim Ribaric - [Github] [https://orcid.org/0000-0001-9229-8569] Rahul Kumar - [Github] [https://orcid.org/0000-0002-4247-6045] Policies play a pivotal role in defining the boundaries of what is permissible and what is not. Among these, academic integrity policies are crucial in outlining acceptable and unacceptable behaviours in academic settings. These policies were available in various formats on institutional websites. This repository contains text versions of academic integrity policies from English-language, publicly supported postsecondary education (PSE) institutions in Canada. We recognize that policies often lag behind innovation (e.g., Barzotto et al., 2019; Marcus, 1981; Rodríguez‐Pose & Wilkie, 2018); consequently, this repository also includes guidelines that are sometimes issued to address the disruption caused by GenAI. We aimed to examine their similarities and differences concerning responsibilities and freedoms outlined in the policies and guidelines. In the research for which these policies were collected, we were particularly interested in investigating how they have evolved (or not) in response to the proliferation of generative artificial intelligence (GenAI). Using a computer script, these policies were collected on July 29, 2024 after their location was populated in the attached CSV file. Examining these policies and guidelines using computerized techniques required tokenization to perform Latent Dirichlet Allocation (LDA) and Term Frequency-Inverse Document Frequency (TFIDF) analyses. These processed files are also included in the repository. Details of the various files and their website locations are provided in the CSV file within the repository, and the README.md contains additional pertinent information. Our preliminary results indicate that the policies are indeed trailing the innovation and disruption brought about by GenAI. For more details, please visit our project website where the published results will also be posted. Document Description & Summary That dataset is comprised of the academic integrity policies of English speaking, publically funding, Canadian institutions current to July 29, 2024. Harvested information is categorized into the following: policies - documents that are binding</> guidelines - documents that are not binding but represent best practices, guidelines, etc. followed up policies - documents that are secondary responses Description of files PSE_Policies_Collection.csv A listing of all of the Canadian colleges and universities with a posted Academic Integrity policy investigated in this study. Columns in data: Name of the PSE U15 or not College/University Province URL of the PSE URL2 (filled if instiution has a policy) URL3 (filled if instiution has a guideline) URL4 (filled if instiution has a followed up policy) Name of downloaded policy document (if applicable) Name of the downloaded guideline policy (if applicable) Name of the downloaded followed up policy (if applicable) Texts of Documents Documents were either HTML or PDF file. These were harvested full-text was extracted and put into a text file with the name of institution, and time stamp of original collection from the web concatenated into the name of the file. Tokenization of Documents In order to run LDA analysis the full-text documents were parsed and tokenized using spACy and NLTK. This process lemmatized the text, created bigrams, and removed stopwords. Each token file follows the same naming structure as the extracted full-text with the addtion of _tokens to the end of the filename. References Barzotto, M., Corradini, C., Fai, F., Labory, S., & Tomlinson, P. R. (2019). Enhancing innovative capabilities in lagging regions: An extra-regional collaborative approach to RIS3. Cambridge Journal of Regions, Economy and Society, 12(2), 213-232. https://doi.org/10.1093/cjres/rsz003 Marcus, A. A. (1981). Policy uncertainty and technological innovation. Academy of Management Review, 6(3), 443-448. https://doi.org/10.5465/amr.1981.4285783 Rodríguez‐Pose, A., & Wilkie, C. (2018). Innovating in less developed regions: what drives patenting in the lagging regions of Europe and North America. Growth and Change, 50(1), 4-37. https://doi.org/10.1111/grow.12280
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.003 | 0.011 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.002 | 0.003 |
| Science and technology studies | 0.007 | 0.001 |
| Scholarly communication | 0.009 | 0.004 |
| Open science | 0.002 | 0.003 |
| Research integrity | 0.007 | 0.005 |
| Insufficient payload (model declined to judge) | 0.642 | 0.472 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".