Methodological Challenges to Collecting Valid Research Data: Researching English Language Barrier Populations in Canada
Bibliographic record
Abstract
Background: Canada has two official languages, English and French. French is the main language in Quebec while English is widely spoken in other provinces and territories. Canadian census data has shown that there is a significant portion of the population in the English speaking provinces that identify French as their mother tongue; and prefer to be served in French when they seek healthcare. This means that they are not able to speak, read, write, or understand the English language in a way that enables them to interact with others daily in English, let alone participate in research conducted in English. Thus the designation, English Language Barrier (ELB). The ELB populations can be difficult to sample for research purposes especially when language affiliation becomes an important identifying variable, thus, can be described as a potentially hard to-reach populations. Inadequate sampling compromises quality of data from which inferences are derived. Aim: The purpose of this paper is to present challenges to collecting valid research data and suggest ways by which such challenges can be overcome in ELB populations in general. Methods: To achieve this, I start by defining ELB populations, followed by a brief description of what constitutes valid data. Subsequently, I discuss evidence on data collection practices and identify major challenges. In the discussion, I present ways by which these challenges can be mitigated; and conclude with a summary of the evidence, the challenges, and suggestions to improve data collection in general. Conclusions: Collection of valid research data among ELB populations can be enhanced by using innovative sampling techniques such as respondent driven sampling, reducing bias and improving on the questionnaire.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.336 | 0.548 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.002 |
| Bibliometrics | 0.007 | 0.014 |
| Science and technology studies | 0.021 | 0.013 |
| Scholarly communication | 0.014 | 0.008 |
| Open science | 0.010 | 0.011 |
| Research integrity | 0.004 | 0.007 |
| Insufficient payload (model declined to judge) | 0.003 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".