Introducing Novel Methods to Identify Fraudulent Responses (Sampling With Sisyphus): Web-Based LGBTQ2S+ Mixed-Methods Study
Bibliographic record
Abstract
BACKGROUND: The myth of Sisyphus teaches about resilience in the face of life challenges. Detransition after an initial gender transition is an emerging experience that requires sensitive and community-driven research. However, there are significant complexities and costs that researchers must confront to collect reliable data to better understand this phenomenon, including the lack of a uniform definition and challenges with recruitment. OBJECTIVE: This paper presents the sampling and recruitment methods of a new study on detransition-related phenomena among lesbian, gay, bisexual, transgender, queer, and 2-spirit (LGBTQ2S+) populations. It introduces a novel protocol for identifying and removing bot, scam, and ineligible responses from survey datasets and presents preliminary descriptive sociodemographic results of the sample. This analysis does not present gender-affirming health care outcomes. METHODS: To attract a large and heterogeneous sample, 3 different study flyers were created in English, French, and Spanish. Between December 1, 2023, and May 1, 2024, these flyers were distributed to >615 sexual and gender minority organizations and gender care providers in the United States and Canada, and paid advertisements totaling >CAD $7400 (US $5551) were promoted on 5 different social media platforms. Although many social media promotions were rejected or removed, the advertisements reached >7.7 million accounts. Study website visitors were directed from 35 different traffic sources, with the top 5 being Facebook (3,577,520/7,777,218, 46%), direct link (2,255,393/7,777,218, 29%), Reddit (1,011,038/7,777,218, 13%), Instagram (466,633/7,777,218, 6%), and X (formerly known as Twitter; 233,317/7,777,218, 3%). A systematic protocol was developed to identify scam, nonsense, and ineligible responses and to conduct web-based Zoom video platform screening with select participants. RESULTS: Of the 1377 completed survey responses, 957 (69.5%) were deemed eligible and included in the analytic dataset after applying the exclusion protocol and conducting 113 virtual screenings. The mean age of the sample was 25.87 (SD 7.77; median 24, IQR 21-29 years). A majority of the participants were White (Canadian, American, or of European descent; 748/950, 78.7%), living in the United States (704/957, 73.6%), and assigned female at birth (754/953, 79.1%). Many participants reported having a sexual minority identity, with more than half the sample (543/955, 56.8%) indicating plurisexual orientations, such as bisexual or pansexual identities. A minority of participants (108/955, 11.3%) identified as straight or heterosexual. When asked about their gender-diverse identities after stopping or reversing gender transition, 33.2% (318/957) reported being nonbinary, 43.2% (413/957) transgender, and 40.5% (388/957) identified as detransitioned. CONCLUSIONS: Despite challenges encountered during the study promotion and data collection phases, a heterogeneous sample of >950 eligible participants was obtained, presenting opportunities for future analyses to better understand these LGBTQ2S+ experiences. This study is among the first to introduce an innovative strategy to sample a hard-to-reach and equity-deserving group, and to present an approach to remove fraudulent responses.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.165 | 0.207 |
| Meta-epidemiology (narrow) | 0.002 | 0.002 |
| Meta-epidemiology (broad) | 0.001 | 0.002 |
| Bibliometrics | 0.004 | 0.003 |
| Science and technology studies | 0.004 | 0.003 |
| Scholarly communication | 0.004 | 0.003 |
| Open science | 0.004 | 0.005 |
| Research integrity | 0.003 | 0.003 |
| Insufficient payload (model declined to judge) | 0.012 | 0.003 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".