An Artificial Intelligence-Based Chatbot to Promote HIV Primary Care Self-Management: a Mixed Method Usability Study
Bibliographic record
Abstract
Context: We developed MARVIN, an artificial intelligence-based chatbot to engage people with HIV in their primary care and support their HIV self-management. Objective: To assess its usability and identify the barriers and facilitators to its acceptance. Study Design and Analysis: A 4-week pilot study using mixed methods. Setting: McGill University Health Centre (Montreal, Canada). Population studied: People with HIV on regular treatment. Intervention/Instrument & Outcome Measures: Participants were asked to have at least 20 conversations within 3 weeks with MARVIN on predetermined topics and then, to complete the Usability Metric for User Experience-lite (UMUX-lite) and Acceptability E-Scale (AES) surveys. Observed mean scores were compared with predetermined thresholds (68/100 and 24/30, respectively). Qualitatively, randomly selected participants were invited to semi-structured focus groups/interviews to discuss their experiences with MARVIN. Verbatim transcriptions were deductively coded using the constructs of the Consolidated Framework for Implementation Research. Barriers and facilitators were identified according to the four subconstructs of the Technology Acceptance Model (TAM): perceived ease of use, perceived usefulness, attitude toward use, and behavioral intention to use. Results: From April to December 2021, 28 participants completed the questionnaires. Their mean age was 40.2 years (SD=11.7), most were male (n=24/28), and over half (n=15/28) preferred to communicate with MARVIN in English. Mean scores for the UMUX-lite and AES were 69.9 and 23.8, both were not significantly below their respective thresholds (p=.76 and p=.42). Nine participants were interviewed. Identified facilitators included user-friendliness, accessibility across devices, confidentiality with a sense of security, and reliability of the information provided. However, lack of topics and functions, limited comprehension, and lack of usage guidance and support were identified as barriers, along with its implementation on only a single platform, Facebook Messenger. Conclusions: MARVIN is easy to use, useful, and acceptable as a self-management tool for People with HIV. The qualitative results highlight the enhanced accessibility of relevant information and sense of interaction and safety using MARVIN as facilitating its usability and acceptance, while the quality of information provided, and the technology’s adaptability are factors that require further attention.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.018 | 0.023 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.001 | 0.000 |
| Insufficient payload (model declined to judge) | 0.002 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".