An Artificial Intelligence-Based Chatbot to Promote HIV Primary Care Self-Management: a Mixed Method Usability Study
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
<h3>Context:</h3> We developed MARVIN, an artificial intelligence-based chatbot to engage people with HIV in their primary care and support their HIV self-management. <h3>Objective:</h3> To assess its usability and identify the barriers and facilitators to its acceptance. <h3>Study Design and Analysis:</h3> A 4-week pilot study using mixed methods. <h3>Setting:</h3> McGill University Health Centre (Montreal, Canada). <h3>Population studied:</h3> People with HIV on regular treatment. <h3>Intervention/Instrument & Outcome Measures:</h3> Participants were asked to have at least 20 conversations within 3 weeks with MARVIN on predetermined topics and then, to complete the Usability Metric for User Experience-lite (UMUX-lite) and Acceptability E-Scale (AES) surveys. Observed mean scores were compared with predetermined thresholds (68/100 and 24/30, respectively). Qualitatively, randomly selected participants were invited to semi-structured focus groups/interviews to discuss their experiences with MARVIN. Verbatim transcriptions were deductively coded using the constructs of the Consolidated Framework for Implementation Research. Barriers and facilitators were identified according to the four subconstructs of the Technology Acceptance Model (TAM): perceived ease of use, perceived usefulness, attitude toward use, and behavioral intention to use. <h3>Results:</h3> From April to December 2021, 28 participants completed the questionnaires. Their mean age was 40.2 years (SD=11.7), most were male (n=24/28), and over half (n=15/28) preferred to communicate with MARVIN in English. Mean scores for the UMUX-lite and AES were 69.9 and 23.8, both were not significantly below their respective thresholds (p=.76 and p=.42). Nine participants were interviewed. Identified facilitators included user-friendliness, accessibility across devices, confidentiality with a sense of security, and reliability of the information provided. However, lack of topics and functions, limited comprehension, and lack of usage guidance and support were identified as barriers, along with its implementation on only a single platform, Facebook Messenger. <h3>Conclusions:</h3> MARVIN is easy to use, useful, and acceptable as a self-management tool for People with HIV. The qualitative results highlight the enhanced accessibility of relevant information and sense of interaction and safety using MARVIN as facilitating its usability and acceptance, while the quality of information provided, and the technology’s adaptability are factors that require further attention.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.005 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.001 | 0.003 |
| Science and technology studies | 0.002 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.001 | 0.000 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.003 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it