Examining Nonverbal Communication in Dyadic Interactions With Virtual Humans Using an Integrated Coding System: Mixed Methods Analysis
Bibliographic record
Abstract
Background: The patient-physician dyad involves both verbal and nonverbal communication. Traditional methods use quantitative or qualitative coding when analyzing dyadic data of nonverbal communication. Quantitative coding methods can capture the frequency of nonverbal communication, while qualitative coding methods can provide descriptive information on the context and nuance of the nonverbal communication expressed. Yet, limited research has examined the integration of quantitative and qualitative coding methods of nonverbal communication between a patient-physician dyad through video recordings. Objective: The objective of this formative study was to demonstrate how nonverbal communication data can be analyzed using a mixed methods analysis approach and propose an integrated coding system using a subset of the original dataset. Methods: A secondary analysis was conducted from the intervention study with a sample of 32 pairs of randomly selected video recordings based on first and second interactions after receiving feedback from a virtual human. A 2-minute segment was used to code nonverbal communication, and a codebook was developed, informed by the literature and inductive qualitative approaches. For the mixed methods analysis, we purposefully selected 2 participants from the sample of 32 who demonstrated high frequency in quantitative and qualitative coding of nonverbal behaviors. We developed a joint display to visually represent the integration of quantitative and qualitative coding methods and developed person-level meta-inferences. Results: This formative study demonstrates an approach to nonverbal communication analysis that mixes qualitative and quantitative methods. The mixed methods results indicated the frequency of participants' (n=32) nonverbal behaviors increased after repeated interactions, including eyebrow raise, nodding, and smiling, in addition to the increased average duration of nonverbal behaviors across interactions. Illustrated through an in-depth example of integrated mixed methods coding of 2 participants from the sample, the integration of quantitative and qualitative data provided insights into nonverbal communication. Quantitatively, we captured the frequency of nonverbal behaviors while qualitatively expanding on the context for nonverbal behaviors and generating person-level meta-inferences. The joint display informed our integrated coding system for mixed methods analysis of nonverbal communication. Conclusions: The resultant integrated coding system may be helpful to researchers engaging in nonverbal communication data of dyads by providing a step-by-step method using a mixed methods analysis approach. This approach can help us to advance methods for analyzing nonverbal communication to enhance the patient-physician dyad and education on nonverbal communication. We encourage applying the integrated coding system across several subdisciplines in health sciences research to identify how it can be further expanded and refined.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.184 | 0.297 |
| Meta-epidemiology (narrow) | 0.002 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.004 |
| Bibliometrics | 0.011 | 0.011 |
| Science and technology studies | 0.003 | 0.003 |
| Scholarly communication | 0.005 | 0.003 |
| Open science | 0.004 | 0.005 |
| Research integrity | 0.001 | 0.002 |
| Insufficient payload (model declined to judge) | 0.004 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".