Dynamic Estimation of Mental Workload and Operator Accuracy for Time-Constrained Binary Classification Tasks
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
Human cognitive states, such as mental workload, play a pivotal role in decision making processes within human automation teams. Although subjective measures of mental workload can be obtained using standard questionnaires, such as the NASA-TLX, their administration is often impractical as it interferes with the primary tasks of the human operator. Therefore, it is of interest to estimate these subjective measures from less intrusive observations. Evidence suggests that mental workload is a dynamic process so incorporating historical measurements could reduce its estimation error. In addition, the estimation of operator performance in human automation teams is essential in optimizing task effectiveness and facilitating efficient resource allocation. In this work, we consider a scenario where a human and an automation solve binary classification tasks under time constraints. We present and compare different dynamic schemes to estimate the operator’s performance, i.e., classification accuracy, and its subjective ratings on subscales of the NASA-TLX questionnaire, which measure mental workload across multiple dimensions. These schemes differ in the information available for estimation. We test these schemes on data collected from a scenario, where a human and an automation perform a series of classification tasks for simulated mobile objects. Our analysis of the interaction data and the estimation schemes indicates that employing dynamic estimation for certain NASA-TLX subscale ratings leads to decreased estimation errors.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.000 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.001 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it