Towards generalized performance metrics for human-robot interaction
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
In order for cognitive robots to act adequately and safely in real world, they must be able to perceive and have abilities of reasoning up to a certain level. Toward this end, performance evaluation metrics are used as important measures to achieve these goals. This paper intends to be a further step towards identifying common metrics for task-oriented human-robot interaction. We believe that within the context of human-robot interaction systems, both human and robot independent actions and joint interactions can significantly affect the quality of the accomplished task, thus proposing a generic performance metric to assess the performance of the human-robot team. Toward the efficient modelling of such metric, we also propose a fuzzy temporal model to evaluate the human trust in automation while interacting with robots and machines to complete some tasks. Trust modelling is critical as it directly influences the interaction time that should be directly and indirectly dedicated toward interacting with the robot. Another fuzzy temporal-based model is also presented to evaluate the human reliability during interaction time, as many research studies state that a large percentage of system failures are due almost equally to humans and machines, and therefore, assessing this important factor in human-robot interaction systems is also crucial. The proposed framework is based on the most recent work in the area of cognitive human-machine interaction and performance evaluation.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.000 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.040 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it