Visual Task Inference in Conjunction Search Using Hidden Markov Models and Token Passing
Bibliographic record
Abstract
The effect of visual task on the pattern and parameters of eye-movements has been long investigated in the oculomotor studies of human vision. However, there is not much done in the inverse process; that is inferring the visual task from eye-movements. Visual search is one of the main ingredients of human vision that plays an important role in our everyday life. In our previous work we developed an ergodic HMM-based model to infer the visual task in pop-out search by locating the focus of covert attention. In this paper, we improve our previous model to infer the task in a conjunction search in an eye-typing application, where users can type a character string by directing their gaze through an on-screen keyboard. In this scenario inferring the task is equivalent to figuring out what word has been eye-typed. The inherent complexity of conjunction search usually calls for off-target fixations before locating the target. However, these off-target fixations are not randomly distributed and show a pattern according to the target. The brain tends to direct the gaze on objects that are seemingly similar to the target. Therefore, we propose a tri-state HMM (TSHMM) to model the attention cognitive process of human brain, where the three states represent the fixations on the target, similar non-target and dissimilar non-target objects. We train a TSHMM for each character by using the Baum-Welch algorithm to capture the dynamics of attention during the search and construct a lexicon network by concatenating the characters and use a technique called token passing to reveal the best state sequence for the test data. The results show a great improvement compared to our previous model. We can further improve the results by setting a-priori constraints on the order of characters by making a dictionary of valid words. Meeting abstract presented at VSS 2012
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.004 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".