Assessment of Technical Quality of Online Video Using Visualization in Place of Experience
Bibliographic record
Abstract
This paper introduces a new method to collect subjective ratings of Technical Quality (TQ) in disrupted video (DV). TQ is related to whether or not the video has disruptions such as impairments (re-buffering, perceived as freezing for a period of time followed by resumption of video playback) or failures (where the video playback stops part way through and fails to complete). The assessment method introduced in this paper avoids the confounding effects of content on TQ ratings and reduces the time and effort necessary to run experiments. Actual videos, as stimuli, are replaced with schematic representations of those videos. We ran an experiment with 37 participants to explore the viability of assessing TQ using visualizations instead of actual videos. The experiment had contrasting conditions that compared TQ ratings of actual videos, vs. schematic representations of videos. The results of the experiment showed that, with appropriate training, ratings of TQ made after viewing the visualizations only were similar to TQ ratings made after actually watching videos with corresponding impairments or failures.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.001 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".