A Visual and Textual Information Fusion-Based Zero-Shot Framework for Hazardous Material Placard Detection and Recognition
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
Automatically detecting and recognizing hazardous material placards using computer vision-based methods ensures safe operations and proper management of dangerous freight transportation. Deep learning-based object detection methods provide viable and practical solutions to varied applications. However, contemporary deep learning-based methods suffer from imbalanced and unseen classes, which are very common in real-life data. Thus, this study, drawing attention to this hitherto neglected challenge in real-world applications, proposes a deep learning-based zero-shot framework to detect and recognize the hazardous material placards of both imbalanced and open classes. A logarithmic weighted cross-entropy is proposed to balance the closed classes during training. In addition, a logarithmic weighted confidence fusion strategy is designed to fuse the separately extracted visual and textual information. The experiments on real-world transportation data demonstrated the proposed framework's effectiveness and superiority over other state-of-the-art methods. Notably, our framework outperforms the previous method with a remarkable margin of 12.8% in the F1 score on the placard dataset. This study solves the imbalanced and open class problem by fusing object visual information and text information, providing a practical industrial application of the zero-shot learning concept.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.000 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it