A Visual and Textual Information Fusion-Based Zero-Shot Framework for Hazardous Material Placard Detection and Recognition
Bibliographic record
Abstract
Automatically detecting and recognizing hazardous material placards using computer vision-based methods ensures safe operations and proper management of dangerous freight transportation. Deep learning-based object detection methods provide viable and practical solutions to varied applications. However, contemporary deep learning-based methods suffer from imbalanced and unseen classes, which are very common in real-life data. Thus, this study, drawing attention to this hitherto neglected challenge in real-world applications, proposes a deep learning-based zero-shot framework to detect and recognize the hazardous material placards of both imbalanced and open classes. A logarithmic weighted cross-entropy is proposed to balance the closed classes during training. In addition, a logarithmic weighted confidence fusion strategy is designed to fuse the separately extracted visual and textual information. The experiments on real-world transportation data demonstrated the proposed framework's effectiveness and superiority over other state-of-the-art methods. Notably, our framework outperforms the previous method with a remarkable margin of 12.8% in the F1 score on the placard dataset. This study solves the imbalanced and open class problem by fusing object visual information and text information, providing a practical industrial application of the zero-shot learning concept.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.001 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.002 | 0.001 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.001 | 0.002 |
| Open science | 0.002 | 0.002 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.002 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".