Semantical representation and retrieval of natural photographs and medical images using concept and context-based feature spaces
Bibliographic record
Abstract
The growth of image content production and distribution over the world has exploded in recent years. This creates a compelling need for developing innovative tools for managing and retrieving images for many applications, such as digital libraries, web image search engines, medical decision support systems, and so on. Until now, content-based image retrieval (CBIR) addresses the problem of finding images by automatically extracting low-level visual features, such as odor, texture, shape, etc. with limited success. The main limitation is due to the large semantic gap that currently exists between the high-level semantic concepts that users naturally associate with images and the low-level visual features that the system is relying upon. Research for the retrieval of images by semantic contents is still in its infancy. A successful solution to bridge or at least narrow the semantic gap requires the investigation of techniques from multiple fields. In addition, specialized retrieval solutions need to emerge, each of which should focus on certain types of image domains, users search requirements and applications objectivity. This work is motivated by a multi-disciplinary research effort and focuses on semantic-based image search from a domain perspective with an emphasis on natural photography and biomedical image databases. More precisely, we propose novel image representation and retrieval methods by transforming low-level feature spaces into concept-based feature spaces using statistical learning techniques. To this end, we perform supervised classification for modeling of semantic concepts and unsupervised clustering for constructing codebook of visual concepts to represent images in higher levels of abstraction for effective retrieval. Generalizing upon vector space model of Information Retrieval, we also investigate automatic query expansion techniques from a new perspective to reduce concept mismatch problem by analyzing their correlations information at both local and global levels in a collection. In addition, to perform retrieval in a complete semantic level, we propose an adaptive fusion-based retrieval technique in content and context-based feature spaces based on relevance feedback information from users. We developed a prototype image retrieval system as a part of the CINDI (Concordia INdexing and DIscovery system) digital library project, to perform exhaustive experimental evaluations and show the effectiveness of our retrieval approaches in both narrow and broad domains of application.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.002 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.003 | 0.003 |
| Science and technology studies | 0.000 | 0.001 |
| Scholarly communication | 0.001 | 0.002 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.001 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".