Speeding Up QA: An Index Structure for Question Queries
Bibliographic record
Abstract
Scalability is a major disadvantage of Web question-answering systems (QA), which produces slow response and tedious search time. The bottleneck lies in the commercial search engine used in simple QA systems. In normal architecture of QA, after finding the related documents in the text corpus, analysis of these documents and retrieval of the answers will cost much time. The reason is that the traditional inverted index used in the commercial search engine is not optimal for the QA systems. One of the solutions is to index the position of answers of QA systems directly in the corpus, not index only the single meaningless words. Thus, the response time of the improved search engine to QA queries will be expected to be almost at the same level as the commercial search engines to searching issued by users. In this paper, we will propose a new inverted index structure for QA systems. By indexing the possible meaningful phrases relative to the position of items (words), our approach can improve the response time without losing the advantages of the inverted index. Due to the larger and larger cheap amount of storing space available in nowadays computers, the extra space used by the approach may be regarded neglectable. Thus, our approach can be used in any large-scale QA system, which always produces enormous quantity of possible answers.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.002 | 0.012 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.005 | 0.006 |
| Science and technology studies | 0.002 | 0.001 |
| Scholarly communication | 0.002 | 0.008 |
| Open science | 0.002 | 0.003 |
| Research integrity | 0.001 | 0.002 |
| Insufficient payload (model declined to judge) | 0.007 | 0.006 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".