Bibliographic record
Abstract
After years of development, and especially since the introduction of some powerful algorithmic approaches including those for link, anchor, title analysis, and so on, web search has become amazingly powerful in its ability to discover and exploit nearly any kind of information within the billions of pages that comprise the Web. However, as powerful and large as current web search engines are, they are still limited in their ability to always deliver key services to their users especially when there is a considerable number of users with different search intentions and needs. Consequently, there is a recent merge of interest and efforts within both industry and academia in developing novel search applications to satisfy different needs. This is especially true for those application domains that have been not fully considered by traditional web search engines. The aim of the present thesis is to propose different ways of extending or using classical link analysis methods, that have been extensively applied in many classical web mining contexts, for several novel search applications which are appearing as a response to recent trends in web search. We consider different semantics related to each application domain and propose different ways of combining these semantic features with classical link analysis to enhance the corresponding web mining task. In particular, we study both analytically and empirically web communities, personalized search, geographically-oriented search, and meta-search emphasizing their retrieval aspect. While most techniques used by traditional information retrieval (IR) are mainly focused on content analysis, the modern WebIR has clearly shown that almost any web mining retrieval task can be enhanced through the exploitation of other features that are inherent in the Web (e.g. anchor text) beyond simple content. In this thesis, we again validate such hypothesis (in most cases through a set of experiments) verifying that the exploitation of features particular to each corresponding web mining task can enhance search effectiveness. For web communities, we propose a model for their extraction and the ranking of their members based on the Random Field Ising Model (RFIM), capable of representing both semantic and linkage information of web pages through node and edge weights. For personalized search, we propose an analytical model for personalized search unifying four critical aspects of the problem, namely link structure, document content, user query, and user preference. For geographically-oriented search, we study how geographic semantics can be combined with classical link analysis to improve geographically-oriented search especially in crawling and ranking aspects. For metasearch, we recast the metasearch problem as a ranking problem on a bipartite graph, constructed for a given query, and we propose a solution based on classical link analysis.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.004 | 0.019 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.002 |
| Bibliometrics | 0.008 | 0.006 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.003 | 0.005 |
| Open science | 0.002 | 0.003 |
| Research integrity | 0.001 | 0.002 |
| Insufficient payload (model declined to judge) | 0.003 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".