Heavy path based super-sequence frequent pattern mining on web log dataset
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
Mining web log datasets has been extensively studied using Frequent Pattern Mining (FPM) and its various other forms. Identifyingfrequent patterns in different sequences can help in analyzing the most common sub-sequences (e.g., the pages visitedtogether). However, this approach would not be able to identify general structures spanning over multiple sequences. In responseto understanding general structures, we introduce a new form of sequential pattern mining called super-sequence frequent patternmining (SS-FPM). In contrast to sub-sequences determined by FPM, SS-FPM determines the super-sequences that can containthe common parts from different sequences. This can be useful in understanding the general behavior/flow of users in web usagemining, classifying web pages and users, making predictions etc. In essence, finding frequent super-sequence patterns turnsout to be related to the well-known heaviest (longest) path problem in graphs, which is known to be NP-hard. Accordingly,we transform a given sequential dataset into a sequence graph and formulate the problem as k-hop heaviest path problem. Wethen propose an efficient heuristic called sequence matrix method using dynamic programming techniques. We compared ourmethod to the existing Heavypath method. The results show that our method is more efficient especially on large datasets.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.003 | 0.001 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.002 | 0.001 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it