Dataset for "Towards Better Evaluation for Dynamic Link Prediction"
Bibliographic record
Abstract
These are the datasets used in Towards Better Evaluation for Dynamic Link Prediction For preparing the datasets, we closely follow the baseline methods' data preparation strategy. The original networks are saved as .csv. The networks are formatted as follows: * Each edge is denoted in one line. * Each line has the following format: source_node, destination_node, timestamp, edge_label, comma-separated arrays of edge features. * Please note that if there is no edge label available, the edge_label column will be filled with 0s only for loading purpose; these labels are not used in the link prediction task. * The first line denotes the network format. * Edge features should include at least one feature. If there is no edge feature available, a 0 value is used for all the edges. The network edge-lists are pre-processed for different methods to use them (Specifically, for preprocessing the data, we use the scripts available in "preprocess_data.py" file of the corresponding baseline). Ater preprocessing the network edge-list, there are three files that are used by the models: * .csv: this file contains the timestamp edge-list. * .npy: this file contains the edge features in the dense `npy` format that has the features in binary format. * .npy: this file contains the node features in the dense `npy` format that contains the node features in binary format. Please note that when the edge features or node features are absent, we use a vector of zeros is used as the node/edge features in line with the baseline methods.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.002 | 0.008 |
| Meta-epidemiology (narrow) | 0.003 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.002 |
| Bibliometrics | 0.003 | 0.003 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.002 | 0.003 |
| Open science | 0.005 | 0.003 |
| Research integrity | 0.003 | 0.003 |
| Insufficient payload (model declined to judge) | 0.028 | 0.044 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".