Data Collection Versus Data Estimation: A Fundamental Trade-Off in Dynamic Networks
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
An important question that often arises in the operation of networked systems is whether to collect the real-time data or to estimate them based on the previously collected data. Various factors should be taken into account such as how informative the data are at each time instant for state estimation, how costly and credible the collected data are, and how rapidly the data vary with time. The above question can be formulated as a dynamic decision making problem with imperfect information structure, where a decision maker wishes to find an efficient way to switch between data collection and data estimation while the quality of the estimation depends on the previously collected data (i.e., duality effect). In this paper, the evolution of the state of each node is modeled as an exchangeable Markov process for discrete features and equivariant linear system for continuous features, where the data of interest are defined in the former case as the empirical distribution of the states, and in the latter case as the weighted average of the states. When the data are collected, they may or may not be credible, according to a Bernoulli distribution. Based on a novel planning space, a Bellman equation is proposed to identify a near-optimal strategy whose computational complexity is logarithmic with respect to the inverse of the desired maximum distance from the optimal solution, and polynomial with respect to the number of nodes. A reinforcement learning algorithm is developed for the case when the model is not known exactly, and its convergence to the near-optimal solution is shown subsequently. In addition, a certainty threshold is introduced that determines when data estimation is more desirable than data collection, as the number of nodes increases. For the special case of linear dynamics, a separation principle is constructed wherein the optimal estimate is computed by a Kalman-like filter, irrespective of the probability distribution of random variables. It is shown that the complexity of finding the proposed sampling strategy, in this special case, is independent of the size of the state space and the number of nodes. Examples of a sensor network, a communication network and a social network are provided.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.003 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.002 |
| Open science | 0.001 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it