Thinking outside the box; fast error estimation for next generation galaxy surveys
Bibliographic record
Abstract
Since the discovery of the accelerated expansion of the universe in the late 90s, the flat $\\Lambda$CDM model has reigned as the best explanation for the cosmological phenomena we have observed. In spite of over two decades of study, the identities and properties of both cold dark matter (CDM) and dark energy ($\\Lambda$) remain a mystery. The goal of modern precision cosmology is to measure cosmological parameters using a multitude of probes in the pursuit of deviations from the $\\Lambda$CDM framework or insights into the properties of dark matter and dark energy. If the probe being considered is the matter power spectrum measured from a galaxy redshift survey, then the precision of the derived parameters is determined by the power spectrum covariance matrix. The analytic form of the covariance matrix is difficult to estimate, so common practice is to run many simulations of the survey volume to get a brute-force estimate. \n \nNext-generation cosmological surveys are set to collect higher resolution data within a larger survey volume than ever before. The complexity and number of simulations that will be required to estimate the covariance matrix of these surveys is threatening to become too computationally expensive for even the most advanced computer clusters. Thus, there is an urgent need to develop novel techniques for reducing the computation time required to achieve such precise covariance estimates. While many proposed methods seek to reduce the number of simulations required, it is also possible to leverage the volume scaling of the covariance matrix, allowing one to reduce the size of the simulations required. \n \nSuper-sample covariance (SSC) is a contribution to the covariance matrix made by modes of the power spectrum that are larger than the volume of a survey or simulation. If this volume scaling of the covariance is to be taken advantage of, then the SSC within the simulations must be accurately modeled. To this end, I review methods of running separate universe (SU) simulations to account for the effects of SSC. While these methods have all been shown to recover the SSC with reasonable accuracy, they have been largely developed and tested in isolation from one another. I present my work in directly comparing the accuracy of these methods in recovering the SSC effect using ensembles of N-body simulations. \n \nEven with SSC accurately modeled, the volume scaling of the covariance does not hold for arbitrarily small volume simulations; at some point, the analytic behaviour of the covariance is expected to break down. I push the volume scaling to its limit by running many thousands of simulations at different volumes and scaling the covariance to match that of a larger volume survey. The SSC term has a nontrivial relation to the simulation volume, preventing it from scaling in the same way as the other components of the covariance. In light of this, I present a way to include SSC such that the scaled covariance could still be recovered with good accuracy. I find the scaled covariance matches the large volume covariance to within $\\sim 4\\%$ or better on most scales, with higher $k$ bins being biased low due to missing a small component of the SSC. The scaled covariance at very low $k$ for very small simulations is substantially lower than the large mock covariance at those scales due to very few modes of that scale being present in the small volume simulations. This creates a skewness in the distribution of power at those scales. By computing the number of modes required to avoid this skewed distribution of power, I derive a way to estimate the minimum simulation volume that could be used to accurately model the covariance at a given scale. The accurate modeling of SSC and optimal leveraging of the volume scaling of the covariance matrix are powerful complementary tools with the potential to substantially reduce the computational cost of covariance matrix estimation for future galaxy survey data.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.009 | 0.067 |
| Meta-epidemiology (narrow) | 0.002 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.001 |
| Bibliometrics | 0.002 | 0.002 |
| Science and technology studies | 0.001 | 0.002 |
| Scholarly communication | 0.003 | 0.006 |
| Open science | 0.004 | 0.005 |
| Research integrity | 0.002 | 0.004 |
| Insufficient payload (model declined to judge) | 0.006 | 0.004 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".