pysal/pysal: PySAL 2.0.0
Bibliographic record
Abstract
Release Notes for PySAL 2.0 This release represents over 15 months of hard work on PySAL. It brings major transitions in the project together with many new enhancements, improved documentation, and many bug fixes and optimizations. <!-- markdown-toc start - Don't edit this section. Run M-x markdown-toc-generate-toc again --> **Table of Contents** - [Release Notes for PySAL 2.0](#release-notes-for-pysal-20) - [Major Changes](#major-changes) - [Entirely New Packages](#entirely-new-packages) - [Significant Enhancements to Existing Packages](#significant-enhancements-to-existing-packages) - [Changes by Package](#changes-by-package) - [libpysal](#libpysal) - [esda](#esda) - [giddy](#giddy) - [inequality](#inequality) - [pointpats](#pointpats) - [spaghetti](#spaghetti) - [mapclassify](#mapclassify) - [splot](#splot) - [spreg](#spreg) - [spglm](#spglm) - [spint](#spint) - [mgwr](#mgwr) - [spvcm](#spvcm) - [Summary Statistics](#summary-statistics) - [Package Activity](#package-activity) - [Contributor Activity](#contributor-activity) - [Contributor by Package Activity](#contributor-by-package-activity) <!-- markdown-toc end --> <a name="major-changes"></a> Major Changes This release, we've had a ton of activity in PySAL, much of which has been conducted in our <em>subpackages</em>, the independently-released components of our library. Because it is too onerous to list all the changes to the API here, please consult the Migrating to PySAL 2.0 page for a detailed discussion of all reorganization-related changes. This document will focus <em>only</em> on improvements to functionality, enhancements, and additional modules added to <code>2.0</code>, over and above the last stable release of the <code>1.0</code> series, PySAL <code>1.14.4</code>. <a name="entirely-new-packages"></a> Entirely New Packages For starters, we've added some entirely-new subpackages to this release candidate: <code>mgwr</code>, <strong>the multi-scale Geographically-weighted regression package for Python.</strong> Don't worry, <code>mgwr</code> fits single-scale geographically-weighted regressions, too. Geographically-weighted regression is a kind of generalized additive model that uses kernel functions in the geographic area around each observation to predict outcomes at that area more accuratly, kind of like Gaussian Process regression for geographic data. <code>spvcm</code>, for spatially-correlated multilevel models. Spatially-correlated multilevel models are models that allow for random effects of nearby areas, regions, or groups to be correlated with one another. This is a Gibbs sampling framework plus diagnostics & plotting tools for general Bayesian analysis of Gibbs samplers. The package also contains simple tools to implement new samplers on top of the infrastructure provided, which is fast, parallel, serializable, iterative-write, and interruptible. <code>spint</code>, for estimating spatial interaction models, such as the production-constrained or consumption-constrained gravity models. <code>spglm</code>, a package for fitting sparse GLMs, focused on performance over sparse categorical data. <code>splot</code>, for spatial vizualization in Python, built on top of the excellent <code>geopandas</code>. This is headed by our Google Summer of Code (2018) student, Stefanie Lumnitz, and will be ongoing throughout the release candidate maturation cycle. <code>pointpats</code>, a package for the statistical analysis of point patterns, geographical colocation, and dispersion. <a name="significant-enhancements-to-existing-packages"></a> Significant Enhancements to Existing Packages We've also had a ton of activity adding new features in our <em>submodules</em>: alpha shapes, a fast implementation (numba-enabled) of the original algorithm generalizing convex hulls to allow for non-convex hulls Contiguity weights from voronoi tesselations/delaunay triangulation graphs for point patterns Fuzzy contiguity weights, for polygonal data where planarity is not strictly enforced (provided in part by <code>geopandas</code>) A family of spatially explicit indicators of mobility association including the inter- and intra-regional indicators as well as the local indicators. The inter- and intra-regional indicators provide a meso-level view on the exchange mobility pattern, shedding light on specific regions hosting frequent rank exchanges while the local indicators examine the concordance relationship between a focal spatial unit and its neighbors for revealing the role of space in shaping the exchange mobility pattern over time. Categorical Spatial Markov, a generalization of the Spatial_Markov class previously implemented to interrogate the role of space in shaping regional income distribution dynamics to incorporate direct variables. Here, the categorical spatial lag is adopted for formalizing the neighborhood. Connected Components checks for our spatial graph matrices Pandas Adjacency list constructors and NetworkX adapters for PySAL graph objects Attach islands, tooling to merge isolated observations into existing spatial graphs Construct Contiguity graphs from linestrings, allowing for easy and fast access to road network data in PySAL <a name="changes-by-package"></a> Changes by Package Overall, there were 1636 commits that closed 368 issues, together with 236 pull requests since our last release on 2017-11-03. <a name="libpysal"></a> libpysal weights.distance.KNN.from_dataframe ignoring radius (#116) Always make spherical KDTrees if radius is passed (#117) [ENH] should <code>weights.util.get_ids()</code> also accept a geodataframe? (#97) enh: add doctests to travis (#2) (#112) sphinx docs need updating (#49) Add notebooks for subpackage contract (#108) Api docs complete (#110) Doctests and start of documentation for libpysal (#109) Add dependencies to requirements_plus.txt for test_db (#107) Weights/util/get ids gdf (#101) missing adjustments to lower case module names (#106) Rel.4.0.0 (#105) REL: 3.0.8 (#104) error importing v3.0.7 (#100) Lower case module names (#98) remove function regime_weights (#96) depreciating regime_weights in the new release? (#94) inconsistency in api? (#93) Ensure consistency in <code>from .module import *</code> in components of libpysal (#95) [WIP] cleanup (#88) docstrings for attributes are defined in properties (#87) docstrings in W class need editing (#64) version name as <strong>version</strong> (#92) remove <code>del</code> statements and modify alphashape <strong>all</strong> (#89) libpysal/libpysal/cg/<strong>init</strong>.py not importing <code>rtree</code> (#90) including rtree in imports (#91) BUG: test_weights_IO.py is using pysal and hard-coded paths (#85) fix hardcoded swm test (#86) check for spatial index if nonplanar neighbors (#84) nonplanar_neighbors fails when sindex is not constructed. (#63) increment version number and add bugfixes, api changes (#79) Spherebug (#82) only warn once for islands/disconnected components (#83) only warn on disconnected components if there are no islands (#81) LEP: Stuff/use pysal/network stuff to provide queen weights on linestring dataframes (#59) swm fix not ported forward from pysal. (#66) import scipy syntax typo in the new issue template (#68) deletion of extra spaces in warning message (#78) Nightli.es build permissions (#77) name of geometry column is hardcoded in nonplanar_neighbors (#75) changed geometry column name from a str to an attribute (#76) Missing example file (#71) if numba isn't present, libpysal warns every time imported (#73) add check for disconnected components (#65) clean up for release (#74) update for new examples (#72) Swm (#70) Remaining concerns left unfixed in #61 (#62) [WIP] Alpha shapes (2D) code (#58) [WIP]: add linestring/multilinestring functionality (#61) Fuzzy contiguity (#57) add berlin example (#56) force UTF8 encoding for the long description read (#55) add guerry example dataset (#45) update georgia shapefile (#53) fix typo add <code>non_planar_joins</code> instead of <code>non_planar_neighbor</code> attr… (#54) add voronoi to the API (#46) ENH: Neighbor detection for nonplanar enforced polygon collections. (#51) Missing example used by gwr tests (#43) Wplot (#50) close the door on 2 for libpysal (#44) To networkx argument name changed (#40) bump micro version (#42) fix networkx adapters (#41) minor version bump for release (#39) Plot weights (#38) forward port of legacy fix #1028 (#37) Adding Voronoi generator for cg as well as Voronoi weights from 2-d points (#36) bump version for release (#34) attach_islands correction: incorporate pr #32 (#33) change data type of contiguity W.neighbors value from set to list (consistent with other weights) (#32) add a function to attach the nearest neighbor to island (#30) fix id2i lookup for string names and make better names (#31) two modules "Wsets.py" and "util.py" depend on each other (#26) add unittest for Wset.w_clip (#29) resolve circular import in Wsets and util (#28) update doctests in weights module to use libpysal instead of pysal (#27) bump stable date (#25) bump version for point release of lag cat fix (#24) <a name="esda"></a> esda enh: updating travis build and rtd (#40) BUG: missing rtd file (#39) REL: 2.0.1 (#38) Prepping for a doc release (#37) docstrings are using pysal legacy (#4) add zenodo doi badge (#36) REL: 2.0.0 (#34) Changing esda setup to handle version programatically (#33) port legacy esda fix for 1013 (#12) notebook links broken (#29) include /tests in release (#32) Add tests to release (#27) Accounting for incoming API changes to <code>libpysal</code> and adding testing against <code>libpysal</code>'s master branch (#26) no <code>varnames</code> in <code>Moran_BV_matrix</code> (#22) add <code>.varnames</code> attribute to <code>Moran_BV</code> objects in <code>Moran_BV_Matrix</code> results (#23) Inconsistent metadata in setup.py (#28) Update license (#30) esda has no read
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.001 | 0.000 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.001 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".