Management of Molecular Data in DINA with SeqDB
Bibliographic record
Abstract
Agriculture and Agri-Food Canada (AAFC) is home to numerous specimen and environmental collections generating highly relational data sets that are analyzed using molecular methods (Sanger and NGS). The need to have a system to properly manage these data sets and to capture accurate, standardized metadata over entire laboratory workflows has been a long-term strategic vision of the Biodiversity group at AAFC. Without robust tracking, many difficulties arise when trying to publish or submit data to external repositories. To even know what work has been carried out on individual collection records over a researchers career becomes a demanding task if the information is retrievable at all. SeqDB was built to resolve these issues by centralizing, standardizing and improving the availability and data quality of source specimen collection data that is being studied using molecular methods. SeqDB also facilitates integration with tools and external repositories in order to take the burden off researchers and technicians having to create adequate systems to track and mobilize their data sets, allowing them to focus on research and collection management. The development of SeqDB aligns with agile development methodologies and attempts to fulfill rapidly emerging needs from genetics and genomics research, which can evolve and fade quickly at times or be without clear requirements. The success of SeqDB as an application supporting DNA sequencing workflows has put it in the same space as other monolithic architectures before it. As the feature set to support the application continues to increase, the number of software developers vs operations and maintenance staff is difficult to rebalance in our organisation. In an effort to manage the scope for the project and ensure we are able to continue to deliver on our mandate, the sequence tracking workflows of the application will become part of the DINA ecosystem (“DIgital information system for NAtural history data”, https://dina-project.net). Other functions of SeqDB such as collections management and taxonomy tree curation, will be replaced with the DINA modules implementing these functions. In order to allow SeqDB to become a module of DINA, it has been decided to refactor the application to base it on a Service Oriented Architecture. By doing so, all molecular data of SeqDB will be exposed as JSON API Web Services (JavaScript object notation application programming interface) allowing other modules, user interfaces and the current SeqDB application to communicate in a standardised way. The new architecture will also bring an important technology upgrade for SeqDB where the front end will eventually become a project in itself.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.015 | 0.027 |
| Meta-epidemiology (narrow) | 0.002 | 0.002 |
| Meta-epidemiology (broad) | 0.002 | 0.002 |
| Bibliometrics | 0.006 | 0.007 |
| Science and technology studies | 0.003 | 0.002 |
| Scholarly communication | 0.014 | 0.009 |
| Open science | 0.008 | 0.010 |
| Research integrity | 0.002 | 0.005 |
| Insufficient payload (model declined to judge) | 0.035 | 0.033 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".