How becoming an open scientist made me fall back in love with neuroscience
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
Three years into my PhD, the most important thing to me was publishing in Nature. As a PhD student in computational neuroscience at Carnegie Mellon, I was building computer simulations to study how groups of neurons should be configured to best represent information. Day in and day out, I tweaked the parameters of my simulations, rummaging around for interesting effects. But what began as a noble pursuit -- understanding the foundations of neuronal computation -- soon devolved into an empty impact factor chase as I became a master in the art of science writing and “selling” my findings. I was miserable. While finishing up my first research project (1), I resolved to make a major change in my research direction. What was the one thing that I could work on that would most benefit and truly impact the field? At the time, neuroscience was a disjointed mess. Thirty years ago, the field was composed of 100 brave pioneers, studying the brain using limited tools but armed with their own creativity. Now, the field had ballooned into 100K individuals, each working towards collectively understanding the minutiae of the brain’s function and dysfunction. Despite the wealth of knowledge my colleagues were discovering, a major challenge was merely keeping up with the research literature in one’s own subfield, much less the greater field at large. I set my sights on tackling this problem of organizing scientific information, and in particular, understanding electrical diversity across neuron types throughout the brain. While hiding from my PhD adviser, I began developing computer algorithms for mining this data from the texts of published papers. I was amazed by how quickly I made progress. In stark contrast to my previous computer language of choice, closed-source Matlab, I readily found free and open-source tools in Python which did much of the heavy lifting in text mining and database management for me. Within a few short months, I had an early working prototype of the NeuroElectro web resource (NeuroElectro.org). But the early success with the project posed an existential dilemma: should we open NeuroElectro up to the public on the internet? It needed tons of polish and was at least 2 years away from publication (2, 3), but my experience from the open source software community suggested that opening it to the public might be a great opportunity to get early feedback and community buy-in. On my other shoulder was my academic intuitions -- what if someone scooped us by analyzing the database’s content before us, reaping the major rewards of building NeuroElectro? After carefully weighing the pros and cons with my adviser and labmates, under the credo: “What would Aaron Swartz do?”, we opened NeuroElectro.org to the public and sent around a few emails to labs which would be interested. As I starting going to conferences and presenting the NeuroElectro resource, the feedback was beyond belief. I would get comments like: “this is amazing! this is exactly what the field needs!”, “this is free? really?”, “I need to email this to my lab immediately!”. Big shots, people I wouldn’t have dreamed of talking to earlier, would chat with me at length and offer their genuine feedback. One of the fathers of my field, Gordon Shepherd, became my mentor and strongest advocate. All of this is to say, I was finally working on something that had true impact, based on the overwhelming support I was getting from the community. And by being open, I was earning the rewards from my work long before publishing it. More recently, the exposure from being open has led to amazing and previously unimaginable opportunities. For example, the multi-billion dollar Human Brain Project invited me out to Geneva, Switzerland to serve as an external advisor on improving their data sharing and perceived openness. During a panel discussion on literature text-mining at Harvard Medical School, I used NeuroElectro to justify the need for a liberal text-mining policy to the mega-publisher Elsevier. Lastly, I’m a member of the Neurodata Without Borders project, working with the field’s leading experimental and informatics experts from HHMI’s Janelia Farm and the Allen Institute for Brain Science, to create a standardized data format for neurophysiology. By participating in these efforts, I’ve been able to organically grow my professional network and pool of potential collaborators. Importantly, being open has led to increased research funding for my work. I’ve received a coveted Canadian Institute for Health Research postdoctoral fellowship and my lab’s R01 grant submission to the NIH was scored very highly. In both cases, reviewers cited our openness and commitment to creating useful informatics tools as key strengths of our research. When I look back on the way I used to do neuroscience compared to how I do it now, the key difference is that today, I have much more fun. I work just as hard, but now, I get to work with smart and creative people who also want their science to have a real impact. Being open and non-secretive about my work-in-progress means that I get feedback early and often, allowing me to refine my efforts for maximal impact and awesomeness. In short, becoming an open scientist made me fall back in love with neuroscience.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.003 | 0.001 |
| Open science | 0.009 | 0.002 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it