Shane Poole · Software & data engineer

Data is potential. Useful data is progress.

Five years of making data usable, then making it matter.

0yrs
Systems in production
0M
Documents processed
0+
Projects shipped
0
Peer-reviewed papers

About

Nobody wants a pipeline. They want the answer at the end of it.

I'm the bioinformatics engineer on the data and development team at the Bove Lab in UCSF's Department of Neurology(opens in a new tab). The lab studies multiple sclerosis along two lines: digital medicine, using wearables, phones, and remote assessment to see patients between clinic visits, and sex- and gender-informed neurology, looking at how hormones and reproductive exposures shape inflammation and repair. My job is the layer underneath both, turning what comes off a device or out of a chart into something a researcher can analyze.

My work starts where the data does, at an instrument in a clinic, a wearable, a phone, a database of records, and follows it until someone can actually use the result. That far end is my favorite part. The dashboard our neurologists open during a visit runs on data I supply: a Google Maps overlay of local MS resources that I helped build, and gait and assessment videos put in order so a clinician can watch how a patient's walk has changed over years. Knowing a doctor reads that with the patient sitting right there changes how carefully I treat every row. Pulling a structured measure out of thousands of narrative notes is worth doing for the same reason. It turns a question nobody could afford to ask into one a researcher can answer this quarter.

Some weeks that means a cohort nobody has assembled before, some weeks a figure for a grant due Friday. The deliverable changes constantly and what it's for never does. The domain happens to be neurology, but the problems are the ones every data team has: sources that disagree with each other, volumes that break the obvious approach, and results that still need to reproduce a year later. Running my own systems that long taught me the handoff matters as much as the build, so I write the glossary, the runbook, and the decision record. A system nobody else can operate isn't finished.

I came to software sideways. Before UCSF I spent three years as a chemist at the US Environmental Protection Agency, running aquatic toxicology studies. I barely wrote code there. What I did write was a VBA macro that killed a copy-and-paste job I had been doing by hand in Excel, and watching an afternoon of tedium collapse into a keystroke is more or less the whole reason I'm here. Then I learned to actually program at 42 Silicon Valley and never went back to the bench. Between the two fields I've co-authored 26 peer-reviewed papers, and my work has been cited over 450 times(opens in a new tab). The environmental research is still being published years after I left it.

Languages

  • Python
  • SQL (T-SQL)
  • R
  • Dart / Flutter
  • Bash

Data

  • ETL & sharded pipelines
  • SQL Server, PostgreSQL
  • DuckDB + Parquet
  • Schema design
  • Statistical modeling

Services

  • FastAPI, Flask
  • Redis + RQ workers
  • Docker & Compose
  • REST API design
  • JWT auth

Platform

  • AWS (S3, RDS, ECR)
  • Object storage
  • nginx
  • CI & deployment
  • pytest

Selected work

012021 to present
Author & maintainer

A shared data platform for research

A Python toolkit that became the group's common data layer. Database access, LLM extraction from clinical notes, and geographic and socioeconomic enrichment all sit behind one importable API, so researchers stop writing one-off scripts against the warehouse. Moving the read path to a local columnar store took typical queries from over 20 seconds to under one. Five years and 30+ downstream notebooks later, it's the first import in almost every analysis the group runs.

PythonDuckDB + ParquetSQL ServerLLM extraction
022026
Author & maintainer

Clinical text classification at scale

A multi-stage pipeline that classifies patient records by combining structured signals (demographics, surgical history, diagnostic codes, medications) with a rules-based NLP waterfall over free-text notes, then settles the ambiguous cases with a temporal ordering pass. Roughly 43 million documents stream through parallel shards with checkpoint and resume, so a failure at hour nine doesn't cost you the first eight.

PythonNLPSharded processingParquet
032021 to present
Primary developer

Multi-modal data capture platform

A platform that follows mobility research data from clinic hardware to finished dataset. Instruments and a clinic-facing recording app push video and structured metrics into an async ingestion service, which validates, transcodes, writes to the study database, and lands media in object storage. A scheduled ETL then reconciles identifiers across buckets, so downstream analysis gets one canonical layout instead of four conventions and a naming argument.

FlaskRedis + RQDockerObject storage
042023 to present
Pipeline author

Video analysis at half a terabyte

Close to 500 GB of gait and assessment video, ingested and turned into something you can run statistics on. Audio is separated and transcribed, and the transcripts feed an LLM that pulls out structured measures. The video itself goes through pose and facial landmark tracking, so a recording of someone walking down a hallway becomes a time series of joint positions. What arrives as an opaque media file leaves as a table.

PythonComputer visionSpeech to textLLM extraction
052026
Backend lead

Participant video app and API

A mobile app and backend that let research participants record and submit standardized activity videos from their own phones, built inside a compliance-restricted cloud environment. Public bucket access was off the table, so uploads proxy through the API as a resumable chunked session. Auth is JWT with Argon2 hashing. Storage and mail each have a single interface and two implementations, which means the whole stack also runs locally with no cloud calls at all.

FastAPIFlutterPostgreSQLAWS
06Personal
Live

31: online card game

Live(opens in a new tab)

A real-time multiplayer version of the card game 31, built from scratch and running in production because I wanted to play it with friends. Game logic, live play, and the infrastructure behind it, on its own domain and paid for out of pocket. Nothing teaches you what architecture actually costs like getting the bill for it.

Full-stackReal-timeAWSRoute 53

Also

LLM extraction pipelines
Structured clinical measures pulled from narrative notes via an institutional Azure OpenAI gateway, with triplicate inference and inter-rater agreement scoring for reliability.
Multi-site imaging infrastructure
Reconciling imaging manifests across seven sources, plus a deterministic de-identified ID service backing a multi-site study.
Wearables and device data
Sync services for consumer health APIs, VR cognitive assessment metrics, and gait instrumentation.
Analysis on request
A standing internal service: 50+ cohort builds, statistical models, and publication figures for collaborators across several labs.
Environmental modeling (EPA)
Dose-response modeling of endocrine disruption, fish transcriptomic datasets, and the adverse outcome pathway networks behind chemical mixture risk assessment in real watersheds.

Photography

Away from the keyboard.

Landscapes, night skies, and the occasional patient bird, shot on a Nikon D610. Mostly a hobby, though a few shoots for friends ended up on their wedding invitations. Click any frame to open it larger.

Contact