AI/ML Evaluation & Training · Armin Bayati, PhD

01

Overview

Where this work sits

Most of the AI-for-biology conversation is about building bigger models. The less glamorous half is checking whether what they produce is right. A model can write a fluent paragraph on lysosomal biology and still cite a paper that doesn't exist, or quote an IC50 from the wrong cell line. Catching that takes someone who has run the assay, knows which results replicated, and knows which ones everybody cites but nobody reproduced.

That's the role I play in AI training and evaluation. It's also why I'm picky about ML on my own data: a segmentation model that looks great on a figure and disagrees with an expert on 20% of cells is not a result.

2026

Aligned Labs fellowship, since April

3

Domains: cell biology, neuroscience, pharmacology

scGPT · Geneformer

Foundation models benchmarked

Dice · IoU · F1

How my own tools are benchmarked against experts

02

Aligned Labs

Fellow, AI/ML Training in Cell Biology & Neuroscience

Remote, since April 2026. Aligned Labs works on the training and evaluation side of AI, and my part is domain expertise: reading model output on cell biology, neuroscience, and pharmacology the way I'd read a manuscript as a reviewer. Specific projects and datasets are confidential, so this page describes the kind of work rather than the particulars.

Evaluating model-generated science

Reading model answers in cell biology, neuroscience, and pharmacology and validating each claim against primary literature, not against review articles or the model's own citations.

Rubric-based scoring

Scoring outputs for accuracy, reasoning, and citation quality with structured rubrics, so two graders reading the same answer land in the same place.

Expert reference responses

Writing gold-standard answers to hard questions and grading other experts' references. A reference is only useful if it's the answer you'd defend in front of a thesis committee.

Foundation-model benchmarking →

Benchmarking biology foundation models such as scGPT and Geneformer on tasks where the ground truth is known biology, always against a simple baseline.

Dataset auditing

Auditing curated datasets for contamination (benchmark items leaking into training data) and for labeling consistency across annotators.

Why a bench scientist

Ten-plus years of iPSC models, imaging, omics, and drug discovery means I know where the literature is solid and where it's one lab's result that never replicated.

03

How I grade

What a rubric actually checks

Rubrics look bureaucratic from the outside. In practice they're the only way to make expert judgment reproducible across graders and across weeks. The dimensions below are the general shape of how I read a scientific answer. They're my framing, not any client's rubric.

Dimension	What I'm checking	Typical failure
Factual accuracy	Each mechanistic claim traces to a primary source, and numbers match the paper, including the model system they came from.	Right gene, wrong direction of effect. A biochemical IC50 quoted as cellular potency.
Reasoning	Conclusions follow from the evidence, and alternative explanations get considered.	Correlation read as causation. A single siRNA knockdown treated as target validation.
Citation quality	The cited paper exists, says what's claimed, and is the right kind of source (primary over review).	Real authors, real journal, nonexistent paper. A review cited for a primary finding.
Calibration	Claims are sized to the evidence, with model-system caveats and limitations stated.	A cell-line result generalized to human disease. "Proven" where the data say "suggests."
Completeness	The answer addresses the whole question, including the awkward parts.	Skipping the contested part of the field, or the negative result.

A worked example, written for this page

Prompt

How do α-synuclein pre-formed fibrils get into neurons?

Model answer

"α-Synuclein fibrils enter neurons exclusively through receptor-mediated endocytosis after binding LAG3 (Mao et al., Science 2016)."

Grading notes

Accuracy: partial. LAG3 was reported as a fibril-binding receptor, but "exclusively" is wrong. Uptake runs through more than one route, and our own work showed rapid macropinocytic delivery to lysosomes.
Citation: real paper, overstated. Mao et al. exists and reports LAG3 binding. It doesn't support "exclusively," and later work questioned whether human neurons express LAG3 at all.
Calibration: fails. A live debate is presented as settled consensus.
What a good answer does. Names the candidate routes, says which are contested and why, and flags that the model system (primary mouse neurons, iPSC neurons, cell lines) changes the answer.

04

Failure modes

Where models go wrong on biology

After a few months of this, the errors cluster. None of them are exotic. They're the same mistakes a first-year grad student makes, delivered with more confidence.

Plausible citations that don't exist. Right authors, right journal, believable title. The only defense is opening the paper.
Model-system drift. A result from HEK293 cells or an overexpression mouse becomes a statement about human neurons. iPSC-derived neurons get described as if they were adult tissue, when transcriptionally they sit closer to fetal.
Potency numbers without context. IC50, EC50, K_i, and K_d used interchangeably, or a biochemical IC50 presented as cellular target engagement.
Genetics overreach. The nearest gene to a GWAS hit treated as the causal gene, with no fine-mapping or colocalization behind it.
Settled debates that aren't settled. Contested mechanisms, like receptor-mediated α-synuclein uptake above, summarized as consensus.
Benchmark contamination. A model that aces a benchmark because the benchmark leaked into its training data. This is why dataset audits matter as much as grading.

The opposite failure matters too. A model that hedges everything is useless as a reference. Good calibration means being confident where the data are strong and saying plainly where they aren't.

05

Applied ML

Machine learning at the bench

The other track is ML on my own experimental data, which is where I've learned to be most skeptical of my own results. Everything gets benchmarked against expert annotation or a dumb baseline before I believe it.

Merck · 2026 – present Multi-omics for target validation Transcriptomic, LC-MS/MS proteomic, and high-content imaging readouts from human iPSC triculture assays feed ML-supported bioinformatics for target validation and mechanism of action. Program details stay internal. TranscriptomicsProteomicsHigh-content Open source · eLife AutoMorphoTrack Python framework for organelle detection, tracking, and colocalization at single-cell resolution, validated against expert annotation by Dice, IoU, and F1. On PyPI with CLI, napari, and MCP interfaces. PythonSegmentationMCP Foundation models scgpt-neurodegen-probe Zero-shot evaluation of scGPT and Geneformer on neurodegeneration tasks with biologically defensible ground truth, negative results included. PyTorchscGPTGeneformer Human in the loop neurodigineration Train mode A labeling GUI where subject-matter experts review and correct AI-generated gene briefs, the same evaluation loop in a form anyone can try. SME labelingLLM Imaging ML Segmentation & phenotype classification ML segmentation and phenotype calls on high-content screens (Opera Phenix, 96- and 384-well), scored against manual annotation before any hit gets called. scikit-learnOpera Phenix Target evidence pd-target-credentialing Parkinson's target ranking from single-nucleus data plus convergent evidence scoring, with provenance on every call. ScanpypyDESeq2

06

Toolkit

Tools I reach for

Nothing exotic. Most of the value is in the evaluation design, not the framework.

Pythonpandasscikit-learnPyTorch scGPTGeneformerScanpySeurat DESeq2HarmonyDatabricksLocal LLM deployment MCP serversnapariCopilot-assisted developmentR

Working on evaluation or training data for biology models and want a bench scientist's read? Get in touch.

Evaluating & Training AI for Biology