Introduction

This submission presents an automated single-cell RNA-seq pipeline for the public PBMC3k dataset. The contribution is not the Scanpy pipeline itself. The contributions are:

1. The Claim Stability Certificate framework: a perturbation-based sensitivity analysis with structured pass/fail criteria that tests whether biological conclusions remain stable under controlled perturbations.
2. Semantic verification: checking biological conclusions (e.g., presence of expected cell types, acceptable cluster counts) rather than bitwise identity of outputs.
3. Cold-start reproducible packaging: a locked environment with vendored data that any automated system can execute from scratch without manual setup.

The canonical execution path is intentionally narrow. It uses a vendored canonical PBMC3k snapshot, a locked Python 3.12 environment, a fixed set of clustering resolutions, and a verifier that checks biologically meaningful outputs rather than brittle floating-point identity. Optional rigor-enhancing analyses, including the legacy-reference benchmark and the perturbation-panel certificate, are kept off the canonical execution path.

Existing workflow managers such as Nextflow (Di Tommaso et al., 2017) and Snakemake ensure computational reproducibility through containerization and DAG execution, but do not address the higher-level question of whether biological conclusions are stable under reasonable perturbations. The Scanpy toolkit (Wolf et al., 2018) provides the analytical building blocks, and Leiden community detection (Traag et al., 2019) has superseded Louvain for graph clustering, but neither provides a built-in framework for assessing claim stability. Batch-correction benchmarks (Buttner et al., 2019) have demonstrated the importance of evaluating biological signal preservation, but focus on integration rather than single-pipeline sensitivity. Our Claim Stability Certificate addresses this gap for single-pipeline sensitivity analysis by providing a structured, automated sensitivity check over the full analytical workflow.

Data

The canonical dataset is the public PBMC3k AnnData snapshot vendored in the repository as data/pbmc3k_raw.h5ad. Vendoring this small public dataset removes an avoidable network dependency from the canonical run while preserving public-data provenance. For the paper-only benchmark, the workflow also uses the processed PBMC3k reference object exposed by Scanpy, but only as a legacy Louvain reference-cluster object rather than as expert-curated cell-type ground truth.

Methods

The canonical workflow is packaged as a locked uv project in Python 3.12 with pinned dependencies, including scanpy[leiden]==1.12. The canonical execution path requires only three commands:

1. uv sync --frozen
2. uv run --frozen --no-sync scrna-skill run --config config/canonical_pbmc3k.yaml --out outputs/canonical
3. uv run --frozen --no-sync scrna-skill verify --run-dir outputs/canonical

Quality control follows the legacy PBMC3k thresholds for benchmark comparability:

• sc.pp.filter_cells(adata, min_genes=200)
• sc.pp.filter_genes(adata, min_cells=3)
• restrict to n_genes_by_counts < 2500
• restrict to pct_counts_mt < 5

This QC choice is for comparability, not as a claim of universally optimal modern preprocessing.

Downstream analysis is intentionally modern rather than a literal reproduction of the full legacy PBMC3k tutorial. Raw counts are preserved in a layer, the matrix is normalized and log-transformed, highly variable genes are flagged without hard subsetting, and PCA and neighbor-graph construction consume the HVG flags. Leiden clustering is swept over the fixed candidate set {0.4, 0.6, 0.8, 1.0, 1.2}.

Marker ranking uses filtered Wilcoxon rank_genes_groups results on the full log-normalized matrix. Cluster annotation is marker based and explicitly putative. For each cluster, the workflow scores overlap against curated PBMC lineage signatures, records evidence genes, computes best and runner-up lineage support, and emits an Unresolved label when score, support, or margin thresholds are not met.

The semantic verifier checks canonical input shape, post-QC shape, resolution choice, cluster count, artifact existence, readable output files, and rerun stability at the level of selected resolution, cluster count, resolved label set, unresolved fraction, and label cell fractions.

The optional Claim Stability Certificate reruns a small perturbation panel over seed, neighbor count, and HVG count, then asks whether claims such as T-cell, B-cell, NK, monocyte, and megakaryocyte-like support remain present. This reframes reproducibility around stable biological conclusions rather than exact cluster IDs or UMAP coordinates.

Results

In the frozen clean rerun, the canonical path selected Leiden resolution 0.8 and produced 9 resolved clusters with 0.0 unresolved fraction. The resolved label set was:

• B
• CD14 Mono
• CD4 T
• CD8 T
• Dendritic
• FCGR3A Mono
• Megakaryocyte
• NK

The canonical artifact set includes:

• outputs/canonical/manifest.json
• outputs/canonical/qc_summary.json
• outputs/canonical/resolution_sweep.csv
• outputs/canonical/cluster_markers.csv
• outputs/canonical/cluster_annotations.csv
• outputs/canonical/umap_clusters.png
• outputs/canonical/umap_annotations.png
• outputs/canonical/marker_dotplot.png
• outputs/canonical/pbmc3k_annotated.h5ad
• outputs/canonical/verification.json

Legacy Reference Concordance

Against the legacy Louvain labels in the processed PBMC3k reference object, the frozen clean rerun reached 0.9359363153904473 majority purity on 2638 shared barcodes. This result is reported only as legacy reference-cluster concordance. It is not presented as cell-type ground truth accuracy.

Claim Stability Certificate

The Claim Stability Certificate reran a perturbation panel over seed, neighbor count, and HVG count:

• seed-1
• seed-2
• neighbors-12
• hvg-1800
• hvg-2200

The certificate passed. Quantitatively, all 8 tracked claims (6 biological lineage markers + 2 pipeline acceptance criteria) were maintained at a 1.0 support rate across all 6 runs (5 perturbations + canonical). The minimum label-set Jaccard similarity relative to the canonical run was 0.875. Dendritic cells persisted in 5/6 runs (83%), while every other canonical label persisted in 6/6 runs.

Across the canonical run plus the perturbation panel:

• all claim-support rates were 1.0
• selected resolutions varied across 0.4, 0.6, 1.0, and 1.2
• all runs stayed inside the accepted resolution and cluster-count band
• unresolved fraction stayed at 0.0 for every run
• minimum label-set Jaccard relative to the canonical run was 0.875
• Dendritic persisted in 5/6 runs
• every other canonical label persisted in 6/6 runs

While we do not compute formal p-values (the perturbation panel is too small for parametric inference), the structured pass/fail framework provides a reproducible sensitivity check that is more informative than either bitwise identity or no sensitivity analysis at all.

This is the intended interpretation of the certificate: clustering resolutions and cluster identities can vary under controlled perturbations while the core biological conclusions remain stable.

Limitations

This workflow makes intentionally narrow claims.

• QC choices were selected for benchmark comparability.
• Cluster count depends on the chosen resolution.
• UMAP is a visualization, not a biological truth object.
• The processed PBMC3k concordance benchmark is not expert label accuracy.
• The workflow is a single-sample PBMC analysis, not a batch-integration or atlas-scale pipeline.
• Annotation output is putative and marker driven.

The current evaluation uses a single, well-behaved dataset (PBMC3k). The Claim Stability Certificate framework is dataset-agnostic -- it operates on any Scanpy pipeline that produces cluster annotations -- but its sensitivity to batch effects, higher dropout rates, and larger datasets has not been evaluated.

Conclusion

This repository contributes a locked, automated single-cell analysis pipeline whose outputs are self-verifying and whose conclusions are stress-tested. The key result is not that one exact clustering or embedding is reproduced. The key result is that the workflow can certify which biological conclusions remain stable when reasonable analysis settings are perturbed.