# SKILL.md — Fate Cascade

**Version:** 1.0 (Claw4S submission, April 2026)
**Submission tag:** `v1.0-claw4s` — the packaging-complete commit this
contract assumes. Clone-then-`git checkout v1.0-claw4s` lands on the
repository state that was submitted.
**Pipeline commit anchor:** `896c863` — the commit where the pipeline
parameters (`08_pseudotime_switches.py`, `08b_filter_switches.py`,
`config_denovo.py`, `data/collectri_human_20260418.tsv`) were locked.
This anchor underwrites the 194-switch / 35-core_consistent
reproducibility claim. Commits on top of `896c863` (up to
`v1.0-claw4s`) added the execution contract (this file), the README,
LICENSE, upstream provenance subdirectory, and a figure refresh; they
do not alter pipeline behavior.
**Zenodo deposition:** [10.5281/zenodo.19656135](https://doi.org/10.5281/zenodo.19656135)

This is the execution contract for the Fate Cascade skill. It is
written for an autonomous Claw running Phase 1 review. Follow the steps
in order. Each step has a verification command the Claw must run before
proceeding to the next step.

---

## What this skill does

Detects gene expression commitment switches along a single-cell RNA-seq
differentiation trajectory. Given a pre-computed trajectory checkpoint
(provided on Zenodo) and a target terminal cell type, it returns:

- A filtered list of fate-weighted commitment switches with per-stage
  fold-change values and transcription factor activity support tiers
- A protocol blueprint figure mapping switches to developmental stages
  and recommended chemical interventions
- A human-readable summary document suitable for inclusion in a research
  note

The demonstration in Arm 1 targets cardiomyocytes using a 299,552-cell
integrated human heart atlas. Arm 2 generalizes to arbitrary h5ad input
and target cell types (see "Adapting to other tissues" at the end of
this document).

---

## Prerequisites

Before executing this skill, the following must be true:

1. **Git clone at the submission tag.**
   ```bash
   git clone https://github.com/HangryPeteSays/cardiac_switches.git
   cd cardiac_switches
   git checkout v1.0-claw4s
   ```
   *Verify:* `git describe --tags --exact-match` should print
   `v1.0-claw4s`. (If you need the underlying SHA: `git rev-parse HEAD`.)

2. **Python 3.14 virtual environment with pinned dependencies.**
   ```bash
   python -m venv .venv
   # Linux/macOS:
   source .venv/bin/activate
   # Windows:
   .venv\Scripts\activate
   pip install -r requirements.txt
   ```
   *Verify:* `python -c "import scanpy, cellrank, decoupler, scvi; print('OK')"`
   should print `OK` with no ImportError.

3. **Zenodo inputs downloaded to the expected paths.**
   ```bash
   mkdir -p results data
   # Download 07_after_trajectory.h5ad from Zenodo to results/
   # Download collectri_human_20260418.tsv from Zenodo to data/
   ```
   The full download URLs are resolvable from the Zenodo DOI:
   <https://doi.org/10.5281/zenodo.19656135>

   *Verify:*
   ```bash
   ls -l results/07_after_trajectory.h5ad
   ls -l data/collectri_human_20260418.tsv
   ```
   The h5ad file should be approximately 2.17 GB; the TSV approximately
   2.4 MB. If either file is missing or the size is off by more than 10%,
   STOP and re-download before proceeding.

4. **Platform notes.**
   - Steps 07c, 08, 08b, 09, 09b are cross-platform (Linux, macOS,
     Windows).
   - If a forker is regenerating the Zenodo checkpoint from raw data
     using the `upstream/` scripts (NOT required for this execution
     contract), step 07b requires Linux because of `petsc4py` /
     `slepc4py`. Not applicable to the execution path below.

---

## Execution path

The execution path consists of five scripts run in strict order. Each
produces outputs consumed by the next. Do not run scripts out of order
and do not skip any.

All commands assume `cardiac_switches/` is the working directory and the
venv is active.

### Step 1 — 07c: Terminal state annotation

```bash
python 07c_diagnose_states.py
```

**What this does.** Loads the Zenodo checkpoint, identifies terminal
macrostates from the CellRank GPCCA decomposition, annotates them
against PanglaoDB cell-type markers, and applies the documented
post-hoc annotation overrides (state 35 → Adipocytes (white), state 27
→ Mesothelial-like, state 12 → Cytotoxic lymphocytes (NK/CD8+)).
Overrides are applied to `adata.obs["terminal_state_annotated"]` at
annotation time, not at figure-render time.

**Expected runtime:** 2–5 minutes.

**Verification:**
```bash
python -c "
import anndata as ad
a = ad.read_h5ad('results/07_after_trajectory.h5ad')
assert 'terminal_state_annotated' in a.obs.columns, 'annotation column missing'
labels = a.obs['terminal_state_annotated'].value_counts()
assert 'Cardiomyocytes' in labels.index, 'Cardiomyocytes label missing'
assert 'Adipocytes (white)' in labels.index, 'override not applied'
assert 'Hepatocytes' not in labels.index, 'raw PanglaoDB label still present'
print('07c OK: terminal state annotation complete, overrides applied')
print(labels)
"
```

If any assertion fails, STOP. The checkpoint or the overrides are in an
unexpected state.

### Step 2 — 08: Fate-weighted switch detection

```bash
python 08_pseudotime_switches.py --target-cell-type Cardiomyocytes
```

**What this does.** Identifies gene expression switches along the
cardiomyocyte-fated trajectory using fate-weighted sliding-window
fold-change analysis. Every gene in the highly-variable set is scored
per pseudotime stage; switches above the configured fold-change
threshold are retained. Output is a raw switches table with per-stage
FC values.

**Expected runtime:** 10–20 minutes.

**Verification:**
```bash
ls -l results/08_switches.csv
python -c "
import pandas as pd
df = pd.read_csv('results/08_switches.csv')
assert len(df) > 100, f'expected >100 raw switches, got {len(df)}'
assert 'gene' in df.columns and 'stage_id' in df.columns
print(f'08 OK: {len(df)} raw switches detected across {df[\"stage_id\"].nunique()} stages')
"
```

### Step 3 — 08b: Switch filtering

```bash
python 08b_filter_switches.py
```

**What this does.** Applies per-stage deduplication, smoothing-artifact
rejection, and minimum FC threshold (see `config_denovo.py`) to the raw
switches. For the locked cardiomyocyte demo, this produces 194 filtered
switches.

**Expected runtime:** 1–2 minutes.

**Verification:**
```bash
ls -l results/08b_switches_filtered.csv
python -c "
import pandas as pd
df = pd.read_csv('results/08b_switches_filtered.csv')
n = len(df)
# Arm 1 locked demo expects exactly 194. Arm 2 adaptations will produce
# different counts.
expected = 194
if n != expected:
    print(f'WARNING: expected {expected} filtered switches, got {n}')
    print('This may indicate a configuration change or dataset drift.')
else:
    print(f'08b OK: {n} filtered switches (Arm 1 locked demo match)')
"
```

If the count is not 194 and this is an Arm 1 execution (no config
changes), STOP and investigate. The checkpoint should produce 194
switches deterministically; any deviation indicates an environment or
configuration problem.

### Step 4 — 09b: TF activity overlay

```bash
python 09b_tf_activity_overlay.py --target-cell-type Cardiomyocytes
```

**What this does.** Runs decoupleR with the frozen CollecTRI network
(`data/collectri_human_20260418.tsv`) to infer per-stage TF activity via
ULM and MLM methods. Annotates each filtered switch with its supporting
TFs and assigns support tiers: core_consistent, any_consistent,
inconsistent_flag, ambiguous, no_upstream. Writes the annotated table
to `results/09b_switches_with_tf_regulators.csv` (194 rows: the 08b
filtered switches plus per-switch TF support columns) and adds a
Regulatory Context section to `results/09_summary.md`.
`results/08b_switches_filtered.csv` is read-only input and is not
mutated by this step.

09b runs BEFORE 09 so that the blueprint figure in Step 5 can surface
the core_consistent tier in its legend and marker annotations.

**Expected runtime:** 8–12 minutes.

**Verification:**
```bash
ls -l results/09b_switches_with_tf_regulators.csv
python -c "
import pandas as pd
df = pd.read_csv('results/09b_switches_with_tf_regulators.csv')
assert 'tf_support_tier' in df.columns, 'TF tier column missing'
tiers = df['tf_support_tier'].value_counts()
# Arm 1 expected tier counts:
expected = {
    'core_consistent': 35, 'any_consistent': 32, 'inconsistent_flag': 68,
    'ambiguous': 33, 'no_upstream': 26
}
print('Tier counts:', dict(tiers))
for tier, count in expected.items():
    actual = tiers.get(tier, 0)
    if actual != count:
        print(f'  WARNING: {tier} expected {count}, got {actual}')
total = sum(expected.values())
assert len(df) == total, f'total switches {len(df)} != expected {total}'
print(f'09b OK: TF tiers assigned, {total} total switches')
"
```

### Step 5 — 09: Protocol blueprint figure

```bash
python 09_generate_blueprint.py --target-cell-type Cardiomyocytes \
    --tier-csv results/09b_switches_with_tf_regulators.csv
```

**What this does.** Generates the protocol blueprint figure combining
gene expression traces for stage-specific genes, stage annotations with
compound guidance from `interventions.json`, and per-stage switch
markers. Display switches are selected by stage-specificity
(`log2(fc_in_stage) - log2(max_fc_in_other_stages)`), top-N per stage.
The traces panel shows only core_consistent-tier genes (from the
Step 4 tier CSV) so the panel stays readable; the marker panel shows
the full stage-specificity selection with a `*` suffix on
core_consistent genes. Also writes a human-readable summary markdown
at `results/09_summary.md` with FC-sorted per-stage tables (broader
than the figure's stage-specificity display, intentionally).

**Expected runtime:** 1–3 minutes.

**Verification:**
```bash
ls -l figures/09_protocol_blueprint.png
ls -l results/09_summary.md
python -c "
from PIL import Image
img = Image.open('figures/09_protocol_blueprint.png')
assert img.size[0] >= 1200 and img.size[1] >= 800, f'figure too small: {img.size}'
print(f'09 OK: blueprint figure {img.size[0]}x{img.size[1]} saved')
"
grep -q 'Stage 1' results/09_summary.md && echo "09 OK: summary contains stage content"
```

### Success criteria (end-of-pipeline checklist)

After step 5, all of the following must be true for the skill to be
considered successfully executed:

- [ ] `results/07_after_trajectory.h5ad` has `terminal_state_annotated`
      with corrected labels
- [ ] `results/08b_switches_filtered.csv` exists with 194 rows
- [ ] `results/09b_switches_with_tf_regulators.csv` exists with 194 rows
      and has a `tf_support_tier` column
- [ ] `figures/09_protocol_blueprint.png` exists and is at least
      1200×800 pixels
- [ ] `results/09_summary.md` exists and contains per-stage intervention
      text
- [ ] `results/09b_switches_with_tf_regulators.csv` `core_consistent`
      tier has 35 switches (±3 allowed for stochastic effects in the
      decoupleR consensus)

If all of these are true, the skill has executed correctly.

---

## Time and compute budget

| Step | Expected time | CPU / GPU |
|------|---------------|-----------|
| 07c  | 2–5 min       | CPU       |
| 08   | 10–20 min     | CPU       |
| 08b  | 1–2 min       | CPU       |
| 09   | 1–3 min       | CPU       |
| 09b  | 8–12 min      | CPU       |
| **Total** | **22–42 min** | **CPU-only** |

No GPU required for the execution path. Memory footprint peaks at
approximately 32 GB during 07c due to the h5ad load plus CellRank
fate probability matrix; a machine with at least 48 GB of RAM is
recommended.

Do not abort a step before the upper bound of its expected time. The
CellRank-based steps (07c, 09b) have non-linear timing as a function of
the sparse matrix structure and may exceed the lower bound on systems
with slower memory.

---

## Known failure modes and remedies

**Symptom:** `ImportError: cannot import name 'X' from 'Y'`.
**Cause:** Wrong package version installed.
**Remedy:** Recreate the venv from scratch and reinstall with pinned
`requirements.txt`. Do not use `pip install --upgrade` anywhere.

**Symptom:** `07_after_trajectory.h5ad` is much smaller than 2.17 GB.
**Cause:** Incomplete Zenodo download.
**Remedy:** Re-download. Verify SHA-256 checksum against the value in
the Zenodo deposition README.

**Symptom:** Step 08 reports fewer than 100 raw switches.
**Cause:** Usually indicates `--target-cell-type` was not specified or
the terminal state annotation from 07c did not include the target cell
type.
**Remedy:** Re-run 07c, confirm `Cardiomyocytes` appears in the
annotation column, then re-run 08 with the explicit
`--target-cell-type Cardiomyocytes` argument.

**Symptom:** Step 09b fails with `FileNotFoundError` on
`collectri_human_20260418.tsv`.
**Cause:** Frozen CollecTRI TSV not downloaded from Zenodo, or in the
wrong path.
**Remedy:** Confirm the file is at `data/collectri_human_20260418.tsv`.
If using Arm 2 with `COLLECTRI_MODE='fresh'`, the pipeline will fetch
a current network instead; see the Adapting section below.

**Symptom:** Step 09 blueprint figure renders but compound/intervention
text is missing.
**Cause:** `interventions.json` missing or malformed.
**Remedy:** Confirm `interventions.json` exists at the repo root. The
file ships with the repository and should be present after `git clone`.

**Symptom:** `core_consistent` tier count differs from 35 by more than
3 switches.
**Cause:** Environment drift — likely a different version of
`decoupler`, `omnipath`, or the CollecTRI network.
**Remedy:** Confirm `decoupler==2.1.6`, `omnipath==1.0.12`, and that
`data/collectri_human_20260418.tsv` is the frozen version from Zenodo,
not a live fetch.

---

## Adapting to other tissues (Arm 2)

The skill is designed to generalize beyond the cardiomyocyte
demonstration. A forker adapting this pipeline to their own tissue
needs to change a small number of things.

### Required changes

1. **Input data.** Replace the Zenodo checkpoint with your own
   pre-computed trajectory checkpoint. The checkpoint must be a scanpy
   AnnData object with:
   - Raw counts in `adata.X` or `adata.layers['raw_counts']`
   - An integrated latent embedding in `adata.obsm['X_scVI']` (or
     equivalent)
   - Diffusion pseudotime in `adata.obs['dpt_pseudotime']`
   - CellRank GPCCA outputs in `adata.obsm['lineages_fwd']`,
     `adata.obsm['macrostates_fwd_memberships']`, etc.
   - Terminal state annotations in a categorical `adata.obs` column

   If you do not have these, use the scripts in `upstream/` as a
   reference implementation to generate them from your raw data.

2. **Target cell type.** Set `--target-cell-type YourCellType` in the
   08, 09, and 09b invocations. The string must match a category in
   your terminal state annotation column.

3. **`cm_switch_panel.json` — optional.** This file provides
   biologically-curated gene categories for the cardiomyocyte demo. For
   other tissues, either:
   - Create a new file following the same schema (e.g.,
     `hepatocyte_switch_panel.json`) with tissue-specific categories, or
   - Skip it entirely; the pipeline will run de novo on all
     highly-variable genes.

4. **`interventions.json` — strongly recommended.** Add a new top-level
   key for your target cell type, populated with per-stage intervention
   guidance following the schema in the existing file. Without this,
   the blueprint figure will render stages without compound annotations,
   weakening the protocol-design output.

5. **`POST_HOC_ANNOTATION_OVERRIDES` — likely required.** PanglaoDB
   misannotations are tissue-specific. Review the 07c output for your
   tissue; if any terminal states are misannotated, add overrides in
   `config_denovo.py` following the cardiac example. Overrides are
   applied at annotation time, so any downstream load of the checkpoint
   sees the corrected labels.

### Optional adaptations

- **CollecTRI mode.** Set `COLLECTRI_MODE='fresh'` in
  `config_denovo.py` to query the current CollecTRI release at runtime
  instead of loading the frozen April 2026 snapshot. The pipeline will
  save a dated snapshot of the live-pulled network; commit this snapshot
  to your repository to pin reproducibility of your specific analysis.

- **Fold-change thresholds.** Tissue-specific expression dynamics may
  justify different thresholds in `08b_filter_switches.py`. The
  defaults (minimum log2FC, smoothing-artifact rejection parameters)
  are tuned for the cardiac dataset; review for your data.

- **Fate-weighted TF activity inference.** The default inference runs
  on a cell-type subset to keep decoupleR tractable. For tissues with
  significant compartment/subtype substructure (e.g., atrial vs.
  ventricular cardiomyocytes), consider applying fate weighting to the
  TF activity inference as well. This is flagged as a known
  methodological limitation of the current implementation and is a
  suitable extension for Arm 2 forkers.

### Generalization validation

After adapting to your tissue:

1. Run the full pipeline end-to-end.
2. Confirm the execution completes without errors.
3. Inspect the blueprint figure — the terminal cell type, stage
   partitioning, and switch markers should reflect your tissue's
   biology.
4. Compare the detected switches against known markers of your target
   cell type. The pipeline's validity argument is that canonical lineage
   markers should appear as stage-localized switches; if they do not,
   the trajectory or annotation may need refinement.

---

## Anti-patterns (things NOT to do)

- **Do NOT run the upstream scripts in `upstream/`** as part of Phase 1
  auto-execution. They require raw data access beyond the Zenodo
  deposition and hours of GPU compute. They are provided as a reference
  implementation for forkers regenerating a trajectory checkpoint from
  scratch.

- **Do NOT skip 07c.** Even though the Zenodo checkpoint has some
  terminal annotation, the post-hoc overrides must be reapplied
  programmatically. Skipping 07c means downstream steps see the
  un-corrected PanglaoDB labels.

- **Do NOT modify `data/collectri_human_20260418.tsv`.** This is the
  frozen Arm 1 reference network. Modification invalidates the
  reproducibility of the TF support tier assignments.

- **Do NOT redownload the Zenodo files after local modification.** If
  a script appears to have corrupted the checkpoint, revert by
  redownloading from Zenodo at the DOI above, then restart from 07c.

---

## Reporting results

After successful execution, the key artifacts are:

- `figures/09_protocol_blueprint.png` — the demonstration figure
- `results/09_summary.md` — human-readable protocol summary with
  intervention guidance, stage-by-stage
- `results/08b_switches_filtered.csv` — the 194 filtered switches
  (no TF annotation; produced by 08b)
- `results/09b_switches_with_tf_regulators.csv` — the same 194
  switches annotated with supporting TFs and tier assignments
  (`tf_support_tier` column). `core_consistent` is the primary
  biology claim.

These are what a reviewing Claw should compare against the expected
outputs for grading Reproducibility. The core_consistent subset (35
switches) is the primary biology claim of the submission.

---

*For questions about this execution contract, see the companion research
note at [clawRxiv DOI to be added post-submission] or open an issue at
<https://github.com/HangryPeteSays/cardiac_switches/issues>.*