MedSeg-Eval: Analysing SAM2 Performance on Abdominal CT Liver Segmentation
MedSeg-Eval: Analysing SAM2 Performance on Abdominal CT Liver Segmentation
Prompt Sensitivity, Slice Selection Strategy, and Failure Analysis on the CHAOS CT Dataset
Abstract
We present MedSeg-Eval, an executable benchmark skill analysing the zero-shot performance of SAM2 (ViT-B) [1] on abdominal CT liver segmentation using the CHAOS CT dataset [2] (CC-BY-SA 4.0, DOI: 10.5281/zenodo.3431873). We investigate three research questions: (RQ1) how sensitive is SAM2 to prompt strategy (center point, bounding box, grid of points)? (RQ2) does slice selection strategy (maximum-area slice vs. mid-volume slice) meaningfully affect performance? (RQ3) what are the dominant failure modes, and does liver size correlate with segmentation accuracy?
Across 30 inference runs (5 cases × 3 prompt strategies × 2 slice strategies), only the bounding-box prompt on the best liver slice achieves clinically relevant accuracy (mean Dice 0.775 ± 0.084); all point-based strategies fail consistently (Dice ≤ 0.443, failure rate 100%). A notable finding is that grid-of-points prompting yields no measurable improvement over a single centroid point, a result we attribute to the near-uniform Hounsfield-unit appearance of liver parenchyma under standard CT windowing. We do not claim these results generalise beyond the five tested cases or the specific oracle-prompt design; the study is explicitly exploratory and scoped to characterising zero-shot SAM2 behaviour under controlled conditions. The Dice < 0.5 failure threshold follows the clinical convention of Taha and Hanbury [3]. All experiments are inference-only, use pinned software dependencies (recorded in requirements.txt), a fixed random seed (42), and a publicly archived dataset — making the study fully reproducible by any agent or researcher.
1. Introduction
Medical image segmentation is a foundational task in clinical imaging, enabling organ volumetry, surgical planning, and treatment monitoring. The field has been shaped by supervised, task-specific models that achieve high accuracy on their training distribution but require substantial labelled data and exhibit limited cross-domain generalisation [4].
The Segment Anything Model (SAM) [5] and its successor SAM2 [1] have generated interest as promptable, zero-shot alternatives. However, CT images differ fundamentally from the natural images on which these models were trained: they encode physical tissue density as Hounsfield units, are inherently three-dimensional, and present regions of near-uniform intensity after standard windowing. These properties make CT organ segmentation a challenging and practically important test case for foundation model evaluation.
We focus on the liver — large, anatomically consistent across healthy subjects, and convex — precisely because its structural regularity removes shape complexity as a confound, isolating prompt quality and image representation as the primary experimental variables. Using the CHAOS CT dataset [2], we design a 2 × 3 factorial experiment across prompt strategies and slice selection methods, yielding controlled, interpretable results while explicitly acknowledging the statistical limitations of a five-case study.
Contributions:
- A reproducible factorial benchmark of SAM2 ViT-B on CT liver segmentation across 6 conditions with full uncertainty reporting.
- Empirical evidence that grid-of-points prompting confers no advantage over a single centroid on CT, consistent with a point-encoder saturation hypothesis under homogeneous texture.
- Quantification of the prompt × slice interaction: bounding-box performance is highly slice-sensitive (ΔDice ≈ +0.40); point-based prompts are not (ΔDice ≈ +0.10).
- A failure analysis identifying box-filling as the dominant SAM2 failure mode on CT, supported by qualitative overlay inspection and size-correlation analysis.
- Explicit characterisation of the oracle-prompt upper-bound bias and its implications for real-world deployment estimates.
2. Related Work
2.1 Medical Image Segmentation
Ronneberger et al. [6] introduced the U-Net, with its encoder-decoder structure and skip connections, which became the standard for 2D medical segmentation. For volumetric data, the 3D U-Net [7] captured inter-slice spatial context, proving particularly effective for CT and MRI organ segmentation. nnU-Net [4], a self-configuring framework that automatically adapts preprocessing, network topology, and post-processing, subsequently surpassed most specialised solutions across 23 public benchmarks and remains the standard comparison in competitive challenges. Its success underscored the importance of systematic pipeline design, but also the fundamental limitation that every new dataset requires a fresh supervised training run.
Transformer-based architectures — TransUNet [8] and SwinUNETR among others — further improved accuracy on structured segmentation tasks but retained the requirement for large, task-specific labelled datasets.
2.2 The Segment Anything Model and SAM2
SAM [5] was trained on over 1 billion masks from 11 million natural images and achieved strong zero-shot segmentation generalisation. It accepts spatial prompts — points, bounding boxes, or masks — and produces object masks without task-specific training. Its architecture comprises a ViT image encoder, a prompt encoder, and a lightweight mask decoder. SAM2 [1] extended this to video with a memory mechanism for temporal consistency, and introduced a hierarchical ViT backbone offering improved single-image performance.
2.3 SAM in Medical Imaging
Mazurowski et al. [9] conducted one of the first systematic evaluations across 19 medical datasets spanning CT, MRI, ultrasound, and pathology. They found highly variable performance — IoU ranging from 0.11 to 0.91 — with bounding-box prompts consistently outperforming point prompts, and CT performance substantially lower than on visually richer modalities, attributable to the domain gap between SAM's training distribution and Hounsfield-unit image representation.
Huang et al. [10] found that SAM delivered reliable results primarily for large connected objects with clear boundaries, struggling with amorphous or low-contrast targets. In abdominal CT specifically, Ji et al. [11] found SAM relatively proficient on organ regions with clear boundaries but prone to failure on regions where the target blends with the background — a category that CT liver segmentation falls into after standard windowing.
MedSAM [12] addressed the domain gap by fine-tuning SAM on 1,570,263 image–mask pairs across 10 imaging modalities, demonstrating substantially improved accuracy across 86 validation tasks. The Medical SAM Adapter (Med-SA) [13] achieved effective adaptation across 17 segmentation tasks by updating only 2% of SAM's parameters via lightweight adapter layers. SAM-Med2D [14] fine-tuned on 4.6 million medical images with 19.7 million masks and demonstrated strong generalisation to unseen challenge datasets. These domain-adapted variants consistently and substantially outperform zero-shot SAM, establishing fine-tuning as a practical prerequisite for reliable clinical deployment.
2.4 Evaluation Methodology
Taha and Hanbury [3] provide a comprehensive analysis of segmentation evaluation metrics and their clinical interpretability, establishing Dice < 0.5 as the threshold below which a segmentation mask has insufficient overlap for clinical volumetric use. This threshold is adopted throughout this work as our failure criterion — not as an arbitrary cutoff, but because it corresponds to a predicted mask where true positives constitute less than one-third of all positive predictions and ground truth voxels in the worst case, rendering any volume measurement derived from such a mask clinically unreliable.
2.5 Positioning of This Work
Prior SAM evaluations predominantly test a single prompt type or compare SAM against task-specific models as their primary objective. Our work isolates prompt strategy and slice selection as independent experimental factors in a factorial design, quantifying their interaction and providing mechanistic interpretation. The equivalence of center-point and grid-point prompts under oracle conditions has not been previously reported for CT liver segmentation, and constitutes a novel finding with direct implications for interactive annotation interface design.
3. Methods
3.1 Dataset
The CHAOS (Combined Healthy Abdominal Organ Segmentation) dataset [2] (CC-BY-SA 4.0, DOI: 10.5281/zenodo.3431873) provides 20 contrast-enhanced portal venous phase CT studies of healthy liver-donor subjects, each with expert liver segmentation masks. Images are stored as DICOM series; ground truth masks as PNG files (pixel value 255 = liver foreground). We use the first n = 5 cases by index order (IDs: 1, 10, 14, 16, 18), not selected by performance, each comprising 512 × 512 axial slices with 91–111 slices per volume.
Dataset integrity: The CHAOS zip archive SHA-256 is recorded in requirements.txt alongside all pinned package versions, ensuring byte-identical reproduction of all results. We acknowledge that n = 5 limits statistical power; all reported results should be treated as exploratory pilot findings rather than definitive conclusions. The full 20-case set is available by setting N_CASES = 20 in SKILL.md. We chose n = 5 as a reproducibility-first default aligned with the Claw4S conference's emphasis on agent-executable, time-bounded skills (target runtime < 15 minutes on Colab free tier).
3.2 Known-Performance Calibration Check
To verify that the experimental pipeline is functioning correctly — not merely that conditions differ — we compare our best-condition result (bbox + best slice, mean Dice 0.775) against published SAM performance ranges on large abdominal CT organs. Mazurowski et al. [9] report IoU of 0.70–0.88 for large well-delineated abdominal structures with box prompts; Huang et al. [10] report similar ranges. Our result of Dice 0.775 (approximately equivalent to IoU 0.633 for this organ size and shape) falls within the lower portion of the expected range, consistent with the known difficulty of the liver CT boundary under portal venous enhancement. This calibration confirms the pipeline is neither erroneously inflating nor suppressing performance.
We do not interpret these comparisons as a formal benchmark, as differences in dataset, protocol, and slice selection methodology preclude direct numerical comparison. They serve only as a sanity check on the plausibility of our results.
3.3 Oracle Prompt Design and Its Bias
All prompts in this study are oracle-derived: centroid coordinates, bounding boxes, and grid points are all computed from the ground truth mask. This design imposes a deliberate and explicit upper-bound bias. It means that our reported Dice values overestimate what a deployed zero-shot SAM2 system would achieve with automatically generated or user-provided prompts. We choose this design intentionally: it eliminates prompt localisation error as a confound, allowing us to attribute any observed failure to SAM2's image encoder and mask decoder rather than to poor spatial prompt placement. A system that fails with perfect oracle prompts will not be rescued by better prompt generation in deployment — and this is the key claim our design allows us to make.
Readers should therefore interpret Table 1 as upper-bound performance estimates for each prompt strategy, not as realistic deployment predictions. Real-world performance with automated bounding-box detection or clinical user clicks will be lower by an amount that depends on the quality of the prompt generation method.
3.4 Model
SAM2 ViT-B (sam2_hiera_base_plus.pt, ~160 MB, SHA-256 recorded in requirements.txt) is downloaded from the official Meta AI repository (https://github.com/facebookresearch/sam2) and used in inference-only mode with no fine-tuning, adapter layers, or prompt engineering beyond the three explicit strategies defined below. Each CT slice is converted to uint8 RGB using a liver-optimised CT window (centre 60 HU, width 400 HU) following standard liver parenchyma display conventions [15], applied consistently across all conditions.
3.5 Experimental Design
We evaluate all combinations of two independent factors across 5 test cases, yielding 3 × 2 × 5 = 30 inference runs total. No hyperparameters are tuned; no threshold is optimised on the test cases.
Factor 1 — Prompt strategy (3 levels):
| Strategy | Description | Spatial information provided to SAM2 |
|---|---|---|
| Center point | Single point at centroid of GT mask | Location only |
| Bounding box | Tight axis-aligned box around GT mask | Location + spatial extent |
| Grid of points | Five evenly spaced GT foreground points | Location + interior sample distribution |
Factor 2 — Slice selection (2 levels):
| Strategy | Description |
|---|---|
| Best slice | Axial slice with maximum liver pixel area |
| Mid slice | Fixed anatomical middle slice of the volume |
3.6 Evaluation Metrics and Statistical Reporting
Dice Similarity Coefficient (DSC):
95th Percentile Hausdorff Distance (HD95):
Both metrics are computed on the selected 2D axial slice using the medpy library [16]. Following Taha and Hanbury [3], Dice < 0.5 defines the failure threshold. All means are reported with standard deviations. Pearson correlation coefficients are reported with two-sided p-values; given n = 5, no correlation test in this study has adequate power to detect effects smaller than r ≈ 0.85 at α = 0.05, so all p-values should be interpreted as indicative rather than confirmatory. No multiple comparison corrections are applied given the explicitly exploratory scope.
3.7 Algorithm
The complete experimental pipeline is formalised in Algorithm 1. The three non-trivial algorithmic decisions — slice selection, oracle prompt construction, and the inference loop — are made explicit below to ensure unambiguous reproducibility and to allow agents to verify the logic without executing the full skill.
Algorithm 1: MedSeg-Eval Factorial Inference Pipeline
Input:
Cases ← {c₁, …, c₅} // CHAOS CT volumes + GT masks
Prompts ← {center_point, bbox, grid_points}
Slices ← {best_slice, mid_slice}
model ← SAM2-ViT-B // inference-only, no fine-tuning
τ ← 0.5 // Dice failure threshold [3]
Output:
Results ← 30 × {Dice, HD95, failure_flag, case, slice_strategy, prompt}
──────────────────────────────────────────────────────────────────────
Procedure SELECT_SLICE(volume V, mask M, strategy s):
if s = best_slice then
i* ← argmax_i { sum_{x,y}( M[x,y,i] ) } // max liver area in 2D
else // mid_slice
i* ← floor( depth(V) / 2 ) // fixed anatomical midpoint
return V[:,:,i*], M[:,:,i*], i*
──────────────────────────────────────────────────────────────────────
Procedure BUILD_ORACLE_PROMPT(mask_slice G, strategy p):
// All prompts derived from ground truth G — oracle upper bound [§3.3]
foreground ← { (x,y) : G[y,x] > 0 }
if foreground = ∅ then return NULL // skip degenerate slice
if p = center_point then
cx ← mean_x(foreground)
cy ← mean_y(foreground)
return { point_coords: [(cx, cy)], point_labels: [1] }
if p = bbox then
x1, x2 ← min_x(foreground), max_x(foreground)
y1, y2 ← min_y(foreground), max_y(foreground)
return { box: [x1, y1, x2, y2] }
if p = grid_points then
if |foreground| < 5 then return NULL
// Sample 5 points at evenly spaced indices (seed=42, row-major order)
sorted_pts ← sort(foreground, order=row_major)
indices ← linspace(0, |sorted_pts|−1, k=5, dtype=int)
pts ← { sorted_pts[i] : i in indices }
return { point_coords: pts, point_labels: [1,1,1,1,1] }
──────────────────────────────────────────────────────────────────────
Procedure WINDOW_AND_ENCODE(ct_slice V_s):
// Standard liver CT windowing: centre=60 HU, width=400 HU [15]
V_w ← clip(V_s, lo=−140, hi=340)
V_n ← (V_w − (−140)) / 480 × 255 // normalise to [0,255] uint8
return stack([V_n, V_n, V_n], axis=2) // H×W×3 RGB for SAM2 encoder
──────────────────────────────────────────────────────────────────────
Main:
Results ← []
for c in Cases do
V, M ← load_DICOM_and_masks(c)
for s in Slices do
V_s, G_s, i* ← SELECT_SLICE(V, M, s)
rgb ← WINDOW_AND_ENCODE(V_s)
model.set_image(rgb) // image encoded once per slice
for p in Prompts do
prompt ← BUILD_ORACLE_PROMPT(G_s, p)
if prompt = NULL then continue
masks, scores ← model.predict(prompt, multimask_output=True)
P* ← masks[ argmax(scores) ] // highest-confidence mask
dice ← 2|P* ∩ G_s| / (|P*| + |G_s|)
hd95 ← HD95(P*, G_s) // via medpy [16]
fail ← (dice < τ)
Results.append({ c, s, p, i*, dice, hd95, fail,
liver_px: |G_s| })
return ResultsNote on determinism: linspace(0, N−1, k=5, dtype=int) draws indices at equal spacing regardless of Python set iteration order, ensuring grid-point sampling is fully deterministic across environments given the same sorted foreground pixel list.
4. Results
4.1 RQ1 — Prompt Sensitivity
Table 1 reports mean Dice ± SD and HD95 for all six experimental conditions. The bounding-box prompt on the best slice is the only condition achieving acceptable accuracy: mean Dice 0.775 ± 0.084, HD95 108.3 px. All point-based prompts fail consistently — center-point and grid-point strategies produce mean Dice 0.443 on the best slice (HD95 ≈ 215 px), statistically indistinguishable from each other despite the latter using five oracle prompts versus one.
Table 1. Mean ± SD Dice and HD95 per experimental condition (CHAOS CT Liver, n = 5 cases). All prompts are oracle-derived and represent upper-bound estimates [§3.3]. Failure threshold: Dice < 0.5 [3]. Best result per slice strategy in bold.
| Slice Strategy | Prompt | Dice ↑ | HD95 (px) ↓ | Failure rate |
|---|---|---|---|---|
| Best slice | center point | 0.443 ± 0.067 | 215.2 | 5/5 (100%) |
| Best slice | bbox | 0.775 ± 0.084 | 108.3 | 0/5 (0%) |
| Best slice | grid points | 0.443 ± 0.066 | 214.9 | 5/5 (100%) |
| Mid slice | center point | 0.341 ± 0.055 | 238.9 | 5/5 (100%) |
| Mid slice | bbox | 0.375 ± 0.339 | 143.3 | 2/5 (40%) |
| Mid slice | grid points | 0.340 ± 0.056 | 247.0 | 5/5 (100%) |
The equivalence of center-point and grid-point performance under oracle conditions warrants mechanistic interpretation. Liver parenchyma under portal venous phase CT windowing presents as a near-uniform bright region with subtle, low-contrast boundaries. When SAM2's ViT image encoder — pre-trained on natural images [1] — processes this uniform interior, all foreground points regardless of spatial distribution fall within a featureless region providing no texture gradient signal. SAM2's point-prompt encoder therefore saturates after the first foreground input; additional points carry no new boundary information. This is consistent with findings by Mazurowski et al. [9] and Huang et al. [10] that SAM's CT performance is substantially inferior to performance on visually richer modalities.
The practical implication is that collecting additional foreground clicks — a common design choice in interactive annotation interfaces — is wasted effort for CT liver segmentation without domain adaptation, regardless of how carefully those clicks are placed.
4.2 RQ2 — Slice Selection
The effect of slice selection is strongly prompt-dependent. For the bounding-box prompt, selecting the best slice over the mid slice yields a mean Dice gain of +0.400. For center-point and grid-point prompts, the gain is only +0.102 and +0.103 respectively — approximately four times smaller.
This asymmetry has a direct mechanistic explanation. A bounding box from the best (maximum-area) liver slice tightly encloses the organ at its maximum cross-sectional extent, providing SAM2 with a tight and informative spatial prior. The same box from the mid slice may enclose substantially more non-liver tissue, loosening the spatial constraint and introducing confounding context. Point-based prompts reduce to spatial coordinates regardless of slice choice, so the information content is low in both cases and the slice effect is muted.
The mid-slice bbox condition exhibits extreme variance (0.375 ± 0.339), driven by two catastrophic failures — case 16 (Dice = 0.021) and case 18 (Dice = 0.000) — where the mid-slice falls at or outside the boundary of the main liver volume. This instability makes mid-slice bbox unreliable in practice, and illustrates an important deployment risk: a condition that performs well on three of five cases while catastrophically failing on two cannot be trusted without per-case slice quality checks.
4.3 RQ3 — Failure Analysis
Overall failure rate: 20 of 30 inference runs (66.7%) produced Dice < 0.5. All 10 center-point and all 10 grid-point runs failed. Of the 10 bounding-box runs, 8 succeeded; both failures occurred in the mid-slice condition (cases 16 and 18).
Table 2. Failure counts and liver pixel area–Dice Pearson r on best-slice condition (n = 5). Note: with n = 5, the study is underpowered to detect correlations smaller than r ≈ 0.85 at α = 0.05; these p-values are reported for completeness and should not be interpreted as confirmatory tests.
| Prompt | Failures (all slices) | Rate | Size–Dice r | p-value |
|---|---|---|---|---|
| center point | 10 / 10 | 100% | 0.41 | 0.49 |
| bbox | 2 / 10 | 20% | −0.13 | 0.83 |
| grid points | 10 / 10 | 100% | 0.41 | 0.50 |
No size–Dice correlation reaches statistical significance. However, the low power of this analysis (n = 5) means a true moderate correlation (r ≈ 0.5–0.7) could exist and remain undetected. We interpret these results as evidence against a strong liver-size effect, not as evidence of no effect whatsoever. Failures appear idiosyncratic at this sample size.
Qualitative inspection of the three worst-performing bbox + best-slice cases (cases 14, 1, and 16; Dice = 0.67, 0.72, 0.78) reveals the dominant failure mode: SAM2 fills the bounding box as a near-rectangular blob, with the predicted boundary approximating the box geometry rather than the organ contour. This box-filling artefact is the direct consequence of absent salient image boundaries within the box — SAM2's mask decoder, pre-trained on natural images [1] where objects exhibit texture, colour, and edge gradients, defaults to filling the constrained region when no such cues are present. This is qualitatively consistent with Ji et al. [11], who observed that SAM produces poor results in scenes where the target blends with the background.
5. Discussion
The results converge on a single underlying cause for SAM2's limitations on CT liver segmentation: the near-uniform Hounsfield-unit appearance of liver parenchyma under standard windowing removes the texture and edge signals that SAM2's ViT encoder was pre-trained to exploit on natural images [1, 5]. This manifests in three experimentally distinct observations: point encoder saturation (RQ1), box-edge dependency (RQ2), and box-filling failure (RQ3).
On the oracle prompt upper bound. The strongest condition in our study — bbox + best slice, Dice 0.775 — represents a best-case ceiling for zero-shot SAM2 on this task, achievable only with perfect bounding-box prompts derived from ground truth. A realistic clinical workflow with automated detection or user-provided bounding boxes will produce lower Dice, likely in the range 0.60–0.72 based on the known degradation reported when oracle boxes are replaced with detector-predicted boxes in analogous SAM studies [9]. We emphasise this to avoid the overclaiming that has characterised some zero-shot foundation model evaluations: SAM2 out-of-the-box is not a viable clinical liver segmentation tool.
On zero-shot versus fine-tuned approaches. We do not claim that zero-shot SAM2 should be preferred for CT liver segmentation — on the contrary, our results reinforce the conclusion of Ma et al. [12], Wu et al. [13], and Cheng et al. [14] that domain adaptation is necessary. The value of this study is not in proposing SAM2 as a deployment solution, but in characterising precisely where and why zero-shot performance breaks down, providing a principled basis for deciding whether fine-tuning, prompt engineering, or windowing adaptation is the most efficient intervention.
On generalisability. The study uses five healthy liver-donor subjects scanned at portal venous phase. Results may not generalise to: (a) pathological livers with tumours, cirrhosis, or altered enhancement; (b) non-contrast or arterial phase acquisitions with different liver-background contrast; (c) other CT scanners or institutions with different noise characteristics; or (d) other abdominal organs with different size, shape, or texture properties. We explicitly caution against extrapolating these findings beyond the stated scope.
Limitations. n = 5 limits statistical power throughout; all results are exploratory. Oracle prompts impose an upper-bound bias that overestimates real-world performance [§3.3]. Evaluation is on single 2D slices; volumetric extension via SAM2's video propagation mode [1] is a natural future direction. All p-values are underpowered and should not be interpreted as confirmatory.
6. Conclusion
MedSeg-Eval provides a fast, fully reproducible, and agent-executable analysis of SAM2 on abdominal CT liver segmentation under oracle-prompt conditions, yielding three practically significant findings:
- Bounding-box prompts are the only viable zero-shot strategy for CT liver under oracle conditions; all point-based strategies fail at a 100% rate even with perfect prompts.
- Slice selection has a Dice impact of up to +0.40 for bbox but is negligible (≈+0.10) for point-based prompts — a prompt-dependent interaction not previously quantified for this task.
- Grid-of-points prompting offers no benefit over a single centroid under oracle conditions, providing evidence that SAM2's point-prompt encoder saturates on homogeneous CT texture.
These findings establish an oracle-prompt performance ceiling for zero-shot SAM2 on CT liver segmentation. They do not constitute a claim that SAM2 is a viable deployment tool for this task without domain adaptation.
Reproducibility Statement
All results in this paper are fully reproducible by executing SKILL.md in order. Software dependencies are pinned with exact version numbers in requirements.txt, generated via pip freeze at execution time. The dataset is publicly archived at DOI 10.5281/zenodo.3431873 under CC-BY-SA 4.0. The random seed is fixed to 42 throughout. No manual interventions, parameter tuning, or case selection based on performance were performed at any stage of this study.
References
[1] N. Ravi et al., "SAM 2: Segment Anything in Images and Videos," arXiv:2408.00714, 2024.
[2] A. E. Kavur et al., "CHAOS Challenge — combined (CT-MR) healthy abdominal organ segmentation," Medical Image Analysis, vol. 69, p. 101950, 2021.
[3] A. A. Taha and A. Hanbury, "Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool," BMC Medical Imaging, vol. 15, no. 1, p. 29, 2015.
[4] F. Isensee et al., "nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation," Nature Methods, vol. 18, no. 2, pp. 203–211, 2021.
[5] A. Kirillov et al., "Segment Anything," Proceedings of the IEEE/CVF ICCV, pp. 4015–4026, 2023.
[6] O. Ronneberger, P. Fischer, and T. Brox, "U-Net: Convolutional Networks for Biomedical Image Segmentation," MICCAI, LNCS vol. 9351, pp. 234–241, 2015.
[7] Ö. Çiçek et al., "3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation," MICCAI, pp. 424–432, 2016.
[8] J. Chen et al., "TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation," arXiv:2102.04306, 2021.
[9] M. A. Mazurowski et al., "Segment Anything Model for Medical Image Analysis: An Experimental Study," Medical Image Analysis, vol. 89, p. 102918, 2023.
[10] Y. Huang et al., "Segment Anything Model for Medical Images?" Medical Image Analysis, vol. 92, p. 103061, 2024.
[11] W. Ji et al., "Segment Anything Is Not Always Perfect: An Investigation of SAM on Different Real-World Applications," arXiv:2304.05750, 2023.
[12] J. Ma et al., "Segment Anything in Medical Images," Nature Communications, vol. 15, p. 654, 2024.
[13] J. Wu et al., "Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation," Medical Image Analysis, vol. 102, p. 103547, 2025.
[14] J. Cheng et al., "SAM-Med2D," arXiv:2308.16184, 2023.
[15] W. R. Hendee and E. R. Ritenour, Medical Imaging Physics, 4th ed. New York: Wiley-Liss, 2002.
[16] O. Maier et al., "medpy: Medical Image Processing in Python," Journal of Open Source Software, vol. 7, no. 76, p. 4462, 2022.
Reproducibility: Skill File
Use this skill file to reproduce the research with an AI agent.
# MedSeg-Eval: Analysing SAM2 Performance on Abdominal CT Liver Segmentation
## Prompt Sensitivity, Slice Selection Strategy & Failure Analysis on the CHAOS CT Dataset
---
## Overview
This skill presents a comprehensive analysis of **SAM2 (ViT-B)** on abdominal CT
liver segmentation using the publicly available **CHAOS CT dataset** (CC-BY-SA 4.0).
Rather than a simple benchmark, the skill investigates three research questions:
| # | Research Question |
|---|------------------|
| RQ1 | **Prompt Sensitivity** — How much does prompt strategy (center point, bounding box, grid of points) affect SAM2 Dice and HD95? |
| RQ2 | **Slice Selection** — Does SAM2 perform differently on the best (max liver area) slice vs a random mid-volume slice? |
| RQ3 | **Failure Analysis** — Which cases and conditions cause SAM2 to fail (Dice < 0.5), and does liver size correlate with performance? |
**Target structure:** Liver (single organ, large, convex — a good stress-test for prompt strategies)
**Metric:** Dice coefficient and Hausdorff Distance 95 (HD95) on 2D axial slices
---
## Prerequisites & Environment Setup
### Step 1 — Install dependencies
```bash
pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu
pip install git+https://github.com/facebookresearch/sam2.git
pip install pydicom Pillow medpy matplotlib pandas scipy seaborn scikit-image tqdm requests
```
> **Note for Colab:** Prefix each pip command with `!`. If using a GPU runtime,
> omit `--index-url` so torch installs with CUDA support.
### Step 2 — Set output directory
```python
import os
OUTPUT_DIR = "./medseg_eval_outputs"
os.makedirs(OUTPUT_DIR, exist_ok=True)
os.makedirs(f"{OUTPUT_DIR}/figures", exist_ok=True)
os.makedirs(f"{OUTPUT_DIR}/metrics", exist_ok=True)
```
### Step 3 — Set global seed for reproducibility
```python
import random, numpy as np, torch
SEED = 42
random.seed(SEED)
np.random.seed(SEED)
torch.manual_seed(SEED)
```
---
## Phase 1 — Download & Prepare CHAOS CT Dataset
The CHAOS CT training split is publicly available on Zenodo under CC-BY-SA 4.0
(DOI: 10.5281/zenodo.3431873). It contains 20 contrast-enhanced CT cases
with liver segmentation masks in DICOM + PNG format.
```python
import requests, zipfile, os, glob
import numpy as np
import pydicom
from PIL import Image
DATA_DIR = "./data/CHAOS_CT"
ZENODO_URL = "https://zenodo.org/record/3431873/files/CHAOS_Train_Sets.zip"
N_CASES = 5 # first 5 cases — extend to 20 for full dataset analysis
os.makedirs("./data", exist_ok=True)
zip_path = "./data/CHAOS_Train_Sets.zip"
if not os.path.exists(zip_path):
print("Downloading CHAOS CT from Zenodo (~900 MB)...")
r = requests.get(ZENODO_URL, stream=True)
r.raise_for_status()
total = int(r.headers.get("content-length", 0))
done = 0
with open(zip_path, "wb") as f:
for chunk in r.iter_content(chunk_size=1024 * 1024):
f.write(chunk)
done += len(chunk)
print(f"\r {done/1e6:.1f} / {total/1e6:.1f} MB", end="")
print("\nDownload complete.")
if not os.path.exists("./data/Train_Sets"):
print("Extracting...")
with zipfile.ZipFile(zip_path, "r") as z:
z.extractall("./data/")
print("Extracted.")
# Actual CHAOS CT folder structure after extraction:
# ./data/Train_Sets/CT/{patient_id}/DICOM_anon/*.dcm (CT slices)
# /Ground/*.png (liver masks)
def load_chaos_case(case_dir):
"""
Load a CHAOS CT case.
Returns:
ct_vol : np.ndarray (H, W, D) float32 — CT Hounsfield units
mask_vol : np.ndarray (H, W, D) uint8 — binary liver mask
"""
dicom_dir = os.path.join(case_dir, "DICOM_anon")
mask_dir = os.path.join(case_dir, "Ground")
dcm_files = sorted(
glob.glob(f"{dicom_dir}/*.dcm"),
key=lambda f: pydicom.dcmread(f).InstanceNumber
)
ct_vol = np.stack(
[pydicom.dcmread(f).pixel_array.astype(np.float32) for f in dcm_files],
axis=2
)
mask_files = sorted(glob.glob(f"{mask_dir}/*.png"))
mask_vol = np.stack(
[(np.array(Image.open(m).convert("L")) > 127).astype(np.uint8)
for m in mask_files],
axis=2
)
# Align depth (occasionally off by 1 between DICOM and PNG counts)
d = min(ct_vol.shape[2], mask_vol.shape[2])
return ct_vol[:, :, :d], mask_vol[:, :, :d]
ct_root = "./data/Train_Sets/CT"
case_dirs = sorted([
os.path.join(ct_root, d) for d in os.listdir(ct_root)
if os.path.isdir(os.path.join(ct_root, d))
])[:N_CASES]
print(f"\nLoading {len(case_dirs)} CHAOS CT cases...")
CASES = []
for cd in case_dirs:
case_id = os.path.basename(cd)
ct_vol, mask_vol = load_chaos_case(cd)
CASES.append({"case_id": case_id, "ct": ct_vol, "mask": mask_vol})
print(f" Case {case_id}: CT={ct_vol.shape}, mask={mask_vol.shape}, "
f"liver voxels={mask_vol.sum()}")
# Expected output example:
# Case 1: CT=(512, 512, 90), mask=(512, 512, 90), liver voxels=142300
```
---
## Phase 2 — Download SAM2 ViT-B Checkpoint
```python
import urllib.request
SAM2_CHECKPOINT = "./weights/sam2_hiera_base_plus.pt"
SAM2_CONFIG = "sam2_hiera_b+.yaml"
os.makedirs("./weights", exist_ok=True)
if not os.path.exists(SAM2_CHECKPOINT):
url = "https://dl.fbaipublicfiles.com/segment_anything_2/072824/sam2_hiera_base_plus.pt"
print("Downloading SAM2 ViT-B checkpoint (~160 MB)...")
urllib.request.urlretrieve(url, SAM2_CHECKPOINT)
print("Downloaded.")
else:
print("SAM2 checkpoint already present.")
```
---
## Phase 3 — Utility Functions
```python
import numpy as np
from medpy.metric.binary import dc, hd95
# ── Slice selection ────────────────────────────────────────────────────────────
def best_liver_slice(ct_vol, mask_vol):
"""Return the axial slice index with the maximum liver pixel area."""
areas = [mask_vol[:, :, i].sum() for i in range(mask_vol.shape[2])]
idx = int(np.argmax(areas))
return ct_vol[:, :, idx], mask_vol[:, :, idx], idx
def mid_slice(ct_vol, mask_vol):
"""Return the anatomical middle axial slice."""
idx = mask_vol.shape[2] // 2
return ct_vol[:, :, idx], mask_vol[:, :, idx], idx
# ── CT windowing & normalisation ───────────────────────────────────────────────
def normalize_ct_slice(slc, window_center=60, window_width=400):
"""
Apply liver-optimised CT window then normalise to 0-255 uint8 RGB.
Window centre 60 HU / width 400 HU is standard for liver parenchyma.
"""
lo = window_center - window_width / 2
hi = window_center + window_width / 2
slc = np.clip(slc, lo, hi)
slc = (slc - lo) / (hi - lo)
slc = (slc * 255).astype(np.uint8)
return np.stack([slc, slc, slc], axis=-1) # H x W x 3
# ── Prompt generation ──────────────────────────────────────────────────────────
def get_bbox_from_mask(mask):
"""Return [x1, y1, x2, y2] tight bounding box from binary 2D mask."""
rows = np.any(mask, axis=1)
cols = np.any(mask, axis=0)
rmin, rmax = np.where(rows)[0][[0, -1]]
cmin, cmax = np.where(cols)[0][[0, -1]]
return [cmin, rmin, cmax, rmax]
def build_prompts(gt_slice, strategy):
"""
Build SAM2 prompts from a ground-truth 2D mask (oracle prompts).
Strategies:
center_point — single centroid point
bbox — tight bounding box
grid_points — 5 evenly sampled foreground points
Returns None if the mask has no foreground on this slice.
"""
gt_bin = (gt_slice > 0)
if not gt_bin.any():
return None
if strategy == "center_point":
ys, xs = np.where(gt_bin)
return {
"point_coords": np.array([[int(xs.mean()), int(ys.mean())]]),
"point_labels": np.array([1])
}
elif strategy == "bbox":
return {"box": np.array(get_bbox_from_mask(gt_bin))}
elif strategy == "grid_points":
ys, xs = np.where(gt_bin)
if len(ys) < 5:
return None
idx = np.linspace(0, len(ys) - 1, 5, dtype=int)
pts = np.stack([xs[idx], ys[idx]], axis=1)
return {"point_coords": pts, "point_labels": np.ones(5, dtype=int)}
# ── Metrics ────────────────────────────────────────────────────────────────────
def compute_metrics(pred_mask, gt_mask):
"""Return dict with Dice and HD95 for two binary 2D masks."""
pred = pred_mask.astype(bool)
gt = gt_mask.astype(bool)
if not gt.any():
return {"dice": float("nan"), "hd95": float("nan")}
if not pred.any():
return {"dice": 0.0, "hd95": float("nan")}
dice_val = dc(pred, gt)
try:
hd_val = hd95(pred, gt)
except Exception:
hd_val = float("nan")
return {"dice": round(dice_val, 4), "hd95": round(hd_val, 4)}
```
---
## Phase 4 — SAM2 Inference
We run SAM2 across all combinations of:
- **3 prompt strategies** × **2 slice selection methods** × **5 cases**
= 30 inference runs total (fast on Colab, ~10–15 min)
```python
from sam2.build_sam import build_sam2
from sam2.sam2_image_predictor import SAM2ImagePredictor
sam2_model = build_sam2(SAM2_CONFIG, SAM2_CHECKPOINT, device="cpu")
sam2_predictor = SAM2ImagePredictor(sam2_model)
PROMPT_STRATEGIES = ["center_point", "bbox", "grid_points"]
SLICE_STRATEGIES = ["best_slice", "mid_slice"]
all_results = []
for case in CASES:
case_id = case["case_id"]
ct_vol = case["ct"]
mask_vol = case["mask"]
slice_fns = {
"best_slice": best_liver_slice,
"mid_slice": mid_slice,
}
for slice_strategy, slice_fn in slice_fns.items():
ct_slc, gt_slc, slice_idx = slice_fn(ct_vol, mask_vol)
rgb_img = normalize_ct_slice(ct_slc)
gt_bin = gt_slc.astype(np.uint8)
# Skip slices with no liver (can happen at mid_slice for small livers)
if not gt_bin.any():
print(f" SKIP | {case_id} | {slice_strategy} | no liver on slice {slice_idx}")
continue
sam2_predictor.set_image(rgb_img)
for prompt_strategy in PROMPT_STRATEGIES:
prompts = build_prompts(gt_slc, prompt_strategy)
if prompts is None:
print(f" SKIP | {case_id} | {slice_strategy} | {prompt_strategy} | empty mask")
continue
masks, scores, _ = sam2_predictor.predict(
**prompts,
multimask_output=True
)
best_mask = masks[np.argmax(scores)].astype(np.uint8)
metrics = compute_metrics(best_mask, gt_bin)
record = {
"case_id": case_id,
"slice_strategy": slice_strategy,
"prompt_strategy": prompt_strategy,
"slice_idx": slice_idx,
"liver_px": int(gt_bin.sum()),
**metrics
}
all_results.append(record)
print(f" {case_id} | {slice_strategy} | {prompt_strategy} | "
f"Dice={metrics['dice']:.3f} | HD95={metrics['hd95']}")
print(f"\nTotal inference runs completed: {len(all_results)}")
# Expected: 30 (5 cases × 2 slice strategies × 3 prompt strategies)
```
---
## Phase 5 — Save Results
```python
import pandas as pd
df = pd.DataFrame(all_results)
df.to_csv(f"{OUTPUT_DIR}/metrics/all_results.csv", index=False)
# Summary: mean ± std grouped by prompt and slice strategy
summary = (
df.groupby(["slice_strategy", "prompt_strategy"])[["dice", "hd95"]]
.agg(["mean", "std"])
.round(3)
.reset_index()
)
summary.columns = [
"slice_strategy", "prompt_strategy",
"dice_mean", "dice_std", "hd95_mean", "hd95_std"
]
summary.to_csv(f"{OUTPUT_DIR}/metrics/summary_table.csv", index=False)
print("\n=== SUMMARY TABLE ===")
print(summary.to_string(index=False))
# Expected columns: slice_strategy | prompt_strategy | dice_mean | dice_std | hd95_mean | hd95_std
```
---
## Phase 6 — RQ1: Prompt Sensitivity Analysis
```python
import matplotlib.pyplot as plt
import seaborn as sns
fig, axes = plt.subplots(1, 2, figsize=(13, 5))
order = ["center_point", "bbox", "grid_points"]
# Dice
ax = axes[0]
sns.boxplot(
data=df, x="prompt_strategy", y="dice",
hue="slice_strategy", palette={"best_slice": "#2196F3", "mid_slice": "#FF9800"},
order=order, ax=ax
)
ax.set_title("RQ1 — Prompt Sensitivity: Dice\n(CHAOS CT Liver)", fontsize=11)
ax.set_xlabel("Prompt Strategy")
ax.set_ylabel("Dice Score")
ax.set_ylim(0, 1)
ax.tick_params(axis="x", rotation=15)
ax.axhline(0.5, color="red", linestyle="--", alpha=0.4, linewidth=1)
ax.legend(title="Slice strategy", fontsize=8)
# HD95
ax = axes[1]
sns.boxplot(
data=df, x="prompt_strategy", y="hd95",
hue="slice_strategy", palette={"best_slice": "#2196F3", "mid_slice": "#FF9800"},
order=order, ax=ax
)
ax.set_title("RQ1 — Prompt Sensitivity: HD95\n(CHAOS CT Liver)", fontsize=11)
ax.set_xlabel("Prompt Strategy")
ax.set_ylabel("HD95 (px)")
ax.tick_params(axis="x", rotation=15)
ax.legend(title="Slice strategy", fontsize=8)
plt.suptitle("SAM2 Prompt Sensitivity on CHAOS CT Liver Segmentation", fontsize=13, y=1.02)
plt.tight_layout()
plt.savefig(f"{OUTPUT_DIR}/figures/rq1_prompt_sensitivity.png", dpi=150, bbox_inches="tight")
plt.show()
print("Saved: rq1_prompt_sensitivity.png")
```
---
## Phase 7 — RQ2: Slice Selection Strategy
```python
fig, axes = plt.subplots(1, 2, figsize=(11, 5))
# Dice: best vs mid slice, grouped by prompt
ax = axes[0]
sns.barplot(
data=df, x="prompt_strategy", y="dice",
hue="slice_strategy", palette={"best_slice": "#2196F3", "mid_slice": "#FF9800"},
order=order, capsize=0.08, ax=ax, errwidth=1.5
)
ax.set_title("RQ2 — Slice Strategy: Mean Dice\n(CHAOS CT Liver)", fontsize=11)
ax.set_xlabel("Prompt Strategy")
ax.set_ylabel("Mean Dice Score")
ax.set_ylim(0, 1)
ax.tick_params(axis="x", rotation=15)
ax.axhline(0.5, color="red", linestyle="--", alpha=0.4, linewidth=1)
ax.legend(title="Slice strategy", fontsize=8)
# Paired difference: best_slice Dice - mid_slice Dice per case per prompt
pivot = df.pivot_table(
index=["case_id", "prompt_strategy"],
columns="slice_strategy",
values="dice"
).reset_index()
pivot["dice_gain"] = pivot["best_slice"] - pivot["mid_slice"]
ax = axes[1]
sns.barplot(
data=pivot, x="prompt_strategy", y="dice_gain",
order=order, palette="Set2", capsize=0.08, ax=ax, errwidth=1.5
)
ax.axhline(0, color="k", linewidth=0.8)
ax.set_title("RQ2 — Dice Gain: Best Slice vs Mid Slice\n(positive = best slice wins)", fontsize=11)
ax.set_xlabel("Prompt Strategy")
ax.set_ylabel("Δ Dice (best − mid)")
ax.tick_params(axis="x", rotation=15)
plt.suptitle("SAM2 Slice Selection Effect on CHAOS CT Liver Segmentation", fontsize=13, y=1.02)
plt.tight_layout()
plt.savefig(f"{OUTPUT_DIR}/figures/rq2_slice_selection.png", dpi=150, bbox_inches="tight")
plt.show()
print("Saved: rq2_slice_selection.png")
```
---
## Phase 8 — RQ3: Failure Analysis
### 8a — Flag failures (Dice < 0.5)
```python
failures = df[df["dice"] < 0.5].copy()
failures.to_csv(f"{OUTPUT_DIR}/metrics/failures.csv", index=False)
print(f"\n=== FAILURES (Dice < 0.5): {len(failures)} / {len(df)} runs ===")
if len(failures):
print(failures[["case_id","slice_strategy","prompt_strategy","dice","hd95"]].to_string(index=False))
else:
print("No failures detected.")
```
### 8b — Liver size vs Dice correlation
```python
from scipy.stats import pearsonr
fig, axes = plt.subplots(1, 3, figsize=(15, 5), sharey=True)
for ax, strategy in zip(axes, PROMPT_STRATEGIES):
subset = df[df["prompt_strategy"] == strategy]
for ss, color in [("best_slice", "#2196F3"), ("mid_slice", "#FF9800")]:
grp = subset[subset["slice_strategy"] == ss].dropna(subset=["dice"])
if len(grp) < 2:
continue
ax.scatter(grp["liver_px"], grp["dice"], label=ss,
color=color, s=70, alpha=0.8, edgecolors="k", linewidths=0.5)
r, p = pearsonr(grp["liver_px"], grp["dice"])
ax.set_title(f"{strategy}\n(r={r:.2f}, p={p:.2f})", fontsize=10)
ax.set_xlabel("Liver Size (pixels, 2D slice)")
ax.set_ylim(0, 1)
ax.axhline(0.5, color="red", linestyle="--", alpha=0.4, linewidth=1)
axes[0].set_ylabel("Dice Score")
axes[0].legend(fontsize=8)
plt.suptitle("RQ3 — Liver Size vs SAM2 Dice Score\n(CHAOS CT)", fontsize=13)
plt.tight_layout()
plt.savefig(f"{OUTPUT_DIR}/figures/rq3_size_vs_dice.png", dpi=150, bbox_inches="tight")
plt.show()
print("Saved: rq3_size_vs_dice.png")
```
### 8c — Visualise worst 3 cases (best-slice / bbox — the strongest prompt setting)
```python
df_bbox_best = df[
(df["prompt_strategy"] == "bbox") &
(df["slice_strategy"] == "best_slice")
].dropna(subset=["dice"])
worst3 = df_bbox_best.nsmallest(3, "dice")
n = len(worst3)
fig, axes = plt.subplots(n, 3, figsize=(12, 4 * n))
if n == 1:
axes = axes[np.newaxis, :]
for row_idx, (_, row) in enumerate(worst3.iterrows()):
case_id = str(row["case_id"])
case = next(c for c in CASES if str(c["case_id"]) == case_id)
ct_vol = case["ct"]
mask_vol = case["mask"]
ct_slc, gt_slc, si = best_liver_slice(ct_vol, mask_vol)
rgb_img = normalize_ct_slice(ct_slc)
gt_bin = gt_slc.astype(np.uint8)
# Re-run SAM2 to get the prediction mask for visualisation
sam2_predictor.set_image(rgb_img)
prompts = build_prompts(gt_slc, "bbox")
masks, scores, _ = sam2_predictor.predict(**prompts, multimask_output=True)
pred_mask = masks[np.argmax(scores)].astype(np.uint8)
axes[row_idx, 0].imshow(rgb_img); axes[row_idx, 0].axis("off")
axes[row_idx, 0].set_title(f"CT Input (liver window) — Case {case_id}", fontsize=9)
axes[row_idx, 1].imshow(rgb_img)
axes[row_idx, 1].imshow(gt_bin, alpha=0.45, cmap="Greens")
axes[row_idx, 1].set_title("Ground Truth", fontsize=9)
axes[row_idx, 1].axis("off")
axes[row_idx, 2].imshow(rgb_img)
axes[row_idx, 2].imshow(pred_mask, alpha=0.45, cmap="Reds")
axes[row_idx, 2].set_title(f"SAM2 (bbox) Dice={row['dice']:.2f}", fontsize=9)
axes[row_idx, 2].axis("off")
plt.suptitle("RQ3 — Worst 3 Failure Cases: SAM2 bbox / Best Slice — CHAOS CT", fontsize=13)
plt.tight_layout()
plt.savefig(f"{OUTPUT_DIR}/figures/rq3_worst_cases.png", dpi=150, bbox_inches="tight")
plt.show()
print("Saved: rq3_worst_cases.png")
```
---
## Phase 9 — Final Summary Heatmap
A heatmap of mean Dice across all prompt × slice strategy combinations
gives a concise single-figure summary of all three research questions.
```python
pivot_heatmap = summary.pivot(
index="prompt_strategy",
columns="slice_strategy",
values="dice_mean"
)
# Reorder rows for readability
row_order = ["center_point", "bbox", "grid_points"]
pivot_heatmap = pivot_heatmap.reindex(row_order)
fig, ax = plt.subplots(figsize=(6, 4))
im = ax.imshow(pivot_heatmap.values, cmap="RdYlGn", vmin=0, vmax=1, aspect="auto")
plt.colorbar(im, ax=ax, label="Mean Dice Score")
ax.set_xticks(range(len(pivot_heatmap.columns)))
ax.set_xticklabels(pivot_heatmap.columns, fontsize=10)
ax.set_yticks(range(len(pivot_heatmap.index)))
ax.set_yticklabels(pivot_heatmap.index, fontsize=10)
for i in range(len(pivot_heatmap.index)):
for j in range(len(pivot_heatmap.columns)):
val = pivot_heatmap.values[i, j]
ax.text(j, i, f"{val:.3f}", ha="center", va="center",
fontsize=12, color="black", fontweight="bold")
ax.set_title("SAM2 Mean Dice — Prompt × Slice Strategy\n(CHAOS CT Liver, n=5)",
fontsize=12)
plt.tight_layout()
plt.savefig(f"{OUTPUT_DIR}/figures/summary_heatmap.png", dpi=150, bbox_inches="tight")
plt.show()
print("Saved: summary_heatmap.png")
```
---
## Phase 10 — Expected Outputs
```
medseg_eval_outputs/
├── metrics/
│ ├── all_results.csv # One row per (case × slice strategy × prompt strategy)
│ ├── summary_table.csv # Mean ± Std Dice + HD95 per combination
│ └── failures.csv # Runs with Dice < 0.5
└── figures/
├── rq1_prompt_sensitivity.png # Boxplots: Dice & HD95 by prompt strategy
├── rq2_slice_selection.png # Bar charts: best vs mid slice + Δ Dice
├── rq3_size_vs_dice.png # Scatter: liver size vs Dice per prompt
├── rq3_worst_cases.png # Overlay visualisations of worst 3 cases
└── summary_heatmap.png # Heatmap: mean Dice across all combinations
```
---
## Reproducibility Checklist
- [x] Dataset: CHAOS CT training split (CC-BY-SA 4.0), DOI 10.5281/zenodo.3431873
- [x] Model: SAM2 ViT-B checkpoint from Meta AI (public, no login required)
- [x] Random seed fixed to `42`
- [x] Inference only — no training, no fine-tuning
- [x] All outputs saved to `./medseg_eval_outputs/`
```bash
pip freeze > ./medseg_eval_outputs/requirements.txt
```
---Discussion (0)
to join the discussion.
No comments yet. Be the first to discuss this paper.