---
name: "HuggingFace Model Naming Patterns vs. Performance"
description: "Analyzes whether models with 'chat', 'instruct', or 'DPO' in the name show different popularity and community engagement metrics on HuggingFace, using likes and downloads as proxy signals."
version: "1.0.0"
author: "Claw 🦞, David Austin"
tags: ["claw4s-2026", "ml-meta-analysis", "huggingface", "model-naming", "instruction-tuning"]
python_version: ">=3.8"
dependencies: []
---

# HuggingFace Model Naming Patterns Analysis

## Overview

This skill downloads model metadata from the HuggingFace API, categorizes models by naming
conventions (chat, instruct, DPO, base, RLHF, GPTQ, AWQ, GGUF, etc.), and analyzes whether
these naming tags correlate with community engagement metrics (downloads, likes). Since
instruction-following benchmark scores are not uniformly available via the API, we use
community engagement as a proxy signal and analyze the distribution of naming patterns
across the model ecosystem.

## Step 1: Create Workspace

```bash
mkdir -p /tmp/claw4s_auto_huggingface-model-naming-patterns
```

**Expected output:** Directory created, exit code 0.

## Step 2: Write Analysis Script

```bash
cat << 'SCRIPT_EOF' > /tmp/claw4s_auto_huggingface-model-naming-patterns/analyze.py
#!/usr/bin/env python3
"""
HuggingFace Model Naming Patterns Analysis
===========================================
Analyzes correlation between model naming conventions and community engagement
metrics on HuggingFace. Uses only Python 3.8+ standard library.

Data source: https://huggingface.co/api/models (pinned query parameters)
"""

import json
import hashlib
import os
import sys
import time
import urllib.request
import urllib.error
import math
import re
from collections import defaultdict
from pathlib import Path

# === Configuration ===
WORKSPACE = Path(__file__).parent
CACHE_DIR = WORKSPACE / "cache"
RESULTS_PATH = WORKSPACE / "results.json"
REPORT_PATH = WORKSPACE / "report.md"

# We fetch text-generation models sorted by downloads, in pages of 100.
# Text-generation is where naming conventions (chat, instruct, DPO) are meaningful.
# We also fetch a general sample for cross-domain comparison.
API_BASE = "https://huggingface.co/api/models"
MODELS_PER_PAGE = 100
NUM_PAGES_TEXTGEN = 20   # 2000 text-generation models
NUM_PAGES_GENERAL = 10   # 1000 general models for comparison

# Naming pattern categories — case-insensitive substring/boundary matching
NAMING_PATTERNS = {
    "chat": re.compile(r"(?:^|[\-_./])chat(?:[\-_./]|$)", re.IGNORECASE),
    "instruct": re.compile(r"(?:^|[\-_./])instruct(?:[\-_./]|$)", re.IGNORECASE),
    "dpo": re.compile(r"(?:^|[\-_./])dpo(?:[\-_./]|$)", re.IGNORECASE),
    "rlhf": re.compile(r"(?:^|[\-_./])rlhf(?:[\-_./]|$)", re.IGNORECASE),
    "gptq": re.compile(r"(?:^|[\-_./])gptq(?:[\-_./]|$)", re.IGNORECASE),
    "awq": re.compile(r"(?:^|[\-_./])awq(?:[\-_./]|$)", re.IGNORECASE),
    "gguf": re.compile(r"gguf", re.IGNORECASE),
    "lora": re.compile(r"(?:^|[\-_./])lora(?:[\-_./]|$)", re.IGNORECASE),
    "fine-tuned": re.compile(r"(?:[\-_./]ft(?:[\-_./]|$)|fine[\-_]?tun)", re.IGNORECASE),
    "base": re.compile(r"(?:^|[\-_./])base(?:[\-_./]|$)", re.IGNORECASE),
}

# Random seed for any sampling (determinism)
RANDOM_SEED = 42


def fetch_url(url, max_retries=3, backoff=2.0):
    """Fetch URL with retry logic and exponential backoff."""
    for attempt in range(max_retries):
        try:
            req = urllib.request.Request(url)
            req.add_header("User-Agent", "Claw4S-Analysis/1.0")
            with urllib.request.urlopen(req, timeout=30) as resp:
                return resp.read()
        except (urllib.error.URLError, urllib.error.HTTPError, OSError) as e:
            if attempt == max_retries - 1:
                raise
            wait = backoff * (2 ** attempt)
            print(f"  Retry {attempt + 1}/{max_retries} after {wait}s: {e}")
            time.sleep(wait)
    return None


def fetch_paginated(label, base_url, num_pages):
    """Fetch multiple pages using cursor-based pagination with caching."""
    CACHE_DIR.mkdir(parents=True, exist_ok=True)

    # Check if we have a complete cached dataset
    manifest_file = CACHE_DIR / f"{label}_manifest.json"
    if manifest_file.exists():
        manifest = json.loads(manifest_file.read_text())
        if manifest.get("pages_fetched", 0) >= num_pages:
            # Load all cached pages, verify SHA256
            all_models = []
            valid = True
            for i in range(manifest["pages_fetched"]):
                cache_file = CACHE_DIR / f"{label}_page_{i:03d}.json"
                sha_file = CACHE_DIR / f"{label}_page_{i:03d}.sha256"
                if not cache_file.exists() or not sha_file.exists():
                    valid = False
                    break
                data = cache_file.read_bytes()
                expected_sha = sha_file.read_text().strip()
                if hashlib.sha256(data).hexdigest() != expected_sha:
                    print(f"  SHA256 mismatch for {label} page {i}, refetching all...")
                    valid = False
                    break
                all_models.extend(json.loads(data))
            if valid:
                print(f"  Loaded {len(all_models)} cached {label} models ({manifest['pages_fetched']} pages)")
                return all_models

    # Fetch fresh data with cursor pagination
    all_models = []
    url = base_url
    for page in range(num_pages):
        print(f"  Fetching {label} page {page + 1}/{num_pages}...")
        req = urllib.request.Request(url)
        req.add_header("User-Agent", "Claw4S-Analysis/1.0")
        try:
            with urllib.request.urlopen(req, timeout=30) as resp:
                link_header = resp.headers.get("Link", "")
                raw = resp.read()
        except (urllib.error.URLError, urllib.error.HTTPError, OSError) as e:
            # Retry logic
            for attempt in range(2):
                wait = 2.0 * (2 ** attempt)
                print(f"    Retry {attempt + 1}/2 after {wait}s: {e}")
                time.sleep(wait)
                try:
                    req = urllib.request.Request(url)
                    req.add_header("User-Agent", "Claw4S-Analysis/1.0")
                    with urllib.request.urlopen(req, timeout=30) as resp:
                        link_header = resp.headers.get("Link", "")
                        raw = resp.read()
                    break
                except (urllib.error.URLError, urllib.error.HTTPError, OSError) as e2:
                    e = e2
            else:
                raise e

        # Cache page
        cache_file = CACHE_DIR / f"{label}_page_{page:03d}.json"
        sha_file = CACHE_DIR / f"{label}_page_{page:03d}.sha256"
        cache_file.write_bytes(raw)
        sha_file.write_text(hashlib.sha256(raw).hexdigest())

        models = json.loads(raw)
        if not models:
            print(f"    Empty page, stopping.")
            break
        all_models.extend(models)

        # Parse next cursor URL from Link header
        next_match = re.search(r'<([^>]+)>;\s*rel="next"', link_header)
        if next_match:
            url = next_match.group(1)
        else:
            print(f"    No next page link, stopping after {page + 1} pages.")
            break

        if page < num_pages - 1:
            time.sleep(0.3)

    # Save manifest
    manifest = {"label": label, "pages_fetched": min(page + 1, num_pages), "total_models": len(all_models)}
    manifest_file.write_text(json.dumps(manifest))

    return all_models


def fetch_models():
    """Fetch model metadata from HuggingFace API with caching.
    Fetches text-generation models (primary dataset) and general models (comparison)."""

    # Primary: text-generation models sorted by downloads
    textgen_url = (
        f"{API_BASE}?sort=downloads&direction=-1"
        f"&pipeline_tag=text-generation&limit={MODELS_PER_PAGE}"
    )
    textgen_models = fetch_paginated("textgen", textgen_url, NUM_PAGES_TEXTGEN)

    # Secondary: general models for cross-domain comparison
    general_url = (
        f"{API_BASE}?sort=downloads&direction=-1"
        f"&limit={MODELS_PER_PAGE}"
    )
    general_models = fetch_paginated("general", general_url, NUM_PAGES_GENERAL)

    # Merge, deduplicating by model ID
    seen_ids = set()
    all_models = []
    for m in textgen_models + general_models:
        mid = m.get("modelId") or m.get("id", "")
        if mid and mid not in seen_ids:
            seen_ids.add(mid)
            all_models.append(m)

    return all_models


def classify_model(model_id):
    """Classify a model by naming patterns. A model can match multiple patterns."""
    tags = []
    for pattern_name, pattern_re in NAMING_PATTERNS.items():
        if pattern_re.search(model_id):
            tags.append(pattern_name)
    if not tags:
        tags.append("unmarked")
    return tags


def safe_median(values):
    """Compute median without numpy."""
    if not values:
        return 0
    s = sorted(values)
    n = len(s)
    if n % 2 == 1:
        return s[n // 2]
    return (s[n // 2 - 1] + s[n // 2]) / 2


def safe_mean(values):
    """Compute mean without numpy."""
    if not values:
        return 0
    return sum(values) / len(values)


def safe_stdev(values):
    """Compute sample standard deviation without numpy."""
    if len(values) < 2:
        return 0
    m = safe_mean(values)
    variance = sum((x - m) ** 2 for x in values) / (len(values) - 1)
    return math.sqrt(variance)


def percentile(values, p):
    """Compute p-th percentile (0-100) without numpy."""
    if not values:
        return 0
    s = sorted(values)
    k = (len(s) - 1) * p / 100
    f = math.floor(k)
    c = math.ceil(k)
    if f == c:
        return s[int(k)]
    return s[f] * (c - k) + s[c] * (k - f)


def welch_t_test(group1, group2):
    """Welch's t-test (unequal variances) - returns t-statistic and approximate p-value."""
    n1, n2 = len(group1), len(group2)
    if n1 < 2 or n2 < 2:
        return 0, 1.0
    m1, m2 = safe_mean(group1), safe_mean(group2)
    v1 = safe_stdev(group1) ** 2
    v2 = safe_stdev(group2) ** 2
    se = math.sqrt(v1 / n1 + v2 / n2) if (v1 / n1 + v2 / n2) > 0 else 1e-10
    t_stat = (m1 - m2) / se

    # Approximate degrees of freedom (Welch-Satterthwaite)
    num = (v1 / n1 + v2 / n2) ** 2
    denom = ((v1 / n1) ** 2 / (n1 - 1) + (v2 / n2) ** 2 / (n2 - 1)) if (
        (v1 / n1) ** 2 / max(n1 - 1, 1) + (v2 / n2) ** 2 / max(n2 - 1, 1)
    ) > 0 else 1
    df = num / max(denom, 1e-10)

    # Approximate two-tailed p-value using normal approximation for large df
    # For df > 30 this is quite reasonable
    z = abs(t_stat)
    # Approximation of 2-tailed p from z (Abramowitz & Stegun)
    if z > 8:
        p_value = 0.0
    else:
        t_val = 1 / (1 + 0.2316419 * z)
        poly = t_val * (0.319381530 + t_val * (-0.356563782 + t_val * (
            1.781477937 + t_val * (-1.821255978 + 1.330274429 * t_val))))
        p_value = 2 * poly * math.exp(-z * z / 2) / math.sqrt(2 * math.pi)

    return t_stat, p_value


def effect_size_cohens_d(group1, group2):
    """Cohen's d effect size."""
    n1, n2 = len(group1), len(group2)
    if n1 < 2 or n2 < 2:
        return 0
    m1, m2 = safe_mean(group1), safe_mean(group2)
    s1, s2 = safe_stdev(group1), safe_stdev(group2)
    pooled_std = math.sqrt(((n1 - 1) * s1**2 + (n2 - 1) * s2**2) / (n1 + n2 - 2))
    if pooled_std == 0:
        return 0
    return (m1 - m2) / pooled_std


def analyze(models_raw):
    """Run the full analysis pipeline."""
    # Extract relevant fields
    models = []
    for m in models_raw:
        model_id = m.get("modelId") or m.get("id", "")
        downloads = m.get("downloads", 0)
        likes = m.get("likes", 0)
        pipeline_tag = m.get("pipeline_tag", "unknown")
        tags = m.get("tags", [])
        if model_id:
            models.append({
                "id": model_id,
                "downloads": downloads,
                "likes": likes,
                "pipeline_tag": pipeline_tag,
                "tags": tags,
                "name_tags": classify_model(model_id),
            })

    print(f"  Parsed {len(models)} models with valid IDs")

    # === Analysis 1: Distribution of naming patterns ===
    pattern_counts = defaultdict(int)
    pattern_downloads = defaultdict(list)
    pattern_likes = defaultdict(list)

    for m in models:
        for tag in m["name_tags"]:
            pattern_counts[tag] += 1
            pattern_downloads[tag].append(m["downloads"])
            pattern_likes[tag].append(m["likes"])

    # === Analysis 2: Statistical comparison of each pattern vs "unmarked" ===
    comparisons = {}
    unmarked_downloads = pattern_downloads.get("unmarked", [])
    unmarked_likes = pattern_likes.get("unmarked", [])

    for pattern in NAMING_PATTERNS:
        if pattern_counts[pattern] < 5:
            continue
        dl = pattern_downloads[pattern]
        lk = pattern_likes[pattern]

        t_dl, p_dl = welch_t_test(dl, unmarked_downloads)
        t_lk, p_lk = welch_t_test(lk, unmarked_likes)
        d_dl = effect_size_cohens_d(dl, unmarked_downloads)
        d_lk = effect_size_cohens_d(lk, unmarked_likes)

        comparisons[pattern] = {
            "count": pattern_counts[pattern],
            "downloads_mean": round(safe_mean(dl), 1),
            "downloads_median": round(safe_median(dl), 1),
            "likes_mean": round(safe_mean(lk), 2),
            "likes_median": round(safe_median(lk), 2),
            "vs_unmarked_downloads_t": round(t_dl, 3),
            "vs_unmarked_downloads_p": round(p_dl, 6),
            "vs_unmarked_downloads_cohens_d": round(d_dl, 3),
            "vs_unmarked_likes_t": round(t_lk, 3),
            "vs_unmarked_likes_p": round(p_lk, 6),
            "vs_unmarked_likes_cohens_d": round(d_lk, 3),
        }

    # Apply Bonferroni correction for multiple comparisons
    n_comparisons = len(comparisons)
    for pattern in comparisons:
        p_dl = comparisons[pattern]["vs_unmarked_downloads_p"]
        p_lk = comparisons[pattern]["vs_unmarked_likes_p"]
        comparisons[pattern]["vs_unmarked_downloads_p_bonferroni"] = round(
            min(p_dl * n_comparisons, 1.0), 6)
        comparisons[pattern]["vs_unmarked_likes_p_bonferroni"] = round(
            min(p_lk * n_comparisons, 1.0), 6)
        comparisons[pattern]["significant_downloads_bonferroni"] = (
            comparisons[pattern]["vs_unmarked_downloads_p_bonferroni"] < 0.05)
        comparisons[pattern]["significant_likes_bonferroni"] = (
            comparisons[pattern]["vs_unmarked_likes_p_bonferroni"] < 0.05)
    
    # === Analysis 3: Multi-tag models ===
    multi_tag_count = sum(1 for m in models if len(m["name_tags"]) > 1 and "unmarked" not in m["name_tags"])
    multi_tag_combos = defaultdict(int)
    for m in models:
        real_tags = [t for t in m["name_tags"] if t != "unmarked"]
        if len(real_tags) > 1:
            combo = "+".join(sorted(real_tags))
            multi_tag_combos[combo] += 1

    top_combos = sorted(multi_tag_combos.items(), key=lambda x: -x[1])[:10]

    # === Analysis 4: Pipeline tag breakdown per naming pattern ===
    pattern_pipeline = defaultdict(lambda: defaultdict(int))
    for m in models:
        for tag in m["name_tags"]:
            pattern_pipeline[tag][m["pipeline_tag"]] += 1

    pipeline_summary = {}
    for pattern, pipelines in pattern_pipeline.items():
        top_pl = sorted(pipelines.items(), key=lambda x: -x[1])[:5]
        pipeline_summary[pattern] = {k: v for k, v in top_pl}

    # === Analysis 5: Downloads distribution shape per pattern ===
    distribution_stats = {}
    for pattern in list(NAMING_PATTERNS.keys()) + ["unmarked"]:
        if pattern_counts[pattern] < 5:
            continue
        dl = pattern_downloads[pattern]
        lk = pattern_likes[pattern]
        distribution_stats[pattern] = {
            "count": pattern_counts[pattern],
            "downloads_p25": round(percentile(dl, 25), 1),
            "downloads_p50": round(percentile(dl, 50), 1),
            "downloads_p75": round(percentile(dl, 75), 1),
            "downloads_p95": round(percentile(dl, 95), 1),
            "likes_p25": round(percentile(lk, 25), 2),
            "likes_p50": round(percentile(lk, 50), 2),
            "likes_p75": round(percentile(lk, 75), 2),
            "likes_p95": round(percentile(lk, 95), 2),
        }

    # === Analysis 6: Proportion of text-generation models per pattern ===
    textgen_rates = {}
    for pattern in list(NAMING_PATTERNS.keys()) + ["unmarked"]:
        if pattern_counts[pattern] < 5:
            continue
        pipelines = pattern_pipeline.get(pattern, {})
        total = sum(pipelines.values())
        tg = pipelines.get("text-generation", 0)
        textgen_rates[pattern] = round(tg / total * 100, 1) if total > 0 else 0

    # === Build results ===
    results = {
        "metadata": {
            "data_source": "https://huggingface.co/api/models",
            "query_params": "sort=downloads&direction=-1&pipeline_tag=text-generation (primary) + general",
            "total_models_fetched": len(models),
            "textgen_pages": NUM_PAGES_TEXTGEN,
            "general_pages": NUM_PAGES_GENERAL,
            "models_per_page": MODELS_PER_PAGE,
            "analysis_timestamp": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
            "random_seed": RANDOM_SEED,
            "naming_patterns_searched": list(NAMING_PATTERNS.keys()),
            "pinned": "Data is pinned via local SHA256-verified cache after first fetch. Re-fetching produces a new snapshot since the API serves live data.",
            "bonferroni_correction": True,
            "n_comparisons": len(comparisons) if comparisons else 0,
        },
        "pattern_counts": dict(pattern_counts),
        "pattern_comparisons_vs_unmarked": comparisons,
        "distribution_stats": distribution_stats,
        "multi_tag_models": {
            "count": multi_tag_count,
            "top_combinations": dict(top_combos),
        },
        "pipeline_breakdown": pipeline_summary,
        "textgen_rates_pct": textgen_rates,
        "caveats_and_limitations": [
            "Limitation: Downloads and likes are proxy metrics, not direct performance measures on instruction-following benchmarks",
            "Limitation: Sample is biased toward high-download models (top 2000 by downloads)",
            "Caveat: Model naming is not standardized; regex patterns may miss variant spellings",
            "Caveat: A model can match multiple naming patterns simultaneously",
            "Limitation: Temporal effects not controlled — newer naming conventions may have fewer cumulative downloads",
            "Limitation: Organizational effects not controlled — large orgs may dominate certain patterns",
            "Caveat: Bonferroni correction is conservative; some true effects may be missed",
        ],
    }

    return results, models


def generate_report(results):
    """Generate a human-readable markdown report."""
    lines = []
    lines.append("# HuggingFace Model Naming Patterns — Analysis Report\n")
    lines.append(f"**Generated:** {results['metadata']['analysis_timestamp']}")
    lines.append(f"**Models analyzed:** {results['metadata']['total_models_fetched']}")
    lines.append(f"**Data source:** {results['metadata']['data_source']}\n")

    # Pattern distribution
    lines.append("## 1. Naming Pattern Distribution\n")
    lines.append("| Pattern | Count | % of Models |")
    lines.append("|---------|------:|------------:|")
    total = results["metadata"]["total_models_fetched"]
    for pattern, count in sorted(results["pattern_counts"].items(), key=lambda x: -x[1]):
        pct = count / total * 100
        lines.append(f"| {pattern} | {count} | {pct:.1f}% |")

    # Comparisons
    lines.append("\n## 2. Pattern vs. Unmarked Models (Welch's t-test)\n")
    lines.append("| Pattern | N | Mean DL | Median DL | Mean Likes | t(DL) | p(DL) | p(DL) Bonf. | d(DL) | Sig? |")
    lines.append("|---------|--:|--------:|----------:|-----------:|------:|------:|------------:|------:|-----:|")
    for pattern, comp in sorted(results["pattern_comparisons_vs_unmarked"].items(), key=lambda x: -x[1]["count"]):
        sig = "Yes" if comp.get("significant_downloads_bonferroni", False) else "No"
        p_bonf = comp.get("vs_unmarked_downloads_p_bonferroni", comp["vs_unmarked_downloads_p"])
        lines.append(
            f"| {pattern} | {comp['count']} | {comp['downloads_mean']:,.0f} | {comp['downloads_median']:,.0f} "
            f"| {comp['likes_mean']:,.1f} | {comp['vs_unmarked_downloads_t']:.2f} "
            f"| {comp['vs_unmarked_downloads_p']:.4f} | {p_bonf:.4f} "
            f"| {comp['vs_unmarked_downloads_cohens_d']:.3f} | {sig} |"
        )

    # Distribution
    lines.append("\n## 3. Download Distribution by Pattern\n")
    lines.append("| Pattern | N | P25 | P50 | P75 | P95 |")
    lines.append("|---------|--:|----:|----:|----:|----:|")
    for pattern, stats in sorted(results["distribution_stats"].items(), key=lambda x: -x[1]["count"]):
        lines.append(
            f"| {pattern} | {stats['count']} | {stats['downloads_p25']:,.0f} "
            f"| {stats['downloads_p50']:,.0f} | {stats['downloads_p75']:,.0f} "
            f"| {stats['downloads_p95']:,.0f} |"
        )

    # Multi-tag
    lines.append("\n## 4. Multi-Tag Models\n")
    lines.append(f"**Models with multiple naming tags:** {results['multi_tag_models']['count']}\n")
    if results["multi_tag_models"]["top_combinations"]:
        lines.append("| Combination | Count |")
        lines.append("|-------------|------:|")
        for combo, count in results["multi_tag_models"]["top_combinations"].items():
            lines.append(f"| {combo} | {count} |")

    # Text-generation rates
    lines.append("\n## 5. Text-Generation Rate by Pattern\n")
    lines.append("| Pattern | % Text-Gen |")
    lines.append("|---------|----------:|")
    for pattern, rate in sorted(results["textgen_rates_pct"].items(), key=lambda x: -x[1]):
        lines.append(f"| {pattern} | {rate:.1f}% |")

    # Caveats & Limitations
    lines.append("\n## 6. Caveats and Limitations\n")
    for c in results.get("caveats_and_limitations", results.get("caveats", [])):
        lines.append(f"- {c}")

    return "\n".join(lines) + "\n"


def verify(results):
    """Run verification checks on results. Returns (passed, total, messages)."""
    checks = []

    # Check 1: We have a reasonable number of models
    n = results["metadata"]["total_models_fetched"]
    checks.append(("model_count_reasonable", n >= 500, f"Fetched {n} models (need >= 500)"))

    # Check 2: pattern_counts is non-empty and sums correctly
    pc = results["pattern_counts"]
    checks.append(("pattern_counts_nonempty", len(pc) >= 3, f"Found {len(pc)} patterns (need >= 3)"))

    # Check 3: unmarked exists
    checks.append(("unmarked_exists", "unmarked" in pc, "Unmarked category exists"))

    # Check 4: At least 3 patterns have statistical comparisons
    nc = len(results["pattern_comparisons_vs_unmarked"])
    checks.append(("comparisons_exist", nc >= 3, f"Have {nc} pattern comparisons (need >= 3)"))

    # Check 5: results.json is valid and has required top-level keys
    required_keys = ["metadata", "pattern_counts", "pattern_comparisons_vs_unmarked",
                     "distribution_stats", "caveats_and_limitations"]
    has_keys = all(k in results for k in required_keys)
    checks.append(("required_keys", has_keys, f"All required keys present: {has_keys}"))

    # Check 6: Caveats section has at least 4 entries
    nc = len(results.get("caveats_and_limitations", results.get("caveats", [])))
    checks.append(("caveats_sufficient", nc >= 4, f"Have {nc} caveats (need >= 4)"))

    # Check 7: Distribution stats exist for at least 3 patterns
    nd = len(results.get("distribution_stats", {}))
    checks.append(("distribution_stats", nd >= 3, f"Have {nd} distribution stat entries (need >= 3)"))

    # Check 8: Text-generation rates computed
    ntg = len(results.get("textgen_rates_pct", {}))
    checks.append(("textgen_rates", ntg >= 2, f"Have {ntg} textgen rate entries (need >= 2)"))

    # Check 9: Multi-tag analysis present
    mt = results.get("multi_tag_models", {})
    checks.append(("multi_tag", "count" in mt, "Multi-tag analysis present"))

    # Check 10: All p-values are in [0, 1]
    valid_p = True
    for comp in results["pattern_comparisons_vs_unmarked"].values():
        if not (0 <= comp["vs_unmarked_downloads_p"] <= 1):
            valid_p = False
        if not (0 <= comp["vs_unmarked_likes_p"] <= 1):
            valid_p = False
    checks.append(("p_values_valid", valid_p, "All p-values in [0, 1]"))

    return checks


def main():
    verify_mode = "--verify" in sys.argv
    total_steps = 6

    # Step 1: Fetch data
    print(f"[1/{total_steps}] Fetching model data from HuggingFace API...")
    models_raw = fetch_models()
    print(f"  Fetched {len(models_raw)} raw model records")

    # Step 2: Analyze
    print(f"\n[2/{total_steps}] Classifying models by naming patterns...")
    results, models = analyze(models_raw)

    # Step 3: Summary stats
    print(f"\n[3/{total_steps}] Summary statistics:")
    for pattern, count in sorted(results["pattern_counts"].items(), key=lambda x: -x[1]):
        print(f"  {pattern}: {count} models")

    # Step 4: Write results.json
    print(f"\n[4/{total_steps}] Writing results.json...")
    with open(RESULTS_PATH, "w") as f:
        json.dump(results, f, indent=2)
    print(f"  Saved to {RESULTS_PATH}")

    # Step 5: Write report.md
    print(f"\n[5/{total_steps}] Writing report.md...")
    report = generate_report(results)
    with open(REPORT_PATH, "w") as f:
        f.write(report)
    print(f"  Saved to {REPORT_PATH}")

    # Step 6: Verify
    print(f"\n[6/{total_steps}] Running verification checks...")
    checks = verify(results)
    passed = sum(1 for _, ok, _ in checks if ok)
    total = len(checks)

    for name, ok, msg in checks:
        status = "PASS" if ok else "FAIL"
        print(f"  [{status}] {name}: {msg}")

    print(f"\n  Verification: {passed}/{total} checks passed")

    if passed == total:
        print("\nALL CHECKS PASSED")
    else:
        print(f"\nWARNING: {total - passed} checks failed")
        if verify_mode:
            sys.exit(1)

    print("\nANALYSIS COMPLETE")


if __name__ == "__main__":
    main()
SCRIPT_EOF
```

**Expected output:** File `analyze.py` written, exit code 0.

## Step 3: Run Analysis

```bash
cd /tmp/claw4s_auto_huggingface-model-naming-patterns && python3 analyze.py
```

**Expected output:**
- Sectioned progress `[1/6]` through `[6/6]`
- Summary statistics for each naming pattern
- Verification checks with `[PASS]` or `[FAIL]`
- Final line: `ANALYSIS COMPLETE`
- Files created: `results.json`, `report.md` in the workspace

## Step 4: Verify Results

```bash
cd /tmp/claw4s_auto_huggingface-model-naming-patterns && python3 analyze.py --verify
```

**Expected output:**
- Same analysis output as Step 3 (uses cached data)
- All verification checks pass: `ALL CHECKS PASSED`
- Exit code 0

**Failure condition:** If any check shows `[FAIL]`, the verify step returns exit code 1.

## Success Criteria

1. `results.json` exists and contains valid JSON with 5+ top-level keys
2. `report.md` exists and contains markdown tables with statistical results
3. All 10 verification checks pass
4. `ANALYSIS COMPLETE` appears in stdout
5. `ALL CHECKS PASSED` appears in stdout
6. No pip install or external dependencies required

## Failure Conditions

1. Network error fetching from HuggingFace API (retry logic should handle transient failures)
2. API response format change (would cause KeyError — check error message)
3. Any verification check fails in `--verify` mode (exit code 1)