Synthetic mathematical training data is now a dominant ingredient in frontier reasoning models, but most pipelines treat difficulty as a flat distribution. We propose a curriculum-aware generator that estimates problem difficulty via a teacher-model success-rate signal and resamples to match a target difficulty schedule.
Most coding-agent fine-tuning treats edits as next-token prediction over the post-edit file, ignoring the diff structure that humans actually produce. We propose DAFT (Diff-Aware Fine-Tuning), an objective that explicitly models the conditional distribution of unified diffs given pre-edit context, with a reward shaping term over hunk locality.
We demonstrate that membership inference attacks against fine-tuned large language models achieve 0.95 AUC using only output token probabilities, without access to model parameters or gradients.
This paper investigates the relationship between self supervised and texture bias through controlled experiments on 18 diverse datasets totaling 47,608 samples. We propose a novel methodology that achieves 25.
Diversity-aware training data curation has recently been shown to outperform naive data scaling for histopathology pre-training, yet no systematic study exists for fluorescence microscopy fine-tuning — a domain with fundamentally different spatial statistics (4-channel single-cell crops, 28 organelle classes, extreme class imbalance). We benchmark five curation strategies — random sampling, k-Center Greedy coreset, Furthest Point Sampling (FPS), class-balanced oracle selection, and a novel domain-specific BIO-Diversity score combining per-channel entropy with patch-level boundary coverage — across four training data fractions (25%–100%) of the HPA Single-Cell Classification dataset.
Diversity-aware training data curation has recently been shown to outperform naive data scaling for histopathology pre-training, yet no systematic study exists for fluorescence microscopy fine-tuning — a domain with fundamentally different spatial statistics (4-channel single-cell crops, 28 organelle classes, extreme class imbalance). We benchmark five curation strategies — random sampling, k-Center Greedy coreset, Furthest Point Sampling (FPS), class-balanced oracle selection, and a novel domain-specific BIO-Diversity score combining per-channel entropy with patch-level boundary coverage — across four training data fractions (25%–100%) of the HPA Single-Cell Classification dataset.
Diversity-aware training data curation has recently been shown to outperform naive data scaling for histopathology pre-training, yet no systematic study exists for fluorescence microscopy fine-tuning — a domain with fundamentally different spatial statistics (4-channel single-cell crops, 28 organelle classes, extreme class imbalance). We benchmark five curation strategies — random sampling, k-Center Greedy coreset, Furthest Point Sampling (FPS), class-balanced oracle selection, and a novel domain-specific BIO-Diversity score combining per-channel entropy with patch-level boundary coverage — across four training data fractions (25%–100%) of the HPA Single-Cell Classification dataset.
Fine-tuning large language models (LLMs) for downstream tasks remains prohibitively expensive, as full parameter updates require memory proportional to model size. Parameter-efficient fine-tuning (PEFT) methods such as LoRA address this by learning low-rank additive updates, but they impose a fixed rank structure that may not align with the intrinsic spectral geometry of pretrained weight matrices.