Re-examine 200 published TWFE DiD studies with staggered treatment adoption from 15 economics journals (2010-2023). Apply Callaway-Sant'Anna (CS) and Sun-Abraham (SA) estimators alongside original TWFE.
Simulation study: generate RCT data with known CATE functions (linear, nonlinear, interaction) at N=200-20000. Apply 4 HTE estimation methods: causal forests, X-learner, R-learner, Bayesian CART.
Re-analyze 100 published synthetic control studies from top economics journals. For each, systematically vary the donor pool: remove 1, 2, or 5 donors (all combinations up to 1000 draws).
Synthetic logs are proposed as a privacy-preserving substitute for production data in anomaly detection research, but claims in the literature are rarely grounded in controlled comparisons between generation methods. We implement four methods—Random (no constraints), Template-based (format-string substitution), Constrained (rule-based causal graph generator), and LLM-based (Claude Haiku prompted with explicit causal specifications)—and evaluate 200 sequences per method (800 total, 5,337 entries) against three pre-defined fidelity criteria: temporal coherence, timing plausibility, and message specificity.
Production logs are inaccessible for ML training due to privacy constraints, yet anomaly detection research requires realistic data. We test whether constrained generation can produce synthetic logs preserving temporal causality in distributed payment system failure cascades.
Tool-using AI agents are increasingly evaluated on benchmarks that measure end-to-end task completion rates. However, high benchmark scores may reflect memorization of tool-calling patterns seen during training rather than genuine compositional reasoning about tool capabilities.