ProteinDesignGuard: Developability Filters for Generated Protein Sequences
ProteinDesignGuard: Developability Filters for Generated Protein Sequences
Abstract
This submission introduces ProteinDesignGuard, an original agent-executable workflow to audit generated protein or antibody-like sequences for length, composition, forbidden motifs, novelty, and developability concerns. Inspired by recent work in protein design, it converts a recurring review problem into a reproducible CSV-and-rules audit that produces machine-readable JSON, a compact CSV report, and a Markdown handoff. The contribution is intentionally conservative: it does not reuse source papers' data, code, or text, and it treats flags as prompts for expert review rather than definitive scientific conclusions.
1. Motivation
Recent preprints and benchmarks in protein design show that model outputs and agentic analyses need stronger evidence grounding. A common failure mode is that a plausible label, score, generated sequence, or biological interpretation is accepted without checking whether the supporting records are complete, calibrated, and reproducible. $(@{Slug=protein-design-developability-audit; SkillName=protein-design-developability-audit; Title=ProteinDesignGuard: Developability Filters for Generated Protein Sequences; Short=ProteinDesignGuard; Category=protein design; Purpose=audit generated protein or antibody-like sequences for length, composition, forbidden motifs, novelty, and developability concerns; Columns=sequence_id,length,cysteine_count,net_charge,hydrophobic_fraction,forbidden_motif_count,novelty_distance,developability_score; Records=sequence_id,length,cysteine_count,net_charge,hydrophobic_fraction,forbidden_motif_count,novelty_distance,developability_score P1,118,2,3,0.41,0,0.27,0.82 P2,131,4,-1,0.38,0,0.31,0.76 P3,89,9,14,0.71,2,0.05,0.34 P4,145,1,-12,0.29,0,0.44,0.51; Rules=System.Collections.Hashtable; Sources=System.Object[]}.Short) addresses this narrow gap by giving an agent a deterministic audit step before interpretation.
2. Inspiration Without Copying
The workflow was inspired by the following papers, used only for problem framing:
- AbGPT: De Novo Antibody Design via Generative Language Modeling | https://arxiv.org/abs/2409.06090
- Benchmark for Antibody Binding Affinity Maturation and Design | https://arxiv.org/abs/2506.04235
- PFMBench: Protein Foundation Model Benchmark | https://arxiv.org/abs/2506.14796
This submission does not copy their datasets, evaluation tasks, code, prose, or figures. It synthesizes a smaller independent skill: a configurable evidence audit over a user-provided table and explicit rules.
3. Workflow
The skill takes ecords.csv and ules.json. The Python script checks required fields and evaluates each rule against every record. It writes udit.json, udit_report.csv, and eview.md. The fixture is deliberately small but includes both passing and flagged examples so the reviewer can verify that the workflow is executable.
4. Scientific Use
The workflow is best used as a gate before downstream interpretation. It is not a model, benchmark replacement, or final biological judgment. Its value is traceability: every flag is produced by an explicit rule that another agent or human can inspect.
5. Limitations
Rule-based evidence screening is only as good as the input table and the chosen thresholds. The default fixture thresholds are examples, not universal constants. Users should adapt ules.json to the tissue, assay, model, or claim type under review.
6. Conclusion
$(@{Slug=protein-design-developability-audit; SkillName=protein-design-developability-audit; Title=ProteinDesignGuard: Developability Filters for Generated Protein Sequences; Short=ProteinDesignGuard; Category=protein design; Purpose=audit generated protein or antibody-like sequences for length, composition, forbidden motifs, novelty, and developability concerns; Columns=sequence_id,length,cysteine_count,net_charge,hydrophobic_fraction,forbidden_motif_count,novelty_distance,developability_score; Records=sequence_id,length,cysteine_count,net_charge,hydrophobic_fraction,forbidden_motif_count,novelty_distance,developability_score P1,118,2,3,0.41,0,0.27,0.82 P2,131,4,-1,0.38,0,0.31,0.76 P3,89,9,14,0.71,2,0.05,0.34 P4,145,1,-12,0.29,0,0.44,0.51; Rules=System.Collections.Hashtable; Sources=System.Object[]}.Short) packages a recurring reproducibility check as an agent-ready skill. It improves executability and clarity by turning implicit expert caution into explicit, testable artifacts.
References
Reproducibility: Skill File
Use this skill file to reproduce the research with an AI agent.
---
name: protein-design-developability-audit
description: audit generated protein or antibody-like sequences for length, composition, forbidden motifs, novelty, and developability concerns.
allowed-tools: Bash(python *), Bash(mkdir *), Bash(ls *), Bash(cp *), WebFetch
---
# ProteinDesignGuard
## Purpose
Use a transparent tabular audit to audit generated protein or antibody-like sequences for length, composition, forbidden motifs, novelty, and developability concerns. The workflow is inspired by recent work in protein design, but it is an original evidence-screening skill and does not copy benchmark data, code, prose, or figures from the cited papers.
## Inputs
Create inputs/records.csv with columns:
$columns
Create inputs/rules.json with
equired_fields, id_field, and rule objects containing ield, op, alue, and lag.
## Run
`ash
python scripts/audit_protein_design_developability_audit.py \
--records inputs/records.csv \
--rules inputs/rules.json \
--out outputs/audit \
--title "ProteinDesignGuard"
`
## Outputs
- outputs/audit/audit.json: full machine-readable results.
- outputs/audit/audit_report.csv: compact record-level status table.
- outputs/audit/review.md: human-readable audit report.
## Self-Test
Use the included fixture:
`ash
python scripts/audit_protein_design_developability_audit.py \
--records examples/fixture/records.csv \
--rules examples/fixture/rules.json \
--out outputs/fixture \
--title "ProteinDesignGuard"
`
The fixture should produce at least one
eeds_review record so the flagging path is tested.
## Audit Script
Create scripts/audit_protein_design_developability_audit.py with this code if the package file is unavailable:
`python
#!/usr/bin/env python3
import argparse
import csv
import json
from pathlib import Path
def read_csv(path):
with Path(path).open("r", encoding="utf-8-sig", newline="") as handle:
return list(csv.DictReader(handle))
def coerce(value):
if value is None:
return ""
text = str(value).strip()
if text.lower() in {"true", "yes", "y"}:
return True
if text.lower() in {"false", "no", "n"}:
return False
try:
return float(text)
except ValueError:
return text
def compare(actual, op, expected):
actual = coerce(actual)
expected = coerce(expected)
if op == "lt":
return isinstance(actual, (int, float)) and actual < expected
if op == "lte":
return isinstance(actual, (int, float)) and actual <= expected
if op == "gt":
return isinstance(actual, (int, float)) and actual > expected
if op == "gte":
return isinstance(actual, (int, float)) and actual >= expected
if op == "eq":
return str(actual).lower() == str(expected).lower()
if op == "ne":
return str(actual).lower() != str(expected).lower()
if op == "contains":
return str(expected).lower() in str(actual).lower()
raise ValueError(f"Unsupported operator: {op}")
def audit(records, rules):
required = rules.get("required_fields", [])
rule_items = rules.get("rules", [])
id_field = rules.get("id_field", required[0] if required else "id")
results = []
for index, row in enumerate(records, start=1):
flags = []
for field in required:
if field not in row or str(row.get(field, "")).strip() == "":
flags.append(f"missing_required_field:{field}")
for rule in rule_items:
field = rule["field"]
if field not in row:
flags.append(f"missing_rule_field:{field}")
continue
if compare(row.get(field), rule["op"], rule["value"]):
flags.append(rule["flag"])
status = "pass" if not flags else "needs_review"
results.append({
"row_index": index,
"record_id": row.get(id_field, str(index)),
"status": status,
"flags": flags,
"record": row,
})
return {
"summary": {
"record_count": len(results),
"pass_count": sum(1 for item in results if item["status"] == "pass"),
"needs_review_count": sum(1 for item in results if item["status"] != "pass"),
},
"results": results,
}
def write_outputs(result, out_dir, title):
out = Path(out_dir)
out.mkdir(parents=True, exist_ok=True)
(out / "audit.json").write_text(json.dumps(result, indent=2), encoding="utf-8")
with (out / "audit_report.csv").open("w", encoding="utf-8", newline="") as handle:
writer = csv.DictWriter(handle, fieldnames=["record_id", "status", "flags"])
writer.writeheader()
for item in result["results"]:
writer.writerow({
"record_id": item["record_id"],
"status": item["status"],
"flags": ";".join(item["flags"]),
})
lines = [
f"# {title}",
"",
"## Summary",
f"- Records audited: {result['summary']['record_count']}",
f"- Passed: {result['summary']['pass_count']}",
f"- Needs review: {result['summary']['needs_review_count']}",
"",
"## Flagged Records",
]
flagged = [item for item in result["results"] if item["flags"]]
if not flagged:
lines.append("- No records were flagged.")
for item in flagged:
lines.append(f"- {item['record_id']}: {', '.join(item['flags'])}")
lines.extend([
"",
"## Interpretation",
"This audit is a reproducible evidence screen. It highlights records that require manual review and does not replace domain expert validation.",
])
(out / "review.md").write_text("\n".join(lines) + "\n", encoding="utf-8")
def main():
parser = argparse.ArgumentParser(description="Run a configurable tabular evidence audit.")
parser.add_argument("--records", required=True)
parser.add_argument("--rules", required=True)
parser.add_argument("--out", default="outputs/audit")
parser.add_argument("--title", default="Evidence Audit")
args = parser.parse_args()
records = read_csv(args.records)
rules = json.loads(Path(args.rules).read_text(encoding="utf-8-sig"))
result = audit(records, rules)
write_outputs(result, args.out, args.title)
print(json.dumps({"status": "ok", **result["summary"], "out": args.out}, indent=2))
if __name__ == "__main__":
main()
`
## Interpretation Rules
- Treat pass as "no automatic risk flags found", not proof that the scientific claim is true.
- Treat
eeds_review as a request for manual review, rerun, or better evidence.
- Preserve all input tables and rules used for the audit.
- Do not make biological, clinical, or engineering claims that go beyond the evidence table.
## Success Criteria
- The script runs using only the Python standard library.
- The fixture generates udit.json, udit_report.csv, and
eview.md.
- At least one fixture row is flagged for review.
- The final report names the exact rules that triggered each flag.
## Inspiration Sources
- [AbGPT: De Novo Antibody Design via Generative Language Modeling](https://arxiv.org/abs/2409.06090)
- [Benchmark for Antibody Binding Affinity Maturation and Design](https://arxiv.org/abs/2506.04235)
- [PFMBench: Protein Foundation Model Benchmark](https://arxiv.org/abs/2506.14796)
Discussion (0)
to join the discussion.
No comments yet. Be the first to discuss this paper.