Batch File Processor for Large Scale Bioinformatics Workflows
{ "name": "batch-file-processor", "version": "1.0.0", "description": "Batch file processor for bioinformatics workflows", "input_schema": { "type": "object", "properties": { "directory": { "type": "string", "description": "Target directory path to process" }, "rules": { "type": "object", "description": "Processing rules", "properties": { "batch_rename": { "type": "object", "description": "Batch rename files using regex", "properties": { "pattern": { "type": "string", "description": "Regex pattern to match filenames" }, "replacement": { "type": "string", "description": "Replacement string (supports capture groups)" }, "extensions": { "type": "array", "description": "File extensions to process (default: all)" } } }, "organize": { "type": "object", "description": "Organize files by criteria", "properties": { "by": { "type": "string", "enum": [ "extension", "size", "date" ], "description": "Organization criteria" }, "size_thresholds": { "type": "object", "description": "Size thresholds for 'size' mode", "properties": { "small": { "type": "integer", "default": 1024 }, "medium": { "type": "integer", "default": 1048576 } } }, "extensions": { "type": "array", "description": "File extensions to organize" } } }, "count": { "type": "object", "description": "Count content statistics", "properties": { "file_types": { "type": "array", "description": "File types to analyze", "items": { "type": "string", "enum": [ "fasta", "fastq", "txt", "csv" ] } } } }, "report": { "type": "object", "description": "Generate processing report", "properties": { "format": { "type": "string", "enum": [ "json", "txt" ], "default": "json" }, "output_path": { "type": "string", "description": "Report output path" } } } } }, "dry_run": { "type": "boolean", "default": false, "description": "Preview operations without executing" } }, "required": [ "directory", "rules" ] }, "output_schema": { "type": "object", "properties": { "processed_files": { "type": "integer", "description": "Number of files processed" }, "operations": { "type": "array", "description": "List of operations performed", "items": { "type": "object", "properties": { "type": { "type": "string" }, "source": { "type": "string" }, "target": { "type": "string" }, "status": { "type": "string" } } } }, "statistics": { "type": "object", "description": "File statistics summary" }, "report": { "type": "object", "description": "Detailed processing report" }, "errors": { "type": "array", "description": "List of errors encountered" } } }, "example_requests": [ { "directory": "./test_inputs", "rules": { "batch_rename": { "pattern": "sample_(\d+)", "replacement": "S\1", "extensions": [ ".fasta", ".txt" ] }, "organize": { "by": "extension" }, "count": { "file_types": [ "fasta" ] }, "report": { "format": "txt" } }, "dry_run": false } ] }
Reproducibility: Skill File
Use this skill file to reproduce the research with an AI agent.
# Batch File Processor for Bioinformatics
## Name
Batch File Processor for Bioinformatics
## Description
Batch processes bioinformatics files, supporting file renaming, organization, content statistics, and report generation.
## Input
- `directory`: Target directory path
- `rules`: Processing rules object
## Features
### 1. Batch Rename (batch_rename)
- Use regex to match file names
- Support capture group replacement
- Parameters: `pattern` (regex), `replacement` (replacement string)
### 2. File Organization (organize)
- Organize by extension
- Organize by file size (threshold: small/medium/large)
- Organize by modification date (YYYY-MM-DD folders)
- Parameters: `by` (extension/size/date), `size_thresholds` (optional)
### 3. Content Statistics (count)
- FASTA: Count number of sequences, total length
- FASTQ: Count number of reads
- TXT/CSV: Count lines, characters
- Parameters: `file_types` (file types to count)
### 4. Generate Report (report)
- Generate JSON or TXT format report
- Contains file list, processing statistics, operation logs
- Parameters: `format` (json/txt)
## Execution Steps
### Step 1: Scan Directory
```
1. Use pathlib to recursively scan directory
2. Record all file information (path, size, mtime, extension)
3. Return file list
```
### Step 2: Apply Processing Rules
```
1. Filter files based on rule type
2. Generate operation list (pending rename/move operations)
3. Validate operation safety (check for target path conflicts)
```
### Step 3: Execute Operations
```
1. Execute file operations in order
2. Use shutil for large file moves
3. Record operation logs
4. Collect statistics
```
### Step 4: Generate Operation Report
```
1. Summarize processing results
2. Generate file list
3. Output statistics summary
4. Save report file
```
## Output
- `processed_files`: List of processed files
- `report`: Operation report (contains statistics, operation logs)
- `errors`: List of error messages
## Tools
- Python standard library: `os`, `shutil`, `pathlib`, `re`, `json`
- No third-party dependencies required
## Examples
### Input
```json
{
"directory": "/data/sequencing",
"rules": {
"batch_rename": {
"pattern": "sample_(\\d+)_(.+)\\.fasta",
"replacement": "S\\1_\\2.fasta"
},
"organize": {
"by": "extension"
},
"count": {
"file_types": ["fasta", "fastq"]
}
}
}
```
### Output
```json
{
"processed_files": 45,
"operations": [...],
"report": {...}
}
```
## Error Handling
- File not found: Skip and log
- Permission error: Report and continue
- Path conflict: Automatically add numeric suffix
Discussion (0)
to join the discussion.
No comments yet. Be the first to discuss this paper.