Cross-Domain Gap Scanning: A Systematic Method for AI-Driven Research Direction Discovery
Cross-Domain Gap Scanning: A Systematic Method for AI-Driven Research Direction Discovery
When AI can execute any research question in hours, the bottleneck shifts from "how to do it" to "what to do." This paper addresses the latter.
1. The Problem: Execution is Solved, Discovery is Not
The current wave of AI research tools — including our own system — can take a well-defined research question and produce a submission-ready manuscript in hours. Data downloading, statistical analysis, figure generation, and manuscript drafting are increasingly automated.
But every one of these systems requires a human to say: "Study the relationship between X and Y using dataset Z."
This is the wrong bottleneck. The research question is the most consequential decision in the entire pipeline. A mediocre question executed perfectly produces a mediocre paper. A brilliant question executed adequately can change a field.
We asked: can an AI system systematically discover research questions that are simultaneously novel, feasible, and impactful?
2. Core Insight: Bridges Between Mature Fields
Traditional literature review finds gaps within a field — incremental improvements, missing subgroups, unexplored moderators. These gaps are real but crowded. Every PhD student in the field is looking at the same gaps.
Our approach is different:
Traditional: Search within Field A for gaps → Incremental research (crowded)
Ours: Map Field A × Field B → Find where both sides are mature
but nobody has built a bridge → Blue oceanThe metaphor: imagine two well-studied cities (e.g., "environmental toxicology" and "psychiatric epidemiology") connected by a river that everyone assumes someone else has bridged. Cross-Domain Gap Scanning systematically checks every river crossing and identifies the ones where no bridge exists.
3. The Six-Phase Method
Phase 1: Asset Inventory
Goal: Understand what we already have — completed studies, available datasets, established variable relationships. This phase adapts to input type:
| Input Type | Action |
|---|---|
| Project directory | Scan outputs, extract exposure/outcome/mediator variables |
| Data file (CSV/XPT) | Profile columns, match to known datasets (e.g., SDMVPSU → NHANES) |
| Paper/DOI | Extract variable relationships, map citation network |
| Domain description | No local assets; Phase 3 switches to broad-scan mode |
| Dataset name | Load known variable modules (e.g., NHANES: 14 modules per cycle) |
| Nothing | Load all available public dataset profiles |
Output: asset_map.md — a variable relationship graph showing which connections already exist and which nodes are isolated.
Phase 2: Structure Selection + Preference Injection
Three structural templates for research programs:
| Structure | Logic | Robustness | Impact | End Product |
|---|---|---|---|---|
| Exposure-driven | One X → many Y | High (shared data) | Medium | Umbrella review |
| Outcome-driven | Many X → one Y | High (independent papers) | Medium | Systematic review + PAF |
| Mechanism chain | A→B→C→D serial | Low (chain fragile) | High | Theoretical contribution |
| Hybrid (recommended) | Main axis + embedded mechanisms | Medium-high | High | Review + original framework |
Critical checkpoint: User preference injection is mandatory at this stage. The system must know:
- Novelty preference: "blank space" vs. "safe incremental"
- Domain constraints: include/exclude specific fields
- Data constraints: restrict to specific datasets
- Practical constraints: clinically translatable vs. pure academic
These preferences directly modify Phase 3's search strategy and scoring weights.
Phase 3: Cross-Domain Gap Scanning (Deep Research-Driven)
This is the core engine. Not brainstorming — systematic evidence-based scanning.
Step 3a: Generate candidate intersections
For every variable pair (A, C) from different domains:
- If A→B has literature (Field 1 is mature)
- And B→C has literature (Field 2 is mature)
- Then "A→B→C serial pathway" is a candidate gap
Step 3b: Deep Research saturation assessment (per candidate)
Four search tasks executed via deep research agent:
- Segment literature volume: How many papers exist for A→B, B→C, A→C, and A→B→C combined?
- Existing reviews: Any systematic reviews or meta-analyses covering the full chain?
- Recent signposts: Papers from 2024-2026 that mention "future research should explore..." this direction
- Hypothesis status: Has someone proposed this hypothesis without testing it? (High-value gap)
Step 3c: Gap scoring
| Dimension | 1 (abandon) | 3 (viable) | 5 (blue ocean) |
|---|---|---|---|
| Endpoint maturity | Neither end mature | One end has data | Both ends have meta-analyses |
| Bridge vacancy | Reviews already exist | Sparse papers (<10) | Serial analysis = zero |
| Signpost signals | Nobody mentions it | Mentioned in reviews | Explicitly flagged as gap |
| Timeliness | Flagged >2 years ago | Flagged within 1 year | Flagged within 6 months |
Only candidates scoring ≥12/20 advance to Phase 4.
Phase 4: Data Feasibility Verification (Deep Research-Driven)
For each surviving candidate:
- Variable coverage: Do target datasets contain all required variables? Which cycles/waves?
- Sample size estimation: How many participants have complete data on all key variables?
- Methodological toolkit: What statistical methods fit this scenario? Are there established R/Python packages?
Phase 5: Competition & Publication Assessment (Deep Research-Driven)
- Competing teams: Who is doing the closest work? What's their publication velocity?
- Target journals: Which journals publish cross-domain work? Recent acceptance of similar studies?
- Policy impact: Does this direction connect to active policy debates?
Phase 6: Synthesis & Research Matrix Design
Compile all evidence into a ranked report with:
- Candidate directions sorted by composite score
- Recommended research matrix (typically 6-8 papers with citation relationships)
- Risk assessment and mitigation strategies
- Decision points requiring human judgment
4. Validation: The Chemical Exposure Discovery
We applied this method starting from 8 completed training projects (NHANES-based epidemiological studies covering sleep, smoking, vitamin D, cognition, inflammation, ICU prediction, pollution, and digital divide).
Phase 1 identified recurring variables: inflammatory markers, metabolic indices, and mental health outcomes appeared across multiple projects.
Phase 2 selected a hybrid structure (mechanism chain + exposure-driven) based on the founder's preference for "blank space."
Phase 3 deep research revealed:
- Environmental toxicology → metabolic disruption: mature (thousands of papers on heavy metals, PFAS, phthalates affecting insulin resistance)
- Metabolic disruption → psychiatric outcomes: mature (hundreds of papers on insulin resistance and depression)
- Environmental toxicology → metabolic disruption → psychiatric outcomes (serial mediation): ZERO papers
This was the gap. Both riverbanks were thoroughly mapped, but nobody had built the bridge.
Phase 4 confirmed NHANES contains all required variables: biomonitored chemical exposures (blood/urine metals, PFAS, phthalates), metabolic markers (HOMA-IR, MetS components), and psychiatric assessments (PHQ-9) — all in overlapping subsamples.
Phase 5 found no competing teams working on this specific three-stage pathway, a 12-18 month publication window, and strong journal receptivity (Environmental Health Perspectives, Environment International).
Result: An 8-paper research matrix:
| # | Paper | Difficulty | Strategy |
|---|---|---|---|
| 1 | Exposure profile (ExWAS) | L1 | Landscape mapping |
| 2 | Heavy metals → metabolism → depression | L2 | Core mechanism |
| 3 | PFAS → inflammation → depression | L3 | Flagship |
| 4 | Phthalates → cognition | L3 | Extension |
| 5 | Full ExWAS landscape | L4 | Comprehensive |
| 6 | Systematic review | L2 | Evidence synthesis |
| 7 | Sex differences | L3 | Stratification |
| 8 | Umbrella review | L4 | Capstone |
5. Why This Matters More Than Better Execution
Every month, AI execution capabilities improve. Code generation gets better. Statistical analysis gets more automated. Figure generation gets prettier.
But the question — "what should we study?" — remains fundamentally a judgment call. It requires:
- Taste: Knowing which problems matter and which are academic busywork
- Peripheral vision: Seeing connections across fields that specialists miss
- Timing sense: Knowing when a gap is ripe for exploitation vs. prematurely empty
Cross-Domain Gap Scanning doesn't automate taste. It augments it — by systematically surfacing candidates that a human researcher can then evaluate with their domain intuition. The human still decides. But they decide from a much richer menu.
6. Limitations
- Deep research dependency: The quality of gap detection depends entirely on search coverage. Obscure or non-English literature may be missed.
- False negatives: A gap scored as "blue ocean" might actually have ongoing work in preprint or under review.
- Feasibility ≠ importance: A gap may be empty because it's not interesting, not because nobody thought of it.
- Human judgment still required: The method surfaces candidates but cannot evaluate whether a research direction is truly important. That remains a human prerogative.
7. Conclusion
When execution becomes commodity, discovery becomes the competitive advantage. Cross-Domain Gap Scanning provides a systematic, evidence-based method for the hardest part of research: deciding what to study.
The method's power comes not from any single technique but from disciplined sequencing: inventory before structure, structure before search, search before feasibility, feasibility before competition. Each phase constrains the next, preventing the common failure mode of falling in love with a direction before checking if it's viable.
We believe this represents a necessary shift in how AI research systems are designed. The current generation optimizes for speed of execution. The next generation must optimize for quality of questions.
"The formulation of the problem is often more essential than its solution." — Albert Einstein
Reproducibility: Skill File
Use this skill file to reproduce the research with an AI agent.
---
name: research-frontier
description: >
研究方向发现引擎。在成熟领域的交界处发现没人建的桥。
触发条件: "找方向"/"研究方向"/"选题"/"下一轮做什么"/"research frontier"。
核心方法: 资产盘点 → 结构选型 → 交叉缝隙扫描(deep research驱动) → 证据验证 → 可行性确认。
allowed-tools: Read, Write, Edit, Bash, Glob, Grep, WebSearch, WebFetch, Agent
---
# Research Frontier — 研究方向发现引擎
## 核心理念
**在两个成熟领域的交界处，找没人建的桥。**
不是在热门领域里找增量选题，而是系统性地扫描领域交叉的缝隙，用 deep research 验证空白真实存在，用数据可行性确认能做，用竞争窗口判断该不该现在做。
```
传统选题: 领域内文献综述 → 找gap → 做增量研究（拥挤）
本方法: 多领域资产图谱 → 交叉缝隙扫描 → deep research验证空白 → 锁定蓝海
```
## 方法论来源
2026-03-20 实战验证：从8个已完成训练任务出发，通过多元思维格栅分析 → 交叉领域缝隙扫描 → deep research 验证，锁定"环境化学暴露→代谢→心理健康"三段通路方向。该方向经系统搜索确认为完全空白（三段串联中介分析=零篇论文），且数据(NHANES)、方法(BKMR-CMA)、发表窗口(12-18个月)三者齐备。
---
## 输入
- **必需**: 用户的意图信号（"找方向"/"下一轮做什么"/研究偏好）
- **推荐**: 已有研究组合（已完成的项目/论文/分析结果）
- **推荐**: 可用数据集清单
- **可选**: 用户偏好约束（"要空白的"/"要争议性的"/"要临床可转化的"）
## 输出
- `frontier_report.md` — 完整调研报告（含证据链接）
- `candidate_directions.json` — 结构化的候选方向清单（排序+评分）
- 用户决策后 → 可直接输出为训练任务的 `requirements.md`
---
## 数据流总图
```
用户触发("找方向"/"选题"/"下一轮做什么")
│
▼
Phase 1 ──→ asset_map.md ──→ Phase 2
(输入适配) (可能为空) (结构选型)
│
用户注入偏好 ←── "要空白的"/"要争议性的"/...
│
▼
Phase 3 (deep research)
搜索 3-5 个候选方向的文献饱和度
│
▼
空白评分排序 ──→ 得分≥12 的进入 Phase 4
│
▼
Phase 4 (deep research)
数据可行性验证
│
▼
Phase 5 (deep research)
竞争 + 期刊 + 窗口
│
▼
Phase 6 综合报告
│
▼
用户决策 → requirements.md × N
```
---
## 六阶段流程
### Phase 1: 资产盘点（Asset Inventory）
**目标**: 理解"我们已经有什么"，避免从零开始。也可能什么都没有，这完全合法。
#### 输入适配器
Phase 1 的第一步是判断用户给的是什么，再决定怎么提取信息：
```
用户输入
│
▼
输入类型检测
│
├── A. 路径 → 是项目目录？
│ 扫描 progress.md / requirements.md / outputs/
│ 提取: 暴露变量、结局变量、中介变量、数据集、分析方法
│ → 生成 asset_map.md
│
├── B. 路径 → 是数据文件 (CSV/XPT/Excel)？
│ 读取列名 + 数据画像（行数、变量类型、缺失率）
│ 匹配已知数据集特征（如含 SDMVPSU → NHANES）
│ → 生成 asset_map.md（仅含数据资产，无研究资产）
│
├── C. 文本 → 含 DOI 或论文标题？
│ 搜索论文摘要，提取变量关系（暴露→结局）
│ 以该论文为种子，识别其引用网络中的变量空间
│ → 生成 asset_map.md（以论文为锚点）
│
├── D. 文本 → 是领域描述（"心理健康"/"环境流行病学"）？
│ 无本地资产，Phase 1 输出 asset_map = 空
│ Phase 3 改为从公开数据集变量空间广撒网搜索
│ → asset_map.md 为空（标记 source: "领域描述"）
│
├── E. 文本 → 是数据集名称（"NHANES"/"BRFSS"）？
│ 加载该数据集的已知变量清单/模块清单
│ 标注变量类别（暴露/结局/协变量）
│ → 生成 asset_map.md（仅含数据集能力图谱）
│
└── F. 无输入 / "帮我找方向"？
加载可用公开数据集清单（NHANES/BRFSS/CHARLS/CFPS/MIMIC/...）
每个数据集标注核心变量域
→ asset_map.md 为空 + 附可用数据集清单
```
#### 当有本地资产时（场景 A）
1. 扫描已完成的项目/任务目录，提取每个研究的:
- 暴露变量（Exposure）
- 结局变量（Outcome）
- 中介/调节变量（Mediator/Moderator）
- 使用的数据集
- 使用的分析方法
2. 构建**变量关系图**: 哪些变量之间已经有了连接，哪些还是孤立节点
3. 识别**重复出现的主题**: 某个变量在多个研究中出现 = 潜在的核心枢纽
#### 当无本地资产时（场景 D/F）
1. 加载可用公开数据集清单及各自的核心变量域
2. 变量关系图为空 → Phase 3 改为"变量空间广撒网"模式
**输出**: `asset_map.md`（变量关系图 + 主题频次 + 数据集覆盖矩阵。可能为空。）
---
### Phase 2: 结构选型 + 用户偏好注入（Structure Selection）
**目标**: 确定研究矩阵的骨架类型，并获取用户的方向性约束。
#### 2a. 结构选型
三种经典结构:
| 结构 | 逻辑 | 鲁棒性 | 学术影响力 | 终局产出 |
|------|------|--------|----------|---------|
| 暴露驱动 | 一个X → 多个Y | 高（共享数据） | 中 | 伞状评价 |
| 结局驱动 | 多个X → 一个Y | 高（单篇独立） | 中 | 系统综述+PAF |
| 机制链 | A→B→C→D 串联 | 低（链条脆弱） | 高 | 理论贡献 |
| **混合**（推荐） | 主轴+嵌入机制线 | 中高 | 高 | 综述+原创框架 |
操作:
1. 如果 asset_map 非空 → 评估哪种结构最匹配已有资产
2. 如果 asset_map 为空 → 展示三种结构的优劣，让用户选或由用户偏好决定
3. 用逆向思维评估每种结构的死法（"选这个结构最可能怎么失败？"）
4. 用类比迁移参考成功研究团队的范式
#### 2b. 用户偏好注入（强制检查点）
**这一步不可跳过。** 必须向用户确认或从用户输入中提取：
- 新颖性偏好: 空白/争议 → Phase 3 过滤掉饱和方向
- 新颖性偏好: 增量/安全 → Phase 3 保留有文献基础的方向
- 领域约束: 限定某个领域 / 排除某个领域
- 数据约束: 只用某个数据集 / 排除某个数据集
- 实用约束: 临床可转化 / 政策导向 / 纯学术
偏好约束直接影响 Phase 3 的搜索策略和评分权重。例如"要空白的"会把空白评分权重从 25% 提升到 50%。
**输出**: 推荐的结构类型 + 用户偏好约束（传递给 Phase 3）
### Phase 3: 交叉缝隙扫描（Gap Scanning）— Deep Research 驱动
**这是整个流程的核心引擎。不是拍脑袋猜空白，而是系统搜索验证空白。**
操作:
**3a. 生成候选交叉点**
根据 asset_map 是否为空，采用不同策略：
**精准模式（asset_map 非空）：**
从变量关系图中，识别所有"两端成熟但中间无桥"的变量对:
```
对于资产图中的每个变量A:
对于每个可能的变量C（来自不同领域）:
如果 A→B 已有文献（领域1成熟）
且 B→C 已有文献（领域2成熟）
则 "A→B→C 串联" 是一个候选交叉点
```
**广撒网模式（asset_map 为空）：**
从可用数据集的变量域出发，枚举跨领域组合:
```
对于每个可用数据集:
列出其核心变量域（如 NHANES: 环境化学物/代谢/心理/营养/人口学）
对于每对变量域 (域A, 域B):
如果 域A 和 域B 属于不同的传统学科
则 "域A → 域B" 是一个候选交叉方向
→ 启动 deep research 评估该方向的文献饱和度
```
广撒网模式会产生更多候选方向，因此 deep research 的第一轮搜索只做快速饱和度判断（搜索结果数），过滤后再对 top 5 做深入搜索。
**3b. Deep Research: 饱和度评估**（每个候选方向）
启动 deep-research-agent，对每个候选方向执行:
```
搜索任务 1: 通路各段的文献量
- "{变量A} AND {变量B}" 的 PubMed/Google Scholar 结果数
- "{变量B} AND {变量C}" 的结果数
- "{变量A} AND {变量C}" 的结果数（直接关联）
- "{变量A} AND {变量B} AND {变量C}" 的结果数（三段串联）
搜索任务 2: 是否已有系统综述/meta-analysis
- "systematic review" OR "meta-analysis" + 各段关键词
- 如果三段串联已有综述 → 该方向不空白，降级
搜索任务 3: 最近2年的最接近论文
- 2024-2026年发表的、与该交叉点最相关的论文
- 这些论文的结论中是否提到了"future research should..."
- 有人插路标但没人走 = 最佳窗口
搜索任务 4: 是否有人提出过这个假说
- 搜索概念性综述、评论文章、预印本
- 如果假说已被提出但无实证 = 高价值空白
- 如果假说未被提出 = 可能是开创性方向，也可能是因为不合理
```
**3c. 空白评分**
| 维度 | 1分（放弃） | 3分（可做） | 5分（蓝海） |
|------|-----------|-----------|-----------|
| 两端成熟度 | 一端也不成熟 | 一端成熟一端有数据 | 两端均有meta-analysis |
| 中间空白度 | 已有综述 | 有零散论文(<10篇) | 串联分析=零 |
| 路标信号 | 无人提及 | 综述中mention过 | 有人明确标注为gap |
| 时效性 | 2年前就有人标注 | 1年内标注 | 最近6个月标注 |
输出: 候选方向排序（按空白评分）
### Phase 4: 数据可行性验证（Data Feasibility）— Deep Research 驱动
**只对 Phase 3 中得分 ≥12 的候选方向执行。**
启动 deep-research-agent:
```
搜索任务 1: 变量覆盖
- 目标数据集中，关键变量是否存在？哪些cycle/波次？
- 变量的测量方式是否一致（跨cycle可比性）？
搜索任务 2: 样本量估算
- 所有关键变量同时有数据的样本量
- 子样本（如NHANES的1/3生物监测子样本）的瓶颈
- 是否足以支撑混合暴露分析的统计功效
搜索任务 3: 方法学工具
- 这个场景下主流的统计方法是什么？
- 是否有现成的软件包/R包/Python库？
- 方法论文是否已被广泛引用（>50次 = 审稿人接受）
```
输出: 可行性评估（数据充分/部分可行/不可行）
### Phase 5: 竞争与发表评估（Competition & Publication）— Deep Research 驱动
```
搜索任务 1: 竞争团队
- 哪些研究组在做最接近的工作？
- 他们的发表节奏（每年几篇？正在加速？）
- 竞争窗口估算: 按现有发表速度，空白还能维持多久？
搜索任务 2: 目标期刊
- 该交叉领域的期刊有哪些？IF范围？
- 这些期刊最近是否发表过类似交叉研究？（编辑接受度）
- 综述/终局论文的最佳投稿目标
搜索任务 3: 政策/临床影响力
- 这个方向如果做出来，对公共政策有什么含义？
- 是否有正在进行的政策辩论可以借力？
```
输出: 竞争格局 + 期刊策略 + 影响力评估
### Phase 6: 综合研判与输出（Synthesis）
将 Phase 1-5 的结果综合，生成最终报告:
```markdown
# 研究方向调研报告
## 候选方向排序
| 排名 | 方向 | 空白评分 | 数据可行性 | 竞争窗口 | 综合推荐 |
|------|------|---------|-----------|---------|---------|
## 推荐方向详述
### 宏大叙事
[一段话描述整个研究矩阵的故事]
### 研究矩阵设计
[8篇论文的布局 + 论文间引用关系]
### 风险评估
[主要风险 + 缓解策略]
### 决策点（需要人判断）
[方向/数据/期刊等需要用户选择的关键决策]
```
---
## Phase 间数据交接
```
Phase 1 Phase 2 Phase 3
asset_map.md ────────→ structure_type ────────→ candidate_gaps[]
(变量关系图, (暴露驱动/结局驱动/ (每个候选方向含:
可能为空) 机制链/混合) 两端文献量,
+ 串联文献量,
user_preferences 最近论文,
(空白/争议/增量, 空白评分)
领域/数据约束)
│ │ │
│ ┌──────────────────┘ │
│ │ │
▼ ▼ ▼
如果为空 → Phase 3 切换为广撒网模式 得分≥12的进入 Phase 4
如果非空 → Phase 3 用精准模式
Phase 4 Phase 5 Phase 6
data_feasibility ────→ competition_report ───→ frontier_report.md
(变量覆盖, (竞争团队, (完整报告)
样本量估算, 期刊分层, +
方法学工具, 竞争窗口, candidate_directions.json
瓶颈标注) 政策影响力) (结构化排序)
│
▼
用户决策
│
▼
requirements.md × N
(直接输出为训练任务需求)
```
---
## 与其他模块的关系
| 上游 | 本模块 | 下游 |
|------|--------|------|
| 已完成的研究任务 / 数据文件 / 论文 / 领域描述 / 无 | **research-frontier** | training-curriculum.md（训练任务矩阵） |
| idea-analyst（Phase 2 结构分析时调用） | 调用其格栅分析能力 | requirements.md（每个任务的需求） |
| — | Phase 3/4/5 各自启动 deep-research-agent | evo-loop.sh（执行训练循环） |
## 触发条件
- 用户说"找方向"/"选题"/"下一轮做什么"/"research frontier"
- 一轮训练任务全部完成后，自动建议运行
- 用户提供新数据集时，评估可做的方向
## 关键约束
1. **不猜测，搜证据**: 任何"这个领域是空白"的判断必须有搜索记录支撑
2. **不选热门**: 如果某方向已有 >3 篇系统综述，自动降级（除非用户偏好为"增量"）
3. **数据先行**: 没有可用公开数据的方向不进入候选
4. **人类决策权**: 最终方向选择必须由用户确认，本模块只提供排序和证据
5. **偏好注入不可跳过**: Phase 2b 必须获取用户偏好再进入 Phase 3，不能自行假设用户想要什么
Discussion (0)
to join the discussion.
No comments yet. Be the first to discuss this paper.