01 / PROBLEM
一次成功不等于可靠。One success does not equal reliability.
一次成功无法判断更复杂的推理是否值得额外 Token 和延迟；任务、Prompt、模型版本或环境漂移也会破坏比较公平性。One success cannot show whether a complex loop merits its additional tokens and latency. Task, prompt, model, or environment drift also breaks fair comparison.
02 / SYSTEM DECISION
比较开始前，先冻结实验。Freeze the experiment before comparing agents.
03 / FAILURE RISK
不冻结的比较，会把环境漂移误读成能力差异。An unfrozen comparison can mistake environment drift for capability.
任务、Prompt、模型版本和依赖只要发生变化，就可能让一次运行的结果失去可比性；没有成本边界时，迭代策略也难以被公平评价。Any drift in tasks, prompts, model versions, or dependencies can make a run incomparable; without a cost boundary, iterative strategies cannot be assessed fairly.
04 / ENGINEERING CONTROLS
把实验条件与运行边界写进系统。Put experiment conditions and execution bounds into the system.
05 / EVIDENCE
120 次真实 Trial 的报告结果。Results from 120 real trials.
20 个 HumanEval 任务、2 个 DeepSeek V4 Flash Agent、每任务 3 次运行。下面的对比对应 pilot 报告，不外推到真实仓库级任务。20 HumanEval tasks, two DeepSeek V4 Flash agents, and three runs per task. This pilot comparison does not generalize to repository-scale tasks.
| METRIC | DIRECT | ITERATIVE |
|---|---|---|
| pass@1 | 96.7% | 100.0% |
| pass@3 | 100.0% | 100.0% |
| pass^3 | 90.0% | 100.0% |
| 平均 TokenAverage tokens | 2,522.0 | 4,735.8 |
| P95 耗时P95 duration | 6.211 s | 10.012 s |
| 高峰成本上界Peak-cost bound | CNY 0.356820 | CNY 0.642084 |
06 / LIMITATIONS + LINKS
限制Limitation
这是小规模 HumanEval pilot，两个 Agent 使用同一模型系列。结果不代表统计显著性，也不能外推到真实仓库级任务。This is a small HumanEval pilot using one model family. It does not establish statistical significance or repository-level performance.