基准 v2：怎么评价一个理论Benchmark v2: How to Judge a Theory

要预测的量The Quantity to Predict

v2 的估计对象是：v2's estimand is:

p(ΔC | Ŝ, b, pos ; π)

读作："在测得流量 Ŝ、背景 b、探测器位置 pos 的条件下，ΔC 服从什么分布？"分号后面的 π 是事先声明的源计数先验（source-count prior）——天上暗源和亮源的比例。E151 证明了这个先验必须声明：同一个模型、同一批数据，只改先验，条件均值就会移动（α = 2.5 时有符号均值从 −0.011σ 移到 −0.115σ）。Read as: "given the measured flux Ŝ, background b, detector position pos, what distribution does ΔC follow?" The π after the semicolon is the pre-declared source-count prior — the ratio of faint to bright sources in the sky. E151 proved this prior must be declared: same model, same data, only the prior changed, and the conditional mean moved (at α = 2.5 the signed mean moved from −0.011σ to −0.115σ).

来源：cognition-os/experiments/E151_prior_truncation_noise_diagnostics.md。Source: cognition-os/experiments/E151_prior_truncation_noise_diagnostics.md.

铁律（v2 版）：条件于观测量加声明的 π；S_true 只能通过 π 积分进入，永远不能作为逐行输入；统计在 ΔC 空间做，绝不按 ML/DET_ML 切样本。按 S_true 检查只算诊断，不进前沿。Iron Rule (v2 wording): condition on the observables plus the declared π; S_true may only enter through the π integral, never as a row-by-row input; statistics are done in ΔC space, never cutting samples on ML/DET_ML. Checks against S_true are diagnostics only and never enter the front.

四层数据The Four Data Layers

C-iso 是 10 月新加的层：只保留孤立点源（2 角分内无延展源、最近邻 ≥ 60 角秒）。理论描述的是孤立点源；E162 第 2 阶段证明最内环的亏缺来自拥挤环境（有延展邻居时残差 −1.0 到 −2.0σ，没有时 −0.2 到 −0.8σ）。C-iso is the layer added in month 10: only isolated point sources (no extended source within 2 arcmin, nearest neighbour ≥ 60 arcsec). The theory describes isolated point sources; E162 stage 2 proved the innermost-ring deficit comes from crowding (with an extended neighbour the residual is −1.0 to −2.0σ, without −0.2 to −0.8σ).

四个误差轴The Four Error Axes

模型插件对每一行输出 ΔC 的预测均值和宽度。计分时把每行换成 z = (ΔC_obs − 均值) / 宽度，然后按（离轴环 × 背景切片 × Ŝ 箱）分箱检查：The model plug-in outputs a predicted mean and width of ΔC for each row. At scoring time each row becomes z = (ΔC_obs − mean) / width, then is checked in (off-axis ring × background slice × Ŝ bin) bins:

另外报告但不进前沿：CRPS（单一数字汇总）、PIT 直方图、偏度、峰度、|z| > 3 的比例、每背景分解，以及压力测试 S1–S5（自截断、先验网格、背景标签、私有位置迁移、分箱加减倍）。Also reported but not entering the front: CRPS (a single-number summary), PIT histogram, skewness, kurtosis, the |z| > 3 fraction, per-background decomposition, and stress tests S1–S5 (self-truncation, prior grid, background labels, private-position transfer, binning halved/doubled).

报告窗口与高计数原则Report Windows & the High-Count Principle

（2026-10-13 用户决定，PARETO_BENCHMARK_V2 §3）：DET_ML 6–10 是主窗口，也是前沿选择唯一依据的窗口；10–20、20–50、50–100 作为报告窗口，> 100 只作背景参考。每个报告窗口给出 A1、A2（原始值和噪声校正值）加两级区间，以及比例残差 frac = mean(ΔC/pred − 1)。(2026-10-13 user decision, PARETO_BENCHMARK_V2 §3): DET_ML 6–10 is the primary window, and the only one used for front selection; 10–20, 20–50 and 50–100 are report windows, > 100 is context only. Each report window reports A1, A2 (raw and noise-corrected) with two-level intervals, plus the fractional residual frac = mean(ΔC/pred − 1).

高计数原则（用户原话）：模型不应该在计数更多的时候反而失效——那就变成为了修正而修正。高计数退化标记：以未修正的 V4c_no_eta 为参照，任何一个报告窗口（不超过 100）里 A1 显著更差（配对区间不含 0）就打标记。标记要不要进入前沿选择，等报告出来后由用户决定（待定）。The high-count principle (user's words): a model must not fail when the source has more counts — otherwise it is correction for its own sake. The high-count degradation flag: with the uncorrected V4c_no_eta as reference, an entry is flagged if its A1 is significantly worse (paired interval excluding 0) in any report window up to 100. Whether the flag later enters front selection is the user's decision after the first report (Pending).

噪声底与 ε-Pareto 前沿Noise Floor & the ε-Pareto Front

有限样本本身会让完美模型得到非零的 A1/A2——这就是噪声底（noise floor）。计分器对每个条目都算出它的噪声底，并给出扣除后的分数 √(score² − floor²)。两个条目的差异要过两级配对自助（two-level paired bootstrap）：先按图像（catalog 上按观测 OBS_ID）重采样，再在簇内重采样，95% 区间含 0 算平局（No significant difference持平）。Finite samples alone give a perfect model non-zero A1/A2 — this is the noise floor. The scorer computes each entry's floor and reports the corrected score √(score² − floor²). Differences between two entries must pass a two-level paired bootstrap: resample by image (OBS_ID on the catalogue) first, then within clusters; a 95% interval containing 0 is a tie (No significant difference).

ε-Pareto 前沿的构造：如果一个条目在所有误差轴上都不比另一个差（差异在噪声内算"不差"），且至少一轴显著更好，后者就被支配。不被任何人支配的条目组成前沿；前沿内并列时参数最少的优先。Construction of the ε-Pareto front: if an entry is no worse than another on every error axis (within-noise differences count as "no worse") and significantly better on at least one, the other is dominated. The undominated entries form the front; ties within the front prefer fewer parameters.

动画 5：C-iso 层当前 10 个条目在 (A1, A2) 平面上的位置。绿色实心点是样本外前沿（2 个零参数模型：T3_joint_2D 与 appendix_eff），灰色是已被支配的条目，红色空心菱形是样本内的 6 参数模型（不与样本外直接比）。Animation 5: the current 10 C-iso entries in the (A1, A2) plane. Green filled points are the out-of-sample front (2 zero-parameter models: T3_joint_2D and appendix_eff), grey are dominated entries, the red hollow diamond is the in-sample 6-parameter model (not compared directly with out-of-sample).

来源：benchmark/leaderboard_v2.json（2026-10-16T18:08:36Z 生成；本页数字照该文件重刷，2026-10-17）。Source: benchmark/leaderboard_v2.json (generated 2026-10-16T18:08:36Z; this page's numbers were refreshed against that file on 2026-10-17).

先验与 L0 设计Priors & the L0 Design

v2 的先验不是重新模拟出来的，而是通过抽样得到的（用户 2026-10-09 提议）：v2's priors are not re-simulated; they are obtained by thinning (user proposal 2026-10-09):

1. L0 模拟从已知的连续母分布（3 个幂律的等权混合，α_q = 0.5/1.5/2.5，范围 [S_min, 1000] counts）逐源抽取 S_true，带种子并逐源记录；the L0 simulation draws S_true per source from a known continuous parent (an equal-weight mixture of 3 power laws, α_q = 0.5/1.5/2.5, range [S_min, 1000] counts), seeded and recorded per source;
2. 每个源存一个固定均匀随机数 u；each source stores one fixed uniform random number u;
3. 要构造任何目标先验 dN/dS ∝ S^−α，只需保留 u < π_α(S)/(M·q(S)) 的源（"减薄"/thinning），不用重新模拟；to build any target prior dN/dS ∝ S^−α, keep only sources with u < π_α(S)/(M·q(S)) ("thinning"), no re-simulation;
4. 每个列表的实际 α 由它保留的 S_true 做最大似然实测（附区间和 KS 检验），实测 α 而不是名义 α 是条目的标签；each list's actual α is measured by maximum likelihood on its retained S_true (with interval and KS test); the measured α, not the nominal one, labels the entry;
5. 注入表记录每一个注入的源——包括从未被探测到的——因为先验属于注入的总体，不属于被计分的行。the injection table records every injected source — including ones never detected — because the prior belongs to the injected population, not to the scored rows.

α 网格：{0, 0.5, 1, 1.5, 2, 2.5}，外加一个从文献 logN-logS 换算的"天空先验"（T8/E159，Mateos 2008 的 2–4.5 keV 计数，经 DR14 曝光分布卷积，窗口内 α_eff ≈ 1.6–2.5）。每个条目的轴值取网格上的最差值（worst over grid）。α grid: {0, 0.5, 1, 1.5, 2, 2.5}, plus a "sky prior" converted from the literature logN-logS (T8/E159, Mateos 2008 2–4.5 keV counts, convolved with the DR14 exposure distribution; α_eff ≈ 1.6–2.5 in the window). Each entry's axis value is the worst over the grid.

为什么"零参数"是目标Why "Zero Parameters" Is the Goal

一个在数据上拟合出来的数，永远可以说"它只是吸收了这批数据的特殊性"。只有零参数的公式——每个系数都来自推导或独立数据——才构成对物理的理解。基准把这一点变成硬规则：A number fitted on the data can always be excused as "absorbing this dataset's quirks". Only zero-parameter formulas — every coefficient from derivation or independent data — constitute understanding of the physics. The benchmark turns this into a hard rule:

• 参数计数是诚实的：迁移来的拟合数也算；parameter counting is honest: transferred fitted numbers count too;
• 在 catalog 行上拟合过参数的条目标记为样本内（in-sample），单独成前沿，不能作为理论的证据；entries whose parameters were fitted on catalogue rows are marked in-sample, form a separate front, and cannot serve as evidence for the theory;
• 论文的 catalog 结论必须由样本外（out-of-sample）条目做出。the paper's catalogue conclusions must come from out-of-sample entries.

当前样本外前沿上的条目（T3_joint_2D、appendix_eff）参数都是 0；样本内前沿上只有 V5c_eta，用了 6 个参数。零参数理论在均值上已经追平它（C-iso 0.132 / 0.153 对 0.125），宽度上 appendix_eff 也基本追平（0.052 对 0.049），T3_joint_2D 的宽度还差一点（0.111）。（前沿口径见 07 R018：前沿用原始分数；头条数字是噪声校正后的。）The entries on the current out-of-sample front (T3_joint_2D, appendix_eff) both have 0 parameters; the in-sample front holds only V5c_eta, with 6. The zero-parameter theory has caught up on the mean (C-iso 0.132 / 0.153 against 0.125), and appendix_eff has essentially caught up on width too (0.052 against 0.049); T3_joint_2D's width still lags (0.111). (Front convention per 07 R018: fronts use raw scores; the headline numbers are noise-corrected.)

附录基线（appendix baseline）是什么（T216/T217，07 R017/R018）：两个附录条目 appendix_eff 与 appendix_psfgen 于 2026-10-16 登记，用来量化"论文附录里那个 PSF 模板"与基准核之间的差别。appendix_eff 用有效模板，appendix_psfgen 用 psfgen 逐位置模板。两者都是零参数、都只作报告与参照，不是新的物理主张：psfgen 模板比核多约 11 % 的俘获核，方向与核的振幅约定（约 12 %）相反，范围 1.06–1.25（M2 b4）、1.083–1.121（1–4 波段）、第 5 波段排除（1.20–1.25）。M31 灵敏度图因此高估 \(\Delta C\) 约 10 % 的读法已 Deprecated——实测 M31 psfgen/解析之比是 1.006–1.064，不是 1.10。它进前沿靠的是宽度，不是均值。What the appendix baseline is (T216/T217, 07 R017/R018): the two appendix entries appendix_eff and appendix_psfgen were registered on 2026-10-16 to quantify the difference between "the PSF template in the paper's appendix" and the benchmark kernel. appendix_eff uses the effective template, appendix_psfgen the psfgen per-position template. Both are zero-parameter and are reports and references only, not new physics claims: the psfgen template carries about 11 % more captured core than the kernel, in the opposite direction to the kernel's amplitude convention (\(\approx\) 12 %), with a range of 1.06–1.25 (M2 b4), 1.083–1.121 (bands 1–4), band 5 excluded (1.20–1.25). The reading that the M31 sensitivity map therefore over-predicts \(\Delta C\) by about 10 % is Deprecated — the measured M31 psfgen/analytic ratio is 1.006–1.064, not 1.10. It sits on the front through width, not through the mean.

注意（2026-10-13 评审更正）：前沿条目用的二维损失经 T11_020 窗口平坦性检验不是闭合项——它在 6–10 窗口对 MOS 最好，但在 3 个窗口不平坦，且在 PN 上扣多了约 +0.06（PN 由 (1−w) 闭合）。条目分数仍然成立，但联合拟合损失的正确形式是当前主攻的开放问题（见工作台）。另外，T5_008 评审发现 6–10 领先条目的加性项 a ≈ −1（多减了约 1 个 ΔC 单位），而 T3_joint_L07 的 a 区间含 0——前者是在用加性项替补缺失的乘性项 m·μ。Note (2026-10-13 review correction): the two-dimensional loss used by the front entries failed the T11_020 window-flatness test — it is not the closing term: best for MOS in the 6–10 window, but not flat across the 3 windows, and it over-subtracts on PN by ~+0.06 (PN is closed by (1−w)). The entries' scores still stand, but the correct form of the joint-fit loss is the currently-pursued open problem (see the Workbench). Also, the T5_008 review found the 6–10 leaders' additive term a ≈ −1 (about 1 ΔC unit of over-subtraction), while T3_joint_L07's a interval contains 0 — the former are using an additive term to stand in for the missing multiplicative term m·μ.

当前条目与前沿（C-iso，DET_ML 6–10）Current Entries & the Front (C-iso, DET_ML 6–10)

来源：benchmark/leaderboard_v2.json（2026-10-16T18:08:36Z 生成；C-iso 层，噪声校正后点值，公开位置）。C-all 层同一文件，趋势相同（T3_joint_2D A1 0.136、A2 0.176）。（前沿口径更正，07 R018：前沿用原始分数，"样本外前沿"列按 class_selections.out-of-sample.front = {T3_joint_2D, appendix_eff}；appendix_psfgen 的 A1 0.885 是一个仅供宽度参考的点，因宽度并列被排除在 C-iso 前沿之外。）Source: benchmark/leaderboard_v2.json (generated 2026-10-16T18:08:36Z; C-iso layer, noise-corrected point values, public positions). C-all is in the same file, with the same trend (T3_joint_2D A1 0.136, A2 0.176). (Front convention corrected, 07 R018: fronts use raw scores; the "out-of-sample front" column follows class_selections.out-of-sample.front = {T3_joint_2D, appendix_eff}; appendix_psfgen's A1 0.885 is a width-only reference point, excluded from the C-iso front on a width tie.)

来源：benchmark/leaderboard_v2.json。Source: benchmark/leaderboard_v2.json.

私有集The Private Split

约 1/3 的模板位置被预先留出（按环分层、确定性哈希选取）。它的名单由 PI 保管，学生和 agent 永远打不开；私有分数只在里程碑时由用户开启，用来检验是否对公开位置过拟合。另外，v1 时代的验证切分（confirmatory split）也保持锁定，只在最终检查时开启一次。About 1/3 of the template positions were held out in advance (stratified by ring, selected by deterministic hashing). The list is kept by the PI and can never be opened by students or agents; private scores are unlocked by the user only at milestones, to check for overfitting to the public positions. The v1-era confirmatory split also stays locked, to be opened once at the final check.