ICL Is Provably Bayesian Inference, audited by Beam 1.0 501B (clean conditions)
The headline flip: two claims that stayed inconclusive through 7 recorded rounds of 27B-class models (Bayes-Gap/context coupling C2, distribution-shift stability C5) came back supported under Beam on clean conditions, and the risk identity C1 was recovered in one attempt (rel_diff 0.0008). C3/C4 honestly falsified on Beam's own implementations; C6/C7 inconclusive with failing controls flagged.
3supported
2falsified
2inconclusive
0not audited
8model calls
58tool runs
1.5wall min
2failures kept
claims
7 audited · click a claim to open the auditor's own summary — metrics, notes, control status
auditor's noteEmpirical squared-loss decomposition over a linear-vs-quadratic task mixture; Bayes predictor is the analytic posterior mean. Positive control uses M equal to the Bayes predictor.
R0.7878
RBG0.3115
RPV0.4769
RBG_plus_RPV0.7884
abs_diff0.0006572
rel_diff0.0008343
threshold0.007878
control_abs_diff0.000
control_passtrue
n_prompts1.500e+4
context_lengths5.000
n_evaluations7.500e+4
auditor's noteSimplified mean-pooling linear model trained by least squares on synthetic linear-regression prompts; Bayes Gap computed against exact ridge Bayes predictor. Positive control is the exact Bayes predictor with zero gap.
r2_joint0.9821
r2_N_only0.3728
r2_p_only0.2170
r2_joint_minus_N0.6092
r2_joint_minus_p0.7651
control_passtrue
control_gap0.000
min_BG0.00003626
max_BG0.003137
joint_params[0.00007645775592589728,4.517778118409092,1.1401628108785087,-0.06861856760903372]
N_params[0.000028699269219444383,0.13785988340743113,1.0306904312228231]
p_params[-0.0009448780078003193,0.004156342276219026,0.47436928457849503]
auditor's noteAudited the Bayes-optimal/sufficiently-pretrained limit: mixture posterior mean versus oracle posterior mean for linear vs quadratic regression tasks. Gap is task-type uncertainty; noise variance cancels in the gap.
n_replicates8000
k_values[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20]
mse_mixture[2.5311410954421505,1.6936353116256344,1.1693228807612412,0.8619791066168755,0.670610393044109,0.5486605115407468,0.45518837884508256,0.41046086908474333,0.3767041347340292,0.3604620520063995,0.343372521263249,0.33116602509784904,0.321013608887388,0.3136771368390933,0.3070020354271825,0.30127974027207827,0.2963323523550766,0.2932789655347235,0.29051123790681954,0.28736427162221295]
mse_oracle[2.4333277030647844,1.5578986752554531,1.0559226207068089,0.7727070152598794,0.5993163836523692,0.5036466134483785,0.4278580682784381,0.38505014237140506,0.35670685384365935,0.3407073245194264,0.32653872282311014,0.3185605084779919,0.31047095880424597,0.3056025674776818,0.2992097561018196,0.2955828360151025,0.29225221261066925,0.2892894778680418,0.2869137304581785,0.28441924120737533]
gap_mixture_minus_oracle[0.09781339237736608,0.1357366363701813,0.11340026005443238,0.0892720913569961,0.0712940093917398,0.04501389809236833,0.027330310566644478,0.025410726713338272,0.01999728089036984,0.0197547274869731,0.01683379844013888,0.012605516619857149,0.010542650083142036,0.008074569361411543,0.007792279325362916,0.005696904256975777,0.004080139744407352,0.003989487666681668,0.0035975074486410685,0.002945030414837624]
initial_gap_k10.09781
gap_k50.07129
relative_gap_k50.7289
monotonic_gapfalse
main_passfalse
control_passtrue
control_max_abs_gap0.000
auditor's noteSynthetic mixture of two Gaussian linear-regression task types; uniform-attention Transformer trained by ERM; Bayes Gap estimated against exact posterior-mean predictor.
m_values[4,8,12]
gap_m[0.2868191254480657,0.37242749085475435,0.4375744210432442]
slope_loggap_logm0.3837
monotonic_mfalse
pn_values[240,360,720]
gap_pn[0.34783293227963713,0.29761611685926964,0.43556143995900903]
slope_loggap_logpn0.2418
monotonic_pnfalse
control_passtrue
oracle_gap0.000
auditor's noteMonte Carlo Bayesian linear regression; model fixed at 0; W1 estimated empirically between input distributions.
corr_wasserstein_bayes_gap0.9617
corr_wasserstein_posterior_variance-0.5269
mean_abs_change_bayes_gap3.144
mean_abs_change_posterior_variance0.009629
ratio_pv_bg0.003063
control_passtrue
control_corr_bg0.9999
control_corr_pv0.1129
bg_source0.7756
pv_source0.2185
auditor's noteTwo-task scalar linear mixture; RPV by posterior over task/sign hypotheses; minimax risk via chi-square expectation; excess fitted to exponential.
r2_excess_fit—
lambda_fit—
d_min1.851
d_min_over_20.9257
monotone_decreasefalse
excess_k10.1486
excess_k_last-0.08384
control_r2—
control_lambda—
control_passfalse
auditor's noteConstant-mean meta-learning task; meta-learner is least squares on all pN prefix examples using mean-pooled context. Bayes Gap = out-of-sample MSE minus exact posterior variance.
spearman_cells0.1157
spearman_replicates0.07560
mean_within_pn_cv0.5533
control_spearman-0.9258
control_cv0.007360
control_passfalse
real_passfalse
cell_bg_means{"N100_p20":0.0008772057230283342,"N200_p10":0.0021689090623655243,"N400_p5":0.004090045031149048,"N500_p8":0.0027560389815750315,"N1000_p4":0.003587362140919049,"N2000_p2":0.0014029362843210335,"N1000_p8":0.002342699212316973}
why the judge trusts this trail
The verdict is bound to the evidence trail, not the outcome — five dimensions, all checked.
5/5PASS
structureclaims.json (7 claims), audit_report.json present
evidence7 audited claims; issues: none
integrity0 claims skipped as untestable, all declared untestable at extraction
cost disclosureper-claim attempt counts recorded; wall time in run files + trace
trace60 events, 8 LLM calls, 8 tool runs, 1 failed attempts preserved
figure evidence
the trail
8 model calls · 58 tool runs · 1.5 min wall · append-only, failures preserved
0:00