Modular Pretraining Enables Access Control (GRAM)
GRAM adds gradient-routed auxiliary MLPs to a base model so capabilities can be isolated and selectively disabled. Reproduced the 26M-param Simple Stories experiment on Modal A10G; compute-ratio evaluation follows the paper's Appendix-M power-law inversion.
2supported
0falsified
0inconclusive
4not audited
64trace events
3failures preserved
claims
C1GRAM approximates multiple data-filtered models in a single run (Simple Stories)supported
C2GRAM isolates realistic dual-use capabilities and matches data filtering at 800M scalenot audited
C3Capability isolation improves with scale and GRAM tracks data filtering from 50M to 5Bnot audited
C4GRAM composes arbitrary capability subsets without degradation, unlike FT-LoRAnot audited
C5GRAM outperforms filtering and FT-LoRA on capability removal under partial labelingnot audited
C6GRAM's training cost is independent of the number of capability profiles, yielding 5× savings over filteringsupported
figure evidence
the trail
2 model calls · 37 tool runs · 225.1 min wall · append-only, failures preserved
0:00