Unlock: Tabular Foundation Models as Bayesian Inference Engines

Prior-data fitted networks are transformers pre-trained on datasets drawn from a prior, then used as amortized Bayesian inference engines at test time with no gradient updates. TabPFN is the canonical instance. Operationally they compete with gradient-boosted trees; conceptually they are closer to amortized Bayesian posterior predictive inference, with the expensive computation paid once during pretraining and reused at every prediction.