PROBABILISTIC ML / EXPLORABLE NOTE

Knowing what the model doesn’t know

A neural network answers every question with the same confident voice — inside its training distribution and far outside it. Bayesian methods replace the single answer with a distribution over answers, so the model can finally say the most useful thing a model can say: I’m not sure.

Everything on this page is live. Scan, dig, drag, click — the theory comes after you’ve felt it.
START HERE / BURIED TREASURE

Find the coin before you dig

Left: the beach — a coin sits hidden under one spot. Right: your belief about where it is. The only connection between the two is a metal detector that beeps over the coin 75% of the time — and false-beeps over plain sand 25% of the time.

TRY THIS

Click a sand spot to scan it. A beep pushes belief toward that spot; silence pushes it away. When you feel sure — or reckless — switch to dig mode and click the spot you want to open.

01 / THE BEACH

Where the coin really is (hidden)

click a spot to scan ↓
02 / YOUR BELIEF

What the scans add up to

BEST GUESS—
11%
1
11%
2
11%
3
11%
4
11%
5
11%
6
11%
7
11%
8
11%
9

taller bar = more belief the coin is under that spot

SCANNING

A coin is buried under one of these nine spots — and no, you don’t get to see where. Your only tool is the detector. Click any sand spot to scan it.

SCANS
0
EMPTY HOLES
0
HOW SURE
—
NAMING / WHAT YOU JUST DID

You were running Bayes’ rule

Every scan, the game did the same silent move. It weighted each spot’s current believability by how well that spot explains what the detector said. Then it rescaled the bars to sum to one. Here is that single move, frozen mid-update:

1 · WHAT YOU BELIEVED

Coins wash up near the water

The waterline is at the left end of the beach, so before any scan your belief already leans left. A starting belief can lean — that’s allowed.

PRIOR p(spot)
2 · WHAT YOU HEARD

BEEP at spot 3

B

Each bar: how well that spot explains the beep. “It’s under spot 3” explains it well (0.75). Any other spot means it was a false beep (0.25).

LIKELIHOOD p(beep | spot)
3 · WHAT YOU BELIEVE NOW

Multiply, spot by spot

Left lean × one beep: spot 3 towers, its waterline neighbors keep a little belief, the far end fades. The faint outline is where you started.

POSTERIOR p(spot | beep)
This is exactly what the beach game did on every scan: multiply what you believed by what you heard, renormalize. That’s the entire method — everything else on this page (regressions, CLIP) is the same move at larger scale. The spread of the surviving bars is your uncertainty.

Now the vocabulary, attached to things you’ve already touched. The belief bars before the scan are the prior. Each spot’s evidence score is the likelihood. The re-weighted bars afterward are the posterior. In symbols — the one equation this whole note runs on:

p(θ∣D)⏟bars after  =  p(D∣θ)⏞evidence score  p(θ)⏞bars beforep(D)⏟rescaling\underbrace{p(\theta\mid\mathcal{D})}_{\text{bars after}}\;=\;\frac{\overbrace{p(\mathcal{D}\mid\theta)}^{\text{evidence score}}\;\overbrace{p(\theta)}^{\text{bars before}}}{\underbrace{p(\mathcal{D})}_{\text{rescaling}}}

On the beach, a beep multiplies the scanned spot’s bar by 0.75 and every other spot’s by 0.25 — silence does the reverse. That’s the whole likelihood. Nothing else happens. That is the algorithm, and it’s the same one running inside any robot that localizes with noisy sensors. The spread of the surviving bars is your uncertainty, measured rather than felt.

THE PAYOFF / BAYESVLM (ICLR 2026)

Teaching CLIP to say “I’m not sure”

That one move scales all the way up. BayesVLM applies it to vision-language models like CLIP, which score an image–text pair with a single deterministic number — the cosine similarity of their embeddings. That number is just as confident on a distribution-shifted input as on a training-like one: a treasure hunter who digs on the very first beep, at a billion-parameter scale. The recipe (each step is unpacked by the experiments below):

  1. STEP 1Laplace where it countsKeep the pretrained weights, and fit a cheap Gaussian stand-in for the posterior (the Laplace approximation) over just the last projection layers of the image and text encoders. No training is repeated — Experiment 03 shows why the stand-in earns its place.
  2. STEP 2Embeddings become GaussiansWith uncertain projection weights, each embedding stops being a point and becomes an (approximately) Gaussian cloud in the joint space:   z∼N(μ,Σ)\;\mathbf{z}\sim\mathcal{N}(\boldsymbol{\mu},\boldsymbol{\Sigma}). Familiar inputs give tight clouds. Shifted inputs give diffuse ones.
  3. STEP 3Uncertainty flows into the scoreThe similarity of two Gaussian clouds is itself a distribution. BayesVLM’s ProbCosine propagates it analytically by moment matching — a mean and a variance for every image–text score, with no Monte Carlo sampling at inference time.
The BayesVLM pipeline: image and text encoders, a Laplace approximation over the projection layers, Gaussian feature projections, and distributions over cosine similarities
The full pipeline, drawn by the authors — encoders, Laplace approximation, Gaussian embeddings, and similarity scores that arrive as distributions. Figure from Baumann, Li, Klasson, Mentu, Karthik, Akata, Solin & Trapp, “Post-hoc Probabilistic Vision-Language Models” (ICLR 2026). All credit to the authors.
s(x,t)=cos⁡(f(x), g(t))  ⟶  s(x,t)∼(E[s], Var[s])s(x,t)=\cos\big(f(x),\,g(t)\big)\;\longrightarrow\; s(x,t)\sim\big(\mathbb{E}[s],\ \mathrm{Var}[s]\big)

The result: predictions keep the same accuracy but become calibrated — stated confidence finally tracks empirical correctness. And the variance itself is a usable signal. High-variance inputs are exactly the ones worth sending to a human, or worth labeling next (uncertainty-guided active learning). The model didn’t get smarter. It got honest, which in deployed systems is often worth more.

The rest of this note unpacks the two ideas that recipe leans on: what model uncertainty looks like when the candidates are infinite, and why a Gaussian stand-in for a posterior is an honest cheat.

UNDER THE MECHANISM / 1 OF 2

Don’t pick a winner. Keep every candidate.

The beach had nine spots. A regression has infinitely many candidates: every line you could draw through the data is one, and the same rule re-weights them all. The next experiment shows that directly. Gray lines are surviving candidates. Where they disagree, the model tells you it doesn’t know — the “diffuse cloud” from Step 2, in miniature. Try to break it: click two points directly above each other and see what no line can fix.

EXPERIMENT 02 / ONE CONFIDENT LINE, MANY HONEST LINES

The fan of gray lines is the model’s doubt

The cyan line is what an ordinary model reports: one answer everywhere. The gray lines are every explanation the data hasn’t ruled out. Where they hug the cyan line, trust it. Where they fan apart, it’s a guess.

TRY THIS

Drag the probe to the empty right side and watch the green bracket stretch. The gray lines disagree wildly there. Then click the plot at the probe to add one data point. Watch the whole fan close around it.

0246810input x — data lives on the left, the unknown on the rightTHE ONE CONFIDENT ANSWERPROBEdisagree by ±1.7
At the probe, the plausible models disagree by ±1.7 — yet the cyan line still answers “3.6” with full confidence. That silent overconfidence away from the data is the domain-shift problem.
UNDER THE MECHANISM / 2 OF 2

The integral nobody can do

What you just saw has a formula too. To predict at a new input x⋆x_\star, ask every surviving candidate and average their answers, weighted by belief — the posterior predictive:

p(y⋆∣x⋆,D)=∫p(y⋆∣x⋆,θ) p(θ∣D) dθp(y_\star\mid x_\star,\mathcal{D})=\int p(y_\star\mid x_\star,\theta)\,p(\theta\mid\mathcal{D})\,d\theta
01

Exact Bayes doesn’t scale

Nine beach spots: trivial. Lines: still closed-form. But a neural network has millions of weights. The candidate space becomes too vast to enumerate, and the integral above becomes hopeless. Every practical method is a different way of cheating. Ensembles train a few networks and treat them as candidates. MC dropout randomizes at test time. Variational inference fits a tractable stand-in by optimization.

02

Laplace: a Gaussian at the peak

The oldest cheat is the most elegant — and it is the one BayesVLM uses. Training already finds the most believable weights θMAP\theta_{\text{MAP}}. Taylor-expand the log-posterior there — the linear term vanishes at a maximum — and what is left is a quadratic, which is exactly a Gaussian:

p(θ∣D)  ≈  N ⁣(θMAP, H−1),H=−∇2log⁡p(θ∣D)∣θMAPp(\theta\mid\mathcal{D})\;\approx\;\mathcal{N}\!\left(\theta_{\text{MAP}},\ \mathbf{H}^{-1}\right),\qquad \mathbf{H}=-\nabla^2\log p(\theta\mid\mathcal{D})\big|_{\theta_{\text{MAP}}}

The Hessian H\mathbf{H} measures curvature. A sharp peak means the data pins the weights down (small variance). A flat one means many settings work equally well (large variance). Crucially, this is post-hoc — no retraining, just one curvature computation around weights you already have.

03

When is the cheat honest?

A Gaussian is symmetric. Real posteriors often aren’t. But posteriors sharpen and symmetrize as data accumulates. The Bernstein–von Mises theorem makes this precise. The experiment below lets you watch the approximation prove itself.

EXPERIMENT 03 / THE GAUSSIAN STAND-IN

What the cheap approximation gets wrong — literally

The gray shape is belief both curves agree on. Cyan is real belief the Gaussian ignores. Pink is belief the Gaussian invents. A perfect stand-in has no color left.

TRY THIS

Drag n down to 1 and the colored error regions bloom — a symmetric Gaussian can’t fake a lopsided posterior. Now drag to 60 and watch the color vanish. More data makes posteriors Gaussian, which is exactly why this cheap trick works on large pretrained models.

123456θ — the parameter being estimatedMISSEDINVENTED
MATCH
90.3%
ERROR
9.7%
At n = 3 the true posterior leans right, so the symmetric Gaussian misses 9.7% of the belief: it understates the long tail (cyan) and overstates the short one (pink).
CLOSING / WHAT TO TAKE WITH YOU

The width is the signal

One move, three scales: belief bars over a beach, a fan of lines over a dataset, Gaussian clouds over CLIP’s embedding space. In every case the answer comes with its own width — and the width, not the answer, is what tells a system when to dig, when to keep scanning, and when to stop and ask for help. For a robot acting in the physical world, that call is the one that matters. Honest beats smart more often than we like to admit.

Further reading: Post-hoc Probabilistic Vision-Language Models (BayesVLM) · Laplace Redux — effortless Bayesian deep learning · Neal, Bayesian Learning for Neural Networks