Normalizing Flows — Visual Experiments¶
Goal: Build a normalizing flow from scratch and watch the change-of-variables formula do real work — turn a plain Gaussian into a complex distribution, recover exact densities, sample, and then run into the wall that the theory warns about (the topology constraint).
A normalizing flow is a chain of invertible maps between a simple base distribution ($\mathcal{N}(0,I)$) and a complex one. Because every step is bijective with a tractable Jacobian, the flow gives something VAEs (only an ELBO bound) and GANs (no likelihood at all) cannot: exact density evaluation. We implement an actual RealNVP in PyTorch and train it.
What we'll explore¶
| # | Experiment | Question answered |
|---|---|---|
| 1 | Change of variables (1D) | How does the Jacobian reshape a density when we warp a variable? |
| 2 | Affine coupling mechanics | Why is a coupling layer invertible and cheap to differentiate? |
| 3 | Composition = expressivity | How does stacking simple layers bend a round Gaussian into complex shapes? |
| 4 | Train a RealNVP | Can a flow learn an exact density and sample from two moons? |
| 5 | The topology wall | Why can't a connected base perfectly model two disconnected modes? |
Linked theory: research/06-normalizing-flows.md · Chạy: uv run jupyter lab từ thư mục latent-anything-theory/ (cần các package trong pyproject.toml).
import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import norm
from sklearn.datasets import make_moons
import torch
import torch.nn as nn
import warnings
warnings.filterwarnings('ignore')
np.random.seed(42)
torch.manual_seed(42)
plt.rcParams.update({
'figure.dpi': 110,
'font.size': 11,
'axes.titlesize': 12,
'axes.labelsize': 11,
'figure.facecolor': 'white',
'axes.facecolor': '#f8f9fa',
'axes.grid': True,
'grid.alpha': 0.35,
})
print('Setup complete. torch', torch.__version__)
Setup complete. torch 2.12.0+cu130
Experiment 1: Change of variables in 1D — the Jacobian reshapes density¶
Question: The whole method rests on one formula: if $x = f(z)$ with $z\sim p_Z$ and $f$ invertible, then $p_X(x) = p_Z(z)\,/\,|f'(z)|$. What does the $1/|f'(z)|$ factor actually do to the shape of a density?
We take a standard Gaussian base, push it through a monotonic warp $f(z) = z + \tanh(z)$, and compare the analytic transformed density (via the formula) against a histogram of pushed samples. No training — just the formula.
np.random.seed(1)
def f(z):
return z + np.tanh(z) # monotonic -> invertible
def fprime(z):
return 1.0 + (1.0 - np.tanh(z) ** 2) # derivative > 0 everywhere
# parametric curve of the transformed density: (x, p_X) sampled along z
z_grid = np.linspace(-5, 5, 600)
x_grid = f(z_grid)
p_z = norm.pdf(z_grid)
p_x = p_z / fprime(z_grid) # change-of-variables formula
# empirical check: push Gaussian samples through f
z_samples = np.random.randn(40000)
x_samples = f(z_samples)
print(f'max |f\'| = {fprime(z_grid).max():.2f} (density compressed here)')
print(f'min |f\'| = {fprime(z_grid).min():.2f} (density stretched here)')
max |f'| = 2.00 (density compressed here) min |f'| = 1.00 (density stretched here)
fig, axes = plt.subplots(1, 3, figsize=(15.5, 4.6))
fig.suptitle('Experiment 1: a monotonic warp + the Jacobian = a new density',
fontsize=12.5, fontweight='bold')
ax = axes[0]
ax.plot(z_grid, p_z, color='#2980b9', lw=2.4)
ax.fill_between(z_grid, p_z, alpha=0.2, color='#2980b9')
ax.set_xlabel('z'); ax.set_ylabel('p_Z(z)')
ax.set_title('Base density N(0, 1)')
ax2 = axes[1]
ax2.plot(z_grid, x_grid, color='#16a085', lw=2.4, label='x = f(z) = z + tanh(z)')
ax2.plot(z_grid, fprime(z_grid), color='#e67e22', lw=2.0, ls='--', label="f'(z) (local stretch)")
ax2.set_xlabel('z'); ax2.set_ylabel('value')
ax2.set_title('The transform and its derivative')
ax2.legend(fontsize=9)
ax3 = axes[2]
ax3.hist(x_samples, bins=120, density=True, alpha=0.45, color='#7f8c8d', label='pushed samples')
ax3.plot(x_grid, p_x, color='#c0392b', lw=2.6, label='analytic p_X = p_Z / |f\'|')
ax3.set_xlabel('x'); ax3.set_ylabel('p_X(x)')
ax3.set_title('Transformed density: formula matches samples')
ax3.legend(fontsize=9)
plt.tight_layout()
plt.show()
What you see¶
What the plot shows: The base is a clean bell curve. The warp $f(z)=z+\tanh(z)$ is steepest near $z=0$ (its derivative peaks at $2$) and flattens to slope $1$ in the tails. The resulting density (right) is lower and wider in the middle and the analytic red curve lands exactly on the histogram of pushed samples.
Why it looks this way: Where $f$ stretches space (large $f'$, near the origin), the same probability mass is spread over a wider interval, so the density must drop — that is precisely the $1/|f'(z)|$ factor. The Jacobian is the bookkeeping that keeps the total probability equal to $1$ after the coordinate change. The perfect match confirms the change-of-variables formula is exact, not approximate.
Key takeaway: A flow sculpts a density purely by warping coordinates and dividing by the local volume change. Everything that follows — coupling layers, deep flows, training — is just a way to build a high-dimensional, learnable version of this single 1D move.
Experiment 2: Affine coupling mechanics — invertible and cheap¶
Question: A general invertible neural net is useless if its Jacobian determinant costs $O(D^3)$. RealNVP's trick is the affine coupling layer: keep half the coordinates fixed, and use them to scale-and-shift the other half. The research file claims this makes the layer (a) exactly invertible with arbitrary neural nets inside, and (b) gives a triangular Jacobian whose log-det is a simple sum. Let's verify both numerically.
We build one coupling layer with random weights, round-trip a batch through it, and inspect its Jacobian on a single point.
torch.manual_seed(2)
class Coupling(nn.Module):
"""RealNVP affine coupling. forward: x -> z (normalizing). inverse: z -> x (generative)."""
def __init__(self, dim=2, hidden=128, mask=None):
super().__init__()
self.register_buffer('mask', mask)
self.net = nn.Sequential(
nn.Linear(dim, hidden), nn.ReLU(),
nn.Linear(hidden, hidden), nn.ReLU(),
nn.Linear(hidden, dim * 2),
)
def _st(self, x):
h = self.net(self.mask * x)
s, t = h.chunk(2, dim=1)
s = torch.tanh(s) * (1 - self.mask) # bounded log-scale, only on transformed dims
t = t * (1 - self.mask)
return s, t
def forward(self, x): # x -> z
s, t = self._st(x)
z = self.mask * x + (1 - self.mask) * ((x - t) * torch.exp(-s))
log_det = (-s).sum(dim=1) # = sum of log|dz/dx| on transformed dims
return z, log_det
def inverse(self, z): # z -> x
s, t = self._st(z) # mask*z == mask*x, so s,t are identical
x = self.mask * z + (1 - self.mask) * (z * torch.exp(s) + t)
return x
mask = torch.tensor([1.0, 0.0]) # keep dim 0, transform dim 1
layer = Coupling(dim=2, mask=mask)
x = torch.randn(2000, 2)
z, ldj = layer(x)
x_rec = layer.inverse(z)
roundtrip_err = (x - x_rec).abs().max().item()
# analytic vs numerical Jacobian on one point
pt = torch.randn(1, 2)
J = torch.autograd.functional.jacobian(lambda v: layer(v)[0].squeeze(0), pt).squeeze()
logdet_numeric = torch.log(torch.abs(torch.det(J))).item()
logdet_analytic = layer(pt)[1].item()
print(f'round-trip max error x -> z -> x : {roundtrip_err:.2e} (machine precision)')
print(f'Jacobian matrix at the point:\n{J.numpy().round(4)}')
print(f'log|det J| analytic = {logdet_analytic:.4f} numerical = {logdet_numeric:.4f}')
round-trip max error x -> z -> x : 2.38e-07 (machine precision) Jacobian matrix at the point: [[ 1. 0. ] [-0.0173 1.1096]] log|det J| analytic = 0.1040 numerical = 0.1040
fig, axes = plt.subplots(1, 2, figsize=(13, 5.2))
fig.suptitle('Experiment 2: the coupling layer is a perfect, cheap bijection',
fontsize=12.5, fontweight='bold')
# --- left: the Jacobian heatmap (triangular) ---
ax = axes[0]
im = ax.imshow(J.numpy(), cmap='RdBu_r', vmin=-abs(J).max(), vmax=abs(J).max())
for (i, j), val in np.ndenumerate(J.numpy()):
ax.text(j, i, f'{val:.2f}', ha='center', va='center', fontsize=13, fontweight='bold')
ax.set_xticks([0, 1]); ax.set_yticks([0, 1])
ax.set_xticklabels(['dx0', 'dx1']); ax.set_yticklabels(['dz0', 'dz1'])
ax.set_title('Jacobian is lower-triangular\n(top-right = 0 => det = product of diagonal)')
ax.grid(False)
fig.colorbar(im, ax=ax, fraction=0.046, pad=0.04)
# --- right: round-trip overlay ---
ax2 = axes[1]
xn = x.numpy(); zn = z.detach().numpy(); rn = x_rec.detach().numpy()
ax2.scatter(xn[:, 0], xn[:, 1], s=8, alpha=0.4, color='#2980b9', label='input x')
ax2.scatter(zn[:, 0], zn[:, 1], s=8, alpha=0.4, color='#e67e22', label='transformed z')
ax2.scatter(rn[:, 0], rn[:, 1], s=4, alpha=0.4, color='#27ae60', label='recovered x (z->x)')
ax2.set_xlabel('dim 0'); ax2.set_ylabel('dim 1')
ax2.set_title(f'Inverse recovers the input exactly (err {roundtrip_err:.0e})')
ax2.legend(fontsize=9)
plt.tight_layout()
plt.show()
What you see¶
What the plot shows: The Jacobian on the left has a hard zero in the top-right corner and a $1$ in the top-left — it is lower-triangular. The analytic and numerical log-determinants agree to four decimals, and the round-trip $x\to z\to x$ returns the input to machine precision (green points sit exactly on the blue inputs on the right).
Why it looks this way: Dimension 0 is passed through untouched ($z_0=x_0$), so $\partial z_0/\partial x_1 = 0$ — that single structural zero makes the matrix triangular, and the determinant collapses to the product of the diagonal ($\exp$ of the summed scales). Invertibility is free because the inverse only needs to evaluate the scale/shift networks, never invert them — which is why $s$ and $t$ can be arbitrary ReLU MLPs.
Key takeaway: The coupling layer buys exact invertibility and an $O(D)$ log-determinant at the cost of only transforming half the coordinates per layer. Alternating which half is frozen (next experiment) lets a stack of these reach every dimension.
Experiment 3: Composition builds expressivity¶
Question: One coupling layer only shears the plane. The research file says rich distributions come from composing many bijections (with alternating masks so every dimension gets transformed). Before any training, how does a round Gaussian deform as we push it through more and more randomly-initialized layers?
We assemble a RealNVP of stacked couplings with random weights and run samples through the generative direction ($z\to x$), snapshotting after 0, 2, 4, and 8 layers.
torch.manual_seed(7)
class RealNVP(nn.Module):
def __init__(self, dim=2, hidden=128, n_layers=6):
super().__init__()
base_mask = torch.tensor([1.0, 0.0])
self.layers = nn.ModuleList([
Coupling(dim, hidden, base_mask if i % 2 == 0 else 1 - base_mask)
for i in range(n_layers)
])
def forward(self, x): # x -> z, accumulate log-det
log_det = torch.zeros(x.shape[0])
z = x
for layer in self.layers:
z, ld = layer(z)
log_det = log_det + ld
return z, log_det
def log_prob(self, x):
z, log_det = self.forward(x)
log_pz = -0.5 * (z ** 2 + np.log(2 * np.pi)).sum(dim=1)
return log_pz + log_det
def inverse(self, z, upto=None): # z -> x (optionally only first `upto` layers)
x = z
layers = list(reversed(self.layers))
if upto is not None:
layers = list(reversed(self.layers[:upto]))
for layer in layers:
x = layer.inverse(x)
return x
def sample(self, n):
return self.inverse(torch.randn(n, 2))
rnd_flow = RealNVP(n_layers=8)
z0 = torch.randn(3000, 2)
snapshots = {k: rnd_flow.inverse(z0, upto=k).detach().numpy() for k in [0, 2, 4, 8]}
print('Generated snapshots after', list(snapshots.keys()), 'random layers')
Generated snapshots after [0, 2, 4, 8] random layers
fig, axes = plt.subplots(1, 4, figsize=(16, 4.2))
fig.suptitle('Experiment 3: a Gaussian deforming through stacked (untrained) coupling layers',
fontsize=12.5, fontweight='bold')
for ax, k in zip(axes, [0, 2, 4, 8]):
pts = snapshots[k]
ax.scatter(pts[:, 0], pts[:, 1], s=5, alpha=0.4, color='#8e44ad')
ax.set_title(f'after {k} layers')
ax.set_xlabel('dim 0'); ax.set_ylabel('dim 1')
ax.set_aspect('equal')
plt.tight_layout()
plt.show()
What you see¶
What the plot shows: At 0 layers the samples are the original round Gaussian blob. Each pair of layers shears and curves it further; by 8 layers the cloud is a visibly warped, non-Gaussian shape — and these are random weights, no fitting at all.
Why it looks this way: Each coupling layer applies an axis-aligned, data-dependent scale-and-shift; alone it can only shear along one axis, but composing layers with alternating masks lets the shears compound into genuinely curved, anisotropic deformations. Crucially every step stays invertible, so the whole stack remains a bijection with a tractable density.
Key takeaway: Expressivity in a flow comes from depth, not from any single clever layer. Training simply chooses the weights of these layers so the deformation maps the base onto the data distribution — which is exactly Experiment 4.
Experiment 4: Train a RealNVP — exact density and sampling¶
Question: Now the payoff. Can the flow, trained by maximum likelihood (the loss is literally $-\mathbb{E}[\log p(x)]$ using the change-of-variables formula), learn the density of a two-moons dataset, evaluate it exactly on a grid, and generate new samples?
We train the stacked flow on two moons and then (a) plot the learned density as a heatmap and (b) draw fresh samples by pushing Gaussian noise through the inverse.
torch.manual_seed(0)
np.random.seed(0)
X, y_moons = make_moons(n_samples=3000, noise=0.07) # keep labels to identify the two moons
X = (X - X.mean(0)) / X.std(0) # standardize
X_t = torch.tensor(X, dtype=torch.float32)
flow = RealNVP(dim=2, hidden=128, n_layers=8)
opt = torch.optim.Adam(flow.parameters(), lr=2e-3)
n_steps = 1500
losses = []
for step in range(n_steps):
opt.zero_grad()
loss = -flow.log_prob(X_t).mean() # negative log-likelihood
loss.backward()
opt.step()
losses.append(loss.item())
if step % 300 == 0 or step == n_steps - 1:
print(f'step {step:4d} NLL = {loss.item():.3f}')
step 0 NLL = 2.950
step 300 NLL = 1.541
step 600 NLL = 1.511
step 900 NLL = 1.472
step 1200 NLL = 1.477
step 1499 NLL = 1.518
# evaluate learned density on a grid
gx, gy = np.mgrid[-2.5:2.5:200j, -2.5:2.5:200j]
grid = torch.tensor(np.stack([gx.ravel(), gy.ravel()], 1), dtype=torch.float32)
with torch.no_grad():
dens = torch.exp(flow.log_prob(grid)).numpy().reshape(gx.shape)
samples = flow.sample(3000).numpy()
fig, axes = plt.subplots(1, 3, figsize=(16, 5.0))
fig.suptitle('Experiment 4: a trained RealNVP learns the two-moons density and samples it',
fontsize=12.5, fontweight='bold')
ax = axes[0]
ax.plot(losses, color='#c0392b', lw=1.8)
ax.set_xlabel('training step'); ax.set_ylabel('negative log-likelihood')
ax.set_title('Training loss = exact NLL, decreasing')
ax2 = axes[1]
cf = ax2.contourf(gx, gy, dens, levels=20, cmap='viridis')
ax2.scatter(X[:, 0], X[:, 1], s=3, color='white', alpha=0.25)
ax2.set_xlabel('dim 0'); ax2.set_ylabel('dim 1')
ax2.set_title('Learned density p(x) (white = data)')
ax2.set_aspect('equal')
fig.colorbar(cf, ax=ax2, fraction=0.046, pad=0.04)
ax3 = axes[2]
ax3.scatter(samples[:, 0], samples[:, 1], s=5, alpha=0.4, color='#16a085')
ax3.set_xlabel('dim 0'); ax3.set_ylabel('dim 1')
ax3.set_title('Samples drawn via z ~ N(0,I) -> inverse flow')
ax3.set_aspect('equal')
plt.tight_layout()
plt.show()
What you see¶
What the plot shows: The negative log-likelihood drops steadily and flattens — the flow is fitting the data by directly maximizing exact likelihood. The middle heatmap is the learned density: two bright crescents tracing the data. The right panel shows fresh samples (Gaussian noise pushed backward through the flow) that reproduce the two-moons shape.
Why it looks this way: Unlike a VAE, there is no reconstruction term and no KL bound — the loss is $-\log p(x)$ computed via $\log p_Z(f(x)) + \log|\det J_f|$, so minimizing it directly maximizes the probability the model assigns to real data. Because the map is bijective, the same trained weights run backward to turn base samples into data samples; density evaluation and generation are two directions of one function.
Key takeaway: This is the unique selling point of flows: a single model that gives you an exact, normalized density and a generator. That exact density is what makes flows attractive for in/out-of-distribution scoring in a latent-inspection toolkit — with the caveat the next experiment exposes.
Experiment 5: The topology wall — a connected base can't tear¶
Question: The research file's deepest limitation: a flow is a homeomorphism, so it preserves topology. The base $\mathcal{N}(0,I)$ is one connected blob; the two moons are nearly two disconnected components. A continuous invertible map cannot tear one blob into two — so what compromise is the flow forced into?
We test connectivity directly: take one point from each moon, draw the straight segment between their latent images, and map that segment back to data space. Because the image of a connected set under a continuous map is connected, this curve must join the two moons — and we check that the density along it stays strictly positive.
with torch.no_grad():
z_data, _ = flow.forward(X_t)
z_data = z_data.numpy()
# one representative point from each moon
iA = np.where(y_moons == 0)[0][0]
iB = np.where(y_moons == 1)[0][0]
with torch.no_grad():
zA, _ = flow.forward(X_t[iA:iA + 1])
zB, _ = flow.forward(X_t[iB:iB + 1])
ts = torch.linspace(0, 1, 300).unsqueeze(1)
z_line = (1 - ts) * zA + ts * zB # connected straight segment in LATENT space
x_line_t = flow.inverse(z_line) # its image in DATA space -> a connected curve
path_dens = torch.exp(flow.log_prob(x_line_t)).numpy()
x_line = x_line_t.numpy()
bridge_min = path_dens.min()
print(f'latent: mean {z_data.mean(0).round(3)}, std {z_data.std(0).round(3)} (~ N(0,I))')
print(f'density along the bridging curve: peak = {path_dens.max():.3f}')
print(f'minimum on the bridge = {bridge_min:.3f} (dips, but strictly > 0 -> modes stay connected)')
latent: mean [ 0.018 -0.024], std [0.941 1.013] (~ N(0,I)) density along the bridging curve: peak = 0.328 minimum on the bridge = 0.077 (dips, but strictly > 0 -> modes stay connected)
fig, axes = plt.subplots(1, 3, figsize=(16, 5.0))
fig.suptitle('Experiment 5: a connected latent segment maps to a connected curve bridging the moons',
fontsize=12.5, fontweight='bold')
# --- left: data mapped to latent, with the straight latent segment ---
ax = axes[0]
th = np.linspace(0, 2 * np.pi, 100)
ax.scatter(z_data[:, 0], z_data[:, 1], s=5, alpha=0.35, color='#bdc3c7')
for r in (1, 2):
ax.plot(r * np.cos(th), r * np.sin(th), color='#c0392b', lw=1.0, ls='--')
zl = z_line.numpy()
ax.plot(zl[:, 0], zl[:, 1], color='#8e44ad', lw=2.6, label='straight segment in latent')
ax.scatter([zA[0, 0]], [zA[0, 1]], color='#2980b9', s=70, zorder=5, label='moon A')
ax.scatter([zB[0, 0]], [zB[0, 1]], color='#e67e22', s=70, zorder=5, label='moon B')
ax.set_xlabel('latent 0'); ax.set_ylabel('latent 1')
ax.set_title('Latent space (~ Gaussian)')
ax.set_aspect('equal'); ax.legend(fontsize=8.5, loc='upper left')
# --- middle: the curve mapped back into data space ---
ax2 = axes[1]
cf = ax2.contourf(gx, gy, dens, levels=20, cmap='viridis')
ax2.scatter(X[y_moons == 0, 0], X[y_moons == 0, 1], s=3, color='#5dade2', alpha=0.4)
ax2.scatter(X[y_moons == 1, 0], X[y_moons == 1, 1], s=3, color='#f5b041', alpha=0.4)
ax2.plot(x_line[:, 0], x_line[:, 1], color='#e74c3c', lw=2.8, label='image of the segment')
ax2.scatter([X[iA, 0]], [X[iA, 1]], color='#2980b9', s=70, zorder=5)
ax2.scatter([X[iB, 0]], [X[iB, 1]], color='#e67e22', s=70, zorder=5)
ax2.set_xlabel('dim 0'); ax2.set_ylabel('dim 1')
ax2.set_title('Data space: the curve joins the two moons')
ax2.set_aspect('equal'); ax2.legend(fontsize=9, loc='upper right')
fig.colorbar(cf, ax=ax2, fraction=0.046, pad=0.04)
# --- right: density along the bridging curve ---
ax3 = axes[2]
tt = ts.numpy().ravel()
ax3.plot(tt, path_dens, color='#8e44ad', lw=2.4)
ax3.fill_between(tt, path_dens, alpha=0.2, color='#8e44ad')
ax3.axhline(bridge_min, color='gray', ls=':', lw=1.4, label=f'bridge floor = {bridge_min:.3f} (> 0)')
ax3.set_ylim(0, None)
ax3.set_xlabel('position along the segment (A -> B)')
ax3.set_ylabel('density p(x)')
ax3.set_title('Density dips through the gap but stays strictly positive')
ax3.legend(fontsize=9)
plt.tight_layout()
plt.show()
What you see¶
What the plot shows: A straight segment between the two moons' latent images (left, purple) maps back to a continuous red curve in data space (middle) that runs from moon A to moon B, joining the two components. The density along that curve (right) dips through the lower-density region between the moons but stays strictly positive — it never touches zero.
Why it looks this way: The flow pushes the everywhere-positive Gaussian through a diffeomorphism, which maps a connected set to a connected set and a strictly-positive density to a strictly-positive density. So the connected latent segment must have a connected image, and that image must carry nonzero probability everywhere. The flow can make such an inter-mode bridge thin, but it cannot sever it — tearing the support into two truly separate pieces would require a discontinuous, non-invertible map.
Key takeaway: This topological rigidity is the headline limitation. It is also why naive likelihood-based OOD detection with flows is unreliable — Kirichenko et al. (NeurIPS 2020) even show flows assigning higher likelihood to some out-of-distribution inputs, because the model spends mass on bridges and low-level statistics it cannot avoid. For Latent-Anything: flows give exact densities, but "high density" is not the same as "in-distribution," so latent OOD scoring needs more than a raw likelihood threshold.
Summary¶
| Experiment | Theme | Key insight | |---|---|---| | 1. Change of variables | Foundation | A density is reshaped by warping coordinates and dividing by the Jacobian $|f'|$ — exactly | | 2. Coupling mechanics | Engineering | Freezing half the dims gives a triangular Jacobian + free inverse, so $s,t$ can be any net | | 3. Composition | Expressivity | Depth, not any single layer, bends a Gaussian into complex shapes while staying bijective | | 4. Trained RealNVP | The payoff | Maximum-likelihood training yields an exact density and a generator from one model | | 5. Topology wall | Limitation | A homeomorphism can't tear a connected base into disconnected modes -> low-density bridges, fragile OOD |
Connection to the broader theory¶
Normalizing flows take the change-of-variables formula (Experiment 1) and make it deep and learnable through coupling layers whose triangular Jacobian keeps everything tractable (Experiments 2-3). Trained by exact maximum likelihood, a flow becomes a rare two-in-one object: an exact density estimator and a sampler (Experiment 4) — which is precisely why Tier 4 of the theory index lists flows as a tool for density estimation in latent space and in/out-of-distribution detection for Layer A. But the topology constraint (Experiment 5) is the boundary: flows preserve connectedness, cannot compress dimension, and high likelihood does not guarantee in-distribution. They complement the geometry of this section — where Slerp and pullback geodesics move through a curved latent space, flows re-coordinatize it into a flat Gaussian base where straight-line operations regain meaning, echoing the flattening idea of FlatVI from Section 1. This closes Tier 3: with linear structure, disentanglement, isotropy, Riemannian geometry, Slerp, and flows in hand, the LatentSpace primitive can be designed on a firm geometric footing.