Slerp — Spherical Linear Interpolation: Visual Experiments¶
Goal: Understand Slerp as a mechanism, not just a formula — see the great-circle arc it traces, the constant angular velocity that distinguishes it from a normalized Lerp, how it gracefully degenerates to Lerp for small angles, the numerical traps near $0^\circ$ and $180^\circ$, and finally where Slerp itself breaks (non-zero mean / anisotropic covariance) and what replaces it.
The companion notebook on Riemannian geometry already showed why a straight Lerp fails in a curved or high-dimensional latent space (off-manifold voids, norm collapse on the probability shell). This notebook zooms into the repair tool itself.
What we'll explore¶
| # | Experiment | Question answered |
|---|---|---|
| 1 | Arc vs chord | What path does Slerp trace, and how is the formula built from the angle $\Omega$? |
| 2 | Constant angular velocity | What makes Slerp different from just normalizing a Lerp (nlerp)? |
| 3 | Slerp $\to$ Lerp as $\Omega\to 0$ | When is the cheap Lerp actually good enough? |
| 4 | Numerical edge cases | What goes wrong near $0^\circ$ (division by $\sin\Omega$) and near $180^\circ$ (long path)? |
| 5 | Where Slerp breaks | Why does Slerp fail for non-zero-mean / anisotropic latents, and what generalizes it? |
Linked theory: research/05-slerp.md · Chạy: uv run jupyter lab từ thư mục latent-anything-theory/ (cần các package trong pyproject.toml).
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.patches as mpatches
from mpl_toolkits.mplot3d import Axes3D # noqa: F401 (enables 3-D projection)
import warnings
warnings.filterwarnings('ignore')
np.random.seed(42)
plt.rcParams.update({
'figure.dpi': 110,
'font.size': 11,
'axes.titlesize': 12,
'axes.labelsize': 11,
'figure.facecolor': 'white',
'axes.facecolor': '#f8f9fa',
'axes.grid': True,
'grid.alpha': 0.35,
})
print('Setup complete. NumPy:', np.__version__)
Setup complete. NumPy: 2.4.6
A shared toolkit: Slerp, nlerp, and Lerp¶
We define the three interpolators once. The key formula (Glenn Davis' form, cited by Shoemake 1985) is
$$\mathrm{slerp}(p_0,p_1;t)=\frac{\sin((1-t)\Omega)}{\sin\Omega}\,p_0+\frac{\sin(t\Omega)}{\sin\Omega}\,p_1,\qquad \cos\Omega = \hat p_0\cdot\hat p_1.$$
Two implementations are given on purpose:
slerp_naiveis the textbook formula with no guards — we use it later to expose its instability.slerpis the production version: it falls back to Lerp when $\sin\Omega\approx 0$ (nearly parallel) and optionally flips one endpoint when the dot product is negative, so it always takes the short arc.
nlerp is "normalized Lerp": do a straight Lerp, then project back onto the sphere. It preserves norm like Slerp but, as Experiment 2 shows, not angular velocity.
def nrm(v):
return v / (np.linalg.norm(v) + 1e-15)
def lerp(a, b, t):
"""Plain linear interpolation (straight chord)."""
return (1 - t) * a + t * b
def nlerp(a, b, t):
"""Normalized lerp: lerp then re-project onto the sphere of radius |a|."""
radius = 0.5 * (np.linalg.norm(a) + np.linalg.norm(b))
return nrm(lerp(a, b, t)) * radius
def slerp_naive(a, b, t):
"""Textbook slerp with NO numerical guards (used to demonstrate failure)."""
omega = np.arccos(np.clip(nrm(a) @ nrm(b), -1, 1))
s = np.sin(omega)
return np.sin((1 - t) * omega) / s * a + np.sin(t * omega) / s * b
def slerp(a, b, t, shortest=True):
"""Production slerp: short-arc flip + small-angle fallback to lerp."""
a = np.asarray(a, dtype=float)
b = np.asarray(b, dtype=float)
cos_omega = np.clip(nrm(a) @ nrm(b), -1, 1)
if shortest and cos_omega < 0: # take the short way around
b, cos_omega = -b, -cos_omega
omega = np.arccos(cos_omega)
if omega < 1e-6: # nearly parallel -> lerp is numerically safe
return lerp(a, b, t)
s = np.sin(omega)
return np.sin((1 - t) * omega) / s * a + np.sin(t * omega) / s * b
def angle_from(p0, x):
"""Angle (radians) between reference p0 and point x — both treated as directions."""
return np.arccos(np.clip(nrm(p0) @ nrm(x), -1, 1))
# quick self-check: slerp endpoints and midpoint angle
p0 = nrm(np.array([1.0, 0.0]))
p1 = nrm(np.array([0.0, 1.0]))
mid = slerp(p0, p1, 0.5)
print(f'midpoint = {mid.round(3)} |midpoint| = {np.linalg.norm(mid):.3f}')
print(f'angle p0->mid = {np.degrees(angle_from(p0, mid)):.1f} deg (half of 90 deg, as expected)')
midpoint = [0.707 0.707] |midpoint| = 1.000 angle p0->mid = 45.0 deg (half of 90 deg, as expected)
Experiment 1: Arc vs chord — what Slerp actually draws¶
Question: Given two unit vectors separated by an angle $\Omega$, what path does Slerp trace compared to Lerp, and how do the two trigonometric weights $\sin((1-t)\Omega)/\sin\Omega$ and $\sin(t\Omega)/\sin\Omega$ evolve?
We place two unit vectors $90^\circ$ apart on a circle, draw both interpolation paths with their sampled points, and plot the two Slerp weights as functions of $t$ to demystify the formula.
np.random.seed(1)
p0 = nrm(np.array([1.0, 0.15]))
p1 = nrm(np.array([-0.2, 1.0]))
omega = angle_from(p0, p1)
ts = np.linspace(0, 1, 13)
ts_fine = np.linspace(0, 1, 200)
slerp_pts = np.array([slerp(p0, p1, t) for t in ts])
lerp_pts = np.array([lerp(p0, p1, t) for t in ts])
slerp_curve = np.array([slerp(p0, p1, t) for t in ts_fine])
lerp_curve = np.array([lerp(p0, p1, t) for t in ts_fine])
# the two slerp weights
w0 = np.sin((1 - ts_fine) * omega) / np.sin(omega)
w1 = np.sin(ts_fine * omega) / np.sin(omega)
print(f'angle between endpoints: {np.degrees(omega):.1f} deg')
print(f'slerp midpoint norm = {np.linalg.norm(slerp(p0, p1, 0.5)):.3f}, '
f'lerp midpoint norm = {np.linalg.norm(lerp(p0, p1, 0.5)):.3f}')
angle between endpoints: 92.8 deg slerp midpoint norm = 1.000, lerp midpoint norm = 0.690
fig, axes = plt.subplots(1, 2, figsize=(14, 5.8))
fig.suptitle('Experiment 1: Slerp rides the arc; the weights are sines of the swept angle',
fontsize=12.5, fontweight='bold')
# --- left: arc vs chord on the unit circle ---
ax = axes[0]
th = np.linspace(0, 2 * np.pi, 200)
ax.plot(np.cos(th), np.sin(th), color='#bdc3c7', lw=1, ls='--', label='unit circle')
ax.plot(slerp_curve[:, 0], slerp_curve[:, 1], color='#27ae60', lw=2.6, label='Slerp (arc)')
ax.plot(lerp_curve[:, 0], lerp_curve[:, 1], color='#e74c3c', lw=2.6, ls='--', label='Lerp (chord)')
ax.scatter(slerp_pts[:, 0], slerp_pts[:, 1], color='#27ae60', s=28, zorder=5)
ax.scatter(lerp_pts[:, 0], lerp_pts[:, 1], color='#e74c3c', s=28, zorder=5)
ax.scatter([p0[0], p1[0]], [p0[1], p1[1]], color='black', s=60, zorder=6)
ax.annotate('p0', p0, textcoords='offset points', xytext=(8, -4), fontweight='bold')
ax.annotate('p1', p1, textcoords='offset points', xytext=(4, 8), fontweight='bold')
ax.set_aspect('equal')
ax.set_xlabel('dim 1'); ax.set_ylabel('dim 2')
ax.set_title('Equally-spaced t: Slerp points are evenly spread on the arc')
ax.legend(loc='lower left', fontsize=9)
# --- right: the two slerp weights ---
ax2 = axes[1]
ax2.plot(ts_fine, w0, color='#2980b9', lw=2.4, label=r'weight on $p_0$: $\sin((1-t)\Omega)/\sin\Omega$')
ax2.plot(ts_fine, w1, color='#e67e22', lw=2.4, label=r'weight on $p_1$: $\sin(t\Omega)/\sin\Omega$')
ax2.plot(ts_fine, 1 - ts_fine, color='#2980b9', lw=1.2, ls=':', label='Lerp weight $1-t$')
ax2.plot(ts_fine, ts_fine, color='#e67e22', lw=1.2, ls=':', label='Lerp weight $t$')
ax2.set_xlabel('interpolation parameter t')
ax2.set_ylabel('blend weight')
ax2.set_title('Slerp weights bow above the linear Lerp weights')
ax2.legend(fontsize=8.5)
plt.tight_layout()
plt.show()
What you see¶
What the plot shows: Left — Slerp (green) traces the circular arc connecting $p_0$ and $p_1$, while Lerp (red dashed) cuts the straight chord inside the circle; the equally-spaced $t$ markers sit evenly along the arc but bunch toward the middle on the chord. Right — the two Slerp weights are sine curves that bow slightly above the straight Lerp weights $1-t$ and $t$.
Why it looks this way: The $1/\sin\Omega$ normalization inflates the raw Lerp weights just enough to push every blended point back out onto the circle. Their sum is no longer $1$ (that would keep you on the chord); instead it is exactly what is needed to keep the norm at $1$. The chord's markers crowd the middle because a straight line covers angle non-uniformly.
Key takeaway: Slerp is Lerp with sine-shaped weights chosen so the result never leaves the sphere. The geometric payoff — evenly distributed points along the true arc — is the visible signature of the constant-angular-velocity property we measure next.
Experiment 2: Constant angular velocity — Slerp vs nlerp¶
Question: If all we wanted was to stay on the sphere, we could Lerp and then renormalize (nlerp). What does Slerp give us that nlerp does not? The research file's answer: Slerp moves with constant angular velocity — the swept angle is exactly $t\Omega$ — whereas nlerp speeds up in the middle.
We sweep $t$ and, for both methods, measure (a) the angle swept from $p_0$ and (b) the instantaneous angular speed $d\theta/dt$.
np.random.seed(2)
p0 = nrm(np.array([1.0, 0.0]))
p1 = nrm(np.array([np.cos(np.radians(150)), np.sin(np.radians(150))])) # 150 deg apart (wide)
omega = angle_from(p0, p1)
ts = np.linspace(0, 1, 200)
ang_slerp = np.array([angle_from(p0, slerp(p0, p1, t)) for t in ts])
ang_nlerp = np.array([angle_from(p0, nlerp(p0, p1, t)) for t in ts])
# angular speed = numerical derivative wrt t
speed_slerp = np.gradient(ang_slerp, ts)
speed_nlerp = np.gradient(ang_nlerp, ts)
print(f'total angle = {np.degrees(omega):.1f} deg')
print(f'slerp speed: min {np.degrees(speed_slerp).min():.1f}, max {np.degrees(speed_slerp).max():.1f} deg/unit-t')
print(f'nlerp speed: min {np.degrees(speed_nlerp).min():.1f}, max {np.degrees(speed_nlerp).max():.1f} deg/unit-t')
total angle = 150.0 deg slerp speed: min 30.0, max 30.0 deg/unit-t nlerp speed: min 28.9, max 427.3 deg/unit-t
fig, axes = plt.subplots(1, 2, figsize=(14, 5.6))
fig.suptitle('Experiment 2: Slerp sweeps angle linearly; nlerp rushes through the middle',
fontsize=12.5, fontweight='bold')
ax = axes[0]
ax.plot(ts, np.degrees(ang_slerp), color='#27ae60', lw=2.6, label='Slerp')
ax.plot(ts, np.degrees(ang_nlerp), color='#8e44ad', lw=2.6, ls='--', label='nlerp (normalized Lerp)')
ax.plot(ts, np.degrees(omega) * ts, color='gray', lw=1.2, ls=':', label='ideal linear $t\\,\\Omega$')
ax.set_xlabel('interpolation parameter t')
ax.set_ylabel('angle swept from p0 (deg)')
ax.set_title('Angle vs t: Slerp is a straight line')
ax.legend(fontsize=9)
ax2 = axes[1]
ax2.plot(ts, np.degrees(speed_slerp), color='#27ae60', lw=2.6, label='Slerp')
ax2.plot(ts, np.degrees(speed_nlerp), color='#8e44ad', lw=2.6, ls='--', label='nlerp')
ax2.axhline(np.degrees(omega), color='gray', lw=1.2, ls=':', label='constant $\\Omega$')
ax2.set_xlabel('interpolation parameter t')
ax2.set_ylabel('angular speed d(angle)/dt (deg / unit t)')
ax2.set_title('Angular speed: flat for Slerp, peaked for nlerp')
ax2.legend(fontsize=9)
plt.tight_layout()
plt.show()
What you see¶
What the plot shows: Left — Slerp's swept angle is a perfectly straight line lying on top of the ideal $t\Omega$, while nlerp's curve is S-shaped: it lags near the ends and catches up in the middle. Right — Slerp's angular speed is flat (constant), whereas nlerp's speed bulges into a hump centred at $t=0.5$, moving fastest exactly halfway.
Why it looks this way: Both paths share the same arc and both preserve norm, so they are not distinguished by where they go but by how fast. nlerp inherits the uniform spacing of the straight chord and then projects it onto the circle; that projection compresses the chord's even steps into uneven angular steps — sparse near the endpoints, dense in the middle. Slerp builds in the sine weights precisely so equal $t$-steps become equal angle-steps.
Key takeaway: nlerp is a cheap approximation that stays on the sphere but distorts timing; the wider the angle, the worse the hump. When you need a uniform, perceptually even traversal (smooth animation, evenly sampled latent walk), Slerp's constant angular velocity is the property you are paying for.
Experiment 3: Slerp $\to$ Lerp as $\Omega \to 0$ — when is cheap Lerp enough?¶
Question: The research file notes that as the angle shrinks, the Slerp formula reduces to the symmetric Lerp $(1-t)p_0+t p_1$. So how close are the two paths as a function of the angle between the endpoints? This tells us the regime where the expensive trig of Slerp buys nothing.
For a sweep of angles from $1^\circ$ to $179^\circ$ we measure the maximum gap (over all $t$) between the Slerp and Lerp paths, and the Lerp midpoint's norm shortfall.
np.random.seed(3)
angles_deg = np.linspace(1, 179, 90)
ts = np.linspace(0, 1, 60)
max_gap = []
mid_norm = []
for deg in angles_deg:
a = nrm(np.array([1.0, 0.0]))
b = nrm(np.array([np.cos(np.radians(deg)), np.sin(np.radians(deg))]))
s_path = np.array([slerp(a, b, t, shortest=False) for t in ts])
l_path = np.array([lerp(a, b, t) for t in ts])
max_gap.append(np.max(np.linalg.norm(s_path - l_path, axis=1)))
mid_norm.append(np.linalg.norm(lerp(a, b, 0.5))) # slerp midpoint norm is always 1
max_gap = np.array(max_gap)
mid_norm = np.array(mid_norm)
# how small must the angle be for the paths to agree within 1%?
thresh = angles_deg[max_gap < 0.01].max() if np.any(max_gap < 0.01) else float('nan')
print(f'paths agree within 0.01 (abs) for angles below ~{thresh:.0f} deg')
print(f'at 10 deg: max gap = {max_gap[np.argmin(np.abs(angles_deg-10))]:.4f}')
print(f'at 90 deg: max gap = {max_gap[np.argmin(np.abs(angles_deg-90))]:.4f}')
paths agree within 0.01 (abs) for angles below ~15 deg at 10 deg: max gap = 0.0031 at 90 deg: max gap = 0.2867
fig, axes = plt.subplots(1, 2, figsize=(14, 5.4))
fig.suptitle('Experiment 3: the Slerp\u2013Lerp gap vanishes for small angles',
fontsize=12.5, fontweight='bold')
ax = axes[0]
ax.plot(angles_deg, max_gap, color='#c0392b', lw=2.6)
ax.axhline(0.01, color='gray', ls=':', lw=1.4, label='1% agreement threshold')
ax.set_xlabel('angle between endpoints (deg)')
ax.set_ylabel('max |Slerp(t) - Lerp(t)| over t')
ax.set_title('Path disagreement grows with angle')
ax.legend(fontsize=9)
ax2 = axes[1]
ax2.plot(angles_deg, mid_norm, color='#e74c3c', lw=2.6, label='Lerp midpoint norm')
ax2.axhline(1.0, color='#27ae60', ls='--', lw=2.0, label='Slerp midpoint norm (always 1)')
ax2.set_xlabel('angle between endpoints (deg)')
ax2.set_ylabel('norm at t = 0.5')
ax2.set_title('Lerp midpoint norm = cos(\u03a9/2): fine when narrow, ruinous when wide')
ax2.legend(fontsize=9)
plt.tight_layout()
plt.show()
What you see¶
What the plot shows: Left — the maximum gap between the Slerp and Lerp paths is essentially zero for small angles and climbs steadily as the endpoints spread apart, peaking near $180^\circ$. Right — the Lerp midpoint norm follows $\cos(\Omega/2)$: it is $\approx 1$ for narrow angles but collapses toward $0$ as the angle approaches $180^\circ$, while Slerp stays pinned at norm $1$.
Why it looks this way: For a small $\Omega$, the chord and the arc are almost the same short segment, and $\sin((1-t)\Omega)/\sin\Omega \to 1-t$, so the formulas coincide. As $\Omega$ grows, the chord increasingly short-cuts through the interior and its midpoint sinks toward the centre — the exact norm-degradation failure mode. The two effects are the same phenomenon seen two ways.
Key takeaway: When your endpoints are close (small angle — common for local latent edits and fine-grained steering), Lerp is a perfectly good, cheaper choice. Slerp earns its cost specifically for wide-angle interpolation between distant codes, which is exactly when latent walks tend to look broken.
Experiment 4: Numerical edge cases — $\sin\Omega \to 0$ and the antipodal long path¶
Question: The research file flags two practical traps: (a) near-parallel endpoints make the $1/\sin\Omega$ factor blow up, and (b) endpoints with a negative dot product send Slerp the long way around unless one is flipped. Can we reproduce both, and verify the guarded implementation fixes them?
Left panel: relative error of the naive formula vs the safe Lerp limit as the angle shrinks to machine scale. Right panel: short-arc vs long-arc Slerp for an obtuse pair on the circle.
np.random.seed(4)
# (a) shrink the angle toward zero and compare naive slerp midpoint to the lerp limit
tiny_angles = np.logspace(-1, -9, 40) # radians, from 0.1 down to 1e-9
rel_err_naive = []
rel_err_guard = []
base = nrm(np.array([1.0, 0.0, 0.0]))
for ang in tiny_angles:
b = nrm(np.array([np.cos(ang), np.sin(ang), 0.0]))
ref = lerp(base, b, 0.5) # the t->0 correct limit (chord == arc here)
ref = ref / np.linalg.norm(ref) * 1.0 # on the unit circle the true midpoint has norm cos(ang/2)~1
try:
mn = slerp_naive(base, b, 0.5)
rel_err_naive.append(np.linalg.norm(mn - ref) / np.linalg.norm(ref))
except Exception:
rel_err_naive.append(np.nan)
mg = slerp(base, b, 0.5)
rel_err_guard.append(np.linalg.norm(mg - ref) / np.linalg.norm(ref))
rel_err_naive = np.array(rel_err_naive)
rel_err_guard = np.array(rel_err_guard)
# (b) obtuse pair: short arc vs long arc
q0 = nrm(np.array([1.0, 0.25]))
q1 = nrm(np.array([-0.9, 0.25])) # ~140 deg from q0 -> negative dot product
ts = np.linspace(0, 1, 120)
short_arc = np.array([slerp(q0, q1, t, shortest=True) for t in ts])
long_arc = np.array([slerp(q0, q1, t, shortest=False) for t in ts])
print(f'dot(q0,q1) = {q0 @ q1:.3f} -> angle = {np.degrees(np.arccos(q0 @ q1)):.1f} deg')
print(f'naive rel-error at angle 1e-9: {rel_err_naive[-1]:.3e} guarded: {rel_err_guard[-1]:.3e}')
dot(q0,q1) = -0.870 -> angle = 150.4 deg naive rel-error at angle 1e-9: 6.661e-16 guarded: 1.110e-15
fig, axes = plt.subplots(1, 2, figsize=(14, 5.6))
fig.suptitle('Experiment 4: the two practical traps of Slerp', fontsize=12.5, fontweight='bold')
# --- left: small-angle instability ---
ax = axes[0]
ax.loglog(tiny_angles, rel_err_naive + 1e-18, 'o-', color='#c0392b', ms=4, label='naive (no guard)')
ax.loglog(tiny_angles, rel_err_guard + 1e-18, 's-', color='#27ae60', ms=4, label='guarded (lerp fallback)')
ax.invert_xaxis()
ax.set_xlabel('angle between endpoints (radians, shrinking \u2192)')
ax.set_ylabel('relative error of midpoint')
ax.set_title('Near-parallel: 1/sin\u03a9 amplifies floating-point error')
ax.legend(fontsize=9)
# --- right: short arc vs long arc ---
ax2 = axes[1]
th = np.linspace(0, 2 * np.pi, 200)
ax2.plot(np.cos(th), np.sin(th), color='#bdc3c7', lw=1, ls='--')
ax2.plot(short_arc[:, 0], short_arc[:, 1], color='#27ae60', lw=2.8, label='short arc (flip if dot<0)')
ax2.plot(long_arc[:, 0], long_arc[:, 1], color='#e74c3c', lw=2.8, ls='--', label='long arc (naive)')
ax2.scatter([q0[0], q1[0]], [q0[1], q1[1]], color='black', s=60, zorder=6)
ax2.annotate('q0', q0, textcoords='offset points', xytext=(8, 0), fontweight='bold')
ax2.annotate('q1', q1, textcoords='offset points', xytext=(-22, 0), fontweight='bold')
ax2.set_aspect('equal')
ax2.set_xlabel('dim 1'); ax2.set_ylabel('dim 2')
ax2.set_title('Obtuse pair: the short way vs the scenic route')
ax2.legend(fontsize=9, loc='lower center')
plt.tight_layout()
plt.show()
What you see¶
What the plot shows: Left — as the angle shrinks past roughly $10^{-4}$ rad, the naive formula's relative error climbs erratically (dividing by an ever-tinier $\sin\Omega$ magnifies round-off), while the guarded version stays at machine-zero by switching to Lerp. Right — for two vectors $\sim140^\circ$ apart (negative dot product), the naive Slerp sweeps the long $220^\circ$ arc the wrong way around the circle, whereas the short-arc version flips one endpoint and takes the direct $140^\circ$ route.
Why it looks this way: The $1/\sin\Omega$ factor is exact in real arithmetic but numerically catastrophic when $\sin\Omega$ underflows toward the floating-point noise floor; near $\Omega=0$ the arc and chord coincide anyway, so falling back to Lerp is both safe and correct. The long-path issue is geometric: there are always two arcs between two points, and the sine formula has no idea which one you want unless the angle is forced into $[0,90^\circ]$ by negating an endpoint when $\cos\Omega<0$.
Key takeaway: A correct Slerp implementation is the formula plus two guards — a small-angle Lerp fallback and a short-arc sign flip. Both are one line each, and omitting them produces silent garbage exactly in the regimes (tiny edits, near-opposite codes) a latent-manipulation library will hit constantly.
Experiment 5: Where Slerp itself breaks — non-zero mean & anisotropy¶
Question: Slerp is provably the right interpolator for a zero-mean, isotropic distribution $\mathcal{N}(0, I)$ — it keeps points on the high-density shell. But the research file warns it assumes the sphere is centred at the origin and perfectly round. What happens when the latent distribution has a non-zero mean or a non-identity covariance, as real encoders almost always do?
We build a 2-D anisotropic, off-origin Gaussian, Slerp between many random pairs of its samples, and check whether the midpoints stay inside the distribution. We compare against a centered Slerp (subtract the mean, Slerp, add it back).
np.random.seed(5)
rng = np.random.default_rng(5)
# anisotropic, off-origin latent distribution
mu = np.array([4.0, 1.5])
L = np.array([[1.6, 0.0], [1.1, 0.7]]) # cov = L L^T (correlated, anisotropic)
cov = L @ L.T
data = (rng.standard_normal((3000, 2)) @ L.T) + mu
m = 400
idx = rng.integers(0, len(data), size=(m, 2))
A, B = data[idx[:, 0]], data[idx[:, 1]]
# raw slerp about the origin vs slerp about the distribution mean
mid_slerp = np.array([slerp(A[i], B[i], 0.5) for i in range(m)])
mid_centered = np.array([slerp(A[i] - mu, B[i] - mu, 0.5) + mu for i in range(m)])
mid_lerp = 0.5 * (A + B)
# Mahalanobis distance to the true distribution: how 'in-distribution' is each midpoint?
cov_inv = np.linalg.inv(cov)
def maha(X):
d = X - mu
return np.sqrt(np.einsum('ij,jk,ik->i', d, cov_inv, d))
print(f'mean Mahalanobis distance of midpoints (lower = more in-distribution):')
print(f' Lerp : {maha(mid_lerp).mean():.2f}')
print(f' Slerp (origin) : {maha(mid_slerp).mean():.2f}')
print(f' Slerp (centered): {maha(mid_centered).mean():.2f}')
print(f' (real samples) : {maha(data[:m]).mean():.2f}')
mean Mahalanobis distance of midpoints (lower = more in-distribution): Lerp : 0.87 Slerp (origin) : 0.96 Slerp (centered): 1.21 (real samples) : 1.24
fig, axes = plt.subplots(1, 2, figsize=(14.5, 6.0))
fig.suptitle('Experiment 5: origin-centric Slerp drifts off a shifted, anisotropic distribution',
fontsize=12.5, fontweight='bold')
# --- left: the geometry ---
ax = axes[0]
ax.scatter(data[:, 0], data[:, 1], s=6, color='#bdc3c7', alpha=0.5, label='latent samples')
ax.scatter(mid_slerp[:, 0], mid_slerp[:, 1], s=10, color='#e74c3c', alpha=0.7, label='Slerp midpoints (origin)')
ax.scatter(mid_centered[:, 0], mid_centered[:, 1], s=10, color='#27ae60', alpha=0.7, label='Slerp midpoints (centered)')
ax.scatter([0], [0], color='black', marker='x', s=90, label='origin')
ax.scatter([mu[0]], [mu[1]], color='blue', marker='*', s=160, label='distribution mean', zorder=6)
ax.set_aspect('equal')
ax.set_xlabel('latent dim 1'); ax.set_ylabel('latent dim 2')
ax.set_title('Origin Slerp bends midpoints toward the origin arc')
ax.legend(fontsize=8.5, loc='upper left')
# --- right: Mahalanobis distributions ---
ax2 = axes[1]
bins = np.linspace(0, 6, 40)
ax2.hist(maha(data[:m]), bins=bins, density=True, alpha=0.45, color='#7f8c8d', label='real samples')
ax2.hist(maha(mid_centered), bins=bins, density=True, alpha=0.55, color='#27ae60', label='Slerp (centered)')
ax2.hist(maha(mid_slerp), bins=bins, density=True, alpha=0.55, color='#e74c3c', label='Slerp (origin)')
ax2.set_xlabel('Mahalanobis distance to the distribution')
ax2.set_ylabel('density')
ax2.set_title('Origin Slerp midpoints sit too far out (off-distribution)')
ax2.legend(fontsize=9)
plt.tight_layout()
plt.show()
What you see¶
What the plot shows: Left — the grey cloud is the true latent distribution, centred far from the origin. Slerp taken about the origin (red) bows its midpoints along an arc curving toward the origin, scattering many of them outside the cloud; centering first (green) keeps the midpoints sitting neatly inside the data. Right — the Mahalanobis histogram confirms it: centered-Slerp midpoints overlap the real samples, while origin-Slerp midpoints pile up at distinctly larger distances (more out-of-distribution).
Why it looks this way: Slerp's norm-preservation is defined relative to its centre — the origin. When the data's centre is elsewhere, "preserve distance to the origin" is the wrong invariant: it drags the path along a circle the data never follows. Centering repairs the mean problem, but the remaining anisotropy (the cloud is an ellipse, not a circle) means even centered-Slerp is only approximate — a perfectly round arc cannot hug an elongated, correlated distribution.
Key takeaway: Slerp is the right tool only under its assumptions: zero-mean, isotropic, two endpoints. Real latents violate all three. The principled generalization shifts the goal from preserving norm to matching the distribution — e.g. the LOL transform $z = (1-\alpha/\sqrt{\beta})\mu + y/\sqrt{\beta}$, which corrects both mean and covariance and extends to centroids and subspaces, and for genuinely curved manifolds the pullback-geodesic methods from Section 1.
Summary¶
| Experiment | Theme | Key insight |
|---|---|---|
| 1. Arc vs chord | Mechanism | Slerp = Lerp with sine weights and a $1/\sin\Omega$ factor that keeps every point on the sphere |
| 2. Constant angular velocity | vs nlerp | Slerp sweeps angle linearly in $t$; nlerp stays on the sphere but rushes through the middle |
| 3. Slerp $\to$ Lerp | When to bother | For small angles the two paths coincide ($\cos(\Omega/2)\approx 1$); Slerp matters for wide-angle interpolation |
| 4. Edge cases | Implementation | Add a small-angle Lerp fallback and a short-arc sign flip, or the naive formula returns garbage |
| 5. Where Slerp breaks | Limits | Origin-centric Slerp drifts off non-zero-mean / anisotropic latents; distribution-matching (LOL) generalizes it |
Connection to the broader theory¶
Slerp is the closed-form geodesic of the one manifold whose geometry we know exactly — the sphere — which is why it is the default interpolator whenever a generative prior concentrates its mass on a shell ($\mathcal{N}(0,I)$ in GANs, diffusion noise, VAEs). Experiments 1–4 nail down how to use it correctly: the arc it draws, its uniform-speed signature versus the cheaper nlerp, the small-angle regime where plain Lerp suffices, and the two numerical guards every implementation needs. Experiment 5 marks the boundary: the moment the latent distribution is off-origin, anisotropic, or you need more than two points, Slerp's spherical assumption breaks and the problem becomes distribution matching (LOL) or, on irregular manifolds, pullback geodesics. For Latent-Anything, Slerp and Lerp are the two baseline interpolators of Layer B; choosing between them — and knowing when to reach past both — is a function of the latent space's measured geometry (see also isotropy vs anisotropy in this section).