PaCMAP — Visual Experiments¶

Goal: See how PaCMAP balances local and global structure by controlling different types of point pairs during optimization.

What we'll explore¶

#	Experiment	Question answered
1	UMAP vs PaCMAP on a bridged manifold	Does PaCMAP keep a global connector without sacrificing local neighborhoods too aggressively?
2	Sweeping MN_ratio	How do mid-near pairs affect the coarse global layout?
3	Sweeping FP_ratio	How much repulsion is useful before the embedding becomes over-stretched?
4	Optimization snapshots	What does PaCMAP's coarse-to-fine schedule look like over time?

---

Linked theory: ../research/06-pacmap.md

In [1]:

import io
import warnings
import contextlib

import numpy as np
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D
from scipy.stats import spearmanr
from sklearn.datasets import make_blobs, make_moons, make_swiss_roll
from sklearn.decomposition import PCA
from sklearn.manifold import trustworthiness
from sklearn.metrics import silhouette_score

import umap
import pacmap

warnings.filterwarnings('ignore')
np.random.seed(42)
plt.rcParams.update({
    'figure.dpi': 110,
    'font.size': 11,
    'axes.titlesize': 12,
    'axes.labelsize': 11,
    'figure.facecolor': 'white',
    'axes.facecolor': '#f8f9fa',
    'axes.grid': True,
    'grid.alpha': 0.35,
})

import io import warnings import contextlib import numpy as np import matplotlib.pyplot as plt from matplotlib.lines import Line2D from scipy.stats import spearmanr from sklearn.datasets import make_blobs, make_moons, make_swiss_roll from sklearn.decomposition import PCA from sklearn.manifold import trustworthiness from sklearn.metrics import silhouette_score import umap import pacmap warnings.filterwarnings('ignore') np.random.seed(42) plt.rcParams.update({ 'figure.dpi': 110, 'font.size': 11, 'axes.titlesize': 12, 'axes.labelsize': 11, 'figure.facecolor': 'white', 'axes.facecolor': '#f8f9fa', 'axes.grid': True, 'grid.alpha': 0.35, })

/home/runner/work/latent-anything/latent-anything/latent-anything-theory/.venv/lib/python3.13/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm

In [2]:

def run_pacmap(X, **kwargs):
    model = pacmap.PaCMAP(random_state=0, **kwargs)
    with contextlib.redirect_stdout(io.StringIO()):
        embedding = model.fit_transform(X, init='pca')
    return embedding


def run_umap(X, **kwargs):
    model = umap.UMAP(random_state=0, **kwargs)
    return model.fit_transform(X)


def first_axis_spearman(reference_values, embedding):
    axis = PCA(n_components=1).fit_transform(embedding).ravel()
    return abs(spearmanr(reference_values, axis).statistic)


def centroid_distance_spearman(reference_points, labels, embedding):
    unique_labels = np.unique(labels)
    ref_centroids = np.array([reference_points[labels == label].mean(axis=0) for label in unique_labels])
    emb_centroids = np.array([embedding[labels == label].mean(axis=0) for label in unique_labels])

    ref_distances = []
    emb_distances = []
    for i in range(len(unique_labels)):
        for j in range(i + 1, len(unique_labels)):
            ref_distances.append(np.linalg.norm(ref_centroids[i] - ref_centroids[j]))
            emb_distances.append(np.linalg.norm(emb_centroids[i] - emb_centroids[j]))
    return spearmanr(ref_distances, emb_distances).statistic


def make_bridge_dataset(seed=0):
    rng = np.random.default_rng(seed)
    X_moons, moon_labels = make_moons(n_samples=520, noise=0.06, random_state=seed)
    bridge_t = np.linspace(0.0, 1.0, 90)
    bridge = np.column_stack([
        -0.15 + 1.3 * bridge_t,
        0.23 + 0.05 * np.sin(np.pi * bridge_t),
    ]) + rng.normal(scale=0.015, size=(bridge_t.shape[0], 2))
    bridge_labels = np.full(bridge.shape[0], 2)
    base = np.vstack([X_moons, bridge]).astype(np.float32)
    labels = np.concatenate([moon_labels, bridge_labels])

    noise = rng.normal(scale=0.18, size=(base.shape[0], 8)).astype(np.float32)
    mixed = np.hstack([
        base,
        noise,
        (base[:, :1] * noise[:, :1]),
        (base[:, 1:2] * noise[:, 1:2]),
    ]).astype(np.float32)
    return mixed, base, labels


def make_hierarchical_cluster_dataset(seed=3):
    rng = np.random.default_rng(seed)
    centers = [(-5, -1), (-4, 1), (-1, -1), (1, 1), (4, -1), (5, 1)]
    base, labels = make_blobs(
        n_samples=[90, 90, 90, 90, 90, 90],
        centers=centers,
        cluster_std=[0.35] * 6,
        random_state=seed,
    )
    base = base.astype(np.float32)
    noise = rng.normal(scale=0.22, size=(base.shape[0], 10)).astype(np.float32)
    mixed = np.hstack([base, noise, base[:, :1] * 0.1, base[:, 1:2] * 0.15]).astype(np.float32)
    return mixed, base, labels


def make_overlap_dataset(seed=7):
    rng = np.random.default_rng(seed)
    centers = [(-3, -1), (-1, 1), (1, -1), (3, 1)]
    base, labels = make_blobs(
        n_samples=[160, 160, 160, 160],
        centers=centers,
        cluster_std=[0.95, 0.95, 0.95, 0.95],
        random_state=seed,
    )
    base = base.astype(np.float32)
    noise = rng.normal(scale=0.25, size=(base.shape[0], 6)).astype(np.float32)
    mixed = np.hstack([base, noise]).astype(np.float32)
    return mixed, base, labels

def run_pacmap(X, **kwargs): model = pacmap.PaCMAP(random_state=0, **kwargs) with contextlib.redirect_stdout(io.StringIO()): embedding = model.fit_transform(X, init='pca') return embedding def run_umap(X, **kwargs): model = umap.UMAP(random_state=0, **kwargs) return model.fit_transform(X) def first_axis_spearman(reference_values, embedding): axis = PCA(n_components=1).fit_transform(embedding).ravel() return abs(spearmanr(reference_values, axis).statistic) def centroid_distance_spearman(reference_points, labels, embedding): unique_labels = np.unique(labels) ref_centroids = np.array([reference_points[labels == label].mean(axis=0) for label in unique_labels]) emb_centroids = np.array([embedding[labels == label].mean(axis=0) for label in unique_labels]) ref_distances = [] emb_distances = [] for i in range(len(unique_labels)): for j in range(i + 1, len(unique_labels)): ref_distances.append(np.linalg.norm(ref_centroids[i] - ref_centroids[j])) emb_distances.append(np.linalg.norm(emb_centroids[i] - emb_centroids[j])) return spearmanr(ref_distances, emb_distances).statistic def make_bridge_dataset(seed=0): rng = np.random.default_rng(seed) X_moons, moon_labels = make_moons(n_samples=520, noise=0.06, random_state=seed) bridge_t = np.linspace(0.0, 1.0, 90) bridge = np.column_stack([ -0.15 + 1.3 * bridge_t, 0.23 + 0.05 * np.sin(np.pi * bridge_t), ]) + rng.normal(scale=0.015, size=(bridge_t.shape[0], 2)) bridge_labels = np.full(bridge.shape[0], 2) base = np.vstack([X_moons, bridge]).astype(np.float32) labels = np.concatenate([moon_labels, bridge_labels]) noise = rng.normal(scale=0.18, size=(base.shape[0], 8)).astype(np.float32) mixed = np.hstack([ base, noise, (base[:, :1] * noise[:, :1]), (base[:, 1:2] * noise[:, 1:2]), ]).astype(np.float32) return mixed, base, labels def make_hierarchical_cluster_dataset(seed=3): rng = np.random.default_rng(seed) centers = [(-5, -1), (-4, 1), (-1, -1), (1, 1), (4, -1), (5, 1)] base, labels = make_blobs( n_samples=[90, 90, 90, 90, 90, 90], centers=centers, cluster_std=[0.35] * 6, random_state=seed, ) base = base.astype(np.float32) noise = rng.normal(scale=0.22, size=(base.shape[0], 10)).astype(np.float32) mixed = np.hstack([base, noise, base[:, :1] * 0.1, base[:, 1:2] * 0.15]).astype(np.float32) return mixed, base, labels def make_overlap_dataset(seed=7): rng = np.random.default_rng(seed) centers = [(-3, -1), (-1, 1), (1, -1), (3, 1)] base, labels = make_blobs( n_samples=[160, 160, 160, 160], centers=centers, cluster_std=[0.95, 0.95, 0.95, 0.95], random_state=seed, ) base = base.astype(np.float32) noise = rng.normal(scale=0.25, size=(base.shape[0], 6)).astype(np.float32) mixed = np.hstack([base, noise]).astype(np.float32) return mixed, base, labels

---

Experiment 1: UMAP vs PaCMAP on a bridged manifold¶

Question: Can PaCMAP preserve a thin global connector while still keeping the two local moon-shaped neighborhoods readable?

The theory note argues that PaCMAP explicitly inserts mid-near pairs to stabilize medium-range geometry. A bridged two-moons dataset is a good stress test because it has both curved local neighborhoods and a weak global connector that many embeddings distort.

In [3]:

np.random.seed(1)
X_bridge, bridge_reference, bridge_labels = make_bridge_dataset(seed=1)
bridge_colors = np.array(['#1f77b4', '#ff7f0e', '#2ca02c'])

np.random.seed(1) X_bridge, bridge_reference, bridge_labels = make_bridge_dataset(seed=1) bridge_colors = np.array(['#1f77b4', '#ff7f0e', '#2ca02c'])

In [4]:

umap_bridge = run_umap(X_bridge, n_neighbors=20, min_dist=0.08)
pacmap_bridge = run_pacmap(
    X_bridge,
    n_components=2,
    n_neighbors=20,
    MN_ratio=0.5,
    FP_ratio=2.0,
    apply_pca=True,
)

umap_trust = trustworthiness(X_bridge, umap_bridge, n_neighbors=15)
pacmap_trust = trustworthiness(X_bridge, pacmap_bridge, n_neighbors=15)
umap_axis = first_axis_spearman(bridge_reference[:, 0], umap_bridge)
pacmap_axis = first_axis_spearman(bridge_reference[:, 0], pacmap_bridge)

fig, axes = plt.subplots(1, 3, figsize=(16, 4.8))
fig.suptitle('Experiment 1: A weak bridge between two nonlinear local neighborhoods', fontsize=13, fontweight='bold')

axes[0].scatter(
    bridge_reference[:, 0],
    bridge_reference[:, 1],
    c=bridge_colors[bridge_labels],
    s=12,
    alpha=0.8,
)
axes[0].set_title('Reference 2D geometry')
axes[0].set_xlabel('Reference x')
axes[0].set_ylabel('Reference y')

axes[1].scatter(
    umap_bridge[:, 0],
    umap_bridge[:, 1],
    c=bridge_colors[bridge_labels],
    s=12,
    alpha=0.8,
)
axes[1].set_title(f'UMAP\ntrust={umap_trust:.3f}, axis corr={umap_axis:.3f}')
axes[1].set_xlabel('Embedding x')
axes[1].set_ylabel('Embedding y')

axes[2].scatter(
    pacmap_bridge[:, 0],
    pacmap_bridge[:, 1],
    c=bridge_colors[bridge_labels],
    s=12,
    alpha=0.8,
)
axes[2].set_title(f'PaCMAP\ntrust={pacmap_trust:.3f}, axis corr={pacmap_axis:.3f}')
axes[2].set_xlabel('Embedding x')
axes[2].set_ylabel('Embedding y')

legend_items = [
    Line2D([0], [0], marker='o', color='w', label='Moon A', markerfacecolor=bridge_colors[0], markersize=8),
    Line2D([0], [0], marker='o', color='w', label='Moon B', markerfacecolor=bridge_colors[1], markersize=8),
    Line2D([0], [0], marker='o', color='w', label='Bridge points', markerfacecolor=bridge_colors[2], markersize=8),
]
axes[2].legend(handles=legend_items, loc='best')

plt.tight_layout()
plt.show()

umap_bridge = run_umap(X_bridge, n_neighbors=20, min_dist=0.08) pacmap_bridge = run_pacmap( X_bridge, n_components=2, n_neighbors=20, MN_ratio=0.5, FP_ratio=2.0, apply_pca=True, ) umap_trust = trustworthiness(X_bridge, umap_bridge, n_neighbors=15) pacmap_trust = trustworthiness(X_bridge, pacmap_bridge, n_neighbors=15) umap_axis = first_axis_spearman(bridge_reference[:, 0], umap_bridge) pacmap_axis = first_axis_spearman(bridge_reference[:, 0], pacmap_bridge) fig, axes = plt.subplots(1, 3, figsize=(16, 4.8)) fig.suptitle('Experiment 1: A weak bridge between two nonlinear local neighborhoods', fontsize=13, fontweight='bold') axes[0].scatter( bridge_reference[:, 0], bridge_reference[:, 1], c=bridge_colors[bridge_labels], s=12, alpha=0.8, ) axes[0].set_title('Reference 2D geometry') axes[0].set_xlabel('Reference x') axes[0].set_ylabel('Reference y') axes[1].scatter( umap_bridge[:, 0], umap_bridge[:, 1], c=bridge_colors[bridge_labels], s=12, alpha=0.8, ) axes[1].set_title(f'UMAP\ntrust={umap_trust:.3f}, axis corr={umap_axis:.3f}') axes[1].set_xlabel('Embedding x') axes[1].set_ylabel('Embedding y') axes[2].scatter( pacmap_bridge[:, 0], pacmap_bridge[:, 1], c=bridge_colors[bridge_labels], s=12, alpha=0.8, ) axes[2].set_title(f'PaCMAP\ntrust={pacmap_trust:.3f}, axis corr={pacmap_axis:.3f}') axes[2].set_xlabel('Embedding x') axes[2].set_ylabel('Embedding y') legend_items = [ Line2D([0], [0], marker='o', color='w', label='Moon A', markerfacecolor=bridge_colors[0], markersize=8), Line2D([0], [0], marker='o', color='w', label='Moon B', markerfacecolor=bridge_colors[1], markersize=8), Line2D([0], [0], marker='o', color='w', label='Bridge points', markerfacecolor=bridge_colors[2], markersize=8), ] axes[2].legend(handles=legend_items, loc='best') plt.tight_layout() plt.show()

Warning: random state is set to 0.

What you see¶

1. What the plot shows: The reference panel contains two curved local neighborhoods connected by a thin bridge. Both methods separate the moon-shaped regions well, but PaCMAP usually keeps the bridge more aligned with the large-scale left-to-right layout instead of treating it as a weak afterthought.
2. Why it looks this way: UMAP mainly reasons through neighbor connectivity. PaCMAP adds mid-near pairs, so the optimizer gets an extra signal about medium-range structure before it fully focuses on nearest-neighbor refinement.
3. Key takeaway: PaCMAP is useful when a latent cloud has meaningful transitions between clusters, not just isolated clusters themselves.

---

Experiment 2: Sweeping MN_ratio¶

Question: How much do mid-near pairs influence the relative arrangement of several nearby clusters?

MN_ratio controls how many mid-near pairs PaCMAP samples compared with nearest-neighbor pairs. If the ratio is too small, the embedding may not receive enough medium-range guidance. If it is too large, the optimizer can over-prioritize the coarse layout and soften local detail.

In [5]:

np.random.seed(2)
X_hier, hier_reference, hier_labels = make_hierarchical_cluster_dataset(seed=3)
mn_ratios = [0.05, 0.5, 1.5]
mn_embeddings = []
mn_metrics = []

for ratio in mn_ratios:
    embedding = run_pacmap(
        X_hier,
        n_components=2,
        n_neighbors=12,
        MN_ratio=ratio,
        FP_ratio=2.0,
        apply_pca=True,
    )
    mn_embeddings.append(embedding)
    mn_metrics.append((
        trustworthiness(X_hier, embedding, n_neighbors=10),
        centroid_distance_spearman(hier_reference, hier_labels, embedding),
    ))

np.random.seed(2) X_hier, hier_reference, hier_labels = make_hierarchical_cluster_dataset(seed=3) mn_ratios = [0.05, 0.5, 1.5] mn_embeddings = [] mn_metrics = [] for ratio in mn_ratios: embedding = run_pacmap( X_hier, n_components=2, n_neighbors=12, MN_ratio=ratio, FP_ratio=2.0, apply_pca=True, ) mn_embeddings.append(embedding) mn_metrics.append(( trustworthiness(X_hier, embedding, n_neighbors=10), centroid_distance_spearman(hier_reference, hier_labels, embedding), ))

Warning: random state is set to 0.

Warning: random state is set to 0.

Warning: random state is set to 0.

In [6]:

fig, axes = plt.subplots(1, 4, figsize=(19, 4.8))
fig.suptitle('Experiment 2: Mid-near pairs shape the global scaffold', fontsize=13, fontweight='bold')

axes[0].scatter(
    hier_reference[:, 0],
    hier_reference[:, 1],
    c=hier_labels,
    cmap='tab10',
    s=14,
    alpha=0.8,
)
axes[0].set_title('Reference cluster layout')
axes[0].set_xlabel('Reference x')
axes[0].set_ylabel('Reference y')

for ax, ratio, embedding, (trust, corr) in zip(axes[1:], mn_ratios, mn_embeddings, mn_metrics):
    ax.scatter(embedding[:, 0], embedding[:, 1], c=hier_labels, cmap='tab10', s=14, alpha=0.8)
    ax.set_title(f'MN_ratio={ratio}\ntrust={trust:.3f}, centroid corr={corr:.3f}')
    ax.set_xlabel('Embedding x')
    ax.set_ylabel('Embedding y')

plt.tight_layout()
plt.show()

fig, axes = plt.subplots(1, 4, figsize=(19, 4.8)) fig.suptitle('Experiment 2: Mid-near pairs shape the global scaffold', fontsize=13, fontweight='bold') axes[0].scatter( hier_reference[:, 0], hier_reference[:, 1], c=hier_labels, cmap='tab10', s=14, alpha=0.8, ) axes[0].set_title('Reference cluster layout') axes[0].set_xlabel('Reference x') axes[0].set_ylabel('Reference y') for ax, ratio, embedding, (trust, corr) in zip(axes[1:], mn_ratios, mn_embeddings, mn_metrics): ax.scatter(embedding[:, 0], embedding[:, 1], c=hier_labels, cmap='tab10', s=14, alpha=0.8) ax.set_title(f'MN_ratio={ratio}\ntrust={trust:.3f}, centroid corr={corr:.3f}') ax.set_xlabel('Embedding x') ax.set_ylabel('Embedding y') plt.tight_layout() plt.show()

What you see¶

1. What the plot shows: All three embeddings keep local clusters recognizable, but the relative spacing between cluster groups changes as MN_ratio changes. Moderate values often produce the cleanest compromise between stable coarse layout and crisp local grouping.
2. Why it looks this way: Mid-near pairs are the part of PaCMAP that talks about medium-range structure. Too few of them leaves the optimizer closer to a pure neighbor-plus-repulsion system; too many can over-constrain the layout.
3. Key takeaway: MN_ratio is the knob that most directly expresses PaCMAP's design philosophy: preserve more than immediate neighborhoods, but do not let medium-range structure dominate everything.

---

Experiment 3: Sweeping FP_ratio¶

Question: What happens when we change the strength of far-pair repulsion?

Further pairs are PaCMAP's anti-collapse mechanism. They prevent unrelated regions from sitting on top of each other, but excessive repulsion can also stretch the embedding more than we want.

In [7]:

np.random.seed(3)
X_overlap, overlap_reference, overlap_labels = make_overlap_dataset(seed=7)
fp_ratios = [0.5, 2.0, 5.0]
fp_embeddings = []
fp_metrics = []

for ratio in fp_ratios:
    embedding = run_pacmap(
        X_overlap,
        n_components=2,
        n_neighbors=15,
        MN_ratio=0.5,
        FP_ratio=ratio,
        apply_pca=True,
    )
    fp_embeddings.append(embedding)
    fp_metrics.append((
        trustworthiness(X_overlap, embedding, n_neighbors=10),
        silhouette_score(embedding, overlap_labels),
    ))

np.random.seed(3) X_overlap, overlap_reference, overlap_labels = make_overlap_dataset(seed=7) fp_ratios = [0.5, 2.0, 5.0] fp_embeddings = [] fp_metrics = [] for ratio in fp_ratios: embedding = run_pacmap( X_overlap, n_components=2, n_neighbors=15, MN_ratio=0.5, FP_ratio=ratio, apply_pca=True, ) fp_embeddings.append(embedding) fp_metrics.append(( trustworthiness(X_overlap, embedding, n_neighbors=10), silhouette_score(embedding, overlap_labels), ))

Warning: random state is set to 0.

Warning: random state is set to 0.

Warning: random state is set to 0.

In [8]:

fig, axes = plt.subplots(1, 4, figsize=(19, 4.8))
fig.suptitle('Experiment 3: Further-pair repulsion controls separation pressure', fontsize=13, fontweight='bold')

axes[0].scatter(
    overlap_reference[:, 0],
    overlap_reference[:, 1],
    c=overlap_labels,
    cmap='tab10',
    s=14,
    alpha=0.8,
)
axes[0].set_title('Reference overlapping blobs')
axes[0].set_xlabel('Reference x')
axes[0].set_ylabel('Reference y')

for ax, ratio, embedding, (trust, silhouette) in zip(axes[1:], fp_ratios, fp_embeddings, fp_metrics):
    ax.scatter(embedding[:, 0], embedding[:, 1], c=overlap_labels, cmap='tab10', s=14, alpha=0.8)
    ax.set_title(f'FP_ratio={ratio}\ntrust={trust:.3f}, silhouette={silhouette:.3f}')
    ax.set_xlabel('Embedding x')
    ax.set_ylabel('Embedding y')

plt.tight_layout()
plt.show()

fig, axes = plt.subplots(1, 4, figsize=(19, 4.8)) fig.suptitle('Experiment 3: Further-pair repulsion controls separation pressure', fontsize=13, fontweight='bold') axes[0].scatter( overlap_reference[:, 0], overlap_reference[:, 1], c=overlap_labels, cmap='tab10', s=14, alpha=0.8, ) axes[0].set_title('Reference overlapping blobs') axes[0].set_xlabel('Reference x') axes[0].set_ylabel('Reference y') for ax, ratio, embedding, (trust, silhouette) in zip(axes[1:], fp_ratios, fp_embeddings, fp_metrics): ax.scatter(embedding[:, 0], embedding[:, 1], c=overlap_labels, cmap='tab10', s=14, alpha=0.8) ax.set_title(f'FP_ratio={ratio}\ntrust={trust:.3f}, silhouette={silhouette:.3f}') ax.set_xlabel('Embedding x') ax.set_ylabel('Embedding y') plt.tight_layout() plt.show()

What you see¶

1. What the plot shows: As FP_ratio increases, unrelated regions usually separate more clearly, and the visual gaps between clusters become stronger. At the same time, the overall map can become more expanded.
2. Why it looks this way: Further pairs only contribute repulsion. Increasing their count changes the optimization pressure so that points not supported by local or mid-range evidence are pushed away more aggressively.
3. Key takeaway: FP_ratio is not about recovering structure by itself; it is about preventing false overlaps and controlling how much empty space the embedding creates.

---

Experiment 4: Optimization snapshots¶

Question: Does PaCMAP really behave like a coarse-to-fine optimizer over time?

The reference implementation changes pair weights across multiple phases. By asking PaCMAP for intermediate snapshots, we can watch the map evolve from a rough scaffold into a refined embedding.

In [9]:

np.random.seed(4)
swiss_points, swiss_t = make_swiss_roll(n_samples=700, noise=0.03, random_state=4)
swiss_base = swiss_points[:, [0, 2]].astype(np.float32)
extra_noise = np.random.normal(scale=0.12, size=(swiss_base.shape[0], 5)).astype(np.float32)
swiss_high = np.hstack([swiss_base, extra_noise]).astype(np.float32)

swiss_states = run_pacmap(
    swiss_high,
    n_components=2,
    n_neighbors=18,
    MN_ratio=0.5,
    FP_ratio=2.0,
    apply_pca=False,
    intermediate=True,
    intermediate_snapshots=[0, 30, 100, 250, 450],
)
selected_states = swiss_states[1:]
selected_steps = [30, 100, 250, 450]

np.random.seed(4) swiss_points, swiss_t = make_swiss_roll(n_samples=700, noise=0.03, random_state=4) swiss_base = swiss_points[:, [0, 2]].astype(np.float32) extra_noise = np.random.normal(scale=0.12, size=(swiss_base.shape[0], 5)).astype(np.float32) swiss_high = np.hstack([swiss_base, extra_noise]).astype(np.float32) swiss_states = run_pacmap( swiss_high, n_components=2, n_neighbors=18, MN_ratio=0.5, FP_ratio=2.0, apply_pca=False, intermediate=True, intermediate_snapshots=[0, 30, 100, 250, 450], ) selected_states = swiss_states[1:] selected_steps = [30, 100, 250, 450]

Warning: random state is set to 0.

Running NN Backend on high-dimensional data. Nearest-neighbor search may be slow!

In [10]:

fig, axes = plt.subplots(1, 4, figsize=(18, 4.8))
fig.suptitle('Experiment 4: PaCMAP builds a scaffold first, then sharpens neighborhoods', fontsize=13, fontweight='bold')

for ax, step, embedding in zip(axes, selected_steps, selected_states):
    scatter = ax.scatter(
        embedding[:, 0],
        embedding[:, 1],
        c=swiss_t,
        cmap='viridis',
        s=10,
        alpha=0.85,
    )
    ax.set_title(f'Iteration {step}')
    ax.set_xlabel('Embedding x')
    ax.set_ylabel('Embedding y')

cbar = fig.colorbar(scatter, ax=axes, shrink=0.8)
cbar.set_label('Swiss-roll progression parameter')
plt.tight_layout()
plt.show()

fig, axes = plt.subplots(1, 4, figsize=(18, 4.8)) fig.suptitle('Experiment 4: PaCMAP builds a scaffold first, then sharpens neighborhoods', fontsize=13, fontweight='bold') for ax, step, embedding in zip(axes, selected_steps, selected_states): scatter = ax.scatter( embedding[:, 0], embedding[:, 1], c=swiss_t, cmap='viridis', s=10, alpha=0.85, ) ax.set_title(f'Iteration {step}') ax.set_xlabel('Embedding x') ax.set_ylabel('Embedding y') cbar = fig.colorbar(scatter, ax=axes, shrink=0.8) cbar.set_label('Swiss-roll progression parameter') plt.tight_layout() plt.show()

What you see¶

1. What the plot shows: Early snapshots often look coarse, with the large-scale arrangement appearing before the final local tightening. Later snapshots sharpen neighborhood detail while keeping most of the broad scaffold already in place.
2. Why it looks this way: The optimizer starts with very strong emphasis on mid-near pairs, then gradually shifts toward ordinary neighbor refinement. That schedule is visible in the geometry of the snapshots.
3. Key takeaway: PaCMAP is not just a static loss; it is a staged optimization procedure whose behavior changes over time.

---

Summary¶

Experiment	Key insight
1. UMAP vs PaCMAP	PaCMAP can better respect weak medium-range connectors while keeping local neighborhoods legible.
2. Sweeping MN_ratio	Mid-near pairs are the main control for stabilizing the global scaffold.
3. Sweeping FP_ratio	Further pairs control repulsion pressure and help prevent false overlap.
4. Optimization snapshots	PaCMAP behaves like a coarse-to-fine optimizer, not a single static balancing act.

Connection to the broader theory¶

These experiments make the theory note concrete: PaCMAP is best understood as a pair-selection strategy plus a staged weighting schedule. In Latent-Anything, that makes it a strong exploratory companion to UMAP when we care about both neighborhood detail and the overall arrangement of latent regions.