Markov Property and State Space — Visual Experiments¶

Goal: See why an observation is not automatically a state, how history can restore predictive sufficiency, and why belief states matter under partial observability.

What we will explore¶

#	Experiment	Question answered
1	Observation aliasing	How can the same observation lead to different futures?
2	State augmentation	How much does adding velocity improve prediction?
3	History ablation	Can old observations still predict the future after conditioning on the current observation?
4	Belief-state filtering	How does a posterior belief summarize noisy observation history?
5	Rollout drift	Why can a low one-step error still produce a bad long rollout?

Linked theory: Markov Property and State Space

In [1]:

import warnings

import matplotlib.pyplot as plt
import numpy as np

warnings.filterwarnings('ignore')

plt.rcParams.update({
    'figure.dpi': 110,
    'font.size': 11,
    'axes.titlesize': 12,
    'axes.labelsize': 11,
    'figure.facecolor': 'white',
    'axes.facecolor': '#f8f9fa',
    'axes.grid': True,
    'grid.alpha': 0.35,
})

import warnings import matplotlib.pyplot as plt import numpy as np warnings.filterwarnings('ignore') plt.rcParams.update({ 'figure.dpi': 110, 'font.size': 11, 'axes.titlesize': 12, 'axes.labelsize': 11, 'figure.facecolor': 'white', 'axes.facecolor': '#f8f9fa', 'axes.grid': True, 'grid.alpha': 0.35, })

---

Experiment 1: One observation, two possible futures¶

Question: Why is position alone not a Markov state for a moving object?

We generate objects with the same range of positions but two hidden velocities. If velocity is omitted, points with nearly identical current observations can move in opposite directions at the next step.

In [2]:

np.random.seed(1)
n_samples = 600
dt = 0.8
position = np.random.uniform(-2.0, 2.0, n_samples)
velocity = np.random.choice([-1.0, 1.0], size=n_samples)
next_position = position + dt * velocity + np.random.normal(0.0, 0.06, n_samples)
near_zero = np.abs(position) < 0.12

fig, axes = plt.subplots(1, 2, figsize=(13, 4.8))
fig.suptitle('Observation aliasing when velocity is hidden', fontsize=14, fontweight='bold')

for value, color, label in [(-1.0, '#3b4cc0', 'velocity = -1'), (1.0, '#b40426', 'velocity = +1')]:
    mask = velocity == value
    axes[0].scatter(position[mask], next_position[mask], s=18, alpha=0.55, color=color, label=label)
axes[0].set_title('Same current position, different next positions')
axes[0].set_xlabel('Current observation $o_t = x_t$')
axes[0].set_ylabel('Next observation $o_{t+1} = x_{t+1}$')
axes[0].legend()

axes[1].hist(next_position[near_zero & (velocity < 0)], bins=14, alpha=0.7, color='#3b4cc0', label='velocity = -1')
axes[1].hist(next_position[near_zero & (velocity > 0)], bins=14, alpha=0.7, color='#b40426', label='velocity = +1')
axes[1].set_title(r'Future distribution near the same observation $x_t \approx 0$')
axes[1].set_xlabel('Next position $x_{t+1}$')
axes[1].set_ylabel('Sample count')
axes[1].legend()

plt.tight_layout()
plt.show()

np.random.seed(1) n_samples = 600 dt = 0.8 position = np.random.uniform(-2.0, 2.0, n_samples) velocity = np.random.choice([-1.0, 1.0], size=n_samples) next_position = position + dt * velocity + np.random.normal(0.0, 0.06, n_samples) near_zero = np.abs(position) < 0.12 fig, axes = plt.subplots(1, 2, figsize=(13, 4.8)) fig.suptitle('Observation aliasing when velocity is hidden', fontsize=14, fontweight='bold') for value, color, label in [(-1.0, '#3b4cc0', 'velocity = -1'), (1.0, '#b40426', 'velocity = +1')]: mask = velocity == value axes[0].scatter(position[mask], next_position[mask], s=18, alpha=0.55, color=color, label=label) axes[0].set_title('Same current position, different next positions') axes[0].set_xlabel('Current observation $o_t = x_t$') axes[0].set_ylabel('Next observation $o_{t+1} = x_{t+1}$') axes[0].legend() axes[1].hist(next_position[near_zero & (velocity < 0)], bins=14, alpha=0.7, color='#3b4cc0', label='velocity = -1') axes[1].hist(next_position[near_zero & (velocity > 0)], bins=14, alpha=0.7, color='#b40426', label='velocity = +1') axes[1].set_title(r'Future distribution near the same observation $x_t \approx 0$') axes[1].set_xlabel('Next position $x_{t+1}$') axes[1].set_ylabel('Sample count') axes[1].legend() plt.tight_layout() plt.show()

What you see¶

1. What the plots show: The left panel contains two distinct next-position branches for every current position. Near $x_t=0$, the right panel shows two separated future modes.
2. Why it looks this way: Position aliases two hidden physical states: moving left and moving right. Conditioning on $x_t$ alone does not block information from the earlier trajectory.
3. Key takeaway: An observation is not a Markov state when omitted variables still change the conditional distribution of the future.

---

Experiment 2: State augmentation restores predictive sufficiency¶

Question: What changes when velocity is added to the state?

We fit two linear one-step predictors on the same data. The observation-only model receives $x_t$; the augmented model receives the candidate state $(x_t, v_t)$.

In [3]:

np.random.seed(2)
indices = np.random.permutation(n_samples)
train_idx, test_idx = indices[:400], indices[400:]

x_only_train = np.column_stack([np.ones(train_idx.size), position[train_idx]])
x_only_test = np.column_stack([np.ones(test_idx.size), position[test_idx]])
state_train = np.column_stack([np.ones(train_idx.size), position[train_idx], velocity[train_idx]])
state_test = np.column_stack([np.ones(test_idx.size), position[test_idx], velocity[test_idx]])

coef_observation = np.linalg.lstsq(x_only_train, next_position[train_idx], rcond=None)[0]
coef_state = np.linalg.lstsq(state_train, next_position[train_idx], rcond=None)[0]
pred_observation = x_only_test @ coef_observation
pred_state = state_test @ coef_state

mse_observation = np.mean((pred_observation - next_position[test_idx]) ** 2)
mse_state = np.mean((pred_state - next_position[test_idx]) ** 2)

fig, axes = plt.subplots(1, 2, figsize=(13, 4.8))
fig.suptitle('Prediction with an observation versus a Markov candidate state', fontsize=14, fontweight='bold')

axes[0].scatter(next_position[test_idx], pred_observation, alpha=0.55, s=22, color='#f2a541', label='position only')
axes[0].scatter(next_position[test_idx], pred_state, alpha=0.55, s=22, color='#2a9d8f', label='position + velocity')
limits = [-3.0, 3.0]
axes[0].plot(limits, limits, '--', color='black', linewidth=1.2, label='perfect prediction')
axes[0].set_xlim(limits)
axes[0].set_ylim(limits)
axes[0].set_title('Predicted versus true next position')
axes[0].set_xlabel('True $x_{t+1}$')
axes[0].set_ylabel('Predicted $x_{t+1}$')
axes[0].legend()

bars = axes[1].bar(['$x_t$ only', '$(x_t, v_t)$'], [mse_observation, mse_state], color=['#f2a541', '#2a9d8f'])
axes[1].set_title('Held-out one-step prediction error')
axes[1].set_xlabel('Predictor input')
axes[1].set_ylabel('Mean squared error')
for bar, value in zip(bars, [mse_observation, mse_state], strict=True):
    axes[1].text(bar.get_x() + bar.get_width() / 2, value, f'{value:.4f}', ha='center', va='bottom')

plt.tight_layout()
plt.show()

np.random.seed(2) indices = np.random.permutation(n_samples) train_idx, test_idx = indices[:400], indices[400:] x_only_train = np.column_stack([np.ones(train_idx.size), position[train_idx]]) x_only_test = np.column_stack([np.ones(test_idx.size), position[test_idx]]) state_train = np.column_stack([np.ones(train_idx.size), position[train_idx], velocity[train_idx]]) state_test = np.column_stack([np.ones(test_idx.size), position[test_idx], velocity[test_idx]]) coef_observation = np.linalg.lstsq(x_only_train, next_position[train_idx], rcond=None)[0] coef_state = np.linalg.lstsq(state_train, next_position[train_idx], rcond=None)[0] pred_observation = x_only_test @ coef_observation pred_state = state_test @ coef_state mse_observation = np.mean((pred_observation - next_position[test_idx]) ** 2) mse_state = np.mean((pred_state - next_position[test_idx]) ** 2) fig, axes = plt.subplots(1, 2, figsize=(13, 4.8)) fig.suptitle('Prediction with an observation versus a Markov candidate state', fontsize=14, fontweight='bold') axes[0].scatter(next_position[test_idx], pred_observation, alpha=0.55, s=22, color='#f2a541', label='position only') axes[0].scatter(next_position[test_idx], pred_state, alpha=0.55, s=22, color='#2a9d8f', label='position + velocity') limits = [-3.0, 3.0] axes[0].plot(limits, limits, '--', color='black', linewidth=1.2, label='perfect prediction') axes[0].set_xlim(limits) axes[0].set_ylim(limits) axes[0].set_title('Predicted versus true next position') axes[0].set_xlabel('True $x_{t+1}$') axes[0].set_ylabel('Predicted $x_{t+1}$') axes[0].legend() bars = axes[1].bar(['$x_t$ only', '$(x_t, v_t)$'], [mse_observation, mse_state], color=['#f2a541', '#2a9d8f']) axes[1].set_title('Held-out one-step prediction error') axes[1].set_xlabel('Predictor input') axes[1].set_ylabel('Mean squared error') for bar, value in zip(bars, [mse_observation, mse_state], strict=True): axes[1].text(bar.get_x() + bar.get_width() / 2, value, f'{value:.4f}', ha='center', va='bottom') plt.tight_layout() plt.show()

What you see¶

1. What the plots show: Position-only predictions collapse toward the average of the two velocity branches, while the augmented-state predictions lie close to the perfect-prediction diagonal. Their held-out MSE differs by orders of magnitude.
2. Why it looks this way: Once velocity is known, the transition $x_{t+1}=x_t+\Delta t\,v_t$ is almost deterministic. Without velocity, the best squared-error predictor averages incompatible futures.
3. Key takeaway: State augmentation can turn apparent stochasticity into predictable dynamics by carrying the missing information forward.

---

Experiment 3: History ablation as a Markov diagnostic¶

Question: Does an older observation still improve prediction after the current observation is known?

For a harmonic oscillator, position alone omits velocity. Two consecutive positions approximately recover velocity, so a predictor using $(x_{t-1},x_t)$ should outperform one using only $x_t$.

In [4]:

np.random.seed(3)
n_steps = 500
delta = 0.16
time = np.arange(n_steps) * delta
signal = np.cos(time) + 0.03 * np.random.randn(n_steps)

target = signal[2:]
current_only = np.column_stack([np.ones(n_steps - 2), signal[1:-1]])
with_history = np.column_stack([np.ones(n_steps - 2), signal[1:-1], signal[:-2]])
split = 320

coef_current = np.linalg.lstsq(current_only[:split], target[:split], rcond=None)[0]
coef_history = np.linalg.lstsq(with_history[:split], target[:split], rcond=None)[0]
pred_current = current_only[split:] @ coef_current
pred_history = with_history[split:] @ coef_history
test_target = target[split:]
mse_current = np.mean((pred_current - test_target) ** 2)
mse_history = np.mean((pred_history - test_target) ** 2)

fig, axes = plt.subplots(1, 2, figsize=(13, 4.8))
fig.suptitle('History ablation reveals missing state information', fontsize=14, fontweight='bold')

window = slice(20, 90)
test_time = time[split + 2:]
axes[0].plot(test_time[window], test_target[window], color='black', linewidth=2.0, label='true next position')
axes[0].plot(test_time[window], pred_current[window], '--', color='#f2a541', label='$x_t$ only')
axes[0].plot(test_time[window], pred_history[window], color='#2a9d8f', label='$(x_{t-1}, x_t)$')
axes[0].set_title('Held-out one-step predictions')
axes[0].set_xlabel('Time')
axes[0].set_ylabel('Next position')
axes[0].legend()

bars = axes[1].bar(['Current only', 'Current + history'], [mse_current, mse_history], color=['#f2a541', '#2a9d8f'])
axes[1].set_title('Residual value of the previous observation')
axes[1].set_xlabel('Predictor input')
axes[1].set_ylabel('Mean squared error')
for bar, value in zip(bars, [mse_current, mse_history], strict=True):
    axes[1].text(bar.get_x() + bar.get_width() / 2, value, f'{value:.5f}', ha='center', va='bottom')

plt.tight_layout()
plt.show()

np.random.seed(3) n_steps = 500 delta = 0.16 time = np.arange(n_steps) * delta signal = np.cos(time) + 0.03 * np.random.randn(n_steps) target = signal[2:] current_only = np.column_stack([np.ones(n_steps - 2), signal[1:-1]]) with_history = np.column_stack([np.ones(n_steps - 2), signal[1:-1], signal[:-2]]) split = 320 coef_current = np.linalg.lstsq(current_only[:split], target[:split], rcond=None)[0] coef_history = np.linalg.lstsq(with_history[:split], target[:split], rcond=None)[0] pred_current = current_only[split:] @ coef_current pred_history = with_history[split:] @ coef_history test_target = target[split:] mse_current = np.mean((pred_current - test_target) ** 2) mse_history = np.mean((pred_history - test_target) ** 2) fig, axes = plt.subplots(1, 2, figsize=(13, 4.8)) fig.suptitle('History ablation reveals missing state information', fontsize=14, fontweight='bold') window = slice(20, 90) test_time = time[split + 2:] axes[0].plot(test_time[window], test_target[window], color='black', linewidth=2.0, label='true next position') axes[0].plot(test_time[window], pred_current[window], '--', color='#f2a541', label='$x_t$ only') axes[0].plot(test_time[window], pred_history[window], color='#2a9d8f', label='$(x_{t-1}, x_t)$') axes[0].set_title('Held-out one-step predictions') axes[0].set_xlabel('Time') axes[0].set_ylabel('Next position') axes[0].legend() bars = axes[1].bar(['Current only', 'Current + history'], [mse_current, mse_history], color=['#f2a541', '#2a9d8f']) axes[1].set_title('Residual value of the previous observation') axes[1].set_xlabel('Predictor input') axes[1].set_ylabel('Mean squared error') for bar, value in zip(bars, [mse_current, mse_history], strict=True): axes[1].text(bar.get_x() + bar.get_width() / 2, value, f'{value:.5f}', ha='center', va='bottom') plt.tight_layout() plt.show()

What you see¶

1. What the plots show: The predictor with one extra historical position follows turning points more accurately and achieves a lower held-out error.
2. Why it looks this way: The difference $x_t-x_{t-1}$ carries velocity information. If adding history improves prediction after conditioning on the proposed state, that state is not predictively sufficient.
3. Key takeaway: History ablation is a practical Markov diagnostic: useful residual history signals missing state variables.

---

Experiment 4: A belief state compresses noisy history¶

Question: How can a compact posterior summarize a long, noisy observation sequence?

A hidden binary mode switches only occasionally. Individual observations are noisy, so the latest observation alone is unreliable. A Bayes filter combines the previous belief, the transition model, and the new likelihood.

In [5]:

np.random.seed(4)
n_steps_belief = 90
stay_probability = 0.96
observation_sigma = 1.15
hidden_mode = np.empty(n_steps_belief, dtype=int)
hidden_mode[0] = 1
for t in range(1, n_steps_belief):
    hidden_mode[t] = hidden_mode[t - 1] if np.random.rand() < stay_probability else -hidden_mode[t - 1]
observations = hidden_mode + np.random.normal(0.0, observation_sigma, n_steps_belief)

belief_positive = np.empty(n_steps_belief)
prior_positive = 0.5
for t, observation in enumerate(observations):
    predicted_positive = stay_probability * prior_positive + (1 - stay_probability) * (1 - prior_positive)
    likelihood_positive = np.exp(-0.5 * ((observation - 1) / observation_sigma) ** 2)
    likelihood_negative = np.exp(-0.5 * ((observation + 1) / observation_sigma) ** 2)
    normalizer = likelihood_positive * predicted_positive + likelihood_negative * (1 - predicted_positive)
    posterior_positive = likelihood_positive * predicted_positive / normalizer
    belief_positive[t] = posterior_positive
    prior_positive = posterior_positive

fig, axes = plt.subplots(2, 1, figsize=(13, 7), sharex=True)
fig.suptitle('Recursive belief update under partial observability', fontsize=14, fontweight='bold')

axes[0].step(np.arange(n_steps_belief), hidden_mode, where='mid', color='black', linewidth=2.0, label='hidden mode')
axes[0].scatter(np.arange(n_steps_belief), observations, color='#6c8ebf', s=24, alpha=0.65, label='noisy observation')
axes[0].set_title('Hidden state and noisy sensor readings')
axes[0].set_xlabel('Time step')
axes[0].set_ylabel('Mode / observation')
axes[0].legend(loc='upper right')

axes[1].plot(belief_positive, color='#7b2cbf', linewidth=2.2, label=r'$P(s_t=+1 \mid o_{0:t})$')
axes[1].axhline(0.5, color='gray', linestyle='--', linewidth=1.2, label='maximum uncertainty')
axes[1].fill_between(np.arange(n_steps_belief), 0, 1, where=hidden_mode > 0, color='#90be6d', alpha=0.16, label='true positive mode')
axes[1].set_ylim(-0.03, 1.03)
axes[1].set_title('Belief state after prediction and correction')
axes[1].set_xlabel('Time step')
axes[1].set_ylabel('Posterior probability')
axes[1].legend(loc='upper right')

plt.tight_layout()
plt.show()

np.random.seed(4) n_steps_belief = 90 stay_probability = 0.96 observation_sigma = 1.15 hidden_mode = np.empty(n_steps_belief, dtype=int) hidden_mode[0] = 1 for t in range(1, n_steps_belief): hidden_mode[t] = hidden_mode[t - 1] if np.random.rand() < stay_probability else -hidden_mode[t - 1] observations = hidden_mode + np.random.normal(0.0, observation_sigma, n_steps_belief) belief_positive = np.empty(n_steps_belief) prior_positive = 0.5 for t, observation in enumerate(observations): predicted_positive = stay_probability * prior_positive + (1 - stay_probability) * (1 - prior_positive) likelihood_positive = np.exp(-0.5 * ((observation - 1) / observation_sigma) ** 2) likelihood_negative = np.exp(-0.5 * ((observation + 1) / observation_sigma) ** 2) normalizer = likelihood_positive * predicted_positive + likelihood_negative * (1 - predicted_positive) posterior_positive = likelihood_positive * predicted_positive / normalizer belief_positive[t] = posterior_positive prior_positive = posterior_positive fig, axes = plt.subplots(2, 1, figsize=(13, 7), sharex=True) fig.suptitle('Recursive belief update under partial observability', fontsize=14, fontweight='bold') axes[0].step(np.arange(n_steps_belief), hidden_mode, where='mid', color='black', linewidth=2.0, label='hidden mode') axes[0].scatter(np.arange(n_steps_belief), observations, color='#6c8ebf', s=24, alpha=0.65, label='noisy observation') axes[0].set_title('Hidden state and noisy sensor readings') axes[0].set_xlabel('Time step') axes[0].set_ylabel('Mode / observation') axes[0].legend(loc='upper right') axes[1].plot(belief_positive, color='#7b2cbf', linewidth=2.2, label=r'$P(s_t=+1 \mid o_{0:t})$') axes[1].axhline(0.5, color='gray', linestyle='--', linewidth=1.2, label='maximum uncertainty') axes[1].fill_between(np.arange(n_steps_belief), 0, 1, where=hidden_mode > 0, color='#90be6d', alpha=0.16, label='true positive mode') axes[1].set_ylim(-0.03, 1.03) axes[1].set_title('Belief state after prediction and correction') axes[1].set_xlabel('Time step') axes[1].set_ylabel('Posterior probability') axes[1].legend(loc='upper right') plt.tight_layout() plt.show()

What you see¶

1. What the plots show: Sensor samples frequently cross zero even when the hidden mode does not change. The posterior belief is smoother, remains confident through isolated noisy readings, and changes after sustained contrary evidence.
2. Why it looks this way: The filter carries information from all previous observations through one scalar posterior, then combines it with transition persistence and the newest likelihood.
3. Key takeaway: Under partial observability, a belief state can be Markov even when the latest observation is not.

---

Experiment 5: One-step accuracy does not guarantee rollout stability¶

Question: Why does an incomplete state accumulate error during autonomous rollout?

We learn a position-only one-step model from many phases of a harmonic oscillator. Its best linear coefficient has low local error, but repeated application loses phase information. A full state containing position and velocity follows the exact rotation dynamics.

In [6]:

np.random.seed(5)
rollout_dt = 0.18
phases = np.random.uniform(0.0, 2 * np.pi, 2000)
train_x = np.cos(phases)
train_next_x = np.cos(phases + rollout_dt)
position_coefficient = np.linalg.lstsq(train_x[:, None], train_next_x, rcond=None)[0][0]
one_step_mse = np.mean((position_coefficient * train_x - train_next_x) ** 2)

horizon = 90
rollout_time = np.arange(horizon) * rollout_dt
true_position = np.cos(rollout_time)

position_only_rollout = np.empty(horizon)
position_only_rollout[0] = 1.0
for t in range(horizon - 1):
    position_only_rollout[t + 1] = position_coefficient * position_only_rollout[t]

rotation = np.array([
    [np.cos(rollout_dt), np.sin(rollout_dt)],
    [-np.sin(rollout_dt), np.cos(rollout_dt)],
])
full_state = np.array([1.0, 0.0])
full_state_rollout = np.empty(horizon)
for t in range(horizon):
    full_state_rollout[t] = full_state[0]
    full_state = rotation @ full_state

fig, axes = plt.subplots(1, 2, figsize=(13, 4.8))
fig.suptitle('Missing state information compounds during rollout', fontsize=14, fontweight='bold')

axes[0].plot(rollout_time, true_position, color='black', linewidth=2.2, label='true trajectory')
axes[0].plot(rollout_time, position_only_rollout, '--', color='#e76f51', linewidth=2.0, label='position-only rollout')
axes[0].plot(rollout_time, full_state_rollout, color='#2a9d8f', linewidth=1.7, label='position + velocity rollout')
axes[0].set_title(f'Autonomous rollout (one-step MSE = {one_step_mse:.4f})')
axes[0].set_xlabel('Time')
axes[0].set_ylabel('Position')
axes[0].legend()

axes[1].plot(rollout_time, np.abs(position_only_rollout - true_position), color='#e76f51', linewidth=2.0, label='position-only error')
axes[1].plot(rollout_time, np.abs(full_state_rollout - true_position), color='#2a9d8f', linewidth=1.7, label='full-state error')
axes[1].set_title('Absolute error across the rollout horizon')
axes[1].set_xlabel('Time')
axes[1].set_ylabel('Absolute position error')
axes[1].legend()

plt.tight_layout()
plt.show()

np.random.seed(5) rollout_dt = 0.18 phases = np.random.uniform(0.0, 2 * np.pi, 2000) train_x = np.cos(phases) train_next_x = np.cos(phases + rollout_dt) position_coefficient = np.linalg.lstsq(train_x[:, None], train_next_x, rcond=None)[0][0] one_step_mse = np.mean((position_coefficient * train_x - train_next_x) ** 2) horizon = 90 rollout_time = np.arange(horizon) * rollout_dt true_position = np.cos(rollout_time) position_only_rollout = np.empty(horizon) position_only_rollout[0] = 1.0 for t in range(horizon - 1): position_only_rollout[t + 1] = position_coefficient * position_only_rollout[t] rotation = np.array([ [np.cos(rollout_dt), np.sin(rollout_dt)], [-np.sin(rollout_dt), np.cos(rollout_dt)], ]) full_state = np.array([1.0, 0.0]) full_state_rollout = np.empty(horizon) for t in range(horizon): full_state_rollout[t] = full_state[0] full_state = rotation @ full_state fig, axes = plt.subplots(1, 2, figsize=(13, 4.8)) fig.suptitle('Missing state information compounds during rollout', fontsize=14, fontweight='bold') axes[0].plot(rollout_time, true_position, color='black', linewidth=2.2, label='true trajectory') axes[0].plot(rollout_time, position_only_rollout, '--', color='#e76f51', linewidth=2.0, label='position-only rollout') axes[0].plot(rollout_time, full_state_rollout, color='#2a9d8f', linewidth=1.7, label='position + velocity rollout') axes[0].set_title(f'Autonomous rollout (one-step MSE = {one_step_mse:.4f})') axes[0].set_xlabel('Time') axes[0].set_ylabel('Position') axes[0].legend() axes[1].plot(rollout_time, np.abs(position_only_rollout - true_position), color='#e76f51', linewidth=2.0, label='position-only error') axes[1].plot(rollout_time, np.abs(full_state_rollout - true_position), color='#2a9d8f', linewidth=1.7, label='full-state error') axes[1].set_title('Absolute error across the rollout horizon') axes[1].set_xlabel('Time') axes[1].set_ylabel('Absolute position error') axes[1].legend() plt.tight_layout() plt.show()

What you see¶

1. What the plots show: The position-only model appears reasonable for one step but decays toward zero when recursively rolled out. The full-state rotation remains on the true periodic trajectory, up to numerical precision.
2. Why it looks this way: Averaging over hidden velocities gives a contraction coefficient. Reapplying that average destroys phase and compounds the representation error at every step.
3. Key takeaway: Multi-step rollout is a stronger state-quality test than one-step prediction because the model must live with its own compressed state.

---

Summary¶

Experiment	Key insight
1. Observation aliasing	The same observation can hide states with incompatible futures.
2. State augmentation	Adding the missing dynamical variable can restore near-deterministic prediction.
3. History ablation	Residual predictive value in history is evidence that the proposed state is insufficient.
4. Belief filtering	A posterior belief recursively compresses noisy history under partial observability.
5. Rollout drift	Small one-step error can hide severe long-horizon failure when state information is missing.

Connection to Latent-Anything¶

A Trajectory should contain states that are sufficient for the operations applied to them, not merely per-frame embeddings. Model adapters and diagnostics therefore need to expose history dependence, uncertainty semantics, action conditioning, and multi-step rollout behavior before a latent can be trusted for planning.