# Learning Video Generation from the Wan2.2 Source Code: From One Prompt to 81 Frames at 720p

One command:

python generate.py --task t2v-A14B --size 1280*720 \
  --ckpt_dir ./Wan2.2-T2V-A14B --prompt "Two anthropomorphic cats boxing"

A few minutes later, you get an 81-frame, 720p, 16fps video. This article explains every step that happens in between. All code comes from the original Wan2.2 repository, and every number can be calculated from the configuration files. I include the formulas as well. After reading, you should be able to answer these questions:

• Why must the frame count be 4n+1? Why won’t 80 frames work?
• How is the 14B parameter count derived from the configuration? How is the 80GB VRAM threshold calculated?
• How does flow matching differ from DDPM mathematically, and why can the former produce a video in 40 steps?
• What knob does --sample_shift 12 turn, and what invisible interaction does it have with MoE expert switching?
• In image-to-video, exactly which lines are changed on top of the text-to-video backbone?

This is not a short article. You can jump by section:

1. VAE: How video is compressed by 46×, where 4n+1 comes from, causal convolution, and streaming inference
2. Text Encoding: umT5, negative prompts, and the prompt expander
3. DiT Backbone: 75,600 tokens, how the 14B parameter count is calculated, AdaLN, 3D RoPE, QK-Norm, and a full shape ledger
4. Flow Matching: Why adding and removing noise can generate content, the mathematics of the straight-line path, UniPC, and the accounting behind shift
5. CFG: The Bayesian derivation of two forward passes, and the role of negative prompts
6. Dual-Expert MoE: The interaction between boundary and shift, the full sampling loop, and how it differs from LLM MoE
7. Condition Injection: Which lines image-to-video changes, and the three techniques shared by five tasks
8. Parallelism: The FLOPs accounting for attention, Ulysses, FSDP, and a VRAM comparison table
9. Failure Modes: Fingers, drift, stiffness, and mesh penetration, with mechanism-level attribution for each
10. Tuning Quick Reference + papers, hands-on path, and glossary

Three Models, One Stage

Video generation is not done by a single model, but by three:

The division of labor is clear: T5 runs only once at the beginning; the VAE encoder is not used at all in t2v because there is no input image; the decoder runs only once at the end. The real compute burner is the diffusion backbone in the middle: it runs for 40 steps, with two forward passes per step.

How the three models are wired together is scripted in generate() in wan/text2video.py: encode text → sample noise → denoising loop → decode into pixels. It is worth memorizing the skeleton of this script first; every later chapter expands on one of its lines.

First, let’s answer a prerequisite question: why doesn’t diffusion operate directly on pixels? Do the math. An 81-frame 720p RGB video has

81 × 720 × 1280 × 3 ≈ 224 million values

After VAE compression, the latent contains only

16 × 21 × 90 × 160 ≈ 4.84 million values

That is 46× fewer. The diffusion backbone has to repeatedly compute over this data for 80 forward passes. When the data is 46× smaller, the cost of attention and convolution collapses along with it. Generating in the compressed space and decoding back to pixels only at the final step is called latent diffusion. It became standard starting with Stable Diffusion, and video generation extends it from 2D to 3D.

Chapter 1: VAE: The Video Is First Shrunk by 46×

Two config lines in wan/configs/wan_t2v_A14B.py determine the shape of every tensor across the entire pipeline:

t2v_A14B.vae_stride = (4, 8, 8)    # time ×4, height ×8, width ×8
t2v_A14B.patch_size = (1, 2, 2)    # DiT patch size

The time dimension is compressed by 4×, and each spatial dimension by 8×. text2video.py uses this to compute the latent shape:

target_shape = (self.vae.model.z_dim, (F - 1) // self.vae_stride[0] + 1,
                size[1] // self.vae_stride[1],
                size[0] // self.vae_stride[2])

Plugging in F=81 and 720×1280 gives (16, 21, 90, 160): 16 latent channels, 21 "latent frames", and a 90×160 spatial grid.

Where 4n+1 Comes From

Notice that the formula for the time dimension is (F - 1) // 4 + 1, not F // 4. The first frame of the video is encoded separately as one latent frame; after that, every 4 frames are compressed into one. 81 frames = 1 + 80, and 80 is exactly divisible by 4, giving 1 + 20 = 21 latent frames. If you pass in 80 or 82 frames, the division does not come out cleanly, the tensor shapes fail to match, and things crash immediately. The requirement that frame_num must be 4n+1 originates from this line.

Why is the first frame special? Because the compression uses causal convolution. The first frame has no history before it, so it can only stand alone. This design also has a side effect: a static image can be viewed as a video with F=1, occupying exactly one latent frame. Images and videos are therefore represented uniformly inside the same VAE, which will matter later when discussing image-to-video.

Causal Convolution and Streaming Inference

Look at the implementation of CausalConv3d in vae2_1.py; it is only a dozen or so lines:

class CausalConv3d(nn.Conv3d):

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self._padding = (self.padding[2], self.padding[2], self.padding[1],
                         self.padding[1], 2 * self.padding[0], 0)
        self.padding = (0, 0, 0)

    def forward(self, x, cache_x=None):
        padding = list(self._padding)
        if cache_x is not None and self._padding[4] > 0:
            cache_x = cache_x.to(x.device)
            x = torch.cat([cache_x, x], dim=2)
            padding[4] -= cache_x.shape[2]
        x = F.pad(x, padding)
        return super().forward(x)

A normal 3D convolution pads symmetrically before and after the time dimension, so each frame’s output can "see" future frames. Here, all temporal padding is moved to the front (2 * self.padding[0], 0: double padding before, zero after), so each frame’s output depends only on the current frame and the past.

Causality enables streaming processing: a video can be split into small chunks and passed through the network sequentially. After each chunk is processed, the last 2 frames (CACHE_T = 2 at the top of the source file) are cached and prepended as cache_x for the next chunk. Mathematically, this is equivalent to convolving the whole video at once. As a result, the VAE’s memory usage is decoupled from the total video length: a 5-second video and a 50-second video have the same peak memory usage in this part of the VAE. The padding[4] -= cache_x.shape[2] line inside forward is just bookkeeping: however much padding the cache replaces, that much less zero-padding is needed.

What Is Inside the Encoder

The _video_vae config sketches the entire network:

cfg = dict(
    dim=96,
    z_dim=z_dim,
    dim_mult=[1, 2, 4, 4],
    num_res_blocks=2,
    attn_scales=[],
    temperal_downsample=[False, True, True],
    dropout=0.0)

It is a standard hierarchical convolutional autoencoder, from the same family as Stable Diffusion’s VAE—the source comments say as much. The base channel count is 96, expanded across four stages via dim_mult as 96→192→384→384. Each stage has two residual blocks. Spatial downsampling happens once between stages, three times in total, corresponding to 8× spatial compression. temperal_downsample=[False, True, True] defines the time-axis schedule: the first stage leaves time unchanged, while the next two stages each halve it, giving 4× temporal compression in total. This is how the 4×8×8 compression ratio is distributed structurally. At the bottleneck there is one spatial self-attention block, adding a bit of global context to the most compressed representation; everything else is convolution. The decoder climbs back up the same staircase in reverse.

Compared with the diffusion backbone, this network is tiny enough to barely matter in the parameter budget—hundreds of millions of parameters versus 14B. But it determines the ceiling of generation quality: details the VAE cannot reconstruct cannot reach the pixels no matter how well the diffusion model sculpts them. For video VAEs, reconstruction quality, compression ratio, and decoding speed form a three-way tradeoff; this is one of the hidden battlegrounds among video models.

Two Versions of the VAE

The Wan2.2 repo contains two VAEs, and they are not interchangeable:

The VAE for ti2v-5B compresses more aggressively—16× spatially. At the same 720p resolution, the latent grid is smaller. This is key to allowing the smaller 5B model to run 121 frames at 24 fps on a 24GB consumer GPU: halving the model size is not enough; the data has to be smaller too. The cost is that reconstruction becomes harder, so the channel count is raised from 16 to 48 to compensate for information capacity.

The "Exchange Rate" of Latent Space

The VAE and the diffusion model are two separately trained systems, connected by a set of hard-coded statistics. In vae2_1.py:

mean = [
    -0.7571, -0.7089, -0.9113, 0.1075, -0.1745, 0.9653, -0.1517, 1.5508,
    0.4134, -0.0715, 0.5517, -0.3632, -0.1922, -0.9497, 0.2503, -0.2921
]
std = [
    2.8184, 1.4541, 2.3275, 2.6558, 1.2196, 1.7708, 2.6052, 2.0743,
    3.2687, 2.1526, 2.8652, 1.5579, 1.6382, 1.1253, 2.8251, 1.9160
]
self.scale = [self.mean, 1.0 / self.std]

The 16 channels each have their own mean and standard deviation, computed over the training set. The raw outputs of the VAE encoder vary greatly in scale across channels—looking at std, the smallest is 1.13 and the largest is 3.27, a threefold difference. The diffusion model assumes the data it sees is close to a standard normal distribution, so the latents must be standardized channel by channel before entering the diffusion model, then multiplied back before decoding. These numbers are the "exchange rate" between this VAE and this diffusion model. Swap in another VAE and all 32 numbers become invalid. This is the root reason the checkpoints of the two VAE versions cannot be mixed.

The VAE’s own training objective is "compress and then decompress as faithfully as possible": reconstruction loss as the base, KL regularization to keep the latent distribution well behaved, and adversarial loss from a GAN discriminator to prevent blurry reconstructions. Once trained, it is frozen. The diffusion model works entirely within the latent space defined by this VAE from beginning to end. This decoupling has a practical advantage: the same VAE can serve many diffusion models. The Wan2.1 VAE is directly reused by four Wan2.2 tasks.

Chapter 2: Where Did the Prompt Go?

umT5-XXL is Google’s multilingual T5 variant, and Wan2.2 uses only its encoder. Its parameter count can be estimated from the configuration in t5.py:

cfg = dict(
    vocab_size=256384, dim=4096, dim_ffn=10240,
    num_heads=64, encoder_layers=24, ...)

Each layer has 4 attention matrices of size 4096×4096, plus 3 gated FFN matrices of size 4096×10240. T5’s FFN has a gate, so there are three matrices rather than two. Across 24 layers, plus the embedding table for a 256k-token vocabulary, the total is about 5.7B parameters, requiring roughly 11.4GB in bf16.

The encoding call path is very short. In t5.py, T5EncoderModel.__call__ is straightforward:

def __call__(self, texts, device):
    ids, mask = self.tokenizer(
        texts, return_mask=True, add_special_tokens=True)
    seq_lens = mask.gt(0).sum(dim=1).long()
    context = self.model(ids, mask)
    return [u[:v] for u, v in zip(context, seq_lens)]

Tokenize, encode, then trim the padding for each item according to the true length from the attention mask. The prompt becomes a sequence of up to 512 vectors, each 4096-dimensional. This is the context. It has exactly one use inside the DiT: in each layer, cross-attention takes video tokens as queries and the context as keys/values. Text does not directly draw any pixels; it continuously gives the model semantic direction at every denoising step.

Why T5 Instead of CLIP?

Early text-to-image models, such as SD 1.x/2.x, used CLIP’s text tower to encode prompts. CLIP is trained on image-text matching. Its text vectors are good at capturing “roughly what this sentence is about,” but weak at representing long sentences, quantifiers, and spatial relationships. Its 77-token length limit also cannot fit detailed scene descriptions. T5 is a pure language model: it encodes text token by token according to the structure of the text itself. With a 512-token limit and per-token cross-attention, the model can distinguish details like “the cat on the left is wearing red boxing gloves.” Starting with Imagen, large T5 models gradually replaced CLIP as the text backbone for generative models. Wan2.2’s choice of the multilingual umT5 has another consideration: Chinese prompts are first-class citizens, and the official negative prompt is simply written in Chinese.

One detail worth knowing: umT5’s relative positional encoding is independent per layer (shared_pos=False in the config). Each of the 24 layers computes its own position bias. Standard T5 computes it once and shares it across all layers. This is a design tradeoff in the multilingual model, and it is easy to misread as a bug when reading the code.

A negative prompt is encoded at the same time. The default value in the config is a long Chinese string, shown here in the original:

wan_shared_cfg.sample_neg_prompt = '色调艳丽，过曝，静态，细节模糊不清，字幕，风格，作品，画作，画面，静止，整体发灰，最差质量，低质量，JPEG压缩残留，丑陋的，残缺的，多余的手指，画得不好的手部，画得不好的脸部，畸形的，毁容的，形态畸形的肢体，手指融合，静止不动的画面，杂乱的背景，三条腿，背景人很多，倒着走'

It is a checklist of “common failure modes.” Its role will be covered in the chapter on CFG. For now, just remember this: the negative prompt also goes through T5, producing context_null.

What If the Prompt Is Too Short? The Prompt Extender

User-written prompts are often just a single sentence, while the model is trained on long, detailed descriptions. Feeding it a prompt that is too short can reduce output quality. --use_prompt_extend enables a preprocessing step: first, an LLM expands your prompt into a longer description with camera, lighting, motion, and visual details, then the result is sent to T5. Two backends are available: DashScope’s online API (qwen-plus) or a locally loaded Qwen2.5. The system prompt used for expansion is selected from system_prompt.py according to the task and language.

There is one engineering detail in multi-GPU setups: expansion runs only once on rank 0, and the result is broadcast to the other GPUs with dist.broadcast_object_list. LLM generation is stochastic. If 8 GPUs each expanded the prompt independently, they would get 8 different prompts, and the video could not be stitched together properly. Broadcasting ensures that all GPUs use the same one.

Chapter 3 DiT: Attention Over 75,600 Tokens

Before entering the backbone, it is worth answering a more fundamental question: what is the essential difference between a video model and an image model?

The answer is not as simple as “one more dimension.” If you call an image model frame by frame to generate 81 images, each frame may look beautiful on its own, but playing them together is a disaster: object shapes drift from frame to frame, lighting jumps, backgrounds flicker. In industry jargon, this is called flicker. Temporal consistency must be explicitly modeled by the model. The industry has gone down two paths: factorized attention (spatial attention and temporal attention are separated; first make each frame internally coherent, then align along the time axis; cheaper, but consistency has a ceiling) and full 3D joint attention (all patches from all frames are placed into the same attention operation, so any token can directly see any position at any moment; strongest consistency, but sequence length explodes). Wan2.2 takes the second path, and this choice directly leads to the protagonist of this chapter: a sequence of 75,600 tokens.

From Latents to Tokens

Before the latents enter the DiT, they are first split into patches. In model.py, this is simply a Conv3d whose stride equals its kernel size:

self.patch_embedding = nn.Conv3d(
    in_dim, dim, kernel_size=patch_size, stride=patch_size)

patch_size = (1, 2, 2): the time dimension is not split, while every 2×2 latent block in space is convolved into a 5120-dimensional token. Counting the total:

tokens per latent frame = 90 × 160 / (2×2) = 3600
total tokens            = 3600 × 21 frames  = 75600

75,600 is the self-attention sequence length when generating a 5-second 720p video. SDXL generating a single 1024 image has a sequence length of 4096, so video is 18 times longer than image. Attention compute grows quadratically with sequence length; 18× the length means about 340× the attention cost. All the VRAM issues and sequence parallelism discussed later ultimately trace back to this number.

How 14B Is Counted

The config gives dim=5120, ffn_dim=13824, num_layers=40. The main parameter counts per layer are:

self-attention:  four Q/K/V/O matrices = 4 × 5120² ≈ 105M
cross-attention: same four matrices     = 4 × 5120² ≈ 105M
FFN:             two matrices           = 2 × 5120 × 13824 ≈ 142M
per layer total ≈ 351M, × 40 layers ≈ 14.05B

The 14B in “A14B” comes from this. Embeddings and modulation parameters are only rounding errors. bf16 uses 2 bytes per parameter, so one expert is 28GB; t2v-A14B has two experts (the next chapter explains why), so the DiT weights alone are 56GB. Add T5’s 11.4GB, the VAE, and activations during sampling, and this is how the 80GB single-GPU threshold gets filled up.

The Full Forward Pass of One Block

WanAttentionBlock.forward is the smallest repeating unit of the whole model, and it is worth reading line by line:

def forward(self, x, e, seq_lens, grid_sizes, freqs, context, context_lens):
    with torch.amp.autocast('cuda', dtype=torch.float32):
        e = (self.modulation.unsqueeze(0) + e).chunk(6, dim=2)

    # self-attention
    y = self.self_attn(
        self.norm1(x).float() * (1 + e[1].squeeze(2)) + e[0].squeeze(2),
        seq_lens, grid_sizes, freqs)
    with torch.amp.autocast('cuda', dtype=torch.float32):
        x = x + y * e[2].squeeze(2)

    # cross-attention & ffn
    def cross_attn_ffn(x, context, context_lens, e):
        x = x + self.cross_attn(self.norm3(x), context, context_lens)
        y = self.ffn(
            self.norm2(x).float() * (1 + e[4].squeeze(2)) + e[3].squeeze(2))
        with torch.amp.autocast('cuda', dtype=torch.float32):
            x = x + y * e[5].squeeze(2)
        return x

    x = cross_attn_ffn(x, context, context_lens, e)
    return x

There are three sections: Self-Attention (75,600 tokens looking at one another to establish spatiotemporal consistency), Cross-Attention (looking at the text context to inject semantics), and FFN. The truly interesting part is e, which is chunked into 6 parts:

Where do these 6 groups of parameters come from? Trace the full journey of timestep t. t is a scalar (for example 927), and first passes through sinusoidal encoding:

def sinusoidal_embedding_1d(dim, position):
    half = dim // 2
    position = position.type(torch.float64)
    sinusoid = torch.outer(
        position, torch.pow(10000, -torch.arange(half).to(position).div(half)))
    x = torch.cat([torch.cos(sinusoid), torch.sin(sinusoid)], dim=1)
    return x

This is the same family as Transformer positional encoding: a set of frequencies from high to low, expanding the scalar t into a 256-dimensional cos/sin waveform. Why not feed t directly into the network as a number? Because neural networks have poor resolution over a single scalar. Sinusoidal expansion makes both “the difference between t=927 and t=925” and “the difference between t=900 and t=100” land at scales the network can easily perceive: high-frequency channels handle fine-grained detail, while low-frequency channels handle coarse-grained structure.

Then it goes through an MLP and a projection:

self.time_embedding = nn.Sequential(
    nn.Linear(freq_dim, dim), nn.SiLU(), nn.Linear(dim, dim))
self.time_projection = nn.Sequential(nn.SiLU(), nn.Linear(dim, dim * 6))

dim * 6 corresponds exactly to those 6 modulation parts. This technique is called AdaLN (adaptive layer norm), one of the core contributions of the DiT paper: instead of concatenating t into the sequence as a token, let t modulate the scale, shift, and residual gates of normalization in every layer. Intuitively, when t=999, the input is almost pure noise, so the network should make broad, aggressive changes; when t=10, the image is mostly formed, so the network should only make small refinements. The same set of weights switches behavior modes through these 6 groups of t-dependent parameters. The DiT paper compared “concatenate t as a token” against “AdaLN modulation,” and the latter achieved significantly better FID under the same compute budget. Since then, it has become the default pattern for diffusion transformers.

There is another easy-to-miss detail in the code: all modulation operations are wrapped in autocast float32. The backbone can run in bf16, but these modulation steps are forced to full precision. Denoising is an iterative process over dozens of steps, and numerical errors in the modulation parameters can accumulate over time. This is where the authors spend VRAM to buy stability.

3D RoPE: Letting Tokens Know Where They Are

75,600 tokens are arranged into a one-dimensional sequence, and attention itself does not know who is next to whom. Positional information is injected through RoPE (rotary positional embeddings). The special part in the video version is that coordinates have three axes. During construction, each head’s 128 dimensions are split into three segments:

d = dim // num_heads          # 5120 // 40 = 128
self.freqs = torch.cat([
    rope_params(1024, d - 4 * (d // 6)),   # time axis: 44 dims
    rope_params(1024, 2 * (d // 6)),       # height axis: 42 dims
    rope_params(1024, 2 * (d // 6))        # width axis: 42 dims
], dim=1)

When applying it, rope_apply concatenates the rotation values for the three coordinates (f, h, w) into a complete 128-dimensional rotation, then applies it in one shot with complex multiplication:

freqs_i = torch.cat([
    freqs[0][:f].view(f, 1, 1, -1).expand(f, h, w, -1),
    freqs[1][:h].view(1, h, 1, -1).expand(f, h, w, -1),
    freqs[2][:w].view(1, 1, w, -1).expand(f, h, w, -1)
], dim=-1).reshape(seq_len, 1, -1)

x_i = torch.view_as_real(x_i * freqs_i).flatten(2)

Three expand calls broadcast the 1D frequency tables across the entire (f, h, w) grid: tokens at different positions in the same frame share the same rotation for the temporal segment and differ in the spatial segments; tokens at the same position in different frames are the opposite. When attention computes the q·k inner product, the relativity of the rotations makes the inner product depend only on the coordinate difference between the two tokens, so translation invariance comes for free. This is a direct extension of image 2D RoPE to video, and the 44/42/42 split means the temporal axis gets roughly the same number of frequency channels as each individual spatial axis, so temporal and spatial relationships are treated on equal footing.

One detail: rope_apply runs entirely in float64 and explicitly disables autocast (@torch.amp.autocast('cuda', enabled=False) on the function header). Rotation is a purely geometric operation; phase error from half precision gets amplified on long sequences. This is another tradeoff where extra compute is spent to buy numerical stability.

QK-Norm

In WanSelfAttention, q and k each pass through RMSNorm after their linear layers:

q = self.norm_q(self.q(x)).view(b, s, n, d)
k = self.norm_k(self.k(x)).view(b, s, n, d)

This has become a standard trick in large-model training over the past couple of years: when the variance of the q·k inner product is uncontrolled, softmax saturates, attention entropy collapses, and training diverges. Normalizing q and k separately locks down the scale of the inner product. Its value is not visible from inference code alone, but without it, a 14B-scale DiT is hard to train stably.

The model exit is a Head with 2 groups of modulation parameters—shift/scale only, no gate. It projects the 5120-dimensional features back to patch_size volume × 16 dimensions, then unpatchify folds them back into the latent shape.

Shape bookkeeping for one forward pass

Putting the whole chapter into one table, here is every shape transformation from model input to model output (t2v-A14B, 81 frames, 720p, single GPU):

The model takes in a shape and outputs the same shape; in between are 40 equal-width refinement layers. The semantics of the output are “velocity” (expanded in the next chapter), corresponding pointwise to the input latent: each position’s value indicates “which direction the content at this position should move.” The text context (512 × 4096, projected by text_embedding to 512 × 5120) exists only as cross-attention key/value throughout, and does not occupy sequence positions.

At this point, the machine that “takes a noisy latent and a timestep t, then emits a same-shaped prediction” is complete. What exactly it predicts is the topic of the next chapter.

Chapter 4: Denoising Follows a Straight Line

Why “Add Noise, Then Denoise” Can Generate

First, let’s add one layer of intuition. The problem a generative model needs to solve is: sample a new example from the “distribution of natural videos.” This distribution has no analytic form, so there is no obvious way to sample from it directly. Diffusion-style methods solve this by building a road: one end is natural data, the other end is standard Gaussian noise, which is easy to sample. Then they train a network to walk back along this road from the noise end to the data end. The noising process builds the road, connecting a complex distribution to a simple one; the denoising network learns, at every position on this road, “which direction should I move to get toward the data?” During training, the network sees massive numbers of “position → direction” examples. During inference, it starts from random noise and follows the learned direction field. The endpoint is a new sample that looks like the training data, but is not equal to any specific training sample. The prompt participates in every direction judgment through cross-attention, constraining the endpoint to the region that “matches this description.”

From DDPM to Flow Matching

Classic DDPM defines this road as follows: the forward process gradually adds Gaussian noise according to a carefully designed variance schedule; the reverse process trains a network to predict the noise added at each step; during sampling, that noise is peeled away step by step according to Bayes’ rule. It works, but the path is a curve produced by a random walk, sampling needs hundreds of steps, and the variance schedule itself is a hyperparameter that has to be tuned.

Flow matching — Wan2.2 uses its simplest form, rectified flow — makes the problem much more direct: between clean data x₀ and pure noise x₁, define the intermediate state by straight-line interpolation:

x_t = (1 - t)·x₀ + t·x₁ ,  t ∈ [0, 1]

Along this straight line, the “velocity” at any time is constant:

dx_t/dt = x₁ - x₀

So the training objective becomes simple enough to write as four lines of pseudocode:

x0 = sample the latent of a real video clip
x1 = sample standard Gaussian noise
t  = uniformly sample a time ∈ [0, 1]
loss = || model((1-t)·x0 + t·x1, t, text) - (x1 - x0) ||²

No variance schedule, no ELBO derivation — just regress a vector field. The repository only contains inference code, but the training loop essentially looks like this. The model’s entire capability comes from repeating these four lines over hundreds of millions of video clips.

During inference, we walk backward: start from pure noise x₁, numerically integrate along the velocity predicted by the network, and integrate back to x₀. Ideally, the trajectory is a straight line and one step would be enough; in reality, the network’s velocity field is not perfect, and after taking one step, the true velocity corresponding to the new position has already changed. So we still need to move in multiple steps, asking for directions after each step. But the straight-line prior creates far fewer detours than DDPM’s random walk: 40 steps can produce a video, whereas in the DDPM era the same quality often required hundreds of steps.

You can see this math directly in the code. The core line in fm_solvers_unipc.py is:

if self.config.prediction_type == "flow_prediction":
    sigma_t = self.sigmas[self.step_index]
    x0_pred = sample - sigma_t * model_output

The derivation is short: write t in the interpolation formula as σ. Then x_t = (1-σ)x₀ + σx₁, and the model outputs v = x₁ - x₀. Therefore:

x_t - σ·v = (1-σ)x₀ + σx₁ - σx₁ + σx₀ = x₀

The current sample minus σ times the predicted velocity is exactly the estimate of the clean data. One line of code, one line of algebra.

UniPC: How to Move Fast and Steadily

Once we have a velocity field, we still need a numerical integrator. The simplest option is Euler’s method:

x ← x - Δσ · v(x, σ)

It is first-order accurate, assuming the velocity stays constant across the whole step. But the actual velocity field is curved. The larger the step, the more the straight-line approximation deviates, and the error accumulates linearly with step size. Either you take many small steps, which is slow, or you use a smarter integrator.

A mature idea from numerical analysis is to use history: the trend of how velocity changed over the last few steps can be extrapolated to estimate how it will bend next. Wan2.2’s default UniPC is a multistep predictor-corrector. Each step has two actions:

• predictor: uses the model outputs from the most recent solver_order steps to fit the velocity change, then extrapolates the next step forward; this is similar to an Adams multistep method. With solver_order=2, the actual accuracy reaches third order.
• corrector: after the model output at the new position is computed, it uses that output to revise the step just taken, gaining one extra order of accuracy without an additional forward pass.

Compare the two: with the same 40 model calls, Euler gives you 40 first-order steps, while UniPC makes each step’s local error a third-order small quantity. In engineering terms, it cuts the step budget for “acceptable quality” from hundreds down to dozens. Video generation requires two 14B forward passes per step, so every saved step is real money. This is also why inference frameworks keep adding longer sampler lists: without changing the model, swapping the integrator gives you speed for free.

The scheduler is also responsible for deciding where the 40 sampling points land. set_timesteps first places points uniformly, then applies a shift transform:

sigmas = shift * sigmas / (1 + (shift - 1) * sigmas)

This transform squeezes points toward the high-noise end, where σ is large. How much squeezing? We can calculate it directly. For 40 steps, counting the two boundaries t≥875 and t≥500:

With shift=12, half of the sampling points are squeezed into the region where t>927, and 37 steps are spent in the first half of the process. The real timestep sequence begins like this with shift=12 and 40 steps; you can verify it yourself by running set_timesteps:

1000.0, 997.9, 995.6, 993.3, 990.8, 988.2, ...

The spacing in the first few steps is only around 2, so the sampler moves very densely there. By the low-noise stage, the spacing expands to dozens, and the remaining interval is crossed in just a few steps.

It is also worth clarifying the relationship between the numbers 1000 and 40, which beginners often mix up: during training, the time axis is discretized into 1000 ticks, num_train_timesteps=1000, and the model sees samples across all ticks. During inference, we simply choose 40 stopping points from this ruler, and the model works as usual at each selected point. The number of steps is a free sampling-time parameter. Changing --sample_steps does not require changing the model. 50 steps and 20 steps are both valid; they just give different integration accuracy. This is also the entry point for “distillation acceleration” work: since the model takes a step at every tick, can we train it to take a much longer step and compress 40 steps into 4? This is exactly what few-step distillation in the industry is trying to do.

Why allocate steps this way? The high-noise stage determines composition, objects, and motion direction; it is the “something from nothing” stage. The low-noise stage mostly polishes details. Tilting the budget toward the harder stage is a tradeoff video generation needs more than image generation: image shift is usually around 3, but motion consistency in video is much harder, so Wan2.2 raises it directly to 12. In the same repository, i2v uses 5.0 because the first frame is already given and structural pressure is lower; ti2v-5B uses 5.0; s2v uses 3. Each task’s shift tells its own difficulty distribution.

--sample_shift does not change the number of steps. It only changes how those steps are distributed. Tuning it means moving the budget between “more stable structure” and “finer details.”

Chapter 5: Why Each Step Runs Two Forward Passes

The core of the sampling loop (text2video.py):

noise_pred_cond = model(latent_model_input, t=timestep, **arg_c)[0]
noise_pred_uncond = model(latent_model_input, t=timestep, **arg_null)[0]

noise_pred = noise_pred_uncond + sample_guide_scale * (
    noise_pred_cond - noise_pred_uncond)

The same latent, the same t, run twice: once with your prompt (arg_c contains context), and once with the negative prompt (arg_null contains context_null). This is classifier-free guidance (CFG).

The principle is worth expanding with a bit of math. To generate “samples that satisfy condition c,” decompose it with Bayes’ rule:

p(x|c) ∝ p(x) · p(c|x)

Take the log and compute the gradient (score):

∇log p(x|c) = ∇log p(x) + ∇log p(c|x)

The first term is the unconditional direction — what the data itself looks like. The second term is the classifier gradient that “makes x look more like it belongs to condition c.” Early methods trained a dedicated noise-robust classifier to provide this second term (classifier guidance), which was both expensive and hard to train. CFG’s insight is that this term can be obtained for free: the model has learned both conditional and unconditional predictions at the same time (by randomly dropping text during training), so subtracting the two gives:

∇log p(c|x) = ∇log p(x|c) - ∇log p(x) ≈ conditional prediction - unconditional prediction

The classifier gradient is hidden inside the difference between the two forward passes. During sampling, this term is amplified by a factor of w:

final direction = uncond + w · (cond - uncond)

Compare that with the three lines of code above: it is exactly the same. sample_guide_scale is w.

How to choose w is purely a tradeoff. At w = 1, it degenerates into conditional generation: the image is natural, but it often “doesn’t understand” the prompt. As w increases, the model is pushed harder toward regions where the “classifier is more confident,” improving compositional obedience at the cost of oversaturated colors, smeared details, and stiff motion, because the latent is pushed away from high-density regions of the training distribution. Image models often use values around 7; video is more sensitive to temporal naturalness, so Wan2.2 only uses 3.0–4.0.

The place of the negative prompt in this derivation is also clear: standard CFG uses empty text for the unconditional branch, while Wan2.2 replaces it with that description of “overexposure, deformation, three legs,” and so on. The difference direction changes from “move closer to your prompt” into “move closer to your prompt while moving away from these flaws,” doing two things in one forward pass.

The role of the negative prompt in this mechanism is also clear: the “uncond” forward pass does not use empty text, but the string describing “low quality, overexposure, deformation,” and so on. The difference direction becomes “move away from these flaws and toward your description,” killing two birds with one stone.

The cost is just as clear: 40 steps × 2 forward passes = 80 full inference passes through the 14B model. Video generation is slow, and half the bill is due to CFG. Some later models use guidance distillation to merge the two forward passes into one. Wan2.2 does not; it simply runs both passes honestly.

Chapter 6: Two 14B Experts Taking Turns

The signature design of the Wan2.2 A14B series is that two 14B experts divide the work across different noise ranges. In the config:

t2v_A14B.boundary = 0.875
t2v_A14B.sample_guide_scale = (3.0, 4.0)  # low noise, high noise

At each step, the sampling loop checks t >= 0.875 × 1000: if true, it uses high_noise_model; otherwise it uses low_noise_model. The motivation follows from the observation in the previous chapter: the high-noise and low-noise phases are doing extremely different jobs. The former determines structure from chaos; the latter polishes textures on a half-finished result. Instead of forcing one network to master both tasks, it is better to let two networks specialize in one each. During training, the two experts are also trained separately on their respective noise ranges; during inference, they take turns according to t.

This is what the official phrase “27B total parameters, 14B active” means: capacity is doubled, but only one expert runs at any given step, so the compute cost equals that of a 14B dense model. The term MoE usually refers to token-level routing, where each token selects a different FFN expert. Wan2.2 uses timestep-level routing instead. The granularity is much coarser, but it avoids a router entirely; the implementation is just one if.

Here is the complete sampling loop. All the pieces discussed in the previous chapters come together in these 20 lines:

for _, t in enumerate(tqdm(timesteps)):
    latent_model_input = latents
    timestep = torch.stack([t])

    model = self._prepare_model_for_timestep(
        t, boundary, offload_model)
    sample_guide_scale = guide_scale[1] if t.item(
    ) >= boundary else guide_scale[0]

    noise_pred_cond = model(
        latent_model_input, t=timestep, **arg_c)[0]
    noise_pred_uncond = model(
        latent_model_input, t=timestep, **arg_null)[0]

    noise_pred = noise_pred_uncond + sample_guide_scale * (
        noise_pred_cond - noise_pred_uncond)

    temp_x0 = sample_scheduler.step(
        noise_pred.unsqueeze(0), t,
        latents[0].unsqueeze(0),
        return_dict=False, generator=seed_g)[0]
    latents = [temp_x0.squeeze(0)]

Each round does five things: choose the expert, choose the guide scale, run two forward passes, combine them with CFG, and let the scheduler take one step. _prepare_model_for_timestep also contains the offload logic that saves single-GPU setups:

if offload_model or self.init_on_cpu:
    if next(getattr(self, offload_model_name).parameters()).device.type == 'cuda':
        getattr(self, offload_model_name).to('cpu')
    if next(getattr(self, required_model_name).parameters()).device.type == 'cpu':
        getattr(self, required_model_name).to(self.device)

With --offload_model True, whenever the expert switches, the off-duty one is moved back to system RAM and the on-duty one is moved onto the GPU. Moving 28GB of weights across PCIe takes several seconds, but fortunately the whole sampling process only switches once: t decreases monotonically, and once it crosses the boundary, it never goes back. So this cost is paid only once.

boundary and shift are tightly coupled, and this is the easiest interaction to miss when tuning. 0.875 looks like “only 12.5% of timesteps go to the high-noise expert,” but as the table in the previous chapter showed, shift=12 aggressively squeezes sampling points toward the high-noise end. In practice, t ≥ 875 covers 26 steps, or 65%. Most of the budget goes to the high-noise expert that determines structure, while the low-noise expert uses the remaining 14 steps to finish the result. If you tune --sample_shift in isolation without thinking about boundary, the workload split between the two experts silently changes, and the resulting quality fluctuations can be baffling.

The two guide_scale values also serve this division of labor: the high-noise phase uses 4.0, keeping the structural stage close to the prompt; the low-noise phase drops to 3.0, applying less force during the detail stage so the image looks more natural.

Recalculating the VRAM bill: two bf16 experts take 56GB of weights, the T5 encoder takes 11.4GB, plus the VAE, activations, KV cache, and so on. An 80GB card can just barely fit everything. When a single card cannot run it, the three-piece survival kit maps directly to entries on that bill: --offload_model True moves the currently unused expert back to RAM, cutting 28GB of resident VRAM; --t5_cpu keeps T5 in system RAM and only moves it onto the GPU during encoding, cutting 11.4GB; --convert_model_dtype lowers weight precision.

How should these three VRAM-saving options be combined? The README examples give the official answer: on an 80GB card, run A14B without adding anything; on a single consumer card, run A14B with --offload_model True --convert_model_dtype; on a 4090 running ti2v-5B, enable all three, including --t5_cpu, because in the 5B scenario, T5’s 11.4GB becomes the largest relative share. The README also gives the reverse hint: if you have more than 80GB of VRAM, remove these flags for a noticeable speedup, because offload is not free.

Finally, let’s clear up a common misunderstanding: only one expert appears at each timestep. The two forward passes are the CFG conditional/unconditional passes; the experts never work simultaneously at the same step. “Two forward passes per step” and “two experts” are orthogonal ideas.

Is This the Same as MoE in LLMs?

No. It is worth remembering the contrast:

The only thing they share is the marketing line: large total parameter count, small active parameter count. Wan2.2’s coarse-grained version sacrifices routing flexibility in exchange for two benefits: no router needs to be trained, because diffusion timesteps are already natural difficulty labels; and the two experts can be trained and upgraded independently. When reading the code, do not let the word “MoE” mislead you. The entire implementation is just the if inside _prepare_model_for_timestep.

Chapter 7 Conditional Injection: Which Lines Image-to-Video Changes

Once you understand t2v, the other tasks are just exercises in "adding conditions to the same backbone." There are only three ways to add conditions:

1. cross-attention: the same path text takes, where the condition is attended to as key/value
2. channel concatenation: the condition tensor is directly concatenated with the noisy latent along the channel dimension, then fed into patch embedding together
3. AdaLN: the condition is mixed into the timestep embedding and modulates each layer’s norm

Image-to-video (i2v-A14B) uses the second method. Look at how image2video.py constructs the condition:

msk = torch.ones(1, F, lat_h, lat_w, device=self.device)
msk[:, 1:] = 0
msk = torch.concat([
    torch.repeat_interleave(msk[:, 0:1], repeats=4, dim=1), msk[:, 1:]
], dim=1)
msk = msk.view(1, msk.shape[1] // 4, 4, lat_h, lat_w)
msk = msk.transpose(1, 2)[0]
...
y = torch.concat([msk, y])

Break it down line by line. msk is the marker for "which frames are known": the first frame (the input image) is marked as 1, and the remaining 80 frames are marked as 0. The following repeat_interleave plus reshape sequence does one thing: it folds the pixel-level mask for 81 frames onto the 21-frame latent time axis. Remember the VAE compression rule: the first frame gets its own latent frame, and every 4 frames after that are merged into one. The mask has to be folded by the same rule: copy the first frame 4 times to complete the grouping, then fold every 4 frames into one group, producing a 4×21×h×w tensor aligned with the latent.

y is the VAE encoder output of the input image itself (with missing frames padded by zeros). The mask and y are concatenated into 20 channels total, then concatenated again with the 16-channel noisy latent along the channel dimension before entering the model. At the beginning of model.py’s forward:

if y is not None:
    x = [torch.cat([u, v], dim=0) for u, v in zip(x, y)]

The i2v version of WanModel has 36 input channels (16 noise + 16 image latent + 4 mask). The first convolution in patch embedding is wider than in the t2v version, while the remaining 40 layers have exactly the same structure. The entire backbone change for image-to-video is simply changing the input channels from 16 to 36.

What the model learns from these 36 channels is: copy the given content where mask=1, generate where mask=0, and keep the two spatially and temporally coherent. The first frame pins down the subject and style of the image, and the following 80 frames unfold motion around it. That is the mechanism behind "making an image move."

Where Conditions Come From in Other Tasks

Read the other three tasks with the same mindset. Focus on "what the condition is, how it is encoded, and through which entrance it enters the model":

ti2v-5B (unified text-image-to-video). The same 5B backbone is paired with a highly compressed 4×16×16 VAE. Its clever trick is that it supports both t2v and i2v without changing the architecture: if an image is provided, encode it, pin it into the first latent-frame position, and use a mask to mark "this frame is known"; if no image is provided, start from pure noise. Known frames and frames to be generated use different effective timesteps within the same batch (known frames are always treated as "already fully denoised"), giving one model two faces.

s2v-14B (speech-driven). The condition is audio. A wav2vec2 encoder turns the waveform into frame-wise features, then re-buckets and aligns them according to "audio frame rate → video frame rate." Each video frame receives its corresponding small segment of audio features, which are injected into the model to drive lip movements and body rhythm. Video length is no longer determined by frame_num; it follows the audio length instead. Extra-long audio is split into multiple segments and generated autoregressively: each segment takes the final few latent frames from the previous segment as a "momentum condition," ensuring motion continuity between segments. This is the only long-video solution in the repository and is worth studying separately.

animate-14B (character animation/replacement). This is the most heavily conditioned task: skeletal pose sequences (driving motion), facial feature sequences (driving expressions), a reference image (locking the character’s appearance, passed through CLIP visual encoding into cross-attention), and, in replacement mode, a background video and mask as well. The preprocessing scripts first extract these assets from the source video, then during inference each condition enters the model through its own route.

The sources of conditions vary widely, but the injection methods are always permutations of the same three: channel concatenation (spatially aligned conditions such as masks, poses, and backgrounds), cross-attention (global semantic conditions such as text and reference-image CLIP features), and frame-wise feature injection (temporally aligned conditions such as audio). Once you understand one task, the rest are just rearrangements.

Chapter 8: What Does 75600² Actually Mean?

Why is multi-GPU parallelism especially important for video generation? Let’s do the attention math carefully. The core computation in a single self-attention layer consists of two matrix multiplications, QK^T and AV. The FLOPs are approximately:

2 × 2 × L² × dim = 4 × 75600² × 5120 ≈ 1.17 × 10¹⁴

That is 117 TFLOPs for one layer. With 40 layers and 80 forward passes, attention alone reaches the order of 3.7 × 10¹⁷ FLOPs. The bf16 peak compute of a single A100 is about 312 TFLOPS, so even under ideal utilization, this part alone would take around twenty minutes, and that does not include the FFN. This is the physical baseline behind why “a 5-second video takes several minutes to tens of minutes to generate.”

If one GPU cannot handle it, split it. Wan2.2’s main approach is Ulysses sequence parallelism. The core communication primitive is in distributed/util.py: a dimension exchange.

def all_to_all(x, scatter_dim, gather_dim, group=None, **kwargs):
    world_size = get_world_size()
    if world_size > 1:
        inputs = [u.contiguous() for u in x.chunk(world_size, dim=scatter_dim)]
        outputs = [torch.empty_like(u) for u in inputs]
        dist.all_to_all(outputs, inputs, group=group, **kwargs)
        x = torch.cat(outputs, dim=gather_dim).contiguous()
    return x

Tracing tensor shapes through an 8-GPU run, with B=1, L=75600, and 40 heads:

Attention requires every token to see the full sequence, so you cannot simply split the sequence and call it done. Ulysses’ answer is: “split heads, not sequence.” Two communication rounds buy each GPU only 1/8 of the attention heads to compute, reducing both attention compute and activation memory to 1/8. The number of heads must be divisible by the number of GPUs: 40 heads with 8 GPUs works perfectly, and generate.py has the corresponding assert.

The communication cost can also be calculated. A full q tensor is 75600 × 5120 bf16 values, about 0.77 GB. Each layer performs four all-to-all operations for q/k/v/output, moving about 3.1 GB of data between GPUs. Across 40 layers and 80 forward passes, that totals roughly 10 TB. It sounds scary, but NVLink inter-GPU bandwidth is on the order of hundreds of GB per second, so amortized over the whole generation process, communication time is far smaller than compute time. The trade is worthwhile. On machines without NVLink, using PCIe instead, the same math may no longer be as favorable, and multi-GPU speedup will shrink noticeably.

A competing route is worth mentioning: Ring Attention follows the path of “split the sequence and pass K/V around the ring.” It can split sequences to arbitrary length, without being constrained by head-count divisibility, but it requires more communication rounds and is more complex to implement. Ulysses is simple to implement and has less communication, but its upper limit is that the number of GPUs cannot exceed the number of heads. Wan2.2 chooses the latter. Forty heads are enough for an 8-GPU machine, but scaling further, such as to a 64-GPU cluster, would require a different scheme.

Parameters not fitting in memory is another axis: FSDP — fully sharded data parallel — splits model weights into N shards distributed across GPUs. When the forward pass reaches a given layer, all GPUs first all-gather that layer’s parameters, compute the layer, and then immediately release them, keeping only their own 1/N shard. On 8 GPUs, the resident DiT weights per GPU drop from 56 GB to 7 GB. The cost is one all-gather per layer, and on NVLink machines, most of this overhead can be overlapped with computation.

Putting the single-GPU and 8-GPU memory accounting side by side:

Sequence parallelism handles compute, while FSDP handles memory. The two are orthogonal, so --dit_fsdp --ulysses_size 8 can be enabled together. The 8-GPU command in the README includes both not by accident: every flag corresponds to a line in the accounting.

Chapter 9: Why the Model Makes the Mistakes It Often Makes

Most typical video-generation failures can be traced back to the principles discussed earlier. This chapter connects the “phenomena” to the “mechanisms,” which is more useful than memorizing a checklist.

Fingers, text, and fine structures collapse. This is the result of two levels of compression working together. The VAE compresses 8×8 pixels into one latent position, and patchify then merges 2×2 latent positions into one token, so one token governs 16×16 pixels. A finger is often only a dozen or so pixels wide; an entire finger may fall into one or two tokens. No matter how powerful attention is, it cannot distinguish the boundaries of five fingers inside a token. Text is even worse: strokes live at pixel level. The only remedies are lowering the compression ratio or increasing resolution, both of which run straight into the compute wall. This is a structural weakness of the current generation of architectures, not the fault of the training data.

Drift in the later part of long videos. t2v generates 81 frames in one shot, with inter-frame consistency enforced by full 3D attention, so it is fine within five seconds. Longer videos (s2v’s segmented autoregression) rely on “the last few frames of the previous segment” to carry state forward, and every handoff is lossy: errors accumulate segment by segment, the character’s face slowly changes, and clothing colors gradually drift. This is isomorphic to drift in long-form LLM text—the original sin of autoregression.

Small motion range and stiff camera work. Multiple factors push in the same direction: CFG pushes the latent toward the region where the “classifier is most confident,” and static compositions are always “safer” than large movements; calm camera shots dominate the training data; words like “static” and “still” in the negative prompt are there to fight this tendency (note that the default negative prompt includes “static, motionless image,” so the authors were clearly annoyed by this problem). Lowering guide_scale and explicitly describing camera motion in the prompt are both ways to loosen the restraints on motion.

Physical inconsistency: clipping, weird liquids, drifting shadows. The model learns “what the next frame usually looks like in pixel statistics,” without any explicit physics engine. Physical correctness only emerges in scenes densely covered by the training data; in sparse-data areas—complex occlusion, fluids, mirror reflections—the weakness shows. This is a known boundary of the purely data-driven path, and the world-model direction being explored in the industry targets exactly this layer.

Large variation between runs with the same prompt. Generation starts from random noise. The prompt only constrains “which semantic region the endpoint lands in”; the concrete appearance inside that region is determined by the seed. This is a feature, not a bug: only by fixing the seed can you make controlled comparisons. That is why all the hands-on experiments in the previous chapter required a fixed seed.

Chapter 10 Symptom-to-Knob: A Tuning Cheat Sheet

Now that the principles are clear, translate them into “what should I turn when I’m unhappy with the output.” The rationale for each row comes from the earlier chapters:

One more reminder beyond the scope of knobs: when adjusting --sample_shift, remember the coupling from Chapter 6. It silently changes how the workload is split between the two experts. When shift drops from 12 to 5, the high-noise expert’s steps fall from 26 to 17, and the style will drift in a way that is visible to the naked eye. It is not purely a single “structure vs. detail” dimension.

Settling the Account

Back to the command at the beginning—now every stage has numbers attached:

• The prompt is transformed by umT5 (5.7B) into ≤512 vectors of 4096 dimensions, and the negative prompt is processed the same way
• Random noise latent: 16 × 21 × 90 × 160, determined by --base_seed; the same seed is reproducible
• Patchified into 75,600 tokens of 5120 dimensions
• 40-step UniPC sampling, with shift=12 squeezing 26 steps into the high-noise region where t≥875, written by the high-noise expert (14B), then switching to the low-noise expert for the final 14 steps
• 2 forwards per step (CFG, guide_scale high-noise tier 4.0, low-noise tier 3.0), for 80 total 14B inferences; attention compute is on the order of 10¹⁷ FLOPs
• The latent is streamed through the causal-convolution VAE decoder back into 81 frames of 720p, then encoded to disk with libx264

Where the time goes can also be ranked qualitatively. T5 encoding runs only once, on the order of seconds; VAE decoding also runs only once, and even though it is convolution over 81 frames, it is still only dozens of seconds; the real bulk is the 80 intermediate 14B forward passes, taking up more than 90% of the full run. Turning on offload adds another layer of weight-transfer time. So every speedup method targets the sampling loop: fewer steps, less CFG (distillation), multi-card attention splitting. Nobody spends much effort optimizing T5 or VAE, because that is not where the bill is.

Each CLI knob corresponds to a specific link in the chain: --sample_steps is the number of integration steps, trading quality against speed; --sample_guide_scale is CFG strength, trading adherence against naturalness; --sample_shift is step allocation, trading the budget between structure and detail while also changing the workload of the two experts; --frame_num is constrained by the VAE compression rule and must be 4n+1; --ulysses_size and --dit_fsdp deal with the compute wall and the memory wall respectively.

At this point, you can look back at the whole picture: there is no magic in this system, only a combination of four things—a compressor that makes the data smaller (VAE), a large network that learns a direction field in the smaller space (DiT), a numerical integration scheme that moves quickly and stably (flow matching + UniPC + shift), and a set of conditioning mechanisms that inject semantics (cross-attention + CFG + task-specific injections). The four evolve independently, each with its own papers, and the repository twists them together. Once you understand the twisting pattern, you can read another video model—Kling, Sora, Veo—and find that the parts differ, but the way they are assembled is much the same.

What to Read Next, What to Build Hands-On

Read the papers along this path; each one corresponds to one or two chapters of this article:

A hands-on path builds intuition faster than papers. I recommend this order:

1. Run the demo with ti2v-5B on a consumer GPU, fixing --base_seed
2. Change only --sample_shift (for example, run 3 / 5 / 12 once each), compare with the same seed, and visually inspect how the budget moves between “structure vs. detail”
3. Change only --sample_guide_scale (1 / 3 / 7), and observe the trade-off between prompt adherence and oversaturation
4. Set a breakpoint in the sampling loop of text2video.py, print t and the latent statistics at each step, and reproduce the table from Chapter 4 by hand
5. Read the mask construction in image2video.py, compare it with Chapter 7, then ask yourself: if the task were “given the first and last frames, generate the frames in between,” how should the mask be changed?

Once you’ve thought through that last question, you’re no longer merely “using” the model—you’re designing its next task.

On the engineering side, this code still has quite a few pitfalls where the implementation is out of sync with the docs. The gotcha list is in another piece: After Reading All the Wan2.2 Source Code. The principles and the pitfalls are best read together.

Appendix: Quick Terminology Reference

In order of appearance, each entry gives one plain-English sentence plus an anchor point:

Any row in this table could expand into a whole chapter. If you forget one, come back and look it up.