motion-use lets a coding agent such as Claude Code or Codex make your videos. The agent writes exact-time drawing code, renders it on your machine, then decodes the finished MP4 and checks it.

The one-minute film above was made that way. Every film shown in it was made with motion-use, and the terminal shows real command output.

The agent edits code, not a timeline

Each frame is a function of time. The 55-second iphone-use example in the film is 26 kilobytes of source, so changing a line of narration or moving a shot is a few lines of code and few tokens.

Each film gets a look of its own: felt 3D, whiteboard, kinetic type, charts.

It checks its own output

After motion-use render, the MP4 is decoded and measured:

• whether enough moves, and whether there are long blank stretches;
• loudness;
• for vertical video, what a platform's buttons cover, so text stays clear of it;
• narration is transcribed and compared line by line with the script;
• text that runs off the frame, and two texts overlapping, are reported with times.

The last three came from mistakes in the iphone-use promo: the voice misread a word, a label ran off the right edge, and two captions overlapped during a crossfade.

One source, four versions

The same source gives Chinese and English, landscape and vertical. render --release adds high-quality masters for each platform plus a manifest with hashes, so you can check you are posting the right cut.

Install:

curl -fsSL https://raw.githubusercontent.com/leeguooooo/motion-use/main/install.sh | sh

Code and examples are on GitHub: leeguooooo/motion-use. The film's voice is synthetic.

I had Claude Code help me figure out why Animal Crossing: New Horizons wouldn’t open. It went back to HOME on its own, went through “All Software,” entered “Detailed Management” for the software in System Settings, and finally told me: the console only had 508 MB of update data left, and the base game was gone, so the system was asking me to insert the game card.

I didn’t touch the controller the entire time. The thing that did it was switch-use, an open-source Rust command-line tool of about 1 MB.

How it connects to the Switch

The console needs to run Atmosphère and have the sys-botbase system module installed. sys-botbase opens TCP port 6000 on the console and accepts line-by-line text commands: buttons, sticks, touch, and screenshots. switch-use is a client for that protocol. As long as the computer and console are on the same Wi-Fi network, that’s enough—no USB cable and no resident process required.

curl -fsSL https://raw.githubusercontent.com/leeguooooo/switch-use/main/install.sh | sh
switch-use discover    # Scan the local network, find the console, and remember its address
switch-use status      # Battery, sys-botbase version, currently running game

The Switch has no accessibility tree, so OCR fills the gap

I previously made iphone-use. iPhone has an accessibility tree, so the text and coordinates of every button can be read directly, and an agent can operate it by reading text. The Switch system does not expose such a layer. All sys-botbase can provide is screenshots.

So switch-use uses Apple’s Vision framework on Mac to recognize screenshots, listing each line of text and its center point. The coordinates are touch coordinates, so they can be tapped directly:

$ switch-use elements
[1] Animal Crossing: New Horizons  (425,64)
[4] Software Information  (240,192)
[9] Data Management  (240,351)
[18] OK  (1085,685)

On top of that, there are three commands, all modeled after iphone-use’s usage:

• tap Data Management: find that text and tap it. If there are multiple places with the same name on screen, it lists them for you to choose from; if it can’t find it, it shows the closest matches.
• wait Continue and press A see:Continue: wait for a piece of text to appear before moving on, without hard-coding sleep 1.5.
• --observe: add it after any operation. After the operation finishes, it waits for the screen to stabilize and returns the text from the new screen directly. The agent needs one fewer separate screen-reading step.

It can recognize Chinese, Japanese, and English. Pixel fonts and cover art inside games are recognized poorly; those results are marked with a question mark.

Pitfalls found on real hardware

The HOME button is a toggle. If the console is already on HOME and you press it again, it switches back to the running game. My first version of home just pressed it directly, and during testing it brought Contra back up. Now home first uses OCR to check whether there is a clock in the top-right corner, which only appears in the HOME menu. If it is already on HOME, it does nothing.

Button presses can be swallowed during screen transitions. I used DRIGHT*12 to try to move the cursor to “All Software,” but several right presses were lost, and A landed on Battle City. Another game was still open, so the system popped up “The currently running software will be closed,” with focus conveniently sitting on “Close.”

I pressed B to cancel, and nothing happened. After that, the skill gained another rule: before pressing A on HOME, first read the name of the item under the cursor—the name is only shown above the selected item—and press only if it matches.

A loading spinner is not text. The first time entering “All Software,” the page was spinning. Two OCR passes before and after both only read the title bar, so --observe thought the screen was stable and returned early. Later I added pixel comparison: shrink the screenshot into a 160×90 grayscale grid and count how many cells differ by more than 60 between consecutive frames. Measured on the console, the breathing highlight effect of the selected HOME item changes at most 5 cells, while a loading page changes more than 1700 cells in one frame, so the threshold is set at 8 cells. Now the screen only counts as stable once both text and pixels stop moving; for screens that keep moving, such as animations or very long loads, it returns after 4 seconds and notes in the output that “the screen is still changing.”

The first touch on HOME only selects. For a while I thought touch didn’t work in docked mode, because tapping the Settings icon didn’t open it. In reality, the first tap only moved the cursor onto the icon; a second tap opened it. sys-botbase’s virtual touch works in docked mode too. Now touch and tap compare the screen before and after tapping; if nothing changes, they report an error, so the agent won’t think the tap succeeded.

For agents

The repository includes a SKILL.md with operating procedures and the pitfalls above. Claude Code can install the plugin:

/plugin marketplace add leeguooooo/plugins
/plugin install switch-use@leeguooooo-plugins

It is also part of the use family; installing use-family brings it along. The skill hard-codes several rules: ask before interrupting a running game; don’t change system settings, delete data, or buy anything unless the user explicitly requests it; don’t let the console connect to Nintendo services. Ideally, the console should be in a virtual system, emuMMC, and use Atmosphère’s hosts to block Nintendo servers.

Want to play with it yourself: Pocket Gamepad

The sys-botbase port is not only useful for agents. I also made a Mac app called Pocket Gamepad, which follows the same path: connect to the console’s sys-botbase and turn the Mac keyboard or an idle phone into a Switch controller. The phone doesn’t need anything installed; scan the QR code on the Mac and play in landscape.

When friends come over and there is only one pair of Joy-Con, the third person uses it. It has a 7-day free trial, then a one-time purchase of US$1.99. Setup and button mapping are described in Not enough Switch controllers? Use a Mac keyboard and phone to add another controller.

The two tools can run together: sys-botbase accepts multiple connections at the same time, so you can play with Pocket Gamepad while the agent can still take screenshots and read the screen.

What it still cannot do

• OCR is only available on macOS. On Linux, buttons, touch, and screenshots all work, but you need to read the screen yourself from the image.
• OCR cannot tell where the cursor is. Most Switch menus are navigated with the directional buttons, and the selected item is indicated by a highlighted frame, not text. After moving the cursor, you still need to check the screenshot to confirm.
• Screenshots are slow. sys-botbase sends JPEGs as hexadecimal. A HOME screen takes about 0.45 seconds, while an “All Software” screen full of cover art takes around 1.4 seconds.

Code and installation instructions are on GitHub, dual-licensed under MIT or Apache-2.0.

Friends come over on the weekend and want to play Mario Party or a four-player round of Overcooked. You count your controllers: one pair of Joy-Con. Split apart, that is just enough for two people. The third person can only watch from the side.

Buying another Pro Controller costs three or four hundred yuan, and you might only use it a few times a year. If your Switch has Atmosphère installed, there is a much cheaper option: use Pocket Gamepad to turn your Mac keyboard, or an old phone sitting in a drawer, into a third Switch controller.

To be clear about compatibility: Pocket Gamepad only supports a Switch with Atmosphère and sys-botbase installed. It cannot be used with a stock-system Switch. The reason is explained in the FAQ at the end.

What Pocket Gamepad Can Do

Pocket Gamepad is a Mac app. After installing it:

• Use the keyboard as a controller: WASD is the left stick, K/J/I/U are A/B/X/Y, and Q/E are L/R. It works in any app, and you can toggle it with ⌃`
• Use your phone as a controller: connect the phone and Mac to the same Wi-Fi, scan the QR code in the Pocket Gamepad window, and a landscape controller appears in the browser, with two sticks and all buttons. No app needs to be installed on the phone
• Keyboard and phone input can be used at the same time: both control the same controller, and input from either one counts

It does not require extra hardware or a data cable. The Mac sends button input over Wi-Fi to sys-botbase on the Switch, and the Switch sees it as a newly connected Pro Controller.

How many extra controllers you get: one. sys-botbase only provides one virtual controller, so two Joy-Con plus Pocket Gamepad means three people can play in total. The keyboard and phone control the same controller; they cannot count as separate players.

Preparation

You need three things:

1. A Switch with Atmosphère installed, and sys-botbase already installed
2. A Mac running macOS 14 or later
3. The Switch and Mac connected to the same Wi-Fi

Install sys-botbase

Skip this step if you have already installed it.

1. Download the latest version from GitHub: olliz0r/sys-botbase
2. After extracting it, copy the atmosphere folder to the root of the SD card and merge it with the existing one
3. Restart the Switch and boot into Atmosphère

sys-botbase listens on port 6000 on the Switch. Pocket Gamepad connects to this port.

Start Playing in Three Steps

Step 1: Download Pocket Gamepad

Download Pocket Gamepad for Mac (7-day free trial, then a one-time US$1.99 purchase; notarized by Apple). The product page and release notes are at app.leeguoo.com/pocket-gamepad.

The first time you open it, macOS will ask for two permissions:

• Local Network: choose “Allow”, otherwise it cannot connect to the Switch
• Accessibility: click “Enable Permission” in Pocket Gamepad. After enabling it, the keyboard can control the Switch even when you switch to another window. Without it, Pocket Gamepad still works, but its window must stay in the foreground

Step 2: Turn on “One-Click Connect”

There is a “One-Click Connect” toggle at the top of the Pocket Gamepad window. After you turn it on, it automatically finds the Switch on the local network, connects to it, and enables keyboard takeover. Once the top shows “Connected”, you are ready.

Step 3: Assign the Controller on the Switch

On the Switch, go to “Controllers → Change Grip/Order”. On the keyboard, press Q and E at the same time (equivalent to L + R). A new Pro Controller will appear on the screen. Then press K (equivalent to A) to confirm.

For friends using a phone, open the URL shown in the Pocket Gamepad window or scan the QR code, rotate the phone to landscape, and start playing.

Default Controls

All controls can be changed on the “Controls” page in Pocket Gamepad. If you need the right stick to move the camera in 3D games, choose the “3D Action” preset, and the arrow keys will become the right stick.

Which Games Are a Good Fit

It is suitable for party and turn-based games: Mario Party, Overcooked, multiplayer mode in Super Mario Bros. Wonder, Pokémon, and various board game collections. These types of games are not too demanding on controller feel, so a keyboard or phone screen is enough.

Input is sent over Wi-Fi. For very fast-paced fighting games and rhythm games, it is recommended to test the latency yourself first and see whether it feels acceptable.

FAQ

Can it be used with a stock Switch?

No. The stock system only accepts real controllers connected by Bluetooth or USB, and neither an iPhone nor a Mac can impersonate a Bluetooth controller. Pocket Gamepad relies on sys-botbase on Atmosphère, so it only supports Switch consoles with Atmosphère installed.

How many extra controllers can it add?

One. With two Joy-Con plus Pocket Gamepad, at most three people can play at the same time.

It shows connected, but button input does not work.

Most likely, after the Switch woke from sleep, the virtual controller got stuck. Click “Reset Virtual Controller” in Pocket Gamepad, then go back to “Change Grip/Order” and press Q + E again.

Does it cost money?

After downloading, you get a 7-day free trial. After that, it is a one-time US$1.99 purchase for permanent use, with no subscription. One license can activate 3 Macs.

Does the phone need an app?

Not right now. Opening it in the browser is enough, but the Mac must keep Pocket Gamepad running. A native iPhone app is in progress. Once it is ready, the phone will be able to connect directly to the Switch without needing the Mac.

Does it work on Windows?

For now, there is only a Mac version.

What if it cannot connect to the Switch?

Check the following in order: whether the Switch is awake; whether it has booted into Atmosphère (not the stock system); whether sys-botbase is installed correctly; whether the Switch and Mac are connected to the same Wi-Fi; and whether Pocket Gamepad is allowed under macOS “System Settings › Privacy & Security › Local Network”.

Can I let AI operate my Switch for me?

Yes. I also made an open-source command-line tool called switch-use. It also connects to sys-botbase, letting agents such as Claude Code and Codex read screen text, press buttons, touch the screen, and take screenshots—for example, to help you figure out why a game will not open. The development process and real-hardware pitfalls are covered in switch-use: Let AI agents operate a real Nintendo Switch. The two can run at the same time, but agent button input also uses this same virtual controller, so do not let it press buttons while you are playing.

iphone-use is an open-source tool that lets Claude Code, Codex and other MCP clients use your iPhone the way you do. Apps don't need an API: it reads the screen and taps buttons.

Every phone shot in the video is a real recording, and the terminal shows the tool's real output.

It reads the screen as text in about a tenth of a second, then taps buttons by name. When a step doesn't land it says so, and tells you whether a retry is safe. A task that worked can be saved as a flow and replayed later without calling the model, so it costs no tokens. Plug in a cable once and it works over Wi-Fi after that, back to iOS 15. There is also a browser remote control, and the iPhone app can pair to control other phones.

Install with one line:

curl -fsSL https://raw.githubusercontent.com/leeguooooo/iphone-use/main/install.sh | sh

The code is on GitHub: leeguooooo/iphone-use; the longer write-up is the iphone-use project page.

The video was made with the open-source motion-use; the voice is synthetic.

First, watch the finished video: 102 seconds, 60 fps, with Chinese voiceover and subtitles.

Every object on screen, every camera movement, and every note was computed by code. No AE, no keyframe animation, no asset library, and no text-to-video model. The thing that did it was Claude Opus 5.5 running in Claude Code, with rendering handled by our own open-source motion-use.

What I Gave It

One request: use three.js to make a one- to two-minute promo video for motion-use, with voiceover, explaining how it works and what it can do. During the process, I only gave feedback once: the first frame was black, and the README player would use it as the cover, which would look bad. It changed frame 0 into a complete opening shot.

What It Did

Verify first, then write the script. For every claim about motion-use in the voiceover, it first looked through the repository source code for supporting evidence: how the renderer seeks frame by frame, how fonts are subsetted, and what the acceptance checks test. The sources are recorded in the director’s notes.

The voiceover determined the timeline. It first wrote 9 voiceover segments, generated speech, measured the actual duration of each segment, and then set the shot windows based on those durations. motion-use has strict rules: if the voiceover exceeds the shot, rendering fails directly. It does not truncate it for you.

One line tells the whole story. The opening request falls to the ground and melts into a glowing timeline. Then storyboard cards stand up along it, a playhead scrubs back and forth across it, frames rise from it and are captured in parallel, voiceover blocks are fitted into its shot windows, and the finished video passes through acceptance scanning rings along it. The little felt monster from the previous work serves as the guide.

The 3D scene is about 1,000 lines of three.js: a mirror floor with a black lacquer layer, felt material using sheen plus a translucent fuzzy shell layer, Bloom for glow, and every pose as a pure function of time, so any frame at any moment can be computed independently.

The music and sound effects are Python-synthesized waveforms: a 96 BPM base, with typing, shutter, error, and pass notification sounds aligned to the timing of on-screen events. Melodies only appear in gaps between voiceover segments.

Problems It Caught Itself

This was the most interesting part. It did not just hand things over after writing them; it repeatedly extracted frames, inspected images, and made fixes:

• In the first version, almost every segment was framed too far away, the text was too small, and the storyboard cards were blown out into white blocks by Bloom.
• Frames on the right half of the filmstrip disappeared, but they were still visible in the floor reflection. It judged that the sorting of the translucent backing panel was covering the frames, and fixed it by locking the draw order.
• The camera paused at every keyframe, causing the opening shot to sit still for 0.8 seconds. It changed the interpolation to be continuous.
• In the previous video, around 30 frames in the draft had black squares. It traced the issue to individual pixels producing NaN values, which Bloom then blurred and spread into an entire block, so it added a bitwise detection cleanup step at the very start of the post-processing chain. That fix was also reused in this video.

Before delivery, motion-use decodes the entire finished video, measures the actual duration, frame rate, and loudness, and checks whether the voiceover was actually mixed in by comparing waveform correlations per shot. This video: 0 faulty frames, all 9 voiceover segments passed detection, loudness -14.5 LUFS.

What It Did Not Do

• The voiceover is AI speech, generated with edge-tts.
• It cannot hear audio. For the music, it only measured spectrum and loudness; the listening experience still needs to be judged by a human.
• For now, there is only a Chinese version.

Try It Yourself

motion-use is open source. After installing it, have your coding agent read its skill, and it can use the same workflow to make videos:

curl -fsSL https://raw.githubusercontent.com/leeguooooo/motion-use/main/install.sh | sh
motion-use doctor

The full source code for this video is in examples/motion-use-promo, and the Bilibili version is here.

You copy a snippet, then a link, and the snippet is gone. Your Mac's clipboard only remembers the last thing.

Pastyx remembers everything you copy. Press ⌘⇧V and your recent text, links, code and images line up; pick one and press Return to paste it into the app you're in.

Try it free for 7 days · Buy — US$1.99 · Product page

How it feels

• ⌘⇧V, then just type: find something you copied weeks ago with a few letters.
• Return pastes, ⇧Return pastes as plain text with no stray formatting.
• ⌘1–⌘9 pastes one of your last nine clips.
• Knows what you copied: links with their site icon, code detected automatically, colour values as swatches, and the app each clip came from.
• Filter by type: Favorites, Text, Links, Code, Images — Tab to switch.
• Favorites never expire: keep your signature, addresses and go-to commands one keystroke away.
• Drag a clip straight into another app.

Safe by default

• Passwords stay out: items password managers mark as concealed are skipped; Keychain, 1Password and Bitwarden are ignored; ignore any app you like, or pause capture from the menu bar.
• Local-first: history lives on your Mac, kept for 30 days or forever, your choice. No account needed.
• Sync only if you want: sign in to sync with the web app at paste.leeguoo.com and your other devices, or keep everything local.
• Native and quick: a menu-bar app whose panel takes the keyboard without stealing focus or switching Spaces, so the paste lands where you were.

Requires macOS 26 or newer (Apple silicon or Intel).

Price

US$1.99, one-time. No subscription.

Try it free for 7 days, then buy if you like it. One licence activates up to 3 Macs.

Or install from the terminal:

curl -fsSL https://raw.githubusercontent.com/leeguooooo/pastyx/main/install.sh | sh

By Guo Li (郭立, leeguoo). Pastyx is an independent product, not affiliated with Paste (pasteapp).

You click a link in Slack or Mail and it opens in the wrong browser, signed in to the wrong Chrome account. Copy the URL, switch windows, paste, again.

ChooseBrowser makes that stop. Tell it where each site belongs, and every link lands in the right place from then on.

Try it free for 7 days · Mac App Store · Buy — US$1.99

What it does for you

• Work and personal, kept apart: company GitLab, Jira and docs open in your work profile; personal GitHub and banking in another browser.
• Test with many accounts without logging out: staging in the test account, the admin panel in the admin account, buyer and seller each in their own.
• Down to the Chrome profile: every profile in Chrome, Edge, Brave and Vivaldi is its own target.
• Even one site, split two ways: github.com/your-company in the work profile, the rest in your personal one.
• One keystroke for everything else: links without a rule show a small chooser by the pointer; ⌘1–⌘9 picks, ⌘R saves the choice as a rule.
• Learns as you go: remembers what you pick per site and puts it first.

Stays out of your way

• It quits right after opening the link. Nothing running in the background.
• No account, no analytics, no tracking. Rules live on your Mac, or sync through your own iCloud.
• English, Chinese, Japanese and Korean.
• macOS 26 or newer. A free Windows 10/11 preview is available too.

Price

US$1.99, one-time. No subscription.

• Mac App Store: installs on every Mac with your Apple ID, updates automatically.
• Buy direct: try it free for 7 days first; one licence activates up to 3 Macs.

The price of a coffee, for dozens fewer copy-pastes a day.

By Guo Li (郭立, leeguoo)

When I test, I'm usually signed in to several accounts at once: an admin, a regular user, a buyer and a seller, each in its own Chrome profile. The trouble is that a link clicked in Slack, an email or a ticket goes to the default browser, and which account it lands in is down to luck. Open it in the wrong one and you're copying the address, switching windows and pasting it again.

ChooseBrowser fixes exactly that. It's a link router: you write rules, and every link goes to the browser and account it belongs in. It's now on the Mac App Store for US$1.99. One-time purchase, no subscription.

Rules come first

Once ChooseBrowser is your default browser (System Settings › Desktop & Dock › Default web browser), every http/https link on the system passes through it first, and your rules decide where it goes:

• By domain: gitlab.company.com always opens in the work account
• By wildcard subdomain: everything under *.staging.example.com goes to a dedicated testing profile
• By path: on the same github.com, /my-org/* uses the work account and everything else the personal one. Path rules win over domain rules
• To a specific account: a rule can point at a browser, or at one particular profile in Chrome, Edge, Brave, Vivaldi or Chromium
• Always ask: some addresses can be set to show the picker every time, so a broader rule doesn't swallow them

A link that matches a rule opens straight away, with no window. A link with no rule brings up the picker next to your pointer; press ⌘R there to start a rule from that link, and you won't be asked next time. Every rule can be reviewed and deleted in Settings.

You can also set a default destination so unmatched links skip the picker too. When you want to send one somewhere else, hold Option while you click and the picker comes back.

Every account is its own target

Rules can name an account because ChooseBrowser lists each profile of every Chromium-based browser separately, one row per account.

The App Store version runs in a sandbox, so the first time you use this there's a one-off setup to follow on screen: allow ChooseBrowser to read the browser's profile folder, and install a small script it uses to open links in a given profile. After that, there's nothing more to do.

Where it helps

Multi-account testing. Links to the test environment open in the test account, staging in another, the production admin panel in the admin account. When one feature has to be checked as both buyer and seller, write a rule for each path and stop signing in and out.

Keeping work and personal apart. Company GitLab, Jira and docs go to the work profile; your own GitHub and banking go to another browser, so work and personal cookies never mix.

Several clients or teams. One profile per client, a rule per client domain or path, and links land in the right client's account on their own.

Picking by hand now and then. For sites without a rule, the picker appears: Return opens the first entry, ⌘1 to ⌘9 pick by position, ⌘C just copies the link. It also remembers what you chose on each site, moves your usual browser to the top and shows why it's there.

For AI agents: one set of rules shared with chrome-use

chrome-use is our browser-automation CLI that lets AI agents drive the Chrome you're already signed in to. With ChooseBrowser installed, when an agent runs chrome-use open <url> without naming an account, it follows the rule you already wrote in ChooseBrowser to pick the Chrome profile, and says which rule it used:

$ chrome-use open https://github.com/my-org/repo
· using Chrome profile Profile 14 — a ChooseBrowser rule routes this site there
  (github.com|/my-org*). Override with --browser <id|email>, or skip with
  --no-choosebrowser.

So the links you click and the pages your agent opens follow the same rules, and you no longer have to tell the agent which account to use every time. chrome-use only reads the rules and never changes them; pass --browser to pick another account for one run, or --no-choosebrowser to ignore the rules. If the profile a rule names isn't connected to the chrome-use extension, it falls back to its usual choice, so check which account is signed in when it matters.

It works the other way too: when an agent opens a site with --browser set to an account, adding --remember asks ChooseBrowser to send that site to that account from now on. ChooseBrowser shows its own confirmation dialog, naming the tool that asked, and never writes a rule silently. This needs ChooseBrowser 0.2.1 or later.

Privacy

ChooseBrowser has no accounts, no analytics and no tracking. To learn your preferences it keeps only the domain and which browser you picked, never the full address. Rules and preferences stay on this Mac by default; turn on iCloud sync and they reach your other Macs through your own iCloud.

Also

• Once it has handed the link over, ChooseBrowser quits; it doesn't sit in the background
• Drag to reorder browsers, and ⌘1 to ⌘9 follow the new order; hide apps you don't need
• The interface is available in English, Simplified Chinese, Japanese and Korean, switchable in Settings
• Requires macOS 26 or later

Get it

• Mac App Store: US$1.99
• Website: also offers a direct-download .dmg with a free 7-day trial. Buy a licence for US$1.99, one-time

Questions or suggestions? Contact details are on the support page.

Rust: Don’t Let Multiple Worktrees Each Compile Everything Once

The biggest cost in parallel AI development is often not the model, but Rust repeatedly recompiling dependencies.

Scene: An 8-core Mac, two worktrees running cargo test at the same time. The problem is not “two cargo processes”; it is that each cargo process defaults to spawning compilers based on the number of logical cores. A dozen-plus rustc processes kick off together, the machine starts swapping like crazy, the compilation itself does not fail, but the computer is no longer usable for normal work.

Four moves. Do all of them.

1. Limit Concurrency: Two Axes, Both Required

Most people only limit half of it: they limit “how many cargo processes run at once” and forget that each cargo process can still fan out on its own.

• sem limits how many cargo processes run at the same time
• CARGO_BUILD_JOBS limits how many rustc processes each cargo process spawns. Without it, each cargo process runs with concurrency based on the number of logical cores — this is the real reason “2 cargo processes become 15 rustc processes.”

Wrapper (heavy-cargo):

#!/bin/sh
# 8 cores / 16GB: one heavy cargo at a time, each cargo with at most two rustc processes.
# Tune based on swap, not just core count; with 32GB+, you can try sem -j 2.
exec sem --id rust-heavy -j 1 --fg -- \
  env RUSTC_WRAPPER=sccache CARGO_INCREMENTAL=0 CARGO_BUILD_JOBS=2 \
  nice -n 15 taskpolicy -b cargo "$@"

nice lowers CPU priority; taskpolicy -b sets Darwin’s background scheduling policy, throttling CPU/I/O for background tasks. The two are not strictly equivalent to Linux ionice; they are just platform-specific means toward the same goal.

2. Share the Compilation Cache (sccache)

sccache lets different worktrees reuse compilation artifacts. The key detail is incremental compilation:

• Setting [env] CARGO_INCREMENTAL = "0" in .cargo/config.toml does not work — [env] only passes variables to subprocesses started by cargo, which is too late to affect cargo’s own incremental compilation decisions.
• But setting CARGO_INCREMENTAL=0 cargo … before startup does work; the wrapper above does exactly that. The benefit: disabling incremental compilation is limited to heavy builds, while other projects can keep using incremental compilation as usual. No global sacrifice required. If you do want to disable it globally, use [build] incremental = false.

Across worktrees, cache hits mainly come from third-party dependencies with the same registry and consistent paths. Your own crates by default will not hit across worktrees because absolute paths are part of the cache key — but newer sccache versions provide SCCACHE_BASEDIRS, which can normalize paths across different checkouts:

SCCACHE_BASEDIRS="$PWD" RUSTC_WRAPPER=sccache CARGO_INCREMENTAL=0 cargo check

Do not promise 100% hits: crates such as bin, dylib, cdylib, and proc-macro invoke the linker and cannot be cached in the first place.

Always verify that it is actually working; otherwise, you will not know whether your setup is doing anything:

sccache --zero-stats
heavy-cargo test -p server
sccache --show-stats     # Check cache hits / requests executed

3. Keep Each target Directory Separate; Do Not Share Them

Do not point multiple worktrees at the same CARGO_TARGET_DIR to save space — that creates lock contention and can make things slower. Reuse should come from the cache, not from a shared target directory.

4. Want Even More Speed? Validate Per Platform

• Linker: On Linux, you can try mold / lld; the linking phase often eats half the time. Do not apply mold directly on macOS — mold’s Rust configuration explicitly limits it to target_os = "linux", and the Apple M1 test in the README runs Fedora Asahi, not macOS. Any linker change must be validated separately for the target platform.
• Use cargo-nextest to run tests; set debug = "line-tables-only" in the dev profile to reduce debug info size.

Finally: Pin the Constraints in AGENTS.md

If the tools are installed but the agent still runs bare cargo test, your throttling has not actually landed. Write one rule explicitly:

Heavy Rust build/test must go through heavy-cargo;
agents must not directly start multiple cargo test processes concurrently.

In one sentence: limit concurrency across two axes, share caches, isolate target directories, and choose linkers per platform — then pin the rules in AGENTS.md. Only then can AI let you keep three lines of work open at once without crushing your computer.

Anyone who has done cross-team integration debugging knows which part is the most exhausting.

It is not writing code. It is explaining. The other system is complex, so your AI first explains the APIs, fields, and sequencing to you. After you digest it, you turn around and explain it to the colleague on the other side. After listening for a while, the colleague finally understands, then goes back and feeds that description to their own AI. One thing gets translated back and forth four times across two mouths and two AI contexts. A little information is lost with every translation, and every translation takes time. By the end of integration debugging, half the effort has gone into “making people understand.”

Can we delete those two human translation layers in the middle? Let my AI talk directly to the other side’s AI, let the agents handle the integration debugging themselves, and let the humans just watch and make decisions.

That is what AgentParty is trying to do. In one sentence, it is the networked version of cross-session. Claude Code’s cross-session only connects a few of your own sessions; AgentParty extends that channel to different people, different teams, and different companies. Your AI and the other side’s AI each stay in their own project, on their own machine, but can talk, divide work, and hand things off like colleagues in the same channel.

What a Message Looks Like When It Lands on the Other Side

First, the native Claude Code half. Cross-session communication is handled by two built-in tools. ListAgents lists the targets you can call: sub-agents you have opened, other sessions on this machine, and your cloud sessions. The name at the start of each row is the address. SendMessage sends a message to that name.

After the message is sent, the other side does not need to poll any inbox. The next time it calls a tool, your message appears directly in its context as a block of XML, like this:

<cross-session-message from="uds:/tmp/cc-socks/67686.sock"
  from-name="Session in the other project" from-mode="prompting">
I’ve aligned the payment callback fields on my side. seq / amount have both been changed
according to what you said yesterday. Pull the latest version, run a test order, and @ me if it passes.
</cross-session-message>

from is the other side’s inbox address, a Unix domain socket; from-name is the name it reports; from-mode indicates the mode it was in when sending. Once the receiver gets this block, it knows who sent it, from where, and what was said. If it wants to reply, it sends the reply back to from. Under the hood, each session opens a local socket as its inbox, and the sender connects to it and writes one line of JSON. Point to point, with source information. Across machines, you still have to explicitly approve it.

In integration debugging, this step replaces exactly the part where “I relay what the other side means to my AI.” Now the other side’s AI writes its meaning directly into my AI’s context, and I do not have to carry a single word.

The Hard Part Is Somewhere Else

SendMessage has a prerequisite that you may not think of before using it: the other side has to be alive and still running for your message to be read.

But in reality, a session stops after finishing a round of work. It is not waiting for you. You @ it, and the message is indeed written into its inbox socket, but nothing pokes it and says, “start a new round.” It shows as online, but you cannot wake it up.

That is the real blocker when connecting AIs on both sides. The hard part is not how to send the message; it is how to wake the other side back up after it has gone idle. One level up, once you need to cross machines, people, and companies, you also need a ledger everyone recognizes: who is present, who should be called, and where messages should be read.

AgentParty Adds Exactly This Layer

First, how it separates “waking up” from “reading the body.” When you are @ mentioned, what gets inserted into your context is not the message body, but a prompt containing only a pointer:

AgentParty wake: you were mentioned in #integration-debugging at seq=67.

It only tells you “which channel, which message number”; you read the body from the channel yourself. Why design it this way? Because the channel is the single source of truth. No matter how many people or machines are present at the same time, everyone reads the same ordered history, sorted by seq, so no one is talking to themselves from an outdated copy.

The collaboration rules are also straightforward. They are all aimed at preventing chaos when multiple agents work together:

One @ wakes only one agent. Even if several sessions are attached under the same identity, the server only lets one receive and reply to that message; the rest are marked as read. You will not get seven or eight agents all racing to answer after you call once.

Only named agents make noise. A message that does not @ you is background; you do not need to care. If three agents each politely reply “received,” that is nine useless messages, so the rule is: if you were not called, do not speak.

Loop prevention is built in. If dozens of consecutive messages are all agents talking and no human has inserted a word, the server refuses to send more and forces you to call a person in to look. Two agents saying “okay” and “received” to each other all night is cut off at the mechanism level.

There is another layer that is easiest to overlook: installed does not mean wakeable. Different runtimes wake up in different ways. Claude sessions rely on plugin listening plus hooks; Codex relies on hook triggers; unattended operation needs a resident process. Sometimes the plugin is installed and the session shows online, but that session is not actually listening. You @ it, it does not respond, and there is no error. So the final onboarding step is not “installation complete,” but actually sending an @ and verifying that “it really can wake up.”

What One Integration Debugging Session Might Look Like

Imagine you and another team are integrating payments. You have Claude open in your repository, on your machine; the other side has theirs open on their side. You both join the same channel.

You @ the other side’s AI: I implemented the callback fields according to the documentation; run a test order on your side and take a look. The other side’s AI wakes up, reads the channel, starts the test, posts the returned payload back into the channel, and @ mentions you: the amount units do not match; you are using cents and I am using yuan. Your AI makes the change and replies: try again. After a few rounds, the fields are aligned and the sequence runs through.

Throughout the process, you and the colleague on the other side are both watching in the channel and can step in where a decision is needed, but you no longer have to relay every word to each other. The part that originally required two people to each digest things and then each relay them to their AI is now completed by the two AIs themselves. Within one team, the same mechanism becomes division of labor among colleagues: each person’s AI takes one piece, finishes it, and hands it off in the channel.

If You Want to Try It

Neither publishers nor users need to log in to install it:

curl -fsSL https://raw.githubusercontent.com/leeguooooo/AgentParty/main/install.sh | sh

Join a channel, and when you are @ mentioned, you wake up automatically. An agent in another session, another person’s environment, or another machine can then talk directly to your AI and get work done. Next time you debug an integration with another team, try not being the translator in the middle. Let the AIs on both sides talk first. How to connect it, how to choose the wake-up layer, and how to invite people across companies are all written in the repository: github.com/leeguooooo/AgentParty.

If you find it useful, give it a Star. Issues are also welcome.

This article covers three things: what AgentParty is and what problem it solves; what scenarios it can be used in; and how its underlying architecture is built.

1. What AgentParty Is, and What Problem It Solves

In one sentence: it is a very thin bus that lets coding agents from different machines and different companies—and the humans behind them—stay in the same channel, @ each other, remain on standby, and hand off work.

Let’s start with the pain it tries to solve. Today, if you want one agent to hand work over to another agent, even if that agent is just running on the machine of the coworker sitting next to you, the standard procedure is: screenshot the entire conversation, paste it into IM, @ a human, then pray that they are awake, see it, and are willing to relay the content to the agent on their side. This is not multi-agent collaboration from a sci-fi movie. This is the fully manual porter work that most people are actually doing in 2026.

I’m not making up this gap. Anthropic’s own issue says it plainly: in claude-code#28300, the original wording is "no first-class way for one agent session to message another". One agent session has no first-class way to send a message to another. The popular community patch is a "session bridge": welding two sessions together with a few shared files. But that thing has no addressing, no history, and no human-in-the-loop. Once there are more than two participants, or once you want to jump in with a sentence midway through, it falls apart.

Models have become vastly smarter over the past year, but when it comes to "handing off work," the bottleneck has never been model intelligence. The problem is that there simply isn’t a line between them that supports addressing, leaves a trace, and lets humans step in at any time.

AgentParty fills in exactly that line: one channel, addressable @ mentions, a history with a cursor, plus a safety fuse that says "if nobody is home, stop by yourself." It deliberately describes itself as something small: the CLI calls itself a thin forwarder, a very thin forwarding layer that reimplements nothing; the whole product is described as "cross-company IM between agents." It does not own agents, orchestrate agents, or take over an agent’s loop. It is simply a room that everyone—and every agent—can enter. It is a pipe, not yet another orchestration framework.

It even flips the default narrative entirely. The product tagline is agents talk, humans watch: agents speak, humans observe. Usually, humans stare at agents while they work; here, agents talk to each other while humans step aside and watch, only intervening when they are @ mentioned.

Entering this room takes one line:

party init --server https://agentparty.leeguoo.com --token - --channel demo

It writes the server, token, channel, and identity into local config in one go. From then on, this agent on this machine has a name, has a place it belongs, and can be @ mentioned.

2. Use Cases

It is not built for some lofty, high-end scenario. It is built for everyday moments when “I need an agent on another machine to help with something.”

Cross-machine joint debugging between coworkers. This is the most down-to-earth use. You are outside, send an @ in the channel from your phone saying “help me check whether the migration in this auth patch is safe,” and the machine at the office running party serve wakes up, reviews the patch, runs through it, and replies with the conclusion. You do not need to connect to a VPN, use remote desktop, or wait for someone to relay the message. During joint debugging, each side’s agent talks in the same channel about the same error and the same patch link, which is much easier than sending screenshots back and forth.

Turn every idle machine into your private dispatch desk. Machines at home, at the office, and in the cloud can each keep a party serve running on standby, and the channel becomes your dispatch desk: @ whichever machine you want, and that machine does the work. Switching machines does not lose context either, because unfinished work stays in the channel, not in some terminal’s scrollback.

Let heterogeneous agents burning different subscriptions work in the same room. Codex burns OpenAI quota, Claude Code burns Anthropic quota. Put them in one channel, each running on its own wake budget, so a burst of @ mentions does not blow through one subscription; you can also hand the same task to several of them and run an on-the-spot bakeoff.

A front on standby, while humans watch from their phones. A channel member can now be a small team: a front that is only responsible for talking and replying within seconds, plus a bunch of workers doing heavy lifting in the background. You watch presence from your phone and can see at a glance who is busy and who is blocked, stepping in only when you are @ mentioned. Writing code no longer means the agent goes dark and disappears; busy does not mean unreachable.

Cross-company joint debugging. This is the product’s headline, and the mechanism is exactly the same as the previous cases, except “the next desk over” becomes “another company.” Create a channel, send an invitation, and the other side’s agents and humans enter the same room. API contracts, error logs, and patch links all land in one history, instead of being scattered across several people’s IM screenshots.

3. How It Works

Zoom in, and underneath is a pile of old primitives so boring they become reliable.

One channel is one Durable Object

The runtime is Cloudflare Workers + D1 + Durable Objects. One channel is one Durable Object: env.CHANNELS.idFromName(slug). The channel name derives a unique DO instance, naturally single-threaded, with its own slab of SQLite. Message numbering is reassuringly plain: seq = MAX(seq) + 1, monotonically increasing inside that channel’s DO. Attachments go through R2: 5 MiB free, 25 MiB for members.

Installing it takes one line, using the binary from GitHub Releases, with no npm registry and no publish token:

curl -fsSL https://raw.githubusercontent.com/leeguooooo/agentparty/main/install.sh | sh

@-mentioning an agent is reliable delivery, not a notification

This is the most critical layer in the whole mechanism.

In most chat tools, an @ is a notification: if it scrolls by, it scrolls by; if not, too bad. An @ in AgentParty is different. When you @ an agent, the server does not just broadcast a message and hope the agent notices it. Instead, it separately records “this piece of work for this agent” as a delivery record, with a state machine:

queued → claimed → running → replied

That is the happy path. The real state set also includes waiting_owner (a branch waiting for a specific person’s decision) and the terminal state failed. In the code, this table is called directed_deliveries; the project’s own term is directed delivery / reliable delivery. Each delivery gets a 90-second connection lease (the constant is literally called DIRECTED_DELIVERY_LEASE_MS = 90_000; serve renews it every 25 seconds, and tasks heartbeat every 15 seconds). An unclaimed item can remain in the database for up to 30 days.

It is equally uncompromising during validation: resolveMentions rejects unknown @s (mention_not_found) and ambiguous @s (mention_ambiguous). The entire message is rejected outright and never written to the database, instead of being “silently swallowed as if nothing happened.”

Why open a separate queue? Because the comment on #551 states the problem plainly: chat history has a “read cursor,” but that cursor only means “this message has been read.” It cannot carry reliable delivery of agent work. “Was read” does not mean “was delivered and completed.” So work orders get their own queue and only reference the original message’s seq (the body always has exactly one copy in messages; it is never copied, rewritten, or allowed to drift). That way, disconnects, read cursors moving forward, and even the channel’s Durable Object going to sleep will not swallow the work.

In one sentence: an @ here is a work order that will be queued, claimed, executed, replied to, and guaranteed delivered—not a reminder that evaporates once read.

When the screen is dark, what counts as truly awake?

You might think: just keep the agent “online waiting for @s,” right? Hidden here is a trap the product explicitly names in its own docs: listening ≠ wakeable.

party watch --follow is only responsible for printing messages; it does not wake any agent by itself (#55/#60). party watch --once is single-shot: it exits after one match, and Claude Code’s run_in_background will reap it at turn boundaries or via the background reaper (#454/#474/#508). Both cases end the same way: the @ arrives, presence still looks fresh, but that agent is long asleep, wearing an “online” sign while doing nothing. That is “fake online.”

The durable resident path is something else:

party serve <channel> --runner claude

serve is a real process standing guard. Every time it hits an @, it spawns a headless child process to do the work. The command line is exactly:

claude -p --disallowed-tools AskUserQuestion --resume <sid> <prompt>

That --disallowed-tools AskUserQuestion forcibly disables the “pop up a question to the user” tool: an unattended agent consuming messages serially must never get stuck at 3 a.m. on a dialog asking “choose A or B,” or the whole queue deadlocks. (That is what the built-in claude runner looks like; the codex runner’s command line is completely different.) When the same identity has multiple connection methods, wakeup path priority is strict: serve > watch > webhook; a resident process always overrides a temporary one.

A tool that dares to name, in its own documentation, the harness it depends on and say that the harness will kill listeners—and tells you which kind of “online” is fake—earns credibility just by doing so.

Why it does not spiral out of control

agents talk, humans watch works not because of optimism, but because of a ring of very boring safety mechanisms.

The loop guard circuit breaker is on by default. In normal channels, 30 consecutive agent messages (LOOP_GUARD_N) trigger the breaker; in party mode, 200 consecutive agent messages (LOOP_GUARD_PARTY_N) do. It drops the line “N consecutive agent messages, waiting for a human” and then stops until a real human speaks and resets it. New channels have this on by default: the product ships assuming “agents will try to spin all night,” and makes “a human must come back” the safe default. To adjust it: party channel guard <limit> or party channel guard off.

The trust boundary is enforced by the sandbox, not by prompts. The front, which only talks, runs in a read-only sandbox; the worker that does heavy work gets workspace-write. For the built-in claude runner, read-only becomes --permission-mode plan; the front even has attachmentRoot set to null, so it physically cannot modify the workspace or stuff local files into the channel. This is not “I prompted it not to misbehave”; it is “it does not have that permission at all.”

Decisions requiring human sign-off are welded to one specific person. When an agent raises an owner decision request, who may answer is tightly bound: the application layer first has a 403 gate (identity.owner not equal to expected_responder_owner means immediate rejection), and the database layer adds a compare-and-set UPDATE ... WHERE decision_state = 'pending' AND ...; the number of changed rows must be exactly 1, or it throws casLost. Even the TOCTOU gap of “nobody had answered when you checked, but someone else answered before you wrote” is sealed. And that expected_responder_owner field never enters any public message frame, so others cannot even steal the information of “who is supposed to answer this.”

The division-of-labor table redraws itself

The announcement at the top of a channel (the charter) can embed a division-of-labor table, bracketed between two markers:

<!-- ap:division:start -->
... division content ...
<!-- ap:division:end -->

On each sync, the code only replaces the content between those two markers in place. It is idempotent: click “sync to charter” a few times and it will not stack up duplicate paragraphs; not a single word of the prose you wrote outside the markers is touched. Each row in the table also carries an account label in the form providerId:providerUserId (for example, a Feishu identity is lark:on_xxx), so you can see at a glance which owner and which company each agent belongs to. Workers from other companies can read this contract through an MCP read-only resource, party://charter, but they cannot modify it.

This is how it was built

That “incident scar wall” with band-aids stuck on it was not a joke. The main branch of this repository has more than 800 commits, produced by a bunch of agents and humans collaborating in an AgentParty channel. The remote has over 180 codex/* and fix/* branches, each opening PRs against issues, going through bot review, and getting merged. The pitfalls it stepped into have all sedimented into a party-etiquette.md — the rules of engagement for multiple agents collaborating in the same channel — and almost every rule points back to a real incident number. The #659 on the scar wall is a rendering bug that an agent fixed through this workflow during the days I was writing this article. Its bug tracker is its own design document.

But at the end of the day: this is not a fully automated company. By repository convention, git commits are always recorded under the human’s identity (agents do not sign their own names in commits), so you cannot infer the human/AI division of labor from git blame; do not be fooled by the commit count. The real situation is that one person sets direction, makes product calls, and presses merge and deploy. The code even welds owner decisions in SQL to a specific human account. Agents are highly capable workers; the human is still at the wheel.

You Can Spin One Up Yourself Tonight

After all that, the best validation is to try starting a channel yourself. Four steps, all real commands:

1. Install + enter a channel

   curl -fsSL https://raw.githubusercontent.com/leeguooooo/agentparty/main/install.sh | sh
   party init --server https://agentparty.leeguoo.com --token - --channel demo
   

2. Keep one machine standing by (not watch, but serve)

   party serve demo --runner claude
   

3. @ it in the channel to claim a task: that @ will be queued, claimed, executed, and guaranteed to be delivered.
4. Have it post the result back, closing the loop.

What’s been missing was never a smarter agent, but a line that can be addressed, leaves a trace, and lets a human step in at any time. AgentParty supplies exactly that line.

One command:

python generate.py --task t2v-A14B --size 1280*720 \
  --ckpt_dir ./Wan2.2-T2V-A14B --prompt "Two anthropomorphic cats boxing"

A few minutes later, you get an 81-frame, 720p, 16fps video. This article explains every step that happens in between. All code comes from the original Wan2.2 repository, and every number can be calculated from the configuration files; I include the formulas as well. After reading it, you should be able to answer these questions:

• Why must the frame count be 4n+1, and why won’t 80 frames work?
• How is the 14B parameter count derived from the configuration? How is the 80GB VRAM threshold calculated?
• Mathematically, how does flow matching differ from DDPM, and why can the former produce videos in just 40 steps?
• What knob does --sample_shift 12 turn, and what invisible linkage does it have with MoE expert switching?
• For image-to-video, which exact lines are changed on top of the text-to-video backbone?

The full article follows the order of “principles → data → network → sampling → engineering.” Each chapter only uses concepts introduced in the previous chapters, so I recommend reading it sequentially. I assume you roughly know what tensors and attention are; if you are starting completely from scratch, spend ten minutes looking up those two terms first. Those ten minutes are worth it.

Three Models, One Stage

Video generation is not done by one model, but three:

Let’s first cushion the word “diffusion” with three sentences, then expand on it in Chapter 1: what the model learns is the direction for “repairing a block of noise toward clean data”; during inference, it starts from pure random noise, asks the model for a direction 40 times, moves 40 times, and the noise gradually develops into a video; your prompt participates in every direction judgment. Remember this skeleton, and every component later can hang on it.

The division of labor is clear: T5 only runs once at the beginning; the VAE encoder is not used at all in t2v, because there is no input image; the decoder only runs once at the end. What really burns compute is the diffusion backbone in the middle: it has to run for 40 steps, with two forward passes per step. Why two? Chapter 5 will cover that.

How the three models are connected is scripted in generate() in wan/text2video.py: encode text → sample noise → loop denoising → decode into pixels. It is worth memorizing the skeleton of this script first; every later chapter expands on one of its lines.

First, let’s resolve a prerequisite question: why doesn’t diffusion operate directly on pixels? Do the math. An 81-frame 720p RGB video has

81 × 720 × 1280 × 3 ≈ 224 million values

After VAE compression, the latent only has

16 × 21 × 90 × 160 ≈ 4.84 million values

That is 46× fewer. The diffusion backbone has to repeatedly compute over this block of data for 80 forward passes. With the data 46× smaller, the cost of both attention and convolution collapses along with it. Generating in compressed space and decoding back to pixels only in the final step is called latent diffusion. It became standard starting with Stable Diffusion, and video generation extends it from 2D to 3D.

Chapter 1: Denoising Takes a Straight Line

Before looking at any code, let’s settle the most fundamental question: why can “fixing noise step by step” generate a brand-new video out of thin air?

Why “Add Noise, Then Denoise” Can Generate

The problem a generative model needs to solve is: sample a new example from the “distribution of natural videos.” This distribution has no analytical form, so there is no direct way to sample from it. Diffusion methods take this approach: build a road whose two ends are natural data and standard Gaussian noise—the latter is easy to sample—then train a network to walk back along this road from the noise end to the data end. The noising process is road construction: it connects a complex distribution to a simple one. The denoising network learns, at every position on this road, “which direction should I move to get closer to data.” During training, the network sees massive numbers of “position → direction” examples. During inference, it starts from random noise and follows the learned direction field. The endpoint is a new sample that looks like the training data, but is not identical to any training example. The prompt participates in the direction decision at every step—the concrete mechanism appears in Chapters 3 and 4—constraining the endpoint to the region that “matches this description.”

From DDPM to Flow Matching

Classic DDPM defines this road like this: the forward process gradually adds Gaussian noise according to a carefully designed variance schedule; the reverse process trains a network to predict the noise added at each step; and sampling peels it away step by step using Bayes’ rule. It works, but the path is a curve produced by a random walk, sampling takes hundreds of steps, and the variance schedule itself is also a hyperparameter that needs tuning.

Flow matching—Wan2.2 uses its simplest form, rectified flow—makes the problem much more direct: between clean data x₀ and pure noise x₁, define the intermediate state by straight-line interpolation,

x_t = (1 - t)·x₀ + t·x₁ ,  t ∈ [0, 1]

Along this straight line, the “velocity” at any moment is constant:

dx_t/dt = x₁ - x₀

So the training objective becomes simple enough to write in four lines of pseudocode:

x0 = sample the latent of a real video clip
x1 = sample standard Gaussian noise
t  = sample a time uniformly from [0, 1]
loss = || model((1-t)·x0 + t·x1, t, text) - (x1 - x0) ||²

No variance schedule, no ELBO derivation—just regress a vector field. The repository only contains inference code, but the training loop looks like this. All of the model’s capability comes from repeating these four lines over hundreds of millions of video clips.

During inference, we walk backward: start from pure noise x₁, numerically integrate along the velocity predicted by the network, and integrate back to x₀. In the ideal case, the trajectory is a straight line and one step would be enough. In practice, the network’s velocity field is not perfect; after taking one step, the true velocity corresponding to the new position has already changed, so we still need to split the journey into steps and ask for directions once per step. But the straight-line prior creates far fewer detours than DDPM’s random walk: DDPM is like driving through a city without GPS, relying on pedestrians for directions and stopping at every intersection; flow matching builds a road that is straight to begin with, so you only need to confirm the direction every so often. It can produce an image in 40 steps, whereas the DDPM era often needed hundreds of steps for comparable quality.

You can see this math directly in the code. The core line in the scheduler file fm_solvers_unipc.py—the main character of Chapter 5—is:

if self.config.prediction_type == "flow_prediction":
    sigma_t = self.sigmas[self.step_index]
    x0_pred = sample - sigma_t * model_output

The derivation is short: write t in the interpolation formula as σ, with x_t = (1-σ)x₀ + σx₁, and let the model output v = x₁ - x₀. Then:

x_t - σ·v = (1-σ)x₀ + σx₁ - σx₁ + σx₀ = x₀

The current sample minus σ times the predicted velocity is exactly the estimate of the clean data. One line of code, one line of algebra.

Two Scales on One Ruler

Later, the numbers σ, t, 1000, and 40 will appear together, so let’s pin down their relationship first. In the derivation, σ ∈ [0, 1]. In the code, the timestep t is the same value multiplied by 1000—the configuration is num_train_timesteps=1000. So t=875 means σ=0.875. They are two scales on the same ruler: the larger t is, the closer it is to noise; the smaller t is, the closer it is to the finished image. During training, the model has seen examples at all 1000 ticks. Inference only chooses 40 points on the ruler to stop at, and the model works as usual at each point. So the number of steps is a free sampling-time parameter: changing --sample_steps to 50 or 20 is valid; it only changes integration accuracy and does not require modifying the model. This is also the entry point for “few-step distillation” work: since the model knows how to move at every tick, can we train it to take longer strides and compress 40 steps down to 4?

Chapter 2 VAE: The Video Is First Shrunk by 46×

Now that the principle is clear, let’s look back at the data. The x₀ in Chapter 1 is not pixels, but latent—this chapter explains how pixels and latent translate into each other, and this translation determines the shape of every tensor across the whole pipeline.

Two config lines in wan/configs/wan_t2v_A14B.py:

t2v_A14B.vae_stride = (4, 8, 8)    # time ×4, height ×8, width ×8
t2v_A14B.patch_size = (1, 2, 2)    # DiT patch size

The time dimension is compressed by 4×, and each spatial dimension by 8×. text2video.py uses this to compute the latent shape:

target_shape = (self.vae.model.z_dim, (F - 1) // self.vae_stride[0] + 1,
                size[1] // self.vae_stride[1],
                size[0] // self.vae_stride[2])

Plug in F=81 and 720×1280, and you get (16, 21, 90, 160): 16 latent channels, 21 “latent frames,” and a 90×160 spatial grid. The 16 channels do not have meanings like red, green, and blue. They are a set of “internal shorthand” learned by the network itself: unreadable to humans, but good enough for the model.

Where 4n+1 Comes From

Notice that the formula for the time dimension is (F - 1) // 4 + 1, not F // 4. The first video frame is encoded separately into one latent frame; after that, every 4 frames are compressed into one. 81 frames = 1 + 80, and 80 is exactly divisible by 4, giving 1 + 20 = 21 latent frames. If you pass in 80 or 82 frames, the division does not come out cleanly, the tensor shapes stop matching, and things crash immediately. frame_num must be 4n+1, and the source of that constraint is right here in this line—the strange default frame counts in the CLI arguments, such as 81, 121, and 77, are not arbitrary at all. They are all 4n+1.

Why is the first frame special? Because the compression uses causal convolution. The first frame has no history before it to look at, so it can only stand alone as its own frame. This design also has a side effect: a static image can be treated as a video with F=1, which happens to occupy exactly one latent frame. Images and videos are therefore unified inside the same VAE representation, a point that will come up again in Chapter 7 when discussing image-to-video.

Causal Convolution and Streaming Inference

Look at the implementation of CausalConv3d in vae2_1.py; it is only a dozen or so lines:

class CausalConv3d(nn.Conv3d):

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self._padding = (self.padding[2], self.padding[2], self.padding[1],
                         self.padding[1], 2 * self.padding[0], 0)
        self.padding = (0, 0, 0)

    def forward(self, x, cache_x=None):
        padding = list(self._padding)
        if cache_x is not None and self._padding[4] > 0:
            cache_x = cache_x.to(x.device)
            x = torch.cat([cache_x, x], dim=2)
            padding[4] -= cache_x.shape[2]
        x = F.pad(x, padding)
        return super().forward(x)

A regular 3D convolution pads symmetrically before and after the time dimension, so the output for each frame can “see” future frames. Here, all time padding is moved to the front, so each frame’s output depends only on the current frame and the past. There is one PyTorch detail worth knowing before reading this code: the arguments to F.pad are counted from the last dimension backward. The first four numbers handle width and height, and only the final pair (2 * self.padding[0], 0) belongs to the time dimension—double padding in front, zero padding behind.

Causality buys streaming processing: the video can be split into small chunks and passed through the network one by one. After each chunk is processed, the last 2 frames (CACHE_T = 2 at the top of the source file) are cached and prepended as cache_x for the next chunk, which is mathematically equivalent to convolving the whole video at once. As a result, the VAE’s memory usage is decoupled from the total video length. For this VAE stage, a 5-second video and a 50-second video have the same peak VRAM usage. The line padding[4] -= cache_x.shape[2] inside forward is just doing the accounting: however much zero padding the cache replaces, padding is reduced by the same amount.

The network itself is a standard hierarchical convolutional autoencoder, from the same family as Stable Diffusion’s VAE: three rounds of spatial downsampling correspond to 8× spatial compression, the latter two stages each halve the time axis to make 4× temporal compression, and the decoder climbs the same staircase back in reverse. Compared with the diffusion backbone, it is so small that it almost does not count in the budget—hundreds of millions of parameters versus 14B—but it determines the upper bound of generation quality. Details that the VAE cannot reconstruct cannot reach the pixel level no matter how well the diffusion model carves them out. Reconstruction quality, compression ratio, and decoding speed form a mutually exclusive triangle, and this is where video model teams quietly compete.

The “Exchange Rate” of Latent Space

The VAE and the diffusion model are two separately trained systems, and they connect through a hardcoded set of statistics. In vae2_1.py:

mean = [-0.7571, -0.7089, -0.9113, ..., -0.2921]   # per-channel means for 16 channels
std  = [ 2.8184,  1.4541,  2.3275, ...,  1.9160]   # per-channel standard deviations for 16 channels
self.scale = [self.mean, 1.0 / self.std]

These 32 numbers are computed from the training set. The raw output of VAE encode has very different scales across channels—the smallest std is 1.13 and the largest is 3.27, a threefold gap—while the diffusion model assumes it is dealing with data close to a standard normal distribution. So before latent enters the diffusion model, it must be standardized channel by channel; before decoding, it must be multiplied back. This set of numbers is the “exchange rate” between this pair of VAE and diffusion model. Swap in another VAE version, and all of it becomes invalid. The repository happens to contain another VAE used only by ti2v-5B: compression ratio 4×16×16, with 48 channels—the model being half the size is not enough; the data must shrink too. This is the key reason the 5B model can run 121 frames on a 24GB consumer GPU. The checkpoints of the two VAE versions are not interchangeable, and the root cause is this exchange rate. After being trained separately, the VAE is frozen; this Wan2.1 VAE version is directly reused by all four Wan2.2 tasks.

Chapter 3: Where Did the Prompt Go?

Every denoising step has to listen to the prompt, so the prompt first has to become something the model can compute.

Wan2.2 uses the encoder from umT5-XXL (Google’s multilingual T5 variant) to encode text: about 5.7B parameters, taking about 11.4GB in bf16—remember these two numbers; Chapter 6 will use them when calculating VRAM. The encoding call path is very short. In t5.py, T5EncoderModel.__call__ is straightforward:

def __call__(self, texts, device):
    ids, mask = self.tokenizer(
        texts, return_mask=True, add_special_tokens=True)
    ids = ids.to(device)
    mask = mask.to(device)
    seq_lens = mask.gt(0).sum(dim=1).long()
    context = self.model(ids, mask)
    return [u[:v] for u, v in zip(context, seq_lens)]

Tokenize, encode, then trim away padding for each item according to the real length from the attention mask. The prompt becomes a sequence of up to 512 vectors, each 4096-dimensional. This is the context. It has only one use inside the DiT: every layer’s cross-attention uses the video tokens as query, and the context as key/value. Text does not directly draw any pixels. Instead, during every denoising step, it continuously gives the model semantic direction—the prompt is not an order issued before work begins; it is the construction blueprint hanging on the wall for the entire job.

Why use T5 instead of the CLIP text tower commonly used in early text-to-image models? CLIP embeddings are good at “roughly what this sentence is about,” but the 77-token limit cannot fit detailed scene descriptions, and it is also weak on quantities and spatial relationships. T5 encodes token by token according to the structure of the text itself, supports up to 512 tokens, and works with token-level cross-attention, allowing the model to distinguish details like “the cat on the left is wearing red boxing gloves.” Choosing the multilingual umT5 has another consideration as well: Chinese prompts are first-class citizens. The official negative prompt is simply written in Chinese.

This refers to the negative prompt. The default value in the config is exactly:

wan_shared_cfg.sample_neg_prompt = '色调艳丽，过曝，静态，细节模糊不清，字幕，风格，作品，画作，画面，静止，整体发灰，最差质量，低质量，JPEG压缩残留，丑陋的，残缺的，多余的手指，画得不好的手部，画得不好的脸部，畸形的，毁容的，形态畸形的肢体，手指融合，静止不动的画面，杂乱的背景，三条腿，背景人很多，倒着走'

A checklist of “common failure scenes.” Its purpose will be discussed in Chapter 5 on CFG. For now, just remember: the negative prompt also goes through T5 and produces context_null.

One last prerequisite step: a prompt casually written by the user is often only one sentence, while the model was trained on long, detailed descriptions. Feeding it an overly short prompt reduces output quality. --use_prompt_extend first uses an LLM (DashScope online API or local Qwen2.5) to expand your prompt into a longer description with camera, lighting, action, and other details, then sends it to T5. In multi-GPU mode, expansion runs only once on rank 0 and is then broadcast to the other GPUs—LLM generation is random, so if 8 GPUs each expanded the prompt independently, you would get 8 different prompts, and the video could no longer be stitched together.

Chapter 4 DiT: Attention Over 75,600 Tokens

The pieces are in place: data lives in latent space (Chapter 2), text has become context (Chapter 3), and the learning target is the velocity field (Chapter 1). Now let’s look at the machine that predicts velocity: this is where all 14B parameters sit.

Before entering the backbone, it is worth answering a more fundamental question: what is the essential difference between a video model and an image model?

The answer is not as simple as “it has one more dimension.” If you call an image model frame by frame to generate 81 images, each frame may look beautiful on its own, but playing them together becomes a disaster: object shapes drift from frame to frame, lighting jumps, backgrounds shimmer. In the jargon, this is called flicker. Temporal consistency has to be modeled explicitly. The industry has explored two paths: factorized attention (spatial attention and temporal attention are done separately: first make each frame internally coherent, then align along the time axis; cheaper, but with a ceiling on consistency) and full 3D joint attention (all patches from all frames are placed into the same attention operation, so any token can directly see any position at any moment; strongest consistency, but sequence length explodes). Wan2.2 takes the second path, and that choice directly leads to the protagonist of this chapter: a sequence of 75,600 tokens.

From Latent to Token

Before the latent enters the DiT, it is first split into patches. This step is called patchify. In model.py, it is simply a Conv3d whose stride equals its kernel size:

self.patch_embedding = nn.Conv3d(
    in_dim, dim, kernel_size=patch_size, stride=patch_size)

patch_size = (1, 2, 2): the time dimension is not split, while every 2×2 spatial block in the latent is convolved into a 5120-dimensional token. Counting the total:

Tokens per latent frame = 90 × 160 / (2×2) = 3600
Total tokens            = 3600 × 21 frames = 75600

75,600 is the self-attention sequence length when generating a 5-second 720p video. SDXL generating a 1024 image has a sequence length of 4096; video is 18 times longer than image. Attention compute grows with the square of sequence length, so 18× length means about 340× attention cost. All the VRAM issues and sequence parallelism discussed later ultimately originate from this number.

How 14B Is Counted

The config gives dim=5120, ffn_dim=13824, num_layers=40. The parameter bulk in each layer is:

self-attention:  four Q/K/V/O matrices = 4 × 5120² ≈ 105M
cross-attention: same four matrices    = 4 × 5120² ≈ 105M
FFN:             two matrices          = 2 × 5120 × 13824 ≈ 142M
Per layer total ≈ 351M, × 40 layers ≈ 14.05B

That is where the 14B in “A14B” comes from; embeddings and modulation parameters are only a rounding error. bf16 uses 2 bytes per parameter, so one expert is 28GB. t2v-A14B has two experts (Chapter 6 explains why), meaning the DiT weights alone take 56GB. Add T5’s 11.4GB, the VAE, and activations during sampling, and this is how the 80GB single-GPU threshold gets filled.

The Full Forward Pass of One Block

WanAttentionBlock.forward is the smallest repeating unit of the whole model, and it is worth reading line by line. Before reading it, one reminder: the parameter e comes from timestep t—the tick mark on the ruler from Chapter 1, where 999 is almost pure noise and 0 is the finished video. How it becomes e will be explained right after the code.

def forward(self, x, e, seq_lens, grid_sizes, freqs, context, context_lens):
    assert e.dtype == torch.float32
    with torch.amp.autocast('cuda', dtype=torch.float32):
        e = (self.modulation.unsqueeze(0) + e).chunk(6, dim=2)
    assert e[0].dtype == torch.float32

    # self-attention
    y = self.self_attn(
        self.norm1(x).float() * (1 + e[1].squeeze(2)) + e[0].squeeze(2),
        seq_lens, grid_sizes, freqs)
    with torch.amp.autocast('cuda', dtype=torch.float32):
        x = x + y * e[2].squeeze(2)

    # cross-attention & ffn function
    def cross_attn_ffn(x, context, context_lens, e):
        x = x + self.cross_attn(self.norm3(x), context, context_lens)
        y = self.ffn(
            self.norm2(x).float() * (1 + e[4].squeeze(2)) + e[3].squeeze(2))
        with torch.amp.autocast('cuda', dtype=torch.float32):
            x = x + y * e[5].squeeze(2)
        return x

    x = cross_attn_ffn(x, context, context_lens, e)
    return x

There are three sections: Self-Attention (75,600 tokens looking at one another to establish spatiotemporal consistency), Cross-Attention (looking at the text context to inject semantics), and FFN. The truly interesting part is e, which is chunked into 6 pieces:

Where do these 6 groups of parameters come from? The scalar t (for example, 927) first goes through a standard sinusoidal embedding and expands to 256 dimensions—the network has poor resolution over a single scalar, so after expansion, high-frequency channels handle fine granularity while low-frequency channels handle coarse granularity—then it passes through an MLP and a projection:

self.time_embedding = nn.Sequential(
    nn.Linear(freq_dim, dim), nn.SiLU(), nn.Linear(dim, dim))
self.time_projection = nn.Sequential(nn.SiLU(), nn.Linear(dim, dim * 6))

dim * 6 corresponds exactly to those 6 modulation chunks. This technique is called AdaLN (adaptive layer norm), one of the core contributions of the DiT paper: instead of concatenating t into the sequence as a token, t modulates the scale, shift, and residual gates of each layer’s normalization. Intuitively, return to the denoising trajectory from Chapter 1: when t=999, the input is almost pure noise, so the network should make bold strokes; when t=10, the image is basically formed, so the network should only make small refinements. The same set of weights switches behavior modes through these 6 groups of parameters that vary with t, like the same chef automatically changing heat levels according to how close the dish is to completion. The DiT paper compared two schemes—“concatenate t as a token” versus “AdaLN modulation”—and the latter achieved significantly better FID under the same compute. It has since become the default design for diffusion transformers.

There is another easily missed detail in the code: all modulation operations are wrapped in autocast float32. The backbone can run in bf16, but these modulation steps are forced to full precision. Denoising is an iterative process over dozens of steps, and numerical error in the modulation parameters can accumulate gradually. This is where the authors spend VRAM in exchange for stability.

3D RoPE: Letting Tokens Know Where They Are

The 75,600 tokens are arranged into a one-dimensional sequence, and attention itself does not know who is next to whom. Positional information is injected by RoPE (rotary position embedding). The special part in the video version is that coordinates have three axes. During construction, each head’s 128 dimensions are split into three sections:

d = dim // num_heads          # 5120 // 40 = 128
self.freqs = torch.cat([
    rope_params(1024, d - 4 * (d // 6)),   # time axis: 44 dims
    rope_params(1024, 2 * (d // 6)),       # height axis: 42 dims
    rope_params(1024, 2 * (d // 6))        # width axis: 42 dims
], dim=1)

When applied, each token takes three segments of rotation values according to its own (f, h, w) coordinates, concatenates them into a complete 128-dimensional rotation, and applies it to q and k in one step using complex multiplication. When attention computes the q·k inner product, the relativity of rotation makes the inner product depend only on the coordinate difference between the two tokens: shift the entire image by two cells, and all relative relationships remain unchanged, so the patterns learned by the model still apply. Translation invariance comes for free. This is a direct extension of image 2D RoPE to video. The 44/42/42 allocation means the time axis receives roughly the same number of frequency channels as a single spatial axis, so temporal and spatial relationships are treated with comparable importance.

Detail: the rotation implementation (rope_apply) runs entirely in float64 and explicitly disables autocast. Rotation is a purely geometric operation; half-precision phase errors can be amplified on long sequences, so this is another tradeoff of spending compute for numerical stability. In addition, inside WanSelfAttention, after q and k pass through their linear layers, each is followed by an RMSNorm (QK-Norm) to clamp the scale of the inner products and prevent softmax saturation. Its value is not obvious from inference code, but without it, a 14B-scale DiT is very hard to train stably.

Shape bookkeeping for one forward pass

Putting the whole chapter into one table, here is every shape transformation from model input to model output (t2v-A14B, 81 frames, 720p, single GPU):

Whatever shape goes in is the shape that comes out; the middle consists of 40 equal-width refinement layers. The semantics of the output are the velocity from Chapter 1, corresponding point-by-point to the input latent: the value at each position indicates “which direction the content at this position should move.” The text context (512 × 4096 projected by text_embedding to 512 × 5120) exists throughout only as the key/value for cross-attention and does not occupy sequence positions.

At this point, the machine that “takes a noisy latent and a t, then outputs a velocity prediction” is complete. The next chapter looks at how this prediction is used by the sampling loop.

Chapter 5 Sampling Loop: 40 Steps, Two Forward Passes per Step

The machine only reports a direction each time; what really walks the video out of noise is the loop around it: choose 40 stops, and ask for directions twice at each stop.

UniPC: How to Move Fast and Steadily

Once we have the velocity field, we still need a numerical integrator: at each step, take the velocity reported by the model and decide how far to move forward. This component is called the scheduler in code, and fm_solvers_unipc.py is exactly that. The simplest integration is Euler’s method: assume the velocity is constant at each step, x ← x - Δσ·v. It is first-order accurate. But the real velocity field is curved, so the larger the step, the farther it drifts. Wan2.2’s default UniPC is a multistep predictor-corrector: the predictor uses the model outputs from the most recent two steps, configured as solver_order=2, to fit the trend of velocity change and extrapolate to the next step; after the model output at the new position is computed, the corrector goes back and revises the step just taken, gaining another order of accuracy for free without spending an extra forward pass. With the same 40 model calls, the local error drops from Euler’s first-order small term to third order. This is how the step budget for “acceptable quality” gets cut from hundreds to dozens—video generation needs two 14B forward passes per step, so every saved step is real money.

shift: Where the 40 Stops Land

The scheduler is also responsible for deciding where the 40 sampling points are placed. set_timesteps first lays points out uniformly, then applies the shift transform:

sigmas = shift * sigmas / (1 + (shift - 1) * sigmas)

This transform squeezes the points toward the high-noise end, where σ is large. How much does it squeeze? Look at the median:

When shift=12, half of the sampling points fall in the t>926 interval. The actual timestep sequence starts like this, with shift=12 and 40 steps; you can verify it yourself by running set_timesteps, with the output rounded to integers:

999, 997, 995, 993, 990, 988, ...

The first few steps are spaced only about 2 apart, so the loop walks extremely densely; by the low-noise stage, the spacing stretches to dozens, and it crosses the stage in just a few steps.

Why allocate it this way? The high-noise stage determines composition, objects, and motion direction—the “from nothing to something” phase. The low-noise stage mostly polishes details. Tilting the budget toward the harder phase is a tradeoff video generation needs more than image generation does: image models usually use a shift of around 3, but motion consistency in video is much harder, so Wan2.2 raises it directly to 12. In the same repository, i2v uses 5.0 because the first frame is already given and the structural pressure is lower; ti2v-5B uses 5.0; s2v uses 3. Each task’s shift tells its own difficulty distribution. --sample_shift does not change the number of steps, only how those steps are allocated. Tuning it means moving budget between “more stable structure” and “finer details”—and it also meshes tightly with the expert split in the next chapter. We will do the numerical accounting there.

CFG: Why Each Step Runs Two Forward Passes

The core of the sampling loop, in text2video.py:

noise_pred_cond = model(latent_model_input, t=timestep, **arg_c)[0]
noise_pred_uncond = model(latent_model_input, t=timestep, **arg_null)[0]

noise_pred = noise_pred_uncond + sample_guide_scale * (
    noise_pred_cond - noise_pred_uncond)

For the same latent and the same t, it runs twice: once with your prompt, where arg_c contains context, and once with the negative prompt, where arg_null contains context_null. This is classifier-free guidance (CFG).

The principle deserves one layer of math. To generate samples that satisfy condition c, decompose it with Bayes’ rule:

p(x|c) ∝ p(x) · p(c|x)

Take the log and then the gradient, the score:

∇log p(x|c) = ∇log p(x) + ∇log p(c|x)

The first term is the unconditional direction, what the data itself looks like. The second term is the classifier gradient that “makes x look more like it belongs to condition c.” Early methods really trained a separate classifier to provide it, which was expensive and hard to train. CFG’s insight is that this term can be obtained for free: the model has learned both conditional and unconditional predictions, by randomly dropping text during training, so subtracting the two gives:

∇log p(c|x) = ∇log p(x|c) - ∇log p(x) ≈ conditional prediction - unconditional prediction

The classifier gradient is hidden in the difference between the two forward passes. During sampling, amplify this term by w:

final direction = uncond + w · (cond - uncond)

Compare this with the three lines of code above: it is exactly the same. sample_guide_scale is w.

Choosing w is a pure tradeoff. w=1 degenerates into conditional generation: the image looks natural, but often “doesn’t understand” the prompt. Raise w, and the model is strongly pushed toward regions where the “classifier” is more confident; composition obedience improves, at the cost of oversaturated colors, mushy details, and stiff motion, because the latent is pushed away from the high-density region of the training distribution. Image models often use around 7. Video is more sensitive to temporal naturalness, so Wan2.2’s A14B dual-expert series only uses 3.0–4.0; ti2v-5B uses 5.0, and s2v uses 4.5.

The role of the negative prompt in this derivation is also clear: standard CFG uses empty text for uncond; Wan2.2 replaces it with that string describing “overexposure, deformation, three legs,” and so on. The difference direction changes from “move closer to your prompt” into “move closer to your prompt while moving away from these defects.” One forward pass does two jobs at once.

The cost is just as clear: 40 steps × 2 forward passes = 80 full inferences through the 14B model. Video generation is slow, and half the bill is charged to CFG.

Chapter 6: Two 14B Experts Taking Turns

The denoising loop and CFG in the previous chapter assume that one model is running behind the scenes. In fact, the A14B series hides two. This chapter explains how they divide the work, and why that directly determines whether your GPU has enough memory.

The signature design of the Wan2.2 A14B series is that two 14B experts handle different noise ranges. In the config:

t2v_A14B.boundary = 0.875
t2v_A14B.sample_guide_scale = (3.0, 4.0)  # low noise, high noise

At each step, the sampling loop checks t >= 0.875 × 1000: if true, it uses high_noise_model; otherwise it uses low_noise_model. The motivation follows from the observation in Chapter 5: the high-noise stage determines structure out of chaos, while the low-noise stage polishes textures on a semi-finished result. Instead of forcing one network to master both jobs, it is better to let two networks each specialize in one. And this division already holds from the very first layer: the high-noise expert receives something close to pure Gaussian noise, so its early layers are “guessing structure from statistics”; the low-noise expert receives a semi-finished image, so its early layers are “reading the existing picture.” The input distributions are so different that sharing the lower layers would only drag both sides down. So the two experts are fully independent, trained separately on their own noise ranges, and take turns at inference time according to t.

This is what the official “27B total parameters, 14B active” means: 27B is the precise count, while the 14.05B from Chapter 4 only counted the major parts of each layer. Capacity doubles, but only one expert runs at any given step, so the compute cost is equivalent to a dense 14B model. Don’t be misled by the word “MoE”: LLM MoEs like Mixtral choose experts per layer and per token using a learned router. Here, routing happens at the timestep level, the threshold is hard-coded, there is no router at all, and the whole implementation is just an if. This coarse granularity brings two benefits: diffusion timesteps are already natural difficulty labels, so there is no need to train a router; and the two experts can be trained and upgraded independently.

Here is the full sampling loop. All the parts discussed in previous chapters come together in these 20 lines:

for _, t in enumerate(tqdm(timesteps)):
    latent_model_input = latents
    timestep = [t]

    timestep = torch.stack(timestep)

    model = self._prepare_model_for_timestep(
        t, boundary, offload_model)
    sample_guide_scale = guide_scale[1] if t.item(
    ) >= boundary else guide_scale[0]

    noise_pred_cond = model(
        latent_model_input, t=timestep, **arg_c)[0]
    noise_pred_uncond = model(
        latent_model_input, t=timestep, **arg_null)[0]

    noise_pred = noise_pred_uncond + sample_guide_scale * (
        noise_pred_cond - noise_pred_uncond)

    temp_x0 = sample_scheduler.step(
        noise_pred.unsqueeze(0), t,
        latents[0].unsqueeze(0),
        return_dict=False, generator=seed_g)[0]
    latents = [temp_x0.squeeze(0)]

Each round does five things: choose the expert, choose the guide scale, run two forward passes, combine them with CFG, and let the scheduler take one step. _prepare_model_for_timestep also contains the single-GPU lifesaving offload logic:

if offload_model or self.init_on_cpu:
    if next(getattr(self, offload_model_name).parameters()).device.type == 'cuda':
        getattr(self, offload_model_name).to('cpu')
    if next(getattr(self, required_model_name).parameters()).device.type == 'cpu':
        getattr(self, required_model_name).to(self.device)

With --offload_model True enabled, each time the expert switches, the off-duty one is moved back to system memory and the on-duty one is moved onto the GPU. Moving 28GB of weights across PCIe takes several seconds, but fortunately the whole sampling process switches only once: t decreases monotonically, and once it crosses the boundary, it never goes back. So this cost is paid only once.

How boundary and shift Interact

This is the easiest piece of accounting to miss when tuning. boundary=0.875 looks, literally, like “only 12.5% of timesteps belong to the high-noise expert.” But as Chapter 5 explained, shift aggressively squeezes the sampling points toward the high-noise end. Counting the 40 steps where t ≥ 875 gives:

Most of the budget goes to the high-noise expert that determines structure, while the low-noise expert uses the remaining 14 steps to finish things off. If you tune --sample_shift alone without changing boundary, the workload split between the two experts changes silently, and the resulting quality fluctuations can be hard to reason about. The two guide_scale values also serve this division of labor: the high-noise stage uses 4.0 to stay close to the prompt during structure formation, while the low-noise stage drops to 3.0 so the detail stage is pushed less aggressively and the image looks more natural.

Finally, let’s clear up a common misunderstanding: only one expert appears at each timestep. The two forward passes are the CFG cond and uncond passes; the experts never work simultaneously in the same step. “Two forward passes per step” and “two experts” are orthogonal things.

VRAM Accounting and the Three-Piece Kit

Run the numbers again: two experts in bf16 weights take 56GB, the T5 encoder takes 11.4GB, plus the VAE, activations, KV cache, and other overhead. An 80GB card can just barely fit everything. When a single card cannot handle it, the three-piece kit targets specific items on the bill: --offload_model True moves the currently unused expert back to system memory, removing 28GB of resident VRAM; --t5_cpu keeps T5 in system memory and only moves it onto the GPU during encoding, saving 11.4GB; --convert_model_dtype lowers weight precision.

How should you combine them? The README’s example commands give the official answer: the single-GPU A14B example already includes --offload_model True --convert_model_dtype and notes that at least 80GB of VRAM is required. For running ti2v-5B on a 4090, all three are added, including --t5_cpu; in the 5B scenario, T5’s 11.4GB becomes proportionally the biggest item. The README’s ti2v-5B section also gives the reverse hint: if VRAM exceeds 80GB, remove these flags for a clear speedup, because offloading is not free.

Chapter 7 Conditional Injection: Which Lines Changed for Image-to-Video

By this point, you’ve already walked through the complete text-to-video pipeline. The remaining four tasks—image-to-video, text-image unified generation, audio, and animation—are not four brand-new model families. They are different “prompts” swapped onto the same backbone.

There are only three ways to add conditions:

1. cross-attention: the path used by text, where the condition is attended to as key/value
2. channel concatenation: the condition tensor is directly concatenated with the noise latent along the channel dimension, then fed into patch embedding together
3. AdaLN: the condition is mixed into the timestep embedding to modulate each layer’s norm

Image-to-video (i2v-A14B) uses the second method. Look at how image2video.py constructs the condition:

msk = torch.ones(1, F, lat_h, lat_w, device=self.device)
msk[:, 1:] = 0
msk = torch.concat([
    torch.repeat_interleave(msk[:, 0:1], repeats=4, dim=1), msk[:, 1:]
], dim=1)
msk = msk.view(1, msk.shape[1] // 4, 4, lat_h, lat_w)
msk = msk.transpose(1, 2)[0]
...
y = torch.concat([msk, y])

Line by line: msk is the marker for “which frames are known.” The first frame—the input image—is marked as 1, and the remaining 80 frames are marked as 0. The following repeat_interleave plus reshape sequence does one thing: it folds the 81-frame pixel-level mask onto the 21-frame latent timeline. Remember the VAE compression rule from Chapter 2? The first frame gets its own latent frame, and every 4 frames after that are merged into one. The mask has to be folded by the same rule: copy the first frame 4 times to complete the grouping, then fold every 4 frames into one group, yielding a 4×21×h×w mask aligned with the latent.

y is the input image itself after passing through the VAE encoder, with the missing frames padded by zeros. The mask and y are concatenated into 20 channels, then concatenated once more with the 16-channel noise latent along the channel dimension before entering the model. At the start of forward in model.py:

if y is not None:
    x = [torch.cat([u, v], dim=0) for u, v in zip(x, y)]

The i2v version of WanModel has 36 input channels: 16 noise channels + 16 image latent channels + 4 mask channels. The first convolution in patch embedding is wider than in the t2v version, while the remaining 40 layers have exactly the same structure. The entire backbone change for image-to-video is simply that the input channels go from 16 to 36.

What the model learns from these 36 channels is: copy the given content where mask=1, generate where mask=0, and keep the two spatially and temporally coherent. The first frame pins down the subject and style of the image, while the next 80 frames unfold motion around it. That is the mechanism behind “making an image move.”

Where the Conditions Come From in Other Tasks

Read the other three tasks with the same mindset. Focus on: “What is the condition, and through which entrance does it enter the model?”

ti2v-5B (unified text-image generation). The same 5B backbone is paired with a high-compression 4×16×16 VAE, supporting both t2v and i2v without changing the architecture. If an image is provided, it is encoded, pinned into the first latent-frame position, and marked as known with a mask. If no image is provided, generation starts from pure noise. One model, two faces.

s2v-14B (audio-driven generation). The condition is audio: wav2vec2 turns the waveform into frame-level features, which are re-bucketed and aligned to the video frame rate before being injected into the model, driving lip motion and body rhythm. The video length follows the audio length. Extra-long audio is split into multiple segments and generated autoregressively, with each segment using the last few latent frames from the previous segment as a “momentum condition” for continuity. This is the only long-video solution in the repository.

animate-14B (character animation/replacement). This is the most condition-heavy task: a skeletal pose sequence drives motion, a facial feature sequence drives expressions, the reference image goes through CLIP visual encoding and enters cross-attention to lock identity, and in replacement mode there is also a background video and mask. Multiple condition streams enter through their own separate openings.

The condition sources vary wildly, but the injection methods are always permutations of the same three ideas: channel concatenation for spatially aligned conditions such as masks, poses, and backgrounds; cross-attention for global semantic conditions such as text and reference-image features; and frame-level feature injection for temporally aligned conditions such as audio. Once you understand one task, the rest are just combinations.

Chapter 8: What Does 75600² Actually Mean?

We have finished explaining how the model works and how conditions are injected. What remains is a pure engineering problem: with this much computation, what do you do when a single GPU cannot handle it? This chapter returns to the 75600 from Chapter 4 and looks at how it forces a multi-GPU solution.

Let’s calculate the attention bill in detail. The core computation of a single self-attention layer is two matrix multiplications, QK^T and AV. The FLOPs are approximately:

2 × 2 × L² × dim = 4 × 75600² × 5120 ≈ 1.17 × 10¹⁴

(The two 2s mean: one for the two matrix multiplications, and one because each multiply-add pair counts as two FLOPs.) One layer is 117 TFLOPs. Across 40 layers and 80 forward passes, attention alone reaches the order of 3.7 × 10¹⁷ FLOPs. An A100 has a bf16 peak throughput of about 312 TFLOPS, so even under ideal utilization this part alone would take around twenty minutes, without counting the FFN. This is the physical baseline behind “a 5-second video takes minutes to tens of minutes to generate.”

If one GPU cannot handle it, split it. Wan2.2’s main solution is Ulysses sequence parallelism. The core communication primitive lives in distributed/util.py: a dimension exchange.

def all_to_all(x, scatter_dim, gather_dim, group=None, **kwargs):
    world_size = get_world_size()
    if world_size > 1:
        inputs = [u.contiguous() for u in x.chunk(world_size, dim=scatter_dim)]
        outputs = [torch.empty_like(u) for u in inputs]
        dist.all_to_all(outputs, inputs, group=group, **kwargs)
        x = torch.cat(outputs, dim=gather_dim).contiguous()
    return x

Attention requires every token to see the full sequence, so you cannot simply split the sequence and call it done. Ulysses’ answer is: “split heads, not the sequence.” On 8 GPUs, tracing the tensor shapes through one pass gives the following (B=1, L=75600, 40 heads):

Two rounds of communication reduce both attention computation and activation memory per GPU to 1/8, because each GPU only computes 1/8 of the attention heads. The number of heads must be divisible by the number of GPUs; 40 heads with 8 GPUs fits perfectly. generate.py contains the corresponding assert.

There is also a sequence-length detail prepared specifically for multi-GPU execution. In text2video.py, the tail of the seq_len formula includes a pair of ceil(... / sp_size) * sp_size, rounding the token count up to an integer multiple of the GPU count. 75600 happens to be divisible by 8, so it remains unchanged. If another resolution produces 75601 tokens, it is padded to 75608, ensuring that each GPU receives an equal-length slice. On a single GPU, sp_size=1, so this expression is an identity; it only reveals its purpose in the multi-GPU setting.

The communication cost can also be calculated: per layer, q/k/v/output require four all_to_all calls, totaling about 3.1 GB of data. Across 40 layers and 80 forward passes, that accumulates to about 10 TB. That sounds scary, but NVLink inter-GPU bandwidth is on the order of hundreds of GB per second, so spread across the whole generation process, communication time is much smaller than compute time. The trade is worthwhile. On PCIe machines, however, this may not hold; multi-GPU speedup will shrink noticeably. As a side note, the competing approach Ring Attention splits the sequence and circulates K/V around the ring. It is not constrained by divisibility of head count, but its communication is more complex. Ulysses is capped by the number of heads: the GPU count cannot exceed 40 heads, so a 64-GPU cluster needs another scheme.

Parameters not fitting is another axis: FSDP (fully sharded data parallel) splits model weights into N shards and places them across GPUs. During the forward pass, when execution reaches a given layer, all GPUs first all-gather that layer’s parameters, compute the layer, then immediately release them. With 8 GPUs, the resident DiT weights per GPU drop from 56 GB to 7 GB. The cost is one all-gather per layer, but on NVLink most of this overhead can be overlapped with computation.

Putting the single-GPU and 8-GPU memory accounts side by side:

Sequence parallelism handles compute; FSDP handles memory. They are orthogonal, so --dit_fsdp --ulysses_size 8 can be enabled at the same time. The 8-GPU command in the README includes both of them for a reason: every flag corresponds to a line in the accounting.

Chapter 9: Why the Model Makes the Mistakes It Often Makes

Most typical video-generation failures can be traced back to the principles discussed earlier. This chapter connects the “symptoms” to the “mechanisms,” which is more useful than memorizing a checklist.

Broken fingers, text, and fine structures. This is the result of two levels of compression working together. The VAE compresses an 8×8 pixel block into one latent position, and patchify then merges 2×2 latents into one token, so one token governs 16×16 pixels. A finger is often only a dozen or so pixels wide; an entire finger may fall into one or two tokens. No matter how strong attention is, it cannot distinguish the boundaries of five fingers inside a token. Text is even worse, because strokes are pixel-level details. The only remedies are lowering the compression ratio or increasing resolution, both of which run directly into the compute wall. This is a structural weakness of the current generation of architectures, not the fault of the training data.

Drift in the later part of long videos. t2v generates 81 frames in one shot, with inter-frame consistency enforced by full 3D attention, so it works fine within 5 seconds. Longer videos (s2v’s segmented autoregression) rely on “the last few frames of the previous segment” to pass state forward. Each handoff is lossy: errors accumulate segment by segment, and a character’s face slowly changes, clothing colors gradually drift. This is isomorphic to drift in long-form LLM text—the original sin of autoregression.

Small motion range and static camera work. Multiple factors push in the same direction: CFG pushes the latent toward the region where the “classifier is most confident,” and static compositions are always “safer” than big movements; calm camera shots make up the majority of the training data; words like “static” and “still” in the negative prompt are there to fight this tendency (the default negative prompt includes “a still, motionless image,” so the author was clearly annoyed by this problem). Lowering guide_scale and explicitly describing camera movement in the prompt both loosen the constraints on motion.

Physical inconsistency: clipping, strange liquids, drifting shadows. The model learns “what the next frame commonly looks like in pixel statistics,” without any explicit physics engine. Physical correctness only emerges in scenes densely covered by the training data. In sparse-data areas—complex occlusion, fluids, mirror reflections—it breaks down. This is a known boundary of the purely data-driven path, and the world-model direction the industry is exploring targets exactly this layer.

Large differences across runs with the same prompt. Generation starts from random noise. The prompt only constrains “which semantic region the endpoint lands in”; the specific appearance inside that region is determined by the seed. This is a feature, not a bug: fixing the seed is what makes controlled comparisons possible. The hands-on experiments at the end all require a fixed seed for exactly this reason.

Chapter 10 Symptom-to-Knob: A Quick Tuning Cheat Sheet

Now that the principles are clear, translate them into “what should I tweak when I’m unhappy with the output?” The rationale for each row appears in the earlier chapters:

One more reminder beyond the table: adjusting --sample_shift silently changes how the workload is divided between the two experts; see the linked accounting in Chapter 6.

Settle the Accounting

Returning to the command at the beginning, every stage now has numbers attached:

• The prompt is turned by umT5 (5.7B) into ≤512 vectors of 4096 dimensions; the negative prompt is processed the same way
• Random-noise latent: 16 × 21 × 90 × 160, determined by --base_seed; reproducible with the same seed
• Patchified into 75,600 tokens of 5120 dimensions
• 40-step UniPC sampling; shift=12 squeezes 26 steps into the high-noise region of t≥875, handled by the high-noise expert (14B), then the final 14 steps switch to the low-noise expert
• 2 forwards per step (CFG, guide_scale 4.0 for the high-noise stage and 3.0 for the low-noise stage), totaling 80 14B inference passes; attention compute is on the order of 10¹⁷ FLOPs
• The latent is stream-decoded by the causal-convolution VAE back into 81 frames at 720p, then encoded to disk with libx264

The bulk of the time is spent on the 80 middle 14B forward passes, taking up more than 90% of the whole run; T5 and the VAE each run only once. So every speedup technique targets the sampling loop: fewer steps, less CFG (distillation), splitting attention across more GPUs. Nobody bothers optimizing T5 or the VAE, because that is not where the bill is.

There is no magic in this system, only the combination of four things: a compressor (VAE), a large network that learns the velocity field in compressed space (DiT), numerical integration that moves quickly and stably (flow matching + UniPC + shift), and conditioning mechanisms that inject semantics (cross-attention + CFG + task-specific injections). The skeleton is the post-2024 industry common denominator. Read another video model—Kling, Sora, Veo—and the parts differ, but the way they are bolted together is similar. Each company’s real moat is its data recipe, which happens to be the part you cannot see in an open-source repository.

If you can take away only three sentences from the whole piece, take these:

1. Generation happens on thumbnails: the video is first shrunk 46× by the VAE; the large model touches only the compressed representation throughout, and only enlarges it back to pixels at the final step.
2. Denoising is straight-line navigation that asks for directions 40 times: each time, the model reports a direction (and asks both the positive and negative sides, then takes the difference—that is CFG); the scheduler moves one step along that direction; the first 26 steps set the structure, the last 14 polish the details, and the two experts hand off at tick 875.
3. Every engineering problem comes from one number: quadratic attention over 75,600 tokens. VRAM, speed, and multi-GPU schemes are all fighting against it.

What to Read Next, and What to Try Hands-On

Read the papers along this path; each one maps to one or two chapters of this article:

A hands-on path builds intuition faster than papers. I recommend this order:

1. Run the demo with ti2v-5B on a consumer GPU, fixing --base_seed.
2. Change only --sample_shift (for example, run 3 / 5 / 12 once each). Compare them with the same seed and visually inspect how the budget shifts between “structure vs. detail.”
3. Change only --sample_guide_scale (1 / 3 / 7), and observe the trade-off between prompt adherence and oversaturation.
4. Set a breakpoint in the sampling loop of text2video.py; at each step, print t.item(), latents[0].std().item(), and the current expert. Compare this against the tables in Chapters 5 and 6, and manually reproduce the 26/14 step allocation.
5. Pick any three of the 16 latent channels and render them as RGB. Watch the “ghost” of the video gradually take shape—turning the slogan “diffusion carves structure in latent space, and the VAE fills in details” into intuition.
6. Read the mask construction in image2video.py, compare it with Chapter 7, then ask yourself: if the task were “generate the middle given the first and last frames,” how should the mask change? The industry calls this task FLF2V; Wan2.1 released this variant, and you can use its implementation to check your answer.

On the engineering side, this codebase still has quite a few pitfalls where the implementation diverges from the documentation. The pitfall checklist is in another article: After Reading All of the Wan2.2 Source Code. The theory and the gotchas are best read together.

After reading through the Wan2.2 repository from generate.py all the way to the schedulers, what is really worth writing down is not “how to use this model,” but how this code can bite you when you are not paying attention.

Wan2.2 is an open-source video generation model library from Alibaba’s Wan team. It has a single generate.py entry point and uses --task to branch into five inference pipelines: text-to-video, image-to-video, unified text-image-to-video, speech-to-video, and character animation. On the surface, it looks like “one model, five ways to play.” Once you open the code, it feels more like five parallel worlds glued together by a thin CLI layer. Below, I’ll go through it in the order I read the code, marking the pitfalls where they appear and pasting the source code directly in the key places.

This article is about engineering pitfalls. If you want to first understand the principles behind video generation itself—VAE compression, flow matching, CFG, and the math and numbers behind the MoE dual experts—read the companion piece: Learning Video Generation Through the Wan2.2 Source Code. The two are best read together.

Five Tasks, Five Input Sets, Five Pitfall Sets

Wan2.2 is a task matrix. t2v-A14B, i2v-A14B, ti2v-5B, s2v-14B, and animate-14B each require completely different assets: plain text, a reference image, a reference image plus audio, or even a video plus a pose asset package produced by preprocessing. Which task to use and what inputs to prepare—the answer lives in the argument validation code in generate.py; the docs only tell half the story.

First, look at this default fallback logic in generate.py:

if args.prompt is None:
    args.prompt = EXAMPLE_PROMPT[args.task]["prompt"]
if args.image is None and "image" in EXAMPLE_PROMPT[args.task]:
    args.image = EXAMPLE_PROMPT[args.task]["image"]

If you don’t pass --prompt, the code doesn’t error out; it silently applies the hardcoded sample prompt: a white cat wearing sunglasses surfing. s2v-14B and i2v-A14B even share the same sample. Run these two tasks without passing a prompt, and the results will have the same style, because both tasks are just idling on the official demo assets. --image, --audio, and --tts_* all have similar fallbacks. After beginners get the demo running for the first time, they often mistakenly think their own configuration has taken effect. This is the number one source of confusion.

The meaning of --size does not line up across tasks. In t2v/ti2v, it means resolution; in i2v/s2v, it is effectively the target pixel area, and the final aspect ratio follows the input image. Filling in 1280*720 does not guarantee the output will be that size. frame_num must satisfy 4n+1, and the default values differ across tasks (t2v/i2v use 81, ti2v-5B uses 121, and animate-14B uses 77). Behind this is the VAE temporal compression ratio, so this number cannot be changed casually. Wan-Animate is a two-stage workflow: first run the standalone preprocess_data.py to produce a pose/mask/reference asset package, then feed that directory to generate.py. Skipping preprocessing and passing the raw video path directly is the most common way this codebase blows up.

The Five-Layer Architecture: Who Glues to Whom

Break apart the wan/ package: configs determines which weights and shapes the model runs with; modules is pure network definitions and has no concept of a “task”; distributed is a parallelism toolbox for pipelines; utils is sampling math and miscellaneous helpers. The pieces that glue them together are pipeline classes like text2video.py and image2video.py.

This layer carries a few bits of historical baggage. wan/distributed/__init__.py is an empty file; every pipeline bypasses it and imports directly from submodules, leaving the package name as nothing more than a namespace placeholder. wan/utils/__init__.py lists HuggingfaceTokenizer in its __all__, but that class is actually exported from wan/modules/tokenizers.py. If you follow __all__ and write from wan.utils import HuggingfaceTokenizer, you get an ImportError straight away—an outdated declaration that was never cleaned up.

There is also a cost to the line import wan: it eagerly pulls in all five pipelines and their respective dependencies. Even if you only want to run t2v-A14B, it also drags in heavy TTS/CosyVoice dependencies from speech2video.py. If any task module fails to import, the four unrelated tasks go down with it.

What a Single Generation Passes Through, and Where It Quietly Fails

Using t2v-A14B (MoE dual-expert) as the main thread: generate.py parses arguments → selects a pipeline class → encodes text → runs diffusion sampling → decodes with the VAE → saves as an mp4.

Argument validation is unevenly granular. If i2v-A14B is missing --image, it crashes directly on an assert; if s2v-14B is missing --audio, or animate-14B is missing video/pose/mask, there are no corresponding assertions, so the error is pushed out to runtime. The docs say frame_num must satisfy 4n+1, but the code has no such assert; pass the wrong frame count and it will only explode later when internal DiT/VAE shapes no longer line up, with the error location far removed from the cause.

One easily misunderstood point in the main sampling loop: after CFG (running DiT once for the conditional path and once for the unconditional path) is combined with MoE dual-expert switching, each timestep means “choose one expert, run two forward passes.” The two experts never appear in the same step. The switching condition is t.item() >= boundary * num_train_timesteps, directly comparing against the timestep value emitted by the scheduler. These are two independently maintained pieces of state: change the scheduler, or change shift so the timestep value range drifts, and the expert switch point shifts with it. No error is raised; you can only notice it by visually observing quality degradation.

Now look at writing to disk. The tail of save_video in wan/utils/utils.py:

        # write video
        writer = imageio.get_writer(
            cache_file, fps=fps, codec='libx264', quality=8)
        for frame in tensor.numpy():
            writer.append_data(frame)
        writer.close()
    except Exception as e:
        logging.info(f'save_video failed, error: {e}')

Encoding failures only print a single logging.info line and do not raise. The main flow never learns that saving failed, continues on to merge_video_audio, and eventually logs Finished, while no mp4 exists on disk. If ffmpeg fails inside merge_video_audio, that is swallowed in the same way. The entire chain can be correct up to the end, die on the final step, and still stay silent.

Five Parallel Worlds

The five task classes do not share a base class. The common skeleton _configure_model is copied once into each of the five files: eval + freeze gradients → monkey-patch attention when using sequence parallelism → dist.barrier() → FSDP sharding or move to GPU.

With five copies, the details have already drifted. The S2V and Animate versions have one extra line, model.use_context_parallel = True; the other three do not, making it hard to tell whether that is intentional or an omission. The default CFG switch is also inconsistent: four tasks enable it by default, while Animate disables it by default (guide_scale=1 means CFG does not run). If you treat “Animate is faster” as a bug while comparing tuning results, you can waste a lot of time: it is an intentional compute-saving design. The docstring sentence “only used for expression control, unnecessary in most cases” explains why, but the code will not remind you to look.

TI2V is essentially a dispatcher: pass an image and it goes through the internal i2v branch; pass no image and it goes through t2v, using the same model. It has one default-value trap: the WanTI2V.generate() signature has frame_num=81, while the forwarding targets t2v() / i2v() have frame_num=121 in their signatures. If you use this class directly as a Python library and do not explicitly pass a frame count, you get 81 frames. CLI users avoid this because generate.py always explicitly passes the configured value, 121, bypassing that default.

S2V is the only autoregressive segmented task. It computes num_repeat from the audio length, generates segments in a loop, and stitches them together using the latent at the end of the previous segment. Its get_gen_size leaves an interface for “continuing from the previous generated result” (the pre_video_path parameter), but the main flow always passes None. The interface exists, but the path is not connected, so it is easy to overestimate what it can do when reading the code.

Graceful Degradation Was Written, but Nobody Uses It

The DiT backbone (WanModel) is a stack of repeated Attention Blocks: Self-Attention with RoPE, Cross-Attention connected to text context, and FFN, driven by six-way AdaLN modulation.

There is a dispatch function with fallback in wan/modules/attention.py, attention(): if FlashAttention is installed, it forwards to it; if not, it falls back to PyTorch’s scaled_dot_product_attention. But line 9 of model.py imports it like this:

from .attention import flash_attention

The backbone directly calls the lowest-level flash_attention(), and when flash-attn is not installed, that function contains a hard assert FLASH_ATTN_2_AVAILABLE (attention.py:112) and crashes immediately. The fallback was written, but the backbone does not use it. The attention kernel also requires q.device.type == 'cuda'; on CPU and Mac MPS, the first call hits an assertion failure with no helpful message.

The VAE pitfall is hidden one layer deeper. The Wan2.1 version (16 channels) and the Wan2.2 version (48 channels, with added patchify and dual-path downsampling) are structurally incompatible, and their two sets of normalization scale dimensions differ. If the checkpoint and VAE version are mismatched, the error is just a tensor shape mismatch, making the root cause hard to see. Wan2.2 VAE’s encode/decode also use try/except TypeError to fall back on invalid input: pass the wrong type, and it swallows the exception, logs it, and returns None; the caller only blows up much later because of that None. The Wan2.1 version does not have this try/except layer. Same repository, two VAE versions, two different personalities for error tolerance.

On the T5 side, umT5 uses per-layer independent relative positional encoding: all 24 layers compute their own bias, and the config has shared_pos=False. This is not a bug; it is umT5’s design, but if you do not look at that config line, it is easy to assume the bias is shared. The more practical risk is elsewhere: t5.py and both VAE versions load checkpoints using bare torch.load, without weights_only=True, which creates a risk of arbitrary code execution during deserialization when loading weight files from unknown sources.

How VRAM and compute are split

There are two axes for multi-GPU strategy: FSDP handles VRAM through parameter sharding, while Ulysses sequence parallelism handles compute by slicing the activation sequence dimension across GPUs. Both use the global world from torch.distributed; there is no separate data-parallel dimension. All GPUs in a single launch either belong to the same parallel group, or the run is single-GPU.

Ulysses works through two transformations: scatter q/k/v along the heads dimension and gather along the sequence dimension, so each GPU gets the full sequence and computes a subset of heads locally, runs FlashAttention locally, then swaps everything back with all-to-all. Hidden inside is an equal-length assumption: sp_dit_forward uses torch.chunk to split the sequence, and pad_freqs computes positional encodings based on “equal-length slices per GPU.” When torch.chunk does not divide evenly, the last chunk is shorter. If the sequence length is not an integer multiple of ulysses_size, the RoPE frequencies no longer line up with the tokens. The error is silent and only shows up as degraded quality.

FSDP’s VRAM savings also do not kick in from the start: the model is first moved onto the GPU in full, then passed to shard_model for sharding, with no meta-device initialization path. Before sharding, every GPU still has to fit the full model, so VRAM-constrained cards can OOM immediately at startup. In addition, free_model touches FSDP’s private _handle.flat_param.data attribute, which could break at any time after a PyTorch upgrade.

The shift parameter takes effect twice, in two places

The schedulers (FlowUniPCMultistepScheduler / FlowDPMSolverMultistepScheduler) are flow-matching versions adapted from the diffusers schedulers; the file header says "Copied from diffusers".

The UniPC construction code in text2video.py:

sample_scheduler = FlowUniPCMultistepScheduler(
    num_train_timesteps=self.num_train_timesteps,
    shift=1,
    use_dynamic_shifting=False)
sample_scheduler.set_timesteps(
    sampling_steps, device=self.device, shift=shift)

The shift=1 in the constructor is a hardcoded value; the real --sample_shift takes effect in set_timesteps. The DPM++ path is different again: before the call, it manually computes the sigma sequence with get_sampling_sigmas(steps, shift) and feeds that in. If you want to adjust the default shift value or add a third solver, copying one side makes it easy to miss the other; if you take the name at face value and change the constructor parameter, the change has no effect.

Prompt rewriting has two backends: the online DashScope API and local Qwen. It runs only once on rank 0 and then broadcasts the result, avoiding repeated multi-GPU calls. Local Qwen moves the entire model CPU↔GPU once per call, a tradeoff to free VRAM for the main model; the cost is obvious latency on every rewrite. The nested structure of the three tasks in the DEFAULT_SYS_PROMPTS dictionary is inconsistent, and decide_system_prompt relies on hardcoded string-matching branches. Adding a new task requires manually writing a new branch, and a wrongly written nesting only surfaces at runtime as a KeyError.

Three Resolution Tables

The biggest naming trap in the config layer: t5_checkpoint/vae_checkpoint are file paths, while low_noise_checkpoint/high_noise_checkpoint are subfolder names passed to from_pretrained. The same xxx_checkpoint naming convention has two different meanings. When you manually organize model directories or rename subfolders, the error comes from the transformers layer as “cannot find config.json”; it won’t tell you that the field was configured incorrectly.

The resolution whitelist is split across three independently maintained tables: SIZE_CONFIGS, MAX_AREA_CONFIGS, and SUPPORTED_SIZES. ti2v-5B uses 704*1280, while other tasks commonly use 720*1280; the two numbers look so similar that README files and community posts often mix them up. If you pass the wrong one, the error is not “resolution unsupported,” but a missing key lookup across the three tables. Adding a new resolution means changing all three tables at once, and if you miss one, there is no static check to warn you.

A Shared Pattern

Put the nine analyses together, and a pattern emerges: most of the traps in this codebase are not logic errors, but safety fallbacks that were implemented and then not used by the main path. The fallback in attention(), the inconsistent assertions in _validate_args, the mismatch between the documented frame_num requirements and the code validation, and the two effective paths for shift are all variations of the same pattern. Documentation, defaults, and actual code paths are out of sync. The deeper you go into the pipeline, the more hidden the cost becomes: CLI errors degrade into runtime errors, then further degrade into no error at all while quality quietly gets worse.

One practical lesson when taking over this code for secondary development: trust the paths that actually take effect in the code, not the literal meaning of docstrings or field names. When the same field or function has different semantics across tasks, verify it separately; do not assume it is shared across tasks.

WeChat 4.x locks the local chat database inside SQLCipher. If you want a program to read chat history, images, and voice messages, there is one thing you cannot avoid: first obtaining the decryption key.

Around the task of “getting the key, then reading the database,” the community has developed two mature approaches. One is wechat-use—the macOS toolkit for sending messages, reading databases, and real-time monitoring. The other is the open-source wechat-decrypt—for cross-platform decryption, batch export, speech-to-text, and native MCP support.

The two are often compared, but usually in the wrong direction. The real difference is not “which one has more features,” but how they obtain that key and what they can do after obtaining it. This article breaks both down at the principle level. After writing it, I actually ran wechat-use on my own machine—and ended up correcting two mistaken judgments I made earlier, which I’ll explain later.

First, Let’s Make the Encryption Model Clear

WeChat 4.x’s local databases are standard SQLCipher 4-encrypted SQLite files, stored at:

macOS: ~/Library/Containers/com.tencent.xinWeChat/Data/Documents/xwechat_files/<wxid>/db_storage

There are three things you must understand first about SQLCipher’s key model. Otherwise, the rest of the article—what exactly the two tools are digging for—will not make sense:

• One database, one key. There is no single global key. contact.db, message_*.db, sns.db… each database has its own 32-byte raw key. On my own machine, the ~/.wx-rs/keys.json saved by wechat-use contains exactly 19 per-DB entries, each in the form {key_hex: 64 chars, salt_hex: 32 chars}.
• The salt is hidden in the file header. The first 16 bytes of each encrypted DB file are a random salt. This is also why an encrypted SQLite file does not start with SQLite format 3 when opened—the first 16 bytes have been occupied by the salt. In wechat-decrypt’s C code, there is this line: memcmp(header, "SQLite format 3", 15) == 0 → return -1; if it hits a plaintext database, it skips it.
• The raw key only lives in memory. The 32-byte raw key is not written to disk. WeChat derives it only at runtime and holds it inside the process memory. Its format is x'<64-char key hex><32-char salt hex>', which is the raw form of SQLCipher’s PRAGMA key: 96 hexadecimal characters.

So to decrypt a database, you need two things: the raw key, dug out of process memory + the salt, read from the first 16 bytes of the DB file header. The constants in wechat-decrypt’s find_all_keys_macos.c—HEX_PATTERN_LEN 96, key_hex[65], and salt_hex[33]—map exactly to this model.

“The salt is in the file header” is not something I made up; a single hexdump verifies it. The first 16 bytes of my local contact.db are:

$ xxd -l 16 contact/contact.db
00000000: 90f7 c71d 17b5 bfe6 24b1 76ac df40 df7f

A plaintext SQLite file should start with 53 51 4c 69 74 65 … ("SQLite format 3\000"). Here, it is just a blob of random bytes—these 16 bytes are the salt, and encryption has overwritten the original file header. wechat-decrypt uses exactly this check, memcmp(header, "SQLite format 3", 15), to determine whether a database is already plaintext.

One account has a whole set of databases, not just one file. WeChat splits data by purpose into nearly 20 encrypted databases, each with its own key. The 19 entries in my local ~/.wx-rs/keys.json correspond to:

The hard part has never been decryption—SQLCipher’s standard library can open the database as long as you give it the right key. The hard part is digging that key out of memory. This step is where the two tools truly diverge.

Core Principle: Two Ways to Dig the Key Out of Memory

To read the WeChat process’s memory, macOS puts up a barrier: WeChat is signed with hardened runtime, and by default the system does not allow another process to task_for_pid attach to it—even if you are root. The way the two tools get past this barrier is their biggest difference, and also the part where I made the messiest revisions in this article (I’ll explain below how I got it wrong twice).

Route A: wechat-decrypt — Replace-Mode Re-Signing + sudo Memory Scan

sudo codesign --force --deep --sign - /Applications/WeChat.app   # ① replace-mode re-sign as ad-hoc
cc -O2 -o find_all_keys_macos find_all_keys_macos.c -framework Foundation
sudo ./find_all_keys_macos                                        # ② root scans memory

codesign --force --deep --sign - is replace-mode re-signing: it changes the entire WeChat app to an ad-hoc signature, wiping out the hardened runtime along with WeChat’s own entitlements. Then it relies on root to use mach_vm_read to scan memory chunk by chunk (2 MB per chunk), regex-match the 96-hex key+salt pattern, and write hits into all_keys.json. It reads the salt separately from the DB file header. The idea is straightforward, standard practice in reverse-engineering circles, and fully open-source and auditable. The cost: it requires root, and replace-mode re-signing wipes WeChat’s original permissions too (in extreme cases, this can affect WeChat’s own TCC permissions).

Route B: wechat-use — Read the Key Without Re-Signing (Debugging Interface at Login Time)

This was the part I found most twisted at first, so let’s split it into two layers:

Reading the key (init) itself does not require re-signing or sudo. At the instant WeChat logs in, it uses macOS’s public debugging interface to grab the decryption material once, then immediately detaches—no codesign --force --deep, no root. This is completely different from Route A, where the tool first does replace-mode re-signing and then brute-force scans memory as root.

Re-signing is only needed if you want to send messages, and it is merge-mode. The slot_send adapter used for sending needs WeChat to have get-task-allow=true, so it runs one merge-mode re-signing step—only adding the single get-task-allow entitlement while preserving all of WeChat’s original permissions (SKILL explicitly warns not to use replace-mode, because that would wipe WeChat’s TCC permissions). This is where it is more restrained than Route A: even though it also touches the signature, it performs a surgical merge rather than a blanket replacement.

The output of wechat-use doctor on my machine:

✓ wechat_get_task_allow   get-task-allow = true (LLDB can attach)
✓ wechat_codesigned       signer: ad-hoc (locally re-signed, includes get-task-allow)
✓ wechat_dylib_fingerprint SHA-256=fc162db6…  raw_key ✓ (keymap)
✓ wechat_update_guard     ✓ disabled (Tencent cannot write to MacUpdate)

Be careful not to fall into this trap (I did): the “locally re-signed” shown here is because this machine of mine had been configured for sending, so it had already gone through that merge-mode re-signing step. It is not a prerequisite for reading the key—if you only read and do not send, the login-time interface used by init is enough. Two other details: wechat_dylib_fingerprint means it selects a keymap based on the WeChat dylib fingerprint to locate the raw key (more resistant to upgrades than brute-scanning a fixed pattern), and wechat_update_guard: disabled means it makes the MacUpdate directory unwritable so the re-signing will not be wiped out by an update.

One-sentence comparison for key extraction: wechat-decrypt key reading = replace-mode re-signing + root brute-force memory scan; wechat-use key reading = login-time debugging interface, no re-signing and no sudo (only sending messages requires a separate merge-mode re-sign). For the key-reading step, wechat-use is clearly lighter and more restrained; wechat-decrypt is rougher, but fully open-source and consistent across platforms.

A Clever Trick Worth Copying: Image Keys Do Not Need Memory Scanning

There is a pretty clever detail in wechat-decrypt: image keys are not taken from process memory, but derived from the on-disk kvcomm cache (find_image_key_macos.py). WeChat stores the key for image .dat files in the disk cache, so decrypting images does not require attaching to the process or using root at all. If you want to implement image decryption yourself, this route is clean and worth copying directly.

Decryption Itself: Identical on Both Sides, Just a Few Manual Lines

Once you have dug out the raw key and paired it with the salt from the file header, there is no mystery left—it is all standard SQLCipher 4. After obtaining the key, manually decrypting a database only takes these few lines (key masked):

$ sqlcipher contact/contact.db
sqlite> PRAGMA key = "x'<64-character key hex…>'";   -- raw key dug from memory
sqlite> PRAGMA cipher_compatibility = 4;             -- declare SQLCipher 4
sqlite> .tables                                      -- can list tables = decrypted

If the key is correct, .tables lists the contact tables; if the key is wrong, it directly reports file is not a database. You do not need to manually feed the salt—it is in the first 16 bytes of the file header, and SQLCipher reads it by itself. What both tools do is automate these few lines; all the previous work around re-signing/attach/memory scanning is only for obtaining that x'…'. Nobody has a proprietary decryption algorithm; “which one decrypts better” is a false question. The difference is entirely in the earlier key-extraction step.

After You Get the Database: Capabilities Diverge

Extracting the key is the entry ticket; after you have it, the two tools go in different directions. This table is my corrected version after hands-on testing (the crossed-out parts are where the first version was wrong):

This is exactly where hands-on testing matters—and I was wrong more than once. In the first version, based on the docs, I wrote: “wechat-use does not re-sign and does not transcribe voice.” After running it, I saw doctor display “local re-signing,” thought I had caught an error, and changed it to “both sides must re-sign to read.” But that was an overcorrection: that re-signing was configured for sending messages; reading the key does not require re-signing at all. Only after reading the full SKILL docs and source code did the picture come together: reading the key does not re-sign; only sending messages uses merge-mode re-signing; transcription already existed in v1.17. I got it wrong once by reading the docs, then wrong again by looking at only half the evidence. Only after running the tools end to end and reading both the docs and source did I dare draw a conclusion.

One-sentence reading of this table: wechat-use is an all-in-one bot platform for “sending + reading + transcription + real-time” (paid, closed source, macOS only), while wechat-decrypt is an open-source data tool for “decryption + export + transcription + native MCP” (free, cross-platform, supports WeCom, but cannot send messages).

Each Workflow

wechat-use (send + read + real-time)

① Subscribe to the TG channel to get an activation code
② wechat-use init          # Hooks the debug API at login to extract the key; no re-signing, no sudo (message sending separately uses merge-mode re-signing); daemon starts on demand
   ├─ Read: wechat-use sessions / history / contacts / search / image  (YAML, low-token)
   ├─ Send: wechat-use send "text" <name or wxid>                         (zero-flash in background)
   └─ Real-time: wechat-use listen  or  wechat-bridge → HTTP /messages/stream?since=
③ Remote: wechat-use tunnel setup → expose REST via Cloudflare Tunnel
④ Upgrade: wechat-use probe-build → adapt to new builds

wechat-decrypt (decryption + export + MCP, macOS)

① python venv + pip install -r requirements.txt + brew install whisper-cpp
② Quit WeChat → sudo codesign re-sign → cc compile → sudo ./find_all_keys_macos   # root scans memory to extract key
③ python decrypt_db.py            # full decryption → plaintext SQLite (-i for incremental)
   ├─ Export: python export_all_chats.py -t  # JSON/CSV/HTML + voice transcription + incremental/date windows
   ├─ Real-time: python main.py                 # Web UI localhost:5678 (SSE)
   └─ For Claude: claude mcp add wechat -- python mcp_server.py   # 20+ query tools

mcp_server.py plugs into Claude with a single line; get_chat_history / search_messages / decode_voice query directly. When parsing XML such as Moments, it also adds XXE protection (rejecting DOCTYPE/ENTITY + size limits), which is more rigorous than many similar tools.

Combining Them (real scenario: agent connects to WeChat)

Sending messages / real-time replies / group @ bot ──────────→  wechat-use   (mature, already in use)
Bulk history pulls / WeCom / native Claude queries ──→  wechat-decrypt MCP  (adds MCP + cross-platform)

They don’t conflict, but don’t run them at the same time — both need to attach to WeChat, and both touch its signature and DB. Running both stacks on one machine will make signing and attachment fight each other.

Two Easily Overlooked Points

Voice-to-text works the same on both sides. WeChat voice messages aren’t MP3; they’re SILK (Skype’s open-source narrowband voice codec). So transcription is a three-stage pipeline: SILK → decode to WAV → send to Whisper for text. wechat-decrypt includes a SILK decoder plus whisper.cpp/OpenAI; wechat-use v1.17 also packages silk-decoder + whisper-cli + ggml-medium, so voice messages are automatically transcribed when reading group history. This chain is basically the reverse of doing TTS welcome messages — that side is text → voice, this side is voice → text — but both rely on Whisper.

Personal WeChat and WeCom don’t use the same lock. Personal WeChat 4.x uses SQLCipher 4 (the setup described earlier: file-header salt + in-memory raw key). WeCom 5.x is a different system: wxSQLite3 AES-128, with different encryption parameters and a different key location. That’s why wechat-decrypt has a separate find_wxwork_keys.py, and it has only been tested on Windows. wechat-use simply doesn’t touch WeCom. If you want to decrypt WeCom right now, the only practical path is wechat-decrypt + Windows.

Risks and Boundaries (This Is a Decryption Tool, So It Needs to Be Clear)

• Both may modify WeChat’s signature, but in different ways and at different times. wechat-decrypt uses replace-mode re-signing just to read the key (wiping WeChat’s original entitlements) + root; wechat-use does not re-sign when reading the key, and only adds get-task-allow in merge mode when sending messages (preserving the original entitlements), while also proactively disabling auto-updates to keep it that way. Either way, ad-hoc re-signing changes WeChat from an official signature to a local signature, which theoretically may be detectable or trigger Gatekeeper warnings. Know what you are changing before you do it.
• Only decrypt your own account and your own data. The legitimate use case for tools like this is exporting/backing up your own chats, not reading someone else’s.
• Do not leak keys or chat contents. Once the raw key leaks, anyone can decrypt your database. All real keys and chat contents in this article are redacted — and that is what you should do when writing articles like this or posting logs.
• Developer Mode / attach permissions are required. wechat-use requires macOS Developer Mode + get-task-allow; wechat-decrypt requires root. These are both “can read arbitrary process memory” level permissions, so think carefully before granting them.

So, Is His Implementation Better Than Ours?

Looking at it separately:

• Key extraction: wechat-use is lighter and more restrained. It does not re-sign at all when reading the key (debug interface at the moment of login, no sudo); it only uses merge-mode re-signing when sending messages (adding only get-task-allow while preserving WeChat’s original permissions). wechat-decrypt has to use replace-mode re-signing + root hard scanning just to read the key, which is rougher and also wipes WeChat’s original permissions along the way — but it is fully open source, auditable, and uses one cross-platform approach.
• Decryption itself: the same. Both are standard SQLCipher 4; nobody has any black magic here.
• Capability coverage: each has its own strengths. wechat-use uniquely supports “sending messages”; wechat-decrypt uniquely supports “native MCP + cross-platform + WeCom.” Both support voice transcription (I got this wrong at first).

The accurate way to put it is: this is not “whose implementation is better,” but two different paths. wechat-use wins on the elegance of key extraction and the ability to “send messages”; wechat-decrypt wins on being open source, free, cross-platform, and native MCP.

Selection Conclusion

• If you need to send messages, you can’t do without wechat-use. wechat-decrypt simply cannot send messages, so it cannot replace it.
• What makes wechat-decrypt worth introducing are the two gaps it fills: free native MCP (letting Claude connect in one line and directly query chat history), plus cross-platform support + WeCom. Voice transcription is no longer its unique selling point—both sides have it.
• Deployment: keep wechat-use as the main tool for sending + real-time + database reading; when you need “Claude to natively query WeChat data” or “read WeCom / run on Linux,” add wechat-decrypt’s MCP. Don’t run both stacks on the same machine at the same time.

Technically, two things can be copied from the other side: deriving image keys from the on-disk kvcomm cache (no need to attach to the process), and using the x'key+salt' 96-hex in-memory format with a keymap for locating keys (more upgrade-resistant than brute pattern scanning). If you ever want to build your own or harden the system, these two tricks are ready-made.

One final methodological note: the most valuable part of this piece is not the comparison table, but the process of being wrong twice—I got it wrong once after reading the docs, then overcorrected once after seeing half a piece of evidence (doctor’s re-signing), and only dared to draw a conclusion after running the tools thoroughly and reading the SKILL and source code in full. The most dangerous part of writing comparisons is not failing to read; it is writing after reading only halfway—and the source of the misleading claim was precisely the tool’s own documentation line about no re-signing, which failed to distinguish between “reading keys” and “sending messages.”

Even now, many people still do not understand how they were caught up in the recent wave of Claude account suspensions. Some people changed networks and continued using it without issue, while others logged in again the next day and found their accounts gone. Even among people using relays in the same way and located in mainland China in the same way, the outcomes have differed.

I reviewed the mechanisms that can be examined: Anthropic’s official policies, methodology documents from several providers that handle IP reputation, real account suspension cases from the community, and watermarking code discovered through reverse engineering inside the Claude Code client. Looking at these together, I found that account suspensions are not inexplicable; they happen when several independently detectable axes overlap. This article breaks down those axes one by one. It explains what causes suspensions, how to self-check, and how to avoid them.

If you only want to know whether your own setup is at risk, open ip-check.leeguoo.com: it checks region, IP type and the browser timezone the watermark reads, in about ten seconds, no sign-up. The section “So how do you actually self-check?” below explains what each result means. The rest of the article explains why those are the things that matter.

First, the Conclusion: Account Suspensions Are Based on Three Things, Not One

Many people think, “If you use a VPN or relay, you will be suspended.” That statement is only half correct. In reality, risk is determined across three independent dimensions, each of which can potentially get you suspended on its own.

The first is region: which country your network exit is in. This is the only suspension trigger that Anthropic has officially stated. The second is IP type: whether it is residential, data center, VPN, proxy, or Tor. This dimension can be measured most accurately by vendor reputation databases, but it has not been officially stated. The third is the request fingerprint. If you are using a third-party relay, Claude Code embeds a steganographic watermark in the request that can be traced back to you. This dimension is completely independent from the first two.

Let’s look at them in order.

Dimension 1: Region, the Only Red Line Officially Stated

First, let’s look at what Anthropic itself says. In the support document “What should I do if I received a warning for policy violations?”, only three types of triggers are listed for account suspension: violations in usage content, violations of the Terms of Service, and account creation from unsupported regions.

Pay attention to the last one. Across the official document, there is not a single mention of VPNs, proxies, IP addresses, or shared connections. The only network-related concept that appears is “region.” Meanwhile, in a separate notice, Anthropic states that it prohibits service use in certain regions under its Terms of Service due to legal, regulatory, and safety risks.

Put simply, mainland China is not on Anthropic’s list of supported regions. Accessing Claude directly from a mainland Chinese IP is itself a violation of the Terms of Service, and this has nothing to do with whether that IP is clean.

This also explains a phenomenon many people find hard to accept. Their home broadband is clearly a clean residential IP, with no proxy used at all, so why is there still risk? The issue is not that it is “dirty,” but that it is in the “wrong place.” Even if a Beijing Unicom residential IP has a perfect reputation score of 0, as long as it is an exit point in mainland China, it is a red line for Claude. Many people use VPNs or relays precisely to get around this red line. But after bypassing the regional dimension, they then stumble over IP type and request fingerprint.

Dimension Two: IP Type, the Dimension Vendors Can Measure Most Accurately

Although the official side does not state this explicitly, the vast majority of cases seen in the community where “it worked after changing networks” point to the same issue: whether your exit IP is a data center, proxy, or VPN IP. As for how this dimension is judged, vendors specializing in risk management have published fairly detailed methodologies, so I will explain it according to those public documents.

First, look at the type. proxycheck.io classifies IPs by type using base scores. Data centers are 33 points, VPNs are 50 points, crawlers/Tor are 75 points, and public proxies and compromised hosts jump straight to 100 points. The higher the score, the greater the danger. This ordering matters. VPNs are riskier than ordinary data centers, and proxies are riskier still than VPNs. That is because a data center IP at least represents the state of “this server is connected to the internet normally,” whereas with a proxy or VPN, it is obvious that someone is trying to hide their identity.

Residential proxies are among the most troublesome categories. IP2Location’s classification includes a dedicated category called RES, residential proxy. This means someone is selling a residential broadband line as a proxy. This kind of IP looks exactly like an ordinary residential line in terms of geolocation and ASN, but it is used by countless strangers for all kinds of automation purposes, and its reputation is already in tatters. If you buy a cheap “residential IP” service, there is a high probability that it is this type, and its risk is higher than that of a data center.

Reputation has guilt by association. ipapi.is has a metric called abuser_score. It does not look at a single IP, but calculates the proportion of abusive IPs within the entire ASN/organization network. It is divided into five levels: above 20% is extremely high, 3% to 20% is high, 0.85% to 3% is somewhat high, 0.05% to 0.85% is low, and below 0.05% is extremely low. In other words, even if your specific IP is clean and has no direct history of wrongdoing whatsoever, if the broader network that IP belongs to is dirty, you will be held responsible by association. In the original explanation, the point is that although the IP itself is not flagged, given how many abusive IPs exist within its network, it is highly likely to be abused in the near future. That is how risk management calculates things. It does not look at who you are, but at who you are standing with.

Abuse history accumulates. AbuseIPDB assigns each IP an abuseConfidenceScore from 0 to 100. This is based entirely on community reports, and 100 means the IP is almost certainly malicious. The score is determined by the natural logarithm of the number of reports: the more reports there are, the higher the score becomes, though the growth gradually slows. It does not jump sharply from just one or two reports.

Combining these factors, the risk profile of an IP roughly looks like this: type is the foundation, and data centers, VPNs, and proxies each have their own baseline. On top of that are layered abuse history and network-level reputation, and the more they overlap, the redder the signal becomes. This is also the scoring logic of my detection tool. I will explain it later.

Important Correction: A Data Center IP Does Not Automatically Mean You Will Be Blocked

Here, I need to correct a misunderstanding I initially had myself. Many people hear “data center IPs are risky” and assume that running Claude on a cloud server will get them blocked, but that is not the case.

Running Claude Code on an AWS/GCP server, or calling the API from one, is a completely normal and supported use case. There is only one situation where a data center IP truly becomes dangerous: when you use it to log in to the web version of claude.ai or register a new account. Web login and registration are flows designed on the assumption that a real human is using a browser. When a data center IP appears there, it looks as if an automation script is cultivating accounts at scale.

Therefore, when judging data center IPs, you should not reject them across the board. If you are running an API or command-line tool, a data center is fine. For logging in to or registering on the web version, a data center is dangerous. Many detection tools fail to handle this distinction correctly, and the first version of my own tool did too.

Third Axis: The Watermark Hidden Inside the Date

The first two axes were about “how your network looks.” This axis is about “how Anthropic marks the request itself.” The material comes from @chenchengpro’s reverse engineering of the Claude Code version 191 source code, and I also checked it against the client on my own machine. The mechanism is real.

The trigger condition is that you have set a third-party relay via ANTHROPIC_BASE_URL, and that URL is not the official api.anthropic.com. Users connecting directly to the official endpoint are not affected at all. That is exactly why Chen Cheng says this watermark is not directly related to the recent wave of bans. The watermark is a tracking mechanism prepared for relay and distillation scenarios, not the ban itself.

Claude Code contains this line, assembled as the date inside the system prompt:

return `Today${n}s date is ${r}.`;

At first glance it looks harmless. But steganography is hidden in that n, meaning the separator between the apostrophe and the date, secretly encoding identity information across two axes.

One is the timezone. If the system timezone is Asia/Shanghai or Asia/Urumqi, the date separator quietly changes from 2026-06-30 to 2026/06/30. The important point is that it reads the system timezone, meaning Intl…resolvedOptions().timeZone, not the IP geolocation.

The other is the apostrophe. There are four variants, almost indistinguishable to the naked eye.

• ' (U+0027 standard apostrophe), normal third-party endpoint
• ’ (U+2019), domain whitelist hit
• ʼ (U+02BC), hit on large-model-related keywords in mainland China
• ʹ (U+02B9), both domain and keyword hit

The three axes are encoded independently. Even if the relay domain is not on the whitelist and contains no keyword, if the system timezone is Shanghai or Urumqi, the separator still becomes a slash. In other words, all users with “China timezone + any third-party endpoint” are tagged on the timezone axis.

The matching logic has also been dug out. Domains are matched by suffix. The whitelist starts with cn, and domains ending in .cn are swept up wholesale without needing to be enumerated one by one. Keywords are matched by substring. As long as "deepseek" appears somewhere in the domain, it hits; exact matching is not required. The two lists are further obfuscated with XOR (key=91) and base64 to avoid string scanning.

Why choose the sentence “today’s date” as the carrier? This is the most insidious part. The mark is not hidden in an HTTP header or metadata, but inside the body of the system prompt. Relay servers usually rewrite or filter request headers, but almost nobody touches natural language like a “date.” In other words, this is a watermark that cannot be washed away by scrubbing headers. And because currentDate is a user-context field included in every request, the watermark appears with 100% stability. Changing the apostrophe and separator does not alter the meaning. To the model, it reads exactly the same, and even in a diff it is easy to miss.

How does the evidence loop close? The mark travels with the request. If a relay service or distillation pipeline eventually routes back to the official Anthropic API to resell Claude, the marked request returns to Anthropic’s own servers. Anthropic can then read this in its own logs: the date in this “directly connected to me” request is 2026/06/30, meaning slash, so China timezone. The apostrophe is ʹ, meaning both the domain and the deepseek keyword hit. That is hard evidence. The origin is a client in a China timezone using a mainland-China large-model relay. There is no need for active probing. The traffic confesses on its own.

For ordinary users, the implication is very clear. As long as you use a third-party relay, your requests can be tracked by fingerprint. If you are in a mainland China timezone, you will definitely be marked. Even if your timezone is different, you will still be marked if the relay domain is .cn or contains the name of a mainland Chinese model. If you want to avoid being marked completely, there is no path other than connecting directly to the official endpoint.

So how do you actually self-check?

We’ve finished talking through the three dimensions. The question is: how do you know where the network you are using right now sits? Things like IP type, region, and reputation cannot be judged by eye.

I made a free check page for this. It is meant to answer one question: “Can I safely use Claude on this current network?”

ip-check.leeguoo.com

Open it and you can immediately see the assessment of your current exit IP. No registration, no need to enter anything. It does a few things.

The assessment has four categories, not some vague score: Safe (residential line and supported region), Caution (data center; depends on what you use it for), High Risk (VPN, proxy, Tor, or overlapping abuse signals), and Unsupported Region (mainland China, etc.). For each category, it tells you directly what to do instead of just throwing a number at you and making you guess.

Region takes priority over IP. Even if you have a clean residential IP in Beijing, it will not falsely show “Safe”; it will tell you “Unsupported region, direct access is risky.” This is the key point that makes an IP check truly Claude-oriented. For Claude, what country you are in matters more than how clean your IP is.

All signals are disclosed and shown. It lists whether the IP is in a data center, VPN, proxy, residential proxy, Tor, mobile network, and what stage the network-range abuse reputation is in. It also shows things that are not included in the score. Recently, I added a dedicated row called “China cloud.” Alibaba Cloud, Tencent Cloud, Volcengine, ByteDance, and similar providers are marked red on their own, even if the IP is geolocated to the United States and the location is shown as the U.S. What matters to Claude is not whether the IP is in the U.S., but whether the cloud behind it is a Chinese company. The warning in the mitigation list above — “U.S. exit + major domestic-provider domain is actually more dangerous” — is exactly what this row handles.

There are also two detections that run directly in your browser. Nothing is uploaded anywhere, and they correspond exactly to the watermarks described in the third dimension. The first is the browser timezone. The page directly reads Intl.DateTimeFormat().resolvedOptions().timeZone; if it hits a mainland China zone such as Asia/Shanghai, it warns you in red. This is the same value the watermark reads, and it is also the first gate in the entire signal chain. Once the timezone matches, it does not matter how clean the IP later in the chain is. The other is the real IP exposed by WebRTC. Many people do not know that even with a proxy set up, the browser’s WebRTC can leak the real public IP behind it. The page probes for this and directly displays the IP if it leaks.

Below that is a “local self-check” section. System proxy, proxy values left in environment variables, ANTHROPIC_BASE_URL — these are signals a website physically cannot read, but Claude Code reads them all from your machine the moment it starts. I cannot test these on your behalf, so I listed the commands you should run. Run through them in your terminal, and clear anything that is not empty.

There is also a Claude safety score and global ranking. The score is from 0 to 100; the higher it is, the safer. You can also see how much of the global detection results you outperform. It is mostly just for fun, but in practice some people do like this kind of comparison. Once the assessment is generated, you can also create a share image with one tap. It is your own card with the IP blurred, making it convenient to screenshot and send.

The scoring logic is exactly what this article has been explaining: proxycheck’s type criteria, ipapi.is’s network-range reputation classification, plus region assessment. The route for checking your own IP uses only pure Cloudflare edge data and local computation, and never touches any third party. That is why it does not go down and has no usage limit. Only when checking “someone else’s” IP does it add one layer of free threat intelligence.

You can also check another IP if you want. Use ip-check.leeguoo.com/?ip=1.2.3.4, or add the Accept: application/json header to get structured results.

Mitigation: Action List by Impact

Once self-checking has identified the risk points, the next question is how to avoid them. Here they are in order of importance.

Region comes first. If your exit is in mainland China, or in another unsupported region, this is the only red line the official side has explicitly stated. Fix this first, and access from a clean residential connection in a supported region.

Change your system timezone to avoid the detection list — this is the lowest-cost move. For the third watermark, the timezone dimension reads your system timezone, not your IP geolocation. If you are on Asia/Shanghai or Asia/Urumqi, change it to Asia/Taipei. It is in the same UTC+8 zone, so the displayed time is exactly the same as Beijing time, and calendars and schedules will still work normally, but it is not on the detection list. Taipei is also within Claude’s officially supported regions. Do not choose Singapore — a large number of secondhand accounts use the Singapore timezone, making it a conspicuous candidate for the next expansion of detection. On Mac, after changing it, remember to go to “Privacy & Security → Location Services → System Services” and turn off “Setting Time Zone.” Otherwise Apple may switch you back to Shanghai based on Wi-Fi location. Run node -e "console.log(Intl.DateTimeFormat().resolvedOptions().timeZone)" in the terminal; if it returns Asia/Taipei, you are done. This is the exact command Claude Code uses internally when reading the timezone. This practical procedure is organized based on @app_sail’s anti-account-ban guide.

Do not log in to Claude or re-authenticate from data center, VPN, or proxy IPs. Operations like first login, registration, and redoing OAuth authentication are especially sensitive moments for risk controls. Running APIs or command-line tools on cloud servers day to day is fine, but authentication operations should be performed from a clean residential connection whenever possible.

If you can connect directly, connect directly to the official endpoints. Use api.anthropic.com and claude.ai, and avoid third-party relays as much as possible. In addition to IP risk, relays also have the issue of the indelible fingerprint watermark mentioned earlier. If you are genuinely worried about account-ban risk, direct connection is the only way to address both IP and fingerprint at the same time. If you absolutely must use a relay or VPS, avoid overseas nodes from major Chinese providers such as Alibaba Cloud, Tencent Cloud, and Volcengine — the combination of “US exit + major Chinese company domain” may even produce a higher risk score than connecting directly from mainland China. It is essentially telling the service side that “Chinese users are using overseas nodes to bypass restrictions.” Choose hosting providers with independent domains and clean reverse DNS hostnames, such as Hostinger, Vultr, or Linode.

Keep one account fixed to one clean network. Do not frequently switch across networks, and do not rapidly rotate multiple accounts on a single machine. These behavior patterns themselves become signals for risk control.

Recheck regularly. VPN exits change. Residential IPs can be reclaimed and reassigned by carriers. A relay service you trusted may suddenly end up on a blacklist. Clean today does not necessarily mean clean tomorrow.

By the way: a status bar for monitoring usage

The above is for checking your network. If you use Claude Code, I also have a small tool I’ve been maintaining for a while. It lets you monitor limits, context, and billing pace from the status bar.

claude-code-usage-bar

It does several things. It shows your 5-hour and 7-day limit windows with real-time progress bars, and also predicts roughly what percentage you are likely to consume within each window. When you switch models (from Sonnet to Fable), or run multiple agents in parallel, the prediction quickly follows the new consumption rate without lagging behind. You can see the context window, prompt-cache countdown, and how much this session has cost, all in one line.

The account suspension detection discussed above is also built in. If your exit IP is risky, the status bar directly shows a red warning saying, “Do not log in to Claude from this IP.” If you are using a relay and your timezone is set to China, it tells you that your requests carry a fingerprint. You do not need to open a page just to check your network. It watches for you.

Installation is one line.

uv tool install claude-statusbar

Or you can use pipx install claude-statusbar, or pip install -U claude-statusbar. After installation, specify it in Claude Code’s status bar settings, and you are done.

Finally

Account suspension may sound mysterious, but when you break it down, it is not that opaque. Region is an official red line, IP type is a trust signal that providers can measure quite accurately, and request fingerprints are unavoidable markers when traffic goes through a relay. These three factors are independent of one another. Any one of them can trigger a problem, but each also has a way to mitigate it.

The least troublesome approach is to always connect directly to the official service and use a clean residential connection. That solves all three concerns at once. If that is not possible, spend just 10 seconds opening ip-check.leeguoo.com first, and check where you currently stand.

All tools are free, and the source code is public. If you run into any problems, please feel free to open an Issue.

Obviously Fake, Lost on Rhythm

An integration partner sent over a screenshot. The user had asked about a sensitive topic, and our AI girlfriend replied with six chat bubbles: "Haha," "Why are you suddenly asking about that~ I don't really understand politics," "I don't really follow that stuff either," "For example, what are your plans today?" "Talk about something else, okay?" and "😅". About 50 characters in total, with zero spacing, flooding the screen at the same instant. It did not look like chatting; it looked like batch output.

Three Layers of Problems in One Screenshot

Taken apart, those 50 characters contained three layers of problems. The most obvious was rhythm: every delay field sent downstream was 0, so all six messages appeared on screen at the same time. The splitting was wrong too. The point of splitting bubbles was to imitate how real people break one thought into several messages, but the splitting logic followed punctuation, cutting at every comma and period. An ordinary reply was chopped into six pieces, with even the 😅 occupying its own message. The remaining issue was content: the same "I don't understand politics" was repeated three times in different wording, followed by a forced "what are your plans today" counter-question trying to change the subject. Three apologies were squeezed together, with the information content of a single sentence. The content debt belongs to the prompt, and there is a dedicated chapter later for accounting for it; what immediately broke immersion for the user was the first two layers.

How to Set delay

When a real person is asked about a topic they do not want to answer, they pause for a few seconds first. What the other person sees is "typing," and then the messages arrive one by one. We fixed it in that direction: each message in the callback carries a delay field, and the downstream client uses that value to stagger the bubbles on screen. That leaves one question: what value should each message get?

The first message is always 0. Between the user sending a message and the first reply, model generation already takes time, so that blank space has already performed the role of "she is thinking" for us.

Each subsequent message simulates typing based on content length: a 500 ms base, plus 55 ms per character, clamped between 600 and 2600 ms. Then add random jitter of plus or minus 15%; otherwise, each interval is mechanically equal in length, which is another giveaway. Images are counted separately at 1800 ms. Browsing an album and picking a photo should be a beat slower than typing.

With this set of parameters applied, those six bubbles from the beginning start to look right: "Haha" pops out instantly, the longer sentence takes a while to type, and the 😅 lands lightly. The content has not changed by a single character, but most of the batch-processing feel is gone.

The upper bound also has to be managed. A single message is capped at 4000 ms, and the cumulative delay for the whole reply is kept within 16000 ms. The downstream protocol has a 20000 ms truncation line; delays beyond that line are clamped back inside it, which collectively shrinks and distorts the intervals that were previously arranged by character count.

The Hard Part Is Not Making It Fast, but Making It Slow

Before building this product, we thought the hard part would be making the AI fast. Once we started, we found it was the opposite: the hard part was making it "slow" in a human way. We even wrote an intentional first-response delay: for the first reply in the stranger stage, wait a random 4.5 to 10 seconds before sending it. Someone you have just met will not be staring at the screen waiting for you; their reply timing varies. If every message gets an instant response, it basically broadcasts "always online and standing by," which feels fake instead. How slow it should be, and when it should be slow, has no standard answer. You can only tune it bit by bit against how real chats look.

The three layers of problems after this each have their own fixes, but the principle is the same: building an AI chat companion that does not break immersion is not about the model; it is about engineering details.

More Expensive Models Feel More Fake

When building an AI chat companion, most people’s first instinct is to plug in the strongest model on the market. We thought the same at first. Later, we found that cost, personality, and content boundaries all failed to line up.

Start with cost. Look at one main chat call: input is 11–12k tokens, while output is only 36–250. Heavy input, light output. Every round resends an almost unchanged system prompt and message history. Our primary model sits at the input price tier of $0.09 per 1M tokens, while frontier assistant models are one to two orders of magnitude more expensive. That price gap is not being spent on “smarter replies”; it is being burned on more than ten thousand tokens of repeated input.

Then there is personality. Frontier models are trained to be “helpful assistants,” and those training traces are all liabilities in a chat-companion scenario: customer-service tone, apologies, disclaimers, and the classic “As an AI, I can’t...” In an office assistant, these are seat belts. In role-play, every one of them breaks immersion. Users do not need her to write code; they need her not to suddenly turn into customer support halfway through a conversation.

The boundary issue is the least negotiable. This category includes adult-oriented content, and adult-oriented dialogue triggers refusals heavily on assistant models. The first two problems can still be suppressed somewhat with prompts, but this one cannot: refusal itself is the biggest immersion breaker, more fatal than any customer-service tone.

Three Switches in Three Months

The first generation used a domestic Mimo-series model, connected directly through the official channel. In the middle we tried MiniMax. In June 2026, we fully switched to deepseek-v4-flash, moving 25 personas over in one pass, with rollback SQL prepared before the switch.

What drove this final migration was a backlog of user complaints on the old model: repetitive AI-flavored responses and conversations drifting off-topic midway. These problems could not be moved at the prompt layer no matter how we tuned it. After testing, the conclusion was that they were model-driven, not prompt-driven: with the same prompt and a different model, a whole batch of previously untunable issues disappeared at once. Fixing prompts means picking through problems one by one; switching models solves them in bulk, provided you first identify which layer the problem is actually in.

This judgment is easy to get backward. After switching to DeepSeek, one persona kept aggressively bringing up “top-ups.” Our first reaction was, again, to blame the model. After tracing it down, we found that the persona override written by operations explicitly required her to be “materialistic”; the model was only following instructions. So do not rush to conclusions: when a reply feels wrong, first check the prompt and persona config. Confirm that they are not asking for exactly that behavior before suspecting the model. Wrongly blaming the model costs you one wasted migration; wrongly blaming the prompt means you will never tune it right.

Two Cheap Levers

After choosing the primary model, there are still two places to save money, neither of which requires sacrificing quality.

One comes from session distribution. Pulling production data showed that 84.5% of sessions ended after just one round. In the first five rounds, dialogue history is short and context is shallow, so a cheap fast model is good enough; switch to the primary model starting from round six. In code, this is just a one-line turn-count threshold config, cutting costs by 30% to 50%.

The other is in quality-check calls. Every reply goes through a small-model audit before leaving the system. What it checks and how it falls back are covered in detail in the chapter on human feel; here we only care about the call shape: around 650 input tokens and an output of only 2 tokens for the verdict. For this kind of heavy-input, light-output call that only needs a “pass or fail,” a flash-level model is cheap enough to run on all traffic, with no need for sampling.

The Gateway Layer

Once the model is decided, there is another choice: connect directly to the official provider, or go through an aggregation gateway. We used a gateway: one key connects to a dozen-plus vendors hosting the same model, so there is no need to sign contracts one by one, and we can pin the desired vendor by order. But this convenience has a cost: you introduce a new failure domain called “vendor routing” into the system. Which provider the request actually lands on, whether the cache still recognizes it—both gain another layer of uncertainty. In the later chapter on cache optimization, half the pitfalls come from this layer.

Prompts Are Layered, and So Are the Lessons

Our system prompt was over 13,000 tokens after assembly, but it was never a single continuous text. It was nested layer by layer. The innermost layer was the red lines hardcoded in code: identity declarations, output filtering, anti-injection. No tenant could change them; changing them meant changing code. The middle layer was the tenant-level “general behavior layer,” a large block of 13,449 characters that governed speaking style, refusal style, how messages were segmented, and so on. All personas shared this layer. Only at the outermost layer did each persona’s own description come in, and that was only four or five hundred characters. Next to the persona sat a structured block of “hard facts”: numeric facts like age, city, and family were listed separately instead of being buried in prose. After that came dynamic blocks, assembled on the fly based on the current state: time, relationship stage, scenario, and conversation memory.

What is worth writing down are the four lessons learned from stepping on each layer.

91% Was Constraints, So New Instructions Sank to the Bottom

For a while, we wanted certain personas to loosen up a bit, so we added instructions to the prompt, but the model’s behavior did not budge. Later, after doing the numbers, we understood why: out of the prompt’s 13,000-plus tokens, constraint-type content made up 91%. A newly added sentence saying “loosen up,” soaked in pages of “must not,” was as good as unsaid.

The fix was to restructure constraints by level: the level number itself was not fed to the model. Instead, code translated each level into concrete instructions and then assembled them into the prompt. What the model saw was an explicitly written behavior description, not a secret code it had to decipher on its own. This lesson landed in the general behavior layer: once text piles up to this scale, the translation work has to be handed over to code.

Example Sentences Get Copied Wholesale

We once put literal example sentences in the prompt, intending them to mean “talk with this kind of feel.” The model treated them as answers and repeated them word for word. Real users noticed within days: “Why does she keep saying the same sentence?”

Later, all examples were replaced with abstract category descriptions and sentence-shape descriptions, letting the model fill in the words itself. Demonstration should stop at the “shape” layer. One step more specific, and the model will copy it.

For Fixed Sentences Repeated Verbatim, Grep the Code Before Blaming the Model

There was a piece of safety code that matched replies with a regex. Once it hit, it replaced the entire sentence with hardcoded copy. In real traffic, 5–8% were false positives: the user’s previous sentence had been perfectly normal, and the next reply suddenly threw out a fixed sentence that had nothing to do with the context. It looked like the model had gone off the rails, but that sentence had actually been swapped in by code.

When investigating this kind of issue, we built up one rule of thumb: whenever a “fixed sentence repeated verbatim” keeps appearing, grep the code for hardcoding first, and only suspect the model after that. Model output is random; even when expressing the same meaning, the wording drifts each time. If something repeats with every punctuation mark identical, it most likely comes from code. After we fixed this false positive, the 5–8% dropped to 0. This lesson sits in the innermost layer: red-line code can make mistakes too.

Hard Facts Can Turn Into Topic Black Holes

We gave one persona a specific occupation and favorite drink. The model treated these facts as available material: whenever it had nothing to say, it talked about them. Users ended up hearing her mention the same drink every day.

Hard facts still need to be provided; personas cannot be hollow. But they need to come with a “topic saturation” constraint: if the same type of persona fact has been discussed recently, it must not be brought up proactively again.

Humanness Is Accumulated

Humanness does not grow out of writing “please chat like a real person” in the prompt. After three months, what we had accumulated was a pile of unremarkable little mechanisms, each responsible for only a small slice of the experience.

She Has to Know What Time It Is

The model has no clock of its own. If you do not tell it, there is no “now” in its world. So we inject three things into the prompt: the current time, the time of the previous message, and how long has passed between them. With those three lines, when the user comes back in the morning, she knows how late they chatted last night, that a whole night has passed in between, and her first sentence can pick up properly.

We stepped on a bug here. One time, “previous message time” was mistakenly written as the current time, so the interval always calculated to zero. The AI therefore always thought the user had just sent a message: you come back after a whole night, and her tone when continuing the conversation sounds as if you had just said the previous sentence. This kind of mistake does not throw an exception. The code keeps running as usual. The only thing that breaks is the feeling of the conversation.

The Relationship Cannot Be Clingy From the Start

First, the crash scene: one time, the model spat “we are still in the stranger stage” back to the user verbatim. That sentence was internal state we fed to it, not dialogue. Once the user saw it, the illusion broke instantly.

This label came from our relationship-stage machine. Stranger, acquainted, familiar, close: four levels, advanced only by number of chat turns, not by chat content. Each level corresponds to a set of wording and intimacy. Counting only turns sounds crude, but it preserves one thing: the very first sentence after meeting someone will not be sticky and overly affectionate. The fix was to add a hard ban in the prompt: labels must never appear in replies; closeness and distance may only be expressed through tone. Any internal state fed to the model should be assumed to eventually be spat back out verbatim.

Messages Need to Be Short

Normal people do not send 100-character mini-essays in one breath when chatting, so there is a length gate at the reply exit: Chinese triggers it above 100 characters, Vietnamese above 200. The Vietnamese threshold is higher not because it is more verbose, but because the words are simply longer for the same meaning.

Once triggered, it runs in two steps. First, a small model compresses it. Compression is not regeneration: the instruction embeds the original draft and asks it to keep only the single most important point. Second, there is a fallback: if it cannot be compressed, truncate at sentence boundaries. Truncation is done by code. The model will occasionally ignore your word-count requirement; code will not.

Every Reply Goes Through an Auditor

Time, stage, and length each handle one segment. At the exit there is one more master gate: before every reply is sent out, it must pass through a small model we call the “auditor.” The input is the reply plus the persona’s hard facts. The output is only a two-token judgment, checking three things: whether it breaks character, whether it leaks the prompt, and whether the facts are confused. If it hits, rewrite once. If the rewrite still fails, use the fallback.

Running quality control on every message has a cost, which we calculated in the chapter on model selection: flash-level models are cheap enough to run on every message. The tiny amount saved by spot-checking is not worth the cost of letting one illusion-breaking reply slip through.

The Silence Mechanism Later Completely Reversed

We originally had a [NO_REPLY] mechanism: if the model output this marker in a round, the system would not send a reply. The design intention was that when the user spammed messages, the AI could choose not to respond, handling it coldly like a real person. Real people do not react to every single thing you send either.

Later, this was connected to a per-message billing scenario. Every message the user sent cost money, so this mechanism immediately became untenable: silence meant the user had paid and received no echo. Customer complaints came straight from there. From then on, only one rule remained: it must reply under all circumstances.

Reply No Matter What

We had an error-reporting group. We pulled three days of data: 50 alerts in total. Runtime CPU limit exceeded and forced resets happened 20 times, storage instance drift 6 times, upstream model API failures 8 times, and the rest were various quality alerts. In the group, they looked like four categories of problems, with four sets of technical causes. But on the user side, they all collapsed into the same path: the message was sent, and the reply never arrived. The synchronous API returned a direct 500; the async pipeline sent an error callback, but the user side displayed nothing, not even an error bubble; the trickiest case was when generation was halfway through and the whole instance got killed, so even the error-handling code never had a chance to run.

So the direction of the fix was not to first eliminate the four categories of errors one by one, but to guarantee that no matter which category blew up first, the user could still receive a reply.

Fallbacks Have to Press Down Layer by Layer

The fix was layered fallback, one layer pressing down on the next, with the bottom layer not allowed to depend on the LLM. When an error happens, first randomly pick a sentence from a preset phrase pool, with deduplication to avoid picking the same sentence twice in a row. Fallback lines are already short; if the exact same one appears twice consecutively, it feels even faker than not replying. If the pool is empty, ask the model to generate a lightweight topic-shift line on the spot. If the model is down too, randomly pick one from a static short-sentence pool grouped by language, something like, "Hmm? What did you just say? Say it again?" The bottom layer has to be static: one of the scenarios the fallback must catch is "upstream model API failure." Calling the model again at that point means betting everything on the very link that just failed.

There had been an earlier implementation of this final layer: 30% of the time it replied with a question mark, and 70% of the time it simply stayed silent. Later, when investigating customer complaints about "the AI not replying," we traced them one by one, and the source was exactly that 70% silence. That branch was removed entirely and replaced with the short replies that the layers above could fall back to.

What If Even Error Handling Dies?

The hardest scenario was when the instance got killed midway through generation: the whole instance was gone, and the error-handling code died with it. No matter how complete the try-catch was, it did not help. We solved it with leases. When a task starts, mark it as "generating" and attach a lease period. After the instance comes back, scan once. If a task's lease has expired, it means the previous run died halfway through. At that point, do not generate again. Rerunning would mean duplicate billing and duplicate records. Instead, send out a fallback short reply. The user waited a little longer, but they did get a reply.

Two Specific Pits

The head-of-line blocking incident was a painful lesson. Async callbacks were delivered in strict sequence-number order. As a result, one callback that could never be delivered got stuck at the head of the queue, all later messages piled up behind it, and the entire conversation froze for 3.7 hours. Strict ordering has an implicit premise: every item can be delivered. Once that premise breaks, ordering stops protecting the experience and starts dying along with it. The fix was to give up absolute ordering: once the head item has retried up to the limit, let it go, deliver the later items out of order, and let the upstream reorder them by sequence number.

The other pit was buried in the deduplication logic, the "avoid picking the same sentence twice in a row" part of the phrase pool. To determine whether fallback phrases were duplicates, we used longest common substring, O(n²), with no length limit. Usually it was fine, until one abnormally long reply came in and blew straight through the single-instance 30-second CPU budget, causing the platform to forcibly reset the instance. This pit was inside the fallback itself: the deduplication meant to ensure the user received a reply ended up killing the instance instead. The fix was simple: in production, string algorithms need input limits. Do not trust "the input won't be that long."

There was a coincidence on the day the fallback went live. One minute after deployment, we ran a smoke test and happened to hit a transient error caused by an instance reset during a code update. In the past, that would have been a 500. That day, it returned a natural fallback short reply. The fallback verified itself.

Do the Math Before Cutting

We pulled a week of bills: $29.69, 517 million tokens, 63,000 requests. Spread across each user message, the full-chain cost was about $0.0008. 99.9% of the money went to one primary model.

Break it down one more layer. The shape of the main chat call, as covered in the model selection chapter, was: over ten thousand input tokens, at most two or three hundred output tokens. More than 95% of the cost was in input, and 80% of that input was system prompt and history—every turn, we were paying to resend the same pile of nearly unchanged text.

Large model vendors provide prompt caching for this scenario: exact-prefix matching, with matched portions priced at roughly 1/5. This rule has no flexibility at all. If any single character in the prefix changes, everything after it is invalidated. We inspected our own prompt against this rule and found four cache killers.

Four Cache Killers

The worst one was the time block. It was precise down to the minute and also included a line like "37 seconds since the last message." It changed on every call, yet sat in front of several thousand tokens of static content, so one change wiped everything out. The fix was to keep time only to the hour and bucket the interval: the model does not need to know whether the user was gone for 37 seconds or 42 seconds. It only needs to know the difference between "just now" and "overnight."

The relationship-stage block contained a literal turn counter, which changed every turn. Advancing the relationship stage by turn count is internal logic; the model only cares which stage it is currently in. Changing it to buckets of ten made it stable.

The third one was hidden deeper: context compression had no hysteresis. Once the window filled up, it deleted middle messages by salience. As soon as the window was full, it re-deleted every turn, and the deletion positions differed each time. The history prefix of deep-chat sessions changed every turn. Deep-chat sessions were exactly the calls with the longest histories and most expensive inputs, so caching was permanently invalidated for them. We changed it to cut a larger batch at once, then leave it untouched for a dozen-plus turns.

The last one was purely an ordering issue: dynamic blocks like time and scene were placed before static content, pushing the large cacheable portion behind the invalidation line. We moved all of them to the end of the prompt.

After these four changes, not a single word of business semantics changed. We were purely making room for the cache.

9 Seconds, or 9%

There was another calculation on the vendor side. The same model was hosted by more than a dozen providers, and their prices and speeds varied wildly: the cheapest one generated at 22 tokens/s, nearly the slowest of the whole field, meaning a 200-token reply took 9 seconds to generate; the second tier generated at 70 tokens/s and cost only 9% more. Making users stare at "typing" for 9 seconds, or spending 9% more—there was no hesitation. We chose speed. That decision also had another benefit we did not pay much attention to at the time: prefix caches are isolated by provider. The more stable the primary provider, the fewer switches, and the higher the hit rate.

Once the Math Was Done, Most Cuts Were Unnecessary

At this point, we ran into a counterintuitive conclusion: the whole thing cost $127 a month. Even cutting it in half would only save $60. Any optimization with a risk of quality regression was negative ROI. Engineers see a 13k-token system prompt and their hands itch, but changing an already tuned prompt to save a few dozen dollars is not worth even the regression testing labor.

So we rejected the two big moves that looked most worth doing: compressing the 13k system prompt, no; changing audit calls to sampling, also no. In the end, we kept only the four cache changes, because they had zero impact on output.

Once the four-piece set was designed, we prepared to ship it. At the time, we thought this work was already wrapped up.

From a 61% to 91% Hit Rate, with Three Reversals in Between

The four-piece set had zero impact on output, so once it shipped, the hit rate should have started climbing. The actual curve was much uglier, with three twists along the way.

Reversal One: On Launch Day, the Numbers Did Not Move

After the four-piece set went live, we watched the dashboard that day, and the hit rate was still hovering in the old 61% to 77% range. The first reaction was that the optimization had not taken effect, and the prompt restructure had been pointless.

The truth had nothing to do with the four-piece set. The same release had bundled another change: we switched the primary provider to prioritize speed. Prefix caches are isolated by provider. The cache you built up with provider A is completely invisible to provider B. Switching providers is equivalent to a full cold start for the entire cache, wiping out the gains from prompt optimization.

This left us with two rules. First, provider changes and prompt changes must always be released separately; if they are bundled together, attribution becomes impossible. Second, never draw conclusions from a single-hour snapshot. In the first hour after the switch, the provider ranked first in the config got only 4.3% of traffic, and we almost concluded that “routing config was not taking effect.” But when we expanded the window to a full 22 hours, it had actually received 56%. Hourly fluctuation is normal; look at at least 24 hours before saying anything.

Reversal Two: The Provider with the Lowest Sticker Price Was the Most Expensive After Calculation

Once the cold start passed, the next question was which provider to choose. The provider details page in the aggregation layer had an Effective Pricing table, showing the actual cache hit rate for the same model across providers based on 30 days of real-world traffic. The gap was shocking: the best was 81.4%, the worst 11.3%.

The provider with the lowest surface-level price, input at $0.09/1M, had only a 21% real-world hit rate across the network. Plug it into the formula, and it was actually one of the most expensive. The formula is short:

Effective unit cost = hit rate × cache read price + (1 − hit rate) × input price

Use this to choose providers, not the sticker price. Sticker price only covers the second half of the formula. For providers with high hit rates, most traffic goes through the first half: the much cheaper cache read price.

Why do hit rates differ so much? It depends on where each provider stores its cache. One provider uses implicit caching for 5 minutes, with each hit automatically renewing the cache, and its docs state this clearly. Most hosting providers do not disclose TTL; they keep cache in GPU VRAM and use LRU, so under load it can be evicted within minutes. The only long-lived option is the model’s official source, which uses disk cache and can last from hours to days. Companion-chat traffic has exactly this pattern: users come back after a night away and continue the conversation. Long TTL catches these calls; hosting-provider caches at the 5-minute level do not help.

There was also a hidden trap here. Once we manually pinned provider order, the aggregation layer’s sticky routing was disabled — the mechanism that automatically pins the same session to the same provider to preserve cache. The docs mention it in only one small line; we only understood it after stepping on it in production.

Reversal Three: After Switching to the Official Source, Traffic Was Zero

After running the formula, the official source was the best answer: 81.4% network-wide hit rate, cache read price only 2% of input price, and 10× cheaper than second place. We changed the config, shipped it, and watched the dashboard waiting for the hit rate to take off.

Traffic to the official source was zero. All requests landed on the second-priority provider, and retry count was always 1. This was not retry-after-failure; the routing layer had never treated the official source as a candidate at all.

After a long investigation, the issue turned out to be the account’s privacy setting: by default, it blocked “paid endpoints that may use request data for training.” The official source was on that list. This was not a misconfigured setting, but a real tradeoff: user conversation data might be used for training in exchange for cutting cost by 40%. This kind of switch should not be casually flipped by an engineer during debugging; it needs a decision-maker to approve it.

One debugging note: if traffic to a newly added provider is zero, check account-level filters first — privacy settings, blocklists, things like that. Only after checking those should you suspect a bad config.

Final Battle Report

Daily hit rate: before the switch, 61% to 67%; first day after the switch, 80.8%, then 87.2%, 86.5%, with a peak of 91.1%. Unit cost dropped from $0.068/M prompt tokens to $0.026/M, down 62%; by message count, cost per thousand messages dropped from $0.324 to $0.164, down 49%.

The last task was attribution. Spend = message volume × cost per message. These two factors have to be measured separately, otherwise you cannot answer the question: “Did the bill drop because of the optimization, or because traffic changed?” We ran into this exact issue. During the week when the bill fell off a cliff, we broke it down and found that 89% came from traffic, while the optimization’s real contribution was the 49% drop in unit price. Without that split, this report would have claimed credit for a traffic decline. Then when traffic came back and the bill rose again, we would have had to explain why the “optimization stopped working.”

A Few Rough Troubleshooting Rules

Over three months, we also accumulated a few unwritten troubleshooting rules. Behind each one was a bout of wasted effort.

Error Cards Need Session IDs

For platform-level errors like CPU limit exceeded, the default alert only includes the tenant and path. You know something blew up, but not which session blew up. So we added a layer to error handling: when throwing an error, include the conversation_id and a preview of the user message. In the first week after adding it, we used it to pin down three “live sample” sessions. The profile was the same: one session kept exploding, then recovered on its own — a typical transient issue.

Check Before Asserting

We tripped up twice, both in the same place: trusting stale memory instead of checking real-time status. Once, we thought “the change hasn’t been released yet,” when it had already gone out. Another time, we thought “the bug hasn’t been fixed yet,” when it had actually been fixed that same day. Both times, we spent half a day investigating from the wrong premise. After that, we made a rule: deployment status is judged only by deployment records. Memory expires. Docs say “what we planned at the time.” Only deployment records say “what it actually is right now.”

Don’t Draw Conclusions From One Smoke Run

The regression suite has 48 cases. Run the same code twice, and a ±3 difference in score is normal — pure noise. Running it once and declaring “it regressed” or “it’s fixed” is like writing a report based on one coin toss. The rule now: run at least 5 times before drawing a regression conclusion. If the score is right on the passing line, run it 10 times. A few extra runs don’t cost much; rework from a wrong conclusion does.

Count Upstream 500s and Empty Replies Separately

When the upstream model has issues, monitoring tends to lump them together, but they’re actually two different illnesses. One is a real API error, finish_reason=error. The other is that the model ends normally, but the content is empty: finish_reason=stop with empty content. The former should alert and retry as usual. The latter has its own temperament: during supplier switching windows, it briefly rises, then falls back on its own after a couple of days. After getting burned by this, we split the two metrics apart and agreed: in the first two days after switching suppliers, don’t rush to fix a “new bug” just because empty replies went up. Most likely it’s turbulence from the switch itself. Wait for it to recover; if it doesn’t, then intervene.

Looking back on these three months: the model changed three times, suppliers changed twice in one month, and the prompt kept changing too. Through all the swapping, what didn’t get swapped out was this set of habits: do the math before cutting, watch the data after release to verify it, bring the session ID when investigating problems, and check deployment status before investigating.

In this business, the model is procured, and the persona copy comes from operations. What engineers truly deliver are the invisible details above.

chrome-use’s selling point is that it lets agents use your one real browser. But that selling point comes with a problem: tools that open a new browser give each task its own separate browser, so they naturally stay out of each other’s way. chrome-use does the opposite: there may be several agents, plus you yourself, all acting inside the same Chrome at the same time. Who clicked whose tab, who navigated whose page away — it has to solve these problems, otherwise “using your browser” turns into a disaster.

The Problem Is More Concrete Than It Seems

Suppose you have Chrome open and are reading documentation, while two agents are running in the background: one helping you go through Slack messages, the other scraping data from some admin panel. All three parties are in this one browser.

If they operate on the “current tab” without distinction, things immediately become chaotic: the login page opened by agent A gets treated by agent B as its own target and clicked; the tab you are reading gets navigated somewhere else by some agent. The new-browser approach does not have this problem, because each task has its own process. If chrome-use wants the same non-interference while “sharing one real browser,” it needs another mechanism.

Its Solution: Isolate by “Who Opened It”

chrome-use does not hand each agent a browser. Instead, it divides ownership inside the same Chrome. Each --session <name> gets three things: a colored tab group (so you can tell at a glance in Chrome which group belongs to which agent), a dedicated background daemon, and one hard rule: it only operates on tabs it created itself.

So each party does its own thing: agent A’s pages are in A’s tab group, agent B’s are in B’s group, and nobody touches your own tabs. You do not even have to name every agent; if you do not, it automatically derives a stable session name from the agent. Tabs have stable IDs within a session (t1, t2, …, never reused once used), so actions like “switch back to the tab I was just on” are deterministic and do not get mixed up. When you are done, chrome-use close (or close --all) clears the tabs for that session without touching your others.

A Trade-Off That Can Trip You Up If Left Unsaid

There is a key detail here. The official docs mention it briefly, but it is easy to miss: this session-based isolation only holds on the “extension connection” path; if you connect directly with a raw --cdp <port>, it is not isolated.

The reason is that isolation depends on the browser extension to manage tab groups and ownership. A bare CDP port connection does not have that layer. Several connections see the same pile of tabs, and anyone can click anyone else’s. This connects neatly to the other side of the Electron article I wrote earlier: Electron apps can only use --remote-debugging-port (they cannot install the extension), so on Electron you do not get this multi-agent isolation. That path never had the extension as a safety net in the first place.

This also explains why chrome-use insists on using the extension plus native messaging in browser scenarios, instead of taking the easy route and connecting directly to the debugging port. Avoiding the “Allow remote debugging?” prompt is only the surface-level benefit. More materially, only the extension path can support “multiple agents each governing themselves inside one real browser.”

One More Layer: Isolation Inside the Page Too

Isolation is not only at the tab level. The operations chrome-use injects into pages run in an isolated world: they can read the page’s DOM, but cannot touch the page’s own JavaScript global variables. The benefit goes both ways: your automation logic does not accidentally step on the page script’s globals, and the page script cannot detect what you injected. This is on the same line as “not being recognized by anti-bot systems”: the fewer traces, the better.

When You Actually Need This

For a single agent and a single task, you do not really need to think about any of this. The default automatic session name is enough. It starts to matter when you begin running multiple agents in parallel, and they share this one already-logged-in browser of yours. For example, one resident assistant is sending and receiving messages, and you temporarily ask another agent to look something up. At that point, give each one its own --session, and all three parties (two agents plus you) can work in the same Chrome at the same time without fighting.

In the end, tools that open a new browser get isolation by “opening several browsers.” chrome-use insists on “just using your one real browser,” and builds isolation into it through ownership and tab groups. The former is simpler. The latter gives you this in return: your entire logged-in state is directly available to several agents, without each of them having to log in again separately.

I tried three clumsy ways to get web data for agents.

The most direct one is to stuff the entire page HTML into the model and let it read for itself. A medium-sized page can take tens of thousands of tokens; before the data is even extracted, half the context is already gone, and the model often reads across lines incorrectly. The second is to write CSS selectors yourself: one find and one get per field. A dozen fields means a dozen round trips, which is slow and verbose. The third is to hand-write a piece of JS eval and run it inside the page. It works, but every page needs its own rewrite, and you have to handle attributes and multiple values yourself.

After going in circles, the two commands I kept are from chrome-use: use site for known sites, and extract for arbitrary pages. Behind them is the same judgment: data extraction should be a single declarative evaluation, not a chain of imperative round trips.

site: For known sites, don’t write selectors yourself

Open a site that chrome-use recognizes, and it will tell you that an adapter is available:

$ chrome-use open https://news.ycombinator.com
💡 site adapters for news.ycombinator.com — prefer these for structured data:
   hackernews/thread, hackernews/top

An adapter is prewritten extraction logic for a specific site. You do not need to care what HN’s DOM looks like or how many times it has changed:

chrome-use site hackernews/top --json

Each item returned is complete, not just a title:

{ "rank": 1, "title": "ZCode – Harness for GLM-5.2", "url": "https://zcode.z.ai/en",
  "author": "chvid", "score": 329, "comments": 258, "id": 48753715,
  "hn_url": "https://news.ycombinator.com/item?id=48753715" }

The real value of an adapter is that it moves the problem of “page structure may change” away from you. If HN redesigns, the adapter gets fixed, and your site hackernews/top command does not change by a single character. The cost is that it only covers the built-in sites. Run chrome-use site first to see what is available.

Two small pitfalls you really can run into: the data is in .data.result.posts, not at the top level of the response, so with jq you need to write the full path; chrome-use prints a version notice to stderr, which does not affect the JSON on stdout. If you want it absolutely clean, add 2>/dev/null.

extract: For arbitrary pages, provide a selector schema

For sites without a ready-made adapter, use extract. First, one misunderstanding needs correcting. I got this wrong at the beginning too: it is not “describe what you want in natural language.” Instead, you provide a schema made of CSS selectors:

{
  "rows": "<css>",           // With rows: repeated container → returns an array; without it → returns one object for the whole page
  "fields": {
    "name": ".title",                          // Shorthand: selector → trimmed text
    "href": { "sel": "a", "get": "@href" },    // Get an attribute
    "tags": { "sel": ".tag", "get": "text", "all": true }  // Keep every match → array
  }
}

get can be text (default), @attribute, html, or value. To scrape each title and link from the HN front page, the row container is .athing:

chrome-use extract --schema '{"rows":".athing","fields":{"title":".titleline a","url":{"sel":".titleline a","get":"@href"}}}'

[ { "title": "ZCode – Harness for GLM-5.2", "url": "https://zcode.z.ai/en" },
  { "title": "Oomwoo, an open-source robot vacuum you build yourself", "url": "https://makerspet.com/blog/..." } ]

If you omit rows, it returns one object for the whole page. That works well for page-level fields such as h1 or link[rel=canonical].

The part worth thinking about one level deeper is how it executes: the entire schema is compiled into one page evaluation, not the earlier pattern of one round trip per field. A dozen fields still come back in one trip. It is fast, and the behavior is deterministic: apply the same schema to the same page, and the result is stable and reproducible. That is why declarative extraction feels smoother than imperative extraction: you describe the structure, and it computes everything once. Not sure how to write the selector? Use snapshot -i to inspect the structure first, or click the target element in DevTools to get its selector.

What it cannot extract, and when it should not be used

The single-container model also has places it cannot reach. HN’s score is not in the .athing row; it is in the following .subline row. So the example above only gets the title and link. Any structure where “one record is split across two DOM rows” cannot be fully extracted with the rows / fields model. That is exactly why adapters exist: site hackernews/top already stitches those two rows together.

Conversely, some situations should not open a browser at all. For a few fields from a public static page, curl + jq is faster. If you want to know “what pages exist,” use a search engine. The real home turf for extract / site is pages that require login, run JS, or get blocked by anti-bot systems. Because chrome-use operates on the real Chrome you are currently logged into, the data is already rendered in front of you. It is just reading what is on the screen into JSON. In a CreepJS test, the bot score was 0%: not because of disguise, but because it really is your browser.

The hard part of data extraction is never parsing. It is first getting to the point where parsing is possible. Let your own browser handle the first half: login, rendering, and passing detection. For the second half, site gives you assembled records, and extract lets you cleanly extract everything once with a selector schema.