DEV Community: c-wind

Finding ZMODEM in a Terminal Byte Stream: How `rz`/`sz` Works Inside an SSH Client

c-wind — Sat, 31 Oct 2026 14:03:32 +0000

Originally published on MistLab. Part 5 of Building an SSH Terminal.

Part 5 of "Building an SSH Terminal". Code excerpts are from MistTerm v1.2.26, an open-source (AGPL) SSH terminal client by MistLab; they are trimmed, and source comments are translated from Chinese.

TL;DR

ZMODEM, implemented by the lrzsz tools rz and sz, is a file-transfer protocol that runs inside the terminal session itself, so it works over SSH without a second connection. The client has to find ZMODEM frame headers in ordinary output, then route the PTY stream to the protocol instead of the emulator. MistTerm only starts a transfer when it sees a valid header and the user has just run sz or rz.

On many production servers the quickest way to move a file is still sz file.tar.gz to download it, or rz to upload one. These are the lrzsz tools, implementing ZMODEM, a file-transfer protocol from 1986 designed for modems and serial lines. It works over SSH because it never needs a separate connection. It runs inside the terminal session, over the same PTY bytes as everything else.

That is also what makes it hard for a terminal. The client has to spot the start of a transfer inside a stream that is otherwise text for the screen. It then has to take the stream away from the emulator, run a protocol over it, and hand it back cleanly.

How does ZMODEM work? Just enough of the protocol

The protocol is described in Chuck Forsberg's ZMODEM specification. The parts that matter here:

Frames start with a header. Header bytes include ZPAD (*) and ZDLE (0x18, the same value as ASCII CAN), followed by a format byte. B is a hex header, encoded entirely in printable hex digits and ended by CR LF. A is binary with CRC-16, C is binary with CRC-32. A hex header looks like ** ZDLE B + two hex digits for the frame type + eight for flags + four for the CRC. Binary headers formally use a single ZPAD.
Frame types relevant to startup: ZRQINIT (type 0), sent by the sender to ask for a receiver; and ZRINIT (type 1), sent by the receiver to say it is ready and to state its capabilities.
ZDLE escaping. In binary data, ZDLE and certain control characters are escaped, so a frame survives links that interpret XON/XOFF.
Cancellation. Five or more consecutive CAN (0x18) bytes abort a session.

The roles follow from the command. sz on the server is the sender: it starts by printing rz\r (a hint to terminals that auto-start a receiver) and a hex ZRQINIT. The client must become the receiver and answer with ZRINIT. rz on the server is the receiver: it announces itself with ZRINIT (lrzsz also prints the text "rz waiting to receive."). The client must become the sender and start with a file header (ZFILE).

ZMODEM over a PTY: detection, then the protocol takes over the stream

How does a terminal detect rz/sz? Not mistaking text for a frame

A terminal that turns into a file receiver whenever **\x18B00 scrolls past would misfire on cat-ing a binary file, on log output, or on someone viewing this article over SSH. MistTerm combines three signals.

Scan a bounded tail, not just the latest read. A header can be split across reads (Part 2). Output from the session is appended to a detection tail, capped at the last 512 bytes, and the scanner runs over that window:

fn push_zmodem_detect_tail(&mut self, data: &[u8]) {
    const TAIL_CAP: usize = 512;
    self.zmodem_pty_detect_tail.extend_from_slice(data);
    if self.zmodem_pty_detect_tail.len() > TAIL_CAP {
        let drop = self.zmodem_pty_detect_tail.len() - TAIL_CAP;
        self.zmodem_pty_detect_tail.drain(..drop);
    }
}

Parse headers structurally. The scanners check the full prefix (ZPAD ZPAD ZDLE + format byte) and decode the type digits. They accept the variants found in the wild, such as a parity-stripped ZPAD (0x80), the single- and double-ZPAD binary forms, and an ASCII **B… form without ZDLE, which the code's comments attribute to the Windows build of lrzsz:

fn hex_scan_for_type(data: &[u8], want: u8) -> bool {
    // at least 6 bytes: `** ZDLE 'B' HH` (HH = frame type as a hex pair)
    for i in 0..data.len().saturating_sub(5) {
        if !is_zpad(data[i]) || !is_zpad(data[i + 1]) || data[i + 2] != zmodem::ZDLE {
            continue;
        }
        let hb = data[i + 3];
        if hb != zmodem::ZHEX && hb != b'B' && hb != b'b' { continue; }
        if let Some(t) = hex_pair(data[i + 4], data[i + 5]) {
            if t == want { return true; }
        }
    }
    false
}

Require intent. A header alone does not start a download. The client also tracks the command line the user submits. If its first word is sz/lsz (any path, .exe allowed), download detection is armed for 30 seconds. Uploads work the same way with rz/lrz, armed for 45 seconds:

pub fn is_sz_shell_command(line: &str) -> bool {
    let base = shell_word_basename(first_shell_word(line));
    let lower = base.to_ascii_lowercase();
    matches!(lower.as_str(), "sz" | "lsz" | "sz.exe" | "lsz.exe")
}

A download starts only when a ZRQINIT is seen and that window is open. echo sz and where rz do not arm it. An upload starts on any of: rz's own prompt text ("rz waiting to receive.", "Awaiting rz", "rz rz rz"), the Windows ASCII **B01 form, or a header seen while an upload is expected. The text check is deliberately specific to lrzsz's messages. The header check is gated by intent.

A naming note for readers of the source: some comments in this code path say that rz "sends ZRQINIT". By the specification, the receiver sends ZRINIT, and ZRQINIT comes from the sender. We list this in the review notes for the series.

Taking over the stream

Once a transfer starts, PTY output belongs to the protocol. Feeding ZMODEM frames to the terminal emulator would put binary garbage on screen and, worse, could be read as control sequences. The handover has a few constraints:

One reader. The shell pump thread owns the SSH channel (Part 2). A transfer must not open a second reader, which would race for bytes. Instead the pump forwards PTY output to the ZMODEM engine through a bypass:

Uploads (local sz → remote rz): ZMODEM frames on the PTY can only reach this point through the SSH pump thread; they must be bypassed to the transfer engine, and the Channel must not be read again from another thread.

Capture before the file dialog. For an upload, the remote rz is already talking when the user is still choosing a file. MistTerm starts capturing the handshake (begin_rz_handshake_capture) as soon as it is detected, before opening the file picker. Otherwise the receiver's ZRINIT is consumed by the emulator, and the sender waits for a header that already went by.
Bounded buffering. While waiting for the first header, or while frames are being reassembled, incoming bytes are buffered up to 512 KiB (RX_INCOMING_CAP). Past that the receive is aborted rather than growing without bound.

The protocol work itself (frame encoding, CRCs, ZDLE escaping, the ZRPOS/ZACK exchange) is done by the zmodem2 crate, vendored in the repository. For comparison testing, setting MISTTERM_ZMODEM_USE_EXTERNAL_SZ=1 makes MistTerm drive a local sz binary instead of the built-in sender.

Two safety details

The remote picks the file name, so treat it as untrusted. The file name arrives in the data subpacket that follows the sender's ZFILE header. A hostile or buggy server could send ../../.bashrc or an absolute path. MistTerm keeps only the final path component and never overwrites:

fn disambiguate_download_path(download_dir: &Path, raw_name: &[u8]) -> Result<PathBuf, String> {
    let base = String::from_utf8_lossy(raw_name);
    let only = Path::new(base.trim())
        .file_name()
        .and_then(|s| s.to_str())
        .filter(|s| !s.is_empty())
        .unwrap_or("zmodem.bin");
    let mut file_path = download_dir.join(only);
    let mut counter = 1u32;
    while file_path.exists() {
        // name_1.ext, name_2.ext, …
    }
    Ok(file_path)
}

Path::file_name returns None for names ending in .., which then fall back to zmodem.bin.

Cancel must jump the queue. When the user cancels an upload, the session queue may still hold file data. A single Ctrl+C is not guaranteed to stop rz/sz, because while they run the PTY is in raw mode. The cancel goes through the priority queue described in Part 2:

// In ZMODEM protocol state a single Ctrl+C may not be recognized by the remote
// rz/sz; send the CAN sequence at high priority, then ETX.
let cancel = vec![
    0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, 0x18, // CAN x8
    0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, 0x08, // erase noise
    0x03, // ETX
];
handle.send_priority_interrupt(cancel)

Eight CANs exceed the spec's five-CAN abort threshold. The backspaces mirror lrzsz's own cancel string (CANs followed by backspaces), erasing the CANs if the line ends up back at a shell. The final ETX covers the case where the remote program has already exited.

Summary

ZMODEM runs inside the terminal stream. The client has to find frame headers within ordinary output.
Detection needs a bounded scan window, a real header parse, and proof the user just submitted sz/rz.
During a transfer, the PTY stream must bypass the emulator through the single channel owner, with bounded buffers.
Treat the sender's file name as untrusted, and send cancellation ahead of queued data.

FAQ

How does rz/sz work over SSH?

ZMODEM runs inside the terminal session over the same PTY bytes as normal output, so no separate connection is needed. The terminal must detect the start of a transfer and take the stream over.

What do ZRQINIT and ZRINIT mean?

ZRQINIT (frame type 0) is sent by the sender to ask for a receiver. ZRINIT (frame type 1) is sent by the receiver to say it is ready and to state its capabilities.

Why doesn't MistTerm start a download whenever it sees a ZMODEM header?

Headers can appear in ordinary output, such as a binary file printed with cat. MistTerm also requires that the user has just submitted an sz or rz command.

How do you cancel a ZMODEM transfer?

Five or more consecutive CAN (0x18) bytes abort a session. MistTerm sends eight, ahead of any queued file data.
Related in this series

← Part 4 · Series index · Part 6 →

Why Non-ASCII Input Breaks over SSH: Locales, SendEnv and the `env` Request

c-wind — Thu, 29 Oct 2026 14:41:19 +0000

Originally published on MistLab. Part 4 of Building an SSH Terminal.

Part 4 of "Building an SSH Terminal". Code excerpts are from MistTerm v1.2.26, an open-source (AGPL) SSH terminal client by MistLab; they are trimmed

TL;DR

Garbled non-ASCII input over SSH usually means the remote program runs in the POSIX/C locale, so line editors such as readline treat each byte as a character. The locale reaches the server through the RFC 4254 env request, which sshd accepts only for variables listed in AcceptEnv. Upstream OpenSSH sends no variables by default; Debian-family packages add SendEnv LANG LC_* and the matching AcceptEnv. The fix is to forward a valid UTF-8 locale and to allow it on the server.

A common SSH puzzle goes like this. Output in Chinese, Japanese or with accented letters displays fine. But when you type such characters at a bash or mysql prompt, the line fills with garbage, or Backspace deletes half a character. The terminal is not at fault. The server-side program is running in the wrong locale, and the fix is a little-known part of the SSH protocol.

Why does non-ASCII input break over SSH? Bytes are only text inside a locale

Part 1 established that the terminal and the remote program exchange bytes. Turning bytes into characters is the program's job, and C programs do it through the C library's locale. After setlocale(LC_ALL, ""), the LC_CTYPE category, chosen from the environment variables LC_ALL, LC_CTYPE and LANG in that order of precedence, decides the character encoding. Under en_US.UTF-8, the three bytes E4 B8 AD are one character, 中, occupying two columns. Under the C/POSIX locale they are three unrelated bytes.

Output usually survives a wrong locale because most programs pass bytes through unchanged, and the terminal decodes UTF-8 itself. Line editors do not. Readline (bash) and libedit (mysql, sqlite3 on some systems) have to know where each character starts in order to echo, move the cursor, redraw and delete. In the C locale, they treat every byte as a separate character. Depending on configuration (input-meta, output-meta), the bytes are echoed as octal escapes or treated as Meta-modified keys, and Backspace removes one byte of a multi-byte character. That is the garbage you see.

Where the remote locale comes from

An SSH session's environment is assembled on the server. sshd sets a few variables itself, PAM modules may add more (on Debian and Ubuntu, pam_env in /etc/pam.d/sshd reads /etc/default/locale), and the login shell's profile scripts can set more. Minimal images, containers and some distributions end up with no LANG at all, which means the POSIX locale.

The client can also contribute, through the env channel request in RFC 4254 §6.4:

byte      SSH_MSG_CHANNEL_REQUEST
uint32    recipient channel
string    "env"
boolean   want reply
string    variable name
string    variable value

The RFC warns that passing arbitrary variables is a security risk and that servers should restrict them. OpenSSH's sshd accepts nothing by default. Variables have to be listed with AcceptEnv in sshd_config.

Locale forwarding: env requests are sent between pty-req and shell

Does ssh send your locale by default?

It is widely believed that the ssh client "sends your locale". Strictly, that is a distribution default, not an OpenSSH default. The upstream ssh_config(5) manual says of SendEnv: "The default is not to send any environment variables." The one exception is TERM, which is not sent through env at all; it travels inside pty-req.

What most users see comes from distribution packaging. Debian's OpenSSH package, and therefore Ubuntu's, ships /etc/ssh/ssh_config with

SendEnv LANG LC_* COLORTERM NO_COLOR

and /etc/ssh/sshd_config with the matching

AcceptEnv LANG LC_* COLORTERM NO_COLOR

Other distributions ship similar pairs. So between two stock Debian-family machines the locale does travel, and that is the behavior a third-party client should reproduce. A client that sends nothing will behave worse than ssh on exactly the servers where AcceptEnv is configured.

(MistTerm's own source comment describes this as "the OpenSSH default". That is imprecise in the way described above, and we list it in the review notes for this series.)

Forwarding the locale: the details

MistTerm sends env requests after pty-req and before shell. Order matters, because once the shell has started, its environment is fixed. Which variables to send, and what to do when there are none, is decided in remote_locale_env:

// src/ssh/client.rs
pub(crate) fn remote_locale_env(get: impl Fn(&str) -> Option<String>) -> Vec<(String, String)> {
    const NAMES: &[&str] = &[
        "LANG", "LC_ALL", "LC_CTYPE", "LC_COLLATE",
        "LC_MESSAGES", "LC_MONETARY", "LC_NUMERIC", "LC_TIME",
    ];
    fn plausible(v: &str) -> bool {
        // macOS Terminal often sets `LC_CTYPE=UTF-8`. That is a macOS-only form that
        // glibc does not accept; sending it makes the remote setlocale fail as a
        // whole and fall back to POSIX, so treat it as unset.
        let bare_codeset = v.eq_ignore_ascii_case("UTF-8") || v.eq_ignore_ascii_case("UTF8");
        !bare_codeset
            && !v.is_empty()
            && v.len() <= 64
            && v.chars()
                .all(|c| c.is_ascii_alphanumeric() || matches!(c, '.' | '_' | '-' | '@'))
    }
    let mut out: Vec<(String, String)> = NAMES.iter().filter_map(|k| {
        let v = get(k)?;
        let v = v.trim();
        plausible(v).then(|| (k.to_string(), v.to_string()))
    }).collect();
    let has_ctype_source = out.iter()
        .any(|(k, _)| k == "LANG" || k == "LC_ALL" || k == "LC_CTYPE");
    if !has_ctype_source {
        out.insert(0, ("LANG".to_string(), DEFAULT_REMOTE_LANG.to_string())); // "en_US.UTF-8"
    }
    out
}

Three cases drive the design:

Values that only make sense locally. macOS commonly exports LC_CTYPE=UTF-8, a bare codeset that the macOS C library accepts. glibc does not. With setlocale(LC_ALL, ""), one invalid category makes the whole call fail and the program stays in the C locale. Forwarding the variable would cause exactly the problem it was meant to fix, so bare UTF-8/UTF8 values are dropped.
Values that are not locale names. Variables are user-controlled, so only short values made of the characters that appear in locale names ([A-Za-z0-9._@-], 64 bytes at most) are sent.
No locale at all. A Windows desktop normally has no LANG, and neither does a GUI app launched without a login environment. If none of LANG, LC_ALL or LC_CTYPE survives, the client sends LANG=en_US.UTF-8, so that an otherwise unconfigured server still gets a UTF-8 LC_CTYPE.

Each request is sent independently, and a rejection is not an error:

for (name, value) in remote_locale_env(|k| std::env::var(k).ok()) {
    match channel.setenv(&name, &value) {
        Ok(()) => log::info!("SSH setenv {}={}", name, value),
        Err(e) => log::info!("SSH setenv {} rejected by server: {}", name, e),
    }
}

How to fix it: what the client can do, and what the server must allow

Forwarding only helps if the server cooperates in two ways:

AcceptEnv must allow the variable. Without it, sshd refuses the request and the session falls back to whatever the server's own configuration provides. That is the same result as with OpenSSH's ssh. The fix is on the server: add AcceptEnv LANG LC_*, or set a system default locale.
The locale must exist on the server. If en_US.UTF-8 is not generated on that machine, setlocale fails again. On glibc systems, locale -a lists what is installed, and many minimal systems also provide C.UTF-8.

When you debug a server, locale inside the session shows what the remote programs actually got.

For MistTerm's local shell tab (a PTY on your own machine, no SSH involved), the same concern is handled by setting the environment directly: TERM=xterm-256color and COLORTERM=truecolor always, and on macOS LANG=en_US.UTF-8 when neither LANG nor LC_ALL is set. An app launched from Finder or the Dock does not inherit a login environment, which is the case this covers.

Summary

Line editors need a correct LC_CTYPE to handle multi-byte input. In the POSIX locale they work byte by byte.
The locale reaches the server through RFC 4254 env requests, sent before shell and filtered by sshd's AcceptEnv.
Upstream OpenSSH sends no variables by default. Debian-family packaging adds SendEnv LANG LC_* (plus COLORTERM, NO_COLOR) and the matching AcceptEnv.
Filter values the remote libc will reject, such as macOS's bare UTF-8, and supply a UTF-8 default when the local machine has none.

FAQ

Why does typed Chinese or accented text turn into garbage over SSH?

The remote program is usually running in the POSIX/C locale, so line editors such as readline handle input byte by byte instead of character by character.

Does ssh send LANG to the server by default?

Upstream OpenSSH sends no environment variables by default. Debian and Ubuntu packages configure SendEnv LANG LC_* in ssh_config and the matching AcceptEnv in sshd_config.

How do I fix garbled input on the server side?

Allow the variables with AcceptEnv LANG LC_* in sshd_config, or set a system default locale, and make sure the locale exists on the server (check with locale -a).

Why should a client not forward LC_CTYPE=UTF-8 from macOS?

glibc does not accept a bare codeset as a locale name. One invalid category makes setlocale(LC_ALL, "") fail, and the program stays in the C locale.
Related in this series

← Part 3 · Series index · Part 5 →

What Happens When You Paste into a Terminal: Bracketed Paste, the 4095-Byte Line, and UTF-8 Boundaries

c-wind — Wed, 28 Oct 2026 14:32:35 +0000

Originally published on MistLab. Part 3 of Building an SSH Terminal.

Part 3 of "Building an SSH Terminal". Code excerpts are from MistTerm v1.2.26, an open-source (AGPL) SSH terminal client by MistLab; they are trimmed

TL;DR

Bracketed paste is an xterm mode (DEC private mode 2004). Once a program enables it, the terminal wraps each paste in ESC[200~ … ESC[201~, so pasted newlines are inserted as text instead of acting as Enter. Readline turns it on by default from version 8.1. Separately, Linux's n_tty caps a canonical-mode line at 4095 bytes plus the newline. Large pastes should be sent in chunks that never split a UTF-8 character.

To a terminal, pasting is typing very fast. The clipboard text is written to the PTY exactly as if the user had typed it. Nothing in the byte stream separates "typed" from "pasted". That causes three separate problems. The shell cannot tell a pasted newline from a pressed Enter. The kernel limits how long one line can be. And splitting a large paste into chunks can cut a multi-byte character in half.

Problem 1: a pasted newline is an Enter. How does bracketed paste work?

Paste three lines into bash without any protocol and bash runs each line as soon as its newline arrives. That is inconvenient for a multi-line snippet. It is dangerous for text copied from a web page, which can contain content you never saw rendered, a newline included.

The fix is bracketed paste, an xterm extension documented in ctlseqs. A program that wants it enables DEC private mode 2004 (CSI ? 2004 h). From then on, the terminal wraps each paste:

ESC [ 2 0 0 ~   <pasted text>   ESC [ 2 0 1 ~

Bracketed paste: the program asks for mode 2004, the terminal wraps the paste

A program that sees the start marker treats everything up to the end marker as literal text. Readline inserts it into the line buffer without executing it, zsh's line editor (since zsh 5.1) does the same, and Vim, documented under :help xterm-bracketed-paste, inserts it without auto-indent.

In GNU Readline, the library behind bash's prompt, bracketed paste became on by default in readline 8.1. The NEWS entry reads "Bracketed paste mode is enabled by default", and the source shows the switch:

/* readline-8.1, rlprivate.h */
#define BRACKETED_PASTE_DEFAULT 1
/* readline-8.1, readline.c */
int _rl_enable_bracketed_paste = BRACKETED_PASTE_DEFAULT;

In readline 8.0 the same variable is initialized to 0. Bash 5.1 ships readline 8.1, so on current distributions an interactive bash asks for mode 2004 unless the user has set enable-bracketed-paste off in .inputrc. Readline also leaves the mode off for terminals it considers unsuitable (see terminal.c), such as TERM=dumb.

The terminal's side of the contract is small but has to be exact. MistTerm checks the mode bit the remote program set (Part 1), normalizes line endings, and wraps the payload:

// src/ui/terminal_paste.rs
/// Normalize newlines to `\n`; when bracketed paste is on, wrap in
/// `ESC[200~ … ESC[201~` and strip forged markers from the body
/// (so pasted content cannot "end the paste" early and be run as commands).
pub fn build_paste_payload(text: &str, bracketed: bool) -> Vec<u8> {
    let normalized = text.replace("\r\n", "\n").replace('\r', "\n");
    if !bracketed {
        return normalized.into_bytes();
    }
    let body = normalized.replace("\x1b[200~", "").replace("\x1b[201~", "");
    let mut out = Vec::with_capacity(body.len() + BP_START.len() + BP_END.len());
    out.extend_from_slice(BP_START);
    out.extend_from_slice(body.as_bytes());
    out.extend_from_slice(BP_END);
    out
}

The marker stripping is the security-relevant line. If clipboard text contains a literal ESC[201~, a naive terminal forwards it, the program believes the paste ended, and whatever follows is interpreted as typed input, newline and all. The unit test pins this behavior:

let p = build_paste_payload("ls\x1b[201~rm -rf x\n", true);
assert_eq!(p, b"\x1b[200~lsrm -rf x\n\x1b[201~");

Bracketed paste only helps if the receiving program enabled it. In a program that did not (a plain cat, a script reading with read, a REPL without support), a pasted newline is still an Enter. That is a property of the program, not something a terminal can change.

Problem 2: why are long pasted lines cut at 4095 bytes?

Programs that leave the TTY in canonical mode (POSIX termios ICANON) do not edit lines themselves. cat, the shell builtin read and Python's input() when stdin is not readline-enabled all fall in this group. The kernel's line discipline buffers input until a newline and implements Backspace, Ctrl-U and similar editing. That buffer is finite.

On Linux the buffer is defined in drivers/tty/n_tty.c:

#define N_TTY_BUF_SIZE 4096

n_tty_receive_buf_common handles the full-buffer case. In canonical mode, when the buffer is full and contains no complete line, the kernel keeps one slot free by re-using the last position. That lets a terminating newline still be accepted, while further characters are discarded. The effective result is that one canonical-mode line holds at most 4095 bytes plus its newline. Anything beyond is silently lost, and the reading program never knows.

Canonical-mode line buffer on Linux

Two details are easy to get wrong:

The MAX_CANON constant in Linux's <linux/limits.h> is 255. It is not the limit that n_tty enforces. POSIX only requires MAX_CANON to be at least 255.
The number is Linux-specific. macOS and the BSDs have a different TTY implementation, and macOS's <sys/syslimits.h> declares MAX_CANON as 1024. We have not measured macOS's actual behavior, so this series makes no claim about it beyond the header value.

A terminal cannot work around this. The limit is in the server's kernel, below sshd. What a terminal can do is warn. SSH gives the client no way to query the remote termios state, so MistTerm uses the best signal it has. If the remote program has not enabled bracketed paste and some line in the clipboard is longer than 4095 bytes, it shows a warning before sending:

let bracketed = self.terminal.is_bracketed_paste();
// In line mode (cat/read, …) a single line > 4095 bytes is truncated by the
// tty layer; the terminal can't avoid that, so tell the user.
if !bracketed && exceeds_canon_line_limit(&normalized) {
    self.pending_toast_error = Some(/* suggest SFTP upload or pasting in vim */);
}

This is a heuristic. A program in raw mode without bracketed paste would trigger a spurious warning. The text is still sent unchanged either way. Interactive shells and editors that enable bracketed paste run the TTY in non-canonical mode and read the bytes themselves, so they are not subject to this limit.

Problem 3: chunking without breaking characters

A paste of several megabytes cannot go to the session thread as one message (Part 2). MistTerm queues it and writes it in pieces of at most PASTE_CHUNK = 4096 bytes. Cutting a byte buffer at an arbitrary offset can split a UTF-8 sequence: a CJK character is three bytes and an emoji four. Most receivers reassemble the bytes correctly, but treating each write as a unit of text is common enough that it is worth not producing broken fragments. UTF-8 makes the boundary easy to find, because continuation bytes always match 10xxxxxx:

Backing up to a UTF-8 character boundary

/// Next chunk length: at most `max`, never splitting a UTF-8 multi-byte
/// character (unless a single character exceeds `max`).
pub fn next_chunk_len(buf: &[u8], max: usize) -> usize {
    if buf.len() <= max {
        return buf.len();
    }
    let mut n = max;
    // UTF-8 continuation bytes look like 10xxxxxx; back up to a character start.
    while n > 0 && (buf[n] & 0xC0) == 0x80 {
        n -= 1;
    }
    if n == 0 { max } else { n }
}

Note that the check is on buf[n], the first byte of the next chunk. If that byte is a continuation byte, the cut is inside a character, and the loop moves back to that character's lead byte.

The send loop is driven by the UI frame clock, not by a blocking loop. A chunk is removed from the pending buffer only after it was accepted by the session queue. If the queue is full, the same chunk is retried after 20 ms, up to 500 times (about ten seconds without progress). Otherwise the next chunk follows 5 ms later:

const GAP: Duration = Duration::from_millis(5);
const RETRY_GAP: Duration = Duration::from_millis(20);
const MAX_RETRIES: u32 = 500; // give up after ≈10 s without progress

If it finally gives up, the user is told how many bytes were not sent. A paste is never silently truncated by the terminal itself.

Summary

Pasted newlines are indistinguishable from Enter unless the program enables bracketed paste (mode 2004). Readline enables it by default from 8.1 onward.
A terminal implementing bracketed paste must strip ESC[200~/ESC[201~ from the pasted text, or the clipboard can break out of the paste.
Linux canonical-mode lines are capped at 4095 bytes plus newline by n_tty. MAX_CANON (255) is not that limit, and other kernels differ.
Chunk large writes on UTF-8 boundaries, and only drop a chunk after it has been accepted.

FAQ

How does bracketed paste work?

A program enables DEC private mode 2004 with CSI ? 2004 h. The terminal then wraps each paste in ESC[200~ and ESC[201~, and the program treats everything between them as literal text.

Is bracketed paste enabled by default in bash?

Readline enables it by default from 8.1, and bash 5.1 ships readline 8.1. Users can turn it off with enable-bracketed-paste off in .inputrc.

Why must a terminal strip ESC[201~ from pasted text?

If the clipboard contains the end marker, the rest of the paste would be treated as typed input, including its newlines.

Why are pasted lines longer than 4095 bytes cut off on Linux?

In canonical mode, Linux's n_tty line discipline holds at most 4095 bytes plus the newline. MAX_CANON (255) is not that limit. A terminal cannot work around it, but it can warn.
Related in this series

← Part 2 · Series index · Part 4 →

SSH Flow Control, and Why an SSH Client Must Keep Reading While It Writes

c-wind — Mon, 26 Oct 2026 15:32:34 +0000

Originally published on MistLab. Part 2 of Building an SSH Terminal.

Part 2 of "Building an SSH Terminal". Code excerpts are from MistTerm v1.2.26, an open-source (AGPL) SSH terminal client by MistLab; they are trimmed

TL;DR

SSH flow control gives each direction of each channel a window: the number of bytes the sender may still send before the receiver grants more with SSH_MSG_CHANNEL_WINDOW_ADJUST (RFC 4254 §5.2). When the send window is zero, the client must keep reading the server's output. Otherwise it and the remote program can wait on each other forever. Keep network I/O off the GUI thread, and bound every wait.

Part 1 treated the terminal as a pipe that hands bytes to a parser. Those bytes come from, and keystrokes go to, an SSH channel on a server PTY. Two properties of that path decide how an interactive client is built: the channel is flow-controlled, and the GUI thread must never wait on the network.

The data path

Data path between the UI, the session thread and the remote shell

When you type l, the UI encodes the key and hands the bytes to the session layer. The SSH library encrypts them into an SSH_MSG_CHANNEL_DATA message (RFC 4254 §5.2). On the server, sshd writes the payload into the master side of a pseudo-terminal. The line discipline in the kernel's tty layer processes it (echo, line editing in canonical mode, signals such as Ctrl+C → SIGINT), and the shell reads from the slave side. Output takes the same path back.

Setting up that path takes three channel requests, all defined in RFC 4254. pty-req (§6.2) carries the terminal type, the size in characters and pixels, and encoded terminal modes. Optional env requests (§6.4) follow; Part 4 covers them. Finally shell (§6.5) starts the user's login shell. Resizing later uses window-change (§6.7). Despite the name, "window" there means the terminal window. It has nothing to do with the flow-control window below.

How does the SSH channel window work?

SSH multiplexes many channels over one TCP connection. To stop one fast channel from swamping the connection or the peer's memory, RFC 4254 §5.2 gives each direction of each channel a window: the number of bytes the sender may still transmit. Sending data consumes window. The receiver adds window back with SSH_MSG_CHANNEL_WINDOW_ADJUST after it has consumed data. When the window reaches zero, the sender must stop until an adjust arrives.

SSH channel window: writes stall at zero until the peer adjusts

For a terminal, the interesting case is a large write: a big paste, or a file upload streamed through the terminal (Part 5). The client keeps writing until its send window is exhausted and then has to wait. What it does while waiting matters.

The server only re-opens the client's window after its side has consumed input, which means the remote program has to be reading. Interactive programs are often busy writing at the same time: a shell echoing the pasted text, or a progress display. If the client is not reading the server's output, the client never re-grants window in the other direction. The server stops sending, the remote program blocks on its own write(), stops reading its input, and the client's send window stays at zero. Neither side can make progress. A writer has to keep reading.

Avoiding the write deadlock: write with drain

MistTerm puts this rule in one function used for all large PTY writes, write_pty_with_drain in src/ssh/manager.rs. The session runs in non-blocking mode (sess.set_blocking(false)), so a write either makes progress or returns immediately. The core of the loop:

const MAX_NO_PROGRESS: usize = 6_000;   // × 150µs sleep ≈ 0.9 s without progress → error
const CHUNK_CEIL: usize = 256 * 1024;

while !rest.is_empty() {
    let len_before = rest.len();
    let win = channel.write_window().remaining as usize;
    if win == 0 {
        // no send window: read output instead, so the peer can make progress
        Self::pump_channel_reads(channel, read_buffer, message_tx, session_id, upload_bypass, None)?;
    } else {
        let chunk = rest.len().min(win).min(CHUNK_CEIL).max(1);
        match channel.write(&rest[..chunk]) {
            Ok(0) => { /* drain reads */ }
            Ok(n) => {
                rest = &rest[n..];
                if !rest.is_empty() { /* drain reads between chunks */ }
            }
            Err(e) if Self::is_retryable_write_error(&e) => { /* drain reads */ }
            Err(e) => return Err(e),
        }
    }
    if rest.len() < len_before {
        no_progress = 0;
    } else {
        no_progress += 1;
        if no_progress > MAX_NO_PROGRESS { return Err(/* TimedOut */); }
        thread::sleep(Duration::from_micros(150));
    }
}

(The real code maps a closed channel to ConnectionAborted at each drain point and handles a library return value larger than the request. These are elided here.)

Three design choices are visible:

Never write more than the window. The chunk size is capped by write_window().remaining and by a fixed ceiling, so a write call never has more queued than the protocol allows.
Read whenever writing cannot proceed. A zero window, a zero-byte write and a would-block error all lead to pump_channel_reads. Output is forwarded to the terminal as it arrives, so the window in the other direction keeps flowing.
Bound the wait. The no-progress counter only grows when a whole iteration moved no bytes. After about 0.9 s of zero progress the write fails and returns control. Without that, a dead peer would hang the session thread forever.

Keep the network off the UI thread

The second rule concerns threads. egui is an immediate-mode GUI: the UI thread redraws every frame, and any blocking call on that thread freezes the window. SSH I/O blocks by nature. A write can sit in the drain loop above, and a library call can wait on the socket.

MistTerm gives each interactive session a dedicated OS thread, the shell pump, which alone owns the channel. The UI communicates with it only through bounded queues:

let (pump_tx, pump_rx) = sync_channel::<ShellPumpCommand>(SHELL_PUMP_QUEUE_CAP); // 512
let (interrupt_tx, interrupt_rx) = sync_channel::<Vec<u8>>(8);
let (resize_tx, resize_rx) = sync_channel::<(u32, u32)>(RESIZE_QUEUE_CAP);       // 16

Each loop iteration of the pump thread first drains the priority queue, then applies pending resizes, then waits up to 8 ms for one command. On timeout it services keepalive and reads output. Adjacent keystroke commands are merged into one write of up to 32 KiB before being sent, so a burst of typing costs one queue entry rather than hundreds.

The UI side never blocks on these queues. Every enqueue uses try_send:

/// Uses `try_send` (non-blocking): if the pump thread is stuck in
/// `write_pty_with_drain` or a long read and the queue is full, the caller
/// (the UI thread) is not blocked; an error is returned so the upper layer
/// can drop or degrade.
fn pump_send(&self, cmd: ShellPumpCommand) -> Result<(), String> {
    self.pump_tx.try_send(cmd).map_err(/* "shell pump queue full (cap=512) …" */)
}

This is a trade-off. A blocking send never loses input, but it ties UI responsiveness to the network. try_send keeps the window responsive and, under extreme backlog, reports a dropped command instead. With a 512-entry queue and keystroke merging, a full queue means the session is already effectively stalled.

The 8-entry priority queue exists for control bytes that must jump the queue, through send_priority_interrupt. In v1.2.26 its caller is the ZMODEM cancel path (Part 5). It sends eight CAN bytes, eight backspaces and Ctrl+C, so an aborted transfer is cancelled ahead of whatever file data is still queued.

Summary

An SSH shell is three RFC 4254 requests (pty-req, env, shell) plus window-change for resizes.
Channel data is window-limited in both directions. A client that only writes during a large transfer can deadlock with the remote program. Read whenever the write cannot proceed.
Bound every wait, and keep all network I/O on a dedicated thread. The GUI talks to it through bounded, non-blocking queues.

Next: paste. Same big write, different ways it fails — bracketed paste, and the 4095-byte line.

FAQ

What is the SSH channel window?

A per-channel, per-direction count of bytes the sender may still transmit (RFC 4254 §5.2). Sending consumes it, and the receiver adds to it with SSH_MSG_CHANNEL_WINDOW_ADJUST after consuming data.

Why can a large paste hang an SSH session?

If the client only writes, it stops reading the server's output. The server stops sending, the remote program blocks on its own output and stops reading input, and the client's send window stays at zero.

Is the SSH window-change request related to flow control?

No. window-change (RFC 4254 §6.7) reports a new terminal size. The flow-control window is unrelated.

Which channel requests start an interactive SSH shell?

pty-req, optional env requests, then shell, all defined in RFC 4254.
Related in this series

← Part 1 · Series index · Part 3 →

From Bytes to Glyphs: How a Terminal Emulator Actually Works

c-wind — Sun, 25 Oct 2026 10:35:51 +0000

Originally published on MistLab. This is Part 1 of the series Building an SSH Terminal.

Part 1 of "Building an SSH Terminal". Code excerpts are from MistTerm v1.2.26, an open-source (AGPL) SSH terminal client by MistLab, written in Rust; excerpts are trimmed.

TL;DR

A terminal emulator turns a byte stream from a program into pixels in three stages. A parser (a resumable state machine) splits the stream into printable characters and control sequences defined by ECMA-48 and xterm. A grid of character cells applies their meaning: cursor, scrolling regions, modes, wide characters, the alternate screen. A renderer then draws the grid. MistTerm uses alacritty_terminal for the parser and grid, and draws the grid itself with egui.

Type ls --color and colored file names appear. It feels immediate, but inside the terminal there are three distinct stages between the network and the screen. The remote program writes a stream of bytes with control sequences mixed in. A parser turns that stream into actions. Those actions mutate a grid of character cells. A renderer draws the grid. This article walks through each stage, what the standards say, and how MistTerm wires a mature emulator core into its own GUI.

How does a program talk to a terminal? Only through a byte stream

There is no API between a terminal and the program running in it. Programs such as bash, vim and top write bytes to standard output, and that is all. Printable characters are displayed. Everything else is an instruction to the terminal, encoded with conventions that go back to the DEC VT100 and were later standardized, in part, as ECMA-48 (ISO 6429):

C0 controls — single bytes below 0x20: carriage return \r, line feed \n, backspace \b, tab \t, bell BEL.
Escape sequences — ESC (0x1B) followed by one or more bytes.
CSI sequences — ESC [, optional parameters, optional intermediates, a final byte in 0x40–0x7E. ESC[31m sets the foreground to red (SGR), ESC[2J erases the display, ESC[10;5H moves the cursor to row 10, column 5.
OSC strings — ESC ] … terminated by BEL or the string terminator ESC \; used for window titles, hyperlinks and the like.
DEC private modes — CSI ? Pm h to set and CSI ? Pm l to reset. Mode 1049 switches to the alternate screen (why vim and less seem to "take over" the screen and give it back on exit). Mode 2004 enables bracketed paste, which is the subject of Part 3.

ECMA-48 defines the syntax and many of the functions. The de facto reference for what a modern terminal is expected to do is xterm's Control Sequences document (ctlseqs), because most software assumes "xterm-compatible". That assumption is negotiated through $TERM: when MistTerm opens a session it requests a PTY with the terminal type xterm-256color, and programs on the server look that name up in the terminfo database to decide which sequences to emit.

// src/ssh/client.rs
channel
    .request_pty(
        "xterm-256color",
        None,
        Some((cols, rows, px_w, px_h)),
    )

Stage 1: how a VT parser state machine works

A byte stream has no message boundaries. One read() may end with ESC[3 and the next begin with 1m. The parser therefore has to keep state across calls and consume input one byte at a time.

The widely used model is Paul Williams' DEC-compatible parser (documented at vt100.net), which defines a dozen or so states — ground, escape, CSI entry/param/intermediate/ignore, OSC string, DCS states and so on — and a transition table for every byte value. Transitions emit actions: print a character, execute a C0 control, dispatch a completed escape, CSI or OSC sequence. Some transitions apply "anywhere": ESC always starts a new sequence, and CAN or SUB abort one.

A simplified VT parser state machine

In Rust this layer is the vte crate from the Alacritty project, whose parser is based on Williams' state machine. It deliberately stops at turning bytes into callbacks. It does not know what "erase in line" means.

Stage 2: the grid, where the semantics live

The meaning of each action is where most of the work is. The visible screen is a rows × columns array of cells. Each cell holds a character plus attributes (foreground, background, bold, underline, inverse, wide-character flags, …). Around it sit the cursor position and its saved copy, the set of active modes, scrolling margins, tab stops, the alternate screen, and scrollback history. The subtle rules are the ones applications notice:

After DECSTBM sets a scrolling region, a line feed on its bottom margin scrolls only that region.
The alternate screen has no scrollback.
Writing in the last column leaves the cursor in a "pending wrap" state. The wrap happens only when the next printable character arrives.
A double-width character occupies two cells. Overwriting either half must clear the other.

Whether tmux, vim and htop draw correctly depends on these details. Writing a parser takes an afternoon. Getting the grid semantics right takes years of comparing against xterm and test suites such as vttest.

That is why MistTerm does not implement them itself. alacritty_terminal packages the parser, grid and semantics from the Alacritty terminal as a library. MistTerm's adapter (src/terminal/alacritty.rs) feeds it bytes in one line:

pub fn feed(&mut self, data: &[u8]) {
    if !data.is_empty() {
        self.content_epoch = self.content_epoch.wrapping_add(1);
    }
    self.parser.advance(&mut self.term, data);
}

parser is an alacritty_terminal::vte::ansi::Processor and term is a Term. advance consumes a whole chunk and keeps any half-finished sequence for the next call. content_epoch is MistTerm's own counter. The UI compares it between frames and skips re-layout when nothing changed.

Mode queries go straight to Term. For example, this is how the paste code knows whether the remote program has asked for bracketed paste:

/// Whether the remote program enabled bracketed paste (DECSET 2004; bash/zsh/vim do).
pub fn is_bracketed_paste(&self) -> bool {
    self.term
        .mode()
        .contains(alacritty_terminal::term::TermMode::BRACKETED_PASTE)
}

The grid size must match the remote PTY. When the window is resized, the local Term is resized and the server is told through the SSH window-change channel request (RFC 4254 §6.7; request_pty_size in the ssh2 crate). Otherwise remote programs keep wrapping at the old width.

Stage 3: drawing the grid

Alacritty itself renders glyphs with OpenGL. MistTerm is a full desktop application with side panels, an SFTP browser and dialogs, built on egui (an immediate-mode GUI) with the wgpu backend. So the terminal area is drawn with egui's text layout too.

From PTY bytes to pixels in MistTerm

For each visible row, cells are converted into an egui LayoutJob. Consecutive cells with identical styling are merged into one run, so the layout engine handles a few dozen text segments instead of thousands of single characters:

let same_run = |a: &TextFormat, b: &TextFormat| {
    a.color == b.color
        && a.extra_letter_spacing == b.extra_letter_spacing
        && a.line_height == b.line_height
        && a.font_id == b.font_id
};
// …
for (ch, color, _bg, flags) in row {
    if flags.intersects(wide_spacer) {
        flush_run(&mut job, &mut run_text, &mut run_fmt);
        job.append("\u{200b}", 0.0, cell_fmt(color, 0.0));
        continue;
    }
    let spacing = if flags.contains(Flags::WIDE_CHAR) { cell_w.max(0.0) } else { 0.0 };
    // same format as the current run → push; otherwise flush and start a new run
}

Two details here exist because of CJK text:

Spacer cells. A wide character is stored as the character itself (flagged WIDE_CHAR) followed by a spacer cell (WIDE_CHAR_SPACER). The spacer must not draw anything, but every row has to keep the same number of cells. MistTerm emits a zero-width space for it.
Forcing the advance width. A monospace font's CJK glyph is not guaranteed to be exactly twice the width of its ASCII glyphs. Laid out naively, the error accumulates along the line and the text drifts away from the cursor. Adding extra_letter_spacing to wide characters forces each one to occupy exactly two cell widths.

Backgrounds are handled separately:

// Glyph layer background is always transparent; ANSI and selection backgrounds are
// painted by the UI, so egui's background expansion can't overlap adjacent lines.
let cell_fmt = |color: Color32, extra_spacing: f32| TextFormat {
    font_id: FontId::monospace(font_size),
    color,
    background: Color32::TRANSPARENT,
    line_height: Some(line_height),
    extra_letter_spacing: extra_spacing,
    valign: egui::Align::BOTTOM,
    ..Default::default()
};

A text engine paints a "background color" around each glyph's bounding box, which can bleed into neighboring lines. A terminal needs solid blocks aligned exactly to the cell grid. So MistTerm paints background rectangles itself from grid coordinates, and the text layer only draws foreground glyphs.

What is left to build

With the emulator core borrowed, what remains is the glue: where the bytes come from, where keystrokes go, when to repaint, and selection, search and scrollback in the GUI. The next article starts with where the bytes come from: the SSH channel, the remote PTY and flow control.

Summary

A terminal and its program share only a byte stream. Control sequences follow ECMA-48 and xterm conventions, and $TERM tells the program which dialect to speak.
Parsing is a resumable state machine (Williams' model, implemented by vte).
The grid carries the hard semantics: margins, wrap state, modes, wide characters, alternate screen. alacritty_terminal provides a mature implementation.
Rendering into a general GUI needs care: merge runs, give wide characters explicit spacing, and paint backgrounds per cell.

FAQ

What is the difference between the parser and the grid in a terminal emulator?

The parser only turns bytes into actions such as "print a character" or "dispatch a CSI sequence". The grid gives those actions their meaning: it holds the character cells, the cursor, scrolling margins, modes and the alternate screen.

What does TERM=xterm-256color do?

It tells programs on the server which terminal type to assume. They look the name up in the terminfo database to decide which control sequences to emit. MistTerm requests a PTY with that terminal type.

Why do CJK characters drift out of alignment in some terminals?

A monospace font's CJK glyph is not guaranteed to be exactly twice as wide as its ASCII glyphs, so small errors accumulate along a line. MistTerm forces each wide character to occupy exactly two cell widths.

Why does vim give the screen back when it exits?

It switches to the alternate screen with DEC private mode 1049 and switches back on exit. The alternate screen has no scrollback.
Related in this series

Series index · Part 2 →