Skip to content

refactor(pty-proxy): migrate output parsing to vte - #4518

Merged
atuin-bot merged 1 commit into
mainfrom
refactor/pty-proxy-one-parser
Oct 22, 2026
Merged

atuin-bot merged 1 commit into
mainfrom
refactor/pty-proxy-one-parser

Conversation

@markovejnovic

@markovejnovic markovejnovic commented Oct 22, 2026 •

Copy link
Copy Markdown
Member

Each read of the child's output went through up to four parsers: the debug highlighter's on the output task, then capture's and the highlighter's again on the emulator thread. All of them were markers.rs, a hand-written scanner. markers.rs becomes parser.rs, which follows the output with atuin-vte, Atuin's fork of vte. atuin-vt100 0.19.2 is built on it too, so the stream parser reads the output as the emulator does.

Behaviour changes

  • 0x9C no longer ends a marker. A UTF-8 terminal reads it as part of a character, and every shell integration ends its markers with BEL. No user-facing change. Our shell integrations use BEL anyways and it's unclear to me why we handled 0x9C.
  • Controls inside a marker are left out of it, and a marker that CAN or SUB cuts off, or that any ESC ends, counts as sent. No user-facing change. Our shell integrations do not send any controls inside markers.
  • An unfinished OSC is held whole, rather than its first bytes. vte doesn't cap it, so neither the emulator (as on main) nor the parser does; a cap belongs in atuin-vte. This means that memory usage of the pty-proxy with malicious applications grows unbounded. main had this bug, except that it was only in the emulator.rs codepath.

Performance

  • P50 keystroke latency increased by 30%. This is due to vte's performance problems. It is around 30us on my machine now.
  • P99 keystroke latency is unchanged.
  • Throughput is unchanged.
  • Pty-proxy under adversarial workloads uses about 2% more CPU on average.

@greptile-apps

greptile-apps Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; no new blocking issue was established.

Summary

The PR parses child output once and shares Output with capture, debug labels, and the screen.

  • The PTY proxy parses child output once and shares marker events.

Reviews (5) · Last reviewed commit: "refactor(pty-proxy): parse with atuin-vt..." · Reviewed by Greptile

Comment thread crates/atuin-pty-proxy/src/parser.rs Resolved
Comment thread crates/atuin-pty-proxy/src/parser.rs Resolved
@socket-security

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addedcargo/​atuin-vte@​0.15.110010093100100
Updatedcargo/​atuin-vt100@​0.19.1 ⏵ 0.19.210010093100100

View full report

@markovejnovic markovejnovic changed the title refactor(pty-proxy): parse the child's output once, with vte refactor(pty-proxy): migrate output parsing to vte Oct 22, 2026
@markovejnovic

Copy link
Copy Markdown
Member Author

/merge

@atuin-bot

atuin-bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

Landed as af714d0. GitHub shows this merged once it reaches main.

Each read of the child's output went through up to four parsers: the debug highlighter's on the output task, then capture's and the highlighter's again on the emulator thread. All of them were `markers.rs`, a hand-written scanner. `markers.rs` becomes `parser.rs`, which follows the output with `atuin-vte`, Atuin's fork of `vte`. `atuin-vt100` 0.19.2 is built on it too, so the stream parser reads the output as the emulator does.

##Behaviour changes

- `0x9C` no longer ends a marker. A UTF-8 terminal reads it as part of a character, and every shell integration ends its markers with `BEL`. **No user-facing change. Our shell integrations use `BEL` anyways and it's unclear to me why we handled `0x9C`.**
- Controls inside a marker are left out of it, and a marker that `CAN` or `SUB` cuts off, or that any `ESC` ends, counts as sent. **No user-facing change. Our shell integrations do not send any controls inside markers.**
- An unfinished OSC is held whole, rather than its first bytes. `vte` doesn't cap it, so neither the emulator (as on main) nor the parser does; a cap belongs in `atuin-vte`. **This means that memory usage of the `pty-proxy` with malicious applications grows unbounded. `main` had this bug, except that it was only in the `emulator.rs` codepath.** 

## Performance

- P50 keystroke latency increased by 30%. This is due to `vte`'s performance problems. It is around 30us on my machine now.
- P99 keystroke latency is unchanged.
- Throughput is unchanged.
- Pty-proxy under adversarial workloads uses about 2% more CPU on average.
@atuin-bot
atuin-bot force-pushed the refactor/pty-proxy-one-parser branch from 8ad26fa to af714d0 Compare October 22, 2026 04:02
@atuin-bot
atuin-bot merged commit af714d0 into main Oct 22, 2026
3 checks passed
@atuin-bot
atuin-bot deleted the refactor/pty-proxy-one-parser branch October 22, 2026 04:02
atuin-bot pushed a commit that referenced this pull request Oct 22, 2026
### Bug Fixes

- *(ai)* Import IntoStaticStr from strum_macros, not strum ([#4480](#4480))
- *(ai)* Improve detection of git repositories ([#4450](#4450))
- *(client)* Resolve relative config paths beside config.toml, not the cwd ([#4479](#4479))
- *(config)* Don't print errors twice ([#4501](#4501))
- *(daemon)* Fetch the server's capabilities until a fetch succeeds
- *(daemon)* Spare a stale pidfile's PID when evicting ([#4512](#4512))
- *(daemon)* Handle daemons not spawned by atuin ([#4516](#4516))
- *(output)* Keep quoted words together in output search ([#4502](#4502))
- *(packfile)* Load only the records inside a manifest's range
- *(pty-proxy)* Standalone init with `atuin pty-proxy init` ([#4449](#4449))
- *(pty-proxy)* Wait for input on a non-blocking inherited stdin ([#4486](#4486))
- *(pty-proxy)* Don't re-exec shells running commands from a string or file ([#4414](#4414))
- *(search)* Don't panic when copying with no results ([#4426](#4426))
- *(server)* Let `ATUIN_*` environment variables override server.toml ([#4456](#4456))
- *(update)* Accept ATUIN_GITHUB_TOKEN ([#4262](#4262))
- Remove duplicated word in the no-history stats message ([#4451](#4451))
- Error printing in `?` handler ([#4448](#4448))
- Normalize relative gitdir paths in worktree resolution ([#4506](#4506))
- Upgrade regressions found in release QA ([#4519](#4519))


### Documentation

- Fix ctrl_n_shortcuts default value ([#4458](#4458))
- List all history list --format variables ([#4460](#4460))
- Fix the --reverse default for history list ([#4472](#4472))
- Mention December default in wrapped --year help ([#4476](#4476))


### Features

- *(common)* AbsPath and AbsPathBuf, Path and PathBuf checked to be absolute ([#4481](#4481))
- *(common)* Find a process's cgroup under cgroup v1 and v2 ([#4513](#4513))
- *(daemon)* Run the daemon as a launchd agent on macOS ([#4517](#4517))
- *(pty-proxy)* Window title follows child process ([#4447](#4447))
- *(secrets)* Keep HashiCorp tokens out of history ([#4445](#4445))


### Miscellaneous Tasks

- Give mono its own nixfmt step and scope atuin's to atuin/
- Say a /merge's landed commit gets CI on main
- Rebuild the apt .deb cache for Ubuntu 24.04 ([#4504](#4504))
- Remove the benchmarks ([#4511](#4511))


### Performance

- *(common)* Compile the vendor regexes once ([#4485](#4485))
- *(common)* Drop the tree watcher's file-id cache ([#4492](#4492))
- *(common)* Release a line reader's buffer at the end of the file ([#4494](#4494))
- *(common)* Keep no tree watcher entry for a rejected file ([#4493](#4493))
- *(daemon)* Warm up only a resumed session's unanswered calls ([#4490](#4490))
- *(packfile)* Compress packs in one shot ([#4484](#4484))


### Refactor

- *(daemon)* Managed daemon service with multiple backends ([#4488](#4488))
- *(daemon)* Move daemon startup into the service module ([#4503](#4503))
- *(daemon)* Keep where the daemon listens behind service::address ([#4514](#4514))
- *(daemon)* Take up the daemon's socket in SocketHolder, not Backend ([#4515](#4515))
- *(deps)* Bump clap_complete_nushell from 4.6.0 to 4.6.2 ([#4473](#4473))
- *(deps)* Bump clap_complete from 4.6.9 to 4.6.11 ([#4396](#4396))
- *(deps)* Bump ratatui from 0.30.0 to 0.30.2 ([#4474](#4474))
- *(deps)* Bump certifi from 2025.11.12 to 2026.10.19 in /docs in the uv group across 1 directory ([#4500](#4500))
- *(pty-proxy)* Run on tokio with the parser on its own runtime ([#4482](#4482))
- *(pty-proxy)* Forward captures to the daemon on the proxy's runtime ([#4487](#4487))
- *(pty-proxy)* Rename the screen's parser to Emulator ([#4510](#4510))
- *(pty-proxy)* Migrate output parsing to vte ([#4518](#4518))


### Testing

- *(daemon)* Run the output index tests under rstest ([#4496](#4496))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants