🔥 Jadwal & Hasil: astral sh/ruff/pull/

charliermarsh · 2026-10-18T23:17:45Z

Summary

We scan ASCII identifier continuations in sixteen-byte batches using a byte lookup table, avoiding per-character decoding. Keyword lookup filters by length and initial byte, then uses a compact hash table with a full-word comparison to reject collisions. Non-ASCII suffixes retain the existing Unicode path.

astral-sh-bot · 2026-10-18T23:21:29Z

Memory usage report

Memory usage unchanged ✅

astral-sh-bot · 2026-10-18T23:21:54Z

`ecosystem-analyzer` results

No diagnostic changes detected ✅

Full report with detailed diff (timing results)

astral-sh-bot · 2026-10-18T23:32:53Z

`ruff-ecosystem` results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

Formatter (stable)

✅ ecosystem check detected no format changes.

Formatter (preview)

✅ ecosystem check detected no format changes.

codspeed · 2026-10-19T00:36:56Z

Merging this PR will improve performance by 4.86%

1 warning

⚠️ Base and head ran on different runner types

Some benchmarks with significant performance changes were compared across different runner types, so the changes might not come from the code. Run base and head on the same runner type, or use CodSpeed Macro Runners for more stable results.

Learn more about Macro Runners

Open the report in CodSpeed to investigate

⚡ 3 improved benchmarks
✅ 259 untouched benchmarks
⏩ 14 skipped benchmarks¹

Performance Changes

	Mode	Benchmark	`BASE`	`HEAD`	Efficiency
⚡	CPU Simulation	`lexer[numpy/globals.py]`	41.6 µs	39.5 µs	+5.38%
⚡	CPU Simulation	`lexer[pydantic/types.py]`	682.6 µs	649.9 µs	+5.03%
⚡	CPU Simulation	`lexer[numpy/ctypeslib.py]`	302.1 µs	290 µs	+4.17%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.

_{Comparing charlie/ruff-ascii-identifiers (6818c90) with main (5fabb6f)}

14 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

MichaReiser · 2026-10-19T21:43:07Z

why?

charliermarsh · 2026-10-19T21:45:26Z

I got the vibe from your comments that you didn't want to proceed with it haha! Do you think it's salvageable? I'm happy to clean it up, I just thought you were left feeling like the tradeoffs weren't right.

MichaReiser · 2026-10-19T21:53:46Z

I think it's a worthwhile change. I just think it requires some more explanation of what happens here and how to update that weird constant when adding/removing kewyords 😆

charliermarsh · 2026-10-19T21:56:49Z

Haha ok. I will clean it up. I'll ask Codex to make it even weirder and more confusing.

MichaReiser · 2026-10-23T17:05:23Z

+# dependencies = []
+# ///
+
+"""Generate the lexer's keyword table and collision-free hash multiplier.


How is this different from using phf?

So, I tried phf::Map, but it seems like it's meaningfully slower because phf still runs SipHash (the main benefit seems to be that it guarantees you don't have collisions?) but these can just fit in a u64.

MichaReiser

Thank you. This makes sense to me. The only part that isn't clear to me (and might be worth documenting), is why we can't use pfh. I'd prefer that over a custom Python script that generates the lookup table.

astral-automations-bot · 2026-10-23T17:50:49Z

Memory usage report

Memory usage unchanged ✅

Results for 6818c90 · Workflow run

astral-automations-bot · 2026-10-23T17:52:56Z

`ecosystem-analyzer` results

No diagnostic changes detected ✅

Full report with detailed diff · Timing results

Results for 6818c90 · Workflow run

## Summary We give ASCII identifiers a dedicated path, avoiding Unicode bookkeeping and checking string prefixes only when a one- or two-letter name reaches a quote. Both paths share the existing batched ASCII scanner, preserving fast scanning for Unicode names with long ASCII suffixes. Stacked on #29463 to measure the incremental improvement. [CodSpeed reports](https://github.com/astral-sh/ruff/pull/29503/checks?check_run_id=118300735426) 6.6–9.8% greater efficiency across all five lexer benchmarks, with no reported regressions. We preserve the existing scanner and add no inlining attributes or unsafe code.

This PR contains the following updates: | Package | Type | Update | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) | |---|---|---|---|---|---| | [DavidAnson/markdownlint-cli2](https://github.com/DavidAnson/markdownlint-cli2) | repository | minor | `v0.23.3` → `v0.24.0` | ![age](https://developer.mend.io/api/mc/badges/age/github-tags/DavidAnson%2fmarkdownlint-cli2/v0.24.0?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/github-tags/DavidAnson%2fmarkdownlint-cli2/v0.23.3/v0.24.0?slim=true) | | [ruff](https://docs.astral.sh/ruff) ([source](https://github.com/astral-sh/ruff), [changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)) | dependency-groups | patch | `==0.17.1` → `==0.17.3` | ![age](https://developer.mend.io/api/mc/badges/age/pypi/ruff/0.17.3?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/ruff/0.17.1/0.17.3?slim=true) | Note: The `pre-commit` manager in Renovate is not supported by the `pre-commit` maintainers or community. Please do not report any problems there, instead [create a Discussion in the Renovate repository](https://github.com/renovatebot/renovate/discussions/new) if you have any questions. --- ### Release Notes <details> <summary>DavidAnson/markdownlint-cli2</summary> ### [`v0.24.0`](DavidAnson/markdownlint-cli2@v0.23.3...v0.24.0) [Compare Source](DavidAnson/markdownlint-cli2@v0.23.3...v0.24.0) </details> <details> <summary>astral-sh/ruff (ruff)</summary> ### [`v0.17.3`](https://github.com/astral-sh/ruff/blob/HEAD/CHANGELOG.md#0173) [Compare Source](astral-sh/ruff@0.17.2...0.17.3) Released on 2026-10-26. ##### Preview features - \[`suspicious`] Add `shared-clock-mock` ([#29074](astral-sh/ruff#29074)) ##### Bug fixes - \[`pyupgrade`] Avoid incorrect conversions to f-strings (`UP032`) ([#25485](astral-sh/ruff#25485)) ##### Performance - Buffer stdout in `ruff format` ([#29631](astral-sh/ruff#29631)) - Specialize ASCII identifier lexing ([#29503](astral-sh/ruff#29503)) - Speed up identifier and keyword lexing ([#29463](astral-sh/ruff#29463)) ##### Documentation - \[`flake8-bandit`] Describe what the rule actually checks for `httpx` (`S113`) ([#27936](astral-sh/ruff#27936)) ##### Contributors - [@itsvedantkumar](https://github.com/itsvedantkumar) - [@charliermarsh](https://github.com/charliermarsh) - [@anishgirianish](https://github.com/anishgirianish) ### [`v0.17.2`](https://github.com/astral-sh/ruff/blob/HEAD/CHANGELOG.md#0172) [Compare Source](astral-sh/ruff@0.17.1...0.17.2) Released on 2026-10-22. ##### Preview features - \[`complexity`] Add `unnecessary-regular-expression-compile` ([#25533](astral-sh/ruff#25533)) ##### Bug fixes - Avoid syntax errors for type variables with starred constraints (`UP040`, `UP046`, `UP047`, `RUF053`) ([#28362](astral-sh/ruff#28362)) - \[`pydocstyle`] Treat backslash-escaped `*args` and `**kwargs` as documented (`D417`) ([#29536](astral-sh/ruff#29536)) - \[`pylint`] Avoid false positive on starred elements (`PLE0643`) ([#29483](astral-sh/ruff#29483)) - \[`pylint`] Mark fixes unsafe when deleting comments (`PLR5501`, `PLW0120`) ([#29414](astral-sh/ruff#29414)) - \[`pyupgrade`] Mark fix unsafe when deleting comments (`UP035`) ([#29413](astral-sh/ruff#29413)) ##### Performance - Avoid intermediate names in lint name resolution ([#29506](astral-sh/ruff#29506)) - Prefilter bidirectional Unicode checks by UTF-8 leading bytes ([#29561](astral-sh/ruff#29561)) - Share postfix parsing and reduce expression copies ([#29480](astral-sh/ruff#29480)) ##### Configuration - \[`pylint`] Add a configurable allowlist for `magic-value-comparison` (`PLR2004`) ([#18961](astral-sh/ruff#18961)) ##### Documentation - Clarify documentation and correct stale guidance ([#29374](astral-sh/ruff#29374)) - \[`pyupgrade`] Remove mentions of variance related to fix safety (`UP040`, `UP047`) ([#29592](astral-sh/ruff#29592)) ##### Contributors - [@fixfon](https://github.com/fixfon) - [@MichaReiser](https://github.com/MichaReiser) - [@zaniebot](https://github.com/zaniebot) - [@FrancescoPezzella](https://github.com/FrancescoPezzella) - [@anishgirianish](https://github.com/anishgirianish) - [@rajveer3x](https://github.com/rajveer3x) - [@charliermarsh](https://github.com/charliermarsh) - [@LckyLike](https://github.com/LckyLke) - [@DaniBodor](https://github.com/DaniBodor) </details> --- ### Configuration 📅 **Schedule**: (in timezone America/Denver) - Branch creation - Between 02:00 AM and 06:59 AM, only on Tuesday (`* 2-6 * * 2`) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://github.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate CLI](https://github.com/renovatebot/renovate).  Reviewed-on: https://git.lichtenwalter.site/rlichten/dir2text/pulls/100

charliermarsh added performance Potential performance improvement parser Related to the parser labels Oct 18, 2026

charliermarsh temporarily deployed to automations October 18, 2026 23:18 — with GitHub Actions Inactive

charliermarsh changed the title ~~Scan ASCII identifiers with byte lookups~~ Batch ASCII identifier scanning Oct 18, 2026

charliermarsh had a problem deploying to automations October 18, 2026 23:33 — with GitHub Actions Error

charliermarsh temporarily deployed to automations October 18, 2026 23:37 — with GitHub Actions Inactive

charliermarsh temporarily deployed to automations October 19, 2026 00:17 — with GitHub Actions Inactive

charliermarsh temporarily deployed to automations October 19, 2026 00:47 — with GitHub Actions Inactive

charliermarsh mentioned this pull request Oct 19, 2026

Reduce atom-parser return value size #29475

Closed

charliermarsh changed the title ~~Batch ASCII identifier scanning~~ Speed up identifier and keyword lexing Oct 19, 2026

charliermarsh temporarily deployed to automations October 19, 2026 08:16 — with GitHub Actions Inactive

charliermarsh changed the base branch from main to charlie/ruff-shared-postfix-atom October 19, 2026 14:02

charliermarsh force-pushed the charlie/ruff-ascii-identifiers branch from a1177bd to f91d049 Compare October 19, 2026 14:02

charliermarsh temporarily deployed to automations October 19, 2026 14:03 — with GitHub Actions Inactive

charliermarsh temporarily deployed to automations October 19, 2026 14:37 — with GitHub Actions Inactive

charliermarsh force-pushed the charlie/ruff-ascii-identifiers branch from ea02032 to d47bd15 Compare October 19, 2026 15:11

charliermarsh temporarily deployed to automations October 19, 2026 15:12 — with GitHub Actions Inactive

charliermarsh temporarily deployed to automations October 19, 2026 16:27 — with GitHub Actions Inactive

charliermarsh added this pull request to stack #29499 October 19, 2026 17:56

charliermarsh mentioned this pull request Oct 19, 2026

Specialize ASCII identifier lexing #29503

Merged

Base automatically changed from charlie/ruff-shared-postfix-atom to main October 19, 2026 20:35

charliermarsh force-pushed the charlie/ruff-ascii-identifiers branch from d47bd15 to 03bf69f Compare October 19, 2026 20:35

charliermarsh temporarily deployed to automations October 19, 2026 20:35 — with GitHub Actions Inactive

charliermarsh marked this pull request as ready for review October 19, 2026 21:05

charliermarsh closed this Oct 19, 2026

charliermarsh reopened this Oct 19, 2026

charliermarsh temporarily deployed to automations October 19, 2026 21:57 — with GitHub Actions Inactive

charliermarsh temporarily deployed to automations October 20, 2026 00:58 — with GitHub Actions Inactive

charliermarsh requested a review from MichaReiser October 23, 2026 14:04

MichaReiser reviewed Oct 23, 2026

View reviewed changes

MichaReiser approved these changes Oct 23, 2026

View reviewed changes

charliermarsh had a problem deploying to automations October 23, 2026 17:42 — with GitHub Actions Error

charliermarsh added 9 commits October 23, 2026 13:44

Scan ASCII identifiers with byte lookups

d90ac20

Scan identifier bytes in fixed-size batches

755524c

Use sixteen-byte identifier batches

91f6345

Skip keyword slicing for long ASCII identifiers

341444d

Compare keyword candidates as bytes

69ac5fc

Recognize Python keywords with a compact hash table

ec5c926

Simplify keyword table construction

c5db21a

Clarify identifier scanning and generate keyword tables

bdbce68

Document the lexer keyword hash tradeoff

6818c90

charliermarsh force-pushed the charlie/ruff-ascii-identifiers branch from 6e5b67b to 6818c90 Compare October 23, 2026 17:45

charliermarsh temporarily deployed to automations October 23, 2026 17:50 — with GitHub Actions Inactive

charliermarsh merged commit de9d86e into main Oct 23, 2026
77 checks passed

charliermarsh deleted the charlie/ruff-ascii-identifiers branch October 23, 2026 17:58

BrewTestBot mentioned this pull request Oct 27, 2026

ruff 0.17.3 Homebrew/homebrew-core#321519

Merged

Conversation

charliermarsh commented Oct 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

astral-sh-bot Bot commented Oct 18, 2026

Memory usage report

Uh oh!

astral-sh-bot Bot commented Oct 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

ecosystem-analyzer results

Uh oh!

astral-sh-bot Bot commented Oct 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

ruff-ecosystem results

Linter (stable)

Linter (preview)

Formatter (stable)

Formatter (preview)

Uh oh!

codspeed Bot commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will improve performance by 4.86%

Performance Changes

Footnotes

Uh oh!

MichaReiser commented Oct 19, 2026

Uh oh!

charliermarsh commented Oct 19, 2026

Uh oh!

MichaReiser commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

charliermarsh commented Oct 19, 2026

Uh oh!

MichaReiser Oct 23, 2026

Choose a reason for hiding this comment

Uh oh!

charliermarsh Oct 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MichaReiser left a comment

Choose a reason for hiding this comment

Uh oh!

astral-automations-bot Bot commented Oct 23, 2026

Memory usage report

Uh oh!

astral-automations-bot Bot commented Oct 23, 2026

ecosystem-analyzer results

Uh oh!

Uh oh!

This branch was previously deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

charliermarsh commented Oct 18, 2026 •

edited

Loading

astral-sh-bot Bot commented Oct 18, 2026 •

edited

Loading

`ecosystem-analyzer` results

astral-sh-bot Bot commented Oct 18, 2026 •

edited

Loading

`ruff-ecosystem` results

codspeed Bot commented Oct 19, 2026 •

edited

Loading

MichaReiser commented Oct 19, 2026 •

edited

Loading

charliermarsh Oct 23, 2026 •

edited

Loading

`ecosystem-analyzer` results