Skip to content

Avoid quadratic runtime in right_hand_split_with_omits on long lines (#5270) - #5475

Merged
cobaltt7 merged 3 commits into
psf:mainfrom
Prateek99-s:fix-subscript-chain-quadratic-5270
Oct 6, 2026
Merged

cobaltt7 merged 3 commits into
psf:mainfrom
Prateek99-s:fix-subscript-chain-quadratic-5270

Conversation

@Prateek99-s

@Prateek99-s Prateek99-s commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fixes #5270.
Closes #5275
Closes #5305

Formatting lines containing long chains of consecutive bracket trailers (such as subscript chains a[0][1]...[160]) suffered from quadratic runtime due to combinatorial omit trials in right_hand_split_with_omits.

Root Cause

Unlike attribute chains which have dot delimiters and are split cleanly in a single pass by delimiter_split, consecutive
subscript chains ][ have no intermediate delimiters and fall back to right_hand_split_with_omits.

generate_trailers_to_omit yields bracket pairs from the tail up to line_length. In long lines with many small trailers
(like [0]...), Black was performing up to 20 full right_hand_split calls per recursion level, only for
is_line_short_enough(lines[0]) to fail on every single omit candidate because each candidate split point was still hundreds of
characters from the start of the line.

Fix

  1. Precompute cumulative prefix lengths for all leaves in line.leaves once at the start of right_hand_split_with_omits.
  2. For each candidate omit, inspect the target opening_bracket the split would break on. If its prefix length from the start of the line alone exceeds mode.line_length, any head line produced by that split is mathematically guaranteed to exceed line_length. We safely skip executing right_hand_split for that omit.
  3. Candidate omits that break on an opening bracket within mode.line_length (e.g. foo(long_args...)[key]) are retained and evaluated normally.
  4. In the final fallback, reuse first_lines instead of redundantly re-running right_hand_split.
  5. Output formatting is 100% invariant across all cases (diff-shades shows 0 changes).

Benchmark

  • $n = 100$ subscripts: ~2.8s -> ~0.35s (~8x speedup, 2,574 RHS calls -> 156 RHS calls)
  • $n = 160$ subscripts: ~9.5s -> ~0.35s (~27x speedup)
  • $n = 300$ subscripts: >30s -> ~0.95s (~30x speedup)

Test Plan

  • Added test_subscript_chain_avoids_quadratic_rhs_splits in tests/test_black.py to verify that right_hand_split invocations remain linearly bounded ($O(n)$) rather than quadratic ($O(n^2)$).
  • Full unit test suite passes.
  • All pre-commit checks pass.

@Prateek99-s
Prateek99-s force-pushed the fix-subscript-chain-quadratic-5270 branch from caba7f3 to c554ed5 Compare October 3, 2026 07:35
@github-actions

github-actions Bot commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

diff-shades results comparing this PR (0fcc9da) to main (a3cb9e1):

--preview style: no changes

--stable style: no changes


What is this? | Workflow run | diff-shades documentation

@Prateek99-s
Prateek99-s force-pushed the fix-subscript-chain-quadratic-5270 branch 2 times, most recently from fbe5509 to c9aaeb0 Compare October 3, 2026 10:28
Comment thread tests/test_black.py Resolved
@cobaltt7

cobaltt7 commented Oct 3, 2026

Copy link
Copy Markdown
Collaborator

This PR also has a significantly malformed description, please fix it

@Prateek99-s
Prateek99-s force-pushed the fix-subscript-chain-quadratic-5270 branch from c9aaeb0 to 9badfad Compare October 6, 2026 12:42
@Prateek99-s

Copy link
Copy Markdown
Contributor Author

Updated the test to wrap right_hand_split and assert that split calls are bounded linearly (mock_rhs.call_count < 2 * n). Without pruning unreachable omits, n = 100 made 2,574 calls (O(nΒ²)), whereas with prefix pruning it stays bounded at 156 calls (O(n)). This avoids fragile wall-clock timings on CI while guaranteeing quadratic backtracking cannot regress.

@cobaltt7

cobaltt7 commented Oct 6, 2026

Copy link
Copy Markdown
Collaborator

Thanks!

@cobaltt7
cobaltt7 merged commit 27c6fd0 into psf:main Oct 6, 2026
58 checks passed
luketainton pushed a commit to luketainton/luke_instant-msg-api that referenced this pull request Oct 10, 2026
This PR contains the following updates:

| Package | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) |
|---|---|---|---|
| [black](https://github.com/psf/black) ([changelog](https://github.com/psf/black/blob/main/CHANGES.md)) | `<26.10.1,>=26.10.0` β†’ `<26.10.2,>=26.10.1` | ![age](https://developer.mend.io/api/mc/badges/age/pypi/black/26.10.1?slim=true) | ![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/black/26.10.0/26.10.1?slim=true) |

---

### Release Notes

<details>
<summary>psf/black (black)</summary>

### [`v26.10.1`](https://github.com/psf/black/blob/HEAD/CHANGES.md#Version-26101)

[Compare Source](psf/black@26.10.0...26.10.1)

##### Highlights

This release fixes a security issue in Black's bundled GitHub Action. Repositories that
use the action are encouraged to update to the latest version of Black immediately. This
update is received automatically when using `psf/black@stable`, and is independent of
the version of Black installed by the action.

When the GitHub Action reads version specifiers from `tool.black.required-version`, it
now only accepts released versions of Black, as it already did for `black` requirements
in dependency lists. Other values, such as URL references, are now rejected. This issue
has been registered as `GHSA-cg8m-r9f2-5wm2`; a CVE identifier is pending.

##### Stable style

- Fix a long Jupyter notebook assignment magic (for example `x = !ls -la`) being wrapped
  in parentheses, which IPython can no longer run; such cells are now left unchanged
  ([#&#8203;5481](psf/black#5481))
- Keep repeated lines outside the selected `--line-ranges` unchanged ([#&#8203;5436](psf/black#5436))
- Fix `--line-ranges` formatting lines that follow a statement when formatting joins or
  re-indents that statement ([#&#8203;5484](psf/black#5484))
- Fix `--line-ranges` crashing with an internal error when an unselected statement has a
  `# fmt: skip` comment on the last line before a closing bracket ([#&#8203;5477](psf/black#5477))
- Prevent moving an encoding declaration (for example `# -*- coding: latin-1 -*-`) onto
  the first two lines of a file ([#&#8203;5487](psf/black#5487))
- Fix adding an extra trailing blank line inside a multi-line module docstring when its
  last line is near the line length limit ([#&#8203;5494](psf/black#5494))

##### Preview style

- Do not split short expressions on delimiters (such as binary operators, comparisons,
  or comprehensions) inside brackets when preceded by a standalone comment ([#&#8203;5455](psf/black#5455))
- Fix the unstable `string_processing` feature removing backslash-newline sequences from
  raw strings, which changed their value ([#&#8203;5482](psf/black#5482))
- Avoid adding unnecessary parentheses around unbreakable right-hand side expressions in
  assignments (such as annotated assignments or subscript targets) ([#&#8203;5473](psf/black#5473))
- Parenthesize expressions with parameter comments when they exceed the line length
  under `--preview` ([#&#8203;5442](psf/black#5442))
- Fix `wrap_long_dict_values_in_parens` dropping the parentheses around a right-hand
  side that contains a dictionary, leaving a line over the length limit ([#&#8203;5492](psf/black#5492))
- Fix the unstable `string_processing` feature adding a space after `*` or `**` in a PEP
  695 type parameter (`* Ts = ...`) when it splits a string in the parameter's default
  ([#&#8203;5504](psf/black#5504))
- Avoid unnecessary line splits of trailers (such as indexing/subscripts) when an
  earlier bracket split exceeds the line length only due to a trailing comment ([#&#8203;5498](psf/black#5498))

##### Configuration

- Add `--cache-dir` to configure the cache directory from the command line ([#&#8203;5433](psf/black#5433))
- Don't cache a file as formatted after formatting or checking it with `--line-ranges`,
  which made a later full `black --check` pass and `black` skip the unformatted lines
  ([#&#8203;5476](psf/black#5476))
- Respect a `.gitignore` that starts with a UTF-8 byte order mark. The mark became part
  of the first pattern, so that pattern never matched and Black formatted files that git
  ignores ([#&#8203;5497](psf/black#5497))

##### Performance

- Avoid quadratic runtime when splitting very long lines with many trailing bracket
  pairs (such as consecutive subscripts) ([#&#8203;5475](psf/black#5475))

##### Integrations

- Fix the GitHub Action crashing on dependency-group includes when `use_pyproject` is
  enabled ([#&#8203;5479](psf/black#5479))
- When the GitHub Action reads version specifiers from `tool.black.required-version`, it
  now only accepts released versions of Black, as it already did for `black`
  requirements in dependency lists. Other values, such as URL references, are now
  rejected. ([#&#8203;5510](psf/black#5510))

</details>

---

### Configuration

πŸ“… **Schedule**: (UTC)

- Branch creation
  - At any time (no schedule defined)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Enabled.

β™» **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.

πŸ”• **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Mend Renovate CLI](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC4xNDkuMCIsInVwZGF0ZWRJblZlciI6IjQ0LjE0OS4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJ0eXBlL2RlcGVuZGVuY2llcyJdfQ==-->

Reviewed-on: https://git.tainton.uk/luke/instant-msg-api/pulls/292
Co-authored-by: renovate[bot] <renovate-bot@git.tainton.uk>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Quadratic formatting time for chains of subscripts (7s for one 160-subscript line; attribute/call chains stay linear)

2 participants