[ethanstoner]

Description

Fixes #3855

With string processing enabled (--unstable), Black produces code it can't parse when the value of a keyword argument, a lambda body or an else branch is an implicit concatenation that StringMerger won't merge, e.g. raw strings:

$ printf "dict(a=r' ' r'')\n" > foo.py
$ black -l 1 --unstable foo.py
error: cannot parse: foo.py:5:4
        r""
       ^
ParseError: bad input

The same happens at the default line length when the first string is too long, e.g. some_function_name(argument=r"<long string>" r" more").

StringMerger refuses raw strings (StringMerger does NOT merge raw strings.), so the concatenation stays split and the earlier delimiter split puts a=r" " and r"" on separate lines. StringParenWrapper then gets the line a=r" " on its own. _assign_match (and _dict_or_lambda_match / _else_match) only look at the leaves in the current line, so they match, and the wrapper turns it into:

dict(
    a=(
        r" "
    )
    r""
)

This PR wraps the whole concatenation in parens instead:

dict(
    a=(
        r" "
        r""
    )
)

That is also what --unstable already does for concatenations it can merge (argument=( with the merged string split inside).

• linegen.py: for a line inside brackets whose only delimiters are between implicitly concatenated strings and which doesn't start with a string (a=STRING STRING, lambda: STRING STRING, else STRING STRING), string_paren_wrap is tried before delimiter_split. If it doesn't match, delimiter_split runs next as before.
• trans.py: _assign_match and _dict_or_lambda_match skip strings concatenated with the matched one (_else_match already accepted anything after the string), so the wrapper puts the whole concatenation inside the parens. do_splitter_match declines when the first string already fits on the line or the line has comments, so those lines are still split at the concatenation as before. It also still refuses a string whose next sibling is a STRING on another line, as a fallback for the cases that don't take this path (e.g. a comment between the strings).

I added three cases to preview_long_strings__regression.py (keyword argument, lambda, else branch) at the default line length, plus one where the first string fits exactly on the line, which main already formats without parens and which must stay that way. With upstream main's trans.py and linegen.py the test fails with black.parsing.InvalidInput: cannot parse: 702:4. With them, it passes.

To check that output which didn't crash before is unchanged, I formatted the inputs in tests/data/cases/, the files in src/black/, and 6636 generated implicit concatenations (raw, bytes, f-string and plain prefixes; keyword arguments, dict values, lambdas, else, return, assignments, call arguments, with and without comments) in stable, --preview and --unstable mode at line lengths 88, 40, 20 and 1, with main and with this branch. None of the 83268 runs changed output. 4479 runs that crash on main now format, all under --unstable.

Validation (Windows, Python 3.12):

• pytest -n auto tests: 501 passed, 3 skipped
• pre-commit run on the changed files: isort, flake8, mypy, prettier and the other hooks pass

Checklist - did you ...

• Implement any code style changes under the --preview style, following the stability policy? (Only affects string_processing, and only input that used to crash)
• Add an entry in CHANGES.md if necessary?
• Add / update tests if necessary?
• Add new / update outdated documentation? (N/A)

[github-actions]

diff-shades results comparing this PR (2a25418) to main (8d5a2d9):

--preview style: no changes

--stable style: no changes

---

What is this? | Workflow run | diff-shades documentation

[cobaltt7]

I think I would prefer both strings to be parenthesized, such as:

dict(
    a=(
        r" "
        r""
    )
)

Your proposed formatting doesn't clearly show that the strings are part of the same value.

[ethanstoner]

Done in c9e79d2: dict(a=r' ' r'') at -l 1 --unstable now gives exactly that form, and the lambda body and else branch get the same treatment (lambda: ( and else ( with both strings inside). This only kicks in when the first string doesn't fit on the line. When it does fit, the concatenation is still split without parens as before, same as stable output, so formatting that didn't crash is unchanged (2a25418 adds a test case for that).

[cobaltt7]

Thanks!