Repository navigation
Apply an explicit DATE_ORDER to every date that search_dates finds (#1512) - #1535
Merged
Merged
Conversation
β¦1512) search_dates parses each date after the first one again, with the date found before it as RELATIVE_BASE, unless the caller set one. Since the thread-safety change in 1.4.3, it does that on a copy of the settings made by Settings.replace() instead of changing the caller's settings in place. The copy did not keep track of which settings the caller set, so an explicit DATE_ORDER counted as the default one and the order of the locale won for every date after the first: with DATE_ORDER "DMY", "01-11-2026" became January 11 instead of November 1. STRICT_DATE_ORDER stopped applying to those dates for the same reason. Have Settings.replace() keep the settings that the caller set unless it is given others. That fixes every copy, not only the one search_dates makes. The only other one, made for language detection, is always made from the default settings, so it does not change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
AdrianAtZyte
approved these changes
Oct 5, 2026
AdrianAtZyte
left a comment
Member
There was a problem hiding this comment.
Please, reword the docstring of
test_search_dates_does_not_check_the_rebuilt_settings, since the rebuilt settings no longer forget which settings the caller set, and drop the# Issue #1512comment from the repro test.
The docstring of test_search_dates_does_not_check_the_rebuilt_settings said that the settings search_dates rebuilds for the dates after the first one "forget which settings the caller set". Settings.replace() keeps track of them now, so say that the rebuilt settings remember which settings the caller set. Also drop the "# Issue #1512" comment from the test of the repro in the issue. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1512.
Root cause
Unless the caller sets
RELATIVE_BASE,search_datesparses each date after the first one again, with the date found before it asRELATIVE_BASE. Up to 1.4.2 it set that on the caller's settings object in place. #1346 made this thread-safe by working on a copy,parser._settings.replace(RELATIVE_BASE=...). ButSettings.replace()only keeps_mod_settings, the record of which settings the caller set, when it is passedmod_settings=. The copy has none, so_try_parsertakes the explicitDATE_ORDERfor the default one and lets the order of the locale win. Search parses the English translation of every language but vi and hu, so that order is MDY.STRICT_DATE_ORDER(#1443) stopped applying to those dates for the same reason, since it only applies with an explicitDATE_ORDER.The issue also suspects the
date_orderchange in_try_parser. That change is not involved:de7b986right before #1346cd9bb13(#1346), 1.4.3, mastercd9bb13with only itssearch.pyhunk revertedThe fix
Settings.replace()keeps the caller's_mod_settingsunless it is given others, so a copy made to change some settings still knows which of the others the caller set. Nothing is mutated: the copy is a new object, as before, and shares the read-only mapping.#1515 (closed unmerged) proposed passing
mod_settings=at the call site instead. The two give identical results on every input below, but fixingreplace()also covers the next copy someone makes. The only other copy, inFullTextLanguageDetector, is always made from the default settings, which have nothing to keep, so it does not change. Neither doessettings.replace(...)on the default settings, which is how aSettingsobject is built by hand.Tests
The diff of
tests/is additions only; no existing test was touched.STRICT_PARSING:DMYin nl, de, fr, ru and en;MDYin vi and hu, whose own orders are DMY and YMD, and in en;YMDin en, es and vi with two-digit years, since a four-digit year first is read as the year under any order;DATE_ORDER, in en, vi and hu, alone and next to other settings: every date is still read in the order of the locale;PREFER_DATES_FROMfor a date with no year, a relative date (in 3 days,Π§Π΅ΡΠ΅Π· 3 Π΄Π½Ρ) counted from the date before it,TIMEZONEwithTO_TIMEZONE, and withRETURN_AS_TIMEZONE_AWARE;STRICT_DATE_ORDER"all"and"year"on the second date;Settings.replace()itself: it keeps an explicitDATE_ORDER, and the default one it copies does not become explicit.On master 10 tests and 20 subtests fail, each because a date is read in the order of the locale instead of the explicit one or, for
STRICT_DATE_ORDER, is not rejected. Only the controls pass there: noDATE_ORDER, MDY in English and thereplace()guard.Verification
pytest: 24,761 passed, 18 skipped, 1 xfailed; master has 24,736 passed with the same skips and xfail. Same result withTZset toPacific/AucklandandPacific/Fiji, as in CI.all,py311,py313,py314,mypy,docs,scriptsandtwinecheckpass.py310,minandmin-allcannot installatherison my machine, so I ran their suites in venvs with the same pins minusatheris, which onlyfuzzing/imports. All pass.pre-commit run --all-filespasses: ruff check, ruff format, zizmor. The pre-commit thattox -e pre-commitinstalls needs a newer git than mine for--all-files, so I ran pre-commit 3.8.0 with the pinned hooks, andtox -e pre-commit -- --files $(git ls-files)as well.Differential run against master over 138,985 results. The texts: 800 generated with two to four complete dates in 11 languages, 195 that mix in relative and yearless dates, and 289 strings from
tests/test_search.py. Each ran under 33 settings, with and withoutRELATIVE_BASE, plus the ngram strategy andparse()on each date alone:DATE_ORDERRELATIVE_BASEset by the callerparse()DATE_ORDER, noRELATIVE_BASETwo independent checks show the changes are the fix:
RELATIVE_BASEset by the caller keeps master off the broken path. With complete dates, this branch without one agrees with master with one on every date that does not take a part from that base. The 1,808 results that differ hold pieces with no full date, like10:30, or two-digit years underPREFER_DATES_FROM: "future", which are resolved against the base.YYYY-MM-DDfirst date that 1.4.2 read as year-day-month under a day-first order (a first date, so untouched by this bug), the later dates that take their month or day from it, and adans 3 jours,that 1.4.2 read as three days ago (Mistake search_dates for ru languageΒ #706).Some changed results look odd. With
DATE_ORDER: "YMD",20 ΠΌΠ°ΡΡΠ°after19 ΠΌΠ°ΡΡΠ° 2001becomes 2020-03-19, because YMD reads a lone number as the year. That is whatparse()returns with the same settings and base, and what 1.4.2 returned in search.Note
The docstring of
test_search_dates_does_not_check_the_rebuilt_settings(#1443) says the rebuilt settings "forget which settings the caller set", which is no longer true. Its assertions still hold. I left it alone so that this PR only adds tests; it can be reworded here or in a follow-up.π€ Generated with Claude Code