Repository navigation
Add a STRICT_DATE_ORDER setting to enforce the year position or the whole DATE_ORDER (#868) - #1443
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #1443 +/- ##
==========================================
+ Coverage 98.94% 98.96% +0.02%
==========================================
Files 239 239
Lines 3521 3590 +69
==========================================
+ Hits 3484 3553 +69
Misses 37 37 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Any thoughts on this LLM feedback?
|
|
Thanks, agreed on the split. I reworked this PR into the
On the other points:
Cross-locale finality. I kept it, because without it the fix only holds with an explicit
|
|
Should we get #1419 merged first? |
|
I think we also need to merge #1431 first. And I think we should then rethink this PR. The documented way for DATE_ORDER to work as as a preference. If we want to be able to enforce specific date orders, we should expose some way for users to opt into that alternative behavior. |
cc8afd4 to
95ea50b
Compare
|
Thanks! With #1419 and #1431 merged, I've rebased this onto master and reworked it along your suggestion. The check is now opt-in through a new setting, With the setting on, the only thing it does in
One question on scope: the setting enforces where the year is, but a day and month that do not fit are still swapped ( |
|
What if we make |
… the year (#868) DATE_ORDER is a preference: any number that is not a valid day or month is read as a two-digit year, so "32 DEC 10" with DATE_ORDER="DMY" is 2032-12-10. With the new setting, which defaults to False, a two-digit year is read only where DATE_ORDER puts the year, and such a date string gives None instead. The rejection is final: no other locale may guess another reading, and the no-spaces parser, which would read the same numbers in a date order again, is skipped. The other parsers of the locale still get their turn. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The setting is now "none" (the default), "year" or "all", so that it can name all three behaviours: DATE_ORDER as a preference, the position of a two-digit year enforced, or the whole order enforced. With "all", a day and a month that do not fit are no longer swapped, neither in the token loop nor through the retry with the swapped order: "13/12/10" with DATE_ORDER="MDY" is None. A four-digit year is still read wherever it is, and a date that starts with one is still read as year, month and day. With "all", a date string that cannot be read in DATE_ORDER at all is final too, so that no other locale reads it differently: with auto-detected languages, "10年20月00日" with DMY would otherwise come back from zh as a relative date, 10 years and 20 months ago. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
95ea50b to
05f2737
Compare
|
A two-digit year is now read only where DATE_ORDER puts the year: the day and month read before it must be the ones the order puts before it. Until now only a day or month that the order puts before the year, but is read after it, was rejected, so "32 DEC 10" with DMY was None but "10 DEC 32" with MYD was still 2032-12-10, and "year" did nothing at all for YMD and YDM, where no part comes before the year. This also rejects #519's last-resort "4-99" with YMD, which is acceptable for an opt-in setting. DMY and MDY are not affected, since nothing comes after the year in them. The day and month swap is now also tested under DYM and MYD, where the year is between them and the swap changes which of them is read before it. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… to enforce "year" and "all" only enforce a DATE_ORDER that the caller sets, since the default one is a preference, so without one they silently did nothing. Raise a SettingValidationError from check_settings instead, which only looks at the settings the caller passed, so the rebuilt settings of search_dates, which forget those, cannot trigger it. "none" does not need a DATE_ORDER. The cases that pinned "no effect without DATE_ORDER" are replaced by tests of the error, of "none", and of search_dates and a reused parser, which keep working. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
"year" now reads as one rule: a two-digit year is read only from the position DATE_ORDER gives the year. Of the details after the examples, only two matter to users, that "year" and "all" need DATE_ORDER, and that '95年12月10日' with MDY is rejected. The rest, such as that a rejection is final across locales, which parsers still run, and that a month name or a missing part is not checked, is in the code comments. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
05f2737 records in auto_order that the end of a day range is dropped when a month name follows it, but it always deleted the entry after the month. In a day-first order, as in "12-12 June 2021" with DMY or "5-5 juin 2021" in French, the day comes before the month, so the deletion raised IndexError, parse_alpha swallowed it and the month name was not read: the date was None even without STRICT_DATE_ORDER, unlike master. Remove the day entry itself instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…_ORDER Moving "4-99" with YMD and "10 DEC 32" with MYD to the rejected cases left no test of #519's two-digit year fallback under STRICT_DATE_ORDER, and left its comment on "10年20月00日", which goes through the day and month swap instead. Also say what the search_dates test checks: that it does not raise. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Without it, "read a two-digit year only from the position DATE_ORDER gives the year" suggests that "32 DEC" with DMY is rejected too, but a month name and a missing part are not checked, and a four-digit year is read wherever it is. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
AdrianAtZyte
left a comment
There was a problem hiding this comment.
- Please, drop the issue numbers from the
_StrictDateOrderErrordocstring and the#1419comment.- In
docs/settings.rst, quote the values ('none','year','all'), and the doctests should not show aNoneline, since that is not what the REPL prints.
…amples Quote the values of the setting in the docs, since they are strings, and drop the None lines from its examples, since the REPL prints nothing for None. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fixes #701, part of #868 (the
DATE_ORDERhalf, opt-in through a new setting. Thedate_formatshalf is left for a separate discussion in #868).With
DATE_ORDERset as shown:parse(...)"none"(default)"year""all""32 DEC 10",DMYNoneNone"10 DEC 32",MYDNoneNone"13/12/10",MDYNone"10 DEC 95",DMYThe setting
STRICT_DATE_ORDERis a string."none"keepsDATE_ORDERa preference, as on master."year"reads a two-digit year only from the positionDATE_ORDERgives the year."all"also keeps the day and month where the order puts them, so no swap (#1431)."year"and"all"raise aSettingValidationErrorifDATE_ORDERis not passed: the default one is only a preference, so there is nothing to enforce.search_datesand a reusedDateDataParserare not affected. With the defaultPARSERS,parse()can only turn a date intoNone. A rejection is final across locales (ruwould read"20 09 29"with MYD as a 2029 date) and skips the no-spaces parser. Only parts read from numbers are checked:"DEC 32"with DMY is still December 2032.Stricter results under the opt-in
With
"year", besides the table: an invalid month ("12/13/10"with DMY), CJK dates under an order that does not start with the year ("95年12月10日"with MDY, documented), rare orders ("40 12"with DYM), and, under YMD, YDM, MYD and DYM, a two-digit year in a position the order does not give ("DEC 10 95"with YMD, #519's"4-99"with YMD).Known gaps
date_formats, the other half of Given a month with 32 days, dateparser fails with some years but parses with others #868.search_dateswithoutRELATIVE_BASEchecks only the first date (Settings.replace()drops the explicit settings, a separate bug). A rejected span is dropped or split differently, and the dates after it resolve against the previous one, so they can change too.PARSERSthat runsabsolute-timefirst, a rejected reading can be replaced by the next parser's."10 95 DEC") is rejected by"year"too, although the year is in place.STRICT_DATE_ORDER, as a bool: to agree before either lands.Tests
127 tests; the full suite (24,606),
pre-commit,mypyand the docs build pass. Differential run against master and this PR's previous head (7,683 strings × everyDATE_ORDERand none × auto/en/ru/ja; 860,496parse()and 268,800search_dates()results):"none": 0 differences from master (430,248 and 134,400 compared), nor in 5,440 day ranges such as"5-5 juin 2021"in every language."year","all": no exception; everyparse()change is a date becomingNone(17,100 and 26,219)."all"is the"year"result orNone, except"00月31日"with MYD and auto-detected languages (as before)."year"against the previous head: 5,632 moreNone, all under YMD, YDM and MYD (DMY and MDY cannot change: nothing comes after the year there)."all"is unchanged.HISTORY.rstis left to the release PR. Suggested bullet:🤖 Generated with Claude Code