Skip to content

fix: preserve complete week dates when parsing custom formats - #1381

Merged
serhii73 merged 1 commit into
scrapinghub:masterfrom
ryanchou1994:fix/week-date-formats
Sep 23, 2026
Merged

serhii73 merged 1 commit into
scrapinghub:masterfrom
ryanchou1994:fix/week-date-formats

Conversation

@ryanchou1994

@ryanchou1994 ryanchou1994 commented Sep 9, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Custom formats containing a year, week number, and weekday are complete dates, but the missing-field detection treats them as missing their month and day. For example, 01.24-1 with %W.%y-%w is overwritten with date preferences instead of returning 2024-01-01. ISO week dates also have their computed year replaced because %G is not recognized.

Preserve the complete date returned by strptime, including calendar years that differ from the ISO week year. Week 0 can reach the previous year and needs calendar.isleap; the private calendar currently loads _strptime by mistake. Copy the standard calendar namespace before rebinding the English names, without re-executing calendar or changing its global constants.

The missing-field check uses a flat year/week/weekday condition after strptime has validated the format.

The regressions cover week numbering conventions, year boundaries, leap days, weekday names, date preferences, escaped directives, incomplete formats, invalid ISO dates, and global calendar isolation.

Fixes #1242.

Validation

  • The original week-date regressions fail on the base implementation; the additional invalid ISO pairing is rejected before and after simplifying the condition.
  • macOS Python 3.13.13 and Linux Python 3.13.15: 24,380 passed, 4 skipped, 1 xfailed, plus 36 subtests on each platform; coverage remains 97%.
  • Focused format, calendar-isolation, and strptime tests: 108 passed plus 36 subtests, with the real French locale available on both platforms.
  • Ruff 0.9.3 lint and format checks passed with the current upstream configuration, including W605. The full suites ran with the new warnings-as-errors setting enabled.

Copilot AI lite review requested due to automatic review settings September 9, 2026 05:51

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@AdrianAtZyte

Copy link
Copy Markdown
Member

The new condition in _get_missing_parts re-encodes rules strptime already enforces: %V requires %G and rejects %Y, and %G requires %V plus a weekday. Since we only reach this code after strptime succeeded, a flat check works and reads better.

@ryanchou1994

Copy link
Copy Markdown
Contributor Author

Thanks, simplified the condition to the three groups: week, weekday, and year. The regex still consumes %% pairs so literal directives do not count. I also added the %G-W%W-%u case to the invalid ISO tests to cover the validation we rely on from strptime.

The focused tests and full suite pass on macOS and Linux with the new warnings-as-errors configuration (24,380 passed, 4 skipped, 1 xfailed, plus 36 subtests on each platform).

@AdrianAtZyte AdrianAtZyte left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks! Two nits, no need for another round if you would rather not:

  • The if "%U" in fmt or "%W" in fmt or "%V" in fmt: guard is redundant with the first set intersection, so the block can be flat.
  • I would drop the comment in strptime.py. It explains why the alternative was not taken, which belongs in the commit message; the code reads fine on its own.

@ryanchou1994

Copy link
Copy Markdown
Contributor Author

Thanks for the approval, @AdrianAtZyte!

@serhii73 when you have a moment, could you take a look and approve the workflow runs? They're still waiting on maintainer approval, so the full test suite hasn't run on this PR yet.

Happy to apply Adrian's two nits here or in a follow-up, whichever you prefer.

@codecov

codecov Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

Codecov Report

βœ… All modified and coverable lines are covered by tests.
βœ… Project coverage is 97.19%. Comparing base (6114c90) to head (99e022b).
⚠️ Report is 3 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #1381   +/-   ##
=======================================
  Coverage   97.18%   97.19%           
=======================================
  Files         236      236           
  Lines        3057     3063    +6     
=======================================
+ Hits         2971     2977    +6     
  Misses         86       86           

β˜” View full report in Codecov by Harness.
πŸ“’ Have feedback on the report? Share it here.

πŸš€ New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@codspeed

codspeed Bot commented Sep 23, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

βœ… 7 untouched benchmarks


Comparing ryanchou1994:fix/week-date-formats (99e022b) with master (6114c90)

Open in CodSpeed

@serhii73

Copy link
Copy Markdown
Collaborator

Thanks!

@serhii73
serhii73 merged commit 2c7ffb9 into scrapinghub:master Sep 23, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

%w | %W and %V | %u format string directives are not supported?

4 participants