Skip to content

Keep the %z offset when parsing with date_formats - #1424

Merged
AdrianAtZyte merged 5 commits into
scrapinghub:masterfrom
AdrianAtZyte:parse-format-offset
Oct 22, 2026
Merged

AdrianAtZyte merged 5 commits into
scrapinghub:masterfrom
AdrianAtZyte:parse-format-offset

Conversation

@AdrianAtZyte

@AdrianAtZyte AdrianAtZyte commented Sep 23, 2026 •

Copy link
Copy Markdown
Member

Fixes #376, resolves #956, closes #840

@codspeed

codspeed Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

βœ… 10 untouched benchmarks


Comparing AdrianAtZyte:parse-format-offset (aba9b7f) with master (78b2fc2)

Open in CodSpeed

@codecov

codecov Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

Codecov Report

βœ… All modified and coverable lines are covered by tests.
βœ… Project coverage is 99.08%. Comparing base (6d32a3e) to head (aba9b7f).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1424      +/-   ##
==========================================
+ Coverage   99.05%   99.08%   +0.03%     
==========================================
  Files         239      239              
  Lines        3900     3935      +35     
==========================================
+ Hits         3863     3899      +36     
+ Misses         37       36       -1     

β˜” View full report in Codecov by Harness.
πŸ“’ Have feedback on the report? Share it here.

πŸš€ New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@AdrianAtZyte
AdrianAtZyte marked this pull request as ready for review September 23, 2026 12:39

@serhii73 serhii73 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this fixes #956 and #376 nicely, and the TIMEZONE / TO_TIMEZONE / RETURN_AS_TIMEZONE_AWARE handling now matches
the free-text parser.

One regression before merging: a format that combines a two-digit year with %z now raises.

import dateparser
dateparser.parse('26 Jan 18 16:32:21 +0000', date_formats=['%d %b %y %H:%M:%S %z'])
  • master: datetime.datetime(2018, 1, 26, 16, 32, 21)
  • this PR: TypeError: can't compare offset-naive and offset-aware datetimes

date_obj is now aware, and _apply_century_preference() compares it with a naive now. It happens with and without
RELATIVE_BASE, and under every PREFER_DATES_FROM. Comparing naive values there fixes it:

date_naive = date_obj.replace(tzinfo=None)
if now_naive < date_naive and prefer_from == "past":
    ...
elif now_naive >= date_naive and prefer_from == "future":
    ...

With that change, '26 Jan 99 16:32:21 +0100' gives 1999 (past/default) and 2099 (future), keeping the +01:00 offset.
Could you add a %y + %z case to the new test?

@serhii73 serhii73 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review (claude code-review, at e15e85d). The full suite passes on this head (24353 passed) and keeping the %z offset is the stated goal, so I'm approving. One thing needs a note before release:

  • User-visible change. parse('2021-09-22 12:30-1030', date_formats=['%Y-%m-%d %H:%M%z']) used to return a naive datetime and now returns 2021-09-22 12:30:00-10:30 with default settings. Callers comparing the result with naive datetimes will get a TypeError, and results change when TIMEZONE or TO_TIMEZONE is set. Please add a changelog line and a sentence in the date_formats docs about %z.

Non-blocking notes:

  • The RETURN_AS_TIMEZONE_AWARE logic now exists in three places (date_parser.py, freshness_date_parser.py, utils), and apply_timezone_from_settings changed from is not True to truthiness; a shared helper would avoid drift.
  • With TIMEZONE='local' (the default) the result stays in the parsed offset, while any other TIMEZONE converts to it. That matches the non-format path, but the tests only cover the Asia/Tokyo case, not local with TO_TIMEZONE.
  • The %y century preference compares naive wall-clock times and ignores the parsed offset, so a date within a day of RELATIVE_BASE can land on the wrong side of PREFER_DATES_FROM (e.g. 1/1/99 00:00 +0100 with a UTC base of 1999-01-01 00:00 and future gives 2099).
  • %Z names (UTC, EST) are still dropped, so %Z results are naive while %z results are aware; worth documenting.

@AdrianAtZyte
AdrianAtZyte merged commit f858b9e into scrapinghub:master Oct 22, 2026
19 checks passed
serhii73 added a commit to AdrianAtZyte/dateparser that referenced this pull request Oct 23, 2026
Keep both sentences of the date_formats docstring paragraphs: the PARSERS
fallback sentence of this PR and the %z/%Z sentence from scrapinghub#1424.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unable to parse date with offset [Bug] Unable to use date_formats argument if timezone already found

2 participants