Skip to content

Prefer tz database offsets for conflicting timezone abbreviations - #1366

Merged
serhii73 merged 1 commit into
masterfrom
fix/issue-1322-tz-database-preference
Aug 12, 2026
Merged

serhii73 merged 1 commit into
masterfrom
fix/issue-1322-tz-database-preference

Conversation

@serhii73

@serhii73 serhii73 commented Aug 12, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

dateparser/timezones.py maps every timezone abbreviation to a single UTC offset. Unrelated
zones frequently share the same letters, and because the table was originally scraped from
Wikipedia, a few abbreviations ended up with a meaning that no tz database zone has.

BST was the reported case (#1321): it resolved to +11, but the only zones named BST
today are the UK ones, at +1. The old value came from Bougainville, and
Pacific/Bougainville is not named BST any more β€” the tz database now prints +11 for it.

So this was never a choice between two valid readings. It was a value nothing uses.

Solution

#1321 changed that one line. #1322 asks for the general fix, so this PR resolves the whole
class rather than the single symptom.

dateparser_scripts/tz_abbreviation_conflicts.py (new) compares dateparser's table against
the tz database and reports two kinds of conflict:

  • unambiguous β€” the tz database names exactly one offset for the abbreviation and
    dateparser disagrees. The tz database offset wins.
  • unsupported β€” the tz database names several offsets and dateparser's value is none of
    them. No single offset can be preferred, so it needs a human, but the current value is
    still wrong.

Run against master it reported exactly two conflicts, both unambiguous, both fixed here:

Abbreviation Was Now Why
BST 39600 (+11) 3600 (+1) British Summer Time β€” the only current meaning (#1321)
HDT -34200 (βˆ’9:30) -32400 (βˆ’9) Hawaii-Aleutian Daylight Time (America/Adak). βˆ’9:30 is what Pacific/Honolulu called HDT back when Hawaii kept βˆ’10:30; no zone has been named that since

Abbreviations the tz database names with several offsets are deliberately left alone,
because there is no tz database answer to prefer: CST (βˆ’6 US, βˆ’5 Cuba, +8 China), CDT,
IST (+1 Ireland, +2 Israel, +5:30 India) and PST (βˆ’8 US, +8 Philippines) keep their
long-standing values. Abbreviations no zone is named after any more, such as AHST and
LMT, are also untouched, so historical text still parses.

The script exits non-zero when it finds anything, and CONTRIBUTING.rst documents when to
run it.

Notes on the approach

  • pytz, not zoneinfo β€” pytz is already a dependency and bundles its own copy of the
    tz database, so the answer is the same on every platform. zoneinfo reads OS-provided data
    and has none on Windows without the separate tzdata package. No new dependency, and no
    runtime cost: parsing still uses the static table.
  • REFERENCE_YEARS is pinned to settled years (2021–2025), and both bounds matter. The
    upper bound stays in the past because the tz database stores future DST rules as
    predictions that change between releases. The lower bound keeps retired meanings out β€”
    reaching back to 1999 picks up Guam's old GST and would report dateparser's GST (+4,
    Gulf) as an unambiguous conflict against +10, even though Guam has used ChST since 2000.
    So the docs say to slide the window forward, never widen it.
  • First entry wins. A few abbreviations are listed more than once (LMT appears four
    times with four different offsets) and only the first can ever match, because
    build_tz_offsets preserves table order and pop_tz_offset_from_string returns the first
    matching pattern. Comparing against the first entry β€” rather than the set of all of them β€”
    is what stops a wrong first entry hiding behind a correct later one.
  • Case-insensitive, because the parser is: the table's ChST and the tz database's
    ChST have to meet under one key.
  • Numeric zone names (+11, -0930) are offsets rather than identities and are excluded;
    dateparser matches numeric offsets through its separate UTC/GMT patterns.
  • Scope. Only the ~48 abbreviations the tz database still prints are checkable, out of
    ~430 in the table; the module docstring states this, along with the fact that pytz ships
    MET as a copy of CET so MET/MEST never reach the comparison.

Tests

A new TestTzDatabasePreference in tests/test_timezone_parser.py. Existing tests are
byte-for-byte unchanged.

Behaviour:

  • BST and HDT resolve to the tz database offset, both at the popping layer and end-to-end
    through parse().
  • CST, CDT, IST and PST keep their offsets, each asserting the retained value is one
    a real zone uses β€” an assertion that stays valid whether or not the abbreviation remains
    ambiguous.
  • test_table_agrees_with_the_tz_database asks the tz database about every shared
    abbreviation, so a regression on any of them is caught, with a coverage floor so it cannot
    pass by comparing nothing.

Because a "no conflicts" assertion would pass even if the checker were broken, the checker is
tested against a frozen table (so the logic tests do not drift with tz database updates): the
pre-fix BST/HDT offsets are reported, an offset no zone uses is reported for an ambiguous
abbreviation, offsets some zone does use are accepted, out-of-window abbreviations are
ignored, numeric names are excluded, repeated abbreviations resolve to their first entry, and
the reference window is asserted to stay in settled years.

Each behaviour was verified by reverting it and confirming the matching test fails (including
widening the window backwards, which fails on the GST false positive described above).

Full suite: 24159 passed, 20 skipped, 1 xfailed. Also run on Python 3.10 and 3.14, and
against the min pin pytz==2024.2 as well as current pytz, with identical results.

Fixes #1322

@codecov

codecov Bot commented Aug 12, 2026 •

Copy link
Copy Markdown

Codecov Report

βœ… All modified and coverable lines are covered by tests.
βœ… Project coverage is 97.10%. Comparing base (b1f603e) to head (e7ec67a).

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #1366   +/-   ##
=======================================
  Coverage   97.10%   97.10%           
=======================================
  Files         236      236           
  Lines        3008     3008           
=======================================
  Hits         2921     2921           
  Misses         87       87           

β˜” View full report in Codecov by Harness.
πŸ“’ Have feedback on the report? Share it here.

πŸš€ New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

dateparser/timezones.py maps each timezone abbreviation to a single UTC
offset. Unrelated zones often share the same letters, and because the
table was scraped from Wikipedia, a few abbreviations ended up with a
meaning no tz database zone has. "BST" resolved to +11, although the
only zones named "BST" today are the UK ones, at +1 (#1321).

Rather than patching that one value, compare the whole table against the
tz database and fix every abbreviation it disagrees with:

- BST: 39600 -> 3600 (British Summer Time)
- HDT: -34200 -> -32400 (Hawaii-Aleutian Daylight Time, America/Adak)

Abbreviations the tz database names with several offsets, such as CST,
IST and PST, are left alone, because there is no single offset to
prefer. Abbreviations no zone is named after any more, such as AHST and
LMT, are kept so historical text still parses.

The comparison lives in dateparser_scripts/tz_abbreviation_conflicts.py
and uses pytz, already a dependency, whose bundled copy of the tz
database gives the same answer on every platform; zoneinfo would depend
on OS-provided data that Windows lacks without the tzdata package.
Parsing is unchanged and still uses the static table.

Fixes #1322

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@serhii73
serhii73 force-pushed the fix/issue-1322-tz-database-preference branch from 955ec9b to e7ec67a Compare August 12, 2026 11:53
@serhii73
serhii73 merged commit 764091d into master Aug 12, 2026
18 checks passed
@serhii73
serhii73 deleted the fix/issue-1322-tz-database-preference branch August 12, 2026 12:52
@serhii73 serhii73 mentioned this pull request Sep 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Prefer choices from the tz database

2 participants