Repository navigation
Prefer tz database offsets for conflicting timezone abbreviations - #1366
Merged
Merged
Conversation
Codecov Reportβ
All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #1366 +/- ##
=======================================
Coverage 97.10% 97.10%
=======================================
Files 236 236
Lines 3008 3008
=======================================
Hits 2921 2921
Misses 87 87 β View full report in Codecov by Harness. π New features to boost your workflow:
|
dateparser/timezones.py maps each timezone abbreviation to a single UTC offset. Unrelated zones often share the same letters, and because the table was scraped from Wikipedia, a few abbreviations ended up with a meaning no tz database zone has. "BST" resolved to +11, although the only zones named "BST" today are the UK ones, at +1 (#1321). Rather than patching that one value, compare the whole table against the tz database and fix every abbreviation it disagrees with: - BST: 39600 -> 3600 (British Summer Time) - HDT: -34200 -> -32400 (Hawaii-Aleutian Daylight Time, America/Adak) Abbreviations the tz database names with several offsets, such as CST, IST and PST, are left alone, because there is no single offset to prefer. Abbreviations no zone is named after any more, such as AHST and LMT, are kept so historical text still parses. The comparison lives in dateparser_scripts/tz_abbreviation_conflicts.py and uses pytz, already a dependency, whose bundled copy of the tz database gives the same answer on every platform; zoneinfo would depend on OS-provided data that Windows lacks without the tzdata package. Parsing is unchanged and still uses the static table. Fixes #1322 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
serhii73
force-pushed
the
fix/issue-1322-tz-database-preference
branch
from
August 12, 2026 11:53
955ec9b to
e7ec67a
Compare
AdrianAtZyte
approved these changes
Aug 12, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
dateparser/timezones.pymaps every timezone abbreviation to a single UTC offset. Unrelatedzones frequently share the same letters, and because the table was originally scraped from
Wikipedia, a few abbreviations ended up with a meaning that no tz database zone has.
BSTwas the reported case (#1321): it resolved to +11, but the only zones namedBSTtoday are the UK ones, at +1. The old value came from Bougainville, and
Pacific/Bougainvilleis not namedBSTany more β the tz database now prints+11for it.So this was never a choice between two valid readings. It was a value nothing uses.
Solution
#1321 changed that one line. #1322 asks for the general fix, so this PR resolves the whole
class rather than the single symptom.
dateparser_scripts/tz_abbreviation_conflicts.py(new) compares dateparser's table againstthe tz database and reports two kinds of conflict:
unambiguousβ the tz database names exactly one offset for the abbreviation anddateparser disagrees. The tz database offset wins.
unsupportedβ the tz database names several offsets and dateparser's value is none ofthem. No single offset can be preferred, so it needs a human, but the current value is
still wrong.
Run against
masterit reported exactly two conflicts, bothunambiguous, both fixed here:BST39600(+11)3600(+1)HDT-34200(β9:30)-32400(β9)America/Adak). β9:30 is whatPacific/HonolulucalledHDTback when Hawaii kept β10:30; no zone has been named that sinceAbbreviations the tz database names with several offsets are deliberately left alone,
because there is no tz database answer to prefer:
CST(β6 US, β5 Cuba, +8 China),CDT,IST(+1 Ireland, +2 Israel, +5:30 India) andPST(β8 US, +8 Philippines) keep theirlong-standing values. Abbreviations no zone is named after any more, such as
AHSTandLMT, are also untouched, so historical text still parses.The script exits non-zero when it finds anything, and
CONTRIBUTING.rstdocuments when torun it.
Notes on the approach
pytz, notzoneinfoβpytzis already a dependency and bundles its own copy of thetz database, so the answer is the same on every platform.
zoneinforeads OS-provided dataand has none on Windows without the separate
tzdatapackage. No new dependency, and noruntime cost: parsing still uses the static table.
REFERENCE_YEARSis pinned to settled years (2021β2025), and both bounds matter. Theupper bound stays in the past because the tz database stores future DST rules as
predictions that change between releases. The lower bound keeps retired meanings out β
reaching back to 1999 picks up Guam's old
GSTand would report dateparser'sGST(+4,Gulf) as an unambiguous conflict against +10, even though Guam has used
ChSTsince 2000.So the docs say to slide the window forward, never widen it.
LMTappears fourtimes with four different offsets) and only the first can ever match, because
build_tz_offsetspreserves table order andpop_tz_offset_from_stringreturns the firstmatching pattern. Comparing against the first entry β rather than the set of all of them β
is what stops a wrong first entry hiding behind a correct later one.
ChSTand the tz database'sChSThave to meet under one key.+11,-0930) are offsets rather than identities and are excluded;dateparser matches numeric offsets through its separate UTC/GMT patterns.
~430 in the table; the module docstring states this, along with the fact that pytz ships
METas a copy ofCETsoMET/MESTnever reach the comparison.Tests
A new
TestTzDatabasePreferenceintests/test_timezone_parser.py. Existing tests arebyte-for-byte unchanged.
Behaviour:
BSTandHDTresolve to the tz database offset, both at the popping layer and end-to-endthrough
parse().CST,CDT,ISTandPSTkeep their offsets, each asserting the retained value is onea real zone uses β an assertion that stays valid whether or not the abbreviation remains
ambiguous.
test_table_agrees_with_the_tz_databaseasks the tz database about every sharedabbreviation, so a regression on any of them is caught, with a coverage floor so it cannot
pass by comparing nothing.
Because a "no conflicts" assertion would pass even if the checker were broken, the checker is
tested against a frozen table (so the logic tests do not drift with tz database updates): the
pre-fix
BST/HDToffsets are reported, an offset no zone uses is reported for an ambiguousabbreviation, offsets some zone does use are accepted, out-of-window abbreviations are
ignored, numeric names are excluded, repeated abbreviations resolve to their first entry, and
the reference window is asserted to stay in settled years.
Each behaviour was verified by reverting it and confirming the matching test fails (including
widening the window backwards, which fails on the
GSTfalse positive described above).Full suite: 24159 passed, 20 skipped, 1 xfailed. Also run on Python 3.10 and 3.14, and
against the
minpinpytz==2024.2as well as currentpytz, with identical results.Fixes #1322