Repository navigation
fix: Parse 1mon ago - #1329
Merged
Merged
fix: Parse 1mon ago#1329
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #1329 +/- ##
==========================================
+ Coverage 97.10% 97.11% +0.01%
==========================================
Files 235 235
Lines 2904 2915 +11
==========================================
+ Hits 2820 2831 +11
Misses 84 84 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
…mpat Python 3.14 changed the json C encoder to access dict subclasses via direct C-level storage rather than calling Python-level __iter__/items(). ruamel.yaml's CommentedMap relies on its Python-level iteration for correct key ordering, so the C shortcut produced a non-deterministic key order on 3.14, causing test_dateparser_data_integrity to fail. Add _to_plain_types() to recursively convert CommentedMap → OrderedDict and CommentedSeq → list (preserving insertion order via Python-level iteration) before passing data to json.dumps in write_complete_data().
Contributor
There was a problem hiding this comment.
Pull request overview
This PR aims to fix issue #1123 by making the freshness/relative-time parser understand compact English month abbreviations like 1mon ago by updating English translation/simplification data and adding regression tests. It also tweaks the data-generation script to produce deterministic JSON output across Python versions.
Changes:
- Add English simplification for
\d+mon(s)→\d+ monthand extend month tokens in EN translation data. - Update freshness parser tests to assert that
1mon ago/2mon ago/3mons agoare parsed as relative months. - Make
write_complete_data.pyconvert ruamel YAML types to plain mapping/sequence types beforejson.dumpsfor stable output.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| tests/test_freshness_date_parser.py | Adds regression coverage asserting *mon(s) ago parses as relative months. |
| dateparser/data/date_translation_data/en.py | Updates EN month tokens and adds a simplification regex for mons?. |
| dateparser_scripts/write_complete_data.py | Normalizes YAML-loaded structures to stable plain types before serialization. |
| dateparser_data/supplementary_language_data/date_translation_data/en.yaml | Mirrors EN month-token + simplification updates in supplementary YAML source. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| "month", | ||
| "months" | ||
| "months", | ||
| "mon", |
Member
There was a problem hiding this comment.
This is also my main concern with this change.
I wonder if you can use AI to expand the tests with inputs that would become problematic after this change and inputs that were problematic before it, and decide which ones is best to leave broken, keeping them as expected failures.
| ], | ||
| "simplifications": [ | ||
| { | ||
| "(\\d+)\\s*mons?\\b": "\\1 month" |
Comment on lines
2365
to
+2379
| @parameterized.expand( | ||
| [ | ||
| param("1mon ago"), # 1116 | ||
| param("1mon ago", ago={"months": 1}, period="month"), # 1123 | ||
| param("2mon ago", ago={"months": 2}, period="month"), # 1123 | ||
| param("3mons ago", ago={"months": 3}, period="month"), # 1123 | ||
| ] | ||
| ) | ||
| def test_known_issues(self, date_string): | ||
| def test_known_issues(self, date_string, ago, period): | ||
| self.given_parser() | ||
| self.given_date_string(date_string) | ||
| self.when_date_is_parsed() | ||
| self.then_error_was_not_raised() | ||
| self.assertEqual(None, self.result["date_obj"]) | ||
| self.then_date_was_parsed_by_freshness_parser() | ||
| self.then_date_obj_is_exactly_this_time_ago(ago) | ||
| self.then_period_is(period) |
…fication
Adding bare 'mon'/'mons' to the English month tokens broke weekday parsing
('Mon, 05 Dec 2022 ...' returned None). Drop those tokens and let the
(\d+[.,]?\d*) mons? -> month simplification handle '1mon ago' while keeping
Monday intact. Broaden the numeric group to allow decimals for consistency
with other relative patterns.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Close #1123