Repository navigation
Stop parsing strings with only skipped words as today - #1440
Merged
AdrianAtZyte merged 6 commits intoOct 27, 2026
Merged
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #1440 +/- ##
=======================================
Coverage 99.18% 99.18%
=======================================
Files 239 239
Lines 4054 4058 +4
=======================================
+ Hits 4021 4025 +4
Misses 33 33 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
AdrianAtZyte
marked this pull request as ready for review
September 24, 2026 10:32
This was referenced Oct 16, 2026
serhii73
approved these changes
Oct 20, 2026
serhii73
left a comment
Collaborator
There was a problem hiding this comment.
Code review (claude code-review, at e198d57). The full suite passes on this head (24612 passed) and I found no case that was right on master and is wrong here; the strings it changes ('year', 'hour minute', 't') returned today's date before.
Non-blocking notes:
- Because the skipped-word-only parse is now rejected in the parser, the language loop falls through to the next locale, so a single letter can produce a different spurious date:
parse('h', settings={'RELATIVE_BASE': datetime(2020,5,5,10)})gave 2020-05-05 (en) on master and now gives 2020-05-04 via Hungarianh(Monday). Both are wrong, but a check in the applicability or skip-word layer would not change which locale wins. all()over an emptyfiltered_tokensis True, so an empty token list now raises "Nothing to parse" too; please note it or add a test.- The check compares token strings and ignores
token_type, and duplicates the skip logic of the loop below it; a single pass counting non-skipped tokens would be simpler. - Tests cover
year,hour minute,tand one search sentence. The docs text aboutPARSERS=[custom-formats]withdate_formatsand the single-letter behaviour across locales have no test.
Keep both sentences of the date_formats docstring paragraphs: the PARSERS fallback sentence of this PR and the %z/%Z sentence from scrapinghub#1424. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
1 task done
Collaborator
|
I guess we can feel free to merge this PR, @AdrianAtZyte |
serhii73
added a commit
that referenced
this pull request
Nov 4, 2026
#1564) Regression from #1401 and #1440. search_dates found the date that a relative date is relative to by looking back through the items parsed so far for the last one that is a date. Since #1401 the look-back goes over every item that has no date, for example a word like "day" or "ngày" that does not parse as a date on its own ("hour" is one too since #1440), so in a long run of such words each word looked back over all the words before it, and each word also copied the whole list of items parsed so far. Keep that date up to date as each item is parsed instead, and hand it to parse_item in place of the list. The date is the same, so the results do not change, but the time spent on a word no longer depends on how many words came before it. The new tests are in their own file, tests/test_search_unparsed_word_runs.py. Two of them count how many times a DateData is read for runs of 200 and 400 words: the count doubles with this change and almost quadruples without it (they fail on master and pass here). Five pin which date each relative date is relative to (they pass on both). Results are unchanged: a differential run of master against this change, with a frozen clock, over 5,156 strings (26,929 comparisons of parse, get_date_data, search_dates and parse_found_objects) showed no difference. Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1020, fixes #1032