Skip to content

Parse English number words from thirteen to ninety-nine - #1416

Merged
AdrianAtZyte merged 4 commits into
scrapinghub:masterfrom
AdrianAtZyte:english-number-words
Oct 22, 2026
Merged

AdrianAtZyte merged 4 commits into
scrapinghub:masterfrom
AdrianAtZyte:english-number-words

Conversation

@AdrianAtZyte

Copy link
Copy Markdown
Member

Resolves #1236

@codspeed

codspeed Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

βœ… 10 untouched benchmarks


Comparing AdrianAtZyte:english-number-words (edc1e7d) with master (bd73ab7)

Open in CodSpeed

@codecov

codecov Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

Codecov Report

βœ… All modified and coverable lines are covered by tests.
βœ… Project coverage is 99.04%. Comparing base (bd73ab7) to head (edc1e7d).

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #1416   +/-   ##
=======================================
  Coverage   99.03%   99.04%           
=======================================
  Files         239      239           
  Lines        3853     3885   +32     
=======================================
+ Hits         3816     3848   +32     
  Misses         37       37           

β˜” View full report in Codecov by Harness.
πŸ“’ Have feedback on the report? Share it here.

πŸš€ New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@AdrianAtZyte
AdrianAtZyte marked this pull request as ready for review September 23, 2026 11:31

@serhii73 serhii73 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, the relative expressions work well ("in twenty minutes", "twenty one days ago", "forty-five seconds ago", "twenty-five minutes later").

One problem before merging: the new words are turned into digits everywhere, not only before a time unit, so ordinary English text now
behaves as if it had bare numbers in it, and search_dates loses dates it used to find:

from datetime import datetime
import dateparser
from dateparser.search import search_dates

S = {"RELATIVE_BASE": datetime(2020, 6, 15, 12)}

search_dates("She turned thirty on 14 February 2018.", settings=S)
# master: [('on 14 February 2018', datetime(2018, 2, 14, 0, 0))]
# this PR: None

search_dates("She turned thirty on February 14, 2018.", settings=S)
# master: [('on February 14, 2018', datetime(2018, 2, 14, 0, 0))]
# this PR: [('2018', datetime(2018, 6, 15, 0, 0))]

search_dates("About fifty students arrived in 2019 and forty left in 2020.", settings=S)
# master: [('in 2019 and', datetime(2019, 6, 15, 0, 0)), ('in 2020', datetime(2020, 6, 15, 0, 0))]
# this PR: [('in 2020', datetime(2020, 6, 15, 0, 0))]

It also adds false positives: dateparser.parse("twenty") now returns the 20th of the current month, parse("Nineteen Eighty-Four") returns
1984-06-19, and search_dates("Chapter Twenty: The return. It was 5 June 2019.") now also returns ('Twenty: The', <20th of this month>).

Could the new entries only apply when a time unit follows, as the old ... later rule did? For example:

- thirteen(?=\s+(?:decades?|years?|y|months?|mo|weeks?|w|days?|d|hours?|hrs?|h|minutes?|mins?|m|seconds?|secs?|s)\b): '13'
- twenty[ -]([1-9])(?=\s+(?:decades?|years?|y|months?|mo|weeks?|w|days?|d|hours?|hrs?|h|minutes?|mins?|m|seconds?|secs?|s)\b): 2\1
- twenty(?=\s+(?:decades?|years?|y|months?|mo|weeks?|w|days?|d|hours?|hrs?|h|minutes?|mins?|m|seconds?|secs?|s)\b): '20'

With that lookahead on each new entry, all the tests added here still pass, and the examples above behave as they do on master.

@serhii73 serhii73 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review (claude code-review, at de74daf). The full suite passes on this head and I found no regression for inputs that parsed before; the notes below are edge cases in the new feature.

Non-blocking notes:

  • one thirty minutes ago and one twenty days ago become 1 30 minutes ago and 1 20 days ago, and the leading 1 is silently dropped, giving 30 minutes or 20 days ago. one thirty is more likely a clock time, so this is wrong output. The unconditional one..twelve rules run before the tens rules; a guard would fix it.
  • The unit lookahead includes the single letters y d h m s w and the word second, so Twenty second street meeting on 5 June 2019 can produce a bogus Twenty second match in search_dates, and twenty-1 days ago is accepted as 21 days.
  • Only 1-99 are handled: one hundred days ago still returns None. Compound forms accept a single space or hyphen only (twenty one days ago with two spaces fails).
  • The same ~200-character unit alternation is copied into 25 rules in en.py and en.yaml, so adding a unit (like fortnight in #1539) means editing 25 places; one rule with a tens-and-units mapping would be shorter and cheaper to run on every parse.
  • The later rule now only accepts digits and relies on rule order. Tests cover only a few later forms and have no negative cases for the unit-ambiguity examples above.

@AdrianAtZyte
AdrianAtZyte merged commit 6d32a3e into scrapinghub:master Oct 22, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Written Number Parsing Not as Expected

2 participants