[serhii73]

Fixes #706.

>>> search_dates('через 2 часа, пожалуйста', languages=['ru'], settings={'RELATIVE_BASE': datetime(2020, 2, 13, 20, 7, 6)})
[('через 2 часа', datetime(2020, 2, 13, 18, 7, 6))]   # master — two hours *before* the base
[('через 2 часа', datetime(2020, 2, 13, 22, 7, 6))]   # this branch

What was left of the issue

The example as it was reported, 'через 2 часа' on its own, already works on master, and I
reproduced the original report against 0.7.6 to be sure it was read right:

# dateparser 0.7.6, RELATIVE_BASE=datetime(2020, 6, 11, 10, 58, 34)
[('2 часа', datetime(2020, 6, 11, 8, 58, 34))]     # wrong span *and* wrong direction

The two halves of the report were one bug. 'через' was simply missing from the ru search
dictionary, so translate_search hit it as an unknown word and flushed the chunk there. That
dropped 'через' from the original chunk — the wrong span — and with it the "in" from the
translation, which is what left '2 hour' to be read as a past date. A later data update added
через → in and fixed both at once. The reporter's guess about two competing candidate matches
is not what happens: there is one (translated, original) chunk pair, aligned token by token.

Half of it is still live, though, in a narrower form: put any punctuation after the unit and the
direction flips again, in every language, English included.

Why punctuation flipped the direction

"через 2 часа" is translated into "in 2 hour", and the "in" is what tells the freshness
parser to count forward. An "in" that came from a preposition ("in January") has to be dropped
instead, and _clear_future_words tells the two apart by looking for a unit of time among the
words next to it.

A unit keeps the punctuation it was written next to, so "через 2 часа, пожалуйста" translates
into "in 2 hour," and the unit is looked up as "hour,":

{"in", "2", "hour,"}.isdisjoint({"day", "week", "month", "year", "hour", "minute", "second"})
# True — no unit found, so the "in" is dropped as a preposition

What is left, "2 hour,", parses as two hours ago.

The fix

Look the unit up without the punctuation it carries. The set is the one translate_search
strips before the dictionary lookup and re-attaches to the translation, and the fix only holds
while the two agree, so it is now a single module-level PUNCTUATION used by both. Falsy tokens
are skipped because the list holds the empty strings that removed skip words leave behind.

This only ever makes the function keep an "in" it used to drop, and only when a time unit is
actually there, so a preposition with no unit next to it is still dropped exactly as before.

Evidence

24251 passed, 16 skipped, 1 xfailed; ruff and ruff-format clean.

Differential run against master over 14,326 inputs — a future and a past phrase per unit per
locale, generated from each locale's own relative-type-regex (1,659 phrases, 121 languages) in
six punctuation variants each, plus every string mined from tests/ run with language detection:

identical	14,014
changed	312
lost (branch returns None)	0
gained	0
new exceptions	0

Every one of the 312 is the bug: a future phrase that master read as past (307 generated
across 16 languages, 5 mined). Each now returns exactly what the same phrase without the
punctuation returns on both master and this branch — the punctuated form stops disagreeing with
the bare one. No past phrase changed.

One of the mined five is worth calling out, because it is not a comma and not Russian:

>>> search_dates("in three weeks' time", settings={'RELATIVE_BASE': datetime(2020, 2, 13, 20, 7, 6)})
[('in three weeks', datetime(2020, 1, 23, 20, 7, 6))]   # master — three weeks *ago*
[('in three weeks', datetime(2020, 3, 5, 20, 7, 6))]    # this branch

tests/test_freshness_date_parser.py:1769 already asserts that parse reads that same string as
three weeks in the future, so search_dates was contradicting parse on it. It is now one of the
new test cases.

Tests

One new test method, test_search_dates_with_a_relative_expression_reads_its_direction, eight
params: через 2 часа bare, with a trailing comma, and inside a sentence; через 3 дня,; both
2 часа назад forms, so the past direction is held in place; the English in 2 hours, please,
since the bug is not language-specific; and in three weeks' time for punctuation that is not a
comma. Reverting locale.py alone fails five of the eight and leaves the other three passing.

No existing test was touched.

Deliberately not fixed

Three neighbours of this bug came up in review. Each behaves identically on master, and none
goes through the "in" decision this PR changes:

• Leading punctuation is re-attached as trailing. translate_search computes the carried
  punctuation as word[len(bare_word):], so "(in" becomes "inn" and search_dates("(in 2 hours)") returns garbage — with or without this PR. A separate bug, one line, worth its own
  issue.
• Typographic quotes and : (’ “ ” « ») are not in the strip set, so in three weeks’ time
  and «через 2 часа» return None before and after; the chunk breaks at the word and never
  reaches this code. parse handles them.
• Two "in"s in one chunk. Only the first is ever examined, so in January in 2 hours, now
  returns what in January in 2 hours already returned on master (a split into January and
  hours). The comma form stops disagreeing with the bare form; the bare form's answer is a
  pre-existing problem.

The "gained 0 / lost 0" above is over the corpus described, which does not contain the third
shape.

🤖 Generated with Claude Code

[codspeed]

Merging this PR will not alter performance

✅ 7 untouched benchmarks

---

_{Comparing fix/issue-706-ru-search-dates-direction (fbc1fb5) with master (5e77cac)}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.21%. Comparing base (5e77cac) to head (fbc1fb5).

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #1391   +/-   ##
=======================================
  Coverage   97.20%   97.21%           
=======================================
  Files         236      236           
  Lines        3081     3084    +3     
=======================================
+ Hits         2995     2998    +3     
  Misses         86       86           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.