Skip to content

Stop caching a Settings instance per RELATIVE_BASE value - #1442

Merged
AdrianAtZyte merged 6 commits into
scrapinghub:masterfrom
AdrianAtZyte:relative-base-leak
Oct 27, 2026
Merged

AdrianAtZyte merged 6 commits into
scrapinghub:masterfrom
AdrianAtZyte:relative-base-leak

Conversation

@AdrianAtZyte

Copy link
Copy Markdown
Member

Resolves #985

@codspeed

codspeed Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

βœ… 10 untouched benchmarks


Comparing AdrianAtZyte:relative-base-leak (1fec2ee) with master (699855a)

Open in CodSpeed

@codecov

codecov Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

Codecov Report

βœ… All modified and coverable lines are covered by tests.
βœ… Project coverage is 99.18%. Comparing base (3b43c90) to head (1fec2ee).
⚠️ Report is 2 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1442      +/-   ##
==========================================
- Coverage   99.18%   99.18%   -0.01%     
==========================================
  Files         239      239              
  Lines        4052     4026      -26     
==========================================
- Hits         4019     3993      -26     
  Misses         33       33              

β˜” View full report in Codecov by Harness.
πŸ“’ Have feedback on the report? Share it here.

πŸš€ New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@AdrianAtZyte
AdrianAtZyte marked this pull request as ready for review September 24, 2026 13:14

@serhii73 serhii73 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review (claude code-review, at 0d7c03a). No correctness bug found. The affected suites pass (1137 tests), and applying the PR onto current master and running the full tests directory gives no failures. Settings that differ only in RELATIVE_BASE now share one key, so the dictionary caches stop growing per call, and parse and search timings are unchanged against the PR's merge-base.

Non-blocking notes:

  • dateparser.utils.registry is removed along with its test. Third-party code importing it will fail with ImportError, so a deprecation shim or a changelog note would keep this non-breaking.
  • The key hashes every setting except RELATIVE_BASE and _mod_settings, so another per-call setting can still grow the caches, and the same language words are rebuilt per distinct TIMEZONE. An allowlist of the settings Dictionary actually reads would fix that.
  • registry_key and get_key keep their registry names; cache_key would be clearer. An all-excluded settings object gets the md5 of the empty string rather than "default".
  • The regression test only checks _split_regex_cache; four other caches use the same key. The test that identical settings give equal Settings was deleted and nothing replaces it, and every replace() (including one per date in search_dates) now builds a fresh Settings.

@AdrianAtZyte
AdrianAtZyte merged commit b4124e0 into scrapinghub:master Oct 27, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RELATIVE_BASE memory leak

2 participants