Skip to content

Allow md5hash to function in fips environments - #1267

Merged
Gallaecio merged 7 commits into
scrapinghub:masterfrom
trevor-scale:trevor/fix_md5_fips
Feb 4, 2026
Merged

Gallaecio merged 7 commits into
scrapinghub:masterfrom
trevor-scale:trevor/fix_md5_fips

Conversation

@trevor-scale

@trevor-scale trevor-scale commented Apr 15, 2025 •

Copy link
Copy Markdown
Contributor

Many secure environments disable the hashlib.md5 function because it is insecure, this causes the line below to fail making dateparser unusable in these environments:

return hashlib.md5("".join(keys).encode("utf-8")).hexdigest()

This is remedied by replacing that line with return hashlib.new("md5", "".join(keys).encode("utf-8"), usedforsecurity=False).hexdigest() which appropriately bypasses the fips security check because dateparser is not using the md5 hash for security applications.

Corresponding issue: #1258

@ghmj2417

Copy link
Copy Markdown

I posted this in #1258 as well. Change usedforsecurity=True to usedforsecurity=False. The suggestion/example provided in issue 1258 was to use hashlib.new but it also included the full command which had usedforsecurity=True to test a failure when using FIPS compliant encryptions/hashes/etc. My apologies for not being clear.

@trevor-scale

Copy link
Copy Markdown
Contributor Author

I posted this in #1258 as well. Change usedforsecurity=True to usedforsecurity=False. The suggestion/example provided in issue 1258 was to use hashlib.new but it also included the full command which had usedforsecurity=True to test a failure when using FIPS compliant encryptions/hashes/etc. My apologies for not being clear.

Thanks for catching this wasn't looking closely when I copy-pasted haha. Will update now!

@trevor-scale

Copy link
Copy Markdown
Contributor Author

I posted this in #1258 as well. Change usedforsecurity=True to usedforsecurity=False. The suggestion/example provided in issue 1258 was to use hashlib.new but it also included the full command which had usedforsecurity=True to test a failure when using FIPS compliant encryptions/hashes/etc. My apologies for not being clear.

Thanks for catching this wasn't looking closely when I copy-pasted haha. Will update now!

@ghmj2417 are you able to approve and merge this, would love to get this added in!

@trevor-scale

Copy link
Copy Markdown
Contributor Author

@Gallaecio can you give this a review and lmk if any changes are needed when you have the chance?

Comment thread dateparser/conf.py Resolved
Co-authored-by: AdriΓ‘n Chaves <adrian@chaves.gal>
@codecov

codecov Bot commented Jun 30, 2025 •

Copy link
Copy Markdown

Codecov Report

βœ… All modified and coverable lines are covered by tests.
βœ… Project coverage is 96.62%. Comparing base (c2ff31c) to head (f0b2675).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #1267   +/-   ##
=======================================
  Coverage   96.62%   96.62%           
=======================================
  Files         235      235           
  Lines        2905     2905           
=======================================
  Hits         2807     2807           
  Misses         98       98           

β˜” View full report in Codecov by Sentry.
πŸ“’ Have feedback on the report? Share it here.

πŸš€ New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Gallaecio

Copy link
Copy Markdown
Contributor

Please, install pre-commit, run pre-commit install, then pre-commit run --all-files, keep running that command until everything is green (I think in this case a single run should suffice, but seeing everything green is nice πŸ™‚), and commit those changes.

@ghmj2417

Copy link
Copy Markdown

hashlib.new was to accommodate Python 3.8. If setup.py is going to require >= 3.9 then hashlib.md5 is fine.

@trevor-scale

Copy link
Copy Markdown
Contributor Author

Finally got the linting done! @Gallaecio are you ok to merge now?

@trevor-scale

Copy link
Copy Markdown
Contributor Author

ghmj2417

/ @ghmj2417

@yannsartori

yannsartori commented Jan 12, 2026 •

Copy link
Copy Markdown

I would be interested in this change as well, right now my workaround is to use unittest.mock.patch

import hashlib
from unittest.mock import patch

import dateparser

settings = ...

@classmethod
def patched_get_key(cls, settings=None):
    if not settings:
        return "default"
    keys = sorted([f"{key}-{settings[key]}" for key in settings])
    return hashlib.md5("".join(keys).encode("utf-8"), usedforsecurity=False).hexdigest()

with patch("dateparser.conf.Settings.get_key", patched_get_key):
    dt = dateparser.parse(s, settings=settings)

@Gallaecio
Gallaecio merged commit 24dac43 into scrapinghub:master Feb 4, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants