Repository navigation
Remove fastText support and enable numpy 2.x compatibility - #1315
Conversation
FastText library has been archived since March 2024 and is no longer maintained. This dependency was blocking numpy 2.x adoption. Changes: - Remove fasttext extra from setup.py dependencies - Add deprecation warnings to fasttext.py module - Update documentation to recommend langdetect as primary option - Update CLI to deprecate --fasttext flag - Add comprehensive test suite for langdetect integration - Remove fasttext from tox.ini test configurations Users should migrate to langdetect: pip install dateparser[langdetect] Closes #1287 Closes #1251
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1315 +/- ##
==========================================
- Coverage 96.63% 96.50% -0.14%
==========================================
Files 235 235
Lines 2915 2887 -28
==========================================
- Hits 2817 2786 -31
- Misses 98 101 +3 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
This PR removes the deprecated fastText library dependency to enable numpy 2.x compatibility. The fastText library has been archived since March 2024 and is no longer maintained, creating a blocker for adopting modern versions of numpy and other libraries that require numpy 2.x.
Changes:
- Removed fastText extra dependency from setup.py (numpy 1.x constraint removed)
- Added deprecation warnings and error handling to fasttext.py module
- Updated CLI to deprecate --fasttext flag and show migration guidance
- Refactored test_language_detect.py to focus on langdetect with comprehensive test coverage
- Cleaned up tox.ini to remove fasttext and numpy version pins from test configurations
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| setup.py | Removed fasttext extra dependency including numpy 1.x constraint |
| tox.ini | Removed fasttext from test extras and numpy 1.x version pins from min dependencies |
| tests/test_language_detect.py | Completely refactored to test langdetect instead of fasttext, with expanded test coverage including basic functionality, integration tests, and mock tests |
| dateparser_cli/cli.py | Deprecated --fasttext flag with helpful error messages directing users to langdetect |
| dateparser/custom_language_detection/fasttext.py | Added deprecation warnings and ImportError when fasttext is not available |
| docs/custom_language_detection.rst | Updated documentation to remove fasttext examples and add deprecation notice, promoting langdetect as the primary option |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| try: | ||
| import fasttext | ||
| except ImportError: | ||
| raise ImportError( | ||
| "fastText is no longer included as a dependency. " | ||
| "The library is archived and unmaintained. " | ||
| "Please use langdetect instead: pip install dateparser[langdetect]" | ||
| ) |
There was a problem hiding this comment.
The deprecation approach has a logical issue. The module shows a DeprecationWarning at import time (lines 8-14), then tries to import fasttext (line 17). If fasttext is not installed, an ImportError is raised (lines 18-23) that tells users to migrate to langdetect. However, the rest of the module code (lines 25-59) still exists and will attempt to use fasttext if it was somehow available. This creates confusion: if a user has fasttext installed, they'll see a deprecation warning but the code will still work. If they don't have it installed, they'll get an ImportError. A clearer approach would be to always raise an error or to remove the functional code entirely, leaving only the deprecation message.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
The previous approach was confusing: - If fasttext was installed, it showed a warning but still worked - If fasttext wasn't installed, it raised an ImportError This creates inconsistent deprecation - users don't know if the feature is truly deprecated or not. Now the module: - Always shows deprecation warning on import - Always raises ImportError when detect_languages() is called - Provides clear migration path to langdetect - No functional fasttext code remains This makes the deprecation clear and consistent.
The module-level 'detect_languages' mock was defined but never used. The MockLangDetectTest class defines its own mock as a class attribute, and other test classes use the actual lang_detect_detect_languages function. This cleanup removes the unused mock to avoid confusion.
serhii73
changed the title
- Resolve conflicts: keep slimmed setup.py (long_description only) - Drop fasttext extra (removed upstream in scrapinghub#1315) - Define version statically as project.version (1.4.0), keep dateparser.__version__ for runtime - Remove dynamic version attr; configure bump-my-version to update pyproject.toml and dateparser/__init__.py
FastText library has been archived since March 2024 and is no longer maintained.
Closes #1287
Closes #1251