Repository navigation
[httpx migration] Import httpx from huggingface_hub, require huggingface_hub>=1.31.0 - #8598
Merged
Merged
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
Contributor
Author
|
Failing CI seems unrelated |
Wauplin
marked this pull request as ready for review
September 10, 2026 15:57
Member
|
cool ! I guess we need to bump the minimal version from 0.25.0 to 1.30.0 (both in setup.py and in the "minimum version" CI) |
`huggingface_hub.utils.httpx` only exists in huggingface_hub>=1.31.0, so the
re-export used to stay forward-compatible with httpx2 requires bumping the
minimum version anyway. Bump it to 1.31.0 (setup.py + minimum-versions CI) and
remove everything that was only there to keep working with older releases:
- `config.HF_HUB_VERSION >= 1.6.0` gates around `BucketNotFoundError` and the
`HfFileSystemResolved{Bucket,Repository}Path` classes, plus the
`raise ImportError("... requires huggingface_hub>=1.6.0")` guards
- `except EntryNotFoundError` around `dirfs.glob`, needed for hfh<=1.7.1
- `requests` exceptions caught next to their `httpx` counterparts, the
`requests` dependency itself, and the direct `httpx` pin (it now comes from
huggingface_hub, so datasets no longer constrains the httpx major)
- `proxies=` passed to `hf_hub_download`, silently ignored since hfh 1.0
- `IS_HF_HUB_1_x`, `require_buckets_support_in_huggingface_hub` and the 0.x
branches in the test helpers
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributor
Author
|
@lhoestq good idea. PR now removes all dead code that was needed for legacy versions of Failing CI all come from HTTP 500 from staging (tests are flaky on hfh as well nowadays ðŽ) |
16 of 18 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Direct httpx imports will break when huggingface_hub migrates to httpx2. This updates them to use the compatibility re-export introduced in huggingface/huggingface_hub#4803, following the migration plan in huggingface/huggingface_hub#4802. Goal is to make datasets compatible with both huggingface_hub 1.x and 2.x when the switch to httpx2 happens.
Following #8598 (comment), this also bumps the minimum
huggingface_hubversion and removes the code that was only there to keep working with older releases.Minimum version
Bumped to
1.31.0(insetup.pyand in the "minimum versions" CI job) rather than1.30.0:huggingface_hub.utils.httpxlanded in 1.31.0, not in 1.30.0 as the migration plan anticipated.requestsandhttpxare also gone from the dependencies.requestswas only needed for huggingface_hub 0.x, and pinninghttpx<1.0.0here would defeat the purpose of the re-export â datasets should not constrain the httpx major version, huggingface_hub owns that now.Legacy code removed
config.HF_HUB_VERSION >= 1.6.0gates inarrow_dataset.py,dataset_dict.py,iterable_dataset.py,load.py,tests/fixtures/hub.pyandtests/test_iterable_dataset.py, with theBucketNotFoundError = None/HfFileSystemResolvedRepositoryPath = HfFileSystemResolvedPathfallbacks and the matchingraise ImportError("... requires huggingface_hub>=1.6.0")guardsexcept EntryNotFoundErrorarounddirfs.glob(...), marked# needed for huggingface_hub<=1.7.1requestsexceptions caught next to theirhttpxcounterparts inload.pyand inCONNECTION_ERRORS_TO_RETRYproxies=download_config.proxiespassed tohf_hub_download, which huggingface_hub has silently ignored since 1.0.DownloadConfig.proxiesis kept for backward compatibility, with a docstring pointing atHTTP_PROXY/set_client_factoryIS_HF_HUB_1_xand its two branches inoffline(),require_buckets_support_in_huggingface_hub(buckets are always available now), therequestshalf ofxfail_if_500_502_http_error, and thehuggingface_hub.file_download.HUGGINGFACE_CO_URL_TEMPLATEmonkeypatch fallbackOnce huggingface_hub 2.0 is out, the last step of the plan is to widen the pin to
<3.0.ðĪ Generated with Claude Code