Cache httpx AsyncClient per event loop to avoid cross-loop reuse crashes - #13410
Merged
cdrini merged 3 commits intoAug 24, 2026
Merged
Conversation
github-project-automation
Bot
moved this to Waiting Review/Merge from Staff
in Ray's Project
cdrini
force-pushed
the
async-client-per-event-loop
branch
3 times, most recently
from
August 24, 2026 19:18
7066670 to
e793962
Compare
cdrini
added
the
Patch Deployed
RayBB
approved these changes
Aug 24, 2026
RayBB
left a comment
RayBB
left a comment
Collaborator
There was a problem hiding this comment.
Looks great and works locally.
My main feedback is I don't think we should use subpreocess to grep. I suggested a fix below.
cdrini
force-pushed
the
async-client-per-event-loop
branch
from
August 24, 2026 23:43
e793962 to
f07a8f1
Compare
Co-Authored-By: Drini Cami <cdrini@gmail.com>
…sions Co-Authored-By: Drini Cami <cdrini@gmail.com>
Co-Authored-By: RayBB <RayBB@users.noreply.github.com> Co-Authored-By: Drini Cami <cdrini@gmail.com>
cdrini
force-pushed
the
async-client-per-event-loop
branch
from
August 24, 2026 23:46
f07a8f1 to
07590ff
Compare
github-project-automation
Bot
moved this from Waiting Review/Merge from Staff
to Done
in Ray's Project
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
We regularly get production
RuntimeError: <asyncio.locks.Event object at 0x7f5816726ed0 [unset]> is bound to a different event looperrors while we do our migration to fastapi when the syncified version of an async method is called from fastapi. These should be removed/remedied, but it's not always possible to migrate everything in one fell swoop. This fix ensures that httpx.AsyncClient never crosses event loops so that these errors never happen, and we can migrate without having sporadic surprise errors on production.A single shared httpx.AsyncClient used from both AsyncBridge's persistent background-thread event loop and a caller's own loop (e.g. FastAPI's) occasionally raised "RuntimeError: ... is bound to a different event loop", since httpx/httpcore/anyio lazily bind pooled-connection internals to whichever loop first contends for them. cache_per_event_loop() keeps one client per loop instead, preserving connection pooling/keep-alive/DNS caching within a loop while keeping different loops isolated.
Technical
Testing
Screenshot
Stakeholders