Skip to content

Add real async iterator for ByteStreams - #3777

Open
akx wants to merge 1 commit into
encode:masterfrom
akx:trio-bytestream-timeout
Open

akx wants to merge 1 commit into
encode:masterfrom
akx:trio-bytestream-timeout

Conversation

@akx

@akx akx commented Feb 24, 2026

Copy link
Copy Markdown

Summary

Previously, __aiter__ was implemented as a generator function, and as such could get garbage collected when not completely exhausted in a way that has (recent versions of) trio complaining:

$ uv pip list | grep io
[... snip ...]
Package                    Version
anyio                      4.12.1
trio                       0.31.0
[... snip ...]
$ uv run pytest tests/ -k test_write_timeout[trio]
[... snip ...]
E ResourceWarning: Async generator 'httpx._content.ByteStream.__aiter__' was garbage collected before it had been exhausted. Surround its use in 'async with aclosing(...):' to ensure that it gets cleaned up as soon as you're done using it.

The above exception was the direct cause of the following exception:

E pytest.PytestUnraisableExceptionWarning: Exception ignored in: <async_generator object ByteStream.__aiter__ at 0x1075593c0>
[... snip ...]

Using a real, if trivial, object to do this causes less complaints.

Fixes #3686
Closes #3700 (fix for the same issue)

Checklist

  • I understand that this PR may be closed in case there was no previous discussion. (This doesn't apply to typos!)
  • I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
    • Retained pristine test coverage to check that the iterator does indeed adhere to the iterator protocol by returning itself when iterated upon.
  • I've updated the documentation accordingly.
    • No documentation changes required.

Previously, `__aiter__` was implemented as a generator function,
and as such could get garbage collected when not completely exhausted
in a way that has recent versions of `trio` complaining.

Using a real, if trivial, object to do this causes less complaints.

Fixes encode#3686

@Wkhanex Wkhanex left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At 357b42fdefa4, I tested independent iterators over the same ByteStream, including an empty payload and a 65,537-byte payload. Both iterators yielded the original bytes; the first remained exhausted on repeated reads, and starting a fresh iterator replayed the original bytes. Dropping a partially consumed iterator and forcing collection produced no async-generator/resource warning under either asyncio or Trio.

All 55 additional assertions passed, along with the existing content module (43 tests). The empty-payload and two-interleaved-iterator cases are useful regression controls for the None sentinel and per-iterator consumed state. AI-assisted CPython 3.12 validation; this is scoped to byte streams and the content tests.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

test_write_timeout(trio) is failing on python3.14

2 participants