Repository navigation
Added the create_task() method to TaskGroup - #1098
Conversation
That isn't in scope for this PR.
tapetersen
left a comment
There was a problem hiding this comment.
Looks fine to me and while a lot of code I've written has benefitted from rewriting to communicat return values through other means sometimes a single handle for cancelling and getting a return value vs. a cancel-scope and a memory-channel/event+var is really all that's needed and deserves a simple construct.
|
|
||
| if self._exception is not None: | ||
| raise self._exception | ||
|
|
There was a problem hiding this comment.
Since these handles now are "leaked" to user code they can be awaited multiple times. Raising the same exception instance will modify the shared-instance's stack-trace (which is what I understand the reason why Outcome has dissallowed unwrapping more than once and now tries to change the api to copy the exception before raising).
I'm quite sure the futures in vanilla asyncio has the same issue so status quo may very well be acceptable, especially if this is intended to be a straight alternative, and we can assume users already need to be aware.
There was a problem hiding this comment.
This is precisely the code I had in that try...except block which I removed. I'm unsure which way to go here.
There was a problem hiding this comment.
As the current behavior is consistent with asyncio I would say it's completely acceptable to leave it as is.
Copying is arguably more correct but I'm really not sure if it's generally safe and how expensive it is given that the vast majority of cases will definitely be the case of it being awaited exactly once.
Purity wise you could even argue that you shoulnd't throw the exception from the task to an outside waiter at all (what if it's a leaked StopIteration ...) and instead wrap it in a TaskErrored like the aborted/cancelled cases with the caller responsible for unwrapping it but I would definitely say that's too far.
I would probably lean towards the best choice being keeping it as it is now unless the copying costs a lot less than I think even if I myself tend to prefer api's doing it the "correct but slower" way. I'm completely fine with either.
There was a problem hiding this comment.
Exception copying is not generally safe. What's on the table is disallowing users from waiting on the same task handle twice, but I'm not sure if such a thing is warranted in the first place.
There was a problem hiding this comment.
What's on the table is disallowing users from waiting on the same task handle twice, but I'm not sure if such a thing is warranted in the first place.
Would it be reasonable to disallow this for now and reconsider supporting it later if a compelling use-case comes up? If we support it now and want to disallow it later, it would be backwards incompatible.
There was a problem hiding this comment.
I am uneasy about exposing the property TaskHandle.exception. raise task_handle.exception seems like a footgun that is too easy to shoot oneself with; it has the same issues that Tobias brought up in this thread.
What is the motivation for exposing TaskHandle.exception publicly?
There was a problem hiding this comment.
What is the motivation for exposing TaskHandle.exception publicly?
Parity with asyncio.Task.
There was a problem hiding this comment.
There is a huge number of ways people could shoot their foot, and surprisingly many will go the great lengths to find such ways. If I removed the exception property, rest assured some enterprising coder would find the _exception attribute and use that to re-raise the exception.
There was a problem hiding this comment.
While I do agree (somewhat) with your motivation here, it's quite unfortunate that sub-optimal asyncio design decisions that somewhat-worked a decade ago now force us to replicate these misteaks.
On the plus side, we can always replace that exception attribute with a property that issues a deprecation warning and tells people to please call reraise() instead (or whichever other reasonable interface we can think of).
There was a problem hiding this comment.
What mistakes are you seeing here? And no, I'm not going slap a deprecation on the exception attribute. It's useful for introspection and monitoring tools. Otherwise you'll have to await on the handle and get the exception re-raised, and even then you'd have to dig it up from __cause__. Is that what you want?
|
How do you feel about the |
We intentionally don't do that anywhere else in the anyio code base. I'm mostly-OK with supporting a coroutine-based task creation method for more convenient migration (though I'd never use it), but what I'm not really OK with is a |
|
I agree that |
With that problem as well as the context linking when raising I'm actually really starting to get convinced, always wrapping exceptions is the better idea and worth a bit clunkier code when trying to catch and handle specific exceptions (you would need an except and a match/isinstance check). |
I'm already on it :) I'll push some changes Soon (tm). |
|
Done, and I like this design better already. |
|
Feel free to bikeshed on the exception and enum names. Now would be a good time for that. |
|
OK, so where do we stand on the design now? Right now the implementation raises the exceptions directly from the task group, but if the handle is waited on, the exception is wrapped so as not to pollute the original |
I don't think that that would prevent us from implementing "exactly once" exception semantics. Here is a quick-and-dirty sketch of an implementation: cef88f6...b6729cc
An observation: this breaks the symmetry between Overall the new "never move an exception" exception semantics seem good to me. I think that the "can move an exception"/"exactly once" exception semantics would be more flexible and would enable more use cases. However, on balance, the "never-move" semantics are IMO likely the better decision here if the primary objective of this API is to make it easier for Analysis that motivated ^ (click to expand)Different options for
Both of these options prevent the user from accidentally suppressing an exception. The two different options will place different restrictions on the use cases for
So, my preference here is somewhat mixed:
Some questions for you all:
|
| case TaskHandle.Status.CANCELLED: | ||
| raise TaskCancelled("the task was cancelled") | ||
| case TaskHandle.Status.ERRORED: | ||
| raise TaskError("the task raised an exception") from self._exception |
There was a problem hiding this comment.
This from doesn't seem to be working:
from __future__ import annotations
import traceback
import anyio
async def raiser():
raise RuntimeError("boom")
async def main() -> None:
async with anyio.create_task_group() as tg:
handle = tg.create_task(raiser())
with anyio.CancelScope(shield=True):
try:
await handle
except BaseException as exc:
print("".join(traceback.format_exception(exc)).rstrip())
print()
raise
anyio.run(main)gives (CPython 3.13.12)
Traceback (most recent call last):
File "/home/ganden/vc/anyio/src/anyio/_core/_tasks.py", line 262, in _run_coro
retval = await self._coro
^^^^^^^^^^^^^^^^
File "/home/ganden/vc/anyio/eg.py", line 9, in raiser
raise RuntimeError("boom")
RuntimeError: boom
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/ganden/vc/anyio/eg.py", line 17, in main
await handle
File "/home/ganden/vc/anyio/src/anyio/_core/_tasks.py", line 321, in __await__
return self.return_value
^^^^^^^^^^^^^^^^^
File "/home/ganden/vc/anyio/src/anyio/_core/_tasks.py", line 315, in return_value
raise TaskError("the task raised an exception") from self._exception
anyio.TaskError: the task raised an exception
+ Exception Group Traceback (most recent call last):
| File "/home/ganden/vc/anyio/eg.py", line 24, in <module>
| anyio.run(main)
| ~~~~~~~~~^^^^^^
| File "/home/ganden/vc/anyio/src/anyio/_core/_eventloop.py", line 77, in run
| return async_backend.run(func, args, {}, backend_options)
| ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/home/ganden/vc/anyio/src/anyio/_backends/_asyncio.py", line 2358, in run
| return runner.run(wrapper())
| ~~~~~~~~~~^^^^^^^^^^^
| File "/home/ganden/.pyenv/versions/3.13/lib/python3.13/asyncio/runners.py", line 118, in run
| return self._loop.run_until_complete(task)
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
| File "/home/ganden/.pyenv/versions/3.13/lib/python3.13/asyncio/base_events.py", line 725, in run_until_complete
| return future.result()
| ~~~~~~~~~~~~~^^
| File "/home/ganden/vc/anyio/src/anyio/_backends/_asyncio.py", line 2341, in wrapper
| return await func(*args)
| ^^^^^^^^^^^^^^^^^
| File "/home/ganden/vc/anyio/eg.py", line 13, in main
| async with anyio.create_task_group() as tg:
| ~~~~~~~~~~~~~~~~~~~~~~~^^
| File "/home/ganden/vc/anyio/src/anyio/_backends/_asyncio.py", line 799, in __aexit__
| raise BaseExceptionGroup(
| "unhandled errors in a TaskGroup", self._exceptions
| ) from None
| ExceptionGroup: unhandled errors in a TaskGroup (2 sub-exceptions)
+-+---------------- 1 ----------------
| Traceback (most recent call last):
| File "/home/ganden/vc/anyio/eg.py", line 17, in main
| await handle
| File "/home/ganden/vc/anyio/src/anyio/_core/_tasks.py", line 321, in __await__
| return self.return_value
| ^^^^^^^^^^^^^^^^^
| File "/home/ganden/vc/anyio/src/anyio/_core/_tasks.py", line 315, in return_value
| raise TaskError("the task raised an exception") from self._exception
| anyio.TaskError: the task raised an exception
+---------------- 2 ----------------
| Traceback (most recent call last):
| File "/home/ganden/vc/anyio/src/anyio/_core/_tasks.py", line 262, in _run_coro
| retval = await self._coro
| ^^^^^^^^^^^^^^^^
| File "/home/ganden/vc/anyio/eg.py", line 9, in raiser
| raise RuntimeError("boom")
| RuntimeError: boom
+------------------------------------
The __context__ that we are setting with this from is missing from the exception group rendering.
There was a problem hiding this comment.
It works on CPython 3.11 and 3.12, but not 3.10, 3.13, and 3.14. Possibly a CPython regression? I will take a closer look tomorrow.
There was a problem hiding this comment.
First, I'd like to point out that from doesn't set __context__, it sets __cause__.
This has everything to do with how exception groups are formatted. Consider this:
try:
raise RuntimeError("boom")
except RuntimeError as exc:
try:
raise RuntimeError("another boom") from exc
except RuntimeError as exc2:
excgrp = ExceptionGroup("group", (exc, exc2))
print("".join(traceback.format_exception(excgrp)).rstrip())
print("".join(traceback.format_exception(excgrp.exceptions[1])).rstrip())This results in:
| exceptiongroup.ExceptionGroup: group (2 sub-exceptions)
+-+---------------- 1 ----------------
| Traceback (most recent call last):
| File "/home/alex/workspace/anyio/bug-trio.py", line 30, in <module>
| raise RuntimeError("boom")
| RuntimeError: boom
+---------------- 2 ----------------
| Traceback (most recent call last):
| File "/home/alex/workspace/anyio/bug-trio.py", line 33, in <module>
| raise RuntimeError("another boom") from exc
| RuntimeError: another boom
+------------------------------------
Traceback (most recent call last):
File "/home/alex/workspace/anyio/bug-trio.py", line 30, in <module>
raise RuntimeError("boom")
RuntimeError: boom
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/alex/workspace/anyio/bug-trio.py", line 33, in <module>
raise RuntimeError("another boom") from exc
RuntimeError: another boom
The __cause__ is indeed preserved but not rendered in the exception group's traceback when the __cause__ is rendered in the traceback.
If we leave out the __cause__ (RuntimeError("boom")) from the exception group, you'll find that the exception is indeed rendered as the cause of the second one:
try:
raise RuntimeError("boom")
except RuntimeError as exc:
try:
raise RuntimeError("another boom") from exc
except RuntimeError as exc2:
excgrp = ExceptionGroup("group", (exc2,))
print("".join(traceback.format_exception(excgrp)).rstrip())Results in:
| exceptiongroup.ExceptionGroup: group (1 sub-exception)
+-+---------------- 1 ----------------
| Traceback (most recent call last):
| File "/home/alex/workspace/anyio/bug-trio.py", line 30, in <module>
| raise RuntimeError("boom")
| RuntimeError: boom
|
| The above exception was the direct cause of the following exception:
|
| Traceback (most recent call last):
| File "/home/alex/workspace/anyio/bug-trio.py", line 33, in <module>
| raise RuntimeError("another boom") from exc
| RuntimeError: another boom
+------------------------------------
To summarize, this behavior has nothing whatsoever to do with the task handle's exception handling logic.
|
But: |
We don't support adding callbacks, and |
|
Sure, I'm just saying asyncio uses "done", but it's fine. |
>>> asyncio.run(main())
<Task finished name='Task-2' coro=<sleep() done, defined at /usr/lib64/python3.10/asyncio/tasks.py:593> result=None>This is where I originally got the |
|
I'd like to get at least two accepting reviews to ensure some consensus. There's one already, so one more is needed. |
m-u-xyz
left a comment
There was a problem hiding this comment.
Well … I'm +.5 on this, because I really dislike asyncio's calling conventions.
But I suppose we can (should, IMHO) add a start_task method (or whatever) later.
WDYM? We already have a |
Yeah, that was my "or whatever" part. Sorry if that was too terse / didn't consider past discussion; I'm a bit stressed out at the moment. 😔 |
|
Thanks, now I can continue to the next steps. |
…ate_task (Future) start_soon is now fire-and-forget; create_task returns the observation Future. Aligns with asyncio.TaskGroup and the AnyIO direction in agronholm/anyio#1098. start_soon is a one-line wrapper over create_task, so the two share one implementation. Also swap the eager async-callable check from inspect.iscoroutinefunction to localpost._utils.is_async_callable, which unwraps functools.partial and detects callable instances with async __call__. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
NOTE Erasing or replacing the contents of this template will result in your pull
request being summarily closed without consideration!
Changes
This adds a mostly asyncio-compatible
create_task()method to theTaskGroupclass.Like its asyncio counterpart, this method accepts a coroutine object and returns an awaitable task handle.
Checklist
If this is a user-facing code change, like a bugfix or a new feature, please ensure that
you've fulfilled the following conditions (where applicable):
tests/) which would fail without your patchdocs/), in case of behavior changes or newfeatures
docs/versionhistory.rst).If this is a trivial change, like a typo fix or a code reformatting, then you can ignore
these instructions.
Updating the changelog
If there are no entries after the last release, use
**UNRELEASED**as the version.If, say, your patch fixes issue #123, the entry should look like this:
If there's no issue linked, just link to your pull request instead by updating the
changelog after you've created the PR.