Excellent, I'll have a look, thanks for the response.
On Tue, Nov 26, 2019 at 3:14 PM Yury Selivanov <yseliva...@gmail.com> wrote: > Hi Justin, > > Yes, we want to have TaskGroups in asyncio, and the only reason we didn't > add them to Python 3.8 is that we also need to support ExceptionGroups (or > MultiErrors). Nathaniel and I plan to start working on that. > > Here's am implementation of TaskGroups that I created for EdgeDB > https://github.com/edgedb/edgedb/blob/master/edb/common/taskgroup.py — > you can use it in the meantime, it works just fine. > > Yury > > > On Tue, Nov 26, 2019 at 3:45 PM, Justin Mayfield <too...@gmail.com> wrote: > >> I'm curious if any work has happened towards supporting a grouping >> concept like Yury has discussed with TaskGroup? I've been away from >> python for a while and I'm finding that the use of gather and wait is >> trickier than I remember it. My code tends to be inelegant when done as >> the docs recommend (using create_task first, etc). I think I benefit from >> a class based context manager primitive like TaskGroup. >> >> Side note, I find I'm missing Javascript's Promise.race() quite a bit too >> (as_completed is often an aesthetic mismatch for my code), so it would be >> neat if the TaskGroup behaved like a set of outstanding work with methods >> like `race()` and `wait()`. >> >> Somewhat related, I recently watched Nathaniel's talk on the happy >> eyeballs algo (https://www.youtube.com/watch?v=oLkfnc_UMcE) and I >> enjoyed the use of this very easy to understand algo as the test basis for >> concurrency designs. However when he described the problem my first >> instinct was that he would the write algo in a top-down style where the >> outer blocks handled scheduling and the inner tasks simply tried to do work >> or failed. It seemed to me the use of events objects, scheduling the next >> task from within an existing task (with conditions) and handling cleanup >> manually via cancel calls was less than perfect and I dare say not >> dissimilar to the reviled GOTO. Could this algo be written in even simpler >> terms? Namely, without calls the any sort of cancel, without the tasks >> themselves knowing anything about the outside scheduling or work set, and >> perhaps most importantly without the use of Event objects which are >> champions of the preemptive world IMO. >> >> Please permit me to indulge in some hypothetical code I would like to be >> able to write someday to perhaps highlight my wishlist for asyncio... >> >> async def might_fail_or_be_slow(): >> await asyncio.sleep(random.random() * 10) >> if random.random() > 0.5: >> raise Exception("job failed") >> return "job worked" >> >> >> async def happy_eyeballs(max_job_wait=0.200): >> async with asyncio.group() as ag: >> for i in range(10): >> ag.add_task(might_fail_or_be_slow()) >> try: >> task = await ag.race(timeout=max_job_wait) >> except asyncio.TimeoutError: >> print("Nothing is ready yet, add another..") >> else: >> try: >> return await task >> except Exception: >> print("Somebody failed, add another..") >> raise Exception("nobody worked") >> >> >> Some notes. This is obviously just pseudo code and I've not thought this >> through very carefully so it's obviously rough and incomplete. For one, >> I'm making the assumption here that the TaskGroup will be somewhat queue >> like. E.g. if you call `race()` the result is removed from the group; >> This might be silly or need to be more explicit, or maybe there could be a >> multitude of grouping classes for each use-case TaskQueue, TaskGroup, >> TaskRace, etc (spitballing). LIkewise it might be useful for the above >> TaskGroup class to have `remove_task(task)` , `wait(timeout=None) -> >> [tasks...]`, and perhaps even something like `results(timeout) -> [results >> from tasks...]` which is a shortcut method for `results = [await t for t in >> await group.wait()]`. >> >> Cheers. >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "python-tulip" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to python-tulip+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/python-tulip/54f410f6-b9b6-4af3-be29-c2e133fb7fe7%40googlegroups.com >> <https://groups.google.com/d/msgid/python-tulip/54f410f6-b9b6-4af3-be29-c2e133fb7fe7%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > > -- > You received this message because you are subscribed to a topic in the > Google Groups "python-tulip" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/python-tulip/dsv_PDcqvsw/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > python-tulip+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/python-tulip/k3gf1sm6.3a8e58ab-eb46-4304-95b2-69f5e3418733%40we.are.superhuman.com > <https://groups.google.com/d/msgid/python-tulip/k3gf1sm6.3a8e58ab-eb46-4304-95b2-69f5e3418733%40we.are.superhuman.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "python-tulip" group. To unsubscribe from this group and stop receiving emails from it, send an email to python-tulip+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/python-tulip/CAHxQA0HT823kbeo8gjh2L4Dfh44BWF%2BVO3JwHi_jVT-w3CBEVg%40mail.gmail.com.
