I honestly have no idea what's being measured, but here are some numbers to compare this to, and then some explanation about async I/O in general.
1. No I/O to a local disk on a modern controller should take milliseconds. The time you are aiming for is below millisecond. That is, writing a block to disk should take less than a millisecond. 2. Linux and some other operating systems have an ability to work asynchronously with network I/O through calls from the epoll family (the story of these system calls is ironic due to how many iterations of failures resulted in what we have today, and then was abandoned in favor of io_uring anyways). I/O to disk, unless you are using io_uring is never asynchronous. Using threads may help *other* code to run asynchronously, but I/O inside threads is still blocking. I have no idea what stats = await asyncio.to_thread(os.stat, url) may be doing. But, looking at these stats: https://en.wikipedia.org/wiki/IOPS, 2 seconds should be enough, on a modern server to write... a lot of data. Let's be modest and assume you are using an Intel SSD with 5K IOPS, then you should be able to write 320 MiB in this time, unless I miscounted the zeroes :D In general though... I think that the subject of threads and I/O are orthogonal and trying to come up with any sort of metric to measure both at the same time somehow... is just not going to give you any sensible description of the situation. So... maybe rethink the problem? Maybe eliminate threads from the equation? Also... I have no idea why Python needs async/await. It's a very confusing and unwieldy interface to epoll. I never found a practical reason to use this, unless in the situation where someone else used this in their library, and I had to use the library. All in all, it's better to pretend this part of Python doesn't exist. It's pointless and poorly written on top of that. On Mon, Jun 23, 2025 at 9:42 PM Mild Shock <janbu...@fastmail.fm> wrote: > > Hi, > > async I/O in Python is extremly disappointing > and an annoying bottleneck. > > The problem is async I/O via threads is currently > extremly slow. I use a custom async I/O file property > predicate. It doesn't need to be async for file > > system access. But by some historical circumstances > I made it async since the same file property routine > might also do a http HEAD request. But what I was > > testing and comparing was a simple file system access > inside a wrapped thread, that is async awaited. > Such a thread is called for a couple of directory > > entries to check a directory tree whether updates > are need. Here some measurement doing this simple > involving some little async I/O: > > node.js: 10 ms (usual Promises and stuff) > JDK 24: 50 ms (using Threads, not yet VirtualThreads) > pypy: 2000 ms > > So currently PyPy is 200x times slower than node.js > when it comes to async I/O. No files were read or > written in the test case, only "mtime" was read, > > via this Python line: > > stats = await asyncio.to_thread(os.stat, url) > > Bye > > Mild Shock schrieb: > > > > Concerning virtual threads the only problem > > with Java I have is, that JDK 17 doesn't have them. > > And some linux distributions are stuck with JDK 17. > > > > Otherwise its not an idea that belongs solely > > to Java, I think golang pioniered them with their > > goroutines. I am planning to use them more heavily > > > > when they become more widely available, and I don't > > see any principle objection that Python wouldn't > > have them as well. It would make async I/O based > > > > on async waithing for a thread maybe more lightweight. > > But this would be only important if you have a high > > number of tasks. > > > > Lawrence D'Oliveiro schrieb: > >> Short answer: no. > >> > >> <https://discuss.python.org/t/add-virtual-threads-to-python/91403> > >> > >> Firstly, anybody appealing to Java as an example of how to design a > >> programming language should immediately be sending your bullshit detector > >> into the yellow zone. > >> > >> Secondly, the link to a critique of JavaScript that dates from 2015, from > >> before the language acquired its async/await constructs, should be > >> another > >> warning sign. > >> > >> Looking at that Java spec, a “virtual thread” is just another name for > >> “stackful coroutine”. Because that’s what you get when you take away > >> implicit thread preemption and substitute explicit preemption instead. > >> > >> The continuation concept is useful in its own right. Why not concentrate > >> on implementing that as a new primitive instead? > >> > > > -- > https://mail.python.org/mailman3//lists/python-list.python.org -- https://mail.python.org/mailman3//lists/python-list.python.org
