Antoine Pitrou wrote: > 1M concurrent database transactions? Does that sound reasonable at > all? Your database administrator probably won't like you.
I agree that 1M concurrent transactions would not be reasonable for the vast majority of database configurations, I didn't mean to specifically imply that 1M would be something reasonable today. That's why I said "a large number of concurrent transactions" instead of specifically saying "1M concurrent transactions". I honestly don't know if any databases are close to capable of handling that many transactions at once, at least not at the present time. But, although we may think it's ridiculous today, who's to say that won't be an occurrence in the future? Imagine a scenario where there was a massive super-cluster of database servers, that performed real-time update transactions every time a single item was checked in and out of some global inventory system. Currently, something like this would likely have to be implemented through batching, where x number of updates have to be queued or x amount of time has to pass before the next transaction is started. Alternatively, each facility or region would have its own local database that synchronizes with the global database every so often. But, there could be a significant advantage in having a near-perfectly synchronized global inventory system, which would only be possible if it was updated in real-time. IMO, the max number of concurrent transactions that the a database system can handle at once is a very clear application of Moore's Law. My point being is that I don't want to arbitrarily restrict how many coroutine objects can exist at once without having a strong reason for doing so AND having a limit that's reasonable in the long term. 1M would have the advantage of being easy to remember, but other than that I see no reason why that should specifically be the limit for the max number of coroutines. As Guido mentioned at the start of the thread, a coroutine object is "just another Python object and has no operating resources associated with it". Other than main memory usage, there's no other external limit to the max number of coroutine objects that can exist at once. Note: Although coroutines were already dropped from PEP 611, I felt that this response was still worthwhile to write. I suspect that the topic of "coroutine object limits" is likely to come up again in the future. On Wed, Dec 11, 2019 at 4:46 PM Antoine Pitrou <solip...@pitrou.net> wrote: > On Mon, 9 Dec 2019 21:42:36 -0500 > Kyle Stanley <aeros...@gmail.com> wrote: > > > > (b) Why limit coroutines? It's just another Python object and has no > > operating resources associated with it. Perhaps your definition of > > coroutine is different, and you are thinking of OS threads? > > > > This was my primary concern with the proposed PEP. At the moment, it's > > rather trivial to create one million coroutines, and the total memory > taken > > up by each individual coroutine object is very minimal compared to each > OS > > thread. > > > > There's also a practical use case for having a large number of coroutine > > objects, such as for asynchronously: > > > > 1) Handling a large number of concurrent clients on a continuously > running > > web server that receives a significant amount of traffic. > > Not sure how that works? Each client has an accepted socket, which is > bound to a local port number, and there are 65536 TCP port numbers > available. Unless you're using 15+ coroutines per client, you probably > won't reach 1M coroutines that way. > > > 2) Sending a large number of concurrent database transactions to run on a > > cluster of database servers. > > 1M concurrent database transactions? Does that sound reasonable at > all? Your database administrator probably won't like you. > > > something like this definitely scales over time. Arbitrarily placing a > > limit on the total number of coroutine objects doesn't make sense to me > for > > that reason. > > There are a lot of arbitrary limits inside a computer system. You just > aren't aware of them because you don't hit them in practice. Claiming > that limits shouldn't exist is just pointless. > > Regards > > Antoine. > > _______________________________________________ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/O3ZODXHEIJ2SM5SZBOVJ4PIAQMSYNXEJ/ > Code of Conduct: http://python.org/psf/codeofconduct/ >
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/PYUSGK4UDMGGDJPQ6E6YHKG2O5D5DTEL/ Code of Conduct: http://python.org/psf/codeofconduct/
