On 7/30/2012 10:31 PM, Leon Smith wrote:
This is not necessarily true, on multiple levels. I mean, some of the programs I write are highly concurrent, and this form of batching would have almost no risk of stalling the network buffer. And the possible use case would be when you are dealing with very small rows, when there would typically be several rows inside a single network packet or network buffer.
With "highly concurrent" you mean multi-threaded? Like one thread reads the rows in batches and pushes them into a queue while another thread processes them from that queue?
If that is the case, then you just added a useless layer of buffering and the need for thread/thread context switches to PQsetSingleRowMode. Libpq's "receiver thread" is the kernel itself. Libpq tries to never read partial kernel buffers already. It always makes sure that there are at least 8K of free space in the inBuffer. In the case you describe above, where several rows fit into a single packet, libpq will receive them with a single system call in one read(2), then the application can get them as fast as possible, without causing any further context switches because they are already in the inBuffer.
I've written that sort of code myself in the past. Look at the Slony worker thread prior to 2.2. We switched to the COPY protocol instead of waiting for the single row mode and got rid of all that extra buffering already (and then some more).
Jan -- Anyone who trades liberty for security deserves neither liberty nor security. -- Benjamin Franklin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
