On Mon, May 8, 2017 at 1:39 PM, David Rowley <david.row...@2ndquadrant.com> wrote: > On 6 May 2017 at 13:44, Thomas Munro <thomas.mu...@enterprisedb.com> wrote: >> Experimentation required... > > Indeed. I do remember long discussions on this before Parallel seq > scan went in, but I don't recall if anyone checked any OS kernels to > see what they did. > > We really need a machine with good IO concurrency, and not too much > RAM to test these things out. It could well be that for a suitability > large enough table we'd want to scan a whole 1GB extent per worker.
I did a bunch of simple experiments this morning to try to observe RA effects, using a couple of different EDB machines running Linux. I wrote a simple program to read large files sequentially using lseek + read, but rotate the reads over N file descriptors to simulate parallel workers. I was surprised to find that I couldn't change cache-cold read performance that way, up to very large numbers of N. I did manage to break it by introducing some artificial disorder, reversing/scrambling the read order of small groups of blocks, but even that required groups over about 16 blocks before performance started to drop (possibly related to the window size which I can't see due to permissions right now). I've also learned that RAID cards sometimes do read-ahead of their own, making matters more complicated. I hope to report more when I figure out all the moving parts... > I did post a patch to have heap_parallelscan_nextpage() use atomics > instead of locking over in [1], but I think doing atomics there does > not rule out also adding batching later. In fact, I think it > structures things so batching would be easier than it is today. +1 -- Thomas Munro http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
