On August 10, 2015 8:24:21 PM GMT+02:00, Fabien COELHO <coe...@cri.ensmp.fr> wrote: > >Hello Andres, > >> You can't allocate 4GB with palloc(), it has a builtin limit against >> allocating more than 1GB. > >Argh, too bad, I assumed very naively that palloc was malloc in >disguise.
It is, but there's some layering (memory pools/contexts) on top. You can get huge allocations with polloc_huge. >Then the file would be fsynced twice: if the fsync is done properly >(data >have already been flushed to disk) then it would not cost much, and >doing >it sometimes twice on some file would not be a big issue. The code >could >also detect such event and log a warning, which would give a hint about > >how often it occurs in practice. Right. At the cost of keeping track of all files... >>>> If the pivot element changes its identity won't the result be >pretty much >>>> random? >>> >>> That would be a very unlikely event, given the short time spent in >>> qsort. >> >> Meh, we don't want to rely on "likeliness" on such things. > >My main argument is that even if it occurs, and the qsort result is >partly >wrong, it does not change correctness, it just mean that the actual >writes >will be less in order than wished. If it occurs, one pivot separation >would be quite strange, but then others would be right, so the buffers >would be "partly sorted". It doesn't matter for correctness today, correct. But it makes out impossible to rely on or too. >Another issue I see is that even if buffers are locked within cmp, the >status may change between two cmp... Sure. That's not what in suggesting. Earlier versions of the patch kept an array of buffer headers exactly because of that. I do not think that locking all >buffers for sorting them is an option. So on the whole, I think that >locking buffers for sorting is probably not possible with the simple >(and >efficient) lightweight approach used in the patch. Yes, the other version has a higher space overhead. I'm not convinced that's meaningful in comparison to shared buffets in space. And rather doubtful it a loss performance wise in a loaded server. All the buffer headers are touched on other cores and doing the sort with indirection will greatly increase bus traffic. >The good news, as I argued before, is that the order is only advisory >to >help with performance, but the correctness is really that all >checkpoint >buffers are written and fsync is called in the end, and does not depend >on >the buffer order. That is how it currently works anyway It's not particularly desirable to have a performance feature that works less well if the server is heavily and concurrently loaded. The likelihood of bogus sort results will increase with the churn rate in shared buffers. Andres --- Please excuse brevity and formatting - I am writing this on my mobile phone. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
