On 29 April 2018 at 00:15, Andres Freund <and...@anarazel.de> wrote: > Hi, > > On 2018-04-28 08:25:53 -0700, Simon Riggs wrote: >> > - Use direct IO. Due to architectural performance issues in PG and the >> > fact that it'd not be applicable for all installations I don't think >> > this is a reasonable fix for the issue presented here. Although it's >> > independently something we should work on. It might be worthwhile to >> > provide a configuration that allows to force DIO to be enabled for WAL >> > even if replication is turned on. >> >> "Use DirectIO" is roughly same suggestion as "don't trust Linux filesystems". > > I want to emphasize that this is NOT a linux only issue. It's a problem > across a number of operating systems, including linux. > > >> It would be a major admission of defeat for us to take that as our >> main route to a solution. > > Well, I think we were wrong to not engineer towards DIO. There's just > too many issues with buffered IO to not have a supported path for > DIO. But given that it's unrealistic to do so without major work, and > wouldn't be applicable for all installations (shared_buffer size becomes > critical), I don't think it matters that much for the issue discussed > here.
20/20 hindsight, really. Not much to be done now. Even with the work you and others have done on shared_buffers scalability, there's likely still improvement needed there if it becomes more important to evict buffers into per-device queues, etc, too. Personally I'd rather not have to write half the kernel's job because the kernel doesn't feel like doing it :( . I'd kind of hoped to go in the other direction if anything, with some kind of pseudo-write op that let us swap a dirty shared_buffers entry from our shared_buffers into the OS dirty buffer cache (on Linux at least) and let it handle writeback, so we reduce double-buffering. Ha! So much for that! -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
