On 11/10/21, 8:10 AM, "David Steele" <da...@pgmasters.net> wrote: > On 11/7/21 1:04 AM, Fujii Masao wrote: >> It's helpful if you share how much this approach reduces >> the amount of overhead. > > FWIW we have a test for this in pgBackRest. Running > `archive_command=pgbackrest archive-push ...` 1000 times via system() > yields an average of 3ms per execution. pgBackRest reports ~1ms of time > here so the system() overhead is ~2ms. These times are on my very fast > workstation and in my experience servers are quite a bit slower.
In the previous thread [0], I noted a 50% speedup for a basic archiving strategy that involved copying the file to a different directory. > I would prefer this module to be in core as our standard implementation > and load by default in a vanilla install. Hm. I think I disagree with putting it in contrib and with making it the default archive library. The first reason is backward compatibility. There has already been quite a bit of discussion about this, and I don't see how we can get away with anything except for maintaining the existing behavior for now. Maybe we could move to a better default down the road, but I'm hesitant to press that issue too much at the moment. The second reason is that the basic_archive module has a couple of deficiencies. For example, it doesn't handle inconvenient server crashes well (e.g., after archiving but before we've renamed the .ready file). A way to fix this might be to compare the archive file with the to-be-archived file and to succeed if they are exactly the same. Another problem is that there is no handling for multiple servers using basic_archive to write WAL to the same location. This is because basic_archive first copies data to a temporary file that is always named "archtemp." This might be fixed by appending a random string to the temporary file or by locking it somehow, but there are still a few things left to figure out. I think it'd be awesome to eventually fix all these issues in basic_archive and to recommend it as a proper archiving strategy, but I'm worried that this will introduce a significant amount of complexity to this patch. I really only intended for basic_archive to be used for testing and to demonstrate that it's possible use the archive module infrastructure to do something useful. If folks really want it in contrib, I'd at least add a big warning about the aforementioned problems in its docs. Nathan [0] https://postgr.es/m/E9035E94-EC76-436E-B6C9-1C03FBD8EF54%40amazon.com
