On Wed, Dec 20, 2023 at 06:47:44PM -0500, Tom Lane wrote: > I have spent some more effort in this area and developed a patch > series that I think addresses all of the performance issues that > we've discussed in this thread, both for pg_upgrade and more > general use of pg_dump/pg_restore. Concretely, it absorbs > the pg_restore --transaction-size switch that I proposed before > to cut the number of transactions needed during restore, and > rearranges the representation of BLOB-related TOC entries to > reduce the client-side memory requirements, and fixes some > ancient mistakes that prevent both selective restore of BLOBs > and parallel restore of BLOBs. > > As a demonstration, I made a database containing 100K empty blobs, > and measured the time needed to dump/restore that using -Fd > and -j 10. HEAD doesn't get any useful parallelism on blobs, > but with this patch series we do: > > dump restore > HEAD: 14sec 15sec > after 0002: 7sec 10sec > after 0003: 7sec 3sec
Wow, thanks for putting together these patches. I intend to help review, but I'm not sure I'll find much time to do so before the new year. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com
