Re: Improving pg_dump performance when handling large numbers of LOBs

Might it be worth a modest amount of time using some basic profiling to see where the time is going? A week is a looonnngg time, even for 150e6 operations. For example, if there an unexpectedly high IO load, some temporary M.2 storage might help? On Tue, 6 Feb 2024, 01:36 Ron Johnson, wrote: >

Re: Improving pg_dump performance when handling large numbers of LOBs

On 2/5/24 12:32, Ken Marshall wrote: On Mon, Feb 05, 2024 at 09:17:53PM +0100, Andreas Joseph Krogh wrote: På mandag 05. februar 2024 kl. 21:10:53, skrev Wyatt Tellis < wyatt.tel...@gmail.com >: No, we don't have the ability to make schema changes and the schema

Re: Improving pg_dump performance when handling large numbers of LOBs

Wyatt Tellis writes: > Yes, the LOBs themselves are tiny, but there are a lot of them (~150 > million) which seem to be slowing down pg_dump. Note, we did not > design/build this system and agree that use of LOBs for this purpose was > not necessary. I don't know of anything much you can do at

Re: Improving pg_dump performance when handling large numbers of LOBs

On Mon, Feb 05, 2024 at 09:17:53PM +0100, Andreas Joseph Krogh wrote: > > På mandag 05. februar 2024 kl. 21:10:53, skrev Wyatt Tellis < > wyatt.tel...@gmail.com >: > > No, we don't have the ability to make schema changes and the schema in the > PG15 copy needs to

Re: Improving pg_dump performance when handling large numbers of LOBs

På mandag 05. februar 2024 kl. 21:10:53, skrev Wyatt Tellis < wyatt.tel...@gmail.com >: No, we don't have the ability to make schema changes and the schema in the PG15 copy needs to match what's in the PG 12 versions Well then, I guess it boils down to how badly

Re: Improving pg_dump performance when handling large numbers of LOBs

Just checking... and I sympathize with your situation. On Mon, Feb 5, 2024 at 2:56 PM Wyatt Tellis wrote: > Yes, the LOBs themselves are tiny, but there are a lot of them (~150 > million) which seem to be slowing down pg_dump. Note, we did not > design/build this system and agree that use of

Re: Improving pg_dump performance when handling large numbers of LOBs

No, we don't have the ability to make schema changes and the schema in the PG15 copy needs to match what's in the PG 12 versions Wyatt On Mon, Feb 5, 2024 at 12:05 PM Andreas Joseph Krogh wrote: > På mandag 05. februar 2024 kl. 20:55:43, skrev Wyatt Tellis < > wyatt.tel...@gmail.com>: > > Yes,

Re: Improving pg_dump performance when handling large numbers of LOBs

På mandag 05. februar 2024 kl. 20:55:43, skrev Wyatt Tellis < wyatt.tel...@gmail.com >: Yes, the LOBs themselves are tiny, but there are a lot of them (~150 million) which seem to be slowing down pg_dump. Note, we did not design/build this system and agree that

Re: Improving pg_dump performance when handling large numbers of LOBs

Yes, the LOBs themselves are tiny, but there are a lot of them (~150 million) which seem to be slowing down pg_dump. Note, we did not design/build this system and agree that use of LOBs for this purpose was not necessary. Wyatt On Mon, Feb 5, 2024 at 11:36 AM Ron Johnson wrote: > On Mon, Feb

Re: Improving pg_dump performance when handling large numbers of LOBs

On 2/5/24 11:35 AM, Ron Johnson wrote: On Mon, Feb 5, 2024 at 2:01 PM Wyatt Tellis wrote: Hi, We've inherited a series of legacy PG 12 clusters that each contain a database that we need to migrate to a PG 15 cluster. Each database contains about 150 million large objects

Re: Improving pg_dump performance when handling large numbers of LOBs

On Mon, Feb 5, 2024 at 2:01 PM Wyatt Tellis wrote: > Hi, > > We've inherited a series of legacy PG 12 clusters that each contain a > database that we need to migrate to a PG 15 cluster. Each database contains > about 150 million large objects totaling about 250GB. > 250*10^9 / (150*10^6) = 1667

Improving pg_dump performance when handling large numbers of LOBs

Hi, We've inherited a series of legacy PG 12 clusters that each contain a database that we need to migrate to a PG 15 cluster. Each database contains about 150 million large objects totaling about 250GB. When using pg_dump we've found that it takes a couple of weeks to dump out this much data.