Re: pgBackRest for a 50 TB database

2023-12-19 Thread Abhishek Bhola
Hello Stephen Just an update on this. After we deployed it on our PROD system, the results were far better than testing. Time taken is around 4-5 hours only. And has been the case for the last 3 months or so. full backup: 20231209-150002F timestamp start/stop: 2023-12-09 15:00:02+09 /

Re: pgBackRest for a 50 TB database

2023-10-05 Thread Stephen Frost
Greetings, On Thu, Oct 5, 2023 at 03:10 Abhishek Bhola wrote: > Here is the update with compress-type=zst in the config file > Process-max is still 30. *But it longer than before, around 27 hours 50 > mins* > > full backup: 20231004-130621F > timestamp start/stop: 2023-10-04

Re: pgBackRest for a 50 TB database

2023-10-05 Thread Abhishek Bhola
Hi Stephen Here is the update with compress-type=zst in the config file Process-max is still 30. *But it longer than before, around 27 hours 50 mins* full backup: 20231004-130621F timestamp start/stop: 2023-10-04 13:06:21+09 / 2023-10-05 15:56:03+09 wal start/stop:

Re: pgBackRest for a 50 TB database

2023-10-03 Thread KK CHN
Greetings, Happy to hear you successfully performed pgBackRest for a 50TB DB. Out of curiosity I would like to know your infrastructure settings. 1. The connectivity protocoal and bandwidth you used for your backend storage ? Is it iSCSI, FC FCoE or GbE ? what's the exact reason for the 26

Re: pgBackRest for a 50 TB database

2023-10-03 Thread Abhishek Bhola
Hi Stephen No, I did not try that. Let me try that now and report the numbers here, both in terms of size and time taken. Thanks for the suggestion. On Tue, Oct 3, 2023 at 10:39 PM Stephen Frost wrote: > Greetings, > > On Mon, Oct 2, 2023 at 20:08 Abhishek Bhola < >

Re: pgBackRest for a 50 TB database

2023-10-03 Thread Stephen Frost
Greetings, On Mon, Oct 2, 2023 at 20:08 Abhishek Bhola wrote: > As said above, I tested pgBackRest on my bigger DB and here are the > results. > Server on which this is running has the following config: > Architecture: x86_64 > CPU op-mode(s):32-bit, 64-bit > Byte Order:

Re: pgBackRest for a 50 TB database

2023-10-02 Thread Abhishek Bhola
Hello, As said above, I tested pgBackRest on my bigger DB and here are the results. Server on which this is running has the following config: Architecture: x86_64 CPU op-mode(s):32-bit, 64-bit Byte Order:Little Endian CPU(s):36 On-line CPU(s) list:

Re: pgBackRest for a 50 TB database

2023-08-27 Thread o1bigtenor
On Sun, Aug 27, 2023 at 10:57 AM Abhishek Bhola wrote: > > Hi > > I am trying to use pgBackRest for all my Postgres servers. I have tested it > on a sample database and it works fine. But my concern is for some of the > bigger DB clusters, the largest one being 50TB and growing by about

Re: pgBackRest for a 50 TB database

2023-08-27 Thread Abhishek Bhola
Hi Stephen Thank you for the prompt response. Hearing it from you makes me more confident about rolling it to PROD. I will have a discussion with the network team once about and hear what they have to say and make an estimate accordingly. If you happen to know anyone using it with that size and

Re: pgBackRest for a 50 TB database

2023-08-27 Thread Stephen Frost
Greetings, * Abhishek Bhola (abhishek.bh...@japannext.co.jp) wrote: > I am trying to use pgBackRest for all my Postgres servers. I have tested it > on a sample database and it works fine. But my concern is for some of the > bigger DB clusters, the largest one being 50TB and growing by about >

pgBackRest for a 50 TB database

2023-08-27 Thread Abhishek Bhola
Hi I am trying to use pgBackRest for all my Postgres servers. I have tested it on a sample database and it works fine. But my concern is for some of the bigger DB clusters, the largest one being 50TB and growing by about 200-300GB a day. I plan to mount NAS storage on my DB server to store my