Re: [Dhis2-users] Major speed issue with pg_restore

2017-07-19 Thread Edward Robinson
Agree, the system has two independent disks, I hadn’t set up any mirroring or striping – I had basically just setup the OS in both cases and performed OS updates. Pretty much a vanilla config on both. My experience is that if the SATA channel drivers aren’t properly loaded, the disks will work

Re: [Dhis2-users] Major speed issue with pg_restore

2017-07-18 Thread Bob Jolliffe
> > > > *From:* Bob Jolliffe [mailto:bobjolli...@gmail.com] > *Sent:* Monday, 17 July 2017 2:33 PM > *To:* Edward Robinson > *Cc:* Lars Helge Øverland ; dhis2-users < > dhis2-users@lists.launchpad.net> > *Subject:* Re: [Dhis2-users] Major speed issue with pg_restore

Re: [Dhis2-users] Major speed issue with pg_restore

2017-07-18 Thread Edward Robinson
Maybe, but 2 days vs 50 minutes is a pretty big difference… From: Bob Jolliffe [mailto:bobjolli...@gmail.com] Sent: Monday, 17 July 2017 2:33 PM To: Edward Robinson Cc: Lars Helge Øverland ; dhis2-users Subject: Re: [Dhis2-users] Major speed issue with pg_restore It is unlikely (though not

Re: [Dhis2-users] Major speed issue with pg_restore

2017-07-17 Thread Bob Jolliffe
Monday, 17 July 2017 8:41 AM > *To:* Edward Robinson > *Cc:* Bob Jolliffe ; dhis2-users < > dhis2-users@lists.launchpad.net> > *Subject:* Re: [Dhis2-users] Major speed issue with pg_restore > > > > Hi Edward, > > > > this is besides the main point but ma

Re: [Dhis2-users] Major speed issue with pg_restore

2017-07-17 Thread Edward Robinson
for the support, Cheers Ed From: Lars Helge Øverland [mailto:l...@dhis2.org] Sent: Monday, 17 July 2017 8:41 AM To: Edward Robinson Cc: Bob Jolliffe ; dhis2-users Subject: Re: [Dhis2-users] Major speed issue with pg_restore Hi Edward, this is besides the main point but make sure you create pg

Re: [Dhis2-users] Major speed issue with pg_restore

2017-07-17 Thread Lars Helge Øverland
> successful restore on Windows. > > Regards > > Ed > > > > *From:* Bob Jolliffe [mailto:bobjolli...@gmail.com] > *Sent:* Saturday, 15 July 2017 3:02 PM > *To:* Edward Robinson > *Cc:* Knut Staring ; dhis2-users < > dhis2-users@lists.launchpad.net>; Hannan K

Re: [Dhis2-users] Major speed issue with pg_restore

2017-07-16 Thread Edward Robinson
...@gmail.com] Sent: Saturday, 15 July 2017 3:02 PM To: Edward Robinson Cc: Knut Staring ; dhis2-users ; Hannan Khan Subject: RE: [Dhis2-users] Major speed issue with pg_restore Ed you never answered my earlier question about the hardware you were running on? On 15 Jul 2017 6:40 p.m., "E

Re: [Dhis2-users] Major speed issue with pg_restore

2017-07-15 Thread Bob Jolliffe
Ed you never answered my earlier question about the hardware you were running on? On 15 Jul 2017 6:40 p.m., "Edward Robinson" wrote: > Sorry for this slow response, time has not been on my side 😊 > Also, thanks Bob, Knut and Hannan for your responses. > > So to fill everyone in, I went through p

Re: [Dhis2-users] Major speed issue with pg_restore

2017-07-15 Thread Edward Robinson
Sorry for this slow response, time has not been on my side 😊 Also, thanks Bob, Knut and Hannan for your responses. So to fill everyone in, I went through postgres.conf with a fine toothed comb. @ Bob, Shared buffers were set to 3200 though I'd set them to 1600 for the restore as per this articl

Re: [Dhis2-users] Major speed issue with pg_restore

2017-07-14 Thread Hannan Khan
Hi Ed Agree with Bob; it deepens on processor, ram disc space and configuration. You database size also not seems too big. Now my usual db backup size is 4.8 GB (compressed) and restoring time is usually 4 hour. Definitely something wrong with the configuration. As I am on vacation, cannot help

Re: [Dhis2-users] Major speed issue with pg_restore

2017-07-14 Thread Bob Jolliffe
[mailto:bobjolli...@gmail.com] > Sent: Friday, 14 July 2017 8:07 AM > To: Edward Robinson > Cc: dhis2-users@lists.launchpad.net > Subject: Re: [Dhis2-users] Major speed issue with pg_restore > > I am also trying to understand how a plain sql dump is only 7G while the > custom com

Re: [Dhis2-users] Major speed issue with pg_restore

2017-07-14 Thread Edward Robinson
dumps like these. I'll post my postgresql.conf and other details shortly. Thanks! -Original Message- From: Bob Jolliffe [mailto:bobjolli...@gmail.com] Sent: Friday, 14 July 2017 8:07 AM To: Edward Robinson Cc: dhis2-users@lists.launchpad.net Subject: Re: [Dhis2-users] Major s

Re: [Dhis2-users] Major speed issue with pg_restore

2017-07-14 Thread Bob Jolliffe
I am also trying to understand how a plain sql dump is only 7G while the custom compressed format is 650G. But given that is true, why aren't you using the plain text dump? On 14 July 2017 at 13:02, Bob Jolliffe wrote: > Hi Edward > > Can you tell us a bit more about the machine. In particular

Re: [Dhis2-users] Major speed issue with pg_restore

2017-07-14 Thread Bob Jolliffe
Hi Edward Can you tell us a bit more about the machine. In particular RAM size and disk type/speed and filesystem type. Given that you are probably not going to be running anything else on this machine while you are restoring I think you might be able to tweak a bit more aggressively just for th

Re: [Dhis2-users] Major speed issue with pg_restore

2017-07-13 Thread Edward Robinson
-ubuntu-16-04-7_171095.html Ed From: Knut Staring [mailto:knu...@gmail.com] Sent: Thursday, 13 July 2017 9:58 PM To: Edward Robinson Cc: dhis2-users@lists.launchpad.net Subject: Re: [Dhis2-users] Major speed issue with pg_restore Perhaps useful to also send this question to a postgres forum? On Jul

Re: [Dhis2-users] Major speed issue with pg_restore

2017-07-13 Thread Knut Staring
Perhaps useful to also send this question to a postgres forum? On Jul 13, 2017 9:55 PM, "Edward Robinson" wrote: > I’ve setup a new Ubuntu 16.04 box with PostgreSQL 9.5.5 and I’m restoring > a fairly large DHIS2 backup but having speed issues. It’s a full pg_dump > in custom format and about 65

[Dhis2-users] Major speed issue with pg_restore

2017-07-13 Thread Edward Robinson
I've setup a new Ubuntu 16.04 box with PostgreSQL 9.5.5 and I'm restoring a fairly large DHIS2 backup but having speed issues. It's a full pg_dump in custom format and about 650Gb compressed (a plain text dump produces a 7Gb file). I made sure inserts were turned off, so that's not the issue,