Re: [Dhis2-users] [Dhis2-devs] restore database out of memory
thank you :-) On Sat, Mar 19, 2016 at 1:32 PM, gerald thomas wrote: > Dear Channara, > I think Dcocos solution should be a break through and it will minimize the > database size. > > Regards, > Gerald > On Mar 19, 2016 05:03, "channara rin" wrote: > >> Thank you, >> let me try testing... >> and by the way, i configured and can run DHIS2 on localhost already. but >> when i plug data into existing DHIS2. it is out of memory. >> >> On Sat, Mar 19, 2016 at 10:08 AM, dcocos wrote: >> >>> Hi, >>> >>> If you dump with (where dbname is your rename) >>> pg_dump dbname -O -T _* -T aggregated* -T analytics* -T completeness* >>> >>> This will dump without the generated tables which you can recreate by >>> running analytics this reduces the size of the import often by an order of >>> magnitude. >>> >>> On Mar 18, 2016, at 10:14 PM, channara rin >>> wrote: >>> >>> Hi bob, >>> when i backup, i use command pg_dumpall. >>> i can restore on my personal laptop with 4GB Memory. but for desktop >>> seem out of memory. >>> i will re-check again on task manager what is still processing. >>> >>> On Fri, Mar 18, 2016 at 11:14 PM, Bob Jolliffe >>> wrote: >>> One other quick thought that is easy to test and eliminate. Postgres out of memory errors on restore can also result from a corrupted dump file. It might be worthwhile to check on another system that your dump is good. On 18 March 2016 at 17:08, Bob Jolliffe wrote: > Ah you are on Windoze. I also don't have much real experience of > running dhis2 other than on linux, but it strikes me that > (i) 4G machine is small but should still "work" > (ii) the databse size you are talking about 100m is quite small and > the restore operation should not be consuming vast amounts of memory > (iii) postgres will run, albeit not very efficiently, with quite small > amounts of memory. > > So ... > > I think you need to look at what else you have running on that machine > which could be consuming the bulk of memory. I think windoze system > tools like task manager and perfmon.exe might show you useful > information. > > Have you modified postgres config or is it default setting? Default > settings would be very conservative but should work. If you have > modified, you might have given postgres the impression it has more > memory available than actually exists. > > On 18 March 2016 at 16:53, channara rin wrote: >> No from hard drive. it has free disk space 100GB for C: and D: is 45GB >> >> On Fri, Mar 18, 2016 at 10:31 PM, Knut Staring wrote: >>> >>> Sorry Channara, I didn't read your question properly, this has nothing to >>> do with JAVA_OPTS. >>> >>> Unfortunately, I have no experience with getting Postgres out of memory >>> errors, though google tells me a lot of people do. >>> >>> Could it perhaps be that your hard drive is filling up? >>> >>> On Fri, Mar 18, 2016 at 3:38 PM, channara rin < rin.chann...@gmail.com> >>> wrote: here in java_opts=-Djava.awt.headless=true -Dfile.encoding=UTF-8 -server -Xms128m -Xmx1024m -XX:NewSize=2048m -XXMaxNewSize=2048m -XX:PermSize=256m -XX:MaxPermSize=1024m -XX:+DisableExplicitGC -xx:+UseConcMarkSweepGC -XX:+DisableExplicitGC On Fri, Mar 18, 2016 at 7:39 PM, Knut Staring wrote: > > Hi Channara, > > What values do you have for JAVA_OPTS? > > On Fri, Mar 18, 2016 at 1:10 PM, channara rin < rin.chann...@gmail.com> > wrote: >> >> hi all, >> do you hav any experience with postgresql database? >> i use command psql/pg_restore into new system. i got out of memory. >> my system server is 4GB memory and file backup size is around 100MB >> >> ___ >> Mailing list: https://launchpad.net/~dhis2-users >> Post to : dhis2-users@lists.launchpad.net >> Unsubscribe : https://launchpad.net/~dhis2-users >> More help : https://help.launchpad.net/ListHelp >> > > > > -- > Knut Staring > Dept. of Informatics, University of Oslo > Norway: +4791880522 > Skype: knutstar > http://dhis2.org >>> >>> >>> >>> -- >>> Knut Staring >>> Dept. of Informatics, University of Oslo >>> Norway: +4791880522 >>> Skype: knutstar >>> http://dhis2.org >> >> >> >> ___ >> Mailing list: https://launchpad.net/~dhis2-users >> Post to : dhis2-users@lists.launchpad.net >> Unsubscribe : https://launchpad.net/~dhis2-user
Re: [Dhis2-users] [Dhis2-devs] restore database out of memory
Dear Channara, I think Dcocos solution should be a break through and it will minimize the database size. Regards, Gerald On Mar 19, 2016 05:03, "channara rin" wrote: > Thank you, > let me try testing... > and by the way, i configured and can run DHIS2 on localhost already. but > when i plug data into existing DHIS2. it is out of memory. > > On Sat, Mar 19, 2016 at 10:08 AM, dcocos wrote: > >> Hi, >> >> If you dump with (where dbname is your rename) >> pg_dump dbname -O -T _* -T aggregated* -T analytics* -T completeness* >> >> This will dump without the generated tables which you can recreate by >> running analytics this reduces the size of the import often by an order of >> magnitude. >> >> On Mar 18, 2016, at 10:14 PM, channara rin >> wrote: >> >> Hi bob, >> when i backup, i use command pg_dumpall. >> i can restore on my personal laptop with 4GB Memory. but for desktop seem >> out of memory. >> i will re-check again on task manager what is still processing. >> >> On Fri, Mar 18, 2016 at 11:14 PM, Bob Jolliffe >> wrote: >> >>> One other quick thought that is easy to test and eliminate. Postgres >>> out of memory errors on restore can also result from a corrupted dump >>> file. It might be worthwhile to check on another system that your >>> dump is good. >>> >>> On 18 March 2016 at 17:08, Bob Jolliffe wrote: >>> > Ah you are on Windoze. I also don't have much real experience of >>> > running dhis2 other than on linux, but it strikes me that >>> > (i) 4G machine is small but should still "work" >>> > (ii) the databse size you are talking about 100m is quite small and >>> > the restore operation should not be consuming vast amounts of memory >>> > (iii) postgres will run, albeit not very efficiently, with quite small >>> > amounts of memory. >>> > >>> > So ... >>> > >>> > I think you need to look at what else you have running on that machine >>> > which could be consuming the bulk of memory. I think windoze system >>> > tools like task manager and perfmon.exe might show you useful >>> > information. >>> > >>> > Have you modified postgres config or is it default setting? Default >>> > settings would be very conservative but should work. If you have >>> > modified, you might have given postgres the impression it has more >>> > memory available than actually exists. >>> > >>> > On 18 March 2016 at 16:53, channara rin >>> wrote: >>> >> No from hard drive. it has free disk space 100GB for C: and D: is 45GB >>> >> >>> >> On Fri, Mar 18, 2016 at 10:31 PM, Knut Staring >>> wrote: >>> >>> >>> >>> Sorry Channara, I didn't read your question properly, this has >>> nothing to >>> >>> do with JAVA_OPTS. >>> >>> >>> >>> Unfortunately, I have no experience with getting Postgres out of >>> memory >>> >>> errors, though google tells me a lot of people do. >>> >>> >>> >>> Could it perhaps be that your hard drive is filling up? >>> >>> >>> >>> On Fri, Mar 18, 2016 at 3:38 PM, channara rin < >>> rin.chann...@gmail.com> >>> >>> wrote: >>> >>> here in java_opts=-Djava.awt.headless=true -Dfile.encoding=UTF-8 >>> -server >>> -Xms128m -Xmx1024m -XX:NewSize=2048m -XXMaxNewSize=2048m >>> -XX:PermSize=256m >>> -XX:MaxPermSize=1024m -XX:+DisableExplicitGC -xx:+UseConcMarkSweepGC >>> -XX:+DisableExplicitGC >>> >>> On Fri, Mar 18, 2016 at 7:39 PM, Knut Staring >>> wrote: >>> > >>> > Hi Channara, >>> > >>> > What values do you have for JAVA_OPTS? >>> > >>> > On Fri, Mar 18, 2016 at 1:10 PM, channara rin < >>> rin.chann...@gmail.com> >>> > wrote: >>> >> >>> >> hi all, >>> >> do you hav any experience with postgresql database? >>> >> i use command psql/pg_restore into new system. i got out of >>> memory. >>> >> my system server is 4GB memory and file backup size is around >>> 100MB >>> >> >>> >> ___ >>> >> Mailing list: https://launchpad.net/~dhis2-users >>> >> Post to : dhis2-users@lists.launchpad.net >>> >> Unsubscribe : https://launchpad.net/~dhis2-users >>> >> More help : https://help.launchpad.net/ListHelp >>> >> >>> > >>> > >>> > >>> > -- >>> > Knut Staring >>> > Dept. of Informatics, University of Oslo >>> > Norway: +4791880522 >>> > Skype: knutstar >>> > http://dhis2.org >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> -- >>> >>> Knut Staring >>> >>> Dept. of Informatics, University of Oslo >>> >>> Norway: +4791880522 >>> >>> Skype: knutstar >>> >>> http://dhis2.org >>> >> >>> >> >>> >> >>> >> ___ >>> >> Mailing list: https://launchpad.net/~dhis2-users >>> >> Post to : dhis2-users@lists.launchpad.net >>> >> Unsubscribe : https://launchpad.net/~dhis2-users >>> >> More help : https://help.launchpad.net/ListHelp >>> >> >>> >> >> ___ >> Mailing list: https://launchpad.net/~dhis2-devs >> Post to : dhis2-d...@lists.launchpad.net >> Un
Re: [Dhis2-users] [Dhis2-devs] restore database out of memory
Thank you, let me try testing... and by the way, i configured and can run DHIS2 on localhost already. but when i plug data into existing DHIS2. it is out of memory. On Sat, Mar 19, 2016 at 10:08 AM, dcocos wrote: > Hi, > > If you dump with (where dbname is your rename) > pg_dump dbname -O -T _* -T aggregated* -T analytics* -T completeness* > > This will dump without the generated tables which you can recreate by > running analytics this reduces the size of the import often by an order of > magnitude. > > On Mar 18, 2016, at 10:14 PM, channara rin wrote: > > Hi bob, > when i backup, i use command pg_dumpall. > i can restore on my personal laptop with 4GB Memory. but for desktop seem > out of memory. > i will re-check again on task manager what is still processing. > > On Fri, Mar 18, 2016 at 11:14 PM, Bob Jolliffe > wrote: > >> One other quick thought that is easy to test and eliminate. Postgres >> out of memory errors on restore can also result from a corrupted dump >> file. It might be worthwhile to check on another system that your >> dump is good. >> >> On 18 March 2016 at 17:08, Bob Jolliffe wrote: >> > Ah you are on Windoze. I also don't have much real experience of >> > running dhis2 other than on linux, but it strikes me that >> > (i) 4G machine is small but should still "work" >> > (ii) the databse size you are talking about 100m is quite small and >> > the restore operation should not be consuming vast amounts of memory >> > (iii) postgres will run, albeit not very efficiently, with quite small >> > amounts of memory. >> > >> > So ... >> > >> > I think you need to look at what else you have running on that machine >> > which could be consuming the bulk of memory. I think windoze system >> > tools like task manager and perfmon.exe might show you useful >> > information. >> > >> > Have you modified postgres config or is it default setting? Default >> > settings would be very conservative but should work. If you have >> > modified, you might have given postgres the impression it has more >> > memory available than actually exists. >> > >> > On 18 March 2016 at 16:53, channara rin wrote: >> >> No from hard drive. it has free disk space 100GB for C: and D: is 45GB >> >> >> >> On Fri, Mar 18, 2016 at 10:31 PM, Knut Staring >> wrote: >> >>> >> >>> Sorry Channara, I didn't read your question properly, this has >> nothing to >> >>> do with JAVA_OPTS. >> >>> >> >>> Unfortunately, I have no experience with getting Postgres out of >> memory >> >>> errors, though google tells me a lot of people do. >> >>> >> >>> Could it perhaps be that your hard drive is filling up? >> >>> >> >>> On Fri, Mar 18, 2016 at 3:38 PM, channara rin > > >> >>> wrote: >> >> here in java_opts=-Djava.awt.headless=true -Dfile.encoding=UTF-8 >> -server >> -Xms128m -Xmx1024m -XX:NewSize=2048m -XXMaxNewSize=2048m >> -XX:PermSize=256m >> -XX:MaxPermSize=1024m -XX:+DisableExplicitGC -xx:+UseConcMarkSweepGC >> -XX:+DisableExplicitGC >> >> On Fri, Mar 18, 2016 at 7:39 PM, Knut Staring >> wrote: >> > >> > Hi Channara, >> > >> > What values do you have for JAVA_OPTS? >> > >> > On Fri, Mar 18, 2016 at 1:10 PM, channara rin < >> rin.chann...@gmail.com> >> > wrote: >> >> >> >> hi all, >> >> do you hav any experience with postgresql database? >> >> i use command psql/pg_restore into new system. i got out of memory. >> >> my system server is 4GB memory and file backup size is around 100MB >> >> >> >> ___ >> >> Mailing list: https://launchpad.net/~dhis2-users >> >> Post to : dhis2-users@lists.launchpad.net >> >> Unsubscribe : https://launchpad.net/~dhis2-users >> >> More help : https://help.launchpad.net/ListHelp >> >> >> > >> > >> > >> > -- >> > Knut Staring >> > Dept. of Informatics, University of Oslo >> > Norway: +4791880522 >> > Skype: knutstar >> > http://dhis2.org >> >> >> >>> >> >>> >> >>> >> >>> -- >> >>> Knut Staring >> >>> Dept. of Informatics, University of Oslo >> >>> Norway: +4791880522 >> >>> Skype: knutstar >> >>> http://dhis2.org >> >> >> >> >> >> >> >> ___ >> >> Mailing list: https://launchpad.net/~dhis2-users >> >> Post to : dhis2-users@lists.launchpad.net >> >> Unsubscribe : https://launchpad.net/~dhis2-users >> >> More help : https://help.launchpad.net/ListHelp >> >> >> > > ___ > Mailing list: https://launchpad.net/~dhis2-devs > Post to : dhis2-d...@lists.launchpad.net > Unsubscribe : https://launchpad.net/~dhis2-devs > More help : https://help.launchpad.net/ListHelp > > ___ Mailing list: https://launchpad.net/~dhis2-users Post to : dhis2-users@lists.launchpad.net Unsubscribe : https://launchpad.net/~dhis2-users More help : https://help.launchpad.net/ListHelp