Re: [ADMIN] Large database help

2001-04-26 Thread Ragnar Kjørstad
On Mon, Apr 23, 2001 at 08:15:05PM -0400, [EMAIL PROTECTED] wrote: Then, please correct me if I'm wrong: I should be able to test your hypothesis by creating a small DB (of say 2MB) and setting up at least a dozen backends to tag it. If I get the same symptoms w/ respect to disk

Re: [ADMIN] Large database help

2001-04-23 Thread Ragnar Kjørstad
On Sun, Apr 22, 2001 at 06:52:26PM -0400, [EMAIL PROTECTED] wrote: I'm spawning 6 backends to query the data. top lists 6 postmaster processes working, and therefore the idle time should hit 0% easily. Also, the hard drive light goes nuts when I'm running this. Here is the pertinent

Re: [ADMIN] Large database help

2001-04-23 Thread xbdelacour
Below are snippets from: strace postmaster -i -D `pwd` -B 48000 A large snippet: . . . socket(PF_UNIX, SOCK_STREAM, 0) = 4 bind(4, {sun_family=AF_UNIX, sun_path=/tmp/.s.PGSQL.5432}, 20) = 0 listen(4, 128) = 0 chmod(/tmp/.s.PGSQL.5432, 0777) = 0

Re: [ADMIN] Large database help

2001-04-23 Thread xbdelacour
I'm trying to figure that out myself :-) According to the strace info I sent in my last message, it is in fact creating a 381MB shmem block.. which makes no sense, I agree. -Xavier At 01:07 AM 4/23/01 -0400, Tom Lane wrote: 27 processes: 24 sleeping, 3 running, 0 zombie, 0 stopped CPU

Re: [ADMIN] Large database help

2001-04-23 Thread Ragnar Kjørstad
On Mon, Apr 23, 2001 at 06:13:38PM -0400, [EMAIL PROTECTED] wrote: A smaller snippet: shmget(5432001, 400385024, IPC_CREAT|0x180|0600) = 2945 shmget(5432001, 400385024, 0) = 2945 shmat(2945, 0, 0) = 0x40176000 I'm no Unix expert, but this would seem to

Re: [ADMIN] Large database help

2001-04-23 Thread Tom Lane
[EMAIL PROTECTED] writes: I'm no Unix expert, but this would seem to indicate that shmget is successfully allocating 400385024/1024/1024=381MB of shared memory. I don't know enough about how the postgres parent/child/shmem scheme works to know why this is working yet the children only

Re: [ADMIN] Large database help

2001-04-23 Thread xbdelacour
By my reading, the machine is definitely swapping, and not writing to a log file (unless its writing obscene amounts of data to the log, which presumably the default settings won't do). postmaster -i -D /home/mg/pgsql -B 100 produces almost identical results in terms of performance and disk

[ADMIN] Large database help

2001-04-22 Thread xbdelacour
Hi everyone, I'm more or less new to PostgreSQL and am trying to setup a rather large database for a data analysis application. Data is collected and dropped into a single table, which will become ~20GB. Analysis happens on a Windows client (over a network) that queries the data in chunks

Re: [ADMIN] Large database help

2001-04-22 Thread Tom Lane
[EMAIL PROTECTED] writes: Hi everyone, I'm more or less new to PostgreSQL and am trying to setup a rather large database for a data analysis application. Data is collected and dropped into a single table, which will become ~20GB. Analysis happens on a Windows client (over a network) that

Re: [ADMIN] Large database help

2001-04-22 Thread Ragnar Kjørstad
On Sun, Apr 22, 2001 at 05:12:20PM -0400, [EMAIL PROTECTED] wrote: My problem is this: during the query process the hard drive is being tagged excessively, while the cpu's are idling at 50% (numbers from Linux command: top), and this is bringing down the speed pretty dramatically since the

Re: [ADMIN] Large database help

2001-04-22 Thread xbdelacour
I'm spawning 6 backends to query the data. top lists 6 postmaster processes working, and therefore the idle time should hit 0% easily. Also, the hard drive light goes nuts when I'm running this. Here is the pertinent information from top. To be clear, I'm NOT spawning a new postmaster per

Re: [ADMIN] Large database help

2001-04-22 Thread Bruce Momjian
[EMAIL PROTECTED] writes: Hi everyone, I'm more or less new to PostgreSQL and am trying to setup a rather large database for a data analysis application. Data is collected and dropped into a single table, which will become ~20GB. Analysis happens on a Windows client (over a network)

Re: [ADMIN] Large database help

2001-04-22 Thread Bruce Momjian
27 processes: 24 sleeping, 3 running, 0 zombie, 0 stopped CPU states: 16.3% user, 3.8% system, 0.0% nice, 79.8% idle Mem: 517292K av, 508400K used, 8892K free, 9K shrd, 197224K buff Swap: 65988K av, 0K used, 65988K free160740K cached I see zero swap used,

Re: [ADMIN] Large database help

2001-04-22 Thread Tom Lane
27 processes: 24 sleeping, 3 running, 0 zombie, 0 stopped CPU states: 16.3% user, 3.8% system, 0.0% nice, 79.8% idle Mem: 517292K av, 508400K used, 8892K free, 9K shrd, 197224K buff Swap: 65988K av, 0K used, 65988K free160740K cached These numbers don't add