Re: [ADMIN] Large database help

2001-04-26 Thread Ragnar Kjørstad
On Mon, Apr 23, 2001 at 08:15:05PM -0400, [EMAIL PROTECTED] wrote: > Then, please correct me if I'm wrong: I should be able to test your > hypothesis by creating a small DB (of say 2MB) and setting up at least a > dozen backends to tag it. If I get the same symptoms w/ respect to disk > activit

Re: [ADMIN] Large database help

2001-04-23 Thread xbdelacour
By my reading, the machine is definitely swapping, and not writing to a log file (unless its writing obscene amounts of data to the log, which presumably the default settings won't do). postmaster -i -D /home/mg/pgsql -B 100 produces almost identical results in terms of performance and disk a

Re: [ADMIN] Large database help

2001-04-23 Thread Tom Lane
[EMAIL PROTECTED] writes: > I'm no Unix expert, but this would seem to indicate that shmget is > successfully allocating 400385024/1024/1024=381MB of shared memory. I don't > know enough about how the postgres parent/child/shmem scheme works to know > why this is working yet the children only r

Re: [ADMIN] Large database help

2001-04-23 Thread xbdelacour
ANote: some disk activety should be expected. Maybe postgresql updates >the log? Or at the very least it will update the atime timestamps for >the files everytime they're read. This shouldn't cause enough disk >activity to become a performance-problem, but if I remember your initial >post correctl

Re: [ADMIN] Large database help

2001-04-23 Thread Ragnar Kjørstad
On Mon, Apr 23, 2001 at 06:13:38PM -0400, [EMAIL PROTECTED] wrote: > A smaller snippet: > > shmget(5432001, 400385024, IPC_CREAT|0x180|0600) = 2945 > shmget(5432001, 400385024, 0) = 2945 > shmat(2945, 0, 0) = 0x40176000 > > I'm no Unix expert, but this would seem

Re: [ADMIN] Large database help

2001-04-23 Thread xbdelacour
I'm trying to figure that out myself :-) According to the strace info I sent in my last message, it is in fact creating a 381MB shmem block.. which makes no sense, I agree. -Xavier At 01:07 AM 4/23/01 -0400, Tom Lane wrote: > >> 27 processes: 24 sleeping, 3 running, 0 zombie, 0 stopped > >> CP

Re: [ADMIN] Large database help

2001-04-23 Thread xbdelacour
Below are snippets from: strace postmaster -i -D `pwd` -B 48000 A large snippet: . . . socket(PF_UNIX, SOCK_STREAM, 0) = 4 bind(4, {sun_family=AF_UNIX, sun_path="/tmp/.s.PGSQL.5432"}, 20) = 0 listen(4, 128) = 0 chmod("/tmp/.s.PGSQL.5432", 0777) = 0 shmget(5

Re: [ADMIN] Large database help

2001-04-23 Thread Ragnar Kjørstad
On Sun, Apr 22, 2001 at 06:52:26PM -0400, [EMAIL PROTECTED] wrote: > I'm spawning 6 backends to query the data. top lists 6 postmaster processes > working, and therefore the idle time should hit 0% easily. Also, the hard > drive light goes nuts when I'm running this. > > Here is the pertinent i

Re: [ADMIN] Large database help

2001-04-22 Thread Tom Lane
>> 27 processes: 24 sleeping, 3 running, 0 zombie, 0 stopped >> CPU states: 16.3% user, 3.8% system, 0.0% nice, 79.8% idle >> Mem: 517292K av, 508400K used, 8892K free, 9K shrd, 197224K buff >> Swap: 65988K av, 0K used, 65988K free160740K cached These numbers don'

Re: [ADMIN] Large database help

2001-04-22 Thread Bruce Momjian
> 27 processes: 24 sleeping, 3 running, 0 zombie, 0 stopped > CPU states: 16.3% user, 3.8% system, 0.0% nice, 79.8% idle > Mem: 517292K av, 508400K used, 8892K free, 9K shrd, 197224K buff > Swap: 65988K av, 0K used, 65988K free160740K cached > I see zero swap use

Re: [ADMIN] Large database help

2001-04-22 Thread Bruce Momjian
> [EMAIL PROTECTED] writes: > > Hi everyone, I'm more or less new to PostgreSQL and am trying to setup a > > rather large database for a data analysis application. Data is collected > > and dropped into a single table, which will become ~20GB. Analysis happens > > on a Windows client (over a ne

Re: [ADMIN] Large database help

2001-04-22 Thread xbdelacour
I'm spawning 6 backends to query the data. top lists 6 postmaster processes working, and therefore the idle time should hit 0% easily. Also, the hard drive light goes nuts when I'm running this. Here is the pertinent information from top. To be clear, I'm NOT spawning a new postmaster per chun

Re: [ADMIN] Large database help

2001-04-22 Thread Ragnar Kjørstad
On Sun, Apr 22, 2001 at 05:12:20PM -0400, [EMAIL PROTECTED] wrote: > My problem is this: during the query process the hard drive is being tagged > excessively, while the cpu's are idling at 50% (numbers from Linux command: > top), and this is bringing down the speed pretty dramatically since the

Re: [ADMIN] Large database help

2001-04-22 Thread xbdelacour
pgsql/PG_VERSION says "7.0". postmaster --version and psql --version don't tell me anything. If the machine is completely dedicated to this database, under what conditions would the kernel make such decisions? Where can I find more information on this? Are there other users with similar requir

Re: [ADMIN] Large database help

2001-04-22 Thread Tom Lane
[EMAIL PROTECTED] writes: > Hi everyone, I'm more or less new to PostgreSQL and am trying to setup a > rather large database for a data analysis application. Data is collected > and dropped into a single table, which will become ~20GB. Analysis happens > on a Windows client (over a network) tha