[PERFORM] Regexps - never completing join.

2008-05-13 Thread Rusty Conover
Hi Guys, I'm using postgresql 8.3.1 and I'm seeing weird behavior between what I expect and what's happening when the query is executed I'm trying to match a table that contains regexps against another table that is full of the text to match against so my query is: select wc_rule.id from

Re: [PERFORM] Problem with 11 M records table

2008-05-13 Thread Ramasubramanian G
Hi , Set this parameter in psotgresql.conf set enable_seqscan=off; And try: -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Scott Marlowe Sent: Tuesday, May 13, 2008 11:32 PM To: idc danny Cc: pgsql-performance@postgresql.org Subject: Re: [PERFORM] Proble

[PERFORM] can I move sort to first outer join ?

2008-05-13 Thread fernando castano
Hi all, This sql is taking too long for the size of my tiny db. Any tips from this alias? I tried moving the sort to the first left outer join (between projects and features tables) using a nested subquery, but postgres tells me only one column could be returned from a subqueyr. TIA, fdo

Re: [PERFORM] RAID controllers for Postgresql on large setups

2008-05-13 Thread PFC
You say that like you don't mind having PCI in a server whose job is to perform massive query over large data sets. I am in my 4th week at a new job. Trying to figure what I am working with. LOOL, ok, hehe, not exactly the time to have a "let's change everything" fit ;) From what I

Re: [PERFORM] RAID controllers for Postgresql on large setups

2008-05-13 Thread James Mansion
PFC wrote: PCI limits you to 133 MB/s (theoretical), actual speed being around 100-110 MB/s. Many servers do have more than one bus. You have to process that data too so its not going to be as much of a limit as you are suggesting. It may be possible to stream a compressed data file to th

Re: [PERFORM] Problem with 11 M records table

2008-05-13 Thread Scott Marlowe
On Tue, May 13, 2008 at 10:57 AM, idc danny <[EMAIL PROTECTED]> wrote: > Hi everybody, > > I'm fairly new to PostgreSQL and I have a problem with > a query: > > SELECT * FROM "LockerEvents" LIMIT 1 OFFSET > 1099 > > The table LockerEvents has 11 Mlillions records on it > and this query take

Re: [PERFORM] Problem with 11 M records table

2008-05-13 Thread Craig James
idc danny wrote: Hi James, Than you for your response. What I want to achieve is to give to the application user 10k rows where the records are one after another in the table, and the application has a paginating GUI ("First page", "Previous page", "Next page", "Last page" - all links & "Jump t

Re: [PERFORM] Problem with 11 M records table

2008-05-13 Thread Craig James
idc danny wrote: Hi everybody, I'm fairly new to PostgreSQL and I have a problem with a query: SELECT * FROM "LockerEvents" LIMIT 1 OFFSET 1099 The table LockerEvents has 11 Mlillions records on it and this query takes about 60 seconds to complete. The OFFSET clause is almost always

Re: [PERFORM] Problem with 11 M records table

2008-05-13 Thread salman
idc danny wrote: Hi everybody, I'm fairly new to PostgreSQL and I have a problem with a query: SELECT * FROM "LockerEvents" LIMIT 1 OFFSET 1099 The table LockerEvents has 11 Mlillions records on it and this query takes about 60 seconds to complete. Moreover, even after making for eac

Re: [PERFORM] Problem with 11 M records table

2008-05-13 Thread Bill Moran
In response to idc danny <[EMAIL PROTECTED]>: > Hi everybody, > > I'm fairly new to PostgreSQL and I have a problem with > a query: > > SELECT * FROM "LockerEvents" LIMIT 1 OFFSET > 1099 This query makes no sense, and I can't blame PostgreSQL for using a seq scan, since you've given it

[PERFORM] Problem with 11 M records table

2008-05-13 Thread idc danny
Hi everybody, I'm fairly new to PostgreSQL and I have a problem with a query: SELECT * FROM "LockerEvents" LIMIT 1 OFFSET 1099 The table LockerEvents has 11 Mlillions records on it and this query takes about 60 seconds to complete. Moreover, even after making for each column in the table

Re: [PERFORM] RAID controllers for Postgresql on large setups

2008-05-13 Thread Vivek Khera
On May 12, 2008, at 11:24 PM, Francisco Reyes wrote: Any PCI controller you have had good experience with? How any other PCI-X/PCI-e controller that you have had good results? The LSI controllers are top-notch, and always my first choice. They have PCI-X and PCI-e versions. -- Sent via

Re: [PERFORM] RAID controllers for Postgresql on large setups

2008-05-13 Thread Vivek Khera
On May 12, 2008, at 10:04 PM, Francisco Reyes wrote: Adaptec 2120 SCSI controller (64MB of cache). The servers have mostly have 12 drives in RAID 10. We are going to redo one machine to compare RAID 10 vs RAID 50. Mostly to see if the perfomance is close, the space gain may be usefull.

Re: [PERFORM] Installation Steps to migrate to Postgres 8.3.1

2008-05-13 Thread Scott Marlowe
On Tue, May 13, 2008 at 6:00 AM, Knight, Doug <[EMAIL PROTECTED]> wrote: > Hi, > Along these lines, the usual upgrade path is a pg_dump/pg_restore set. > However, what if your database is large (> 50GB), and you have to > minimize your downtime (say less than an hour or two). Any suggestions > on h

Re: [PERFORM] Installation Steps to migrate to Postgres 8.3.1

2008-05-13 Thread Knight, Doug
Hi, Along these lines, the usual upgrade path is a pg_dump/pg_restore set. However, what if your database is large (> 50GB), and you have to minimize your downtime (say less than an hour or two). Any suggestions on how to handle that kind of situation? It sure would be nice to have some kind of too

Re: [PERFORM] RAID controllers for Postgresql on large setups

2008-05-13 Thread Merlin Moncure
On Tue, May 13, 2008 at 8:00 AM, Francisco Reyes <[EMAIL PROTECTED]> wrote: > PFC writes: > > > >You say that like you don't mind having PCI in a server whose job > is to perform massive query over large data sets. > > > > I am in my 4th week at a new job. Trying to figure what I am worki

Re: [PERFORM] RAID controllers for Postgresql on large setups

2008-05-13 Thread Francisco Reyes
PFC writes: You say that like you don't mind having PCI in a server whose job is to perform massive query over large data sets. I am in my 4th week at a new job. Trying to figure what I am working with. From what I see I will likely get as much improvement from new hardware as from re-doing

Re: [PERFORM] RAID controllers for Postgresql on large setups

2008-05-13 Thread PFC
Will it pay to go to a controller with higher memory for existing machines? The one machine I am about to redo has PCI which seems to somewhat limit our options. Urgh. You say that like you don't mind having PCI in a server whose job is to perform massive query over large data s

Re: [PERFORM] Installation Steps to migrate to Postgres 8.3.1

2008-05-13 Thread Scott Marlowe
On Mon, May 12, 2008 at 11:40 PM, Gauri Kanekar <[EMAIL PROTECTED]> wrote: > Hi, > > We want to migrate from postgres 8.1.3 to postgres 8.3.1. > Can anybody list out the installation steps to be followed for migration. > Do we require to take care of something specially. First, I'd recommend updat

Re: [PERFORM] RAID controllers for Postgresql on large setups

2008-05-13 Thread Scott Marlowe
On Mon, May 12, 2008 at 8:04 PM, Francisco Reyes <[EMAIL PROTECTED]> wrote: > Inheritted a number of servers and I am starting to look into the hardware. > > So far what I know from a few of the servers > Redhat servers. > 15K rpm disks, 12GB to 32GB of RAM. > Adaptec 2120 SCSI controller (64MB of