Re: [PERFORM] Dell Hardware Recommendations

2007-08-09 Thread david
On Thu, 9 Aug 2007, Decibel! wrote: On Thu, Aug 09, 2007 at 08:58:19PM -0500, Scott Marlowe wrote: On Thu, Aug 09, 2007 at 05:29:18PM -0500, Scott Marlowe wrote: On 8/9/07, Decibel! <[EMAIL PROTECTED]> wrote: Also, a good RAID controller can spread reads out across both drives in each mirror

[PERFORM] Implementing an regex filter

2007-08-09 Thread Enrico Weigelt
Hi folks, I've coded an auction crawler, which queries several auction platforms for user-defined keywords. The results are then put into queues an sorted out from there. For example each auction query can be passed along with an maxprice value, which is put into the result records. Every few

Re: [PERFORM] Update table performance

2007-08-09 Thread Piotr Kołaczkowski
Hi, update valley set test='this is a test' Such query updates ALL of your records in the table. 5 million records * 47 fields - that can be several gigabytes of data. The system has to scan that gigabytes to change every record. This is a huge task. Try vacuuming and see if it helps. It can he

[PERFORM] select count(*) performance

2007-08-09 Thread runic
Hello Group, I'm new in PostgreSQL Business, therefore please forgive me a "newbie" Question. I have a table with ca. 1.250.000 Records. When I execute a "select count (*) from table" (with pgAdmin III) it takes about 40 secs. I think that takes much to long. Can you please give me hints, where I

Re: [PERFORM] select count(*) performance

2007-08-09 Thread slawekj
> I'm new in PostgreSQL Business, therefore please forgive me a "newbie" > Question. I have a table with ca. 1.250.000 Records. When I execute > a "select count (*) from table" (with pgAdmin III) it takes about 40 > secs. > I think that takes much to long. Can you please give me hints, where > I c

[PERFORM] Performance on writable views

2007-08-09 Thread Enrico Weigelt
Hi folks, I'm often using writable views as interfaces to clients, so they only see "virtual" objects and never have to cope with the actual storage, ie. to give some client an totally denormalized view of certain things, containing only those information required for certain kind of operatio

Re: [PERFORM] Update table performance

2007-08-09 Thread Kenneth Marshall
Mark, You are not alone in the fact that when you post your system specifications, CPU and memory are always listed while the disk I/O subsystem invariably is not. This is a very disk intensive operation and I suspect that your disk system is maxed-out. If you want it faster, you will need more I/

Re: [PERFORM] Dell Hardware Recommendations

2007-08-09 Thread Greg Smith
On Thu, 9 Aug 2007, Joe Uhl wrote: The MD1000 holds 15 disks, so 14 disks + a hot spare is the max. With 12 250GB SATA drives to cover the 1.5TB we would be able add another 250GB of usable space for future growth before needing to get a bigger set of disks. 500GB drives would leave alot mor

Re: [PERFORM] Dell Hardware Recommendations

2007-08-09 Thread Decibel!
On Thu, Aug 09, 2007 at 08:58:19PM -0500, Scott Marlowe wrote: > > On Thu, Aug 09, 2007 at 05:29:18PM -0500, Scott Marlowe wrote: > > > On 8/9/07, Decibel! <[EMAIL PROTECTED]> wrote: > > > > > > > Also, a good RAID controller can spread reads out across both drives in > > > > each mirror on a RAID1

Re: [PERFORM] Dell Hardware Recommendations

2007-08-09 Thread Scott Marlowe
oops, the the wrong list... now the right one. On 8/9/07, Decibel! <[EMAIL PROTECTED]> wrote: > You forgot the list. :) > > On Thu, Aug 09, 2007 at 05:29:18PM -0500, Scott Marlowe wrote: > > On 8/9/07, Decibel! <[EMAIL PROTECTED]> wrote: > > > > > Also, a good RAID controller can spread reads out

Re: [PERFORM] Dell Hardware Recommendations

2007-08-09 Thread Joe Uhl
Thanks for the input. Thus far we have used Dell but I would certainly be willing to explore other options. I found a "Reference Guide" for the MD1000 from April, 2006 that includes info on the PERC 5/E at: http://www.dell.com/downloads/global/products/pvaul/en/pvaul_md1000_solutions_guide.pdf

Re: [PERFORM] Dell Hardware Recommendations

2007-08-09 Thread Arjen van der Meijden
On 9-8-2007 23:50 Merlin Moncure wrote: Where the extra controller especially pays off is if you have to expand to a second tray. It's easy to add trays but installing controllers on a production server is scary. For connectivity-sake that's not a necessity. You can either connect (two?) extr

Re: [PERFORM] Dell Hardware Recommendations

2007-08-09 Thread Decibel!
On Thu, Aug 09, 2007 at 05:50:10PM -0400, Merlin Moncure wrote: > Raid 10 is usually better for databases but in my experience it's a > roll of the dice. If you factor cost into the matrix a SAS raid 05 > might outperform a SATA raid 10 because you are getting better storage > utilization out of t

Re: [PERFORM] Dell Hardware Recommendations

2007-08-09 Thread Merlin Moncure
On 8/9/07, Joe Uhl <[EMAIL PROTECTED]> wrote: > We have a 30 GB database (according to pg_database_size) running nicely > on a single Dell PowerEdge 2850 right now. This represents data > specific to 1 US state. We are in the process of planning a deployment > that will service all 50 US states.

Re: [PERFORM] Dell Hardware Recommendations

2007-08-09 Thread Decibel!
On Thu, Aug 09, 2007 at 03:47:09PM -0400, Joe Uhl wrote: > We have a 30 GB database (according to pg_database_size) running nicely > on a single Dell PowerEdge 2850 right now. This represents data > specific to 1 US state. We are in the process of planning a deployment > that will service all 50

[PERFORM] Dell Hardware Recommendations

2007-08-09 Thread Joe Uhl
We have a 30 GB database (according to pg_database_size) running nicely on a single Dell PowerEdge 2850 right now. This represents data specific to 1 US state. We are in the process of planning a deployment that will service all 50 US states. If 30 GB is an accurate number per state that means t

Re: [PERFORM] Update table performance

2007-08-09 Thread Decibel!
On Thu, Aug 09, 2007 at 06:04:09PM +0530, Merlin Moncure wrote: > That said, 'update' is the slowest operation for postgresql relative > to other databases that are not MVCC. Actually, it depends on how you do MVCC. In Oracle, DELETE is actually the most expensive operation, because they have to n

Re: [PERFORM] mid 2007 "best bang for the buck" hardware opinions

2007-08-09 Thread Vivek Khera
On Aug 8, 2007, at 11:34 PM, justin wrote: So whats the thoughts on a current combined rack/disks/cpu combo around the $10k-$15k point, currently? I just put into production testing this setup: SunFire X4100M2 (2x Opteron Dual core) with 20Gb RAM and an LSI PCI-e dual-channel 4Gb Fibre ch

Re: [PERFORM] When/if to Reindex

2007-08-09 Thread Steven Flatt
On 8/8/07, Tom Lane <[EMAIL PROTECTED]> wrote: > > I'd make the same remark about Steven's case: if possible, don't create > the indexes at all until you've loaded the table fully. We considered this, however in some of our 12-hour partitions, there are upwards of 50 or 60 million rows near the e

Re: [PERFORM] Update table performance

2007-08-09 Thread Trevor Talbot
On 8/9/07, Michael Stone <[EMAIL PROTECTED]> wrote: > On Thu, Aug 09, 2007 at 06:04:09PM +0530, Merlin Moncure wrote: > >keep an eye for the HOT feature which will hopefully make 8.3 that > >will highly reduce the penalty for (small) updates in many cases. > > Is there an overview somewhere about h

Re: [PERFORM] Update table performance

2007-08-09 Thread Michael Stone
On Thu, Aug 09, 2007 at 06:04:09PM +0530, Merlin Moncure wrote: keep an eye for the HOT feature which will hopefully make 8.3 that will highly reduce the penalty for (small) updates in many cases. Is there an overview somewhere about how this feature works and what it is expected to do? There

Re: [PERFORM] When/if to Reindex

2007-08-09 Thread Bill Moran
In response to "Decibel!" <[EMAIL PROTECTED]>: > On Wed, Aug 08, 2007 at 03:27:57PM -0400, Bill Moran wrote: > > I've had similar experience. One thing you didn't mention that I've noticed > > is that VACUUM FULL often bloats indexes. I've made it SOP that > > after application upgrades (which u

Re: [PERFORM] Update table performance

2007-08-09 Thread Merlin Moncure
On 8/8/07, Mark Makarowsky <[EMAIL PROTECTED]> wrote: > Can you provide more detail on what you mean by your > two suggestions below: > > Yeah, I've used "vertical partitioning" very > successfully in the past, though I've never done it > for just a single field. I'll typically leave the few > most