Re: [PERFORM] max fsm pages question

2008-07-09 Thread Vivek Khera
On Jul 8, 2008, at 3:24 PM, Bill Moran wrote: If you don't handle this, that table will continue to grow in size on the disk, taking up space unnecessarily and probably negatively impacting performance. s/probably/definitely/ Also, if it was #3 on Bill's list, one thing to do is look for ind

Re: [PERFORM] Hardware vs Software RAID

2008-06-26 Thread Vivek Khera
On Jun 25, 2008, at 11:35 AM, Matthew Wakeling wrote: On Wed, 25 Jun 2008, Greg Smith wrote: A firewire-attached log device is an extremely bad idea. Anyone have experience with IDE, SATA, or SAS-connected flash devices like the Samsung MCBQE32G5MPP-0VA? I mean, it seems lovely - 32GB, a

Re: [PERFORM] RAID controllers for Postgresql on large setups

2008-05-13 Thread Vivek Khera
On May 12, 2008, at 11:24 PM, Francisco Reyes wrote: Any PCI controller you have had good experience with? How any other PCI-X/PCI-e controller that you have had good results? The LSI controllers are top-notch, and always my first choice. They have PCI-X and PCI-e versions. -- Sent via

Re: [PERFORM] RAID controllers for Postgresql on large setups

2008-05-13 Thread Vivek Khera
On May 12, 2008, at 10:04 PM, Francisco Reyes wrote: Adaptec 2120 SCSI controller (64MB of cache). The servers have mostly have 12 drives in RAID 10. We are going to redo one machine to compare RAID 10 vs RAID 50. Mostly to see if the perfomance is close, the space gain may be usefull.

Re: [PERFORM] Replication Syatem

2008-04-29 Thread Vivek Khera
On Apr 29, 2008, at 10:16 AM, Tom Lane wrote: Greg Smith <[EMAIL PROTECTED]> writes: The model here assumes that you'll need that space again for the next time you UPDATE or INSERT a row. So instead VACUUM just keeps those available for database reuse rather than returning it to the opera

Re: [PERFORM] recommendations for web/db connection pooling or DBD::Gofer reviews

2008-04-11 Thread Vivek Khera
On Apr 10, 2008, at 5:28 PM, Mark Stosberg wrote: So, the front-end proxy would have a number of max connections, say 200, and it would connect to another httpd/mod_perl server behind with a lower number of connections, say 20. If the backend httpd server was busy, the proxy connection to

Re: [PERFORM] migration of 7.4 to 8.1

2008-03-12 Thread Vivek Khera
On Mar 12, 2008, at 2:43 AM, sathiya psql wrote: My question is that how to migrate my database to 7.4 to 8.1 that is not only dumping the db and extracting that in 8.1 .. If i do that whether it will work without problem, or i have to do some manual changes is my question... the pg dump

Re: [PERFORM] How to choose a disc array for Postgresql?

2008-03-03 Thread Vivek Khera
On Mar 3, 2008, at 12:16 AM, Greg Smith wrote: I've collected up many of the past list comments on this subject and put a summary athttp://www.postgresqldocs.org/index.php/SCSI_vs._IDE/SATA_Disks I'll add a recommendation of Partners Data Systems http://www.partnersdata.com/ as a great ven

Re: [PERFORM] How to choose a disc array for Postgresql?

2008-03-03 Thread Vivek Khera
On Mar 2, 2008, at 11:23 PM, Scott Marlowe wrote: And I've never had any of the problems you list with LSI cards. The only issue I've seen is mediocre RAID-10 performance on their cards I don't fault the LSI card. The 320-2X is by far one of the fastest cards I've ever used, and the most

Re: [PERFORM] How to choose a disc array for Postgresql?

2008-03-03 Thread Vivek Khera
On Mar 2, 2008, at 11:02 PM, Steve Poe wrote: It seems the RAID card manufacturers have more to do with failures than the drives themselves. Have you found a RAID card you did not have to drop to U160? The only array for which I've had to drop to U160 on an LSI card is the Dell array. I th

Re: [PERFORM] How to choose a disc array for Postgresql?

2008-03-02 Thread Vivek Khera
On Mar 2, 2008, at 2:37 AM, Steve Poe wrote: I need to consider a vendor for the new disc array (6- to 8 discs). The local vendor (in the San Francisco Bay Area), I've not been completely pleased with, so I am considering using Dell storage connecting to an retail version LSI MegaRAID 320-2X ca

Re: [PERFORM] 12 disks raid setup

2008-02-29 Thread Vivek Khera
On Feb 29, 2008, at 9:51 AM, Franck Routier wrote: my Raid controller is an Adaptec 31205 SAS/RAID controller. The battery was an option, but I didn't know it at purchase time. So I have no battery, but the whole system is on an UPS. Go find one on ebay or google search, and plug it in. Ad

Re: [PERFORM] 7 hrs for a pg_restore?

2008-02-21 Thread Vivek Khera
On Feb 21, 2008, at 12:28 PM, Guillaume Cottenceau wrote: I have made a comparison restoring a production dump with default and large maintenance_work_mem. The speedup improvement here is only of 5% (12'30 => 11'50). At one point I was evaluating several server vendors and did a bunch of DB

Re: [PERFORM] Vacuum and FSM page size

2008-01-23 Thread Vivek Khera
On Jan 23, 2008, at 1:29 PM, Thomas Lozza wrote: We have an installation of Postgres 8.1.2 (32bit on Solaris 9) with a DB size of about 250GB on disk. The DB is subject to fair amount of inserts, deletes and updates per day. Running VACUUM VERBOSE tells me that I should allocate around 20M

Re: [PERFORM] With 4 disks should I go for RAID 5 or RAID 10

2007-12-27 Thread Vivek Khera
On Dec 26, 2007, at 4:28 PM, [EMAIL PROTECTED] wrote: now, if you can afford solid-state drives which don't have noticable seek times, things are completely different ;-) Who makes one with "infinite" lifetime? The only ones I know of are built using RAM and have disk drive backup with in

Re: [PERFORM] With 4 disks should I go for RAID 5 or RAID 10

2007-12-27 Thread Vivek Khera
On Dec 26, 2007, at 10:21 AM, Bill Moran wrote: I snipped the rest of your message because none of it matters. Never use RAID 5 on a database system. Ever. There is absolutely NO reason to every put yourself through that much suffering. If you hate yourself that much just commit suicide,

Re: [PERFORM] dell versus hp

2007-11-15 Thread Vivek Khera
On Nov 14, 2007, at 5:36 PM, Jeff Frost wrote: I believe these were both on ext3. I thought I had some XFS results available for comparison, but I couldn't find them. You'd see similar with the UFS2 file system on FreeBSD. ---(end of broadcast)---

Re: [PERFORM] dell versus hp

2007-11-09 Thread Vivek Khera
On Nov 8, 2007, at 3:56 PM, Alan Hodgson wrote: You can't touch RAID 10 for performance or reliability. The only reason to use RAID 5 or RAID 6 is to get more capacity out of the same drives. Maybe you can't, but I can. I guess I have better toys than you :-) ---(

Re: [PERFORM] dell versus hp

2007-11-08 Thread Vivek Khera
On Nov 8, 2007, at 1:22 PM, Scott Marlowe wrote: I've heard the newest adaptecs, even the perc implementations aren't bad. I have a pair of Adaptec 2230SLP cards. Worst. Just replaced them on Tuesday with fibre channel cards connected to external RAID enclosures. Much nicer. -

Re: [PERFORM] dell versus hp

2007-11-08 Thread Vivek Khera
On Nov 6, 2007, at 1:10 PM, Greg Smith wrote: elsewhere. But once you have enough disks in an array to spread all the load over that itself may improve write throughput enough to still be a net improvement. This has been my expeience with 14+ disks in an array (both RAID10 and RAID5).

Re: [PERFORM] dell versus hp

2007-11-08 Thread Vivek Khera
On Nov 6, 2007, at 5:12 AM, Tore Halset wrote: Here are our current alternatives: Two things I recommend. If the drives are made by western digital, run away. If the PERC5/i is an Adaptec card, run away. Max out your cache RAM on the RAID card. 256 is the minimum when you have such b

Re: [PERFORM] hp ciss on freebsd

2007-11-08 Thread Vivek Khera
On Nov 5, 2007, at 8:19 AM, Claus Guttesen wrote: I will get four 72 GB sas-disks at 15K rpm. Reading the archives suggest raid 1+0 for optimal read/write performance, but with a solid raid-controller raid 5 will also perform very well when reading. If you only have 4 drives, I'd recommend no

Re: [PERFORM] Postgres 7.4.2 hanging when vacuum full is run

2007-09-28 Thread Vivek Khera
On Sep 28, 2007, at 10:28 AM, Radhika S wrote: 20775 ?S 0:00 postgres: abc myDB [local] idle in transaction 20776 ?S 0:00 postgres: abc myDB [local] idle 17509 ?S 0:06 postgres: abc myDB [local] VACUUM waiting 24656 ?S 0:00

Re: [PERFORM] SAN vs Internal Disks

2007-09-07 Thread Vivek Khera
On Sep 6, 2007, at 2:42 PM, Scott Marlowe wrote: I'd recommend against Dell unless you're at a company that orders computers by the hundred lot. My experience with Dell has been that unless you are a big customer you're just another number (a small one at that) on a spreadsheet. I order mayb

Re: [PERFORM] [Solved] Postgres performance problem

2007-08-31 Thread Vivek Khera
On Aug 30, 2007, at 2:08 PM, Mark Lewis wrote: If you're not running regular VACUUMs at all but are instead exclusively running VACUUM FULL, then I don't think you would see warnings about running out of fsm enties, which would explain why you did not notice the bloat. I haven't confirmed th

Re: [PERFORM] Dell Hardware Recommendations

2007-08-13 Thread Vivek Khera
On Aug 10, 2007, at 4:36 PM, Merlin Moncure wrote: I'm not so sure I agree. They are using LSI firmware now (and so is everyone else). The servers are well built (highly subjective, I admit) and configurable. I have had some bad experiences with IBM gear (adaptec controller though), and whit

Re: [PERFORM] Dell Hardware Recommendations

2007-08-10 Thread Vivek Khera
On Aug 9, 2007, at 3:47 PM, Joe Uhl wrote: PowerEdge 1950 paired with a PowerVault MD1000 2 x Quad Core Xeon E5310 16 GB 667MHz RAM (4 x 4GB leaving room to expand if we need to) PERC 5/E Raid Adapter 2 x 146 GB SAS in Raid 1 for OS + logs. A bunch of disks in the MD1000 configured in Raid 10 f

Re: [PERFORM] mid 2007 "best bang for the buck" hardware opinions

2007-08-09 Thread Vivek Khera
On Aug 8, 2007, at 11:34 PM, justin wrote: So whats the thoughts on a current combined rack/disks/cpu combo around the $10k-$15k point, currently? I just put into production testing this setup: SunFire X4100M2 (2x Opteron Dual core) with 20Gb RAM and an LSI PCI-e dual-channel 4Gb Fibre ch

Re: [PERFORM] When/if to Reindex

2007-08-08 Thread Vivek Khera
On Jul 18, 2007, at 1:08 PM, Steven Flatt wrote: Some background: we make extensive use of partitioned tables. In fact, I'm really only considering reindexing partitions that have "just closed". In our simplest/most general case, we have a table partitioned by a timestamp column, each pa

Re: [PERFORM] Vacuum full considered useful ;)

2007-07-14 Thread Vivek Khera
On Jul 14, 2007, at 11:50 AM, Patric de Waha wrote: Yesterday I switched from 8.1 to 8.2. So I needed to dump the dbase and reimport it. The dbase after 4 months of running without "vacuum full" reached 60 gigabyte of diskspace. Now after a fresh import it only has 5 gigabyte!

Re: [PERFORM] PostgreSQL publishes first real benchmark

2007-07-09 Thread Vivek Khera
On Jul 9, 2007, at 1:02 PM, Joshua D. Drake wrote: It is also the reason that those in the know typically ignore all benchmarks and do their own testing. Heresy! ---(end of broadcast)--- TIP 6: explain analyze is your friend

Re: [PERFORM] Best use of second controller with faster disks?

2007-06-14 Thread Vivek Khera
On Jun 13, 2007, at 10:36 PM, Francisco Reyes wrote: FreeBSD, indeed. The vendor, Partners Data Systems, did a wonderful This one? http://www.partnersdata.com that's the one. job ensuring that everything integrated well to the point of talking with various FreeBSD developers, LSI engi

Re: [PERFORM] Performance Testing Utility

2007-06-13 Thread Vivek Khera
On Jun 13, 2007, at 6:25 AM, Christo Du Preez wrote: Is there some kind of performance testing utility available for postgresql Something I can run after installing postgresql to help me identify if my installation is optimal. Your own app is the only one that will give you meaningful resul

Re: [PERFORM] Best use of second controller with faster disks?

2007-06-13 Thread Vivek Khera
On Jun 12, 2007, at 8:33 PM, Francisco Reyes wrote: Vivek Khera writes: what raid card have you got? 2 3ware cards. I believe both are 9550SX i'm playing with an external enclosure which has an areca sata raid in it and connects to the host via fibre channel. What is the OS? Fr

Re: [PERFORM] Best use of second controller with faster disks?

2007-06-12 Thread Vivek Khera
On Jun 11, 2007, at 9:14 PM, Francisco Reyes wrote: RAID card 1 with 8 drives. 7200 RPM SATA RAID10 RAID card 2 with 4 drives. 10K RPM SATA RAID10 what raid card have you got? i'm playing with an external enclosure which has an areca sata raid in it and connects to the host via fibre ch

Re: [PERFORM] max_fsm_pages, shared_buffers and checkpoint_segments

2007-05-31 Thread Vivek Khera
On May 23, 2007, at 4:40 PM, Peter Schuller wrote: Sounds like you need to increase your shared memory limits. Unfortunately this will require a reboot on FreeBSD :( No, it does not. You can tune some of the sysv IPC parameters at runtime. the shmmax and shmall are such parameters.

Re: [PERFORM] does VACUUM ANALYZE complete with this error?

2007-05-23 Thread Vivek Khera
On May 23, 2007, at 9:26 AM, Susan Russo wrote: I've played 'catch up' wrt adjusting max_fsm_pages (seems to be a regular event), however am wondering if the vacuum analyze which reports the error was actually completed? Yes, it completed. However not all pages with open space in them are

Re: [PERFORM] Tips & Tricks for validating hardware/os

2007-05-23 Thread Vivek Khera
On May 23, 2007, at 2:32 AM, Andreas Kostyrka wrote: You forgot pulling some RAID drives at random times to see how the hardware deals with the fact. And how it deals with the rebuild afterwards. (Many RAID solutions leave you with worst of both worlds, taking longer to rebuild than a rest

Re: [PERFORM] performance drop on 8.2.4, reverting to 8.1.4

2007-05-23 Thread Vivek Khera
On May 18, 2007, at 11:40 AM, Liviu Ionescu wrote: 8.1 might have similar problems, but the point here is different: if what was manually tuned to work in 8.1 confuses the 8.2 planner and performance drops so much (from 2303 to 231929 ms in my case) upgrading a production machine to 8.2 i

Re: [PERFORM] 121+ million record table perf problems

2007-05-21 Thread Vivek Khera
On May 18, 2007, at 2:30 PM, Andrew Sullivan wrote: Note also that your approach of updating all 121 million records in one statement is approximately the worst way to do this in Postgres, because it creates 121 million dead tuples on your table. (You've created some number of those by killing

Re: [PERFORM] postgres: 100% CPU utilization

2007-04-23 Thread Vivek Khera
On Apr 23, 2007, at 12:09 PM, Scott Marlowe wrote: And do you have 32 or 64 Megs of memory in that machine? Cause honestly, that's the kinda hardware I was running 7.0.2 on, so you might as well get retro in your hardware department while you're at it. I think you're being too conservati

Re: [PERFORM] Finding bloated indexes?

2007-04-13 Thread Vivek Khera
On Apr 13, 2007, at 4:01 PM, Dan Harris wrote: Is there a pg_stat_* table or the like that will show how bloated an index is? I am trying to squeeze some disk space and want to track down where the worst offenders are before performing a global REINDEX on all tables, as the database is rou

Re: [PERFORM] how small to split a table?

2007-03-20 Thread Vivek Khera
On Mar 20, 2007, at 11:20 AM, Heiko W.Rupp wrote: partition through the master table abould halfed the speed with 4 partitions and made a 50% increase for 2 partitions. Please note: this is not representative in any kind! I fully intend to build knowledge of the partitions into the insert

[PERFORM] how small to split a table?

2007-03-20 Thread Vivek Khera
I've got one logging table that is over 330 million rows to store 6 months' worth of data. It consists of two integers and a 4-character long string. I have one primary key which is the two integers, and an additional index on the second integer. I'm planning to use inheritance to split t

Re: [PERFORM] Configuration settings for 32GB RAM server

2006-12-04 Thread Vivek Khera
On Dec 4, 2006, at 12:10 PM, Mark Lonsdale wrote: - 4 physical CPUs (hyperthreaded to 8) i'd tend to disable hyperthreading on Xeons... shared_buffers – 50,000 - >From what Id read, increasing this number higher than this wont have any advantages ? if you can, increase it until you

Re: [PERFORM] VACUUMs take twice as long across all nodes

2006-11-02 Thread Vivek Khera
On Oct 27, 2006, at 2:07 PM, Tom Lane wrote: 8.2, but in existing releases I can't see much you can do about it except REINDEX when things get slow. This will be so nice for me. I have one huge table with a massive amount of churn and bulk deletes. I have to reindex it once every other

Re: [PERFORM] New hardware thoughts

2006-10-23 Thread Vivek Khera
On Oct 23, 2006, at 5:08 PM, Joshua D. Drake wrote: They don't randomly change the controllers under the same name. If you order a PERC4e/Si controller you will get the same controller every time. Actually Vivek this isn't true. Yes the hardware will likely be the same, but the firmware

Re: [PERFORM] Optimizing disk throughput on quad Opteron

2006-10-23 Thread Vivek Khera
On Oct 23, 2006, at 4:59 PM, Joshua D. Drake wrote: If I had $50k budget, I'd be buying the SunFire X4500 and running Solaris + ZFS on it. However, you're limited to 2 dual core Opterons, it seems. The HP 585 will give you quad dual core :) but can you sling the bits to and from the dis

Re: [PERFORM] New hardware thoughts

2006-10-23 Thread Vivek Khera
On Oct 20, 2006, at 10:58 AM, Dave Cramer wrote: My advice is to find another supplier. check the archives for Dell. Not necessarily bad to go with Dell. There are *some* of their controllers that are wicked fast in some configurations. However, finding which ones are fast is very trick

Re: [PERFORM] Optimizing disk throughput on quad Opteron

2006-10-23 Thread Vivek Khera
On Oct 21, 2006, at 11:43 AM, John Philips wrote: Can you guys see any glaring bottlenecks in my layout? Any other suggestions to offer (throw in more controllers, different RAID layout, etc.)? Our budget limit is $50k. If I had $50k budget, I'd be buying the SunFire X4500 and running Sol

Re: [PERFORM] Opteron vs. Xeon "benchmark"

2006-09-22 Thread Vivek Khera
On Sep 22, 2006, at 4:58 AM, nicky wrote: till 100 simultaneous visitors, the Xeon performs 24% better with MSQL 4.1.20, 30% better in MySQL 5.0.20a and 37% better in PostgreSQL 8.2-dev. In short, the Socket F Opteron doesn't stand a chance, although the Woodcrest scales better and has suc

Re: [PERFORM] performance problems.

2006-08-31 Thread Vivek Khera
On Aug 31, 2006, at 3:08 PM, Tom Lane wrote: Vivek Khera <[EMAIL PROTECTED]> writes: Curious... See Message-ID: <[EMAIL PROTECTED]> from the October 2003 archives. (I'd provide a full link to it, but the http://archives.postgresql.org/pgsql-performance/ archives are botch

Re: [PERFORM] performance problems.

2006-08-31 Thread Vivek Khera
On Aug 30, 2006, at 7:48 PM, Dave Cramer wrote: Actually unless you have a ram disk you should probably leave random_page_cost at 4, shared buffers should be 2x what you have here, maintenance work mem is pretty high effective cache should be much larger 3/4 of 4G or about 36 I've be

Re: [PERFORM] performance problems.

2006-08-31 Thread Vivek Khera
On Aug 30, 2006, at 12:26 PM, Jim C. Nasby wrote: You misunderstand how effective_cache_size is used. It's the *only* memory factor that plays a role in cost estimator functions. This means it should include the memory set aside for caching in shared_buffers. Also, hibufspace is only talkin

Re: [PERFORM] performance problems.

2006-08-30 Thread Vivek Khera
= 40 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Vivek Khera, Ph.D.MailerMailer, LLC Rockville, MD http://www.MailerMailer.com/ +1-301-869-4449 x806 smime.p7s Description: S/MIME cryptographic signature

Re: [PERFORM] Dell PowerEdge 2950 performance

2006-08-15 Thread Vivek Khera
On Aug 15, 2006, at 4:21 PM, Bucky Jordan wrote: ... from Vivek... which is an issue with freebsd and bonnie++ since it doesn't know that freebsd can use large files natively (ie, no large file hacks necessary). the freebsd port of bonnie takes care of this, if you use that instead of compilin

Re: [PERFORM] Dell PowerEdge 2950 performance

2006-08-15 Thread Vivek Khera
On Aug 15, 2006, at 2:50 PM, Luke Lonergan wrote: I don't know why I missed this the first time - you need to let bonnie++ pick the file size - it needs to be 2x memory or the results you get will not be accurate. which is an issue with freebsd and bonnie++ since it doesn't know that fr

Re: [PERFORM] Dell PowerEdge 2950 performance

2006-08-14 Thread Vivek Khera
On Aug 14, 2006, at 3:56 PM, Bucky Jordan wrote: Seems to me the PERC5 has issues with layered raid (10, 50) as others have suggested on this list is a common problem with lower end raid cards. For now, I'm going with the RAID 5 option, however if I have time, I would like to test having t

Re: [PERFORM] Dell PowerEdge 2950 performance

2006-08-14 Thread Vivek Khera
On Aug 9, 2006, at 11:56 AM, Bucky Jordan wrote:Here’s the hardware:2xDual Core 3.0 Ghz CPU (Xeon 5160- 1333Mhz FSB, 4 MB shared cache per socket)8 GB RAM (DDR2, fully buffered, Dual Ranked, 667 Mhz)6x300 10k RPM SAS drivesPerc 5i w/256 MB battery backed cacheIs the PERC 5/i dual channel?  If so, a

Re: [PERFORM] Performances with new Intel Core* processors

2006-07-31 Thread Vivek Khera
On Jul 31, 2006, at 12:30 PM, Arjen van der Meijden wrote: For a database system, however, processors hardly ever are the main bottleneck, are they? So you should probably go for a set of "fast processors" from your favorite supplier and focus mainly on lots of memory and fast disks. Wheth

Re: [PERFORM] suggested RAID controller for FreeBSD 6.1 + PostgreSQL 8.1

2006-07-10 Thread Vivek Khera
On Jul 5, 2006, at 7:46 PM, Kenji Morishige wrote: like to know what an ideal RAID controller that would be compatible with FreeBSD 6.1 would be these days. LSI MegaRAID 320-2X and put half disks on one channel, half on the other, and MIRROR+STRIPE them (ie, RAID10). There is nothing fa

Re: [PERFORM] Opteron/FreeBSD/PostgreSQL performance poor

2006-07-05 Thread Vivek Khera
On Jul 5, 2006, at 10:43 AM, andy rost wrote: We're in the process of porting from Informix 9.4 to PostgreSQL 8.1.3. Our PostgreSQL server is an AMD Opteron Dual Core 275 with two 2.2 Ghz 64-bit processors. There are two internal drives and an external enclosure containing 14 drives (confi

Re: [PERFORM] Which processor runs better for Postgresql?

2006-06-15 Thread Vivek Khera
On Jun 15, 2006, at 1:10 PM, Steve Poe wrote: Vivek, Thanks for your feedback. Which Dell server did you purchase? I have many many dell rackmounts: 1550, 1650, 1750, 1850, and SC1425 and throw in a couple of 2450. I *really* like the 1850 with built-in SCSI RAID. It is fast enough t

Re: [PERFORM] Which processor runs better for Postgresql?

2006-06-15 Thread Vivek Khera
On Jun 13, 2006, at 2:02 PM, Steve Poe wrote: Can anyone share what their experience has been with Intel's dual core CPUs and/or Dell's new servers? I'm one of the few Dell fans around here... but I must say that I don't buy them for my big DB servers specifically since they don't curren

Re: [PERFORM] [GENERAL] Arguments Pro/Contra Software Raid

2006-05-10 Thread Vivek Khera
On May 10, 2006, at 12:41 AM, Greg Stark wrote: Well, dollar for dollar you would get the best performance from slower drives anyways since it would give you more spindles. 15kRPM drives are *expensive*. Personally, I don't care that much for "dollar for dollar" I just need performance.

Re: [PERFORM] Arguments Pro/Contra Software Raid

2006-05-09 Thread Vivek Khera
On May 9, 2006, at 11:51 AM, Joshua D. Drake wrote: Sorry that is an extremely misleading statement. SATA RAID is perfectly acceptable if you have a hardware raid controller with a battery backup controller. And dollar for dollar, SCSI will NOT be faster nor have the hard drive capacity

Re: [PERFORM] Memory and/or cache issues?

2006-05-08 Thread Vivek Khera
On May 8, 2006, at 1:30 PM, Jim C. Nasby wrote: Yeah, I prefer my surgeons to work this way too. training is for the birds. I think you read too quickly past the part where Tim said he'd taking a week-long training class. s/training/apprenticeship/g; ---(end of

Re: [PERFORM] Memory and/or cache issues?

2006-05-08 Thread Vivek Khera
On May 6, 2006, at 10:53 AM, mcelroy, tim wrote: development side than DBA by the way), but there is no better way to learn and understand better than actual day-to-day working experience. Yeah, I prefer my surgeons to work this way too. training is for the birds. smime.p7s Descripti

Re: [PERFORM] Slow restoration question

2006-05-03 Thread Vivek Khera
On May 3, 2006, at 9:19 AM, Jeff Trout wrote: Bonnie++ is able to use very large datasets. It also tries to figure out hte size you want (2x ram) - the original bonnie is limited to 2GB. but you have to be careful building bonnie++ since it has bad assumptions about which systems can do

Re: [PERFORM] postgresql transaction id monitoring with nagios

2006-05-02 Thread Vivek Khera
On May 2, 2006, at 3:03 PM, Alvaro Herrera wrote: Something seems wrong... I just ran your script against my development database server which is vacuumed daily and it said I was 53% of the way to 2B. Seemed strange to me, so I re-ran "vacuum -a - z" to vacuum all databases (as superuser), rer

Re: [PERFORM] postgresql transaction id monitoring with nagios

2006-05-02 Thread Vivek Khera
On May 2, 2006, at 2:26 PM, Tony Wasson wrote: The script detects a wrap at 2 billion. It starts warning once one or more databases show an age over 1 billion transactions. It reports critical at 1.5B transactions. I hope everyone out there is vacuuming *all* databases often. Something seems

Re: [PERFORM] hardare config question

2006-05-01 Thread Vivek Khera
On May 1, 2006, at 1:58 PM, Erik Myllymaki wrote: Of course now i am in a dangerous situation - using volatile write cache without a BBU. It should be against the law to make RAID cards with caches that are not battery backed. If I were to use a UPS to ensure a soft shutdown in the eve

Re: [PERFORM] hardare config question

2006-04-28 Thread Vivek Khera
On Apr 28, 2006, at 11:37 AM, Erik Myllymaki wrote: When I had this installed on a single SATA drive running from the PE1800's on-board SATA interface, this operation took anywhere from 65-80 seconds. With my new RAID card and drives, this operation took 272 seconds!? switch it to RAID10

Re: [PERFORM] Large (8M) cache vs. dual-core CPUs

2006-04-27 Thread Vivek Khera
On Apr 25, 2006, at 5:09 PM, Ron Peacetree wrote: ...and even if you do buy Intel, =DON"T= buy Dell unless you like causing trouble for yourself. Bad experiences with Dell in general and their poor PERC RAID controllers in specific are all over this and other DB forums. I don't think that

Re: [PERFORM] Large (8M) cache vs. dual-core CPUs

2006-04-27 Thread Vivek Khera
On Apr 25, 2006, at 2:14 PM, Bill Moran wrote: Where I'm stuck is in deciding whether we want to go with dual-core pentiums with 2M cache, or with HT pentiums with 8M cache. In order of preference: Opterons (dual core or single core) Xeon with HT *disabled* at the BIOS level (dual or single

Re: [PERFORM] Inserts optimization?

2006-04-20 Thread Vivek Khera
On Apr 14, 2006, at 8:00 AM, Marc Cousin wrote: So, you'll probably end up being slowed down by WAL fsyncs ... and you won't have a lot of solutions. Maybe you should start with trying to set fsync=no as a test to confirm that (you should have a lot of iowaits right now if you haven't dis

Re: [PERFORM] Inserts optimization?

2006-04-20 Thread Vivek Khera
On Apr 13, 2006, at 2:59 PM, Francisco Reyes wrote: This particular server is pretty much what I inherited for now for this project.and its Raid 5. There is a new server I am setting up soon... 8 disks which we are planning to setup 6 disks in RAID 10 2 Hot spares In RAID 10 would it matte

Re: [PERFORM] Restore performance?

2006-04-10 Thread Vivek Khera
On Apr 10, 2006, at 3:55 AM, Jesper Krogh wrote: I'd run pg_dump | gzip > sqldump.gz on the old system. That took about 30 hours and gave me an 90GB zipped file. Running cat sqldump.gz | gunzip | psql into the 8.1 database seems to take about the same time. Are there any tricks I can use to

Re: [PERFORM] Sun Fire T2000 and PostgreSQL 8.1.3

2006-04-06 Thread Vivek Khera
On Apr 6, 2006, at 12:47 AM, Leigh Dyer wrote: I'm sure those little SAS drives would be great for web servers and other non-IO-intensive tasks though -- I'd love to get some X4100s in to replace our Poweredge 1750s for that. It's a smart move overall IMHO, For this purpose, bang for the

Re: [PERFORM] Sun Fire T2000 and PostgreSQL 8.1.3

2006-04-06 Thread Vivek Khera
On Apr 5, 2006, at 9:11 PM, Marcelo Tada wrote: What are you think about the Sun Fire X64 X4200 Server? I use the X4100 and like it a lot. I'm about to buy another. I see no advantage to the X4200 unless you want the extra internal disks. I use an external array. ---

Re: [PERFORM] Sun Fire T2000 and PostgreSQL 8.1.3

2006-04-06 Thread Vivek Khera
On Apr 5, 2006, at 5:58 PM, August Zajonc wrote: Most involve some AMD Opertons, lots of spindles with a good raid controller preferred to one or two large disks and a good helping of ram. Be interesting to get some numbers on the sunfire machine. I can highly recommend the SunFire X4100, how

Re: [PERFORM] freebsd/softupdates for data dir

2006-04-06 Thread Vivek Khera
On Apr 5, 2006, at 6:07 PM, Jim Nasby wrote: More importantly, it allows the system to come up and do fsck in the background. If you've got a large database that's a pretty big benefit. That's a UFS2 feature, not a soft-updates feature. ---(end of broadcast)

Re: [PERFORM] freebsd/softupdates for data dir

2006-04-04 Thread Vivek Khera
On Apr 3, 2006, at 10:10 PM, Mark Kirkwood wrote: I've always left them on, and never had any issues...(even after unscheduled power loss - which happened here yesterday). As I understand it, the softupdate code reorders *metadata* operations, and does not alter data operations - so the ef

Re: [PERFORM] Decide between Postgresql and Mysql (help of

2006-03-29 Thread Vivek Khera
On Mar 28, 2006, at 11:55 AM, Marcos wrote: The application will be a chat for web, the chats will be stored in the server. In a determined interval of time... more or less 2 seconds, the application will be looking for new messages. We bought software for this purpose (phplive). It is b

Re: [PERFORM] Decide between Postgresql and Mysql (help of

2006-03-29 Thread Vivek Khera
On Mar 28, 2006, at 1:59 PM, Scott Marlowe wrote: Generally you'll find the PostgreSQL gotchas are of the sort that make you go "oh, that's interesting" and the MySQL gotchas are the kind that make you go "Dear god, you must be kidding me!" But that's just my opinion, I could be wrong. I

Re: [PERFORM] Decide between Postgresql and Mysql (help of comunity)

2006-03-29 Thread Vivek Khera
On Mar 28, 2006, at 1:57 PM, Madison Kelly wrote: From what I understand, PostgreSQL is designed with stability and reliability as key tenants. MySQL favors performance and ease of use. An From my point of view, mysql favors single-user performance over all else. Get into multiple upda

pgsql-performance@postgresql.org

2006-03-21 Thread Vivek Khera
On Mar 21, 2006, at 12:59 PM, Jim C. Nasby wrote: atapci1: And note that this is using FreeBSD gmirror, not the built-in raid controller. I get similar counter-intuitive slowdown with gmirror SATA disks on an IBM e326m I'm evaluating. If/when I buy one I'll get the onboard SCSI RAID in

Re: [PERFORM] Best OS & Configuration for Dual Xeon w/4GB & Adaptec

2006-03-21 Thread Vivek Khera
On Mar 21, 2006, at 2:04 PM, PFC wrote: especially since I have desktop PCI and the original poster has a real server with PCI-X I think. that was me :-) but yeah, I never seem to get full line speed for some reason. i don't know if it is because of inadequate measurement tools or what..

Re: [PERFORM] Best OS & Configuration for Dual Xeon w/4GB & Adaptec

2006-03-21 Thread Vivek Khera
On Mar 20, 2006, at 6:27 PM, PFC wrote: Expensive SCSI hardware RAID cards with expensive 10Krpm harddisks should not get humiliated by such a simple (and cheap) setup. (I'm referring to the 12-drive RAID10 mentioned before, not the other one which was a simple 2-disk mirror). Toms hardwa

Re: [PERFORM] Best OS & Configuration for Dual Xeon w/4GB & Adaptec

2006-03-21 Thread Vivek Khera
On Mar 21, 2006, at 6:03 AM, Mark Kirkwood wrote: The so-called limit (controllable via various sysctl's) is on the amount of memory used for kvm mapped pages, not cached pages, i.e - its a subset of the cached pages that are set up for immediate access (the Thanks... now that makes sens

Re: [PERFORM] Best OS & Configuration for Dual Xeon w/4GB & Adaptec RAID 2200S

2006-03-20 Thread Vivek Khera
On Mar 20, 2006, at 6:04 PM, Miguel wrote: Umm, in my box i see better seektimes but worst transfer rates, does it make sense? i think i have something wrong, the question i cant answer is what tunning am i missing? Well, I forgot to mention I have 15k RPM disks, so the transfers should

Re: [PERFORM] Best OS & Configuration for Dual Xeon w/4GB & Adaptec RAID 2200S

2006-03-20 Thread Vivek Khera
If you do put on FreeBSD 6, I'd love to see the output of "diskinfo - v -t" on your RAID volume(s). Not directly related ... i have a HP dl380 g3 with array 5i controlled (1+0), these are my results [...] is this good enough? Is that on a loaded box or a mostly quiet box? Those number se

Re: [PERFORM] 1 TB of memory

2006-03-20 Thread Vivek Khera
On Mar 20, 2006, at 2:44 PM, Merlin Moncure wrote: For my use it was worth the price. However, given the speed increase of other components since then, I don't think I'd buy one today. Parallelism (if you can do it like Luke suggested) is the way to go. Thats an interesting statement. My pe

Re: [PERFORM] Best OS & Configuration for Dual Xeon w/4GB & Adaptec RAID 2200S

2006-03-20 Thread Vivek Khera
On Mar 17, 2006, at 5:11 PM, Kenji Morishige wrote: In summary, my questions: 1. Would running PG on FreeBSD 5.x or 6.x or Linux improve performance? FreeBSD 6.x will definitely get you improvements. Many speedup improvements have been made to both the generic disk layer and the speci

Re: [PERFORM] 1 TB of memory

2006-03-20 Thread Vivek Khera
On Mar 17, 2006, at 5:07 PM, Scott Marlowe wrote: Open Source SSD via iSCSI with commodity hardware... hmmm. sounds like a useful project. sh! don't give away our top secret plans! ---(end of broadcast)--- TIP 6: explain analyze is yo

Re: [PERFORM] 1 TB of memory

2006-03-20 Thread Vivek Khera
On Mar 17, 2006, at 8:55 AM, Merlin Moncure wrote: I like their approach...ddr ram + raid sanity backup + super reliable power system. Their prices are on jupiter (and i dont mean jupiter, fl) but hopefully there will be some competition and the invetible Nothing unique to them. I have a 4

Re: [PERFORM] Vacuum template databases, Urgent: Production probl

2006-03-14 Thread Vivek Khera
On Mar 14, 2006, at 4:19 PM, mcelroy, tim wrote: Humm, well I am running 8.0.1 and use that option and see the following in my vacuum output log: vacuumdb: vacuuming database "template1" it has done so since at least 7.4, probably 7.3. the "-a" flag really does what is says. ---

Re: [PERFORM] Reliability recommendations

2006-02-24 Thread Vivek Khera
On Feb 24, 2006, at 11:32 AM, Scott Marlowe wrote: My bad experiences were with the 2600 series machines. We now have some 2800 and they're much better than the 2600/2650s I've used in the past. Yes, the 2450 and 2650 were CRAP disk performers. I haven't any 2850 to compare, just an 18

Re: [PERFORM] Reliability recommendations

2006-02-24 Thread Vivek Khera
On Feb 24, 2006, at 9:29 AM, Bruce Momjian wrote: Dell often says part X is included, but part X is not the exact same as part X sold by the original manufacturer. To hit a specific price point, Dell is willing to strip thing out of commodity hardware, and often does so even when performance

Re: [PERFORM] Looking for a tool to "*" pg tables as ERDs

2006-02-23 Thread Vivek Khera
On Feb 23, 2006, at 11:38 AM, Ron Peacetree wrote: Where "*" == {print | save to PDF | save to format | display on screen} Anyone know of one? There's a perl module, GraphViz::DBI::General, which does a rather nifty job of taking a schema and making a graphviz "dot" file from it, which

  1   2   3   >