Re: [PERFORM] Follow-Up: How to improve db performance with $7K?
Vivek Khera wrote: On Mar 31, 2005, at 9:01 PM, Steve Poe wrote: Now, we need to purchase a good U320 RAID card now. Any suggestions for those which run well under Linux? Not sure if it works with linux, but under FreeBSD 5, the LSI MegaRAID cards are well supported. You should be able to pick up a 320-2X with 128Mb battery backed cache for about $1k. Wicked fast... I'm suprized you didn't go for the 15k RPM drives for a small extra cost. Wow, okay, so I'm not sure where everyone's email went, but I got over a weeks worth of list emails at once. Several of you have sent me requests on where we purchased our systems at. Compsource was the vendor, www.c-source.com or www.compsource.com.The sales rep we have is Steve Taylor or you can talk to the sales manager Tom.I've bought hardware from them for the last 2 years and I've been very pleased. I'm sorry wasn't able to respond sooner. Steve, The LSI MegaRAID cards are where its at. I've had -great- luck with them over the years. There were a few weird problems with a series awhile back where the linux driver needed tweaked by the developers along with a new bios update. The 320 series is just as Vivek said, wicked fast. Very strong cards. Be sure though when you order it to specificy the battery backup either with it, or make sure you buy the right one for it. There are a couple of options with battery cache on the cards that can trip you up. Good luck on your systems! Now that I've got my email problems resolved I'm definitely more than help to give any information you all need. ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PERFORM] How to improve db performance with $7K?
You can purchase a whole new dual opteron 740,with 6 gigs of ram, a case to match and 6 74 gig ultra320 sca drives for about $7k I know because that's what I bought one for 2 weeks ago. Using Tyan's dual board. If you need some details and are willing to go that route, let me know and I'll get you the information. Sincerely, Will LaShell Steve Poe wrote: Situation: An 24/7 animal hospital (100 employees) runs their business on Centos 3.3 (RHEL 3) Postgres 7.4.2 (because they have to) off a 2-CPU Xeon 2.8MHz, 4GB of RAM, (3) SCSI disks RAID 0 (zcav value 35MB per sec). The databse is 11GB comprised over 100 tables and indexes from 1MB to 2GB in size. I recently told the hospital management team worst-case scenerio they need to get the database on its own drive array since the RAID0 is a disaster wating to happen. I said ideally a new dual AMD server with 6/7-disk configuration would be ideal for safety and performance, but they don't have $15K. I said a seperate drive array offer the balance of safety and performance. I have been given budget of $7K to accomplish a safer/faster database through hardware upgrades. The objective is to get a drive array, but I can use the budget any way I see fit to accomplish the goal. Since I am a dba novice, I did not physically build this server, nor did I write the application the hospital runs on, but I have the opportunity to make it better, I'd thought I should seek some advice from those who have been down this road before. Suggestions/ideas anyone? Thanks. Steve Poe ---(end of broadcast)--- TIP 8: explain analyze is your friend ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [PERFORM] Help with count(*)
Hannu Krosing wrote: Christopher Browne kirjutas R, 14.11.2003 kell 16:13: Martha Stewart called it a Good Thing when [EMAIL PROTECTED] (Rajesh Kumar Mallah) wrote: INFO: "profiles": found 0 removable, 369195 nonremovable row versions in 43423 pages DETAIL: 246130 dead row versions cannot be removed yet. Nonremovable row versions range from 136 to 2036 bytes long. It seems as though you have a transaction open that is holding onto a whole lot of old rows. I have seen this happen somewhat-invisibly when a JDBC connection manager opens transactions for each connection, and then no processing happens to use those connections for a long time. The open transactions prevent vacuums from doing any good... Can't the backend be made to delay the "real" start of transaction until the first query gets executed ? That seems counter intuitive doesn't it? Why write more code in the server when the client is the thing that has the problem? Will ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PERFORM] RedHat Enterprise Linux ES 3 ?!?!
On Thu, 2003-10-23 at 08:44, Robert Treat wrote:
> On Thu, 2003-10-23 at 11:27, Will LaShell wrote:
> > > Also, I cannot find any list of packages included in Enterprise Linux
> > 2.1 /
> > > 3. Does anyone know if PostgreSQL is included and if so, what version?
> >
> > You have two options as I understand it for PG under RHEL. You can
> > install the PG source from Postgres themselves, or you can use the
> > Postgresql Red Hat Edition. Bruce I think can give you more information
> > on this product. http://sources.redhat.com/rhdb/index.html This is the
> > link to it.
> >
>
> Bruce works for SRA, not Red Hat, so he's probably not your best option
> to talk to on PRHE... While there are Red Hat employees floating around
Gah that's right. *beats self*
> these lists, I'd first suggest reading over the website and then either
> emailing the PRHE lists or one of it's team members depending on the
> specifics of any questions.
Don't forget you can always call the RedHat sales people as well. They
usually have good product knowledge especially since you are talking
about the Advanced Server lines.
> Robert Treat
> --
> Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL
Will
signature.asc
Description: This is a digitally signed message part
Re: [PERFORM] RedHat Enterprise Linux ES 3 ?!?!
On Thu, 2003-10-23 at 01:40, Alexander Priem wrote: > Hi guys, > > This basically continues the other thread about the PERC4 RAID controller, > but since it is a bit off-topic I thought to start another thread. Thanks > for all your help so far :) > > Earlier today I read about the newly released RedHat Enterprise Linux ES > version 3. This version should include out-of-the-box megaraid_2 drivers, so > it would support the Dell PERC4/Di RAID controller. > > However, it is very much more expensive than RedHat Linux 9. RH Linux 9 is > free and the Enterpise ES edition will cost between 400 and several 1.000's > of dollars, depending on the support you want to go with it. > > Do any of you guys have experience with the previous version of Enterprise > Linux (that would be version 2.1) or even better, are any of you already > using version 3? > > Would you recommend this over RedHat Linux 9? I think that with RH Linux 9 > it would be easier to get all the latest versions of components I need (RPMs > for PostgreSQL, Apache, Samba etc.), while my guess would be that Enterprise > Linux would be more difficult to upgrade... The reason to get RHEL over RH9 or the upcoming Fedora releases is for stability. They have a -much- longer stability period, release cycle, and support lifetime. You get RHEL if you want a distribution that you can get commercial support for, install the server and then not touch it. For production machines of this nature you'll pretty much never have the latest and greatest packages. Instead you'll have the most completely stable packages. The two distribution types are really apples and oranges. They are both fruit ( they are both linux distros ) but they sure taste different. > Also, I cannot find any list of packages included in Enterprise Linux 2.1 / > 3. Does anyone know if PostgreSQL is included and if so, what version? You have two options as I understand it for PG under RHEL. You can install the PG source from Postgres themselves, or you can use the Postgresql Red Hat Edition. Bruce I think can give you more information on this product. http://sources.redhat.com/rhdb/index.html This is the link to it. > > Kind regards, > Alexander Priem. Hope this helps, Will signature.asc Description: This is a digitally signed message part
Re: [PERFORM] RAID controllers etc... was: PostgreSQL data on a
Heya On Wed, 2003-10-22 at 01:13, Alexander Priem wrote: > So I guess the PERC4/Di RAID controller is pretty good. It seems that > RedHat9 supports it out-of-the-box (driver 1.18f), but I gather from the > sites mentioned before that upgrading this driver to 1.18i would be > better... Actually upgrading to the Megaraid_2 driver would be even better. There are a -ton- of performance enhancements with it. Depending on your performance needs and testing capabilities, I would highly recommend trying it out. Will signature.asc Description: This is a digitally signed message part
Re: [PERFORM] RAID controllers etc... was: PostgreSQL data on a NAS device ?
On Tue, 2003-10-21 at 13:36, scott.marlowe wrote: > On 21 Oct 2003, Will LaShell wrote: > > > On Tue, 2003-10-21 at 08:40, scott.marlowe wrote: > > > > > So that brings up my question, which is better, the Perc4 or Perc3 > > > controllers, and what's the difference between them? I find Dell's > > > tendency to hide other people's hardware behind their own model numbers > > > mildly bothersome, as it makes it hard to comparison shop. > > > > Perc4 has n LSI 1030 chip > > http://docs.us.dell.com/docs/storage/perc4di/en/ug/features.htm > > > > > > Perc3 > > depending on the model can be a couple of things but I think they are > > all U160 controllers and not U320 > > Thanks. I googled around and found this page: > > http://www.domsch.com/linux/ > > Which says what each model is. It looks like the "RAID" controller they > wanna charge me for is about $500 or so, so I'm guessing it's the medium > range Elite 1600 type controller, i.e. U160, which is plenty for the > machine / drive number we'll be using. > > Has anyone played around with the latest ones to get a feel for them? I > want a battery backed controller that runs well under linux and also BSD > that isn't gonna break the bank. I'd heard bad stories about the > performance of the Adaptec RAID controllers, but it seems the newer ones > aren't bad from what I've found googling. We own 2 Elite 1650 and we love them. It would be nice to have had U320 capable controllers but the cards are completely reliable. I recommend the LSI controllers to everyone because I've never had a problem with them. signature.asc Description: This is a digitally signed message part
Re: [PERFORM] PostgreSQL data on a NAS device ?
On Tue, 2003-10-21 at 08:40, scott.marlowe wrote: > So that brings up my question, which is better, the Perc4 or Perc3 > controllers, and what's the difference between them? I find Dell's > tendency to hide other people's hardware behind their own model numbers > mildly bothersome, as it makes it hard to comparison shop. Perc4 has n LSI 1030 chip http://docs.us.dell.com/docs/storage/perc4di/en/ug/features.htm Perc3 depending on the model can be a couple of things but I think they are all U160 controllers and not U320 Will signature.asc Description: This is a digitally signed message part
Re: [PERFORM] PostgreSQL data on a NAS device ?
Hello Alexander, On Mon, 2003-10-20 at 06:04, Alexander Priem wrote: > Even better than the four-disk NAS I mentioned earlier is the following: > > Promise UltraTrak RM8000. This is a so-called SCSI-to-IDE RAID system. > Basically it's a RAID setup of eight IDE disks, using a hardware RAID > engine, that's connected to (in this case) the PostgreSQL server via a SCSI > Ultra160 interface (!). So the server won't know any better than that > there's a SCSI disk attached, but in reality it's a IDE RAID setup. It > supports RAID levels 0, 1, 0+1, 5, 50 and JBOD and supports hot-swapping. We have a Promise FasTrak 4000 in our development server connected to 120 Gig western digital 8mb cache drives. Basically the fastest drives we could get for an ide configuration. This system works well, however there are a few things you need to consider. The biggest is that you have very limited control over your devices with the Promise controllers. The bios of the raid controller doesn't have many options on it. You basically plug everything together, and just hope it works. It usually does, but there have been times in the past that really gave us a scare. And we had a situation that in a hard poweroff ( UPS died ) we suffered complete corruptions of 2 of our 4 drives. Performance wise it is =okay= but definitely not on par with either our Megaraid elite 1650 controller or a solution I'm going to suggest to you later in this mail. Your biggest hit is going to be multiple simultaneous accesses. The controller and drives just can't keep up to it. Realistically with my experiences I cannot recommend this solution for a production machine, even with the budget constraints you have put forth. > > Such a NAS config would cost around EUR 3700 (ex. VAT), using 8x40 Gb IDE > disks (7200rpm). > > A SCSI RAID-10 setup using 6x18Gb (15000rpm) disks would cost around EUR > 6000 (ex. VAT) so it's a big difference... I'm not sure where you have your figures, but I would like to propose the following solution for you. for your boot device use either a single ide drive and keep an exact duplicate of the drive in the event of a drive failure, or use 2 drives and use software raid to mirror the two. In this manner you can spend approx $100 USD for each drive and no additional cost for your controller as you will use the motherboards IDE controller. For your postgresql partition or even /var use software raid on an adaptec 29320-R SCSI controller. ( http://www.adaptec.com/worldwide/product/proddetail.html?sess=no&language=English+US&prodkey=ASC-39320-R&cat=%2fTechnology%2fSCSI%2fUltra320+SCSI ) cost: $399 USD IF you bought it from adaptec Match this with 6 Seagate 10k 36G Cheetah U320 scsi drives: ( http://www.c-source.com/csource/newsite/ttechnote.asp?part_no=207024 ) for a cost of $189 USD per drive. If you have 6 of them it brings the total price for your drives to $1134 USD. Total cost for this would be approx $1633 before shipping costs. We use this configuration in our two file servers and have nothing but positive results. If you are totally unable to use software raid you could still buy 6 of those drives, and spend approx $900 USD on an LSI Megaraid 1650 controller. I really believe you'll find either of those options to be superior in terms of price for you. Sincerely, Will LaShell > Does anyone have experience with this NAS device or other "SCSI-to-IDE" RAID > systems? Are they OK in terms of performance and reliability? > Kind regards, > Alexander. > > > ---(end of broadcast)--- > TIP 7: don't forget to increase your free space map settings signature.asc Description: This is a digitally signed message part
Re: [PERFORM] advice on raid controller
On Mon, 2003-09-29 at 06:48, scott.marlowe wrote: > I've used the megaraid / LSI cards in the past and they were pretty good > in terms of reliability, but the last one I used was the 328 model, from 4 > years ago or so. that one had a battery backup option for the cache, and > could go to 128 Meg. We tested it with 4/16 and 128 meg ram, and it was > about the same with each, but we didn't do heavy parallel testing either. > > Here's the page on the megaraid cards at lsilogic.com: > > http://www.lsilogic.com/products/stor_prod/raid/ultra320products.html > > On Sun, 28 Sep 2003, Matt Clark wrote: > > > As others have mentioned, you really ought to get battery-backed cache if > > you're doing any volume of writes. The ability to do safe write-back > > caching makes an *insane* difference to write performance. > > > > The site you link to also has that for only 15% more money: > > http://uk.azzurri.com/product/product.cgi?productId=80 > > > > No experience with the card(s) I'm afraid. > > > > In general though, U320 will only be faster than U160 for large sequential > > reads, or when you have silly numbers of disks on a channel (i.e. more than > > 4/channel). If you have silly numbers of disks, then RAID5 will probably be > > better, if you have 4 disks total then RAID1+0 will probably be better. In > > between it depends on all sorts of other factors. Bear in mind though that > > if you *do* have silly numbers of disks then more channels and more cache > > will count for more than anything else, so spend the money on that rather > > than latest-and-greatest performance for a single channel. Just to add my thoughts, we use the MegaRaid Elite 1650 in 3 servers here that drive our core databases. We paired up the controllers with the Seagate Cheetah 10k drives, we could have purchased the X15's which are Seagate's 15k version, but due to budget constraints and lack of true performance increase from the 10k to the 15k rpm drives we didn't opt for them. I have to say that I've been 100% pleased with the performance and reliability of the Megaraid controllers. They do everything a good controller should and they are very easy to manage. The driver is actively maintained by the guys at LSI and their tech support personnel are very good as well. If you want any specific information or have any questions about our experience or configuration please feel free to contact me. Sincerely, Will LaShell > > HTH > > > > Matt > > > > > -Original Message- > > > From: [EMAIL PROTECTED] > > > [mailto:[EMAIL PROTECTED] Behalf Of Richard > > > Jones > > > Sent: 27 September 2003 18:25 > > > To: [EMAIL PROTECTED] > > > Subject: [PERFORM] advice on raid controller > > > > > > > > > Hi, i'm on the verge of buying a "MegaRAID SCSI 320-2" raid controller. > > > I need it to build a db server using 4x ultra320 scsi disks > > > i'm thinking raid 1+0 but will try with raid5 too and compare > > > > > > Does anyone have any experience with this model, good or bad i'd like to > > > know.. thanks :) > > > > > > as seen: > > > http://uk.azzurri.com/product/product.cgi?productId=188 > > > > > > Regards, > > > Richard. > > > > > > PS: whoever mentioned starting a site with raid controller > > > reviews, excellent > > > idea - its hard to find decent info on which card to buy. signature.asc Description: This is a digitally signed message part
Re: [PERFORM] best arrangement of 3 disks for (insert) performance
I would like to point out though on the PERC controllers that are LSI based ( Megaraid ) there -are- settings that can be changed to fix any o the performance issues. Check the linux megaraid driver list archives to see the full description. I've seen it come up many times and basically all the problems have turned up resolved. Will On Fri, 2003-09-12 at 10:03, Thom Dyson wrote: > > The Dell PERC controllers have a very strong reputation for terrible > performance. If you search the archives of the Dell Linux Power Edge list > (dell.com/linux), you will find many, many people who get better > performance from software RAID, rather than the hw RAID on the PERC. > Having said that, the 3/SC might be one of the better PERC controllers. I > would spend and hour or two and benchmark hw vs. sw before I committed to > either one. > > Thom Dyson > Director of Information Services > Sybex, Inc. > > On 9/12/2003 9:55:40 AM, Richard Jones <[EMAIL PROTECTED]> wrote: > > The machine is coming from dell, and i have the option of a > > PERC 3/SC RAID Controller (32MB) > > or software raid. > > > > does anyone have any experience of this controller? > > its an additional £345 for this controller, i'd be interested to know > what > > people think - my other option is to buy the raid controller separately, > > which appeals to me but i wouldnt know what to look for in a raid > > controller. > > > > that raid controller review site sounds like a good idea :) > > > > Richard. > > > > ---(end of broadcast)--- > TIP 4: Don't 'kill -9' the postmaster signature.asc Description: This is a digitally signed message part
Re: [PERFORM] Tuning PostgreSQL
On Tue, 2003-07-29 at 08:14, Vivek Khera wrote: > > "GS" == Greg Stark <[EMAIL PROTECTED]> writes: > > GS> "scott.marlowe" <[EMAIL PROTECTED]> writes: > > GS> But you have to actually test your setup in practice to see if it > GS> hurts. A big data warehousing system will be faster under RAID5 > GS> than under RAID1+0 because of the extra disks in the > GS> stripeset. The more disks in the stripeset the more bandwidth you > GS> get. > > Anyone have ideas on 14 spindles? I just ordered a disk subsystem > with 14 high speed (U320 15kRPM) SCSI disks to hook up with a dell > PERC3/DC controller (only 128MB cache, though). Hey one comment on this. With dell Perc3/DC you should check the megaraid-devel list to find the best BIOS settings for maximum performance. There have been many comments on it and trials to get it going really well. All told though I totally love the LSI Megaraid ( which is what the perc3/dc is ) controllers. We use the Elite 1650 with seagate cheetah drives for a nice little array. --Will signature.asc Description: This is a digitally signed message part
