Re: Tips for LARGE system
On Sun, Oct 20, 2002 at 10:52:22PM -0700, Steven Roussey wrote: Other people can tell you more about replication and how it is different from clustering or load balancing. And you can use it for backup in a way too. It depends on your requirements though. Yeah. And you can use replication to enable load-balancing. They work quite well together. If things are partitionable, then some thinking about that upfront will do you a world of good later. Depend on your application though. Indeed. Partitioning before you NEED to can save a ton of hassle. 7) I'd also appreciate any input from people who have used official mysql support before. We have used their support and it was excellent. Heck, their free support is great. The paid stuff just gurantees it. Jeremy -- Jeremy D. Zawodny | Perl, Web, MySQL, Linux Magazine, Yahoo! [EMAIL PROTECTED] | http://jeremy.zawodny.com/ MySQL 3.23.51: up 75 days, processed 1,588,578,807 queries (242/sec. avg) - Before posting, please check: http://www.mysql.com/manual.php (the manual) http://lists.mysql.com/ (the list archive) To request this thread, e-mail [EMAIL PROTECTED] To unsubscribe, e-mail [EMAIL PROTECTED] Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php
RE: Tips for LARGE system
handling from 3M to 20M records/day with potentially a lot of processing, live inserts/ updates etc. How many records do you forsee? (There are some things to look up in the manual to give MySQL a hint at the total size of the table so it can start using longer internal pointers.) How big are the records? What kind of processing? Are you separating your transactional system from the reporting system? What is the pattern of use of the records? Unfortunately, your information is not enough to go on. For example, we do 200,000,000+ queries a day and barely break a load of 1.00 on Linux. But we have had a lot of time to optimize things. 3) Are there any guidelines in estimating more presicely what hardware I will need? Will your application we processor or disk based? I'd guess disk based, so get lots of RAM (fast RAM -- like DDR or whatever), and as many fast disks as you can afford and stripe them. Personally, I built our machine from parts at Frys Electronics. I think you could put something together for a reasonable amount of money. Do note, it has been my experience that hardware costs are inversely exponentially proportional to the optimization of the database structure and queries. Fully normalizing and then selectively denormalizing will huge differences in throughput. As will optimizing queries. (Specifically to the database in use helps tons too.) 4) I cannot foresee all the possible growth, nor will the initial budget be huge. Is it then woth planning for building out / clustering for some redunacy and some load balancing upfront Other people can tell you more about replication and how it is different from clustering or load balancing. And you can use it for backup in a way too. It depends on your requirements though. If things are partitionable, then some thinking about that upfront will do you a world of good later. Depend on your application though. 7) I'd also appreciate any input from people who have used official mysql support before. We have used their support and it was excellent. Sincerely, Steven Roussey http://Network54.com/ - Before posting, please check: http://www.mysql.com/manual.php (the manual) http://lists.mysql.com/ (the list archive) To request this thread, e-mail [EMAIL PROTECTED] To unsubscribe, e-mail [EMAIL PROTECTED] Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php
RE: Tips for LARGE system
Hi Bigger than my experience, so just a suggestion - have you looked at replication? If you have a master which does all the processing and (multiple) slaves which handle all the reads then you might benefit from cheaper Intel hardware and have a more robust system to boot. Good luck! Peter --- Excellence in internet and open source software --- Sunmaia www.sunmaia.net tel. 0121-242-1473 --- -Original Message- From: spiros [mailto:moka;hol.gr] Sent: 21 October 2002 00:00 To: [EMAIL PROTECTED] Subject: Tips for LARGE system I need to come up with an intra-company system recommendation: We are looking at a possibly huge system getting data from a LAN(No web-based app or anything-in fact it will NOT be connected to the web) handling from 3M to 20M records/day with potentially a lot of processing, live inserts/ updates etc. I have a demo version(running on a PIII 2x1000MHz, 2GB RAM Linux machine) but it cannot handle the full load. I am thinking of keeping Mysql 4-0.4. and going to an alpha machine(running linux). The question is whether anyone has done this before. So I am looking for tips and recommendations(before buying any hardware): 1) Are there any known or expected problems with Mysql on alpha machines? 2) Do I need a special linux version(the kernel I use says 2.4.18-4GB, but I need much more RAM and in fact this is the main reason for going to an alpha machine)?(never used linux on anything except Pentiums) 3) Are there any guidelines in estimating more presicely what hardware I will need? 4) I cannot foresee all the possible growth, nor will the initial budget be huge. Is it then woth planning for building out / clustering for some redunacy and some load balancing upfront 5) Suppose I get this system running. What are the problems in keeping it running? Critics say that a)data management is a pain b) scalability is pretty non-existant (just sticking more RAM). I though it would be much cheaper to stick in more RAM than going to another RDBMS c) it lacks a load of fundamental features of large RDBMS. I thought 4.1 should be out between end of Octobera nd January, and it will have triggers, no? 6) How is a hot backup organized in Mysql(i.e. backing up the data while the database is running) 7) I'd also appreciate any input from people who have used official mysql support before. Hopefully there will be a set of issues: a) hardware requirements b) setting up and configuring(i.e. specifying InnoDB buffer sizes) c) performance tuneup. Again, I'd appreciate any info from mysql people or people who used mysql support before. My problem is that from mysql support I am told we can discuss all these after signing a support contract, but to get there I need to come up with a company realistic cost estimate that includes the necessary hardware... I'd appreciate any input from people running such systems. I have never used any other RDBMS, so I am cannot judge if the critisism is justified or not and would appreciate any help on this Thanks, S.Alexiou - Before posting, please check: http://www.mysql.com/manual.php (the manual) http://lists.mysql.com/ (the list archive) To request this thread, e-mail [EMAIL PROTECTED] To unsubscribe, e-mail [EMAIL PROTECTED] Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php - Before posting, please check: http://www.mysql.com/manual.php (the manual) http://lists.mysql.com/ (the list archive) To request this thread, e-mail [EMAIL PROTECTED] To unsubscribe, e-mail [EMAIL PROTECTED] Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php
