Re: Server Stats
On Tue, 05 Oct 1999, Frank Livaudais wrote: > Hi, > > We run a large web site (about 90-120 million page view a month) and it is > all asp. We also have antoher web site with about 10 million page view a > month running mod perl. I am looking for anyone else who is running a large > site (> than 75 million page views a month) that are using Mod perl. Both > sites are almost entirely dynamic. > > I would like to know the number of db and Web servers people are using and > what kind they are (Compaq, Dell, Sun, etc). > > We are trying to convice everyone to convert the big site over to a mod perl > environment but we need some extra numbers of people who also are doing > large-scale sites. Check out slashdot: http://slashdot.org> which serves up about half a million hits a day (all dynamic) from a single dual xeon box running mod_perl. -- Details: FastNet Software Ltd - XML, Perl, Databases. Tagline: High Performance Web Solutions Web Sites: http://come.to/fastnet http://sergeant.org Available for Consultancy, Contracts and Training.
Re: Server Stats
> > We run a large web site (about 90-120 million page view a month) and it is > > all asp. We also have antoher web site with about 10 million page view a > > month running mod perl. I am looking for anyone else who is running a large > > site (> than 75 million page views a month) that are using Mod perl. Both > > sites are almost entirely dynamic. > > > > I would like to know the number of db and Web servers people are using and > > what kind they are (Compaq, Dell, Sun, etc). > > > > We are trying to convice everyone to convert the big site over to a mod perl > > environment but we need some extra numbers of people who also are doing > > large-scale sites. > > Check out slashdot: http://slashdot.org> which serves up about half a > million hits a day (all dynamic) from a single dual xeon box running > mod_perl. What database is this running? Does anyone know? We are looking to run Oracle 8i on a Sun box with Solaris. Our biggest hit has been on the database side. I think our mod perl side can take it, but are there special ways we could configure it to interact better with a database server, etc?
Re: Server Stats
> > Check out slashdot: http://slashdot.org> which serves up about half a > > million hits a day (all dynamic) from a single dual xeon box running > > mod_perl. > > What database is this running? Does anyone know? http://slashdot.org/faq.shtml#Q9 > We are looking to run Oracle 8i > on a Sun box with Solaris. Our biggest hit has been on the database side. I > think our mod perl side can take it, but are there special ways we could configure > it to interact better with a database server, etc? Read the DBI docs, and then have a look at http://www.symbolstone.org/technology/perl/DBI/doc/conferences/ for more. - Perrin
Re: Server Stats
On Tue, 5 Oct 1999, Frank Livaudais wrote: > > million hits a day (all dynamic) from a single dual xeon box running > > mod_perl. > > What database is this running? MySQL. -- Michael Allen Plump | Snorks + Kr0n = yip yip yip! | [EMAIL PROTECTED] "Salsa jazz music reminds me of the time my grandpa ate his left ear." --Joe Dietz
Re: Server Stats
On Tue, 05 Oct 1999, Frank Livaudais wrote: > > Check out slashdot: http://slashdot.org> which serves up about half a > > million hits a day (all dynamic) from a single dual xeon box running > > mod_perl. > > What database is this running? Does anyone know? We are looking to run Oracle 8i > on a Sun box with Solaris. Our biggest hit has been on the database side. I > think our mod perl side can take it, but are there special ways we could configure > it to interact better with a database server, etc? It's running MySQL, and Oracle is your problem. The trouble is it's just too damn slow (no flame wars please, unless you've run side by side comparisons to prove me wrong with 8i - my tests were on Oracle 7). Either switch to Sybase (significantly faster while still offering all the features of Oracle) or realise that you don't need all those features and switch to MySQL. (MySQL doesn't have views, triggers, transactions and a bunch of other important features that Oracle provides). -- Details: FastNet Software Ltd - XML, Perl, Databases. Tagline: High Performance Web Solutions Web Sites: http://come.to/fastnet http://sergeant.org Available for Consultancy, Contracts and Training.
Re: Server Stats
> "FL" == Frank Livaudais <[EMAIL PROTECTED]> writes: >> Check out slashdot: http://slashdot.org> which serves up about half a >> million hits a day (all dynamic) from a single dual xeon box running >> mod_perl. FL> What database is this running? Does anyone know? We are looking FL> to run Oracle 8i on a Sun box with Solaris. Our biggest hit has FL> been on the database side. I Slashdot runs MySQL (http://www.tcx.se/) which is a very light-weight but quite featureful database. We use it quite extensively for our own projects with great success. It does lack some SQL features, so you need to decide if it is "good enough" for you. Chances are it is. -- =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Vivek Khera, Ph.D.Khera Communications, Inc. Internet: [EMAIL PROTECTED] Rockville, MD +1-301-545-6996 PGP & MIME spoken herehttp://www.kciLink.com/home/khera/
Re: Server Stats
Matt Sergeant wrote: > > On Tue, 05 Oct 1999, Frank Livaudais wrote: > > > Check out slashdot: http://slashdot.org> which serves up about half a > > > million hits a day (all dynamic) from a single dual xeon box running > > > mod_perl. > > > > What database is this running? Does anyone know? We are looking to run Oracle 8i > > on a Sun box with Solaris. Our biggest hit has been on the database side. I > > think our mod perl side can take it, but are there special ways we could configure > > it to interact better with a database server, etc? > > It's running MySQL, and Oracle is your problem. The trouble is it's just > too damn slow (no flame wars please, unless you've run side by side > comparisons to prove me wrong with 8i - my tests were on Oracle 7). Either > switch to Sybase (significantly faster while still offering all the > features of Oracle) or realise that you don't need all those features and > switch to MySQL. (MySQL doesn't have views, triggers, transactions and a > bunch of other important features that Oracle provides). Oracle could never be as fast as MySQL at returning a bunch of data from a simple query. The additional features will slow it down. However, I've seen it come close with good DBAs and good hardware, and even pass MySQL on more complex queries where its strengths come into play. It should be able to compete favorably with Sybase in this department, if properly tuned and cared for, but I've never used Sybase so I can't speak from experience. The main thing it has over MySQL is better concurrency support. MySQL will start to give you locking problems if you get a bunch of users doing queries that don't finish immediately and then need to update data. Oracle handles this very well with row-level locking and a multi-versioning feature that no other commercial database currently has. For a description of some of the things that make Oracle different from MySQL, read Phillip Greenspun's latest on the subject at http://www.linuxworld.com/linuxworld/lw-1999-09/lw-09-aolserver_3.html. He's kind of harsh on MySQL, but he does point out the choices you're making by going with one or the other. Personally, I think both have their place and your application needs and environment (as well as how many millions you have to give to Oracle) will determine which one you should use. - Perrin
Re: Server Stats
On Wed, 06 Oct 1999, Perrin Harkins wrote: > Matt Sergeant wrote: > > > > On Tue, 05 Oct 1999, Frank Livaudais wrote: > > > > Check out slashdot: http://slashdot.org> which serves up about half a > > > > million hits a day (all dynamic) from a single dual xeon box running > > > > mod_perl. > > > > > > What database is this running? Does anyone know? We are looking to run Oracle >8i > > > on a Sun box with Solaris. Our biggest hit has been on the database side. I > > > think our mod perl side can take it, but are there special ways we could >configure > > > it to interact better with a database server, etc? > > > > It's running MySQL, and Oracle is your problem. The trouble is it's just > > too damn slow (no flame wars please, unless you've run side by side > > comparisons to prove me wrong with 8i - my tests were on Oracle 7). Either > > switch to Sybase (significantly faster while still offering all the > > features of Oracle) or realise that you don't need all those features and > > switch to MySQL. (MySQL doesn't have views, triggers, transactions and a > > bunch of other important features that Oracle provides). > > Oracle could never be as fast as MySQL at returning a bunch of data from > a simple query. The additional features will slow it down. However, > I've seen it come close with good DBAs and good hardware, and even pass > MySQL on more complex queries where its strengths come into play. It > should be able to compete favorably with Sybase in this department, if > properly tuned and cared for, but I've never used Sybase so I can't > speak from experience. The main thing it has over MySQL is better > concurrency support. MySQL will start to give you locking problems if > you get a bunch of users doing queries that don't finish immediately and > then need to update data. Oracle handles this very well with row-level > locking and a multi-versioning feature that no other commercial database > currently has. I'm assuming this is with Oracle 8 (I did a direct evaluation against Sybase with Oracle 7.3). My experience even with Oracle 8 however has been "dog slow" - and every time has been tuned by Oracle paid consultants. And the more complex the query gets the slower I've seen Oracle get. It's not just my opinion either - I've heard this from lots of other people. > For a description of some of the things that make Oracle different from > MySQL, read Phillip Greenspun's latest on the subject at > http://www.linuxworld.com/linuxworld/lw-1999-09/lw-09-aolserver_3.html. > He's kind of harsh on MySQL, but he does point out the choices you're > making by going with one or the other. Personally, I think both have > their place and your application needs and environment (as well as how > many millions you have to give to Oracle) will determine which one you > should use. Sadly Phillip Greenspun, while a great writer, isn't that fabulous technically (although he's on the right track by not recommending NT). See how he also recommends HP-UX as the fastest and most stable Unix around. (NB: I get an awful lot of work on Oracle systems, so I'm not saying that I'd never use it - just that it's always been slower than anything else I've ever used - and I've used Oracle on some very large iron). -- Details: FastNet Software Ltd - XML, Perl, Databases. Tagline: High Performance Web Solutions Web Sites: http://come.to/fastnet http://sergeant.org Available for Consultancy, Contracts and Training.
Re: Server Stats
Matt Sergeant writes: > On Wed, 06 Oct 1999, Perrin Harkins wrote: > > Matt Sergeant wrote: > > > > > > On Tue, 05 Oct 1999, Frank Livaudais wrote: > > > > > Check out slashdot: http://slashdot.org> which serves up about half a > > > > > million hits a day (all dynamic) from a single dual xeon box running > > > > > mod_perl. > > > > > > > > What database is this running? Does anyone know? We are looking to run >Oracle 8i > > > > on a Sun box with Solaris. Our biggest hit has been on the database side. I > > > > think our mod perl side can take it, but are there special ways we could >configure > > > > it to interact better with a database server, etc? > > > > > > It's running MySQL, and Oracle is your problem. The trouble is it's just > > > too damn slow (no flame wars please, unless you've run side by side > > > comparisons to prove me wrong with 8i - my tests were on Oracle 7). Either > > > switch to Sybase (significantly faster while still offering all the > > > features of Oracle) or realise that you don't need all those features and > > > switch to MySQL. (MySQL doesn't have views, triggers, transactions and a > > > bunch of other important features that Oracle provides). > > > > Oracle could never be as fast as MySQL at returning a bunch of data from > > a simple query. The additional features will slow it down. However, > > I've seen it come close with good DBAs and good hardware, and even pass > > MySQL on more complex queries where its strengths come into play. It > > should be able to compete favorably with Sybase in this department, if > > properly tuned and cared for, but I've never used Sybase so I can't > > speak from experience. My experience (admittedly limited with Oracle) is that Sybase requires quite a bit less resources than Oracle for a given performance level (ie transactions per second). I also think that Sybase has superior backup and restore facilities. Also Sybase has a totally free version out for linux (ie it's not supported but it's free for both development and deployment) which could be interesting for a startup, and potentially migrate to more serious hardware for the dataserver later, as needed... Michael -- Michael Peppler -||- Data Migrations Inc. [EMAIL PROTECTED]-||- http://www.mbay.net/~mpeppler Int. Sybase User Group -||- http://www.isug.com Sybase on Linux mailing list: [EMAIL PROTECTED]
Re: Server Stats
Our issue comes from the fact we will have tables with millions and millions of records and thousands of simultaneous users. So we need to power that Oracle brings, we are just looking for ways to tweak the performance
Re: Server Stats
I read the RDBMS versus article and was interested in the AOLserver solution. Assuming that I don't mind programming Tcl versus Perl, what are the pro's and cons of Apache and mod_perl versus AOLserver? Rex Matt Sergeant wrote: > > Sadly Phillip Greenspun, while a great writer, isn't that fabulous > technically (although he's on the right track by not recommending NT). See > how he also recommends HP-UX as the fastest and most stable Unix around. >
Re: Server Stats
At 13:38 6-10-99 -0500, Frank Livaudais wrote: >Our issue comes from the fact we will have tables with millions and >millions of records and >thousands of simultaneous users. So we need to power that Oracle brings, >we are just >looking for ways to tweak the performance I don't think (never tried it myself, but according to the MySQL docs they do use it at TCX) MySQL has much of a problem with millions and millions of records. Thousands of simultaneous users might be a big problem though - for ANY database. Really, what you want to run depends heavily on how you'll be using the database. How many concurrent sessions (not users - sessions) do you expect, will one db server do or will there be a need for a cluster of them - and if you have to go the cluster way, might you be able to spread database tables over database servers. Will you do mostly reads, or will you need to write a lot as well. Can you use static row sizes or is there a need for variable length fields, etc, etc. All such have an impact on performance. MySQL is quick, it's by far the fastest you can get at most operations. On the other hand, you can't easily spread the load over multiple servers, you can forget about commit/rollback, you won't have stored procedures (yet) - and there's probably more I can't think of right now. I wouldn't dream of running our customer database on MySQL - that kind of data I won't trust to it simply because other databases have a lot more to offer to ensure reliability and integrity. On the other hand, I wouldn't ask of a sloth like Oracle or Informix to serve me the data for persistent user sessions. There is not much of a need for the reliability (at a certain performance cost) the sloths offer for such data. Yet, don't forget that the best database is no database. :) If you can cache data instead of running expensive queries, you'll see performance increase significantly. It's how I'm rewriting a system with HTML::Mason and seeing some 3-4 times the performance the old system had (or lacked). Reducing the number of queries to 1 per request instead of some 6 queries per request really makes a difference in my test configurations. Not in the least because it will allow your database to serve other more useful queries instead. Grtz, Pascal -- Pascal Eeftinck - arcade^xs4all.nl - Perl is not a language, it's a way of life.
Re: Server Stats
On Thu, 7 Oct 1999, Pascal Eeftinck wrote: > MySQL is quick, it's by far the fastest you can get at most operations. On the > other hand, you can't easily spread the load over multiple servers, you can > forget about commit/rollback, you won't have stored procedures (yet) - and > there's probably more I can't think of right now. Referential integrity? :-) -- -- Tom Mornini -- InfoMania Printing and Prepress
Re: Server Stats
Matt Sergeant <[EMAIL PROTECTED]> writes: > Sadly Phillip Greenspun, while a great writer, isn't that fabulous > technically (although he's on the right track by not recommending NT). See > how he also recommends HP-UX as the fastest and most stable Unix around. Yeah, but have you seen the kit they gave him! -- David Hodgkinson, Technical Director, Sift PLChttp://www.sift.co.uk Editor, "The Highway Star" http://www.deep-purple.com Dave endorses Yanagisawa saxes, Apache, Perl, Linux, MySQL, emacs, gnus
Re: Server Stats
Hi. I realise this is getting off-topic, so I suppose replies should go direct to me unless they'll interest the list. I work on a site that makes use of mod_perl, Apache and MySQL. We are currently toying around with our server set-up, trying to spread the load across multiple machines. For web-serving, this is fairly simple, but we're concerned about our MySQL server. Currently, different apps sit on different boxes, each with its own MySQL. However, for ease of upgrading, we're thinking of moving all MySQL databases to dedicated machine(s). On 7 Oct, Pascal Eeftinck wrote: > MySQL is quick, it's by far the fastest you can get at most operations. On the > other hand, you can't easily spread the load over multiple servers This is our current concern. Is the single machine a good way to go? If one app takes down MySQL (which unfortunately does happen once in a while) then all apps lose their database. But if the machine gets bogged down, we can throw more ram/disk space at it. Is it possible to run MySQL across multiple servers? Should we be looking at a solution with multiple database servers instead of one machine? At the hardware level, this would be more reliable, but at the script level, we'd have to keep track of multiple machines, and being a lazy perl monkey, I want all my scripts to talk to the database in an identical manner. Basically, what I'm after is a few words of advice :) Perhaps I should take this to a MySQL mailing list, but given the frequency of the word "MySQL" on this list I thought I'd chance my arm. Thanks in advance. Austin -- Austin Plunkett - perl monkey
Re: Server Stats
>Hi. I realise this is getting off-topic, so I suppose replies should go >direct to me unless they'll interest the list. I think it's of a general interest to the list really. :) >I work on a site that makes use of mod_perl, Apache and MySQL. We are >currently toying around with our server set-up, trying to spread the >load across multiple machines. For web-serving, this is fairly simple, >but we're concerned about our MySQL server. Currently, different apps >sit on different boxes, each with its own MySQL. However, for ease of >upgrading, we're thinking of moving all MySQL databases to dedicated >machine(s). Note that once you do find a stable version of MySQL, unless you need new features there's hardly much of a reason to upgrade. I've found 3.22.25 and up to be pretty stable, while older versions would go down every now and then (on a Solaris 2.5 Ultra 2). Not that crashing has been much of a problem - the safe_mysqld script checks the databases [with only once a problem due to lack of diskspace in creating the temporary tables] and restarts the server. I know the server crashed once every 1-2 days, but I've never heard any complaint over it having done that. With the newer versions it stays up a lot longer, if it even crashes at all. > > MySQL is quick, it's by far the fastest you can get at most > operations. On the > > other hand, you can't easily spread the load over multiple servers > >This is our current concern. Is the single machine a good way to go? If >one app takes down MySQL (which unfortunately does happen once in a >while) then all apps lose their database. But if the machine gets >bogged down, we can throw more ram/disk space at it. Is it possible to >run MySQL across multiple servers? Should we be looking at a solution >with multiple database servers instead of one machine? At the >hardware level, this would be more reliable, but at the script level, >we'd have to keep track of multiple machines, and being a lazy perl >monkey, I want all my scripts to talk to the database in an identical >manner. I'd best direct you to section 19.1 of the MySQL manual for this. But basically it boils down to you having to do all the hard work - it isn't too hard to maintain a mirror server of your database (through the update log) but once you start updating your slave server as well you will get out of sync with your data on the master. So if you only need to read from your database it might be worth the performance to have a slave server to only read from. Another alternative are of course different databases, if they are not or only very slightly related then run multiple servers with these different databases. (Perhaps one might contain all your persistent session data, and another your customer data - people with a persistent session are not necessarily your customer too). What's probably also an important factor is whether you have one database and a set of scripts accessing it, or lots of databases and more scripts accessing them. The latter will give you problems with persistent database connections and the database server will have to work a lot harder to cache all the different sets of data. And do try to keep your queries down. If you can cache data then by all means go for it. I have a CGI server with a MySQL database on it and dozens of CGI's accessing different databases on that server. It has to do an average of some 5500 SQL queries per 5 minutes, peaking at 12200 queries per 5 minutes at peak load. It consumes 60% of the CGI's server CPU capacity, and it takes a lot of simultaneous connects with this peak load. As it takes so much performance it will also bring down the number (and performance) of the mod_perl CGI's it can serve in a snowball effect. I'm sure once I'm done converting the heavily used scripts to HTML::Mason and implementing a lot of caching there the load on the MySQL server will be a lot less and leave room for the useful stuff. :) As someone else already mentioned, the MySQL people are working hard on making a distributed database possible. I'm curious to see what will come of that, although I don't think I'll use the feature myself. Grtz, Pascal -- Pascal Eeftinck - arcade^xs4all.nl - Perl is not a language, it's a way of life
Re: Server Stats
On Thu, 21 Oct 1999, Pascal Eeftinck wrote: > As someone else already mentioned, the MySQL people are working hard on > making a distributed database possible. I'm curious to see what will > come of that, although I don't think I'll use the feature myself. And there are other far more important things they should work on first - this is like closing the gate after the horse has bolted without things like decent locking and transactions. Although perhaps I'm mistaken and they have those things higher on the priority list. -- Details: FastNet Software Ltd - XML, Perl, Databases. Tagline: High Performance Web Solutions Web Sites: http://come.to/fastnet http://sergeant.org Available for Consultancy, Contracts and Training.
Re: Server Stats
>this is like closing the gate after the horse has bolted without things >like decent locking and transactions. Although perhaps I'm mistaken and You can rest assured that they know what they are doing. :-) It is also worth upgrading to newer versions. The newest versions not deemed stable just yet no longer use ISAM, are much faster, and will allow for a host of new features. stay tuned. ed
