I Raise my hat and salut Zaid Ameereh, you are the man :-) and everyone who contributed Maturely into this thread ...
On 11/18/07, Ala'a Ibrahim <[EMAIL PROTECTED]> wrote: > > Hi Zaid, > I'm so sorry, it seams that you are right, as you said, every human is > prone to making errors, and I admit it, the number was 1,500 RPS, I'm sos > sorry for making such mistake, I guess I need to recheck with the admins on > the sessions number. > > I'm sorry > > On Nov 18, 2007 2:56 AM, Zaid Amireh <[EMAIL PROTECTED]> wrote: > > > > > Hi Ala'a, > > > > Every human being is prone to making errors, I am a human being and I > > might be wrong if you heartily believe so, if you and your > > administration team were able to reach 15,000 RPS on commodity > > hardware then I take off my hat in respect to you all, I obviously > > need to change my life time career and passion :) > > > > Anyway, it seems that you already know much more than I do about this > > subject, I will step aside from this conversation. > > > > virtually yours > > > > Zaid > > > > > > On Nov 17, 2007 6:55 PM, Ala'a Ibrahim <[EMAIL PROTECTED] > wrote: > > > Hi Zaid, > > > I' so sorry again, but if I didn't say it I wouldn't be sleeping for > > days > > > :), thanks for the complement, or whatever it was. > > > > > > anyway back to business, this number I got from the system admins, I > > don't > > > work on that machine, and I don't really know if it's true or not, but > > I'm > > > sure this is not happening 24 hours, but as far as I know, it reaches > > > something like this during the day, and I also guess there is > > something > > > wrong with your calculation. > > > > > > let's start by, everytime apache is installed we make sure to change > > the > > > max simultaneous connections is changed to 1024, and once again it's > > not > > > RPS, it's the number of requests that can be handled simultaneously, > > also, > > > for me I've worked on optimizing stuff, and without playing with the > > OS, > > > only PHP, Apache, and of course with APC. > > > I managed with an AMD 64 Athlon processor to get a php application > > that was > > > launched online, to take it up to about 15,000 RPS, also dude, if it > > takes 1 > > > sec for each page to respond, I guess you are in deep trouble. > > > > > > Also the use of client caching is so important, I don't think that > > anybody > > > takes the css, JS, and images files, every time the browser makes a > > request, > > > this would be so waste of bandwidth, that's why we use Expires, > > > Last-Modification-Date, and the E-Tag headers sent, so the user won't > > get > > > the same data twice, so mostly in the end, the user in most of the > > times > > > just gets the HTML. > > > > > > also given that we have the DB on another machine, that would remove > > it's > > > burden from the sever, it's on another server on the same Rack. > > > > > > and don't think that even when reaching that number the server is > > standing > > > doing his job well, it would be dying out there, and that's why there > > is the > > > need for another machine to share this burden with. > > > > > > So for me I guess 200,000 is so believable to have, just after doing > > a good > > > job. > > > > > > > > > > > > On 11/17/07, Zaid Amireh < [EMAIL PROTECTED]> wrote: > > > > > > > > > > > > > > > > Dear Ala'a, > > > > > > > > You are a smart guy whom I respect. I am not going to reply to your > > > > highlighted bold comment for obvious maturity reasons :) , on the > > > > other hand, the number you claimed (200,000) is beyond absurd and > > > > following is my explanation: > > > > > > > > In PHP, the default value of session.gc_maxlifetime in 1440 seconds, > > > > which equals 24 minutes, I will assume that your clients are > > browsing > > > > 3 pages during their session lifetime which is a very low number, > > but > > > > stick with me, it gets interesting. > > > > > > > > 200,000 * 3 = 600,000 page requests in 24 minutes. > > > > > > > > each page would contain an average of 5 media elements, also way too > > > > low but I'm not gonna stress test you here, so we end up with 6 > > > > requests for each page. > > > > > > > > so 600,000 * 6 requests served in 24 mins = ( (600,000 * 6) / > > (24*60) > > > > ) thats 2500 requests per second. > > > > > > > > You do know that apache's default of MaxClients (which is max > > > > concurrent requests if you don't have keep alive on) is 256 and that > > > > you have to change the source code to increase that, right? ever > > > > wondered why its hard coded at 256? > > > > > > > > lets assume you did increase the cap and you are happily serving > > 2500 > > > > RPS, I don't know what hardware or software you have, but I can > > > > extrapolate on the environment that I operate since its all php and > > > > apache after all. > > > > > > > > I have a dual core Xeon with 2 gigs of RAM, hardware RAID 1, > > 100mbits > > > > uplink, its a decent Dell PowerEdge machine that has been grinding > > web > > > > requests for the last couple of years, it took me 2 months to fully > > > > tweak the OS and applications, starting from the VM parameters in > > the > > > > kernel, to the TCP params in the kernel, all the way to tuning gcc > > > > compile options to produce an optimized apache binary, APC for php > > > > caching, etc ..., trust me, its a blazing fast machine for serving > > PHP > > > > and I hold extreme pride in what I made out of this particular > > > > machine. > > > > > > > > How many clients can this machine serve concurrently? (this machine > > > > does not hold the DB btw, it has only apache) > > > > At 35 requests per second the CPU is at 20% utilization. > > > > At 55 requests per second the CPU is at 40% utilization. > > > > At 75 requests per second the CPU is at 70% utilization. > > > > I was able to squeez 89 RPS at 100% CPU. (and the CPU was the > > > > bottleneck, also bear in mind that the NIC has an onboard processor, > > > > thus networking takes almost nothing from the CPU). > > > > > > > > Want fancy charts with real data? I have attached two charts that > > show > > > > apache's children state and cpu utilization for the last hour, I can > > > > also provide you with the last 12 months charts that include apache > > > > children stats (busy, idle, waiting...), cpu utilization, memory > > > > utilization, hard disk i/o, all that stuff. > > > > > > > > So for the love of the flying spaghetti monster don't tell me there > > is > > > > a *single* machine out there serving 2500 PHP and static content > > > > requests per second unless you back up your claim with real numbers. > > > > > > > > Good luck with your cluster and if you need anything else we are > > here > > > > to help as much as we can :) > > > > > > > > virtually yours > > > > > > > > Zaid > > > > > > > > > > > > > > > > > > > > > > > > On Nov 17, 2007 2:35 PM, Ala'a Ibrahim < [EMAIL PROTECTED]> > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > On 11/17/07, Zaid Amireh < [EMAIL PROTECTED] > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > you kidding me? 200,000 *concurrent* HTTP sessions on *one* > > server? > > > > > > whats your session time window? 24 hours? > > > > > > > > > > > > seriously mate, if you want help, tell me some thing I would > > believe. > > > > > > > > > > > > Zaid > > > > > > > > > > > > On Nov 15, 2007 5:03 PM, Ala'a Ibrahim < [EMAIL PROTECTED] > > > > wrote: > > > > > > > well, the problem is that it's a legacy system, and we are not > > > planning > > > > > to > > > > > > > do any development on it right now, we are having about > > 200,000 > > > > > concurrent > > > > > > > sessions on that system, and the database server is also > > already > > > loaded, > > > > > > > (given too that the apache server is more loaded) that's why > > it's > > > being > > > > > > > clustered upon 2 servers, also the servers are on the same > > rack, so > > > I > > > > > don't > > > > > > > think the network latency would be a problem. > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 11/15/07, Ammar Ibrahim < [EMAIL PROTECTED]> wrote: > > > > > > > > I completely agree, I *guess* it would run much faster on a > > > properly > > > > > > > designed database. All your queries will be based on an > > indexed > > > column > > > > > > > (session_id). A nice trick would be to create a table that is > > in > > > memory, > > > > > > > this way you would have blazing speed, but then again, you > > need to > > > study > > > > > the > > > > > > > possibility of the database server going offline and the > > > consequences. > > > > > > > Databases are much easier to scale than a file system based > > > solution. > > > > > And > > > > > > > keep in mind if you apply your own session mechanism that > > doesn't > > > use a > > > > > > > database you might run into problems that are very hard to > > > trace/debug > > > > > (race > > > > > > > conditions). You need to test properly with the load you are > > > getting. > > > > > There > > > > > > > are tools to help you simulate the number of sessions that you > > have. > > > > > > > > > > > > > > > > What you could also do is sharding, which is the technique > > of > > > > > splitting > > > > > > > the sessions across more than one database. e.g. you can use a > > > hashing > > > > > > > function to determine to which Database the session is stored > > into. > > > As > > > > > an > > > > > > > example: > > > > > > > > > > > > > > > > Let's assume you want to create 5 data stores for sessions. > > Tables > > > > > might > > > > > > > be in the same database or a combination of servers/tables. To > > make > > > it > > > > > > > simple let's say we have the tables sessions1 - session5 in > > one db. > > > By > > > > > > > writing a function that allows sessions to be mapped to their > > > > > corresponding > > > > > > > table, this way you can achieve splitting your number of > > records by > > > a > > > > > factor > > > > > > > of 5. If you had a million sessions, on average you will have > > around > > > > > 200K > > > > > > > records in each table. This is a very nice technique for > > database > > > > > > > optimization in general, the draw back is complexity in > > programming, > > > and > > > > > if > > > > > > > you work on a project with less experienced developers I > > highly > > > don't > > > > > > > recommend sharding all your app. Just minimize it to Sessions, > > which > > > is > > > > > > > gonna be transparent to the developers working on the project > > as > > > they > > > > > will > > > > > > > be using the built in session mechanism in PHP, and they don't > > have > > > to > > > > > worry > > > > > > > about what's going on in the background. > > > > > > > > > > > > > > > > It wouldn't be hard to benchmark and see the best fit to > > your > > > problem. > > > > > > > > > > > > > > > > - Ammar > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Nov 15, 2007 11:38 AM, Zaid Amireh < [EMAIL PROTECTED] > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > How many concurrent sessions do you have? whats the size > > of a > > > > > typical > > > > > > > > > session file? you have to know your workload to find the > > best > > > > > > > > > solution. > > > > > > > > > > > > > > > > > > Unless you have 1000+ concurrent sessions the only load > > you will > > > > > > > > > imposing on the DB server is the network overhead, > > sessions come > > > and > > > > > > > > > go and so the table would stay small, with some extra > > indexes > > > here > > > > > and > > > > > > > > > there you should be golden. > > > > > > > > > > > > > > > > > > Then again, even with 1000+ sessions, if the table has > > proper > > > > > indexes, > > > > > > > > > I would assume its still faster than a remote filesystem, > > too > > > many > > > > > > > > > factors to write about in this email. > > > > > > > > > > > > > > > > > > Zaid > > > > > > > > > > > > > > > > > > > > > > > > > > > On Nov 15, 2007 10:12 AM, Ala'a Ibrahim < > > [EMAIL PROTECTED]> > > > > > wrote: > > > > > > > > > > Well, I'm avoiding the use of database, I don't want to > > add an > > > > > extra > > > > > > > load on > > > > > > > > > > the DB server, and don't won't to get a new server for > > the > > > > > sessions > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 11/15/07, Ammar Ibrahim < [EMAIL PROTECTED]> > > wrote: > > > > > > > > > > > Database? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Nov 15, 2007 7:48 AM, Ala'a Ibrahim < > > > [EMAIL PROTECTED]> > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > I'm trying to do a small apache cluster, everything > > works > > > > > fine, > > > > > > > except > > > > > > > > > > for PHP sessions, as they are stored on the filesystem, > > I > > > tried > > > > > using > > > > > > > an NFS > > > > > > > > > > share to hold the data, it worked fine on the testing > > > > > environment,but > > > > > > > on the > > > > > > > > > > live servers, it wasn't a good idea, as the NFS share > > kept > > > hanging > > > > > up, > > > > > > > so > > > > > > > > > > I'm thinking of using a DRBD filesystem, connected on > > all the > > > > > nodes, > > > > > > > > > > something like a Replica RAID on all the servers, in the > > > testing > > > > > > > > > > environment, it worked fine, but I wonder if it would be > > on > > > the > > > > > live > > > > > > > one, so > > > > > > > > > > if anyone thinks this is a bad idea, has a better > > solution, or > > > has > > > > > > > some > > > > > > > > > > comments please share it. > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > Ala'a A. Ibrahim > > > > > > > > > > > > http://guru.alaa-ibrahim.com/ > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > Ala'a A. Ibrahim > > > > > > > > > > http://guru.alaa-ibrahim.com/ > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > --------------------------- > > > > > > > > > Netiquette -> http://www.dtcc.edu/cs/rfc1855.html > > > > > > > > > Netiquette Nazi -> > > > > > > > > > > > > http://redwing.hutman.net/%7Emreed/warriorshtm/netiquettenazi.htm > > > > > > > > > --------------------------- > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Ala'a A. Ibrahim > > > > > > > http://guru.alaa-ibrahim.com/ > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > --------------------------- > > > > > > Netiquette -> http://www.dtcc.edu/cs/rfc1855.html > > > > > > Netiquette Nazi -> > > > > > > > > http://redwing.hutman.net/%7Emreed/warriorshtm/netiquettenazi.htm > > > > > > --------------------------- > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > --------------------------- > > > > > > > > Netiquette -> http://www.dtcc.edu/cs/rfc1855.html > > > > Netiquette Nazi -> > > > > http://redwing.hutman.net/%7Emreed/warriorshtm/netiquettenazi.htm > > > > --------------------------- > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > Ala'a A. Ibrahim > > > http://guru.alaa-ibrahim.com/ > > > > > > > > > > > > > > > > > > -- > > --------------------------- > > Netiquette -> http://www.dtcc.edu/cs/rfc1855.html > > Netiquette Nazi -> > > http://redwing.hutman.net/%7Emreed/warriorshtm/netiquettenazi.htm > > --------------------------- > > > > > > > > -- > > Ala'a A. Ibrahim > > http://guru.alaa-ibrahim.com/ > > > > > > -- Khamis Siksek http://saksoook.blogspot.com --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Jolug" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/Jolug?hl=en-GB -~----------~----~----~----~------~----~------~--~---
