Have you configured your app to time out bot sessions after page view? Bots can't retain sessions, so every page view would spawn a separate session, thereby flooding memory. You can do bot detection in your onRequestStart() to set the session timeout to something like 2 seconds, as opposed to the typical. This will decrease your memory footprint during bot crawling, which helps a little. Ben Nadel has some posts on this on his blog.
I ran an app for 5 years that did 150k page views every few minutes, 500k query requests per server every 4 hours (we had 8 web servers), and 3k+ templates of legacy code. You can do it, you just have to find your pain points and address them. A thorough review of what's happening with variables throughout the application life span can be very beneficial. Is this variable being used? Is it in the right scope? Every bit counts. Steve 'Cutter' Blades Adobe Community Professional Adobe Certified Expert Advanced Macromedia ColdFusion MX 7 Developer ____________ http://blog.cutterscrossing.com Co-Author "Learning Ext JS 3.2" Packt Publishing 2010 https://www.packtpub.com/learning-ext-js-3-2-for-building-dynamic-desktop-style-user-interfaces/book "The best way to predict the future is to help create it" On 6/28/2011 5:12 PM, Richard Steele wrote: > Yes, it's a hugely complex site with 170k page views a day. We pull some of > the data we show from Amazon's services further slowing it down (much of > which can't be cached according to their terms of service). Yes, I agree, > getting rid of any slow running queries is the most important thing to do and > we do have some work to do in that realm. But 95% of the time the site works > just fine. So it's a lot of programming time to make that 96% of the time or > greater. We do have FusionReactor and it is extremely helpful. But it can't > tell us the IP addresses hitting the site if we have a bot crawling through > 10k dynamically produced links or if some other website is hammering the > instances. We see the load balancer IP address instead. Thus we need to go > into the server logs and analyze what is happening. > > >> A site? A single site? >> >> In my experience, performance issues come down to a few things: Poor >> (or >> no) scoping choices, poor SQL (both query execution plans and general >> >> server health), and an improperly tuned JVM. Most of the time, for me, >> >> it appears to be the DB. Either the queries are terribly written, >> there >> aren't any indexes on tables, or both (and more). >> >> Have your tried FusionReactor yet? FR can be a great resource for >> finding and resolving bottlenecks and poorly written code, and I think >> >> there's a 10 trial download. Hiring guys like Mike Brunt and Charlie >> Arehart are well worth the money too. >> >> Steve 'Cutter' Blades >> Adobe Community Professional >> Adobe Certified Expert >> Advanced Macromedia ColdFusion MX 7 Developer >> ____________ >> http://blog.cutterscrossing.com >> >> >> Co-Author "Learning Ext JS 3.2" Packt Publishing 2010 >> https://www.packtpub. > com/learni> > ng-ext-js-3-2-for-building-dynamic-desktop-style-user-interfaces/book >> "The best way to predict the future is to help create it" >> >> >> On 6/28/2011 12:33 PM, Richard Steele wrote: >>> Yes, for sure.... in an ideal world. However even though we have CF >> Enterprise, we can only have two instances for this particular site >> due to hardware and budget constraints. When traffic increases due to >> some link to the site going viral (which is unusual), it would be >> impossible for the server to keep up with the requests even if the >> requests were all processed under a second. Then requests would build >> up in the queue awaiting processing. People then get impatient and >> start clicking the link over and over, further exacerbating the >> problem. >>> >>> > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:345880 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
