On 4 Sep 2013, at 00:03, "Daevid Vincent" <dae...@daevid.com> wrote:
>> -----Original Message----- >> From: Stuart Dallas [mailto:stu...@3ft9.com] >> Sent: Tuesday, September 03, 2013 2:37 PM >> To: Daevid Vincent >> Cc: php-general@lists.php.net; 'Jim Giner' >> Subject: Re: [PHP] refernces, arrays, and why does it take up so much >> memory? [SOLVED] >> >> On 3 Sep 2013, at 21:47, "Daevid Vincent" <dae...@daevid.com> wrote: >>> We get over 30,000 hits per second, and even with lots of caching, 216MB >>> vs 70-96MB is significant and the speed hit is only about 1.5 seconds more >>> per page. >>> >>> Here are three distinctly different example pages that exercise >>> different parts of the code path: >>> >>> PAGE RENDERED IN 7.0466279983521 SECONDS >>> MEMORY USED @START: 262,144 - @END: 26,738,688 = 26,476,544 BYTES >>> MEMORY PEAK USAGE: 69,730,304 BYTES >>> >>> PAGE RENDERED IN 6.9327299594879 SECONDS >>> MEMORY USED @START: 262,144 - @END: 53,739,520 = 53,477,376 BYTES >>> MEMORY PEAK USAGE: 79,167,488 BYTES >>> >>> PAGE RENDERED IN 7.558168888092 SECONDS >>> MEMORY USED @START: 262,144 - @END: 50,855,936 = 50,593,792 BYTES >>> MEMORY PEAK USAGE: 96,206,848 BYTES >> >> Knowing nothing about your application I'm obviously not in a strong >> position to comment, but seven seconds to generate a page would be >> unacceptable to me and any of my clients. > > It's a "one time hit" and the rest is served from a cache for the next 24 > hours which serves very very fast after that initial rendering. It's just we > have so many thousands of pages that this becomes an issue -- especially > when webcrawlers hit us and thread-out so MANY pages are trying to render at > the same time, especially the ones towards the end where they haven't been > cached since rarely do real people get that far... Like you know, pages 900, > 901, 902, etc... with new content each day, page 1 today is now page 2 > tomorrow, so it's a constant thorn. At 30k requests per second, is it a one-time hit, or is it 225k hit because in the 7 seconds it takes the cache to be built you have that many clients also building the cache? Or is this already an offline script, in which case how long it takes is largely irrelevant. What is your caching strategy? What is cached? At what granularity? When is the cache updated (i.e. on demand or on change)? Why does a page need to retrieve so much data? Can that data not be summarised/processed ahead of demand? >> I'll put money on it being >> possible to cut that time by changing your caching strategy. The memory >> usage is also ridiculous - does a single page really display that amount > of >> data? Granted, there are some applications that cannot be optimised beyond > a >> certain point, but those numbers make me sad! > > HA! It was over 400MB per page a few weeks ago. I keep whittling it down, > but I think I'm hitting the lower limit at this point. That is nuts! What's the website? > It's a tough balance between database hits, cache hits, network traffic > (memcached), disk i/o, page speed, load balancing, etc. All we can do is try > things and tweak and see what works and what brings the servers to their > binary knees. Without knowing anything about the site there's little I can say, but if you want to take this off-list I'm happy to talk about it. I have a fair amount of experience with high-traffic web applications so it's possible I might be able to help. -Stuart -- Stuart Dallas 3ft9 Ltd http://3ft9.com/ -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
