Re: Cake CPU hungry
Do you know that for sure? If you lets say load the Users model in AppController, then it is more then likely that *all* of your other models are loaded b/c they are somehow associated with User. In AppController I only turn on othAuth and afaik it doesn't load any models if authorization is not required in a Controller. Here's part of debug($this) run in my pages_controller : [name] = Pages [helpers] = Array ( [0] = Html [1] = Text [2] = Form [3] = OthAuth [4] = tqlTools ) [uses] = [components] = Array ( [0] = othAuth [1] = Session ) . . [othAuth] = othAuthComponent Object ( ... [controller] = PagesController Object *RECURSION* ) No model loaded, I only don't like this *RECURSION* .. in a controller with like 6 models loaded there's a LOT of RECURSION.. do you have it as well ? My suggestion is to not work / rely on benchmark solutions that just hammer at your server too much - its very complex to create a realistic scenario that way (i.e. 20 users holding down on your front page or some other action is very different from each of them doing different things). The best way to make your app faster should be to get some profiler going and see what functions / parts of your app take the longest to load and then optimize them. Yeah, I'm taking a closer look on xdebug, I will post some results if I manage to run it. From there I'd keep an eye closely on the server load during traffic peaks which you can use to generate yourself some statistics to estimate when your current horse power will not be enough anymore. Then it is time to cluster and scale your architecture. So from some point there's a question where to put your resources : trying to optimize the code or buying and adding hardware to cope with a higher load. Anyway, I'm not an expert on the matter and you may be able diagnose things with your current benchmark setup as well - however I think its not the easiest path. Good luck and success with your app! Thanks! We've got more then 5000 facebook users in a first week and 1000 daily active users (new facebook metric), up to 13000 pageviews/ day so it's coming along nicely :) Best, Andrzej --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: Cake CPU hungry
(i.e. 20 users holding down on your front page or some other action is very different from each of them doing different things) Yeah, I'm taking a closer look on xdebug, I will post some results if I manage to run it. Ups, I meant to write 'Holding down F5 on your front page here, but I think you got that. Yeah I'm currently looking into getting xdebug to run myself - you don't happen to use OS X for development so you could share your findings ^^? In AppController I only turn on othAuth and afaik it doesn't load any models if authorization is not required in a Controller. Here's part of debug($this) run in my pages_controller : You may get better results using get_included_files http://de.php.net/manual/en/function.get-included-files.php / get_declared_classes http://de.php.net/manual/en/function.get-declared-classes.php or using a profiler, but yeah your results seem to indicate no models being loaded I think. No model loaded, I only don't like this *RECURSION* .. in a controller with like 6 models loaded there's a LOT of RECURSION.. do you have it as well ? There is not much you can do against that recursion and its not really a problem for performance (I think). What you could try is to write your own debug() function that keeps a hash of items already iterated through and makes sure inifinite recursions will not be followed (my experience has shown that the print_r behavior here is inconsistent). So from some point there's a question where to put your resources : trying to optimize the code or buying and adding hardware to cope with a higher load. I'm usually for throwing more CPU cycles at things instead of development time as the latter is more expensive and better spend on improving the user experience / maintainability rather then the speed. That being said, this is not meant as an excuse for not looking into obvious bottle necks / scalability issues and resolving them. But after that, you reach a point where optimizing things becomes more and more difficult and may even make your application harder to maintain, thats the point where I'd say throw more servers at it instead of developers. Thanks! We've got more then 5000 facebook users in a first week and 1000 daily active users (new facebook metric), up to 13000 pageviews/ day so it's coming along nicely :) Oh its a Facebook app - cool : ). Add me to your friends and tell me what app it is so I can give it a try (my profile is: http://hs.facebook.com/profile.php?id=1114020732). Oh and congratulations to your success so far, thats pretty cool. Hope you'll find some time into sharing your experiences in using and scaling CakePHP for this at some point with the community : ). -- Felix -- My Blog: http://www.thinkingphp.org My Business: http://www.fg-webdesign.de thequietlab wrote: Do you know that for sure? If you lets say load the Users model in AppController, then it is more then likely that *all* of your other models are loaded b/c they are somehow associated with User. In AppController I only turn on othAuth and afaik it doesn't load any models if authorization is not required in a Controller. Here's part of debug($this) run in my pages_controller : [name] = Pages [helpers] = Array ( [0] = Html [1] = Text [2] = Form [3] = OthAuth [4] = tqlTools ) [uses] = [components] = Array ( [0] = othAuth [1] = Session ) . . [othAuth] = othAuthComponent Object ( ... [controller] = PagesController Object *RECURSION* ) No model loaded, I only don't like this *RECURSION* .. in a controller with like 6 models loaded there's a LOT of RECURSION.. do you have it as well ? My suggestion is to not work / rely on benchmark solutions that just hammer at your server too much - its very complex to create a realistic scenario that way (i.e. 20 users holding down on your front page or some other action is very different from each of them doing different things). The best way to make your app faster should be to get some profiler going and see what functions / parts of your app take the longest to load and then optimize them. Yeah, I'm taking a closer look on xdebug, I will post some results if I manage to run it. From there I'd keep an eye closely on the server load during traffic peaks which you can use to generate yourself some statistics to estimate when your current horse power will not be enough anymore. Then it is time to cluster and scale your architecture. So from some point there's a question where to put your resources : trying to optimize the code or buying and adding hardware to cope with a higher load. Anyway, I'm not an expert on the matter and you may be able diagnose things with your current benchmark setup as well - however I think its
Re: Cake CPU hungry
For xdebug please read this pdf http://derickrethans.nl/files/phparch-xdebug-qa.pdf link from http://www.sitepoint.com/blogs/2006/04/20/php-frontend-for-xdebug-profiling/ Use linux and KCacheGrind. For windows:dwnload xdebug dll, and put in php/ext folder php.ini add: zend_extension_ts=C:\AppServ\php5\ext\php_xdebug.dll xdebug.auto_trace=1 xdebug.auto_profile=1 xdebug.auto_profile_mode=0 ;config ;xdebug.remote_enable=on ;xdebug.profiler_enable_trigger=on ;xdebug.profiler_output_dir=C:\AppServ\www ;xdebug.profiler_output_name=timestamp ;xdebug.profiler_enable=on There is KCacheGrind-wannabe for Windows - WinCacheGrind, for processing output. It's old (2005) software... $persistModel will cache constructed models and it's recursive associations Your app can work without mod_rewrite. Using mod_rewrite, apache seek .htaccess file in each directory in path ( from root ). @majna yeah.. I was planning to use xdebug, could you maybe provide any links on how to set it up properly ? I'll take a look at var $persistModel, debug is 0. As I already noticed var $uses is a point to optimize. What do you mean by : do not use mod_rewrite ? Cake url's are based on mod_rewrite. Thanks, Andrzej --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: Cake CPU hungry
@majna thanks for the links. I just tried $persistModel and, at first page render I was extremely happy to see all the models cached but then when I refreshed I got this : Warning (2): Invalid argument supplied for foreach() [CORE\cake\libs \object.php, line 239] Object::__openPersistent() - CORE\cake\libs\object.php, line 239 Object::_persist() - CORE\cake\libs\object.php, line 203 Controller::constructClasses() - CORE\cake\libs\controller \controller.php, line 409 Dispatcher::dispatch() - CORE\cake\dispatcher.php, line 295 [main] - APP\webroot\index.php, line 81 and I could probably live with it .. but that was not enough.. I also saw : Fatal error: Model::findAll() [a href='http://php.net/function.Model- findAll'function.Model-findAll/a]: The script tried to execute a method or access a property of an incomplete object. Please ensure that the class definition quot;SlugBehaviorquot; of the object you are trying to operate on was loaded _before_ unserialize() gets called or provide a __autoload() function to load the class definition in G: \wamp\libs\cake\cake\libs\model\model.php on line 1432 that was it for $persistModel.. it happens to all behaviors.. I still didn't get deeper into this one, any thoughts ? Your app can work without mod_rewrite. Using mod_rewrite, apache seek .htaccess file in each directory in path ( from root ). I'm using lighttpd which is having all the rewrites in .conf file, more like httpd.conf then .htaccess... and I like friendly urls too much to not use rewrites ;) Best, Andrzej --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: Cake CPU hungry
This persistModel tip was for cake 1.1.x and it works well. 1.2 has problem with behaviours and persistModel. Report ticket? mod_rewrite is a must have, I know :) On Sep 1, 9:37 pm, thequietlab [EMAIL PROTECTED] wrote: @majna thanks for the links. I just tried $persistModel and, at first page render I was extremely happy to see all the models cached but then when I refreshed I got this : Warning (2): Invalid argument supplied for foreach() [CORE\cake\libs \object.php, line 239] Object::__openPersistent() - CORE\cake\libs\object.php, line 239 Object::_persist() - CORE\cake\libs\object.php, line 203 Controller::constructClasses() - CORE\cake\libs\controller \controller.php, line 409 Dispatcher::dispatch() - CORE\cake\dispatcher.php, line 295 [main] - APP\webroot\index.php, line 81 and I could probably live with it .. but that was not enough.. I also saw : Fatal error: Model::findAll() [a href='http://php.net/function.Model- findAll'function.Model-findAll/a]: The script tried to execute a method or access a property of an incomplete object. Please ensure that the class definition quot;SlugBehaviorquot; of the object you are trying to operate on was loaded _before_ unserialize() gets called or provide a __autoload() function to load the class definition in G: \wamp\libs\cake\cake\libs\model\model.php on line 1432 that was it for $persistModel.. it happens to all behaviors.. I still didn't get deeper into this one, any thoughts ? Your app can work without mod_rewrite. Using mod_rewrite, apache seek .htaccess file in each directory in path ( from root ). I'm using lighttpd which is having all the rewrites in .conf file, more like httpd.conf then .htaccess... and I like friendly urls too much to not use rewrites ;) Best, Andrzej --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: Cake CPU hungry
ab -n 1000 -c 20 means that you have 20 concurrent users making 1000 requests. The important part here is the concurrent bit. There's a wide variety of things you can do to optimise your controller, which are way beyond the scope of a single post, especially since you don't really say anything about what your controller is doing. Turning on caching might help. The key to this is most likely memory based. Is your 70% usage just cpu, or is there wait time? Is memory swapping? You can do all sorts of things to cap rates, reduce linux's aggressive swapiness, or improve the use of disk. If memory's not the problem, maybe you could move cake's cache into ram. Cake can be very memory greedy if you use reconnect on datasources, or load a lot of models, or have a lot of relations. Lookup the expects() function on the bakery or look at bind(), unbind() if you want to deal with that the hard way. Other than that, it's probably a close investigation of you controller code and some optimisation there. Note also that ab will not provide a good test, since it does not download embedded resources (images / scripts / css etc) since it does not have any html parsing. You might do well to look at something like jakarta Jmeter, which is very good, if a little daunting for the beginner, or some of the dodgy cheap windows stuff like Website Optimizer (I think). Of course it may be that you just need a better server (the spec you stated is not exactly top of the range), or a separate database server / server cluster. You'd probably benefit hugely from a reverse proxy accelerator on a Cake site as well to take away all the js, css and image routes / rewrites / requests. Just some ideas. Hope they help. It's all a bit open-ended but I may be able to help more with a better description of the controller. Simon --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: Cake CPU hungry
try profiling with xdebug, use var $persistModel = true; in controller set debug to 0, cache views if u can, or just query results, persistent mysql connection , avoid var $uses() ClassRegistry may help sometimes do not use mod_rewrite On Aug 31, 3:46 pm, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: ab -n 1000 -c 20 means that you have 20 concurrent users making 1000 requests. The important part here is the concurrent bit. There's a wide variety of things you can do to optimise your controller, which are way beyond the scope of a single post, especially since you don't really say anything about what your controller is doing. Turning on caching might help. The key to this is most likely memory based. Is your 70% usage just cpu, or is there wait time? Is memory swapping? You can do all sorts of things to cap rates, reduce linux's aggressive swapiness, or improve the use of disk. If memory's not the problem, maybe you could move cake's cache into ram. Cake can be very memory greedy if you use reconnect on datasources, or load a lot of models, or have a lot of relations. Lookup the expects() function on the bakery or look at bind(), unbind() if you want to deal with that the hard way. Other than that, it's probably a close investigation of you controller code and some optimisation there. Note also that ab will not provide a good test, since it does not download embedded resources (images / scripts / css etc) since it does not have any html parsing. You might do well to look at something like jakarta Jmeter, which is very good, if a little daunting for the beginner, or some of the dodgy cheap windows stuff like Website Optimizer (I think). Of course it may be that you just need a better server (the spec you stated is not exactly top of the range), or a separate database server / server cluster. You'd probably benefit hugely from a reverse proxy accelerator on a Cake site as well to take away all the js, css and image routes / rewrites / requests. Just some ideas. Hope they help. It's all a bit open-ended but I may be able to help more with a better description of the controller. Simon --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: Cake CPU hungry
thanks for your answers, @Simon ab -n 1000 -c 20 means that you have 20 concurrent users making 1000 requests. The important part here is the concurrent bit. ok, sorry I said it wrong - I know what does it mean but I'm not sure what should be the values to simulate traffic at a range like 100,000 users per day. I know it depends on the peak hours etc. but is there a way to estimate some values to know what to expect ? There's a wide variety of things you can do to optimise your controller, which are way beyond the scope of a single post, especially since you don't really say anything about what your controller is doing. Turning on caching might help. These are mostly pages you view after authorization so traditional cake views caching is not of any use. I use memcache to save on database queries and it seems like mysql is not a bottleneck here. I can imagine my controller can be optimized - it's just a first phase of the deployment - but the same (or almost the same) happens with static pages which don't use any models. The key to this is most likely memory based. Is your 70% usage just cpu, or is there wait time? Is memory swapping? You can do all sorts of things to cap rates, reduce linux's aggressive swapiness, or improve the use of disk. If memory's not the problem, maybe you could move cake's cache into ram. ok, I'll take a look on the memory but it doesn't seem to be a problem, at least memory is not swapping. Cake can be very memory greedy if you use reconnect on datasources, or load a lot of models, or have a lot of relations. Lookup the expects() function on the bakery or look at bind(), unbind() if you want to deal with that the hard way. I'm using expects(), it's pretty handy. My problem might be loading lot of models but still 'pages' doesn't load any models. Other than that, it's probably a close investigation of you controller code and some optimisation there. What controller does is nothing complicated - just takes data from db - 2 queries cached with memcached so they don't even get called (cache is dynamically changing on updates) Note also that ab will not provide a good test, since it does not download embedded resources (images / scripts / css etc) since it does not have any html parsing. You might do well to look at something like jakarta Jmeter, which is very good, if a little daunting for the beginner, or some of the dodgy cheap windows stuff like Website Optimizer (I think). Sure but I just wanted to stress the application itself. Just some ideas. Hope they help. It's all a bit open-ended but I may be able to help more with a better description of the controller. THANKS for it Simon! I know it's a pretty big area and lots of things matters. I will be checking things one after another and will try to optimize it. One thing I noticed.. when I'm doing debug($this); and it dumps everything there's a lot of *RECURSION* at many points.. objects just holding themselves at many points.. I'm not sure but it might be quite a bottleneck. @majna yeah.. I was planning to use xdebug, could you maybe provide any links on how to set it up properly ? I'll take a look at var $persistModel, debug is 0. As I already noticed var $uses is a point to optimize. What do you mean by : do not use mod_rewrite ? Cake url's are based on mod_rewrite. Thanks, Andrzej --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Cake PHP group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: Cake CPU hungry
I'm using expects(), it's pretty handy. My problem might be loading lot of models but still 'pages' doesn't load any models. Do you know that for sure? If you lets say load the Users model in AppController, then it is more then likely that *all* of your other models are loaded b/c they are somehow associated with User. My suggestion is to not work / rely on benchmark solutions that just hammer at your server too much - its very complex to create a realistic scenario that way (i.e. 20 users holding down on your front page or some other action is very different from each of them doing different things). The best way to make your app faster should be to get some profiler going and see what functions / parts of your app take the longest to load and then optimize them. From there I'd keep an eye closely on the server load during traffic peaks which you can use to generate yourself some statistics to estimate when your current horse power will not be enough anymore. Then it is time to cluster and scale your architecture. Anyway, I'm not an expert on the matter and you may be able diagnose things with your current benchmark setup as well - however I think its not the easiest path. Good luck and success with your app! -- Felix -- My Blog: http://www.thinkingphp.org My Business: http://www.fg-webdesign.de thequietlab wrote: thanks for your answers, @Simon ab -n 1000 -c 20 means that you have 20 concurrent users making 1000 requests. The important part here is the concurrent bit. ok, sorry I said it wrong - I know what does it mean but I'm not sure what should be the values to simulate traffic at a range like 100,000 users per day. I know it depends on the peak hours etc. but is there a way to estimate some values to know what to expect ? There's a wide variety of things you can do to optimise your controller, which are way beyond the scope of a single post, especially since you don't really say anything about what your controller is doing. Turning on caching might help. These are mostly pages you view after authorization so traditional cake views caching is not of any use. I use memcache to save on database queries and it seems like mysql is not a bottleneck here. I can imagine my controller can be optimized - it's just a first phase of the deployment - but the same (or almost the same) happens with static pages which don't use any models. The key to this is most likely memory based. Is your 70% usage just cpu, or is there wait time? Is memory swapping? You can do all sorts of things to cap rates, reduce linux's aggressive swapiness, or improve the use of disk. If memory's not the problem, maybe you could move cake's cache into ram. ok, I'll take a look on the memory but it doesn't seem to be a problem, at least memory is not swapping. Cake can be very memory greedy if you use reconnect on datasources, or load a lot of models, or have a lot of relations. Lookup the expects() function on the bakery or look at bind(), unbind() if you want to deal with that the hard way. I'm using expects(), it's pretty handy. My problem might be loading lot of models but still 'pages' doesn't load any models. Other than that, it's probably a close investigation of you controller code and some optimisation there. What controller does is nothing complicated - just takes data from db - 2 queries cached with memcached so they don't even get called (cache is dynamically changing on updates) Note also that ab will not provide a good test, since it does not download embedded resources (images / scripts / css etc) since it does not have any html parsing. You might do well to look at something like jakarta Jmeter, which is very good, if a little daunting for the beginner, or some of the dodgy cheap windows stuff like Website Optimizer (I think). Sure but I just wanted to stress the application itself. Just some ideas. Hope they help. It's all a bit open-ended but I may be able to help more with a better description of the controller. THANKS for it Simon! I know it's a pretty big area and lots of things matters. I will be checking things one after another and will try to optimize it. One thing I noticed.. when I'm doing debug($this); and it dumps everything there's a lot of *RECURSION* at many points.. objects just holding themselves at many points.. I'm not sure but it might be quite a bottleneck. @majna yeah.. I was planning to use xdebug, could you maybe provide any links on how to set it up properly ? I'll take a look at var $persistModel, debug is 0. As I already noticed var $uses is a point to optimize. What do you mean by : do not use mod_rewrite ? Cake url's are based on mod_rewrite. Thanks, Andrzej --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google
