Re: CakePHP and scalability
> http://www.mysqlperformanceblog.com/ very interesting. From reading http://www.mysqlperformanceblog.com/2009/09/19/multi-column-indexes-vs-index-merge/ it would appear that for join tables, indexes with both keys would be best, rather than a single key per column. Good tip for the day that :) Cheers, Jon > > I hope you have seen this blog? They have had a number of post lately > about indexes and always post about optimization in one way or the > other (funny enough). > > If you have added indexes which "did not improve much" you may be > defining them wrong for the query you make. You may also need to > modify your queries so that they can make use of the indexes. When > adding an index it is good to run an "explain" on the query before and > after to see that the index is being used. > > 300'000rows for a join-table doesn't sound like too much for one > server unless you really need to scan most of that table for a lot of > queries... like showing a users friends friends friends... I have that > and more in both "normal" tables and joining tables, though your app > probably sees more requests than mine (not a public app). > > I wanted to get to the point that I used to have huge problems with > performance. 9/10 of them needed php related optimizations like > Containable or doing 3 queries instead of a for loop or a Set::extract > (). Once I needed to cache the returned data from Mysql... when > generating reporting data and identical queries could be issued 200 > times during the same "request". > > > > On Sep 21, 11:00 pm, byqsri wrote: >> My table is about 30 records with only 5 fields >> It's a jointable of a HABTM relation. >> But I use this table in some INNER JOIN in some queries that are very >> slowly. >> I use memcache. >> In these cases what is the best practise to consider? >> On 21 Set, 20:20, "Ma'moon" wrote: >> >> >> >> > i was successfully able to handle tables with more than 200 records >> > "yes, above 2 million records!" with CakePHP and with a very acceptable >> > performance and speed, the first thing that you should be thinking of is >> > sharding the database into smaller chunks "DBs", second thing is enabling >> > "memcached" to reduce DB access "specially reducing the master DB hits", >> > thirdly enable APC or whatever bytecode cacher you might be able to use >> > "this will gain your web server a very good performance boost", and finally >> > set the debug level for you cake app to "0" to cache the tables "DESC" >> > operations. >> > Taking all the above into consideration CakePHP can be a very powerful >> > framework to handle huge databases!. >> >> > On Mon, Sep 21, 2009 at 8:50 PM, Miles J wrote: >> >> > > Well how big is your table? I have tables with over 200,000+ rows and >> > > it works just fine. >> >> > --http://phpirate.net > > > -- jon bennett w: http://www.jben.net/ iChat (AIM): jbendotnet Skype: jon-bennett --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "CakePHP" group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to cake-php+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: CakePHP and scalability
http://www.mysqlperformanceblog.com/ I hope you have seen this blog? They have had a number of post lately about indexes and always post about optimization in one way or the other (funny enough). If you have added indexes which "did not improve much" you may be defining them wrong for the query you make. You may also need to modify your queries so that they can make use of the indexes. When adding an index it is good to run an "explain" on the query before and after to see that the index is being used. 300'000rows for a join-table doesn't sound like too much for one server unless you really need to scan most of that table for a lot of queries... like showing a users friends friends friends... I have that and more in both "normal" tables and joining tables, though your app probably sees more requests than mine (not a public app). I wanted to get to the point that I used to have huge problems with performance. 9/10 of them needed php related optimizations like Containable or doing 3 queries instead of a for loop or a Set::extract (). Once I needed to cache the returned data from Mysql... when generating reporting data and identical queries could be issued 200 times during the same "request". On Sep 21, 11:00 pm, byqsri wrote: > My table is about 30 records with only 5 fields > It's a jointable of a HABTM relation. > But I use this table in some INNER JOIN in some queries that are very > slowly. > I use memcache. > In these cases what is the best practise to consider? > On 21 Set, 20:20, "Ma'moon" wrote: > > > > > i was successfully able to handle tables with more than 200 records > > "yes, above 2 million records!" with CakePHP and with a very acceptable > > performance and speed, the first thing that you should be thinking of is > > sharding the database into smaller chunks "DBs", second thing is enabling > > "memcached" to reduce DB access "specially reducing the master DB hits", > > thirdly enable APC or whatever bytecode cacher you might be able to use > > "this will gain your web server a very good performance boost", and finally > > set the debug level for you cake app to "0" to cache the tables "DESC" > > operations. > > Taking all the above into consideration CakePHP can be a very powerful > > framework to handle huge databases!. > > > On Mon, Sep 21, 2009 at 8:50 PM, Miles J wrote: > > > > Well how big is your table? I have tables with over 200,000+ rows and > > > it works just fine. > > > --http://phpirate.net --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "CakePHP" group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to cake-php+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: CakePHP and scalability
My table is about 30 records with only 5 fields It's a jointable of a HABTM relation. But I use this table in some INNER JOIN in some queries that are very slowly. I use memcache. In these cases what is the best practise to consider? On 21 Set, 20:20, "Ma'moon" wrote: > i was successfully able to handle tables with more than 200 records > "yes, above 2 million records!" with CakePHP and with a very acceptable > performance and speed, the first thing that you should be thinking of is > sharding the database into smaller chunks "DBs", second thing is enabling > "memcached" to reduce DB access "specially reducing the master DB hits", > thirdly enable APC or whatever bytecode cacher you might be able to use > "this will gain your web server a very good performance boost", and finally > set the debug level for you cake app to "0" to cache the tables "DESC" > operations. > Taking all the above into consideration CakePHP can be a very powerful > framework to handle huge databases!. > > On Mon, Sep 21, 2009 at 8:50 PM, Miles J wrote: > > > Well how big is your table? I have tables with over 200,000+ rows and > > it works just fine. > > --http://phpirate.net --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "CakePHP" group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to cake-php+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: CakePHP and scalability
i was successfully able to handle tables with more than 200 records "yes, above 2 million records!" with CakePHP and with a very acceptable performance and speed, the first thing that you should be thinking of is sharding the database into smaller chunks "DBs", second thing is enabling "memcached" to reduce DB access "specially reducing the master DB hits", thirdly enable APC or whatever bytecode cacher you might be able to use "this will gain your web server a very good performance boost", and finally set the debug level for you cake app to "0" to cache the tables "DESC" operations. Taking all the above into consideration CakePHP can be a very powerful framework to handle huge databases!. On Mon, Sep 21, 2009 at 8:50 PM, Miles J wrote: > > Well how big is your table? I have tables with over 200,000+ rows and > it works just fine. > > > -- http://phpirate.net --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "CakePHP" group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to cake-php+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: CakePHP and scalability
Well how big is your table? I have tables with over 200,000+ rows and it works just fine. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "CakePHP" group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to cake-php+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: CakePHP and scalability
My big problem is that I have some tables there are very and very huge. I have add indexes but the situation is not improved much Is someone can suggest me how can I manage very large table with mysql and CakePHP without scaling mysql? On 21 Set, 17:25, "j0n4s.h4rtm...@googlemail.com" wrote: > In general, besides the noted tips: > - Take a look at lazyloader/lazyloading models > - Do not use Model->recursive > 0, use Containable. You might look > into Translateable as well > - DEBUG sql outputs, see where the heavy load is happening. > > On Sep 21, 9:56 am, "marco.rizze...@gmail.com" > > wrote: > > Hi > > I have developed my web application with CakePHP. > > For the moment I use one Apache server and one MySql server. > > Now I have noted that my application is getting slow because the > > numbers of users is increasing. > > So I have decided that is the time to scale my architecture. > > I have tried to find some suggestions on this forum about how scale > > CakePHP but I haven't found anything. > > Can someone help me about this with some example, link, tutorial etc.? > > Many Thanks > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "CakePHP" group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to cake-php+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: CakePHP and scalability
While not having much of experience with scaling, scaling the DB by using InnoDB + UUIDs and Replication should not be a huge Problem. What I would do for the application is adding a proxy that redirects like www1.domain.tld www2.domain.tld and so on, though I have no knowledge about implementation it works for huge download services like RS very well, and THEY got massive traffic and heavy cpu usage for sure. On Sep 21, 10:09 am, Bert Van den Brande wrote: > First you need to find out why your application is getting slower, this can > be a database issue or a PHP scripting issue ... or a combination of both :) > > Lots of caching options are provided by Cake, those should get you a long > way in improving the performance of your site. > > Scaling to multiple web- and database servers is imho only a last resort, as > this will increase the overhead in maintenance, backup and deploying > significantly. > > Friendly greetings, > Bert > > On Mon, Sep 21, 2009 at 9:56 AM, marco.rizze...@gmail.com < > > marco.rizze...@gmail.com> wrote: > > > Hi > > I have developed my web application with CakePHP. > > For the moment I use one Apache server and one MySql server. > > Now I have noted that my application is getting slow because the > > numbers of users is increasing. > > So I have decided that is the time to scale my architecture. > > I have tried to find some suggestions on this forum about how scale > > CakePHP but I haven't found anything. > > Can someone help me about this with some example, link, tutorial etc.? > > Many Thanks --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "CakePHP" group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to cake-php+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: CakePHP and scalability
In general, besides the noted tips: - Take a look at lazyloader/lazyloading models - Do not use Model->recursive > 0, use Containable. You might look into Translateable as well - DEBUG sql outputs, see where the heavy load is happening. On Sep 21, 9:56 am, "marco.rizze...@gmail.com" wrote: > Hi > I have developed my web application with CakePHP. > For the moment I use one Apache server and one MySql server. > Now I have noted that my application is getting slow because the > numbers of users is increasing. > So I have decided that is the time to scale my architecture. > I have tried to find some suggestions on this forum about how scale > CakePHP but I haven't found anything. > Can someone help me about this with some example, link, tutorial etc.? > Many Thanks --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "CakePHP" group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to cake-php+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: CakePHP and scalability
Hi marco, > I have developed my web application with CakePHP. > For the moment I use one Apache server and one MySql server. > Now I have noted that my application is getting slow because the > numbers of users is increasing. > So I have decided that is the time to scale my architecture. > I have tried to find some suggestions on this forum about how scale > CakePHP but I haven't found anything. > Can someone help me about this with some example, link, tutorial etc.? Just checking, but have you optimised your app already? * caching views * using contain and fields for every model in the contain to reduce the mysql hit * caching the data returned * using Memcached over file system caching * installing eAccellorator to speed up php execution Are you on shared, vps or a dedicated box? If one of the latter two, could you upgrade your server with more ram/cpu? What about switching to a more efficient server like ngnix ? Otherwise, I think you could move to one server for DB, and one (or more) for the web server. Cheers, Jon -- jon bennett w: http://www.jben.net/ iChat (AIM): jbendotnet Skype: jon-bennett --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "CakePHP" group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to cake-php+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: CakePHP and scalability
First you need to find out why your application is getting slower, this can be a database issue or a PHP scripting issue ... or a combination of both :) Lots of caching options are provided by Cake, those should get you a long way in improving the performance of your site. Scaling to multiple web- and database servers is imho only a last resort, as this will increase the overhead in maintenance, backup and deploying significantly. Friendly greetings, Bert On Mon, Sep 21, 2009 at 9:56 AM, marco.rizze...@gmail.com < marco.rizze...@gmail.com> wrote: > > Hi > I have developed my web application with CakePHP. > For the moment I use one Apache server and one MySql server. > Now I have noted that my application is getting slow because the > numbers of users is increasing. > So I have decided that is the time to scale my architecture. > I have tried to find some suggestions on this forum about how scale > CakePHP but I haven't found anything. > Can someone help me about this with some example, link, tutorial etc.? > Many Thanks > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "CakePHP" group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to cake-php+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
Re: CakePHP and scalability
hi marco, It's early, not had coffee yet - and I missed this crucial comment: >> For the moment I use one Apache server and one MySql server. As you've already separated mysql and apache, how about adding another apache server, using memcached to share sessions between the 2, and something at the front to randomly decide which server a request goes to. Might be useful if your assets and uploads are in a central place as well, so you don't need to duplicate them or worry about how you route traffic between the 2 servers. I've not done this directly, but it shouldn't be much different from load balancing any other (php) web app. cheers, j -- jon bennett w: http://www.jben.net/ iChat (AIM): jbendotnet Skype: jon-bennett --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "CakePHP" group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to cake-php+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~--~~~~--~~--~--~---
