Hi Gareth, that sounds like a nifty way you came up with, and it is indeed often used, even within javascript to cache the result of ajax calls by their url hash. What I was talking about earlier regarding high-level caching, is that if you apply the caching to the view layer, just saving the html using symfony's building caching configuration, those queries would also only run once, and you would avoid complicating your code.. Does that make sense?
Daniel On Mar 9, 11:54 pm, Gareth McCumskey <gmccums...@gmail.com> wrote: > I have actually come across a rather interesting way to use the memory cache > as specified in the book > (http://www.symfony-project.org/book/1_1/18-Performance#chapter_18_sub... > ) > > The way it is described to store data in the cache needs three things; a > unique name that can be easily determined, the value related to that name > and how long that value stays cached. In our application, our database > records will very very rarely see any updates. Once records are inserted > they are only ever retrieved making them ideal for caching. But we cannot > store every database record into memory. We do have queries that run very > frequently, however, and each Criteria object i sunique for each query with > the various values and so on. The problem is you cannot store an object, > like the Criteria object that is built up before you run a Propel query with > doSelect or similar methods. So, if you serialize the Criteria object, you > have a string. But this is a very long string (one of the serialized > Criteria objects I tested with was over 1000 characters long). But you can > convert it to a hash value. Naturally there is one problem with a hash; the > possibility that two different string would create the same hash. So create > two hashes and concatenate them, an SHA1 hash and an MD5 hash. This can then > be the name of the item you are storing in memory. Run the DB query and you > have the value of the query, which can then also be stored into the cache. > > As a brief example for what I mean by overriding the doSelect method for a > model class: > > private static function doSelect($c) > { > $serialized_c = serialize($c); > $md5_serialized = md5($serialized_c); > $sha1_serialized = sha1($serialized_c); > > $cache = new sfAPCCache(); > > if ($cache->has('doSelect'.$md5_serialized.$sha1_serialized)) > { > $query_value = > $cache->get('doSelect'.$md5_serialized.$sha1_serialized); > } > else > { > $query_value = parent::doSelect($c); > $cache->set('doSelect'.$md5_serialized.$sha1_serialized, $query_value, > 60); > } > > return $query_value; > > } > > Any comments would be great on this or any problems with doing this kind of > thing that I may not have seen please feel free to let me know > > On Tue, Mar 10, 2009 at 8:06 AM, Gareth McCumskey <gmccums...@gmail.com>wrote: > > > Thanks for that. I have actually been looking at the function cache amongst > > others and there is a lot we can do there as our DB records, once inserted, > > are not likely to change. In fact if they do it means we are having a > > problem as we store email data for a number of companies in them. Therefore > > function caching and even memory caching records as we extract them from db > > would probably help us a lot. It does mean more work code-wise and isn't a > > "quick-fix", so we plan to start looking at this once we hit Beta where > > performance will be a major requirement. > > > The old system is faster simply because it follows no design pattern except > > procedural and that is where its speed lies. There are no ORM's, classes or > > anything like that, and SQL queries are sent straight through to the > > database using handcoded, dynamic SQL queries as opposed to an ORM generated > > one and the resultsets are manipulated directly in each "view". In fact > > there are only views, there is little seperation of business logic and > > presentation. > > > The reason we need symfony for this new version is that we are going to be > > adding more advanced features that would "complicate" the product beyond > > what a procedural style would allow us to maintain. We are already > > struggling to keep the older system maintained and enhanced for our > > customers as it is. symfony, Propel and even Prototype with scriptaculous > > help alleviate these maintenance and extensibility issues. > > > On Mon, Mar 9, 2009 at 9:13 PM, Richtermeister <nex...@gmail.com> wrote: > > >> Hi Gareth, > > >> after reading all this I feel your time is most likely best spent in > >> smart caching, since it sounds like the DB is not your bottleneck. > >> What's easy to overlook when working with symfony, is that compared to > >> straight procedural "get data -> display data" scripts, rendering > >> templates with hydrated objects is slower, albeit more flexible. So, > >> if your previous site was coded in a bad way, it was probably using a > >> lot of "view specific" code, so it's hard to compete with that on pure > >> speed considerations. The only way to mitigate that is by using all > >> forms of caching, and yes, this may include the function cache > >> (although I don't like it much). However, the "higher up" you can > >> cache, i.e. a complete action, the less you have to cache on the model > >> level. > > >> Just my 2 cents. Good luck, > >> Daniel > > >> On Mar 9, 8:42 am, Sumedh <sumedh.inam...@gmail.com> wrote: > >> > My 2 cents...slow query log in mysql should help a lot... > > >> > Please let us know your insights at the end of your exercise... :) > > >> > On Mar 9, 3:41 pm, Gareth McCumskey <gmccums...@gmail.com> wrote: > > >> > > I just tried using Propel 1.3 on our application and while I would > >> love to > >> > > continue using it (as it seemed to produce a little more efficieny) we > >> can't > >> > > use it for now because the servers that the app will run on are Centos > >> 4 > >> > > with PHP 5.1.x as its maximum version for now. The sysadmins here say > >> that > >> > > to force an upgrade to 5.2.x would be a hard task as to retain RedHat > >> > > support it means they would need to upgrade to Centos 5. > > >> > > I am currently looking at the chapter about Optimising symfony and the > >> > > function cache seems to be something we cna consider doing in a lot of > >> our > >> > > model calls from the action to help speed things up, especially for > >> model > >> > > methods that access historical data (i.e. stuff dated in the past that > >> > > obviously wont change on subsequent calls) but these are relatively > >> large > >> > > coding changes which we will probably only do during our beta > >> development > >> > > phase. > > >> > > I am still looking through more advise recieved from this post and I > >> have to > >> > > thank everyone for their input. I honestly didn't expect this response > >> and > >> > > it has been fantastic and very helpful. > > >> > > On Mon, Mar 9, 2009 at 12:49 AM, Crafty_Shadow <vankat...@gmail.com> > >> wrote: > > >> > > > Symfony 1.1 came by default with Propel 1.2 > >> > > > You can try upgrading to 1.3 (it isn't really a trivial task, but it > >> > > > shouldn't be a big problem) > >> > > > There is thorough explanation on the symfony site how to do it: > >> > > >http://www.symfony-project.org/cookbook/1_1/en/propel_13 > >> > > > It should fare a measurable increase in performance. Also, a site > >> that > >> > > > makes good use of cache should have caching for absolutely > >> everything > >> > > > not session-dependent. I find it hard to imagine a php app, no > >> matter > >> > > > how fast, that would run faster than symfony's cached output. > > >> > > > Alvaro: > >> > > > Is your plugin based on Propel 1.3? > >> > > > If you believe you have made significant improvements to Propel, why > >> > > > not suggest them for version 2.0, which is still under heavy > >> > > > development? > > >> > > > On Mar 8, 4:33 pm, alvaro <harryjek...@gmail.com> wrote: > >> > > > > At the company I developed a symfony plugin to optimize the Propel > >> > > > > queries and also the Propel hydrate method, improving even 5 times > >> > > > > query speed and also memory usage. > > >> > > > > The plugins supports joins and thanks to PHP features the plugin > >> > > > > returns Propel objects populated with custom AS columns. > > >> > > > > We are thinking on release it on the following weeks so stay tuned > >> :) > > >> > > > > Regards, > > >> > > > > Alvaro > > >> > > > > On Mar 8, 2009, at 10:20 PM, Gareth McCumskey wrote: > > >> > > > > > We have put numerous caching techniques into effect, from Cache- > >> > > > > > Expires headers to compression of static files like js and html > >> > > > > > files. Currently we use symfony 1.1 and Propel as the ORM. We > >> have > >> > > > > > identified the bottleneck generally as being the application > >> > > > > > processing after the db queries have run to extract the data. > > >> > > > > > The entire point of my question was to get some info on general > >> tips > >> > > > > > and tricks we can try out to see if anything helps or if perhaps > >> we > >> > > > > > have missed any obvious issues that may actually be the cause of > >> the > >> > > > > > slow performance we are getting. As it is I have gotten quite a > >> few > >> > > > > > and look forward to getting into the office tomorrow to try them > >> > > > > > out. Anymore is greatly appreciated. > > >> > > > > > Of course I am looking through the code to see if there is > >> anyway we > >> > > > > > can streamline it on that end, but every little bit helps. > > >> > > > > > Gareth > > >> > > > > > On Sun, Mar 8, 2009 at 12:27 PM, Crafty_Shadow < > >> vankat...@gmail.com> > >> > > > > > wrote: > > >> > > > > > Gareth, you didn't mention what version of symfony you were > >> using, > >> > > > > > also what ORM (if any). > >> > > > > > The best course of optimization will depend on those. Also, as > >> already > >> > > > > > mentioned, caching is your best friend. > > >> > > > > > On Mar 8, 9:43 am, Gareth McCumskey <gmccums...@gmail.com> > >> wrote: > >> > > > > > > Well, consider a single database table that looks something > >> like > >> > > > > > this: > > >> > > > > > > From_address > >> > > > > > > to_address (possibly multiple addresses comma-seperated) > >> > > > > > > headers > >> > > > > > > spam_report > >> > > > > > > subject > > >> > > > > > > And we would have millions of those records in the database. > >> > > > > > Repeated > >> > > > > > > entries, especially on to_address, means the data > > ... > > read more » --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "symfony users" group. To post to this group, send email to symfony-users@googlegroups.com To unsubscribe from this group, send email to symfony-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/symfony-users?hl=en -~----------~----~----~----~------~----~------~--~---
