Thanks for that. I have actually been looking at the function cache amongst others and there is a lot we can do there as our DB records, once inserted, are not likely to change. In fact if they do it means we are having a problem as we store email data for a number of companies in them. Therefore function caching and even memory caching records as we extract them from db would probably help us a lot. It does mean more work code-wise and isn't a "quick-fix", so we plan to start looking at this once we hit Beta where performance will be a major requirement.
The old system is faster simply because it follows no design pattern except procedural and that is where its speed lies. There are no ORM's, classes or anything like that, and SQL queries are sent straight through to the database using handcoded, dynamic SQL queries as opposed to an ORM generated one and the resultsets are manipulated directly in each "view". In fact there are only views, there is little seperation of business logic and presentation. The reason we need symfony for this new version is that we are going to be adding more advanced features that would "complicate" the product beyond what a procedural style would allow us to maintain. We are already struggling to keep the older system maintained and enhanced for our customers as it is. symfony, Propel and even Prototype with scriptaculous help alleviate these maintenance and extensibility issues. On Mon, Mar 9, 2009 at 9:13 PM, Richtermeister <[email protected]> wrote: > > Hi Gareth, > > after reading all this I feel your time is most likely best spent in > smart caching, since it sounds like the DB is not your bottleneck. > What's easy to overlook when working with symfony, is that compared to > straight procedural "get data -> display data" scripts, rendering > templates with hydrated objects is slower, albeit more flexible. So, > if your previous site was coded in a bad way, it was probably using a > lot of "view specific" code, so it's hard to compete with that on pure > speed considerations. The only way to mitigate that is by using all > forms of caching, and yes, this may include the function cache > (although I don't like it much). However, the "higher up" you can > cache, i.e. a complete action, the less you have to cache on the model > level. > > Just my 2 cents. Good luck, > Daniel > > > > > > On Mar 9, 8:42 am, Sumedh <[email protected]> wrote: > > My 2 cents...slow query log in mysql should help a lot... > > > > Please let us know your insights at the end of your exercise... :) > > > > On Mar 9, 3:41 pm, Gareth McCumskey <[email protected]> wrote: > > > > > I just tried using Propel 1.3 on our application and while I would love > to > > > continue using it (as it seemed to produce a little more efficieny) we > can't > > > use it for now because the servers that the app will run on are Centos > 4 > > > with PHP 5.1.x as its maximum version for now. The sysadmins here say > that > > > to force an upgrade to 5.2.x would be a hard task as to retain RedHat > > > support it means they would need to upgrade to Centos 5. > > > > > I am currently looking at the chapter about Optimising symfony and the > > > function cache seems to be something we cna consider doing in a lot of > our > > > model calls from the action to help speed things up, especially for > model > > > methods that access historical data (i.e. stuff dated in the past that > > > obviously wont change on subsequent calls) but these are relatively > large > > > coding changes which we will probably only do during our beta > development > > > phase. > > > > > I am still looking through more advise recieved from this post and I > have to > > > thank everyone for their input. I honestly didn't expect this response > and > > > it has been fantastic and very helpful. > > > > > On Mon, Mar 9, 2009 at 12:49 AM, Crafty_Shadow <[email protected]> > wrote: > > > > > > Symfony 1.1 came by default with Propel 1.2 > > > > You can try upgrading to 1.3 (it isn't really a trivial task, but it > > > > shouldn't be a big problem) > > > > There is thorough explanation on the symfony site how to do it: > > > >http://www.symfony-project.org/cookbook/1_1/en/propel_13 > > > > It should fare a measurable increase in performance. Also, a site > that > > > > makes good use of cache should have caching for absolutely everything > > > > not session-dependent. I find it hard to imagine a php app, no matter > > > > how fast, that would run faster than symfony's cached output. > > > > > > Alvaro: > > > > Is your plugin based on Propel 1.3? > > > > If you believe you have made significant improvements to Propel, why > > > > not suggest them for version 2.0, which is still under heavy > > > > development? > > > > > > On Mar 8, 4:33 pm, alvaro <[email protected]> wrote: > > > > > At the company I developed a symfony plugin to optimize the Propel > > > > > queries and also the Propel hydrate method, improving even 5 times > > > > > query speed and also memory usage. > > > > > > > The plugins supports joins and thanks to PHP features the plugin > > > > > returns Propel objects populated with custom AS columns. > > > > > > > We are thinking on release it on the following weeks so stay tuned > :) > > > > > > > Regards, > > > > > > > Alvaro > > > > > > > On Mar 8, 2009, at 10:20 PM, Gareth McCumskey wrote: > > > > > > > > We have put numerous caching techniques into effect, from Cache- > > > > > > Expires headers to compression of static files like js and html > > > > > > files. Currently we use symfony 1.1 and Propel as the ORM. We > have > > > > > > identified the bottleneck generally as being the application > > > > > > processing after the db queries have run to extract the data. > > > > > > > > The entire point of my question was to get some info on general > tips > > > > > > and tricks we can try out to see if anything helps or if perhaps > we > > > > > > have missed any obvious issues that may actually be the cause of > the > > > > > > slow performance we are getting. As it is I have gotten quite a > few > > > > > > and look forward to getting into the office tomorrow to try them > > > > > > out. Anymore is greatly appreciated. > > > > > > > > Of course I am looking through the code to see if there is anyway > we > > > > > > can streamline it on that end, but every little bit helps. > > > > > > > > Gareth > > > > > > > > On Sun, Mar 8, 2009 at 12:27 PM, Crafty_Shadow < > [email protected]> > > > > > > wrote: > > > > > > > > Gareth, you didn't mention what version of symfony you were > using, > > > > > > also what ORM (if any). > > > > > > The best course of optimization will depend on those. Also, as > already > > > > > > mentioned, caching is your best friend. > > > > > > > > On Mar 8, 9:43 am, Gareth McCumskey <[email protected]> > wrote: > > > > > > > Well, consider a single database table that looks something > like > > > > > > this: > > > > > > > > > From_address > > > > > > > to_address (possibly multiple addresses comma-seperated) > > > > > > > headers > > > > > > > spam_report > > > > > > > subject > > > > > > > > > And we would have millions of those records in the database. > > > > > > Repeated > > > > > > > entries, especially on to_address, means the data is hugely > > > > > > redundant. By > > > > > > > normalising we are turning a text search across millions of > > > > > > records with > > > > > > > redundant repeated data into a text search over a unique list, > > > > > > then an > > > > > > > integer search over primary key (which of course is indexed). > > > > > > > > > On Sun, Mar 8, 2009 at 9:37 AM, Lawrence Krubner > > > > > > <[email protected]>wrote: > > > > > > > > > > On Mar 8, 3:26 am, Gareth McCumskey <[email protected]> > wrote: > > > > > > > > > We had a speed increase because we had a lot of text > searches > > > > > > in the old > > > > > > > > > system, all going through text fields where the same values > > > > > > were repeated > > > > > > > > > over and over. Its therefore a lot faster to search a much > > > > > > smaller table, > > > > > > > > > where the text fields are unique, and find the value once, > > > > > > then use an ID > > > > > > > > > comparison, being much faster to match integers than text. > > > > > > > > > > In sounds like you got a speed boost from doing intelligent > > > > > > indexing. > > > > > > > > What you are describing sounds more like indexing than > > > > > > normalization, > > > > > > > > at least to me. > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "symfony users" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/symfony-users?hl=en -~----------~----~----~----~------~----~------~--~---
