OK, so let me update briefly on some changes I have done. The only bad part here is, none of the changes *singly* have made a large difference in performance, but all together it seems to be running much better.
I set Apache to have 100 process threads rather than 250 I disabled DEP on Windows (which was enabled by default) I disabled the source browsing from all hosted trac environments I disabled all authentication to the /chrome folder, my <Location directive for the chrome folder looks as such: <Location /chrome> Satisfy Any Allow from all ExpiresActive On ExpiresDefault "access plus 30 days" </Location> I ran some IO tests on the S3 drive vs the host OS drive, and while the S3 drive did show some high variances on speed, it also had peaks of very quick performance (as I would expect from a highly shared resource), so I do not believe that is my sole IO issue. Performance seems much better overall now, and normal page requests are no longer peaking the CPU quite so badly, but the wiki page still consumes 100% CPU for approx 1 - 2 seconds, then drops back down. I think (at this late time at night on the weekend) that performance is acceptable now, but I will need to get it under load before I know for certain. Each one of these changes have seemed to cause an improvement, and all together it seems pretty happy now =) Thanks to everyone for all your idea's. I believe my next steps here will be to update to the 1.6.2 release of SVN, and I believe I will also update the Sqlite databases to either MySQL or PostgreSQL and see if I can find some additional performance gains that way. If anyone has a recommendation on which way to go with the Trac database for performance and ease of upgrading 50 environments or so, please feel free to chime in! Hope everyone is having a great evening, Chris Peterson Alagad, Inc. On 6/5/09 12:57 PM, "Christian Boos" <cb...@neuf.fr> wrote: > > Chris Peterson wrote: >> I work with Doug on this issue, and I wanted to provide some >> additional details about our configuration and what I am seeing. >> >> The server is an Amazon EC2 small instance, running on a dual core >> Opteron 2.6 ghz w/ 1.66 gig of ram >> The Trac install lives in the default c:\python25 folder, and the >> trac / svn data lives on an S3 drive. >> > > Ah well, you're the author of comment:57 - didn't you indicate there > that changing the log_level was helping? > Also, did you test on some similar local machine as well? > > >> We have 49 trac environments running here, 4 of which have their own v- >> host configured, the rest are for internal use and share a root url. >> All of these environments also have SVN setup. The majority of the >> shared instances have a common configuration base trac.ini file, with >> individual files for environment specific configurations. >> >> I started out with the 0.11 install of Trac, and have since upgraded >> to the 0.12dev-r8263 release from SVN (using easy_install and pointing >> to the SVN repo location), to try and address our performance issues. >> Under 'About Trac', here is my current config: >> >> Trac: 0.12dev-r8263 >> Python: 2.5.4 (r254:67916, Dec 23 2008, 15:10:54) [MSC v.1310 >> 32 bit (Intel)] >> setuptools: 0.6c9 >> SQLite: 3.5.2 >> pysqlite: 2.4.0 >> Genshi: 0.6dev-r1052 >> Babel: - >> mod_python: 3.3.1 >> Pygments: 1.0 >> Subversion: 1.5.4 (r33841) >> jQuery: 1.1.3.1 >> > Side note, should be jQuery: 1.3.2, no? (that's what is checked in into > trunk, AFAICT). >> I have Apache connecting using mod_python, but this slow performance >> and 100% CPU peak usage persists even if I load a single environment >> using tracd and shut-down the apache instance. > > Ok, that's very interesting: you're able to reproduce the perfomance > issue with tracd serving a single environment. > Let's forget Apache for now and focus on tracd. How exactly do you start > it, btw? > >> I have gone through the >> site-packages and, one by one, removed each plugin from the folder and >> re-started the instance, and I still saw the performance issues (I was >> trying to isolate the issue to a single plugin, without luck). >> > > That's also interesting: Trac trunk alone /without any plugin/ shows the > same performance issue. > Did you try to disable the browser component as well? ([components] > trac.versioncontrol.* = disabled) > I see you're using svn 1.6.0, which might be an issue, as it was mostly > untested - I use 1.6.2. > > >> I applied the patch as suggested in >> http://trac.edgewall.org/ticket/7785#comment:13 >> and have seen no improvement. I also have logging currently disabled >> completely in the hopes of improving performance, still without luck. >> >> I did a directory output from my site-packages folder, if it will >> provide better detail as to plugins I have installed, that you can >> view here: http://pastebin.com/fea11695 >> >> The worst performance seems to be on the wiki pages, but viewing >> tickets spins the CPU up quite high as well. > > I wonder if that could be related to wiki processing? Are you really > positive that /no/ plugins were used? (btw, you should also post a log > file at DEBUG level, starting from the startup of tracd till viewing a > few of those pages where the CPU spikes). > > We did have some performance issues in the past for some specific > "pathological" markups, maybe you could post some typical wiki content > (unless the performance problem also happens for unmodified Trac pages > from the TracGuide). > >> At one point I attempted >> to implement the Psyco package (http://psyco.sourceforge.net/) to >> improve performance, but all plugins have been updated without it >> again as I saw no performance gains from using it (the package is >> still in my site-packages, but the code is not in use) >> >> At this point I am (sadly) thinking that I may need to re-configure a >> new server with a clean / new install of python and trac and see how >> it runs, I really do not know what else to investigate to resolve this >> problem. Any idea's, as wild as they may be, would be greatly >> appreciated! >> > > Well, maybe it's due to something specific to Amazon EC2, so please try > first to reproduce the issue on a single isolated environment using a > bare bone tracd from trunk running on a "normal" XP or Vista system of > yours, then do the same minimalistic test on Amazon EC2. > > -- Christian > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Trac Users" group. To post to this group, send email to trac-users@googlegroups.com To unsubscribe from this group, send email to trac-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/trac-users?hl=en -~----------~----~----~----~------~----~------~--~---
