On Sun, Nov 14, 2010 at 12:00 AM, Robert Collins <[email protected]> wrote: > We had a period - about 36 hours long - where the xmlrpc internal > server was simply saturated and overloaded. During this period folk > would experience a backtrace as described in > https://bugs.launchpad.net/launchpad/+bug/674416.
There were no reports that I can find of this symptom overnight - I looked at new Launchpad bugs, lp-ops, #bzr, #launchpad and #launchpad-dev. I think we're clean. \o/ Colin Watson has noted that we have some cleanup to do - we need to trigger a 'branchChanged' event for all the branches which had that event crash. > Time Out Counts by Page ID > > Hard Soft Page ID > 384 578 CodehostingApplication:CodehostingAPI > 94 832 CodeImportSchedulerApplication:CodeImportSchedulerAPI > 89 14 Person:+commentedbugs > 14 39 BugTask:+index > 10 16 Archive:EntryResource:getBuildSummariesForSourceIds > 4 0 https://api.edge.launchpad.net > 4 0 ProjectGroup:+milestones > 2 37 Distribution:+bugtarget-portlet-bugfilters-stats > 2 7 DistroSeriesLanguage:+index > 2 4 Distribution:+archivemirrors > (from https://devpad.canonical.com/~lpqateam/lpnet-oops.html#time-outs) > > Again, if it was over-saturation causing the issue, we should expect > the first two rows to stay constant (or nearly so) over the remainder > of the day. Its been stable for a good 30+ minutes now - in fact > +commented bugs just passed the codeimportschedulerAPI, with the xml > resources staying constant. This is a good sign. Current figures are: Time Out Counts by Page ID Hard Soft Page ID 384 578 CodehostingApplication:CodehostingAPI 173 36 Person:+commentedbugs 94 832 CodeImportSchedulerApplication:CodeImportSchedulerAPI 20 107 BugTask:+index 11 1 DistributionSourcePackage:+addquestion 10 18 Archive:EntryResource:getBuildSummariesForSourceIds 5 2 ProjectGroup:+milestones 4 153 Distribution:+bugtarget-portlet-bugfilters-stats 4 32 MaloneApplication:+bugs 4 0 https://api.edge.launchpad.net So the two xmlrpc calls have not failed over the last 7 hours. Excellent. -Rob _______________________________________________ Mailing list: https://launchpad.net/~launchpad-dev Post to : [email protected] Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
