On October 22, 2009, Graham Binns wrote: > Hi folks, > > We've been seeing repeated timeouts in the +filebug page where users are > coming from apport (so +filebugs/$token, to be precise). These timeouts > often look like they're happening in the LoginStatus code, but some > investigation proves this to be a red herring. > > My current theory is that: > > 1. A user comes to +filebug/$token > 2. FileBugViewBase.publishTraverse() handles the token, fetches the > LibraryFileAlias to which it points and then passes that to a > FileBugDataParser. > 3. FileBugViewBase.publishTraverse() calls FileBugDataParser.parse() > 4. Time passes. > 5. More time passes. > 6. FileBugDataParser.parse() completes and the request continues, but > parse() has taken so long to run (~30 seconds) that by the time the > LoginStatus code is being run the timeout limit kicks in and the > request is given the nuclear boot of doom. > > FileBugDataParser.parse() is in fact pretty much one big while loop > (lib/lp/bugs/browser/bugtarget.py:187), looping over the contents of the > file that apport has attached and dealing with them appropriately. I'm > pretty certain that the problem we're having is just one of too much > data; the files that were being uploaded by apport in the cases I looked > at were circa 90MB in size, and they're going to take a while to parse, > whichever way you look at it. > > Now, as far as I can tell - without studying the loop in detail and > trying to find ways to slim it down - the only real way to fix this is > to move the processing of the apport data elsewhere, so that it doesn't > impact on the user's session. As I see it, the options are: > > 1. Create a script that processes apport data and make it possible for > the +filebug process to tell it "Hey, this LibraryFileAlias is mine, > please process it and update this bug appropriately" after the bug > has been filed. > 2. Make it so that the apport data get processed before the user is > pointed at +filebug, so that the requisite data are available to > +filebug as via a series of queries instead of locked away in a > BLOB. > 3. A variation on option 1, whereby +filebug will only use the > asynchronous method for files over a certain size, e.g. 25MB or so). >
Another possibility is that when the user hits +filebug, a page showing a spinner and telling the user "Processing apport data...". That page would fire a job to parse the data, and then you poll every 5 seconds to see if the parse completed. You can do it using AJAX, but you'll need to have a non-JS version that only refresh the whole page every 10 seconds for people without JS. -- Francis J. Lacoste [email protected]
signature.asc
Description: This is a digitally signed message part.
_______________________________________________ Mailing list: https://launchpad.net/~launchpad-dev Post to : [email protected] Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
