On Wed, Dec 31, 2014 at 6:15 AM, <[email protected]> wrote: > Hi Nathan, hi all, > > I've just debugged and fixed this memory leak. Please check with newest > default branch tip if you like. > > The issue was that a list of warnings was getting duplicated with each new > process. So you'd get > exponentially increasing memory usage, and if you have either a large > amount of warnings or a large > amount of documents, boom :) > > Thanks for the patience, > Georg >
Thanks for the quick fix! I confirm that this fixes the issue in our docs build. I *really* appreciate you putting in the time to fix this, especially over the holiday. > > *Gesendet:* Mittwoch, 31. Dezember 2014 um 01:23 Uhr > *Von:* "Nathan Goldbaum" <[email protected]> > *An:* [email protected] > *Betreff:* Memory leak for parallel docs build > Hi all, > > For the past few weeks, the developers for the yt project have > been struggling with an issue in our docs build. > > We recently switched to the latest development version of sphinx > to take advantage of the new parallel reading capabilities. I'm > currently using the tip of the default branch (Sphinx 1.3b3). > > I'm using the sphinx build for the yt project: > > https://bitbucket.org/yt_analysis/yt/src/yt/doc/ > > I've modified our Sphinx configuration to do a parallel build on > 8 cores and to not build some custom directives we use to > evaluate code in the docs build: > > https://bpaste.net/show/a541fd4a6aa8 > > The memory errors still happen with our custom sphinx extensions > turned on I've just disabled them here to isolate the issue to the > autosummary extension. > > The reading phase proceeds without issues, but during the write > phase after 10% of the documents have been written, I see (via top) > memory spikes on all the worker processes - each worker consumes > 10s of gigabytes of ram. Shortly after this, the whole build > crashes. I don't quite understand the traceback, but I think > it's happening when Sphinx tries to serialize and transmit a huge > amount of data between workers or the master process. I've > pasted the output from the build here, along with the traceback sphinx > reports when the build crashes: > > https://bpaste.net/show/39abed83b80f > > One extra hint is that if I turn off the autosummary extension, > the build proceeds fine in parallel. My guess is that we're > exposing a scaling issue in sphinx or docutils that only becomes > important for a build that contains a really large number of > output pages. > > We've been using the autosummary extension for years with no > trouble, and have been using parallel writing since it became > available in Sphinx 1.2. I guess this is due to a change in the > way parallel writing works in Sphinx 1.3? Another hint is that we've > been able to reproduce this on three different build machines now. > > Does anyone have any hints on debugging memory leaks like this? > I'd like to be able to get a traceback on one of the worker > processes once the memory usage spike has started. > > Thanks for your help! > > -Nathan > > > -- > You received this message because you are subscribed to the Google Groups > "sphinx-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "sphinx-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/sphinx-users. For more options, visit https://groups.google.com/d/optout.
