Hello Thomas, 2012/4/16 Thomas Petazzoni <[email protected]>: > From the investigations Maxime and I conducted today, it seems like > Mapnik keeps the .shp (and related) files opened once the rendering is > terminated. We use such files for the grid and the administrative > boundary shape, one per page in multi-page renderings. So for a 50 > pages multi-page rendering, there are about 104 (50*2 + 2 for the cover > + 2 for the overview) shape files that remain open.
I am glad to see that your are making progress in locating this annoying bug. Locating it is half fixing it. ;-) > An option would be to move the rendering process into a separate Unix > process for every rendering, so that all resources (memory, file > descriptors, etc.) are cleaned up. I think this option is the most tolerant to this error and other similar errors that would certainly occur in the future. While it might not be the cleanest from an idealistic point of view, it uses the isolation features of our underlying OS and ensures that everything is properly cleaned-up. In fact, I thought it was the way the rendering pipeline was implemented. :-) Regarding performance, I think (but not measured) that the overhead of forking a process is negligible compared to other overheads we have, like database queries. > Another option would be to fix the problem itself, but it seems to be > within Mapnik internals and/or the Python API that does not provide the > way of telling Mapnik to release the resources it has taken. But if you want to follow this path, have you tried to trigger the garbage collector through gc.collect()? I don't know Python well so there might be another method to call, to finalize certain objects for example. http://docs.python.org/library/gc.html Best regards, david
