Hi Jon, Interesting... a drop from 35 to 23 seconds seems quite poor so I wonder if there is no concurrency occurring at all, just perhaps a better "packing" of the execution. Alternatively, perhaps the cost of running the threads is quite high. With just two tile render jobs, I doubt if there is an I/O bottleneck. I had a recent experience with running two separate processes (on a quad core machine) which take about an hour each, is that the execution time is still about the same. i.e. almost perfect parallelism.
I'll report the results of my own thread vs. process tests in the next day or two. Regards, Warren -----Original Message----- From: Jon Burgess [mailto:[email protected]] Sent: 06 September 2009 12:54 To: Warren Vick Cc: [email protected] Subject: Re: [Mapnik-users] Multi Threading On Sun, 2009-09-06 at 10:46 +0100, Jon Burgess wrote: > > On Sun, 2009-09-06 at 09:28 +0100, Warren Vick wrote: > > I've just discovered that Python is a language which does not let > > multiple threads execute at the same time, because the run parallel > > runtime support is based on a central lock (the "Global Interpreter > > Lock"). So, I guess the only way to utilise multiple > processors/cores > > is to run separate processes? Or, do all Python programs running on > a > > system share the same runtime? > > Mapnik can render lots of tiles in parallel via the python bindings. > The > GIL is released during the main rendering call so that multiple > threads > can run in parallel. I use the feature in the python mod_tile > renderd.py. > > The generate_tiles.py has not been updated because it was originally > intended as a simple example program, not a serious tool for rendering > millions of tiles. I'll take a look at adding in some multithreading > into generate_tiles.py. I have checked in a version of generate_tiles.py which uses multiple rendering threads. On a 2 core machine this reduces the rendering time for the "world 0 - 4" tiles from 35 to 23 seconds. The new code also makes use of some of the more recent Mapnik features like the buffer_size and png256 output format. Jon _______________________________________________ Mapnik-users mailing list [email protected] https://lists.berlios.de/mailman/listinfo/mapnik-users
