Re: [mkgmap-dev] [PATCH v10] make maps in parallel
On Tue, May 19, 2009 at 10:33:01AM +0100, Mark Burton wrote: Further testing shows that you can get different output on subsequent runs on the same input (ignoring the time stamps). So far, all of the differences appear to be caused by data being output in a different order rather than the data itself being different. So, I believe that the resulting maps are still good. You should understand that the ordering of some of the map's data structures is not important as long as all the cross-references match up. Has anyone written a lint program for *.img files that would validate all the cross-references? Or a program to pretty-print the data structures in sorted format? The sorted pretty-print should be identical across runs. Marko ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] [PATCH v10] make maps in parallel
Hi Marko, Has anyone written a lint program for *.img files that would validate all the cross-references? Or a program to pretty-print the data structures in sorted format? The sorted pretty-print should be identical across runs. There are various programs around for printing out the stuff in IMG files. I have a hacked version of imgdecode that is somewhat more useful than the original and there is always the display code written by Steve. I am simply using cmp -l to do a byte-for-byte comparison of the files generated by various runs of mkgmap and then using imgdecode and display to locate where the differences reside. Cheers, Mark ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] [PATCH v10] make maps in parallel
Yes, get back to work, you slacker! Many thanks for the feedback. That's a good speedup for a 2 core machine. When you have the time, please also try the quick-distance patch. real4m13.508s user4m39.725s sys 0m16.205s Time duration: 254 secs. real3m25.462s user5m21.408s sys 0m15.697s Time duration: 205 secs. The first is without --max-jobs and the second is with and both are a considerable improvement than before the quick-distance patch Cheers Paul ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] [PATCH v10] make maps in parallel
Hi Paul, Thanks for the figures. real4m13.508s user4m39.725s sys 0m16.205s Time duration: 254 secs. real3m25.462s user5m21.408s sys 0m15.697s Time duration: 205 secs. The first is without --max-jobs and the second is with and both are a considerable improvement than before the quick-distance patch Yes, it's a good time saver and when it's combined with the multicore patch I'm seeing around 350% speedup! Cheers, Mark ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] [PATCH v10] make maps in parallel
Hi Clinton, I have also tested this patch on my Windows machine: the error which I previously reported regarding missing files no longer occurs. Sorry for not responding earlier, but I was away. No problem, glad it has fixed the issue. A superficial examination of the map revealed no noticeable differences or problems compared to maps compiled without the parallel code. I'll also test later on with Mac OS. OK. Thanks! The patch looks good so far. Excellent, we now have several reports that the missing file problem has been fixed (methinks that there is either a bug in the Java Futures stuff or it works somewhat differently than the documentation suggests). Cheers, Mark ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] [PATCH v10] make maps in parallel
Mark Burton wrote: Excellent, we now have several reports that the missing file problem has been fixed several++ Like everybody else: maps seem to be fine, with superficial testing... Regards, Martin ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
