Re: [mkgmap-dev] [PATCH v5]splitter memory usage

2011-11-17 Thread Felix Hartmann
On 17.11.2011 22:38, GerdP wrote: > Hello Felix, > > okay, thanks again for these numbers. Just one remark regarding speed: > My profiling programs says that ~ 3% of the time is now spent in the > routines that store the data (Node2AreaMap and below). So, I can't get this > much better. Most of t

Re: [mkgmap-dev] [PATCH v5]splitter memory usage

2011-11-17 Thread Michael Prinzing
On Wed, 16 Nov 2011 22:37:31 +0100, WanMil wrote: >I will merge the branch back to trunk after getting some feedback on the >list that the changes are worthy to merge. The patched splitter is the first version since r161 that is usable on my system (Win XP 32 bit, Java 1.7 , 1.7GB RAM). It is ru

Re: [mkgmap-dev] [PATCH v5]splitter memory usage

2011-11-17 Thread Michael Prinzing
On Wed, 16 Nov 2011 22:37:31 +0100, WanMil wrote: >I will merge the branch back to trunk after getting some feedback on the >list that the changes are worthy to merge. The patched splitter is the first version since r161 that is usable on my system (Win XP 32 bit, Java 1.7 , 1.7GB RAM). It is ru

Re: [mkgmap-dev] [PATCH v5]splitter memory usage

2011-11-17 Thread GerdP
Hello Felix, okay, thanks again for these numbers. Just one remark regarding speed: My profiling programs says that ~ 3% of the time is now spent in the routines that store the data (Node2AreaMap and below). So, I can't get this much better. Most of the time is spent reading and writing pbf and c

Re: [mkgmap-dev] [PATCH v5]splitter memory usage

2011-11-17 Thread Felix Hartmann
Sorry, all of my my previous measurements were actually done with splitter r188. So also the fastutil.jar missing problem was of course caused by it. I have updated all tests with r189. Note that optimize-mem at least on my system (Core7 with 4 Cores available, 8GB Ram, Win 7 x64 server) is onl

Re: [mkgmap-dev] [PATCH v5]splitter memory usage

2011-11-17 Thread Felix Hartmann
Well I think you didn't read my last message yet I think currently about 6GB of free RAM are needed to have max-area=1024 to be working (assuming max-nodes 900 000). What is the actual maximum? Could one set max-area=1300? (that is about what I think is max on my machine with max-nodes 900

Re: [mkgmap-dev] [PATCH v5]splitter memory usage

2011-11-17 Thread Felix Hartmann
Europe max-nodes 900.000 (resulting in 975 tiles, extract is about 4 weeks old already - hence 4 passes down to 1 pass ) old splitter: 52min 38 seconds (3158 seconds) splitter_patched: 29min 45 seconds (1785 seconds) splitter patched with optimize-mem: 30min 41seconds (1841 seconds) So I would

Re: [mkgmap-dev] [PATCH v5]splitter memory usage

2011-11-17 Thread Felix Hartmann
some more speed observations: country germany.pbf from Geofabrik splitter_patched with optimize-mem: 3 min 6 seconds splitter_patched without optimize-mem: 3min 2 seconds old splitter: 3 min 42 seconds I am currently running against europe.pbf, will post the results later. But it seems that the

Re: [mkgmap-dev] [PATCH v5]splitter memory usage: Exception in thread "main" java.lang.NoClassDefFoundError: it/unimi/dsi/fastutil/longs/Long2ShortFunction

2011-11-17 Thread Felix Hartmann
Seems to be still some fastutil.jar problem around? I tried to use the splitter.jar from the mkgmap download page WITHOUT the modified fastutil.jar (latest as of today): Exception in thread "main" java.lang.NoClassDefFoundError: it/unimi/dsi/fastutil/longs/Long2ShortFunction It does work howeve

Re: [mkgmap-dev] [PATCH v5]splitter memory usage

2011-11-16 Thread WanMil
Thanks Gerd, I have commited it to the branch. I will merge the branch back to trunk after getting some feedback on the list that the changes are worthy to merge. Unfortunately I do not have time to check that myself at the moment. Have fun! WanMil > Hello, > > here is the final patch > http:

[mkgmap-dev] [PATCH v5]splitter memory usage

2011-11-16 Thread GerdP
Hello, here is the final patch http://gis.638310.n2.nabble.com/file/n7001044/memory_patch_v5.patch memory_patch_v5.patch It has to be applied against the memory branch. Final notes: - the modified fastutil.jar is no longer needed, the trunk version version works fine - the program works almost