Hi Bernhard,
thanks for the feedback. I found a reason for the long run time, please check
my posts reg. r3861 and also this
one:
http://www.mkgmap.org.uk/pipermail/mkgmap-dev/2017q1/026489.html
Gerd
Von: mkgmap-dev im Auftrag von
Bernhard Hiller
Gesendet: Mittwoch, 22. März 2017 21:07:37
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: Re: [mkgmap-dev] Performance with large files
Hi Gerd,
it was one single tile which took more than 2 hours:
4313.osm.pbf took ~7242 sec
It covers a large area, but not as big as the tile mentioned previously:
4313: 1765376,741376 to 2203648,1310720
# : 37.880859,15.908203 to 47.285156,28.125000
The input file "4313.osm.pbf" measures 12.5 MB - quite a normal
size; the resulting img file is 4.2 MB, that's below average. Strange
that that takes such long.
The --polygon-file option (with a polygon from 2-19°E, 45-55°N) reduced
the time spent by mkgmap to just below 1 hour, no tile took more than
100s. Thanks for that trick.
I am still curious why the creation of a tile covering a large area, but
hardly containing nodes requires so much time. Also with the polygon
mentioned above, a large almost empty tile exists (South of Belgium,
because I did not include France) which took longest (97s).
Bernhard
Am 22.03.2017 um 06:26 schrieb Gerd Petermann:
> Hi Bernhard,
>
> I assume that the merged file doesn't contain a bbox. In that case you may
> see a few nodes from ferry routes in the input file and
> splitter will calculate a bbox containing them. I did not see this problem
> when merging Belgium + Netherlands with osmconvert.
>
> Anyway, you may solve the problem in the future by using appropriate bounds,
> e.g. with osmconvert or with the --polygon-file option
> in spltter.
>
> Gerd
>
>
> Von: mkgmap-dev im Auftrag von Gerd
> Petermann
> Gesendet: Dienstag, 21. März 2017 19:15:13
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Performance with large files
>
> Hi Bernhard,
>
> okay, that is a good guess. How do you combine the files? Do you download
> single country extracts from geofabrik
> and merge them with osmconvert? Or maybe another tool?
>
> The splitter log file may show some hints why the area is so large, and maybe
> you should check splitter option
> no-trim.
>
> Gerd
>
> Von: mkgmap-dev im Auftrag von
> Bernhard Hiller
> Gesendet: Dienstag, 21. März 2017 18:56:51
> An: mkgmap-dev@lists.mkgmap.org.uk
> Betreff: Re: [mkgmap-dev] Performance with large files
>
> Hi Gerd,
>
> mkgmap is running now, I'll likely report tomorrow.
>
> But I already saw that the first tile took very long, and in the areas
> file it is:
> 4311: 1765376,-935936 to 2822144,139264
> # : 37.880859,-20.083008 to 60.556641,2.988281
>
> So, I suspect that here could be a problem: some of the newly added
> extracts inflates the area covered by the map extremely. I first thought
> of the Dutch Antilles in the Caribean, but that's farther west and
> farther south. I cannot see how that area comes into the map.
>
> The maps I created before, typically contained Germany, Chechia,
> Austria, Liechtenstein, and Switzerland. That took normally less than an
> hour per map.
>
> Now I added Luxemburg, Belgium and Netherlands also, which does not
> increase file size a lot. But the time for map creation was increased
> enormously.
>
> Bernhard
>
> Am 21.03.2017 um 09:17 schrieb Gerd Petermann:
>> Hi Bernhard,
>>
>> other tests did not show new results. Any idea why you got so different
>> numbers?
>>
>> Gerd
>>
>> Von: mkgmap-dev im Auftrag von Gerd
>> Petermann
>> Gesendet: Montag, 20. März 2017 07:40:00
>> An: Development list for mkgmap
>> Betreff: Re: [mkgmap-dev] Performance with large files
>>
>> Hi Bernhard,
>>
>> reg. splitter:
>> If I got that right you used pbf as input format for the "Germany" result
>> and *.o5m for "Central Europe".
>> Since the o5m format allows faster processing the splitter times are okay
>> for me.
>>
>> reg. mkgmap:
>> I've added a few lines in mkgmap to report the calculation time for each
>> tile. As you might know,
>> mkgmap starts a few threads (depending on the max-jobs option) and each
>> thread processes on tile at a time.
>> The threads share only a few data structures, and mkgmap doesn't collect
>> much information for each tile in
>> memory, so there is no good reason for an increase of run time per
>> processed tile.
>>
>> My system is probably close to yours, a machine with 8 GB Memory and a 4
>> core CPU (i5) running a 64 Windows.
>>
>> The attached patch adds a few lines to report the calculation time for a
>> tile. The patched version is here:
>> http://files.mkgmap.org.uk/download/339/mkgmap.jar
>>
>>I've used the patched version to compile a map with the OpenfietsMap Lite
>> style of an area around (and including) Germany
