subject:"Re\: \[mkgmap\-dev\] splitter"

Re: [mkgmap-dev] Splitter Java Heap

2024-03-21 Thread Gerd Petermann

Hi Felix,

if you feed the same input (same file(s) , same options) into identical
splitter and run it on identical JRE with identical JRE options but on two
different machines you still have to compare the number of parallel threads
(max-threads).

Gerd

Von: mkgmap-dev im Auftrag von Felix
Herwegh
Gesendet: Mittwoch, 20. März 2024 20:39
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: Re: [mkgmap-dev] Splitter Java Heap

Hi Gerd,

Thanks!

>From the java --version output (in the first mail) I assumed already beeing on
>a 64-bit JRE and, following your advice, set -Xmx4G. Bingo, succes in the
>first pass on the weak machine.

JVM Memory Info: Current 3990MB (1967MB used, 2023MB free) Max 4096MB

I still wonder how to read the figures though, as for the former successfull
passes only about 1300 MB where reported used. and every pass reportedly still
had free memory, although "current" was = "max." on the fails.

So, how to figure out an appropriate allocation? By map and/or machine?

Altough reading up on splitters documentation I seem to have been derailled by
the description for --max-areas parameter

Higher numbers ... require more memory

Note that the first stage of the processing has a fixed memory overhead
regardless of what this is set to so if you are running out of memory before
the areas.list file is generated, you need to either increase your -Xmx value...

Therefore I fiddled with max-areas, but didn't try to use -Xmx myself.

But, I now see, why --max-areas didn't change a thing in my case, as -for the
map in question- I get:

Processing 293 areas in a single pass

// Felix

On 20.03.24 15:16, Gerd Petermann wrote:

Hi Felix,

I guess your laptop is running a 32-bit JRE, so your first step should be to
install a 64 bit version if you have a 64 bit OS.
With this done you can increase the max heap to e.g. 4GB with something like
java -Xmx4G -jar splitter.jar
If you cannot increase the memory you can switch of the --keep-complete option
(with the corresponding disadvantages)

Gerd

Von: mkgmap-dev
<mailto:mkgmap-dev-boun...@lists.mkgmap.org.uk>
im Auftrag von Felix Herwegh <mailto:mlmm...@herwegh.de>
Gesendet: Mittwoch, 20. März 2024 15:03
An: mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk>
Betreff: [mkgmap-dev] Splitter Java Heap

Hi,

switching to my Ultrabook (6 GB) while travelling I recently faced some kind of
borderline condition with splitter. On the first run it throws
"OutOfMemoryError: Java heap space", on closely subsequent runs without any
modifications it does not. Repeating the task after some delay fails again. I
guess, there might be some self-optimization involved for this.

fail:
...
40.000.000 ways parsed... id=888262666
Number of stored tile combinations in multiTileDictionary: 4.525
41.000.000 ways parsed... id=929920953
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at
uk.me.parabola.splitter.tools.SparseLong2IntMap$ChunkMem.(SparseLong2IntMap.java:189)
at
uk.me.parabola.splitter.tools.SparseLong2IntMap.saveCurrentChunk(SparseLong2IntMap.java:627)
at
uk.me.parabola.splitter.tools.SparseLong2IntMap.replaceCurrentChunk(SparseLong2IntMap.java:886)
at
uk.me.parabola.splitter.tools.SparseLong2IntMap.put(SparseLong2IntMap.java:691)
at
uk.me.parabola.splitter.SplitProcessor.processWay(SplitProcessor.java:149)
at
uk.me.parabola.splitter.AbstractMapProcessor.consume(AbstractMapProcessor.java:84)
at uk.me.parabola.splitter.OSMFileHandler.execute(OSMFileHandler.java:157)
at uk.me.parabola.splitter.Main.writeTiles(Main.java:542)
at uk.me.parabola.splitter.Main.start(Main.java:132)
at uk.me.parabola.splitter.Main.main(Main.java:81)
Elapsed time: 8m 0s Memory: Current 1466MB (1339MB used, 127MB free) Max
1466MB

success:
...
48.000.000 ways parsed... id=1262369277
Writing relations Tue Mar 19 10:50:36 CET 2024
100.000 relations parsed... id=1783690
200.000 relations parsed... id=4148045
300.000 relations parsed... id=7895430
400.000 relations parsed... id=11681672
500.000 relations parsed... id=15581604
coord Map: 312.851.126 stored long/int pairs require ca. 3 bytes per pair.
14.225.657 chunks are used, the avg. number of values in one 64-chunk is 21.
coord Map details: ~852 MB, including 88 array(s) with 8 MB

way Map: 48.015.926 stored long/int pairs require ca. 3 bytes per pair.
3.974.651 chunks are used, the avg. number of values in one 64-chunk is 12.
way Map details: ~123 MB, including 10 array(s) with 8 MB

JVM Memory Info: Current 1466MB (1357MB used, 109MB free) Max 1466MB
Full Node tests: 62.230.523
Quick Node tests: 282.354.912
Thread worker-2 has finished
...

My main machine has 24 GB of main memory, and runs troublefree using the
following memory allocation on the same task:

JVM Memory Info: Curre

Re: [mkgmap-dev] Splitter Java Heap

2024-03-20 Thread Felix Herwegh


Hi Gerd,

Thanks!

From the java --version output (in the first mail) I assumed already 
beeing on a 64-bit JRE and, following your advice, set -Xmx4G. Bingo, 
succes in the first pass on the weak machine.


   /JVM Memory Info: Current 3990MB (1967MB used, 2023MB free) Max 4096MB/

I still wonder how to read the figures though, as for the former 
successfull passes only about 1300 MB where reported used. and every 
pass reportedly still had free memory, although "current" was = "max." 
on the fails.


So, how to figure out an appropriate allocation? By map and/or machine?

Altough reading up on splitters documentation I seem to have been 
derailled by the description for  --max-areas parameter


   /Higher numbers ... require more memory/

   /Note that the first stage of the processing has a fixed memory
   overhead regardless of what this is set to so if you are running out
   of memory before the areas.list file is generated, you need to
   either increase your -Xmx value.../

Therefore I fiddled with max-areas, but didn't try to use -Xmx myself.

But, I now see, why --max-areas didn't change a thing in my case, as 
-for the map in question- I get:


   /Processing 293 areas in a single pass
   /

// Felix


On 20.03.24 15:16, Gerd Petermann wrote:

Hi Felix,

I guess your laptop is running a 32-bit JRE, so your first step should be to 
install a 64 bit version if you have a 64 bit OS.
With this done you can increase the max heap to e.g. 4GB with something like
java -Xmx4G -jar splitter.jar 
If you cannot increase the memory you can switch of the --keep-complete option 
(with the corresponding disadvantages)

Gerd



Von: mkgmap-dev  im Auftrag von Felix 
Herwegh
Gesendet: Mittwoch, 20. März 2024 15:03
An:mkgmap-dev@lists.mkgmap.org.uk
Betreff: [mkgmap-dev] Splitter Java Heap

Hi,

switching to my Ultrabook (6 GB) while travelling I recently faced some kind of 
borderline condition with splitter. On the first run it throws "OutOfMemoryError: 
Java heap space", on closely subsequent runs without any modifications it does not. 
Repeating the task after some delay fails again. I guess, there might be some 
self-optimization involved for this.

fail:
...
40.000.000 ways parsed... id=888262666
   Number of stored tile combinations in multiTileDictionary: 4.525
41.000.000 ways parsed... id=929920953
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
 at 
uk.me.parabola.splitter.tools.SparseLong2IntMap$ChunkMem.(SparseLong2IntMap.java:189)
 at 
uk.me.parabola.splitter.tools.SparseLong2IntMap.saveCurrentChunk(SparseLong2IntMap.java:627)
 at 
uk.me.parabola.splitter.tools.SparseLong2IntMap.replaceCurrentChunk(SparseLong2IntMap.java:886)
 at 
uk.me.parabola.splitter.tools.SparseLong2IntMap.put(SparseLong2IntMap.java:691)
 at 
uk.me.parabola.splitter.SplitProcessor.processWay(SplitProcessor.java:149)
 at 
uk.me.parabola.splitter.AbstractMapProcessor.consume(AbstractMapProcessor.java:84)
 at uk.me.parabola.splitter.OSMFileHandler.execute(OSMFileHandler.java:157)
 at uk.me.parabola.splitter.Main.writeTiles(Main.java:542)
 at uk.me.parabola.splitter.Main.start(Main.java:132)
 at uk.me.parabola.splitter.Main.main(Main.java:81)
Elapsed time: 8m 0s   Memory: Current 1466MB (1339MB used, 127MB free) Max 
1466MB

success:
...
48.000.000 ways parsed... id=1262369277
Writing relations Tue Mar 19 10:50:36 CET 2024
100.000 relations parsed... id=1783690
200.000 relations parsed... id=4148045
300.000 relations parsed... id=7895430
400.000 relations parsed... id=11681672
500.000 relations parsed... id=15581604
coord Map: 312.851.126 stored long/int pairs require ca. 3 bytes per pair. 
14.225.657 chunks are used, the avg. number of values in one 64-chunk is 21.
coord Map details: ~852 MB, including 88 array(s) with 8 MB

way Map: 48.015.926 stored long/int pairs require ca. 3 bytes per pair. 
3.974.651 chunks are used, the avg. number of values in one 64-chunk is 12.
way Map details: ~123 MB, including 10 array(s) with 8 MB

   JVM Memory Info: Current 1466MB (1357MB used, 109MB free) Max 1466MB
Full Node tests:  62.230.523
Quick Node tests: 282.354.912
Thread worker-2 has finished
...

My main machine has 24 GB of main memory, and runs troublefree using the 
following memory allocation on the same task:

JVM Memory Info: Current 3342MB (2378MB used, 964MB free) Max 6000MB

Splitter 653 so far is involved without explicit memory allocation (java -jar 
.../splitter-latest/splitter.jar ...), using

java --version
openjdk 11.0.22 2024-01-16
OpenJDK Runtime Environment (build 11.0.22+7-post-Debian-1deb10u1)
OpenJDK 64-Bit Server VM (build 11.0.22+7-post-Debian-1deb10u1, mixed mode, 
sharing)

Following up on splitter tuning hints (areas.list gets generated in each case) 
I reduced --max-areas= from 4096 to 2048 to 1024, but to no avail (not even 
significantly on the runtimes), once I figured out the effect above. It fails 
on

Re: [mkgmap-dev] Splitter Java Heap

2024-03-20 Thread Gerd Petermann

Hi Felix,

I guess your laptop is running a 32-bit JRE, so your first step should be to 
install a 64 bit version if you have a 64 bit OS.
With this done you can increase the max heap to e.g. 4GB with something like
java -Xmx4G -jar splitter.jar 
If you cannot increase the memory you can switch of the --keep-complete option 
(with the corresponding disadvantages)

Gerd



Von: mkgmap-dev  im Auftrag von Felix 
Herwegh 
Gesendet: Mittwoch, 20. März 2024 15:03
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: [mkgmap-dev] Splitter Java Heap

Hi,

switching to my Ultrabook (6 GB) while travelling I recently faced some kind of 
borderline condition with splitter. On the first run it throws 
"OutOfMemoryError: Java heap space", on closely subsequent runs without any 
modifications it does not. Repeating the task after some delay fails again. I 
guess, there might be some self-optimization involved for this.

fail:
...
40.000.000 ways parsed... id=888262666
  Number of stored tile combinations in multiTileDictionary: 4.525
41.000.000 ways parsed... id=929920953
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at 
uk.me.parabola.splitter.tools.SparseLong2IntMap$ChunkMem.(SparseLong2IntMap.java:189)
at 
uk.me.parabola.splitter.tools.SparseLong2IntMap.saveCurrentChunk(SparseLong2IntMap.java:627)
at 
uk.me.parabola.splitter.tools.SparseLong2IntMap.replaceCurrentChunk(SparseLong2IntMap.java:886)
at 
uk.me.parabola.splitter.tools.SparseLong2IntMap.put(SparseLong2IntMap.java:691)
at 
uk.me.parabola.splitter.SplitProcessor.processWay(SplitProcessor.java:149)
at 
uk.me.parabola.splitter.AbstractMapProcessor.consume(AbstractMapProcessor.java:84)
at uk.me.parabola.splitter.OSMFileHandler.execute(OSMFileHandler.java:157)
at uk.me.parabola.splitter.Main.writeTiles(Main.java:542)
at uk.me.parabola.splitter.Main.start(Main.java:132)
at uk.me.parabola.splitter.Main.main(Main.java:81)
Elapsed time: 8m 0s   Memory: Current 1466MB (1339MB used, 127MB free) Max 
1466MB

success:
...
48.000.000 ways parsed... id=1262369277
Writing relations Tue Mar 19 10:50:36 CET 2024
100.000 relations parsed... id=1783690
200.000 relations parsed... id=4148045
300.000 relations parsed... id=7895430
400.000 relations parsed... id=11681672
500.000 relations parsed... id=15581604
coord Map: 312.851.126 stored long/int pairs require ca. 3 bytes per pair. 
14.225.657 chunks are used, the avg. number of values in one 64-chunk is 21.
coord Map details: ~852 MB, including 88 array(s) with 8 MB

way Map: 48.015.926 stored long/int pairs require ca. 3 bytes per pair. 
3.974.651 chunks are used, the avg. number of values in one 64-chunk is 12.
way Map details: ~123 MB, including 10 array(s) with 8 MB

  JVM Memory Info: Current 1466MB (1357MB used, 109MB free) Max 1466MB
Full Node tests:  62.230.523
Quick Node tests: 282.354.912
Thread worker-2 has finished
...

My main machine has 24 GB of main memory, and runs troublefree using the 
following memory allocation on the same task:

JVM Memory Info: Current 3342MB (2378MB used, 964MB free) Max 6000MB

Splitter 653 so far is involved without explicit memory allocation (java -jar 
.../splitter-latest/splitter.jar ...), using

java --version
openjdk 11.0.22 2024-01-16
OpenJDK Runtime Environment (build 11.0.22+7-post-Debian-1deb10u1)
OpenJDK 64-Bit Server VM (build 11.0.22+7-post-Debian-1deb10u1, mixed mode, 
sharing)

Following up on splitter tuning hints (areas.list gets generated in each case) 
I reduced --max-areas= from 4096 to 2048 to 1024, but to no avail (not even 
significantly on the runtimes), once I figured out the effect above. It fails 
on all first runs and succeeds on all shortly following next ones.

Unfortunately its not possible to increase main hardware memory on the small 
machine, but system tools report only about 2...3 GB being used anyway.
Is it possible to tweak Java to overcome the problem without hurting the maps, 
preferably by machine, to be able to run identical scripts? Some pointers would 
be appreciated, also on how to monitor the Java memory situation.

Thanks, Felix


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Splitter feature request

2023-12-06 Thread Carlos Dávila


Hi Gerd,

As this new feature would be for areas.list passed to splitter, I 
thought it could assign objects to tiles (inlcuding L-shaped ones) in 
one step, assuming it can do so once it knows the area of each tile, but 
I'm probably wrong, as I don't really know how splitter works internally.


Right/left hand drive side driving is quite difficult to solve in land 
borders, but L-shaped tiles can help avoiding the need of tiny tiles in 
those areas. For areas with a lot of islands, where some drive on the 
right and others on the left, L-shaped tiles would really help.


El 6/12/23 a las 12:23, Gerd Petermann escribió:

Hi Carlos,

since nobody else reacted:
I wonder in what situation an L-shaped tile could be the result of a 
split-process.
Do you think about some kind of post-processor which would recombine 2 or more 
tiles so that they still don't
reach the maxnodes boundary? Thinking about the way how a split is done, I 
think it can happen that the combination of two connected tiles are still below 
the limit,
but I would assume that an L-shape is rather rare.
Anyway, if splitter would write non-rectangular tiles we need
a) a further output file (or a new output format) to store the tile-boundary 
for input in mkgmap
b) some new logic in mkgmap to cut ways and polygons along that boundary

It might be easier to implement b) when boundary lines are only horizontal or 
vertical, but I am pretty sure this wouldn't solve most of the problems
reg.  right/left hand side driving.

ciao
Gerd



Von: mkgmap-dev  im Auftrag von Carlos Dávila 

Gesendet: Sonntag, 3. Dezember 2023 12:26
An: Development list for mkgmap
Betreff: [mkgmap-dev] Splitter feature request

Hi all

I usually let splitter decide tiles based on max-nodes value, but for
some maps I need to supply splitter an areas.list file which I adjust
manually. Reasons for that include reducing the total number of tiles,
avoiding cutting of some islands and separating right/left hand side
driving areas or reducing number of tiles with both sides driving. For
these cases, it would be helpful to be able to build "L shaped tiles",
as in the attached image. I think these tiles could be defined by three
coordinates (A,B,C), which would define the two rectangles in which the
L shaped tile can be split, and a switch (1,2,3,4) to determine de
orientation of the "L".

1: tile = A-lat, A-lon to B-lat, B-lon + A-lat,B-lon to C-lat,C-lon

2: tile = B-lat, A-lon to C-lat, C-lon + A-lat,B-lon to B-lat,C-lon

3: tile = A-lat, A-lon to B-lat, B-lon + B-lat,A-lon to C-lat,C-lon

4: tile = A-lat, A-lon to C-lat, B-lon + A-lat,B-lon to B-lat,C-lon

This raises several questions. First of all, if is feasible, then if it
is worth the effort to implement it, how would if affect splitter
performance and if it makes sense for other users. Perhaps it could be a
first step towards irregular tiles...
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Splitter feature request

2023-12-06 Thread Gerd Petermann

Hi Carlos,

since nobody else reacted:
I wonder in what situation an L-shaped tile could be the result of a 
split-process.
Do you think about some kind of post-processor which would recombine 2 or more 
tiles so that they still don't
reach the maxnodes boundary? Thinking about the way how a split is done, I 
think it can happen that the combination of two connected tiles are still below 
the limit,
but I would assume that an L-shape is rather rare.
Anyway, if splitter would write non-rectangular tiles we need
a) a further output file (or a new output format) to store the tile-boundary 
for input in mkgmap
b) some new logic in mkgmap to cut ways and polygons along that boundary

It might be easier to implement b) when boundary lines are only horizontal or 
vertical, but I am pretty sure this wouldn't solve most of the problems
reg.  right/left hand side driving.

ciao
Gerd



Von: mkgmap-dev  im Auftrag von Carlos 
Dávila 
Gesendet: Sonntag, 3. Dezember 2023 12:26
An: Development list for mkgmap
Betreff: [mkgmap-dev] Splitter feature request

Hi all

I usually let splitter decide tiles based on max-nodes value, but for
some maps I need to supply splitter an areas.list file which I adjust
manually. Reasons for that include reducing the total number of tiles,
avoiding cutting of some islands and separating right/left hand side
driving areas or reducing number of tiles with both sides driving. For
these cases, it would be helpful to be able to build "L shaped tiles",
as in the attached image. I think these tiles could be defined by three
coordinates (A,B,C), which would define the two rectangles in which the
L shaped tile can be split, and a switch (1,2,3,4) to determine de
orientation of the "L".

1: tile = A-lat, A-lon to B-lat, B-lon + A-lat,B-lon to C-lat,C-lon

2: tile = B-lat, A-lon to C-lat, C-lon + A-lat,B-lon to B-lat,C-lon

3: tile = A-lat, A-lon to B-lat, B-lon + B-lat,A-lon to C-lat,C-lon

4: tile = A-lat, A-lon to C-lat, B-lon + A-lat,B-lon to B-lat,C-lon

This raises several questions. First of all, if is feasible, then if it
is worth the effort to implement it, how would if affect splitter
performance and if it makes sense for other users. Perhaps it could be a
first step towards irregular tiles...
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter with --polygon-desc-file crashes

2022-02-16 Thread 7770

Hi.
Just after splitter reports that it stats with the second country (estonia in 
lower case is the country name within the polygon file).
--resolution=15 combined with --max-nodes=75000 did not change...

Some of the output:

Rounded map coverage is (55.634765625,19.7314453125) to 
(60.029296875,28.2568359375)
Splitting nodes into areas containing a maximum of 75 000 nodes each...
splitting distinct part of latvia
Highest node count in a single grid element is 83 999
Highest node count in a single grid element within the bounding polygon is 
77 110
...
continues.

Then splitter reports "splitting distinct part of estonia" thereafter:

java.lang.NullPointerException
at 
uk.me.parabola.splitter.solver.DensityMap.getNodeCount(DensityMap.java:156)
at 
uk.me.parabola.splitter.solver.EnhancedDensityMap.prepare(EnhancedDensityMap.java:
83)
at 
uk.me.parabola.splitter.solver.EnhancedDensityMap.(EnhancedDensityMap.java:
43)
at 
uk.me.parabola.splitter.solver.SplittableDensityArea.prepare(SplittableDensityArea.java:
339)
at 
uk.me.parabola.splitter.solver.SplittableDensityArea.split(SplittableDensityArea.java:
177)
at 
uk.me.parabola.splitter.solver.SplittableDensityArea.split(SplittableDensityArea.java:
237)
at 
uk.me.parabola.splitter.solver.AreasCalculator.calcAreas(AreasCalculator.java:
228)
at uk.me.parabola.splitter.Main.split(Main.java:227)
at uk.me.parabola.splitter.Main.start(Main.java:121)
at uk.me.parabola.splitter.Main.main(Main.java:81)


I can add the full output if it may be of any use.

Regards
Karl



On onsdag 16 februari 2022 18:35:51 CET Gerd Petermann wrote:
> Hi Karl,
> 
> I've not tried it yet. What is the error message from splitter?
> When experimenting with small files it sometimes makes sense to increase
> the resolution (14 or 15) and lower the --max-nodes values to get closer to
> a large scale result (like > 100 tiles or so)
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag von 7770
> <7...@foskan.eu> Gesendet: Mittwoch, 16. Februar 2022 17:52
> An: mkgmap-dev@lists.mkgmap.org.uk
> Betreff: [mkgmap-dev] splitter with --polygon-desc-file crashes
> 
> Hi.
> I have asked long ago about this particular problem i have and it was long
> time ago i last tried, but now i did again. It relates to splitter and usage
> of --polygon-desc-file=
> 
> This is how i have tried:
> 1.  Download data for Estonia and Latvia.
> https://download.geofabrik.de/europe/estonia-latest.osm.pbf
> https://download.geofabrik.de/europe/latvia-latest.osm.pbf
> 
> 2. convert to o5m and combine
> osmconvert estonia-latest.osm.pbf -o=estonia-latest.osm.o5m;
> osmconvert latvia-latest.osm.pbf -o=latvia-latest.osm.o5m;
> osmconvert estonia-latest.osm.o5m latvia-latest.osm.o5m -o=ee_lv.o5m;
> 
> 3. Prepare an osm-polygon file with one area for Estonia and one for Latvia.
> (file attached)
> 
> 4. Run splitter:
> java -Xmx2000m -jar ${SPLITTER} \
> --polygon-desc-file=./polygon_ee_lv.osm \
> --output-dir=ee_lv/splitted/ \
> --max-nodes=75 \
> --no-trim \
> ee_lv.o5m
> 
> 
> While running this is seems like the Latvia part is fine, but the Estonia
> crashes.
> 
> 
> The error message doesn't say much (that i understand).
> But maybe it makes some sense to you?
> I imagine something is wrong with the polygon file, but i dont know what.
> 
> The polygon files is partially made in JOSM and partially manaully to re-use
> the same segments/edges which belong to both countries. (I could see that
> Gerd was using this concept in some example file for Germany and the alps,
> but i could not add two ways over one edge in JOSM, so i made manual
> chanegs in the file).
> I tried also making two separate boxes, one for each country, but that was
> just as bad (not reusing edges).
> 
> Anything that could be suggested that i change or try?
> 
> PS. i am trying this on small scale for two smaller countries, later i will
> apply it because i need to split europe in parts of max 4GB.
> 
> Regards
> Karl
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev



___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter with --polygon-desc-file crashes

2022-02-16 Thread Gerd Petermann

Hi Karl,

I've not tried it yet. What is the error message from splitter?
When experimenting with small files it sometimes makes sense to increase
the resolution (14 or 15) and lower the --max-nodes values to get closer to a 
large scale
result (like > 100 tiles or so)

Gerd


Von: mkgmap-dev  im Auftrag von 7770 
<7...@foskan.eu>
Gesendet: Mittwoch, 16. Februar 2022 17:52
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: [mkgmap-dev] splitter with --polygon-desc-file crashes

Hi.
I have asked long ago about this particular problem i have and it was long
time ago i last tried, but now i did again. It relates to splitter and usage
of --polygon-desc-file=

This is how i have tried:
1.  Download data for Estonia and Latvia.
https://download.geofabrik.de/europe/estonia-latest.osm.pbf
https://download.geofabrik.de/europe/latvia-latest.osm.pbf

2. convert to o5m and combine
osmconvert estonia-latest.osm.pbf -o=estonia-latest.osm.o5m;
osmconvert latvia-latest.osm.pbf -o=latvia-latest.osm.o5m;
osmconvert estonia-latest.osm.o5m latvia-latest.osm.o5m -o=ee_lv.o5m;

3. Prepare an osm-polygon file with one area for Estonia and one for Latvia.
(file attached)

4. Run splitter:
java -Xmx2000m -jar ${SPLITTER} \
--polygon-desc-file=./polygon_ee_lv.osm \
--output-dir=ee_lv/splitted/ \
--max-nodes=75 \
--no-trim \
ee_lv.o5m


While running this is seems like the Latvia part is fine, but the Estonia
crashes.


The error message doesn't say much (that i understand).
But maybe it makes some sense to you?
I imagine something is wrong with the polygon file, but i dont know what.

The polygon files is partially made in JOSM and partially manaully to re-use
the same segments/edges which belong to both countries. (I could see that Gerd
was using this concept in some example file for Germany and the alps, but i
could not add two ways over one edge in JOSM, so i made manual chanegs in the
file).
I tried also making two separate boxes, one for each country, but that was
just as bad (not reusing edges).

Anything that could be suggested that i change or try?

PS. i am trying this on small scale for two smaller countries, later i will
apply it because i need to split europe in parts of max 4GB.

Regards
Karl
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter r609 released

2021-07-01 Thread Gerd Petermann

Hi Felix,

the poly file for South America contains few points (< 40). I think splitter 
switched into the mode which tries hard to fit tiles into the polygon.
If you post a link to the corresponding densities-out.txt and log I can tell 
you more details.

Gerd


Von: mkgmap-dev  im Auftrag von Felix 
Hartmann 
Gesendet: Donnerstag, 1. Juli 2021 09:58
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] splitter r609 released

Well here are the results for 615:
If you would like the log files for any of those runs - tell me. Significantly 
better with polygon file are Norway, Russia, Australia-Oceania and Asia. This 
version has much less problem countries like 614. It would be too big to attach 
all of them here. If you like all of them then I can upload them to my server.

South America is really bad using polygon-file.

"for netherlands use polygon-file - cnt1 = 89 cnt0 = 88"
"for great-britain use polygon-file - cnt1 = 111 cnt0 = 110"
"for germany use polygon-file - cnt1 = 258 cnt0 = 257"
"for liechtenstein do not use polygon-file - cnt1 = 1 cnt0 = 6"
"for monaco do not use polygon-file - cnt1 = 1 cnt0 = 2"
"for slovenia do not use polygon-file - cnt1 = 27 cnt0 = 28"
"for ukraine use polygon-file - cnt1 = 62 cnt0 = 61"
"for norway use polygon-file - cnt1 = 129 cnt0 = 119"
"for switzerland do not use polygon-file - cnt1 = 29 cnt0 = 30"
"for poland use polygon-file - cnt1 = 132 cnt0 = 128"
"for sweden use polygon-file - cnt1 = 60 cnt0 = 54"
"for finland do not use polygon-file - cnt1 = 65 cnt0 = 84"
"for denmark use polygon-file - cnt1 = 33 cnt0 = 32"
"for andorra do not use polygon-file - cnt1 = 1 cnt0 = 4"
"for estonia use polygon-file - cnt1 = 9 cnt0 = 8"
"for saarland do not use polygon-file - cnt1 = 4 cnt0 = 17"
"for hamburg do not use polygon-file - cnt1 = 3 cnt0 = 12"
"for hessen do not use polygon-file - cnt1 = 17 cnt0 = 18"
"for bayern do not use polygon-file - cnt1 = 47 cnt0 = 48"
"for berlin do not use polygon-file - cnt1 = 5 cnt0 = 13"
"for australia-oceania use polygon-file - cnt1 = 131 cnt0 = 109"
"for south-america do not use polygon-file - cnt1 = 334 cnt0 = 463"
"for africa use polygon-file - cnt1 = 665 cnt0 = 663"
"for asia use polygon-file - cnt1 = 1592 cnt0 = 1549"
"for russia use polygon-file - cnt1 = 426 cnt0 = 408"
"for central-america use polygon-file - cnt1 = 61 cnt0 = 58"
"for morocco do not use polygon-file - cnt1 = 19 cnt0 = 27"
"for tanzania use polygon-file - cnt1 = 75 cnt0 = 74"
"for mozambique use polygon-file - cnt1 = 24 cnt0 = 23"
"for azerbaijan do not use polygon-file - cnt1 = 3 cnt0 = 4"
"for iran use polygon-file - cnt1 = 16 cnt0 = 15"
"for malaysia-singapore-brunei do not use polygon-file - cnt1 = 16 cnt0 = 17"
"for china use polygon-file - cnt1 = 106 cnt0 = 101"
"for india do not use polygon-file - cnt1 = 109 cnt0 = 112"
"for indonesia do not use polygon-file - cnt1 = 191 cnt0 = 200"
"for philippines use polygon-file - cnt1 = 52 cnt0 = 51"
"for afghanistan do not use polygon-file - cnt1 = 11 cnt0 = 20"
"for australia use polygon-file - cnt1 = 67 cnt0 = 66"
"for argentina use polygon-file - cnt1 = 32 cnt0 = 27"
"for brazil use polygon-file - cnt1 = 177 cnt0 = 172"
"for chile do not use polygon-file - cnt1 = 31 cnt0 = 33"
"for paraguay use polygon-file - cnt1 = 17 cnt0 = 16"
"for canada use polygon-file - cnt1 = 274 cnt0 = 272"
"for us-northeast do not use polygon-file - cnt1 = 109 cnt0 = 110"
"for us-pacific use polygon-file - cnt1 = 21 cnt0 = 18"
"for us-south use polygon-file - cnt1 = 262 cnt0 = 261"
"for us-west use polygon-file - cnt1 = 229 cnt0 = 227"
"for greenland use polygon-file - cnt1 = 4 cnt0 = 2"
"for mexico use polygon-file - cnt1 = 44 cnt0 = 42"
"for reunion do not use polygon-file - cnt1 = 3 cnt0 = 5"

On Wed, 30 Jun 2021 at 12:30, Gerd Petermann 
mailto:gpetermann_muenc...@hotmail.com>> wrote:
Hi Felix,

I've released r615 to fix the problems with Saarland (now 17 tiles with polygon 
instead of crash, 4 tiles without) and Norway (121 with polygon, 128 without)
Numbers for max-nodes=140

Gerd



Von: mkgmap-dev 
mailto:mkgmap-dev-boun...@lists.mkgmap.org.uk>>
 im Auftrag von Felix Hartmann 
mailto:extremecar...@gmail.com>>
Gesendet: Mittwoch, 30. Juni 2021 10:52
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] splitter r609 released

well yeah by now my list is down to very few countries (different then with 
older splitter.jar) that profit fr

Re: [mkgmap-dev] splitter r609 released

2021-07-01 Thread Felix Hartmann

Well here are the results for 615:
If you would like the log files for any of those runs - tell me.
Significantly better with polygon file are Norway, Russia,
Australia-Oceania and Asia. This version has much less problem countries
like 614. It would be too big to attach all of them here. If you like all
of them then I can upload them to my server.

South America is really bad using polygon-file.

"for netherlands use polygon-file - cnt1 = 89 cnt0 = 88"
"for great-britain use polygon-file - cnt1 = 111 cnt0 = 110"
"for germany use polygon-file - cnt1 = 258 cnt0 = 257"
"for liechtenstein do not use polygon-file - cnt1 = 1 cnt0 = 6"
"for monaco do not use polygon-file - cnt1 = 1 cnt0 = 2"
"for slovenia do not use polygon-file - cnt1 = 27 cnt0 = 28"
"for ukraine use polygon-file - cnt1 = 62 cnt0 = 61"
*"for norway use polygon-file - cnt1 = 129 cnt0 = 119" *
"for switzerland do not use polygon-file - cnt1 = 29 cnt0 = 30"
"for poland use polygon-file - cnt1 = 132 cnt0 = 128"
"for sweden use polygon-file - cnt1 = 60 cnt0 = 54"
"for finland do not use polygon-file - cnt1 = 65 cnt0 = 84"
"for denmark use polygon-file - cnt1 = 33 cnt0 = 32"
"for andorra do not use polygon-file - cnt1 = 1 cnt0 = 4"
"for estonia use polygon-file - cnt1 = 9 cnt0 = 8"
"for saarland do not use polygon-file - cnt1 = 4 cnt0 = 17"
"for hamburg do not use polygon-file - cnt1 = 3 cnt0 = 12"
"for hessen do not use polygon-file - cnt1 = 17 cnt0 = 18"
"for bayern do not use polygon-file - cnt1 = 47 cnt0 = 48"
"for berlin do not use polygon-file - cnt1 = 5 cnt0 = 13"
*"for australia-oceania use polygon-file - cnt1 = 131 cnt0 = 109" *
"for south-america do not use polygon-file - cnt1 = 334 cnt0 = 463"
"for africa use polygon-file - cnt1 = 665 cnt0 = 663"
*"for asia use polygon-file - cnt1 = 1592 cnt0 = 1549" *
*"for russia use polygon-file - cnt1 = 426 cnt0 = 408" *
"for central-america use polygon-file - cnt1 = 61 cnt0 = 58"
"for morocco do not use polygon-file - cnt1 = 19 cnt0 = 27"
"for tanzania use polygon-file - cnt1 = 75 cnt0 = 74"
"for mozambique use polygon-file - cnt1 = 24 cnt0 = 23"
"for azerbaijan do not use polygon-file - cnt1 = 3 cnt0 = 4"
"for iran use polygon-file - cnt1 = 16 cnt0 = 15"
"for malaysia-singapore-brunei do not use polygon-file - cnt1 = 16 cnt0 =
17"
"for china use polygon-file - cnt1 = 106 cnt0 = 101"
"for india do not use polygon-file - cnt1 = 109 cnt0 = 112"
"for indonesia do not use polygon-file - cnt1 = 191 cnt0 = 200"
"for philippines use polygon-file - cnt1 = 52 cnt0 = 51"
"for afghanistan do not use polygon-file - cnt1 = 11 cnt0 = 20"
"for australia use polygon-file - cnt1 = 67 cnt0 = 66"
"for argentina use polygon-file - cnt1 = 32 cnt0 = 27"
"for brazil use polygon-file - cnt1 = 177 cnt0 = 172"
"for chile do not use polygon-file - cnt1 = 31 cnt0 = 33"
"for paraguay use polygon-file - cnt1 = 17 cnt0 = 16"
"for canada use polygon-file - cnt1 = 274 cnt0 = 272"
"for us-northeast do not use polygon-file - cnt1 = 109 cnt0 = 110"
"for us-pacific use polygon-file - cnt1 = 21 cnt0 = 18"
"for us-south use polygon-file - cnt1 = 262 cnt0 = 261"
"for us-west use polygon-file - cnt1 = 229 cnt0 = 227"
"for greenland use polygon-file - cnt1 = 4 cnt0 = 2"
"for mexico use polygon-file - cnt1 = 44 cnt0 = 42"
"for reunion do not use polygon-file - cnt1 = 3 cnt0 = 5"

On Wed, 30 Jun 2021 at 12:30, Gerd Petermann <
gpetermann_muenc...@hotmail.com> wrote:

> Hi Felix,
>
> I've released r615 to fix the problems with Saarland (now 17 tiles with
> polygon instead of crash, 4 tiles without) and Norway (121 with polygon,
> 128 without)
> Numbers for max-nodes=140
>
> Gerd
>
>
> 
> Von: mkgmap-dev  im Auftrag von
> Felix Hartmann 
> Gesendet: Mittwoch, 30. Juni 2021 10:52
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] splitter r609 released
>
> well yeah by now my list is down to very few countries (different then
> with older splitter.jar) that profit from --polygon-file.
> I feel like I will only use polygon-file for Asia continent map and for
> Russia - they fit your description of spanning over 180° and needing
> polygon-file to be split into less tiles. They pretty consistently split
> better with polygon-file on several splitter versions.
>
> Then maybe retest a year later.
>
> On Wed, 30 Jun 2021 at 11:25, Gerd Petermann <
> gpetermann_muenc...@hotmail.com<mailto:gpetermann_muenc...@ho

Re: [mkgmap-dev] splitter r609 released

2021-06-30 Thread Gerd Petermann

Hi Felix,

I've released r615 to fix the problems with Saarland (now 17 tiles with polygon
instead of crash, 4 tiles without) and Norway (121 with polygon, 128 without)
Numbers for max-nodes=140

Gerd

Von: mkgmap-dev im Auftrag von Felix
Hartmann
Gesendet: Mittwoch, 30. Juni 2021 10:52
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] splitter r609 released

well yeah by now my list is down to very few countries (different then with
older splitter.jar) that profit from --polygon-file.
I feel like I will only use polygon-file for Asia continent map and for Russia
- they fit your description of spanning over 180° and needing polygon-file to
be split into less tiles. They pretty consistently split better with
polygon-file on several splitter versions.

Then maybe retest a year later.

On Wed, 30 Jun 2021 at 11:25, Gerd Petermann
mailto:gpetermann_muenc...@hotmail.com>> wrote:
Hi Felix,

sure, it's not good if splitter fails to split correct data. Still, an
automated process should check the return code of the program. If splitter
returns a values != 0 you can be sure that something is wrong (same with
mkgmap)

It's okay to split small files like Liechtenstein, no problem with that. Just
don't expect to always get fewer tiles when you use a polygon-file ;)

Gerd

Von: mkgmap-dev
mailto:mkgmap-dev-boun...@lists.mkgmap.org.uk>>
im Auftrag von Felix Hartmann
mailto:extremecar...@gmail.com>>
Gesendet: Mittwoch, 30. Juni 2021 10:10
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] splitter r609 released

yes my script only counts the produced tiles. And if it's 0 that's very bad for
any automated process. 0 should only happen if the input data is broken (yes
then it should happen).
(and why do I split countries like Liechtenstein - well first I do not know if
the extract would fit into one tile without splitting, second if the download
is broken for some reason - then splitter rightfully cannot split it - and the
old maps on the server will not be overwritten by maps based on a broken
extract from geofabrik).

On Wed, 30 Jun 2021 at 11:03, Gerd Petermann
mailto:gpetermann_muenc...@hotmail.com><mailto:gpetermann_muenc...@hotmail.com<mailto:gpetermann_muenc...@hotmail.com>>>
wrote:
Hi Felix,

I can now reproduce the poor result with norway when I use both --polygon-file
and --precomp-sea.
Problem seems to be that splitter decides to try the slower algo first. After a
while it gives up and tries the alternative faster algo and that would find a
good solution soon but splitter gives up too early, so this is really not an
improvement.
Without the polygon the slower algo finds a good solution.

I am working on a new solution which executes both algos in parallel and
selects the better result. This works much better for this test case but
sometimes worse when option --num-tiles is used.

reg. Saarland with polygon: r614 stops with an error message (and returns 1 to
signal a bad split). Your scripts seems to ignore this and just counts the
number of written tiles?
I'll try to find a fix so that splitter doesn't try to fit the tiles into the
polygon when this happens...

Gerd

Von: mkgmap-dev
mailto:mkgmap-dev-boun...@lists.mkgmap.org.uk><mailto:mkgmap-dev-boun...@lists.mkgmap.org.uk<mailto:mkgmap-dev-boun...@lists.mkgmap.org.uk>>>
im Auftrag von Felix Hartmann
mailto:extremecar...@gmail.com><mailto:extremecar...@gmail.com<mailto:extremecar...@gmail.com>>>
Gesendet: Mittwoch, 30. Juni 2021 08:55
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] splitter r609 released

okay yeah I think I misunderstood the polygon-file a bit. But why is the split
for Norway so much worse now? Before it was possible to get it down to 115
tiles (using the polygon-file option) now its 168 instead. Not using
polygon-file no change. (guess the 1 tile more is maybe added data somewhere).

On Wed, 30 Jun 2021 at 07:59, Gerd Petermann
mailto:gpetermann_muenc...@hotmail.com><mailto:gpetermann_muenc...@hotmail.com<mailto:gpetermann_muenc...@hotmail.com>><mailto:gpetermann_muenc...@hotmail.com<mailto:gpetermann_muenc...@hotmail.com><mailto:gpetermann_muenc...@hotmail.com<mailto:gpetermann_muenc...@hotmail.com>>>>
wrote:
Hi Felix,

reg. Saarland: I'll try to reproduce. Looks like an error.
reg. Liechtenstein:
I think we still have different ideas what the polygon-file option is about.
The documentation is a bit outdated since r433 (see
https://www.mkgmap.org.uk/websvn/revision.php?repname=splitter=433 )
but the basic idea was that splitter should try to produce tiles which don't
overlap the given polygon too much. This sometimes means that it produces more
and smaller tiles.
Splitter should be able to find good splits for geofabrik downloads with

Re: [mkgmap-dev] splitter r609 released

2021-06-30 Thread Felix Hartmann

well yeah by now my list is down to very few countries (different then with
older splitter.jar) that profit from --polygon-file.
I feel like I will only use polygon-file for Asia continent map and for
Russia - they fit your description of spanning over 180° and needing
polygon-file to be split into less tiles. They pretty consistently split
better with polygon-file on several splitter versions.

Then maybe retest a year later.

On Wed, 30 Jun 2021 at 11:25, Gerd Petermann <
gpetermann_muenc...@hotmail.com> wrote:

> Hi Felix,
>
> sure, it's not good if splitter fails to split correct data. Still, an
> automated process should check the return code of the program. If splitter
> returns a values !=  0 you can be sure that something is wrong (same with
> mkgmap)
>
> It's okay to split small files like Liechtenstein, no problem with that.
> Just don't expect to always get fewer tiles when you use a polygon-file ;)
>
> Gerd
>
> 
> Von: mkgmap-dev  im Auftrag von
> Felix Hartmann 
> Gesendet: Mittwoch, 30. Juni 2021 10:10
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] splitter r609 released
>
> yes my script only counts the produced tiles. And if it's 0 that's very
> bad for any automated process. 0 should only happen if the input data is
> broken (yes then it should happen).
> (and why do I split countries like Liechtenstein - well first I do not
> know if the extract would fit into one tile without splitting, second if
> the download is broken for some reason - then splitter rightfully cannot
> split it - and the old maps on the server will not be overwritten by maps
> based on a broken extract from geofabrik).
>
> On Wed, 30 Jun 2021 at 11:03, Gerd Petermann <
> gpetermann_muenc...@hotmail.com<mailto:gpetermann_muenc...@hotmail.com>>
> wrote:
> Hi Felix,
>
> I can now reproduce the poor result with norway when I use both
> --polygon-file and --precomp-sea.
> Problem seems to be that splitter decides to try the slower algo first.
> After a while it gives up and tries the alternative faster algo and that
> would find a good solution soon but splitter gives up too early, so this is
> really not an improvement.
> Without the polygon the slower algo finds a good solution.
>
> I am working on a new solution which executes both algos in parallel and
> selects the better result. This works much better for this test case but
> sometimes worse when option --num-tiles is used.
>
> reg. Saarland with polygon: r614 stops with an error message (and returns
> 1 to signal a bad split). Your scripts seems to ignore this and just counts
> the number of written tiles?
> I'll try to find a fix so that splitter doesn't try to fit the tiles into
> the polygon when this happens...
>
> Gerd
>
> 
> Von: mkgmap-dev  mkgmap-dev-boun...@lists.mkgmap.org.uk>> im Auftrag von Felix Hartmann <
> extremecar...@gmail.com<mailto:extremecar...@gmail.com>>
> Gesendet: Mittwoch, 30. Juni 2021 08:55
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] splitter r609 released
>
> okay yeah I think I misunderstood the polygon-file a bit. But why is the
> split for Norway so much worse now? Before it was possible to get it down
> to 115 tiles (using the polygon-file option) now its 168 instead. Not using
> polygon-file no change. (guess the 1 tile more is maybe added data
> somewhere).
>
> On Wed, 30 Jun 2021 at 07:59, Gerd Petermann <
> gpetermann_muenc...@hotmail.com<mailto:gpetermann_muenc...@hotmail.com
> ><mailto:gpetermann_muenc...@hotmail.com gpetermann_muenc...@hotmail.com>>> wrote:
> Hi Felix,
>
> reg. Saarland: I'll try to reproduce. Looks like an error.
> reg. Liechtenstein:
> I think we still have different ideas what the polygon-file option is
> about.
> The documentation is a bit outdated since r433 (see
> https://www.mkgmap.org.uk/websvn/revision.php?repname=splitter=433 )
> but the basic idea was that splitter should try to produce tiles which
> don't overlap the given polygon too much. This sometimes means that it
> produces more and smaller tiles.
> Splitter should be able to find good splits for geofabrik downloads
> without a polygon-file, but there are use cases: If the user only wants a
> part of a download in the map they can use the corresponding polygon.
>
> You seem to suggest that splitter should just use the polygon to be able
> to ignore data that is in the input file but not in the polygon and never
> try to fit the tiles into the polygon?
>
> Gerd
>
>
>
>
>
>
>
> 
> Von: mkgmap-dev  mkgmap-dev-boun...@lists.mkgmap.org.uk> mkgmap-

Re: [mkgmap-dev] splitter r609 released

2021-06-30 Thread Gerd Petermann

Hi Felix,

It's okay to split small files like Liechtenstein, no problem with that. Just
don't expect to always get fewer tiles when you use a polygon-file ;)

Gerd

Von: mkgmap-dev im Auftrag von Felix
Hartmann
Gesendet: Mittwoch, 30. Juni 2021 10:10
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] splitter r609 released

On Wed, 30 Jun 2021 at 11:03, Gerd Petermann
mailto:gpetermann_muenc...@hotmail.com>> wrote:
Hi Felix,

I am working on a new solution which executes both algos in parallel and
selects the better result. This works much better for this test case but
sometimes worse when option --num-tiles is used.

Gerd

Von: mkgmap-dev
mailto:mkgmap-dev-boun...@lists.mkgmap.org.uk>>
im Auftrag von Felix Hartmann
mailto:extremecar...@gmail.com>>
Gesendet: Mittwoch, 30. Juni 2021 08:55
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] splitter r609 released

On Wed, 30 Jun 2021 at 07:59, Gerd Petermann
mailto:gpetermann_muenc...@hotmail.com><mailto:gpetermann_muenc...@hotmail.com<mailto:gpetermann_muenc...@hotmail.com>>>
wrote:
Hi Felix,

You seem to suggest that splitter should just use the polygon to be able to
ignore data that is in the input file but not in the polygon and never try to
fit the tiles into the polygon?

Gerd

Von: mkgmap-dev
mailto:mkgmap-dev-boun...@lists.mkgmap.org.uk><mailto:mkgmap-dev-boun...@lists.mkgmap.org.uk<mailto:mkgmap-dev-boun...@lists.mkgmap.org.uk>>>
im Auftrag von Felix Hartmann
mailto:extremecar...@gmail.com><mailto:extremecar...@gmail.com<mailto:extremecar...@gmail.com>>>
Gesendet: Dienstag, 29. Juni 2021 23:04
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] splitter r609 released

I found (server) time to run my test again - using splitter v614 - search limit
100
Problem cases in bold. I also added the number of tiles that splitter v602
needed (well on one week older geofabrik extract - but that should not make
such a difference for Norway or South America). I feel 602 with search-limit
100 was producing more consistent results.

"for netherlands use polygon-file - cnt1 = 93 cnt0 = 88"
"for alps do not use polygon-file - cnt1 = 225 cnt0 = 226"
"for liechtenstein do not use polygon-file - cnt1 = 1 cnt0 = 6"
I thought that the new version fixes the splitting if it is not actually needed.

"for mona

Re: [mkgmap-dev] splitter r609 released

2021-06-30 Thread Felix Hartmann

yes my script only counts the produced tiles. And if it's 0 that's very bad
for any automated process. 0 should only happen if the input data is broken
(yes then it should happen).
(and why do I split countries like Liechtenstein - well first I do not know
if the extract would fit into one tile without splitting, second if the
download is broken for some reason - then splitter rightfully cannot split
it - and the old maps on the server will not be overwritten by maps based
on a broken extract from geofabrik).

On Wed, 30 Jun 2021 at 11:03, Gerd Petermann <
gpetermann_muenc...@hotmail.com> wrote:

> Hi Felix,
>
> I can now reproduce the poor result with norway when I use both
> --polygon-file and --precomp-sea.
> Problem seems to be that splitter decides to try the slower algo first.
> After a while it gives up and tries the alternative faster algo and that
> would find a good solution soon but splitter gives up too early, so this is
> really not an improvement.
> Without the polygon the slower algo finds a good solution.
>
> I am working on a new solution which executes both algos in parallel and
> selects the better result. This works much better for this test case but
> sometimes worse when option --num-tiles is used.
>
> reg. Saarland with polygon: r614 stops with an error message (and returns
> 1 to signal a bad split). Your scripts seems to ignore this and just counts
> the number of written tiles?
> I'll try to find a fix so that splitter doesn't try to fit the tiles into
> the polygon when this happens...
>
> Gerd
>
> 
> Von: mkgmap-dev  im Auftrag von
> Felix Hartmann 
> Gesendet: Mittwoch, 30. Juni 2021 08:55
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] splitter r609 released
>
> okay yeah I think I misunderstood the polygon-file a bit. But why is the
> split for Norway so much worse now? Before it was possible to get it down
> to 115 tiles (using the polygon-file option) now its 168 instead. Not using
> polygon-file no change. (guess the 1 tile more is maybe added data
> somewhere).
>
> On Wed, 30 Jun 2021 at 07:59, Gerd Petermann <
> gpetermann_muenc...@hotmail.com<mailto:gpetermann_muenc...@hotmail.com>>
> wrote:
> Hi Felix,
>
> reg. Saarland: I'll try to reproduce. Looks like an error.
> reg. Liechtenstein:
> I think we still have different ideas what the polygon-file option is
> about.
> The documentation is a bit outdated since r433 (see
> https://www.mkgmap.org.uk/websvn/revision.php?repname=splitter=433 )
> but the basic idea was that splitter should try to produce tiles which
> don't overlap the given polygon too much. This sometimes means that it
> produces more and smaller tiles.
> Splitter should be able to find good splits for geofabrik downloads
> without a polygon-file, but there are use cases: If the user only wants a
> part of a download in the map they can use the corresponding polygon.
>
> You seem to suggest that splitter should just use the polygon to be able
> to ignore data that is in the input file but not in the polygon and never
> try to fit the tiles into the polygon?
>
> Gerd
>
>
>
>
>
>
>
> 
> Von: mkgmap-dev  mkgmap-dev-boun...@lists.mkgmap.org.uk>> im Auftrag von Felix Hartmann <
> extremecar...@gmail.com<mailto:extremecar...@gmail.com>>
> Gesendet: Dienstag, 29. Juni 2021 23:04
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] splitter r609 released
>
> I found (server) time to run my test again - using splitter v614 - search
> limit 100
> Problem cases in bold. I also added the number of tiles that splitter v602
> needed (well on one week older geofabrik extract - but that should not make
> such a difference for Norway or South America). I feel 602 with
> search-limit 100 was producing more consistent results.
>
> "for netherlands use polygon-file - cnt1 = 93 cnt0 = 88"
> "for alps do not use polygon-file - cnt1 = 225 cnt0 = 226"
> "for liechtenstein do not use polygon-file - cnt1 = 1 cnt0 = 6"
> I thought that the new version fixes the splitting if it is not actually
> needed.
>
> "for monaco do not use polygon-file - cnt1 = 1 cnt0 = 2"
> "for slovenia do not use polygon-file - cnt1 = 27 cnt0 = 28"
> "for ukraine use polygon-file - cnt1 = 62 cnt0 = 61"
> "for norway do not use polygon-file - cnt1 = 129 cnt0 = 168"
> Big degradation here - with the older splitter it was: "for norway use
> polygon-file - cnt1 = 128 cnt0 = 115" (also using 100 search limit)
>
> "for switzerland do not use polygon-file - cnt1 = 29 cnt0 = 30"
> "for poland do

Re: [mkgmap-dev] splitter r609 released

2021-06-30 Thread Gerd Petermann

Hi Felix,

I can now reproduce the poor result with norway when I use both --polygon-file 
and --precomp-sea.
Problem seems to be that splitter decides to try the slower algo first. After a 
while it gives up and tries the alternative faster algo and that would find a 
good solution soon but splitter gives up too early, so this is really not an 
improvement.
Without the polygon the slower algo finds a good solution.

I am working on a new solution which executes both algos in parallel and 
selects the better result. This works much better for this test case but 
sometimes worse when option --num-tiles is used.

reg. Saarland with polygon: r614 stops with an error message (and returns 1 to 
signal a bad split). Your scripts seems to ignore this and just counts the 
number of written tiles?
I'll try to find a fix so that splitter doesn't try to fit the tiles into the 
polygon when this happens...

Gerd


Von: mkgmap-dev  im Auftrag von Felix 
Hartmann 
Gesendet: Mittwoch, 30. Juni 2021 08:55
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] splitter r609 released

okay yeah I think I misunderstood the polygon-file a bit. But why is the split 
for Norway so much worse now? Before it was possible to get it down to 115 
tiles (using the polygon-file option) now its 168 instead. Not using 
polygon-file no change. (guess the 1 tile more is maybe added data somewhere).

On Wed, 30 Jun 2021 at 07:59, Gerd Petermann 
mailto:gpetermann_muenc...@hotmail.com>> wrote:
Hi Felix,

reg. Saarland: I'll try to reproduce. Looks like an error.
reg. Liechtenstein:
I think we still have different ideas what the polygon-file option is about.
The documentation is a bit outdated since r433 (see 
https://www.mkgmap.org.uk/websvn/revision.php?repname=splitter=433 )
but the basic idea was that splitter should try to produce tiles which don't 
overlap the given polygon too much. This sometimes means that it produces more 
and smaller tiles.
Splitter should be able to find good splits for geofabrik downloads without a 
polygon-file, but there are use cases: If the user only wants a part of a 
download in the map they can use the corresponding polygon.

You seem to suggest that splitter should just use the polygon to be able to 
ignore data that is in the input file but not in the polygon and never try to 
fit the tiles into the polygon?

Gerd








Von: mkgmap-dev 
mailto:mkgmap-dev-boun...@lists.mkgmap.org.uk>>
 im Auftrag von Felix Hartmann 
mailto:extremecar...@gmail.com>>
Gesendet: Dienstag, 29. Juni 2021 23:04
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] splitter r609 released

I found (server) time to run my test again - using splitter v614 - search limit 
100
Problem cases in bold. I also added the number of tiles that splitter v602 
needed (well on one week older geofabrik extract - but that should not make 
such a difference for Norway or South America). I feel 602 with search-limit 
100 was producing more consistent results.

"for netherlands use polygon-file - cnt1 = 93 cnt0 = 88"
"for alps do not use polygon-file - cnt1 = 225 cnt0 = 226"
"for liechtenstein do not use polygon-file - cnt1 = 1 cnt0 = 6"
I thought that the new version fixes the splitting if it is not actually needed.

"for monaco do not use polygon-file - cnt1 = 1 cnt0 = 2"
"for slovenia do not use polygon-file - cnt1 = 27 cnt0 = 28"
"for ukraine use polygon-file - cnt1 = 62 cnt0 = 61"
"for norway do not use polygon-file - cnt1 = 129 cnt0 = 168"
Big degradation here - with the older splitter it was: "for norway use 
polygon-file - cnt1 = 128 cnt0 = 115" (also using 100 search limit)

"for switzerland do not use polygon-file - cnt1 = 29 cnt0 = 30"
"for poland do not use polygon-file - cnt1 = 127 cnt0 = 128"
"for sweden use polygon-file - cnt1 = 60 cnt0 = 54"
"for finland do not use polygon-file - cnt1 = 65 cnt0 = 84"
"for czech-republic use polygon-file - cnt1 = 63 cnt0 = 62"
"for denmark use polygon-file - cnt1 = 33 cnt0 = 32"
"for austria use polygon-file - cnt1 = 56 cnt0 = 54"
"for andorra do not use polygon-file - cnt1 = 1 cnt0 = 4"
"for estonia use polygon-file - cnt1 = 9 cnt0 = 8"
"for saarland use polygon-file - cnt1 = 4 cnt0 = 0"
Seems using a polygon-file no spit at all - however it should still write the 
data again out into the newly defined filename.

"for hamburg do not use polygon-file - cnt1 = 3 cnt0 = 12"
"for hessen do not use polygon-file - cnt1 = 17 cnt0 = 18"
"for bayern do not use polygon-file - cnt1 = 47 cnt0 = 48"
"for berlin do not use polygon-file - cnt1 = 5 cnt0 = 13"
"for australia-oceania use polygon-file - cnt1 = 112 cnt0 = 109"
"for south-america do not use polygon-fil

Re: [mkgmap-dev] splitter r609 released

2021-06-30 Thread Gerd Petermann

Hi Felix,

don't know why norway changed that much.  It would really safe time if you 
would provide a link to the splitter log and the densities-out.file. Those two 
small files are all that I need to reproduce your results.
I see no regression with my older norway extract and r614.

Now I have to download the full norway extract to find out...

Gerd


Von: mkgmap-dev  im Auftrag von Felix 
Hartmann 
Gesendet: Mittwoch, 30. Juni 2021 08:55
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] splitter r609 released

okay yeah I think I misunderstood the polygon-file a bit. But why is the split 
for Norway so much worse now? Before it was possible to get it down to 115 
tiles (using the polygon-file option) now its 168 instead. Not using 
polygon-file no change. (guess the 1 tile more is maybe added data somewhere).

On Wed, 30 Jun 2021 at 07:59, Gerd Petermann 
mailto:gpetermann_muenc...@hotmail.com>> wrote:
Hi Felix,

reg. Saarland: I'll try to reproduce. Looks like an error.
reg. Liechtenstein:
I think we still have different ideas what the polygon-file option is about.
The documentation is a bit outdated since r433 (see 
https://www.mkgmap.org.uk/websvn/revision.php?repname=splitter=433 )
but the basic idea was that splitter should try to produce tiles which don't 
overlap the given polygon too much. This sometimes means that it produces more 
and smaller tiles.
Splitter should be able to find good splits for geofabrik downloads without a 
polygon-file, but there are use cases: If the user only wants a part of a 
download in the map they can use the corresponding polygon.

You seem to suggest that splitter should just use the polygon to be able to 
ignore data that is in the input file but not in the polygon and never try to 
fit the tiles into the polygon?

Gerd








Von: mkgmap-dev 
mailto:mkgmap-dev-boun...@lists.mkgmap.org.uk>>
 im Auftrag von Felix Hartmann 
mailto:extremecar...@gmail.com>>
Gesendet: Dienstag, 29. Juni 2021 23:04
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] splitter r609 released

I found (server) time to run my test again - using splitter v614 - search limit 
100
Problem cases in bold. I also added the number of tiles that splitter v602 
needed (well on one week older geofabrik extract - but that should not make 
such a difference for Norway or South America). I feel 602 with search-limit 
100 was producing more consistent results.

"for netherlands use polygon-file - cnt1 = 93 cnt0 = 88"
"for alps do not use polygon-file - cnt1 = 225 cnt0 = 226"
"for liechtenstein do not use polygon-file - cnt1 = 1 cnt0 = 6"
I thought that the new version fixes the splitting if it is not actually needed.

"for monaco do not use polygon-file - cnt1 = 1 cnt0 = 2"
"for slovenia do not use polygon-file - cnt1 = 27 cnt0 = 28"
"for ukraine use polygon-file - cnt1 = 62 cnt0 = 61"
"for norway do not use polygon-file - cnt1 = 129 cnt0 = 168"
Big degradation here - with the older splitter it was: "for norway use 
polygon-file - cnt1 = 128 cnt0 = 115" (also using 100 search limit)

"for switzerland do not use polygon-file - cnt1 = 29 cnt0 = 30"
"for poland do not use polygon-file - cnt1 = 127 cnt0 = 128"
"for sweden use polygon-file - cnt1 = 60 cnt0 = 54"
"for finland do not use polygon-file - cnt1 = 65 cnt0 = 84"
"for czech-republic use polygon-file - cnt1 = 63 cnt0 = 62"
"for denmark use polygon-file - cnt1 = 33 cnt0 = 32"
"for austria use polygon-file - cnt1 = 56 cnt0 = 54"
"for andorra do not use polygon-file - cnt1 = 1 cnt0 = 4"
"for estonia use polygon-file - cnt1 = 9 cnt0 = 8"
"for saarland use polygon-file - cnt1 = 4 cnt0 = 0"
Seems using a polygon-file no spit at all - however it should still write the 
data again out into the newly defined filename.

"for hamburg do not use polygon-file - cnt1 = 3 cnt0 = 12"
"for hessen do not use polygon-file - cnt1 = 17 cnt0 = 18"
"for bayern do not use polygon-file - cnt1 = 47 cnt0 = 48"
"for berlin do not use polygon-file - cnt1 = 5 cnt0 = 13"
"for australia-oceania use polygon-file - cnt1 = 112 cnt0 = 109"
"for south-america do not use polygon-file - cnt1 = 335 cnt0 = 463"
old splitter "for south-america do not use polygon-file - cnt1 = 337 cnt0 = 
339"  (using default search limit)

"for africa do not use polygon-file - cnt1 = 653 cnt0 = 663"
"for asia use polygon-file - cnt1 = 1697 cnt0 = 1549"
"for russia use polygon-file - cnt1 = 429 cnt0 = 408"
0ld identical

"for central-america use polygon-file - cnt1 = 60 cnt0 = 58"
"for antarctica use polygon-file - cnt1 = 7 cnt0 = 6"
"for morocco do not use polygon-file - cnt1 = 19 cnt

Re: [mkgmap-dev] splitter r609 released

2021-06-30 Thread Felix Hartmann

okay yeah I think I misunderstood the polygon-file a bit. But why is the
split for Norway so much worse now? Before it was possible to get it down
to 115 tiles (using the polygon-file option) now its 168 instead. Not using
polygon-file no change. (guess the 1 tile more is maybe added data
somewhere).

On Wed, 30 Jun 2021 at 07:59, Gerd Petermann <
gpetermann_muenc...@hotmail.com> wrote:

> Hi Felix,
>
> reg. Saarland: I'll try to reproduce. Looks like an error.
> reg. Liechtenstein:
> I think we still have different ideas what the polygon-file option is
> about.
> The documentation is a bit outdated since r433 (see
> https://www.mkgmap.org.uk/websvn/revision.php?repname=splitter=433 )
> but the basic idea was that splitter should try to produce tiles which
> don't overlap the given polygon too much. This sometimes means that it
> produces more and smaller tiles.
> Splitter should be able to find good splits for geofabrik downloads
> without a polygon-file, but there are use cases: If the user only wants a
> part of a download in the map they can use the corresponding polygon.
>
> You seem to suggest that splitter should just use the polygon to be able
> to ignore data that is in the input file but not in the polygon and never
> try to fit the tiles into the polygon?
>
> Gerd
>
>
>
>
>
>
>
> 
> Von: mkgmap-dev  im Auftrag von
> Felix Hartmann 
> Gesendet: Dienstag, 29. Juni 2021 23:04
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] splitter r609 released
>
> I found (server) time to run my test again - using splitter v614 - search
> limit 100
> Problem cases in bold. I also added the number of tiles that splitter v602
> needed (well on one week older geofabrik extract - but that should not make
> such a difference for Norway or South America). I feel 602 with
> search-limit 100 was producing more consistent results.
>
> "for netherlands use polygon-file - cnt1 = 93 cnt0 = 88"
> "for alps do not use polygon-file - cnt1 = 225 cnt0 = 226"
> "for liechtenstein do not use polygon-file - cnt1 = 1 cnt0 = 6"
> I thought that the new version fixes the splitting if it is not actually
> needed.
>
> "for monaco do not use polygon-file - cnt1 = 1 cnt0 = 2"
> "for slovenia do not use polygon-file - cnt1 = 27 cnt0 = 28"
> "for ukraine use polygon-file - cnt1 = 62 cnt0 = 61"
> "for norway do not use polygon-file - cnt1 = 129 cnt0 = 168"
> Big degradation here - with the older splitter it was: "for norway use
> polygon-file - cnt1 = 128 cnt0 = 115" (also using 100 search limit)
>
> "for switzerland do not use polygon-file - cnt1 = 29 cnt0 = 30"
> "for poland do not use polygon-file - cnt1 = 127 cnt0 = 128"
> "for sweden use polygon-file - cnt1 = 60 cnt0 = 54"
> "for finland do not use polygon-file - cnt1 = 65 cnt0 = 84"
> "for czech-republic use polygon-file - cnt1 = 63 cnt0 = 62"
> "for denmark use polygon-file - cnt1 = 33 cnt0 = 32"
> "for austria use polygon-file - cnt1 = 56 cnt0 = 54"
> "for andorra do not use polygon-file - cnt1 = 1 cnt0 = 4"
> "for estonia use polygon-file - cnt1 = 9 cnt0 = 8"
> "for saarland use polygon-file - cnt1 = 4 cnt0 = 0"
> Seems using a polygon-file no spit at all - however it should still write
> the data again out into the newly defined filename.
>
> "for hamburg do not use polygon-file - cnt1 = 3 cnt0 = 12"
> "for hessen do not use polygon-file - cnt1 = 17 cnt0 = 18"
> "for bayern do not use polygon-file - cnt1 = 47 cnt0 = 48"
> "for berlin do not use polygon-file - cnt1 = 5 cnt0 = 13"
> "for australia-oceania use polygon-file - cnt1 = 112 cnt0 = 109"
> "for south-america do not use polygon-file - cnt1 = 335 cnt0 = 463"
> old splitter "for south-america do not use polygon-file - cnt1 = 337 cnt0
> = 339"  (using default search limit)
>
> "for africa do not use polygon-file - cnt1 = 653 cnt0 = 663"
> "for asia use polygon-file - cnt1 = 1697 cnt0 = 1549"
> "for russia use polygon-file - cnt1 = 429 cnt0 = 408"
> 0ld identical
>
> "for central-america use polygon-file - cnt1 = 60 cnt0 = 58"
> "for antarctica use polygon-file - cnt1 = 7 cnt0 = 6"
> "for morocco do not use polygon-file - cnt1 = 19 cnt0 = 27"
> old identical
>
> "for congo-democratic-republic do not use polygon-file - cnt1 = 37 cnt0 =
> 38"
> "for mozambique use polygon-file - cnt1 = 24 cnt0 = 23"
> "for azerbaijan do not use polygon-file - cnt1 = 3 cnt0 = 4"

Re: [mkgmap-dev] splitter r609 released

2021-06-29 Thread Gerd Petermann

Hi Felix,

reg. Saarland: I'll try to reproduce. Looks like an error.
reg. Liechtenstein:
I think we still have different ideas what the polygon-file option is about.
The documentation is a bit outdated since r433 (see 
https://www.mkgmap.org.uk/websvn/revision.php?repname=splitter=433 )
but the basic idea was that splitter should try to produce tiles which don't 
overlap the given polygon too much. This sometimes means that it produces more 
and smaller tiles.
Splitter should be able to find good splits for geofabrik downloads without a 
polygon-file, but there are use cases: If the user only wants a part of a 
download in the map they can use the corresponding polygon.

You seem to suggest that splitter should just use the polygon to be able to 
ignore data that is in the input file but not in the polygon and never try to 
fit the tiles into the polygon?

Gerd








Von: mkgmap-dev  im Auftrag von Felix 
Hartmann 
Gesendet: Dienstag, 29. Juni 2021 23:04
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] splitter r609 released

I found (server) time to run my test again - using splitter v614 - search limit 
100
Problem cases in bold. I also added the number of tiles that splitter v602 
needed (well on one week older geofabrik extract - but that should not make 
such a difference for Norway or South America). I feel 602 with search-limit 
100 was producing more consistent results.

"for netherlands use polygon-file - cnt1 = 93 cnt0 = 88"
"for alps do not use polygon-file - cnt1 = 225 cnt0 = 226"
"for liechtenstein do not use polygon-file - cnt1 = 1 cnt0 = 6"
I thought that the new version fixes the splitting if it is not actually needed.

"for monaco do not use polygon-file - cnt1 = 1 cnt0 = 2"
"for slovenia do not use polygon-file - cnt1 = 27 cnt0 = 28"
"for ukraine use polygon-file - cnt1 = 62 cnt0 = 61"
"for norway do not use polygon-file - cnt1 = 129 cnt0 = 168"
Big degradation here - with the older splitter it was: "for norway use 
polygon-file - cnt1 = 128 cnt0 = 115" (also using 100 search limit)

"for switzerland do not use polygon-file - cnt1 = 29 cnt0 = 30"
"for poland do not use polygon-file - cnt1 = 127 cnt0 = 128"
"for sweden use polygon-file - cnt1 = 60 cnt0 = 54"
"for finland do not use polygon-file - cnt1 = 65 cnt0 = 84"
"for czech-republic use polygon-file - cnt1 = 63 cnt0 = 62"
"for denmark use polygon-file - cnt1 = 33 cnt0 = 32"
"for austria use polygon-file - cnt1 = 56 cnt0 = 54"
"for andorra do not use polygon-file - cnt1 = 1 cnt0 = 4"
"for estonia use polygon-file - cnt1 = 9 cnt0 = 8"
"for saarland use polygon-file - cnt1 = 4 cnt0 = 0"
Seems using a polygon-file no spit at all - however it should still write the 
data again out into the newly defined filename.

"for hamburg do not use polygon-file - cnt1 = 3 cnt0 = 12"
"for hessen do not use polygon-file - cnt1 = 17 cnt0 = 18"
"for bayern do not use polygon-file - cnt1 = 47 cnt0 = 48"
"for berlin do not use polygon-file - cnt1 = 5 cnt0 = 13"
"for australia-oceania use polygon-file - cnt1 = 112 cnt0 = 109"
"for south-america do not use polygon-file - cnt1 = 335 cnt0 = 463"
old splitter "for south-america do not use polygon-file - cnt1 = 337 cnt0 = 
339"  (using default search limit)

"for africa do not use polygon-file - cnt1 = 653 cnt0 = 663"
"for asia use polygon-file - cnt1 = 1697 cnt0 = 1549"
"for russia use polygon-file - cnt1 = 429 cnt0 = 408"
0ld identical

"for central-america use polygon-file - cnt1 = 60 cnt0 = 58"
"for antarctica use polygon-file - cnt1 = 7 cnt0 = 6"
"for morocco do not use polygon-file - cnt1 = 19 cnt0 = 27"
old identical

"for congo-democratic-republic do not use polygon-file - cnt1 = 37 cnt0 = 38"
"for mozambique use polygon-file - cnt1 = 24 cnt0 = 23"
"for azerbaijan do not use polygon-file - cnt1 = 3 cnt0 = 4"
"for malaysia-singapore-brunei do not use polygon-file - cnt1 = 16 cnt0 = 17"
"for china use polygon-file - cnt1 = 104 cnt0 = 101"
"for india do not use polygon-file - cnt1 = 109 cnt0 = 112"
"for indonesia do not use polygon-file - cnt1 = 175 cnt0 = 200"
"for japan use polygon-file - cnt1 = 163 cnt0 = 160"
"for philippines do not use polygon-file - cnt1 = 50 cnt0 = 51"
"for afghanistan do not use polygon-file - cnt1 = 11 cnt0 = 20"
old identical

"for australia use polygon-file - cnt1 = 67 cnt0 = 66"
"for argentina use polygon-file - cnt1 = 31 cnt0 = 27"
"for brazil use polygon-file - cnt1 = 176 cnt0 = 172"
"for chile do not use polygon-file - cnt1 = 31 cnt0 = 33"
"for paraguay u

Re: [mkgmap-dev] splitter r609 released

2021-06-29 Thread Felix Hartmann

I found (server) time to run my test again - using splitter v614 - search
limit 100
Problem cases in bold. I also added the number of tiles that splitter v602
needed (well on one week older geofabrik extract - but that should not make
such a difference for Norway or South America). I feel 602 with
search-limit 100 was producing more consistent results.

"for netherlands use polygon-file - cnt1 = 93 cnt0 = 88"
"for alps do not use polygon-file - cnt1 = 225 cnt0 = 226"
*"for liechtenstein do not use polygon-file - cnt1 = 1 cnt0 = 6" *
*I thought that the new version fixes the splitting if it is not actually
needed.*

"for monaco do not use polygon-file - cnt1 = 1 cnt0 = 2"
"for slovenia do not use polygon-file - cnt1 = 27 cnt0 = 28"
"for ukraine use polygon-file - cnt1 = 62 cnt0 = 61"
*"for norway do not use polygon-file - cnt1 = 129 cnt0 = 168" *
*Big degradation here - with the older splitter it was: "for norway use
polygon-file - cnt1 = 128 cnt0 = 115" (also using 100 search limit)*

"for switzerland do not use polygon-file - cnt1 = 29 cnt0 = 30"
"for poland do not use polygon-file - cnt1 = 127 cnt0 = 128"
"for sweden use polygon-file - cnt1 = 60 cnt0 = 54"
"for finland do not use polygon-file - cnt1 = 65 cnt0 = 84"
"for czech-republic use polygon-file - cnt1 = 63 cnt0 = 62"
"for denmark use polygon-file - cnt1 = 33 cnt0 = 32"
"for austria use polygon-file - cnt1 = 56 cnt0 = 54"
"for andorra do not use polygon-file - cnt1 = 1 cnt0 = 4"
"for estonia use polygon-file - cnt1 = 9 cnt0 = 8"
*"for saarland use polygon-file - cnt1 = 4 cnt0 = 0" *
*Seems using a polygon-file no spit at all - however it should still write
the data again out into the newly defined filename.*

"for hamburg do not use polygon-file - cnt1 = 3 cnt0 = 12"
"for hessen do not use polygon-file - cnt1 = 17 cnt0 = 18"
"for bayern do not use polygon-file - cnt1 = 47 cnt0 = 48"
"for berlin do not use polygon-file - cnt1 = 5 cnt0 = 13"
"for australia-oceania use polygon-file - cnt1 = 112 cnt0 = 109"
*"for south-america do not use polygon-file - cnt1 = 335 cnt0 = 463" *
*old splitter "for south-america do not use polygon-file - cnt1 = 337 cnt0
= 339"  (using default search limit)*

"for africa do not use polygon-file - cnt1 = 653 cnt0 = 663"
"for asia use polygon-file - cnt1 = 1697 cnt0 = 1549"
*"for russia use polygon-file - cnt1 = 429 cnt0 = 408" *
*0ld identical*

"for central-america use polygon-file - cnt1 = 60 cnt0 = 58"
"for antarctica use polygon-file - cnt1 = 7 cnt0 = 6"
*"for morocco do not use polygon-file - cnt1 = 19 cnt0 = 27" *
*old identical*

"for congo-democratic-republic do not use polygon-file - cnt1 = 37 cnt0 =
38"
"for mozambique use polygon-file - cnt1 = 24 cnt0 = 23"
"for azerbaijan do not use polygon-file - cnt1 = 3 cnt0 = 4"
"for malaysia-singapore-brunei do not use polygon-file - cnt1 = 16 cnt0 =
17"
"for china use polygon-file - cnt1 = 104 cnt0 = 101"
"for india do not use polygon-file - cnt1 = 109 cnt0 = 112"
"for indonesia do not use polygon-file - cnt1 = 175 cnt0 = 200"
"for japan use polygon-file - cnt1 = 163 cnt0 = 160"
"for philippines do not use polygon-file - cnt1 = 50 cnt0 = 51"
*"for afghanistan do not use polygon-file - cnt1 = 11 cnt0 = 20" *
*old identical *

"for australia use polygon-file - cnt1 = 67 cnt0 = 66"
"for argentina use polygon-file - cnt1 = 31 cnt0 = 27"
"for brazil use polygon-file - cnt1 = 176 cnt0 = 172"
"for chile do not use polygon-file - cnt1 = 31 cnt0 = 33"
"for paraguay use polygon-file - cnt1 = 17 cnt0 = 16"
"for us-midwest do not use polygon-file - cnt1 = 152 cnt0 = 154"
"for us-northeast do not use polygon-file - cnt1 = 109 cnt0 = 110"
"for us-pacific use polygon-file - cnt1 = 20 cnt0 = 18"
"for us-south do not use polygon-file - cnt1 = 259 cnt0 = 261"
"for us-west do not use polygon-file - cnt1 = 224 cnt0 = 227"
"for greenland use polygon-file - cnt1 = 4 cnt0 = 2"
"for mexico use polygon-file - cnt1 = 43 cnt0 = 42"
"for reunion do not use polygon-file - cnt1 = 3 cnt0 = 5"

On Fri, 25 Jun 2021 at 16:51, Gerd Petermann <
gpetermann_muenc...@hotmail.com> wrote:

> Hi all,
>
> I think with r609 there should be no need to use a polygon file. See
> https://www.mkgmap.org.uk/websvn/revision.php?repname=splitter=609
>
> Please let me know when this version produces much worse results compared
> to older releases (using the same options and input) and provide the files
> densities-out.txt and the log.
>
> If you are interested in good splits you should check the splitter log for
> "Solution is not nice. Can't find a better solution"
>
> When this is printed splitter did not find a good split. Expect almost
> empty tiles in this case.
> It is likely that this happens when rather small files are split with a
> polygon and the default resolution. Normal users don't do this, but some
> map providers try to use the same options for very different downloads ...
>
> Gerd
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
>

Re: [mkgmap-dev] splitter default resolution 13 and overview map from default style

2021-06-02 Thread Ticker Berkin

Hi all

Outermost level should be as Gerd suggests
5 levels of overview seems excessive - what about:

overview-levels = 4:17, 5:15, 6:13

Ticker


On Tue, 2021-06-01 at 13:29 +, Gerd Petermann wrote:
> Hi all,
> 
> I am not sure but I think it is a bad idea to have 
> overview-levels = 4:17, 5:16, 6:15, 7:14, 8:12
> in the default style.
> The lowest resolution is 12 while splitter uses the default 13 for
> its resolution. This also means that tiles from splitter are aligned
> to a resolution of 13. The lowest level in the overview map is
> therefore sometimes much more distorted at tile boundaries than the
> next higher. I think it should be changed to 
> overview-levels = 4:17, 5:16, 6:15, 7:14, 8:13
> 
> I've never thought about this relationship between splitter and the
> overview map. Just wondered why my test data looks so ugly at
> resolution 12.
> An alternative would be to remove the lowest level completely.
> Maybe mkgmap should print a warning when tiles are not aligned for
> the lowest resolution used in the map?
> 
> Any thoughts? 
> 
> Gerd
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter default resolution 13 and overview map from default style

2021-06-01 Thread Joris Bo

Hallo Gerd



You just solved a long time wish of getting country borders more nice at low 
zoom:

Left with overview 8:12  and right with lowest  8:13 (which is much nicer !)





Basecamp zoomlevel 300km

Gulf of Genua



[cid:image001.png@01D75704.89BA1B80]





Met vriendelijke groeten,



Joris Bo

jori...@hotmail.com



-Oorspronkelijk bericht-
Van: mkgmap-dev 
mailto:mkgmap-dev-boun...@lists.mkgmap.org.uk>>
 Namens Gerd Petermann
Verzonden: dinsdag 1 juni 2021 15:30
Aan: mkgmap-dev@lists.mkgmap.org.uk
Onderwerp: [mkgmap-dev] splitter default resolution 13 and overview map from 
default style



Hi all,



I am not sure but I think it is a bad idea to have overview-levels = 4:17, 
5:16, 6:15, 7:14, 8:12 in the default style.

The lowest resolution is 12 while splitter uses the default 13 for its 
resolution. This also means that tiles from splitter are aligned to a 
resolution of 13. The lowest level in the overview map is therefore sometimes 
much more distorted at tile boundaries than the next higher. I think it should 
be changed to overview-levels = 4:17, 5:16, 6:15, 7:14, 8:13



I've never thought about this relationship between splitter and the overview 
map. Just wondered why my test data looks so ugly at resolution 12.

An alternative would be to remove the lowest level completely.

Maybe mkgmap should print a warning when tiles are not aligned for the lowest 
resolution used in the map?



Any thoughts?



Gerd

___

mkgmap-dev mailing list

mkgmap-dev@lists.mkgmap.org.uk

https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Splitter only writes points, not ways on huge file?

2021-05-29 Thread Felix Hartmann

Hi Gerd,

thanks I assume there are no ways - and phyghtmap does not work correctly
either on such a huge area - has maybe a 64GB limit which would be broken?
or does not work on the poly file with two areas. I will run it again now
against both single areas (one has finished already and looks okay - but
that was the tiny area). The other area will need 15 days or so as
phyghtmap on ubuntu is using 1 thread only (can use 2 threads - but that is
only maybe 1% faster. if more threads just wasting memory but not being any
faster). So I will wait and see in 2 weeks...


osmconvert also needed like 60 minutes or so to put out:
timestamp min: 2021-05-13T08:41:54Z
timestamp max: 2021-05-13T08:41:54Z
lon min: 44.999
lon max: 179.9991666
lat min: -11.0025000
lat max: 81.8573611
nodes: 26761028462
ways: 0
relations: 0
node id min: 1000
node id max: 26771065386


so clearly no ways.

On Sat, 29 May 2021 at 08:47, Gerd Petermann <
gpetermann_muenc...@hotmail.com> wrote:

> Hi Felix,
> the log shows
> "way Map is empty"
> so splitter doesn't find any ways in asia10m_.osm.pbf.
>
> So, either the file doesn't contain ways or splitter fails to find them.
> Maybe use
> osmconvert --out-statistics asia10m_.osm.pbf
> to find out more.
>
> Reg. performance when splitting huge files with --keep-complete
> use o5m format and --max-areas=4096 (or higher)
>
> I'll have to check why splitter needs so much time in the
> "Problem-list-generator" passes. Without ways or relations there is no need
> to even execute that.
>
> The asia.poly contains two rings. I think splitter can copy with that but
> maybe phyghtmap cannot? Try with two separate *.poly files
>
> Gerd
>
> 
> Von: mkgmap-dev  im Auftrag von
> Felix Hartmann 
> Gesendet: Freitag, 28. Mai 2021 19:29
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Splitter only writes points, not ways on huge
> file?
>
> Attached the full splitter.log - looks fine. Is it most likely that the
> error is on phyghtmap?
>
> On Fri, 28 May 2021 at 13:12, Felix Hartmann  <mailto:extremecar...@gmail.com>> wrote:
> running now again with resolution 13, I had tried if changing resolution
> to 12 would solve the problem - but is does not.
>
> On Fri, 28 May 2021 at 13:11, Felix Hartmann  <mailto:extremecar...@gmail.com>> wrote:
> Asia.poly is attached - I did not save the splitter.log - doing this right
> now (but will take some hours - splitting this huge file takes a lot of
> time)
>
> The log starts with:
> Splitter version 597 compiled 2020-01-30T14:10:47+
> boundary-tags=use-exclude-list
> cache=
> description=
> geonames-file=
> handle-element-version=remove
> ignore-osm-bounds=false
> keep-complete=true
> mapid=7540
> max-areas=2048
> max-nodes=1200
> max-threads=7
> mixed=false
> no-trim=true
> num-tiles=
> output=pbf
> output-dir=
> overlap=auto
> polygon-desc-file=
> polygon-file=
> precomp-sea=
> problem-file=
> problem-report=
> resolution=13
> route-rel-values=
> search-limit=20
> split-file=
> status-freq=120
> stop-after=dist
> wanted-admin-level=5
> write-kml=
> Elapsed time: 0s   Memory: Current 1012MB (7MB used, 1005MB free) Max
> 25000MB
> Time started: Fri May 28 12:06:59 CEST 2021
> Map is being split for resolution 13:
>  - area boundaries are aligned to 0x800 map units (0.0439453125 degrees)
>  - areas are multiples of 0x800 map units wide and high
> Processing asia10m_.osm.pbf
> Bounding box -179.03 -13.0116501 179.03
> 84.52666
> 10,000,000 nodes parsed... id=1999
> 20,000,000 nodes parsed... id=2999
> 30,000,000 nodes parsed... id=3999
> 40,000,000 nodes parsed... id=4999
> 50,000,000 nodes parsed... id=5999
> 60,000,000 nodes parsed... id=6999
> 70,000,000 nodes parsed... id=7999
> 80,000,000 nodes parsed... id=8999
> 90,000,000 nodes parsed... id=
> 100,000,000 nodes parsed... id=10999
>
> On Fri, 28 May 2021 at 12:25, Gerd Petermann <
> gpetermann_muenc...@hotmail.com<mailto:gpetermann_muenc...@hotmail.com>>
> wrote:
> Hi Felix,
>
> the pbf file really contains only nodes.
> Can you attach the splitter log file and the asia.poly, please?
>
> Gerd
>
> 
> Von: mkgmap-dev  mkgmap-dev-boun...@lists.mkgmap.org.uk>> im Auftrag von Felix Hartmann <
> extremecar...@gmail.com<mailto:extremecar...@gmail.com>>
> Gesendet: Freitag, 28. Mai 2021 11:18
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Splitter only writes points, not ways on huge
> file?
>
> Hi Gerd,
> so

Re: [mkgmap-dev] Splitter only writes points, not ways on huge file?

2021-05-28 Thread Gerd Petermann

Hi Felix,
the log shows
"way Map is empty"
so splitter doesn't find any ways in asia10m_.osm.pbf.

So, either the file doesn't contain ways or splitter fails to find them. Maybe 
use
osmconvert --out-statistics asia10m_.osm.pbf
to find out more.

Reg. performance when splitting huge files with --keep-complete
use o5m format and --max-areas=4096 (or higher)

I'll have to check why splitter needs so much time in the 
"Problem-list-generator" passes. Without ways or relations there is no need to 
even execute that.

The asia.poly contains two rings. I think splitter can copy with that but maybe 
phyghtmap cannot? Try with two separate *.poly files

Gerd


Von: mkgmap-dev  im Auftrag von Felix 
Hartmann 
Gesendet: Freitag, 28. Mai 2021 19:29
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] Splitter only writes points, not ways on huge file?

Attached the full splitter.log - looks fine. Is it most likely that the error 
is on phyghtmap?

On Fri, 28 May 2021 at 13:12, Felix Hartmann 
mailto:extremecar...@gmail.com>> wrote:
running now again with resolution 13, I had tried if changing resolution to 12 
would solve the problem - but is does not.

On Fri, 28 May 2021 at 13:11, Felix Hartmann 
mailto:extremecar...@gmail.com>> wrote:
Asia.poly is attached - I did not save the splitter.log - doing this right now 
(but will take some hours - splitting this huge file takes a lot of time)

The log starts with:
Splitter version 597 compiled 2020-01-30T14:10:47+
boundary-tags=use-exclude-list
cache=
description=
geonames-file=
handle-element-version=remove
ignore-osm-bounds=false
keep-complete=true
mapid=7540
max-areas=2048
max-nodes=1200
max-threads=7
mixed=false
no-trim=true
num-tiles=
output=pbf
output-dir=
overlap=auto
polygon-desc-file=
polygon-file=
precomp-sea=
problem-file=
problem-report=
resolution=13
route-rel-values=
search-limit=20
split-file=
status-freq=120
stop-after=dist
wanted-admin-level=5
write-kml=
Elapsed time: 0s   Memory: Current 1012MB (7MB used, 1005MB free) Max 25000MB
Time started: Fri May 28 12:06:59 CEST 2021
Map is being split for resolution 13:
 - area boundaries are aligned to 0x800 map units (0.0439453125 degrees)
 - areas are multiples of 0x800 map units wide and high
Processing asia10m_.osm.pbf
Bounding box -179.03 -13.0116501 179.03 84.52666
10,000,000 nodes parsed... id=1999
20,000,000 nodes parsed... id=2999
30,000,000 nodes parsed... id=3999
40,000,000 nodes parsed... id=4999
50,000,000 nodes parsed... id=5999
60,000,000 nodes parsed... id=6999
70,000,000 nodes parsed... id=7999
80,000,000 nodes parsed... id=8999
90,000,000 nodes parsed... id=
100,000,000 nodes parsed... id=10999

On Fri, 28 May 2021 at 12:25, Gerd Petermann 
mailto:gpetermann_muenc...@hotmail.com>> wrote:
Hi Felix,

the pbf file really contains only nodes.
Can you attach the splitter log file and the asia.poly, please?

Gerd


Von: mkgmap-dev 
mailto:mkgmap-dev-boun...@lists.mkgmap.org.uk>>
 im Auftrag von Felix Hartmann 
mailto:extremecar...@gmail.com>>
Gesendet: Freitag, 28. Mai 2021 11:18
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] Splitter only writes points, not ways on huge file?

Hi Gerd,
sorry - fixed. I attached the file. I could also put the whole 60GB file for 
analysis. But do not know if you could download such a huge file or analyse it 
better.
https://openmtbmap.org/75400407.osm.pbf

Felix

On Fri, 28 May 2021 at 09:03, Gerd Petermann 
mailto:gpetermann_muenc...@hotmail.com><mailto:gpetermann_muenc...@hotmail.com<mailto:gpetermann_muenc...@hotmail.com>>>
 wrote:
Hi Felix,

the link to the pbf file seems to be wrong?  I get an 404

Gerd


Von: mkgmap-dev 
mailto:mkgmap-dev-boun...@lists.mkgmap.org.uk><mailto:mkgmap-dev-boun...@lists.mkgmap.org.uk<mailto:mkgmap-dev-boun...@lists.mkgmap.org.uk>>>
 im Auftrag von Felix Hartmann 
mailto:extremecar...@gmail.com><mailto:extremecar...@gmail.com<mailto:extremecar...@gmail.com>>>
Gesendet: Freitag, 28. Mai 2021 07:56
An: Development list for mkgmap
Betreff: [mkgmap-dev] Splitter only writes points, not ways on huge file?

I used phyghtmap to create a huge file (60GB) of 10m equidistance osm.pbf 
contourlines data.

phyghtmap --jobs=2 --polygon=/home/contourlines/bounds/asia.poly --step=10  
--no-zero-contour --void-range-max=-420 --output-prefix="Asia10m" 
--line-cat=500,100--start-node-id=1000 --start-way-id=1000 
--source=view1,srtm1,view3,srtm3 --max-nodes-per-way=230 --max-nodes-per-tile=0 
--pbf

I am not fully sure that this pbf file is correct, but I created it exactly the 
same way as smaller countries. Maybe its also phyghtmap going wrong. But I 
doubt it with phyghtmaps incremental approach of just

Re: [mkgmap-dev] Splitter only writes points, not ways on huge file?

2021-05-28 Thread Felix Hartmann

running now again with resolution 13, I had tried if changing resolution to
12 would solve the problem - but is does not.

On Fri, 28 May 2021 at 13:11, Felix Hartmann 
wrote:

> Asia.poly is attached - I did not save the splitter.log - doing this right
> now (but will take some hours - splitting this huge file takes a lot of
> time)
>
> The log starts with:
> Splitter version 597 compiled 2020-01-30T14:10:47+
> boundary-tags=use-exclude-list
> cache=
> description=
> geonames-file=
> handle-element-version=remove
> ignore-osm-bounds=false
> keep-complete=true
> mapid=7540
> max-areas=2048
> max-nodes=1200
> max-threads=7
> mixed=false
> no-trim=true
> num-tiles=
> output=pbf
> output-dir=
> overlap=auto
> polygon-desc-file=
> polygon-file=
> precomp-sea=
> problem-file=
> problem-report=
> resolution=13
> route-rel-values=
> search-limit=20
> split-file=
> status-freq=120
> stop-after=dist
> wanted-admin-level=5
> write-kml=
> Elapsed time: 0s   Memory: Current 1012MB (7MB used, 1005MB free) Max
> 25000MB
> Time started: Fri May 28 12:06:59 CEST 2021
> Map is being split for resolution 13:
>  - area boundaries are aligned to 0x800 map units (0.0439453125 degrees)
>  - areas are multiples of 0x800 map units wide and high
> Processing asia10m_.osm.pbf
> Bounding box -179.03 -13.0116501 179.03
> 84.52666
> 10,000,000 nodes parsed... id=1999
> 20,000,000 nodes parsed... id=2999
> 30,000,000 nodes parsed... id=3999
> 40,000,000 nodes parsed... id=4999
> 50,000,000 nodes parsed... id=5999
> 60,000,000 nodes parsed... id=6999
> 70,000,000 nodes parsed... id=7999
> 80,000,000 nodes parsed... id=8999
> 90,000,000 nodes parsed... id=
> 100,000,000 nodes parsed... id=10999
>
> On Fri, 28 May 2021 at 12:25, Gerd Petermann <
> gpetermann_muenc...@hotmail.com> wrote:
>
>> Hi Felix,
>>
>> the pbf file really contains only nodes.
>> Can you attach the splitter log file and the asia.poly, please?
>>
>> Gerd
>>
>> 
>> Von: mkgmap-dev  im Auftrag von
>> Felix Hartmann 
>> Gesendet: Freitag, 28. Mai 2021 11:18
>> An: Development list for mkgmap
>> Betreff: Re: [mkgmap-dev] Splitter only writes points, not ways on huge
>> file?
>>
>> Hi Gerd,
>> sorry - fixed. I attached the file. I could also put the whole 60GB file
>> for analysis. But do not know if you could download such a huge file or
>> analyse it better.
>> https://openmtbmap.org/75400407.osm.pbf
>>
>> Felix
>>
>> On Fri, 28 May 2021 at 09:03, Gerd Petermann <
>> gpetermann_muenc...@hotmail.com<mailto:gpetermann_muenc...@hotmail.com>>
>> wrote:
>> Hi Felix,
>>
>> the link to the pbf file seems to be wrong?  I get an 404
>>
>> Gerd
>>
>> 
>> Von: mkgmap-dev > mkgmap-dev-boun...@lists.mkgmap.org.uk>> im Auftrag von Felix Hartmann <
>> extremecar...@gmail.com<mailto:extremecar...@gmail.com>>
>> Gesendet: Freitag, 28. Mai 2021 07:56
>> An: Development list for mkgmap
>> Betreff: [mkgmap-dev] Splitter only writes points, not ways on huge file?
>>
>> I used phyghtmap to create a huge file (60GB) of 10m equidistance osm.pbf
>> contourlines data.
>>
>> phyghtmap --jobs=2 --polygon=/home/contourlines/bounds/asia.poly
>> --step=10  --no-zero-contour --void-range-max=-420
>> --output-prefix="Asia10m" --line-cat=500,100--start-node-id=1000
>> --start-way-id=1000 --source=view1,srtm1,view3,srtm3
>> --max-nodes-per-way=230 --max-nodes-per-tile=0 --pbf
>>
>> I am not fully sure that this pbf file is correct, but I created it
>> exactly the same way as smaller countries. Maybe its also phyghtmap going
>> wrong. But I doubt it with phyghtmaps incremental approach of just
>> constantly writing out data.
>>
>> When splitting it with splitter - and opening up one of the osm.pbf files
>> in JOSM - I noticed it contains only the contourline nodes, no lines.
>> I think the Asia continent file spans 360 degrees - so maybe that causes
>> problems with splitter?
>>
>>
>>
>> I have uploaded a splitted file here:
>> https://openmtbmap.org/75400407.osm.pbf
>> If you load it in Josm, you will notice it is alll the points but no
>> lines...
>>
>> --
>> Felix Hartman - Openmtbmap.org & VeloMap.org
>>
>> ___
>> mkgmap-dev mailing list
>> mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk>
>> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>
>>
>> --
>> Felix Hartman - Openmtbmap.org & VeloMap.org
>>
>> ___
>> mkgmap-dev mailing list
>> mkgmap-dev@lists.mkgmap.org.uk
>> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>
>
>
> --
> Felix Hartman - Openmtbmap.org & VeloMap.org
>
>

-- 
Felix Hartman - Openmtbmap.org & VeloMap.org
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Splitter only writes points, not ways on huge file?

2021-05-28 Thread Felix Hartmann

Asia.poly is attached - I did not save the splitter.log - doing this right
now (but will take some hours - splitting this huge file takes a lot of
time)

The log starts with:
Splitter version 597 compiled 2020-01-30T14:10:47+
boundary-tags=use-exclude-list
cache=
description=
geonames-file=
handle-element-version=remove
ignore-osm-bounds=false
keep-complete=true
mapid=7540
max-areas=2048
max-nodes=1200
max-threads=7
mixed=false
no-trim=true
num-tiles=
output=pbf
output-dir=
overlap=auto
polygon-desc-file=
polygon-file=
precomp-sea=
problem-file=
problem-report=
resolution=13
route-rel-values=
search-limit=20
split-file=
status-freq=120
stop-after=dist
wanted-admin-level=5
write-kml=
Elapsed time: 0s   Memory: Current 1012MB (7MB used, 1005MB free) Max
25000MB
Time started: Fri May 28 12:06:59 CEST 2021
Map is being split for resolution 13:
 - area boundaries are aligned to 0x800 map units (0.0439453125 degrees)
 - areas are multiples of 0x800 map units wide and high
Processing asia10m_.osm.pbf
Bounding box -179.03 -13.0116501 179.03
84.52666
10,000,000 nodes parsed... id=1999
20,000,000 nodes parsed... id=2999
30,000,000 nodes parsed... id=3999
40,000,000 nodes parsed... id=4999
50,000,000 nodes parsed... id=5999
60,000,000 nodes parsed... id=6999
70,000,000 nodes parsed... id=7999
80,000,000 nodes parsed... id=8999
90,000,000 nodes parsed... id=
100,000,000 nodes parsed... id=10999

On Fri, 28 May 2021 at 12:25, Gerd Petermann <
gpetermann_muenc...@hotmail.com> wrote:

> Hi Felix,
>
> the pbf file really contains only nodes.
> Can you attach the splitter log file and the asia.poly, please?
>
> Gerd
>
> 
> Von: mkgmap-dev  im Auftrag von
> Felix Hartmann 
> Gesendet: Freitag, 28. Mai 2021 11:18
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Splitter only writes points, not ways on huge
> file?
>
> Hi Gerd,
> sorry - fixed. I attached the file. I could also put the whole 60GB file
> for analysis. But do not know if you could download such a huge file or
> analyse it better.
> https://openmtbmap.org/75400407.osm.pbf
>
> Felix
>
> On Fri, 28 May 2021 at 09:03, Gerd Petermann <
> gpetermann_muenc...@hotmail.com<mailto:gpetermann_muenc...@hotmail.com>>
> wrote:
> Hi Felix,
>
> the link to the pbf file seems to be wrong?  I get an 404
>
> Gerd
>
> 
> Von: mkgmap-dev  mkgmap-dev-boun...@lists.mkgmap.org.uk>> im Auftrag von Felix Hartmann <
> extremecar...@gmail.com<mailto:extremecar...@gmail.com>>
> Gesendet: Freitag, 28. Mai 2021 07:56
> An: Development list for mkgmap
> Betreff: [mkgmap-dev] Splitter only writes points, not ways on huge file?
>
> I used phyghtmap to create a huge file (60GB) of 10m equidistance osm.pbf
> contourlines data.
>
> phyghtmap --jobs=2 --polygon=/home/contourlines/bounds/asia.poly
> --step=10  --no-zero-contour --void-range-max=-420
> --output-prefix="Asia10m" --line-cat=500,100--start-node-id=1000
> --start-way-id=1000 --source=view1,srtm1,view3,srtm3
> --max-nodes-per-way=230 --max-nodes-per-tile=0 --pbf
>
> I am not fully sure that this pbf file is correct, but I created it
> exactly the same way as smaller countries. Maybe its also phyghtmap going
> wrong. But I doubt it with phyghtmaps incremental approach of just
> constantly writing out data.
>
> When splitting it with splitter - and opening up one of the osm.pbf files
> in JOSM - I noticed it contains only the contourline nodes, no lines.
> I think the Asia continent file spans 360 degrees - so maybe that causes
> problems with splitter?
>
>
>
> I have uploaded a splitted file here:
> https://openmtbmap.org/75400407.osm.pbf
> If you load it in Josm, you will notice it is alll the points but no
> lines...
>
> --
> Felix Hartman - Openmtbmap.org & VeloMap.org
>
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk>
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
>
> --
> Felix Hartman - Openmtbmap.org & VeloMap.org
>
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>


-- 
Felix Hartman - Openmtbmap.org & VeloMap.org


asia.poly
Description: Binary data
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Splitter only writes points, not ways on huge file?

2021-05-28 Thread Gerd Petermann

Hi Felix,

the pbf file really contains only nodes.
Can you attach the splitter log file and the asia.poly, please?

Gerd


Von: mkgmap-dev  im Auftrag von Felix 
Hartmann 
Gesendet: Freitag, 28. Mai 2021 11:18
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] Splitter only writes points, not ways on huge file?

Hi Gerd,
sorry - fixed. I attached the file. I could also put the whole 60GB file for 
analysis. But do not know if you could download such a huge file or analyse it 
better.
https://openmtbmap.org/75400407.osm.pbf

Felix

On Fri, 28 May 2021 at 09:03, Gerd Petermann 
mailto:gpetermann_muenc...@hotmail.com>> wrote:
Hi Felix,

the link to the pbf file seems to be wrong?  I get an 404

Gerd


Von: mkgmap-dev 
mailto:mkgmap-dev-boun...@lists.mkgmap.org.uk>>
 im Auftrag von Felix Hartmann 
mailto:extremecar...@gmail.com>>
Gesendet: Freitag, 28. Mai 2021 07:56
An: Development list for mkgmap
Betreff: [mkgmap-dev] Splitter only writes points, not ways on huge file?

I used phyghtmap to create a huge file (60GB) of 10m equidistance osm.pbf 
contourlines data.

phyghtmap --jobs=2 --polygon=/home/contourlines/bounds/asia.poly --step=10  
--no-zero-contour --void-range-max=-420 --output-prefix="Asia10m" 
--line-cat=500,100--start-node-id=1000 --start-way-id=1000 
--source=view1,srtm1,view3,srtm3 --max-nodes-per-way=230 --max-nodes-per-tile=0 
--pbf

I am not fully sure that this pbf file is correct, but I created it exactly the 
same way as smaller countries. Maybe its also phyghtmap going wrong. But I 
doubt it with phyghtmaps incremental approach of just constantly writing out 
data.

When splitting it with splitter - and opening up one of the osm.pbf files in 
JOSM - I noticed it contains only the contourline nodes, no lines.
I think the Asia continent file spans 360 degrees - so maybe that causes 
problems with splitter?



I have uploaded a splitted file here: https://openmtbmap.org/75400407.osm.pbf
If you load it in Josm, you will notice it is alll the points but no lines...

--
Felix Hartman - Openmtbmap.org & VeloMap.org

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk<mailto:mkgmap-dev@lists.mkgmap.org.uk>
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


--
Felix Hartman - Openmtbmap.org & VeloMap.org

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Splitter only writes points, not ways on huge file?

2021-05-28 Thread Felix Hartmann

Hi Gerd,
sorry - fixed. I attached the file. I could also put the whole 60GB file
for analysis. But do not know if you could download such a huge file or
analyse it better.
https://openmtbmap.org/75400407.osm.pbf

Felix

On Fri, 28 May 2021 at 09:03, Gerd Petermann <
gpetermann_muenc...@hotmail.com> wrote:

> Hi Felix,
>
> the link to the pbf file seems to be wrong?  I get an 404
>
> Gerd
>
> 
> Von: mkgmap-dev  im Auftrag von
> Felix Hartmann 
> Gesendet: Freitag, 28. Mai 2021 07:56
> An: Development list for mkgmap
> Betreff: [mkgmap-dev] Splitter only writes points, not ways on huge file?
>
> I used phyghtmap to create a huge file (60GB) of 10m equidistance osm.pbf
> contourlines data.
>
> phyghtmap --jobs=2 --polygon=/home/contourlines/bounds/asia.poly
> --step=10  --no-zero-contour --void-range-max=-420
> --output-prefix="Asia10m" --line-cat=500,100--start-node-id=1000
> --start-way-id=1000 --source=view1,srtm1,view3,srtm3
> --max-nodes-per-way=230 --max-nodes-per-tile=0 --pbf
>
> I am not fully sure that this pbf file is correct, but I created it
> exactly the same way as smaller countries. Maybe its also phyghtmap going
> wrong. But I doubt it with phyghtmaps incremental approach of just
> constantly writing out data.
>
> When splitting it with splitter - and opening up one of the osm.pbf files
> in JOSM - I noticed it contains only the contourline nodes, no lines.
> I think the Asia continent file spans 360 degrees - so maybe that causes
> problems with splitter?
>
>
>
> I have uploaded a splitted file here:
> https://openmtbmap.org/75400407.osm.pbf
> If you load it in Josm, you will notice it is alll the points but no
> lines...
>
> --
> Felix Hartman - Openmtbmap.org & VeloMap.org
>
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>


-- 
Felix Hartman - Openmtbmap.org & VeloMap.org
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Splitter only writes points, not ways on huge file?

2021-05-28 Thread Gerd Petermann

Hi Felix,

the link to the pbf file seems to be wrong?  I get an 404

Gerd


Von: mkgmap-dev  im Auftrag von Felix 
Hartmann 
Gesendet: Freitag, 28. Mai 2021 07:56
An: Development list for mkgmap
Betreff: [mkgmap-dev] Splitter only writes points, not ways on huge file?

I used phyghtmap to create a huge file (60GB) of 10m equidistance osm.pbf 
contourlines data.

phyghtmap --jobs=2 --polygon=/home/contourlines/bounds/asia.poly --step=10  
--no-zero-contour --void-range-max=-420 --output-prefix="Asia10m" 
--line-cat=500,100--start-node-id=1000 --start-way-id=1000 
--source=view1,srtm1,view3,srtm3 --max-nodes-per-way=230 --max-nodes-per-tile=0 
--pbf

I am not fully sure that this pbf file is correct, but I created it exactly the 
same way as smaller countries. Maybe its also phyghtmap going wrong. But I 
doubt it with phyghtmaps incremental approach of just constantly writing out 
data.

When splitting it with splitter - and opening up one of the osm.pbf files in 
JOSM - I noticed it contains only the contourline nodes, no lines.
I think the Asia continent file spans 360 degrees - so maybe that causes 
problems with splitter?



I have uploaded a splitted file here: https://openmtbmap.org/75400407.osm.pbf
If you load it in Josm, you will notice it is alll the points but no lines...

--
Felix Hartman - Openmtbmap.org & VeloMap.org

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Splitter download broken

2021-02-02 Thread Carlos Dávila


You can update it with the attached (minor) patch

El 1/2/21 a las 22:24, Steve Ratcliffe escribió:

Hi


Download for splitter r597 (and all older versions) is broken.
Please check.


Thanks for reporting, I have rebuild the latest release.

It was removed because it is over a year old!

Best wishes
Steve
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Index: src/uk/me/parabola/splitter/ProblemLists.java
===
--- src/uk/me/parabola/splitter/ProblemLists.java	(revisión: 592)
+++ src/uk/me/parabola/splitter/ProblemLists.java	(copia de trabajo)
@@ -53,7 +53,7 @@
 		long startProblemListGenerator = System.currentTimeMillis();
 		ArrayList distinctAreas = getNonOverlappingAreas(realAreas);
 		if (distinctAreas.size() > realAreas.size()) {
-			System.err.println("Waring: The areas given in --split-file are overlapping.");
+			System.err.println("Warning: The areas given in --split-file are overlapping.");
 			Set overlappingTiles = new TreeSet<>();
 			for (int i = 0; i < realAreas.size(); i++) {
 Area a1 = realAreas.get(i);
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Splitter download broken

2021-02-01 Thread Steve Ratcliffe


Hi


Download for splitter r597 (and all older versions) is broken.
Please check.


Thanks for reporting, I have rebuild the latest release.

It was removed because it is over a year old!

Best wishes
Steve
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter performance with "osm.pbf" compared to "o5m"

2020-12-22 Thread Gerd Petermann

Hi Franco,

just noticed that I used o5m for output in both tests, so my numbers for pbf 
are probably a bit to small. Anyway, I think this just proves that it was a 
good idea to implement the o5m support ;)

Gerd


Von: mkgmap-dev  im Auftrag von Franco 
Bez 
Gesendet: Dienstag, 22. Dezember 2020 20:41
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: Re: [mkgmap-dev] splitter performance with "osm.pbf" compared to "o5m"

Hi Gerd,

thanks for your tests.

On your machine it's still more than double, on my machine almost
tripple the time for pbf compared to o5m.

Thanks for the hint with "ways missing nodes", I'll check this.

Ciao,

Franco

Am 22.12.20 um 10:51 schrieb Gerd Petermann:
> Hi Franco,
>
> the differences in my numbers are not that drastical. I tried with Germany  
> from geofabrik and got 5 minutes 21 seconds.
> for the o5m format and 12 min for pbf. Maybe the pbf reader could be improved 
> to use multiple threads, but that's not my strength.
>
> BTW: Your logs show many warnings like "Sorry, way 4216363 is missing 29 
> node(s)." . Maybe you should check your tool chain if these ways are relavant 
> for you.
>
> Gerd
>
> 
> Von: mkgmap-dev  im Auftrag von 
> Franco Bez 
> Gesendet: Samstag, 19. Dezember 2020 13:37
> An: mkgmap-dev@lists.mkgmap.org.uk
> Betreff: Re: [mkgmap-dev] splitter performance with "osm.pbf" compared to 
> "o5m"
>
> |Hi Gerd,|
>
> |here are the two log files.
> |
>
> |http://files.mkgmap.org.uk/download/491/logs.tgz|
>
> |Ciao,|
>
> |Franco
> |
>
> Am 19.12.20 um 12:09 schrieb Gerd Petermann:
>> Hi Franco,
>>
>> OK, I guess 2048 areas should be enough and memory is for sure enough. The 
>> major reason that I added o5m support to splitter was that this format was 
>> much faster to read, esp. when the file is read multiple times some passes 
>> only need relations or only ways. The pbf format didn't support this as 
>> well, but I thought that I also improved handling of that format.
>> If you like you can zip the two logs and upload them to 
>> http://files.mkgmap.org.uk .
>>
>> I'll do some tests on my own machine again.
>>
>> Gerd
>>
>> 
>> Von: mkgmap-dev  im Auftrag von 
>> Franco Bez 
>> Gesendet: Samstag, 19. Dezember 2020 11:45
>> An: mkgmap-dev@lists.mkgmap.org.uk
>> Betreff: Re: [mkgmap-dev] splitter performance with "osm.pbf" compared to
>>"o5m"
>>
>> Hi Gerd,
>>
>> thanks for the fast reply.
>>
>> I tried "java -ea -Xmx16G -Xms16G" but this doen't make any difference
>> on my machine, AMD Ryzen 7 3700X 32GB Ram 2 SSDs 1TB each
>>
>> see the log extracts below.
>>
>> Ciao,
>>
>>   Franco
>>
>

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter performance with "osm.pbf" compared to "o5m"

2020-12-22 Thread Franco Bez

Hi Gerd,

thanks for your tests.

On your machine it's still more than double, on my machine almost
tripple the time for pbf compared to o5m.

Thanks for the hint with "ways missing nodes", I'll check this.

Ciao,

Franco

Am 22.12.20 um 10:51 schrieb Gerd Petermann:
> Hi Franco,
>
> the differences in my numbers are not that drastical. I tried with Germany  
> from geofabrik and got 5 minutes 21 seconds.
> for the o5m format and 12 min for pbf. Maybe the pbf reader could be improved 
> to use multiple threads, but that's not my strength.
>
> BTW: Your logs show many warnings like "Sorry, way 4216363 is missing 29 
> node(s)." . Maybe you should check your tool chain if these ways are relavant 
> for you.
>
> Gerd
>
> 
> Von: mkgmap-dev  im Auftrag von 
> Franco Bez 
> Gesendet: Samstag, 19. Dezember 2020 13:37
> An: mkgmap-dev@lists.mkgmap.org.uk
> Betreff: Re: [mkgmap-dev] splitter performance with "osm.pbf" compared to 
> "o5m"
>
> |Hi Gerd,|
>
> |here are the two log files.
> |
>
> |http://files.mkgmap.org.uk/download/491/logs.tgz|
>
> |Ciao,|
>
> |Franco
> |
>
> Am 19.12.20 um 12:09 schrieb Gerd Petermann:
>> Hi Franco,
>>
>> OK, I guess 2048 areas should be enough and memory is for sure enough. The 
>> major reason that I added o5m support to splitter was that this format was 
>> much faster to read, esp. when the file is read multiple times some passes 
>> only need relations or only ways. The pbf format didn't support this as 
>> well, but I thought that I also improved handling of that format.
>> If you like you can zip the two logs and upload them to 
>> http://files.mkgmap.org.uk .
>>
>> I'll do some tests on my own machine again.
>>
>> Gerd
>>
>> 
>> Von: mkgmap-dev  im Auftrag von 
>> Franco Bez 
>> Gesendet: Samstag, 19. Dezember 2020 11:45
>> An: mkgmap-dev@lists.mkgmap.org.uk
>> Betreff: Re: [mkgmap-dev] splitter performance with "osm.pbf" compared to
>>"o5m"
>>
>> Hi Gerd,
>>
>> thanks for the fast reply.
>>
>> I tried "java -ea -Xmx16G -Xms16G" but this doen't make any difference
>> on my machine, AMD Ryzen 7 3700X 32GB Ram 2 SSDs 1TB each
>>
>> see the log extracts below.
>>
>> Ciao,
>>
>>   Franco
>>
>

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter performance with "osm.pbf" compared to "o5m"

2020-12-22 Thread Gerd Petermann

Hi Franco,

the differences in my numbers are not that drastical. I tried with Germany  
from geofabrik and got 5 minutes 21 seconds.
for the o5m format and 12 min for pbf. Maybe the pbf reader could be improved 
to use multiple threads, but that's not my strength.

BTW: Your logs show many warnings like "Sorry, way 4216363 is missing 29 
node(s)." . Maybe you should check your tool chain if these ways are relavant 
for you.

Gerd


Von: mkgmap-dev  im Auftrag von Franco 
Bez 
Gesendet: Samstag, 19. Dezember 2020 13:37
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: Re: [mkgmap-dev] splitter performance with "osm.pbf" compared to "o5m"

|Hi Gerd,|

|here are the two log files.
|

|http://files.mkgmap.org.uk/download/491/logs.tgz|

|Ciao,|

|Franco
|

Am 19.12.20 um 12:09 schrieb Gerd Petermann:
> Hi Franco,
>
> OK, I guess 2048 areas should be enough and memory is for sure enough. The 
> major reason that I added o5m support to splitter was that this format was 
> much faster to read, esp. when the file is read multiple times some passes 
> only need relations or only ways. The pbf format didn't support this as well, 
> but I thought that I also improved handling of that format.
> If you like you can zip the two logs and upload them to 
> http://files.mkgmap.org.uk .
>
> I'll do some tests on my own machine again.
>
> Gerd
>
> 
> Von: mkgmap-dev  im Auftrag von 
> Franco Bez 
> Gesendet: Samstag, 19. Dezember 2020 11:45
> An: mkgmap-dev@lists.mkgmap.org.uk
> Betreff: Re: [mkgmap-dev] splitter performance with "osm.pbf" compared to 
>   "o5m"
>
> Hi Gerd,
>
> thanks for the fast reply.
>
> I tried "java -ea -Xmx16G -Xms16G" but this doen't make any difference
> on my machine, AMD Ryzen 7 3700X 32GB Ram 2 SSDs 1TB each
>
> see the log extracts below.
>
> Ciao,
>
>   Franco
>

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter performance with "osm.pbf" compared to "o5m"

2020-12-19 Thread Franco Bez

|Hi Gerd,|

|here are the two log files.
|

|http://files.mkgmap.org.uk/download/491/logs.tgz|

|Ciao,|

|Franco
|

Am 19.12.20 um 12:09 schrieb Gerd Petermann:
> Hi Franco,
>
> OK, I guess 2048 areas should be enough and memory is for sure enough. The 
> major reason that I added o5m support to splitter was that this format was 
> much faster to read, esp. when the file is read multiple times some passes 
> only need relations or only ways. The pbf format didn't support this as well, 
> but I thought that I also improved handling of that format.
> If you like you can zip the two logs and upload them to 
> http://files.mkgmap.org.uk .
>
> I'll do some tests on my own machine again.
>
> Gerd
>
> 
> Von: mkgmap-dev  im Auftrag von 
> Franco Bez 
> Gesendet: Samstag, 19. Dezember 2020 11:45
> An: mkgmap-dev@lists.mkgmap.org.uk
> Betreff: Re: [mkgmap-dev] splitter performance with "osm.pbf" compared to 
>   "o5m"
>
> Hi Gerd,
>
> thanks for the fast reply.
>
> I tried "java -ea -Xmx16G -Xms16G" but this doen't make any difference
> on my machine, AMD Ryzen 7 3700X 32GB Ram 2 SSDs 1TB each
>
> see the log extracts below.
>
> Ciao,
>
>   Franco
>



signature.asc
Description: OpenPGP digital signature
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter performance with "osm.pbf" compared to "o5m"

2020-12-19 Thread Gerd Petermann

Hi Franco,

OK, I guess 2048 areas should be enough and memory is for sure enough. The 
major reason that I added o5m support to splitter was that this format was much 
faster to read, esp. when the file is read multiple times some passes only need 
relations or only ways. The pbf format didn't support this as well, but I 
thought that I also improved handling of that format.
If you like you can zip the two logs and upload them to 
http://files.mkgmap.org.uk .

I'll do some tests on my own machine again.

Gerd


Von: mkgmap-dev  im Auftrag von Franco 
Bez 
Gesendet: Samstag, 19. Dezember 2020 11:45
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: Re: [mkgmap-dev] splitter performance with "osm.pbf" compared to   
"o5m"

Hi Gerd,

thanks for the fast reply.

I tried "java -ea -Xmx16G -Xms16G" but this doen't make any difference
on my machine, AMD Ryzen 7 3700X 32GB Ram 2 SSDs 1TB each

see the log extracts below.

Ciao,

  Franco


O5M
Splitter version 597 compiled 2020-01-30T14:10:47+
boundary-tags=use-exclude-list
cache=
description=
geonames-file=/home/franco/map_build/cities15000.zip
handle-element-version=remove
ignore-osm-bounds=false
keep-complete=true
mapid=6501
max-areas=2048
max-nodes=120
max-threads=16 (auto)
mixed=false
no-trim=false
num-tiles=
output=o5m
output-dir=
overlap=0
polygon-desc-file=
polygon-file=
precomp-sea=/home/franco/map_build/precomp/sea-latest.zip
problem-file=
problem-report=
resolution=13
route-rel-values=
search-limit=20
split-file=
status-freq=120
stop-after=dist
wanted-admin-level=5
write-kml=dach.kml
Elapsed time: 0s   Memory: Current 16384MB (19MB used, 16365MB free) Max
16384MB
Time started: Sat Dec 19 10:10:21 CET 2020
Time finished: Sat Dec 19 10:15:44 CET 2020
Total time taken: 5 minutes 22 seconds

---PBF--
Splitter version 597 compiled 2020-01-30T14:10:47+
boundary-tags=use-exclude-list
geonames-file=/home/franco/map_build/cities15000.zip
handle-element-version=remove
ignore-osm-bounds=false
keep-complete=true
mapid=6501
max-areas=2048
max-nodes=120
max-threads=16 (auto)
mixed=false
no-trim=false
num-tiles=
output=pbf
output-dir=
overlap=0
polygon-desc-file=
polygon-file=
precomp-sea=/home/franco/map_build/precomp/sea-latest.zip
problem-file=
problem-report=
resolution=13
route-rel-values=
search-limit=20
split-file=
status-freq=120
stop-after=dist
wanted-admin-level=5
write-kml=dach.kml
Elapsed time: 0s   Memory: Current 16384MB (19MB used, 16365MB free) Max
16384MB
Time started: Sat Dec 19 10:33:28 CET 2020
Time finished: Sat Dec 19 10:51:12 CET 2020
Total time taken: 17 minutes 43 seconds



___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter performance with "osm.pbf" compared to "o5m"

2020-12-19 Thread Franco Bez

Hi Gerd,

thanks for the fast reply.

I tried "java -ea -Xmx16G -Xms16G" but this doen't make any difference
on my machine, AMD Ryzen 7 3700X 32GB Ram 2 SSDs 1TB each

see the log extracts below.

Ciao,

  Franco


O5M
Splitter version 597 compiled 2020-01-30T14:10:47+
boundary-tags=use-exclude-list
cache=
description=
geonames-file=/home/franco/map_build/cities15000.zip
handle-element-version=remove
ignore-osm-bounds=false
keep-complete=true
mapid=6501
max-areas=2048
max-nodes=120
max-threads=16 (auto)
mixed=false
no-trim=false
num-tiles=
output=o5m
output-dir=
overlap=0
polygon-desc-file=
polygon-file=
precomp-sea=/home/franco/map_build/precomp/sea-latest.zip
problem-file=
problem-report=
resolution=13
route-rel-values=
search-limit=20
split-file=
status-freq=120
stop-after=dist
wanted-admin-level=5
write-kml=dach.kml
Elapsed time: 0s   Memory: Current 16384MB (19MB used, 16365MB free) Max
16384MB
Time started: Sat Dec 19 10:10:21 CET 2020
Time finished: Sat Dec 19 10:15:44 CET 2020
Total time taken: 5 minutes 22 seconds

---PBF--
Splitter version 597 compiled 2020-01-30T14:10:47+
boundary-tags=use-exclude-list
geonames-file=/home/franco/map_build/cities15000.zip
handle-element-version=remove
ignore-osm-bounds=false
keep-complete=true
mapid=6501
max-areas=2048
max-nodes=120
max-threads=16 (auto)
mixed=false
no-trim=false
num-tiles=
output=pbf
output-dir=
overlap=0
polygon-desc-file=
polygon-file=
precomp-sea=/home/franco/map_build/precomp/sea-latest.zip
problem-file=
problem-report=
resolution=13
route-rel-values=
search-limit=20
split-file=
status-freq=120
stop-after=dist
wanted-admin-level=5
write-kml=dach.kml
Elapsed time: 0s   Memory: Current 16384MB (19MB used, 16365MB free) Max
16384MB
Time started: Sat Dec 19 10:33:28 CET 2020
Time finished: Sat Dec 19 10:51:12 CET 2020
Total time taken: 17 minutes 43 seconds





signature.asc
Description: OpenPGP digital signature
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter performance with "osm.pbf" compared to "o5m"

2020-12-19 Thread Gerd Petermann

Hi Franco,

first see 
https://wiki.openstreetmap.org/wiki/Mkgmap/help/splitter#Tuning_for_best_performance.
I think the difference should not be that big when splitter has enough memory, 
so maybe you should add the -Xmx JRE parameter.

Gerd



Von: mkgmap-dev  im Auftrag von Franco 
Bez 
Gesendet: Samstag, 19. Dezember 2020 09:40
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: [mkgmap-dev] splitter performance with "osm.pbf" compared to "o5m"

Hi all,

I noticed that splitter has a major performance lag on osm.pbf files
compared to o5m.

splitting off my DACH region in pbf  format takes 15min 45sec, while
splitting the same region in o5m format is finished after 5min 17sec.

So it takes almost 3 times as long splitting pbf data.

mkgmap on the other hand has almost no difference at all when building
the DACH map from the pbf-tiles, compared to o5m-tiles.

mkgmap with pbf takes 12min 28sec, with o5m 12min 09sec.

here are my command lines:

java -ea  -XX:+AggressiveHeap  -jar splitter-r597/splitter.jar  >
splitter.log  --geonames-file=cities15000.zip --mapid=6501
--output=pbf  --precomp-sea=sea-latest.zip --write-kml=dach.kml
--keep-complete  --overlap=0  --max-nodes=120 dach.osm.pbf

java -ea  -XX:+AggressiveHeap  -jar splitter-r597/splitter.jar  >
splitter.log  --geonames-file=cities15000.zip --mapid=6501
--output=o5m  --precomp-sea=sea-latest.zip --write-kml=dach.kml
--keep-complete  --overlap=0  --max-nodes=120 dach.o5m


Any ideas why pbf splitting is so slow compared to o5m splitting ?


Ciao,

  Franco

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter questions

2020-10-22 Thread Ticker Berkin

Hi Karl

If you are developing these for a device that supports multiple .img
files, the option
  --add-boundary-nodes-at-admin-boundaries=NUM
allows you to build individual countries (or groups), have them all
loaded on the device, select the appropriate maps and have routing
between them.

Having larger tiles can cause 2 problems that I know of:
1/ mkgmap runs out of memory
2/ some tile .img data structures use a variable number of bytes
depending on the number of items and stepping over a threshold can
cause a jump in the final .img size and, if I remember correctly, a
more significant jump in size of some of the global indexes (MDR
structures) if just 1 tile needs more bytes.
mkgmap generates a warning when it needs to use 3 bytes to reference
Cities, Zip codes, Exit facilities or their linked highways.

Ticker

On Thu, 2020-10-22 at 09:41 +, Gerd Petermann wrote:
> Hi Karl,
> 
> do you want to distribute the maps for others or do you need them
> only for yourself?
> In the latter case you may use a poly file to cut out the area you
> are interested in. I do this with my maps (which I don't share with
> others)
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag
> von 7770 <7...@foskan.eu>
> Gesendet: Donnerstag, 22. Oktober 2020 11:36
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] splitter questions
> 
> Hi Gerd.
> 
> I managed to run the splitter and it completed well with
> java -Xmx1800m  and setting the --max-nodes=160.
> 570 osm.pbf files were created.
> 
> For even larger maps
> --max-nodes= higher should use less memory, but will those tiles be
> slower on
> a GPS unit and how much larger value is reasonable?
> 
> 
> The main reason for creating such a large mapdata set is to make
> cross border
> routing to work.
> Is there some other way i could produce the maps?
> For example split and build map for each country separately (based on
> data
> from geofabrik) and then stitch the countries and routing data
> together, is
> this possible?
> 
> Regards
> Karl.
> 
> 
> On torsdag 22 oktober 2020 kl. 11:17:00 CEST Gerd Petermann wrote:
> > Hi Karl,
> > 
> > the main purpose of --keep-complete is to have complete
> > multipolygons in
> > each tile to avoid empty parts of large lakes or flooded areas near
> > large
> > lakes. There is also an effect on routing regarding turn
> > restrictions and
> > route relations.
> > 
> > Anyhow, if you really want to produce a very large map on such a
> > limited
> > system you probably run into other problems when calculating the
> > global
> > index in mkgmap.
> > 
> > Gerd
> > 
> > 
> > 
> > Von: mkgmap-dev  im Auftrag
> > von 7770
> > <7...@foskan.eu> Gesendet: Donnerstag, 22. Oktober 2020 10:59
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] splitter questions
> > 
> > Hi Gerd.
> > Yes, it is 64-bit, but i don't have a lot of memory.
> > --keep-complete=false, can it impact routing possibilities (routing
> > may not
> > work well) or is it mainly affecting how the map will look?
> > 
> > 
> > Regards
> > Karl
> > 
> > On torsdag 22 oktober 2020 kl. 10:49:23 CEST Gerd Petermann wrote:
> > > Hi Karl,
> > > 
> > > if you are running a 64 Bit OS make sure to install a 64 Bit java
> > > runtime.
> > > This allows to increase the -Xmx option. You probably need > 5G
> > > for this.
> > > With a 32 Bit OS you can only try to disable the keep-complete
> > > option.
> > > 
> > > Gerd
> > > 
> > > 
> > > Von: mkgmap-dev  im
> > > Auftrag von
> > > 7770 <7...@foskan.eu> Gesendet: Donnerstag, 22. Oktober 2020
> > > 10:45
> > > An: mkgmap-dev@lists.mkgmap.org.uk
> > > Betreff: [mkgmap-dev] splitter questions
> > > 
> > > Hi.
> > > 
> > > I am trying to split data for some 15 countries. First step is to
> > > combine
> > > the country data (with osmconvert) to one large o5m file and then
> > > try to
> > > split the combined file.
> > > The o5m file is about 14 GB in size.
> > > 
> > > When running splitter i give java -Xmx1600m
> > > options to splitter are: --max-nodes=150 --no-trim
> > > 
> > > Splitter calculates around 640 areas/tiles to be created.
> > > If i understand correctly, ways are parsed completely and then
> > > starts
> > &

Re: [mkgmap-dev] splitter questions

2020-10-22 Thread Gerd Petermann

Hi Karl,

do you want to distribute the maps for others or do you need them only for 
yourself?
In the latter case you may use a poly file to cut out the area you are 
interested in. I do this with my maps (which I don't share with others)

Gerd


Von: mkgmap-dev  im Auftrag von 7770 
<7...@foskan.eu>
Gesendet: Donnerstag, 22. Oktober 2020 11:36
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] splitter questions

Hi Gerd.

I managed to run the splitter and it completed well with
java -Xmx1800m  and setting the --max-nodes=160.
570 osm.pbf files were created.

For even larger maps
--max-nodes= higher should use less memory, but will those tiles be slower on
a GPS unit and how much larger value is reasonable?


The main reason for creating such a large mapdata set is to make cross border
routing to work.
Is there some other way i could produce the maps?
For example split and build map for each country separately (based on data
from geofabrik) and then stitch the countries and routing data together, is
this possible?

Regards
Karl.


On torsdag 22 oktober 2020 kl. 11:17:00 CEST Gerd Petermann wrote:
> Hi Karl,
>
> the main purpose of --keep-complete is to have complete multipolygons in
> each tile to avoid empty parts of large lakes or flooded areas near large
> lakes. There is also an effect on routing regarding turn restrictions and
> route relations.
>
> Anyhow, if you really want to produce a very large map on such a limited
> system you probably run into other problems when calculating the global
> index in mkgmap.
>
> Gerd
>
>
> 
> Von: mkgmap-dev  im Auftrag von 7770
> <7...@foskan.eu> Gesendet: Donnerstag, 22. Oktober 2020 10:59
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] splitter questions
>
> Hi Gerd.
> Yes, it is 64-bit, but i don't have a lot of memory.
> --keep-complete=false, can it impact routing possibilities (routing may not
> work well) or is it mainly affecting how the map will look?
>
>
> Regards
> Karl
>
> On torsdag 22 oktober 2020 kl. 10:49:23 CEST Gerd Petermann wrote:
> > Hi Karl,
> >
> > if you are running a 64 Bit OS make sure to install a 64 Bit java runtime.
> > This allows to increase the -Xmx option. You probably need > 5G for this.
> > With a 32 Bit OS you can only try to disable the keep-complete option.
> >
> > Gerd
> >
> > 
> > Von: mkgmap-dev  im Auftrag von
> > 7770 <7...@foskan.eu> Gesendet: Donnerstag, 22. Oktober 2020 10:45
> > An: mkgmap-dev@lists.mkgmap.org.uk
> > Betreff: [mkgmap-dev] splitter questions
> >
> > Hi.
> >
> > I am trying to split data for some 15 countries. First step is to combine
> > the country data (with osmconvert) to one large o5m file and then try to
> > split the combined file.
> > The o5m file is about 14 GB in size.
> >
> > When running splitter i give java -Xmx1600m
> > options to splitter are: --max-nodes=150 --no-trim
> >
> > Splitter calculates around 640 areas/tiles to be created.
> > If i understand correctly, ways are parsed completely and then starts
> > relations.
> >
> > After 100 000 relations i get:
> > * Full GC *
> > and soon thereafter:
> > Exception in thread "main" Exception in thread "worker-0"
> >
> >  java.lang.OutOfMemoryError: GC overhead limit exceeded... and many more
> >
> > lines.
> >
> > After this, one thread (out of two) dies but the other seems to hang doing
> > nothing... and i have to manually kill it later.
> >
> >
> >
> > Now, i can give java a little bit more memory to java, say up to 1800m.
> >
> > setting  --max-nodes= larger should use less memory says documentation.
> > With a larger max-nodes, will those tiles be faster or slower to display
> > on
> > a GPS unit?
> > What is a reasonable value, 10 000 000 nodes or 2 000 000?
> >
> >
> > Would --max-areas help my situation?
> > If i use --max-areas=350 (approx half of total expected tiles), based on
> > your experience would it be enough, or shall i better try with a value of
> > 220 (~1/3 of the total tiles)?
> >
> > Regards
> > Karl
> >
> >
> > ___
> > mkgmap-dev mailing list
> > mkgmap-dev@lists.mkgmap.org.uk
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > ___
> > mkgmap-dev mailing list
> > mkgmap-dev@lists.mkgmap.org.uk
> > htt

Re: [mkgmap-dev] splitter questions

2020-10-22 Thread 7770

Hi Gerd.

I managed to run the splitter and it completed well with
java -Xmx1800m  and setting the --max-nodes=160.
570 osm.pbf files were created.

For even larger maps
--max-nodes= higher should use less memory, but will those tiles be slower on 
a GPS unit and how much larger value is reasonable?


The main reason for creating such a large mapdata set is to make cross border 
routing to work.
Is there some other way i could produce the maps?
For example split and build map for each country separately (based on data 
from geofabrik) and then stitch the countries and routing data together, is 
this possible?

Regards
Karl.


On torsdag 22 oktober 2020 kl. 11:17:00 CEST Gerd Petermann wrote:
> Hi Karl,
> 
> the main purpose of --keep-complete is to have complete multipolygons in
> each tile to avoid empty parts of large lakes or flooded areas near large
> lakes. There is also an effect on routing regarding turn restrictions and
> route relations.
> 
> Anyhow, if you really want to produce a very large map on such a limited
> system you probably run into other problems when calculating the global
> index in mkgmap.
> 
> Gerd
> 
> 
> 
> Von: mkgmap-dev  im Auftrag von 7770
> <7...@foskan.eu> Gesendet: Donnerstag, 22. Oktober 2020 10:59
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] splitter questions
> 
> Hi Gerd.
> Yes, it is 64-bit, but i don't have a lot of memory.
> --keep-complete=false, can it impact routing possibilities (routing may not
> work well) or is it mainly affecting how the map will look?
> 
> 
> Regards
> Karl
> 
> On torsdag 22 oktober 2020 kl. 10:49:23 CEST Gerd Petermann wrote:
> > Hi Karl,
> > 
> > if you are running a 64 Bit OS make sure to install a 64 Bit java runtime.
> > This allows to increase the -Xmx option. You probably need > 5G for this.
> > With a 32 Bit OS you can only try to disable the keep-complete option.
> > 
> > Gerd
> > 
> > 
> > Von: mkgmap-dev  im Auftrag von
> > 7770 <7...@foskan.eu> Gesendet: Donnerstag, 22. Oktober 2020 10:45
> > An: mkgmap-dev@lists.mkgmap.org.uk
> > Betreff: [mkgmap-dev] splitter questions
> > 
> > Hi.
> > 
> > I am trying to split data for some 15 countries. First step is to combine
> > the country data (with osmconvert) to one large o5m file and then try to
> > split the combined file.
> > The o5m file is about 14 GB in size.
> > 
> > When running splitter i give java -Xmx1600m
> > options to splitter are: --max-nodes=150 --no-trim
> > 
> > Splitter calculates around 640 areas/tiles to be created.
> > If i understand correctly, ways are parsed completely and then starts
> > relations.
> > 
> > After 100 000 relations i get:
> > * Full GC *
> > and soon thereafter:
> > Exception in thread "main" Exception in thread "worker-0"
> > 
> >  java.lang.OutOfMemoryError: GC overhead limit exceeded... and many more
> > 
> > lines.
> > 
> > After this, one thread (out of two) dies but the other seems to hang doing
> > nothing... and i have to manually kill it later.
> > 
> > 
> > 
> > Now, i can give java a little bit more memory to java, say up to 1800m.
> > 
> > setting  --max-nodes= larger should use less memory says documentation.
> > With a larger max-nodes, will those tiles be faster or slower to display
> > on
> > a GPS unit?
> > What is a reasonable value, 10 000 000 nodes or 2 000 000?
> > 
> > 
> > Would --max-areas help my situation?
> > If i use --max-areas=350 (approx half of total expected tiles), based on
> > your experience would it be enough, or shall i better try with a value of
> > 220 (~1/3 of the total tiles)?
> > 
> > Regards
> > Karl
> > 
> > 
> > ___
> > mkgmap-dev mailing list
> > mkgmap-dev@lists.mkgmap.org.uk
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > ___
> > mkgmap-dev mailing list
> > mkgmap-dev@lists.mkgmap.org.uk
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> 
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev




___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter questions

2020-10-22 Thread Gerd Petermann

Hi Karl,

the main purpose of --keep-complete is to have complete multipolygons in each 
tile to avoid empty parts of large lakes or flooded areas near large lakes.
There is also an effect on routing regarding turn restrictions and route 
relations.

Anyhow, if you really want to produce a very large map on such a limited system 
you probably run into other problems when calculating the global index in 
mkgmap.

Gerd



Von: mkgmap-dev  im Auftrag von 7770 
<7...@foskan.eu>
Gesendet: Donnerstag, 22. Oktober 2020 10:59
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] splitter questions

Hi Gerd.
Yes, it is 64-bit, but i don't have a lot of memory.
--keep-complete=false, can it impact routing possibilities (routing may not
work well) or is it mainly affecting how the map will look?


Regards
Karl

On torsdag 22 oktober 2020 kl. 10:49:23 CEST Gerd Petermann wrote:
> Hi Karl,
>
> if you are running a 64 Bit OS make sure to install a 64 Bit java runtime.
> This allows to increase the -Xmx option. You probably need > 5G for this.
> With a 32 Bit OS you can only try to disable the keep-complete option.
>
> Gerd
>
> 
> Von: mkgmap-dev  im Auftrag von 7770
> <7...@foskan.eu> Gesendet: Donnerstag, 22. Oktober 2020 10:45
> An: mkgmap-dev@lists.mkgmap.org.uk
> Betreff: [mkgmap-dev] splitter questions
>
> Hi.
>
> I am trying to split data for some 15 countries. First step is to combine
> the country data (with osmconvert) to one large o5m file and then try to
> split the combined file.
> The o5m file is about 14 GB in size.
>
> When running splitter i give java -Xmx1600m
> options to splitter are: --max-nodes=150 --no-trim
>
> Splitter calculates around 640 areas/tiles to be created.
> If i understand correctly, ways are parsed completely and then starts
> relations.
>
> After 100 000 relations i get:
> * Full GC *
> and soon thereafter:
> Exception in thread "main" Exception in thread "worker-0"
>  java.lang.OutOfMemoryError: GC overhead limit exceeded... and many more
> lines.
>
> After this, one thread (out of two) dies but the other seems to hang doing
> nothing... and i have to manually kill it later.
>
>
>
> Now, i can give java a little bit more memory to java, say up to 1800m.
>
> setting  --max-nodes= larger should use less memory says documentation.
> With a larger max-nodes, will those tiles be faster or slower to display on
> a GPS unit?
> What is a reasonable value, 10 000 000 nodes or 2 000 000?
>
>
> Would --max-areas help my situation?
> If i use --max-areas=350 (approx half of total expected tiles), based on
> your experience would it be enough, or shall i better try with a value of
> 220 (~1/3 of the total tiles)?
>
> Regards
> Karl
>
>
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev




___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter questions

2020-10-22 Thread 7770

Hi Gerd.
Yes, it is 64-bit, but i don't have a lot of memory.
--keep-complete=false, can it impact routing possibilities (routing may not 
work well) or is it mainly affecting how the map will look?


Regards
Karl

On torsdag 22 oktober 2020 kl. 10:49:23 CEST Gerd Petermann wrote:
> Hi Karl,
> 
> if you are running a 64 Bit OS make sure to install a 64 Bit java runtime.
> This allows to increase the -Xmx option. You probably need > 5G for this.
> With a 32 Bit OS you can only try to disable the keep-complete option.
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag von 7770
> <7...@foskan.eu> Gesendet: Donnerstag, 22. Oktober 2020 10:45
> An: mkgmap-dev@lists.mkgmap.org.uk
> Betreff: [mkgmap-dev] splitter questions
> 
> Hi.
> 
> I am trying to split data for some 15 countries. First step is to combine
> the country data (with osmconvert) to one large o5m file and then try to
> split the combined file.
> The o5m file is about 14 GB in size.
> 
> When running splitter i give java -Xmx1600m
> options to splitter are: --max-nodes=150 --no-trim
> 
> Splitter calculates around 640 areas/tiles to be created.
> If i understand correctly, ways are parsed completely and then starts
> relations.
> 
> After 100 000 relations i get:
> * Full GC *
> and soon thereafter:
> Exception in thread "main" Exception in thread "worker-0"
>  java.lang.OutOfMemoryError: GC overhead limit exceeded... and many more
> lines.
> 
> After this, one thread (out of two) dies but the other seems to hang doing
> nothing... and i have to manually kill it later.
> 
> 
> 
> Now, i can give java a little bit more memory to java, say up to 1800m.
> 
> setting  --max-nodes= larger should use less memory says documentation.
> With a larger max-nodes, will those tiles be faster or slower to display on
> a GPS unit?
> What is a reasonable value, 10 000 000 nodes or 2 000 000?
> 
> 
> Would --max-areas help my situation?
> If i use --max-areas=350 (approx half of total expected tiles), based on
> your experience would it be enough, or shall i better try with a value of
> 220 (~1/3 of the total tiles)?
> 
> Regards
> Karl
> 
> 
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev




___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter questions

2020-10-22 Thread Gerd Petermann

Hi Karl,

if you are running a 64 Bit OS make sure to install a 64 Bit java runtime. This 
allows to increase the -Xmx option. You probably need > 5G for this.
With a 32 Bit OS you can only try to disable the keep-complete option.

Gerd


Von: mkgmap-dev  im Auftrag von 7770 
<7...@foskan.eu>
Gesendet: Donnerstag, 22. Oktober 2020 10:45
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: [mkgmap-dev] splitter questions

Hi.

I am trying to split data for some 15 countries. First step is to combine the
country data (with osmconvert) to one large o5m file and then try to split the
combined file.
The o5m file is about 14 GB in size.

When running splitter i give java -Xmx1600m
options to splitter are: --max-nodes=150 --no-trim

Splitter calculates around 640 areas/tiles to be created.
If i understand correctly, ways are parsed completely and then starts
relations.

After 100 000 relations i get:
* Full GC *
and soon thereafter:
Exception in thread "main" Exception in thread "worker-0"
 java.lang.OutOfMemoryError: GC overhead limit exceeded... and many more
lines.

After this, one thread (out of two) dies but the other seems to hang doing
nothing... and i have to manually kill it later.



Now, i can give java a little bit more memory to java, say up to 1800m.

setting  --max-nodes= larger should use less memory says documentation.
With a larger max-nodes, will those tiles be faster or slower to display on a
GPS unit?
What is a reasonable value, 10 000 000 nodes or 2 000 000?


Would --max-areas help my situation?
If i use --max-areas=350 (approx half of total expected tiles), based on your
experience would it be enough, or shall i better try with a value of 220 (~1/3
of the total tiles)?

Regards
Karl


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Splitter 597 xml format output problem

2020-09-02 Thread Dave

Hi Gerd,

Thanks will try that next time. My initial problem has been solved.

Dave

-Original Message-
From: mkgmap-dev  On Behalf Of Gerd
Petermann
Sent: 02 September 2020 12:20
To: Development list for mkgmap 
Subject: Re: [mkgmap-dev] Splitter 597 xml format output problem

Hi Dave,

question is why JOSM refuses to load version 0.5 To solve your problem you
may use splitter option --handle-element-version=fake

Gerd


Von: mkgmap-dev  im Auftrag von Dave

Gesendet: Mittwoch, 2. September 2020 11:52
An: 'Development list for mkgmap'
Betreff: [mkgmap-dev] Splitter 597 xml format output problem

Hi,

While trying to troubleshoot a map creation problem I had I decided to
output the data from Splitter using --output=xml rather than pbf so I could
open the resultant file in JOSM version 16812 (I wasn't sure if the
unexpected results were caused by Splitter or Mkgmap), JOSM gave the
following error message:

Could not read file '2641.osm'. Error is: Unsupported version: 0.5 (at
line 2, column 56). 100 bytes have been read

As a work around I edited the osm version to 0.6 and added visible="true"
version="1" to node and way id lines and was able to open the file. (was a
simple file only containing lines/ways no polygons etc.)

This is not something I plan to do often but thought you should be aware.

Dave


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Splitter 597 xml format output problem

2020-09-02 Thread Gerd Petermann

Hi Dave,

question is why JOSM refuses to load version 0.5
To solve your problem you may use splitter option --handle-element-version=fake

Gerd


Von: mkgmap-dev  im Auftrag von Dave 

Gesendet: Mittwoch, 2. September 2020 11:52
An: 'Development list for mkgmap'
Betreff: [mkgmap-dev] Splitter 597 xml format output problem

Hi,

While trying to troubleshoot a map creation problem I had I decided to
output the data from Splitter using --output=xml rather than pbf so I could
open the resultant file in JOSM version 16812 (I wasn't sure if the
unexpected results were caused by Splitter or Mkgmap), JOSM gave the
following error message:

Could not read file '2641.osm'. Error is: Unsupported version: 0.5 (at
line 2, column 56). 100 bytes have been read

As a work around I edited the osm version to 0.6 and added visible="true"
version="1" to node and way id lines and was able to open the file. (was a
simple file only containing lines/ways no polygons etc.)

This is not something I plan to do often but thought you should be aware.

Dave


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter and options --description / --geonames-file

2020-04-10 Thread Ticker Berkin

Hi

I'd go for the idea of having the specific option eg --add-google-plus.

Also, as a slight modification to the patch that increases the area
beyond the tile looking for a city, maybe limit it double the tile size
before giving up and just describing the tile with the mapName.

Ticker

On Fri, 2020-04-10 at 06:46 +, Gerd Petermann wrote:
> Hi Randolph,
> 
> when I want to visualize the calulcated areas I use --write
> -kml=splitter.kml. The output file splitter.kml can be used in JOSM
> (with a plugin) as well as in Google Earth,
> if you prefer that. The use in JOSM helps me to find issues where a
> tile border might be involved.
> 
> Anyhow, if others are interested I would use your patch with a
> different option, something like --add-google-plus which would either
> add the code to the city name or to the given description or to an
> empty string so that splitter would always generate a unique tile
> description.
> 
> Gerd
> 
> 
> 
> Von: Randolph J. Herber 
> Gesendet: Freitag, 10. April 2020 02:17
> An: Development list for mkgmap; Gerd Petermann; Mike Baggaley;
> Ticker Berkin
> Betreff: Re: [mkgmap-dev] splitter and options --description / -
> -geonames-file
> 
> Hi, all!
> 
> Yes, it helps.
> 
> It was not what I had in mind.
> 
> [Fragment of  template.args showing the OLC or Google Plus
> Codes that were  added.]
> 
> These codes can be directly used to show the area in question: EG
> -Alexandria:8G3F5W00+<https://plus.codes/8G3F5W00+>
> 
> [Image of  Alexandria, Egypt, area referenced.]
> 
> and for an Egyptian area with no cities with a population as large as
> 15 000: EgyptUC:7GJM+<https://plus.codes/7GJM+>
> 
> [Large area insouthern Egypt with no cities as large as 15
> thousand people.]
> 
> This area is in southern Egypt.
> 
> The differences in AreaList.java, in Unix format are in the attached
> differences file.
> 
> The modified AreaList.java files in the attached java file.
> 
> Can you now understand what I want and why?
> 
> Randolph J. Herber
> 
> 
> On 4/9/2020 9:48 AM, Gerd Petermann wrote:
> 
> Hi all,
> 
> sorry, I think some mails where not sent to the list. I've attached a
> patch that adds a number to the city name when --geonames-file is
> used and multiple tiles
> are close to that city.
> 
> I fear I don't know in what situation these names are important. So,
> please try if this improves something.
> 
> Gerd

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter and options --description / --geonames-file

2020-04-09 Thread Gerd Petermann

Hi all,

sorry, I think some mails where not sent to the list. I've attached a patch 
that adds a number to the city name when --geonames-file is used and multiple 
tiles
are close to that city.

I fear I don't know in what situation these names are important. So, please try 
if this improves something.

Gerd

Von: mkgmap-dev  im Auftrag von Gerd 
Petermann 
Gesendet: Mittwoch, 8. April 2020 16:40
An: Development list for mkgmap; Ticker Berkin
Betreff: Re: [mkgmap-dev] splitter and options --description/   
--geonames-file

Hi Randolph,

in what situation do you use these codes? I am happy with the mapids generated 
by splitter.

Gerd

Von: mkgmap-dev  im Auftrag von 
Randolph J. Herber 
Gesendet: Mittwoch, 8. April 2020 15:13
An: Development list for mkgmap; Ticker Berkin
Betreff: Re: [mkgmap-dev] splitter and options --description /  --geonames-file

Hi, all,

The Definition of Google Plus Codes, a.k.a. Opern Location Codes, can be
found at https://en.wikipedia.org/wiki/Open_Location_Code

The complete code also specifies, by its length, the size of the area
being described. When using them in splitter, I have never seen a tile
whose code was longer than 8 characters, an area 250m east to west and
125m north to south. I use the largest OLC centered on the tile in
question that fits within the tile.

Randolph J. Herber

On 4/8/2020 4:59 AM, Ticker Berkin wrote:
> Hi Gerd
>
> I'd say that if the splitter is going to generate descriptions per tile
> (geonames) it should generate one for each tile and, if there isn't a
> centre of a city within the tile and no default description, it should
> use a modified form of the nearest city to make it unique from a
> possible adjacent tile that might use the same city.
>
> Maybe something like "GB-~Basingstoke", but no solution is ideal and
> there could still be duplicates.
>
> The splitter could keep track of which names were have been used and,
> for the ones outside the tile, added an increasing suffix number, so
> would have "GB-Basingstoke-2" etc
>
> Ticker
>
> On Wed, 2020-04-08 at 08:38 +, Gerd Petermann wrote:
>> Hi all,
>>
>> this is a follow up of http://gis.19327.n8.nabble.com/documentation-i
>> mprovement-tp5962609p5962747.html
>>
>> Since I don't use these options for my maps I am unsure what is best.
>> The --geonames-file option tries to find a city within the calculated
>> tile. If none is found it either writes the value
>> given with --description or just a comment.
>> I think it would be best to always write the nearest city.
>>
>> Gerd
>> ___
>> mkgmap-dev mailing list
>> mkgmap-dev@lists.mkgmap.org.uk
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

geonames.patch
Description: geonames.patch
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter and options --description / --geonames-file

2020-04-08 Thread Gerd Petermann

Hi Randolph,

in what situation do you use these codes? I am happy with the mapids generated 
by splitter.

Gerd

Von: mkgmap-dev  im Auftrag von 
Randolph J. Herber 
Gesendet: Mittwoch, 8. April 2020 15:13
An: Development list for mkgmap; Ticker Berkin
Betreff: Re: [mkgmap-dev] splitter and options --description /  --geonames-file

Hi, all,

The Definition of Google Plus Codes, a.k.a. Opern Location Codes, can be
found at https://en.wikipedia.org/wiki/Open_Location_Code

The complete code also specifies, by its length, the size of the area
being described. When using them in splitter, I have never seen a tile
whose code was longer than 8 characters, an area 250m east to west and
125m north to south. I use the largest OLC centered on the tile in
question that fits within the tile.

Randolph J. Herber

On 4/8/2020 4:59 AM, Ticker Berkin wrote:
> Hi Gerd
>
> I'd say that if the splitter is going to generate descriptions per tile
> (geonames) it should generate one for each tile and, if there isn't a
> centre of a city within the tile and no default description, it should
> use a modified form of the nearest city to make it unique from a
> possible adjacent tile that might use the same city.
>
> Maybe something like "GB-~Basingstoke", but no solution is ideal and
> there could still be duplicates.
>
> The splitter could keep track of which names were have been used and,
> for the ones outside the tile, added an increasing suffix number, so
> would have "GB-Basingstoke-2" etc
>
> Ticker
>
> On Wed, 2020-04-08 at 08:38 +, Gerd Petermann wrote:
>> Hi all,
>>
>> this is a follow up of http://gis.19327.n8.nabble.com/documentation-i
>> mprovement-tp5962609p5962747.html
>>
>> Since I don't use these options for my maps I am unsure what is best.
>> The --geonames-file option tries to find a city within the calculated
>> tile. If none is found it either writes the value
>> given with --description or just a comment.
>> I think it would be best to always write the nearest city.
>>
>> Gerd
>> ___
>> mkgmap-dev mailing list
>> mkgmap-dev@lists.mkgmap.org.uk
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter and options --description / --geonames-file

2020-04-08 Thread Randolph J. Herber


Hi, all,

The Definition of Google Plus Codes, a.k.a. Opern Location Codes, can be 
found at https://en.wikipedia.org/wiki/Open_Location_Code


The complete code also specifies, by its length, the size of the area 
being described. When using them in splitter, I have never seen a tile 
whose code was longer than 8 characters, an area 250m east to west and 
125m north to south. I use the largest OLC centered on the tile in 
question that fits within the tile.


Randolph J. Herber

On 4/8/2020 4:59 AM, Ticker Berkin wrote:

Hi Gerd

I'd say that if the splitter is going to generate descriptions per tile
(geonames) it should generate one for each tile and, if there isn't a
centre of a city within the tile and no default description, it should
use a modified form of the nearest city to make it unique from a
possible adjacent tile that might use the same city.

Maybe something like "GB-~Basingstoke", but no solution is ideal and
there could still be duplicates.

The splitter could keep track of which names were have been used and,
for the ones outside the tile, added an increasing suffix number, so
would have "GB-Basingstoke-2" etc

Ticker

On Wed, 2020-04-08 at 08:38 +, Gerd Petermann wrote:

Hi all,

this is a follow up of http://gis.19327.n8.nabble.com/documentation-i
mprovement-tp5962609p5962747.html

Since I don't use these options for my maps I am unsure what is best.
The --geonames-file option tries to find a city within the calculated
tile. If none is found it either writes the value
given with --description or just a comment.
I think it would be best to always write the nearest city.

Gerd
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter and options --description / --geonames-file

2020-04-08 Thread Randolph J. Herber


Hi, all!

It would be best if each title has its own, separate identity.

Randolph J. Herber

On 4/8/2020 3:38 AM, Gerd Petermann wrote:

Hi all,

this is a follow up of 
http://gis.19327.n8.nabble.com/documentation-improvement-tp5962609p5962747.html

Since I don't use these options for my maps I am unsure what is best.
The --geonames-file option tries to find a city within the calculated tile. If 
none is found it either writes the value
given with --description or just a comment.
I think it would be best to always write the nearest city.

Gerd
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter and options --description / --geonames-file

2020-04-08 Thread Ticker Berkin

Hi Gerd

I'd say that if the splitter is going to generate descriptions per tile
(geonames) it should generate one for each tile and, if there isn't a
centre of a city within the tile and no default description, it should
use a modified form of the nearest city to make it unique from a
possible adjacent tile that might use the same city.

Maybe something like "GB-~Basingstoke", but no solution is ideal and
there could still be duplicates.

The splitter could keep track of which names were have been used and,
for the ones outside the tile, added an increasing suffix number, so
would have "GB-Basingstoke-2" etc

Ticker

On Wed, 2020-04-08 at 08:38 +, Gerd Petermann wrote:
> Hi all,
> 
> this is a follow up of http://gis.19327.n8.nabble.com/documentation-i
> mprovement-tp5962609p5962747.html
> 
> Since I don't use these options for my maps I am unsure what is best.
> The --geonames-file option tries to find a city within the calculated
> tile. If none is found it either writes the value
> given with --description or just a comment.
> I think it would be best to always write the nearest city.
> 
> Gerd
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Splitter Build Failure

2020-01-30 Thread News

Ok I've got it now. I noticed in the following that I successfully 
downloaded ivy-2.5.0.jar but then seemed to be trying to use "Apache Ivy 
2.3.0-local-2017090712 - 2017090712" which was a local 
Apache-Ivy I had previously installed in order to try and fix this 
before asking the list. The updated build.xml and uninstalling the 
apache-ivy package did the trick


Thanks

Paul


download-ivy:
[mkdir] Created dir: 
/home/miscellaneous/Maps/tile-splitter/splitter/lib/build
[mkdir] Created dir: 
/home/miscellaneous/Maps/tile-splitter/splitter/lib/compile
[mkdir] Created dir: 
/home/miscellaneous/Maps/tile-splitter/splitter/lib/test
  [get] Getting: 
https://repo1.maven.org/maven2/org/apache/ivy/ivy/2.5.0/ivy-2.5.0.jar
  [get] To: 
/home/miscellaneous/Maps/tile-splitter/splitter/lib/build/ivy-2.5.0.jar


init-ivy:
Trying to override old definition of task antlib:org.apache.ivy.ant:buildobr
Trying to override old definition of task antlib:org.apache.ivy.ant:buildobr
[ivy:configure] impossible to define new type: class not found: 
org.apache.ivy.plugins.signer.bouncycastle.OpenPGPSignatureGenerator in 
[] nor Ivy classloader
[ivy:configure] :: Apache Ivy 2.3.0-local-2017090712 - 
2017090712 :: http://ant.apache.org/ivy/ ::
[ivy:configure] :: loading settings :: file = 
/home/miscellaneous/Maps/tile-splitter/splitter/ivysettings.xml


resolve-compile:
[ivy:retrieve]
[ivy:retrieve] :: problems summary ::
[ivy:retrieve]  WARNINGS
[ivy:retrieve]  module not found: xpp3#xpp3;1.1.4c
[ivy:retrieve]   local: tried
[ivy:retrieve]/home/user/.ivy2/local/xpp3/xpp3/1.1.4c/ivys/ivy.xml
[ivy:retrieve]-- artifact xpp3#xpp3;1.1.4c!xpp3.jar:
[ivy:retrieve]/home/user/.ivy2/local/xpp3/xpp3/1.1.4c/jars/xpp3.jar
[ivy:retrieve]   shared: tried
[ivy:retrieve]/home/user/.ivy2/shared/xpp3/xpp3/1.1.4c/ivys/ivy.xml
[ivy:retrieve]-- artifact xpp3#xpp3;1.1.4c!xpp3.jar:
[ivy:retrieve]/home/user/.ivy2/shared/xpp3/xpp3/1.1.4c/jars/xpp3.jar
[ivy:retrieve]   public: tried
[ivy:retrieve] 
http://repo1.maven.org/maven2/xpp3/xpp3/1.1.4c/xpp3-1.1.4c.pom

[ivy:retrieve]-- artifact xpp3#xpp3;1.1.4c!xpp3.jar:
[ivy:retrieve] 
http://repo1.maven.org/maven2/xpp3/xpp3/1.1.4c/xpp3-1.1.4c.jar

[ivy:retrieve]   mkgmap: tried
[ivy:retrieve] 
http://ivy.mkgmap.org.uk/repo/xpp3/xpp3/1.1.4c/ivys/ivy.xml

[ivy:retrieve]-- artifact xpp3#xpp3;1.1.4c!xpp3.jar:
[ivy:retrieve] 
http://ivy.mkgmap.org.uk/repo/xpp3/xpp3/1.1.4c/jars/xpp3.jar

[ivy:retrieve]  ::
[ivy:retrieve]  ::  UNRESOLVED DEPENDENCIES ::
[ivy:retrieve]  ::
[ivy:retrieve]  :: xpp3#xpp3;1.1.4c: not found
[ivy:retrieve]  ::
[ivy:retrieve]  ERRORS
[ivy:retrieve]  SERVER ERROR: HTTPS Required 
url=http://repo1.maven.org/maven2/xpp3/xpp3/1.1.4c/xpp3-1.1.4c.pom
[ivy:retrieve]  SERVER ERROR: HTTPS Required 
url=http://repo1.maven.org/maven2/xpp3/xpp3/1.1.4c/xpp3-1.1.4c.jar

[ivy:retrieve]
[ivy:retrieve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS

BUILD FAILED
/home/miscellaneous/Maps/tile-splitter/splitter/build.xml:113: 
impossible to resolve dependencies:

resolve failed - see output for details


On 30/01/2020 15:13, Steve Ratcliffe wrote:

Hi

I didn't fix any thing before you sent this message, but I found I needed to 
upgrade ivy for it to work. So I've committed that now.

mkgmap already used a later version.

Steve

On 30 January 2020 13:45:55 GMT, Gerd Petermann 
 wrote:

Hi all,

I just tried again after renaming .ivy cache directory with a clean checkout of 
spliter and mkgmap and both builds worked. A few minutes ago it didn't.
Maybe Steve has already fixed something. I've never cared much about the ivy 
stuff.

Gerd


Von: mkgmap-dev  im Auftrag von DD8KQ 

Gesendet: Donnerstag, 30. Januar 2020 14:44
An: Development list for mkgmap; News
Betreff: Re: [mkgmap-dev] Splitter Build Failure

I have a short look to your failures. For me, it looks like you have to
change the url-requests form http to https. But i don't know, where you
have to change it. 'repo1.maven.org' don't want to communicate with
unsecure http anymore.

Am 30.01.2020 um 14:04 schrieb News:

Hi Gerd

Sorry. This still fails. Full info below

[ivy:configure] impossible to define new type: class not found:
org.apache.ivy.plugins.signer.bouncycastle.OpenPGPSignatureGenerator
in [] nor Ivy classloader
[ivy:configure] :: Apache Ivy 2.3.0-local-2017090712 -
2017090712 :: http://ant.apache.org/ivy/ ::
[ivy:configure] :: loading settings :: file =
/home/miscellaneous/Maps/tile-splitter/splitter/ivysettings.xml

resolve-compile:
[ivy:retrieve]
[ivy:retrieve] :: problems summary ::
[ivy:retrieve]  WARNINGS
[ivy:retrieve]  module not found: xpp3#xpp3;1.1.4c
[iv

Re: [mkgmap-dev] Splitter Build Failure

2020-01-30 Thread Steve Ratcliffe

Hi

I didn't fix any thing before you sent this message, but I found I needed to 
upgrade ivy for it to work. So I've committed that now.

mkgmap already used a later version.

Steve

On 30 January 2020 13:45:55 GMT, Gerd Petermann 
 wrote:
>Hi all,
>
>I just tried again after renaming .ivy cache directory with a clean checkout 
>of spliter and mkgmap and both builds worked. A few minutes ago it didn't.
>Maybe Steve has already fixed something. I've never cared much about the ivy 
>stuff.
>
>Gerd
>
>
>Von: mkgmap-dev  im Auftrag von DD8KQ 
>
>Gesendet: Donnerstag, 30. Januar 2020 14:44
>An: Development list for mkgmap; News
>Betreff: Re: [mkgmap-dev] Splitter Build Failure
>
>I have a short look to your failures. For me, it looks like you have to
>change the url-requests form http to https. But i don't know, where you
>have to change it. 'repo1.maven.org' don't want to communicate with
>unsecure http anymore.
>
>Am 30.01.2020 um 14:04 schrieb News:
>> Hi Gerd
>>
>> Sorry. This still fails. Full info below
>>
>> [ivy:configure] impossible to define new type: class not found:
>> org.apache.ivy.plugins.signer.bouncycastle.OpenPGPSignatureGenerator
>> in [] nor Ivy classloader
>> [ivy:configure] :: Apache Ivy 2.3.0-local-2017090712 -
>> 2017090712 :: http://ant.apache.org/ivy/ ::
>> [ivy:configure] :: loading settings :: file =
>> /home/miscellaneous/Maps/tile-splitter/splitter/ivysettings.xml
>>
>> resolve-compile:
>> [ivy:retrieve]
>> [ivy:retrieve] :: problems summary ::
>> [ivy:retrieve]  WARNINGS
>> [ivy:retrieve]  module not found: xpp3#xpp3;1.1.4c
>> [ivy:retrieve]   local: tried
>> [ivy:retrieve] /home/user/.ivy2/local/xpp3/xpp3/1.1.4c/ivys/ivy.xml
>> [ivy:retrieve]    -- artifact xpp3#xpp3;1.1.4c!xpp3.jar:
>> [ivy:retrieve] /home/user/.ivy2/local/xpp3/xpp3/1.1.4c/jars/xpp3.jar
>> [ivy:retrieve]   shared: tried
>> [ivy:retrieve] /home/user/.ivy2/shared/xpp3/xpp3/1.1.4c/ivys/ivy.xml
>> [ivy:retrieve]    -- artifact xpp3#xpp3;1.1.4c!xpp3.jar:
>> [ivy:retrieve] /home/user/.ivy2/shared/xpp3/xpp3/1.1.4c/jars/xpp3.jar
>> [ivy:retrieve]   public: tried
>> [ivy:retrieve]
>> http://repo1.maven.org/maven2/xpp3/xpp3/1.1.4c/xpp3-1.1.4c.pom
>> [ivy:retrieve]    -- artifact xpp3#xpp3;1.1.4c!xpp3.jar:
>> [ivy:retrieve]
>> http://repo1.maven.org/maven2/xpp3/xpp3/1.1.4c/xpp3-1.1.4c.jar
>> [ivy:retrieve]   mkgmap: tried
>> [ivy:retrieve]
>> http://ivy.mkgmap.org.uk/repo/xpp3/xpp3/1.1.4c/ivys/ivy.xml
>> [ivy:retrieve]    -- artifact xpp3#xpp3;1.1.4c!xpp3.jar:
>> [ivy:retrieve]
>> http://ivy.mkgmap.org.uk/repo/xpp3/xpp3/1.1.4c/jars/xpp3.jar
>> [ivy:retrieve] ::
>> [ivy:retrieve]  ::  UNRESOLVED DEPENDENCIES ::
>> [ivy:retrieve] ::
>> [ivy:retrieve]  :: xpp3#xpp3;1.1.4c: not found
>> [ivy:retrieve] ::
>> [ivy:retrieve]  ERRORS
>> [ivy:retrieve]  SERVER ERROR: HTTPS Required
>> url=http://repo1.maven.org/maven2/xpp3/xpp3/1.1.4c/xpp3-1.1.4c.pom
>> [ivy:retrieve]  SERVER ERROR: HTTPS Required
>> url=http://repo1.maven.org/maven2/xpp3/xpp3/1.1.4c/xpp3-1.1.4c.jar
>> [ivy:retrieve]
>> [ivy:retrieve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
>>
>> BUILD FAILED
>> /home/miscellaneous/Maps/tile-splitter/splitter/build.xml:113:
>> impossible to resolve dependencies:
>> resolve failed - see output for details
>>
>> Thanks
>>
>> Paul
>>
>> On 30/01/2020 13:10, Gerd Petermann wrote:
>>> Hi Paul,
>>>
>>> please update, see
>>> http://www.mkgmap.org.uk/websvn/revision.php?repname=splitter=596
>>>
>>> Gerd
>>>
>>> 
>>> Von: mkgmap-dev  im Auftrag
>>> von News 
>>> Gesendet: Donnerstag, 30. Januar 2020 13:05
>>> An: 'Development list for mkgmap'
>>> Betreff: [mkgmap-dev] Splitter Build Failure
>>>
>>> I recently clean reinstalled my PC (openSuSE Leap 15.1) and today was
>>> the first time I tried to build splitter and received the following
>>> errors
>>>
>>> BUILD FAILED
>>> /home/miscellaneous/Maps/tile-splitter/splitter/build.xml:95: Can't get
>>> http://repo1.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar to
>>> /home/miscellaneous/Maps/tile-splitter/splitter/lib/build/ivy-2.2.0.jar
>>>
>>>

Re: [mkgmap-dev] Splitter Build Failure

2020-01-30 Thread Gerd Petermann

Hi all,

I just tried again after renaming .ivy cache directory with a clean checkout of 
spliter and mkgmap and both builds worked. A few minutes ago it didn't.
Maybe Steve has already fixed something. I've never cared much about the ivy 
stuff.

Gerd


Von: mkgmap-dev  im Auftrag von DD8KQ 

Gesendet: Donnerstag, 30. Januar 2020 14:44
An: Development list for mkgmap; News
Betreff: Re: [mkgmap-dev] Splitter Build Failure

I have a short look to your failures. For me, it looks like you have to
change the url-requests form http to https. But i don't know, where you
have to change it. 'repo1.maven.org' don't want to communicate with
unsecure http anymore.

Am 30.01.2020 um 14:04 schrieb News:
> Hi Gerd
>
> Sorry. This still fails. Full info below
>
> [ivy:configure] impossible to define new type: class not found:
> org.apache.ivy.plugins.signer.bouncycastle.OpenPGPSignatureGenerator
> in [] nor Ivy classloader
> [ivy:configure] :: Apache Ivy 2.3.0-local-2017090712 -
> 2017090712 :: http://ant.apache.org/ivy/ ::
> [ivy:configure] :: loading settings :: file =
> /home/miscellaneous/Maps/tile-splitter/splitter/ivysettings.xml
>
> resolve-compile:
> [ivy:retrieve]
> [ivy:retrieve] :: problems summary ::
> [ivy:retrieve]  WARNINGS
> [ivy:retrieve]  module not found: xpp3#xpp3;1.1.4c
> [ivy:retrieve]   local: tried
> [ivy:retrieve] /home/user/.ivy2/local/xpp3/xpp3/1.1.4c/ivys/ivy.xml
> [ivy:retrieve]    -- artifact xpp3#xpp3;1.1.4c!xpp3.jar:
> [ivy:retrieve] /home/user/.ivy2/local/xpp3/xpp3/1.1.4c/jars/xpp3.jar
> [ivy:retrieve]   shared: tried
> [ivy:retrieve] /home/user/.ivy2/shared/xpp3/xpp3/1.1.4c/ivys/ivy.xml
> [ivy:retrieve]    -- artifact xpp3#xpp3;1.1.4c!xpp3.jar:
> [ivy:retrieve] /home/user/.ivy2/shared/xpp3/xpp3/1.1.4c/jars/xpp3.jar
> [ivy:retrieve]   public: tried
> [ivy:retrieve]
> http://repo1.maven.org/maven2/xpp3/xpp3/1.1.4c/xpp3-1.1.4c.pom
> [ivy:retrieve]    -- artifact xpp3#xpp3;1.1.4c!xpp3.jar:
> [ivy:retrieve]
> http://repo1.maven.org/maven2/xpp3/xpp3/1.1.4c/xpp3-1.1.4c.jar
> [ivy:retrieve]   mkgmap: tried
> [ivy:retrieve]
> http://ivy.mkgmap.org.uk/repo/xpp3/xpp3/1.1.4c/ivys/ivy.xml
> [ivy:retrieve]    -- artifact xpp3#xpp3;1.1.4c!xpp3.jar:
> [ivy:retrieve]
> http://ivy.mkgmap.org.uk/repo/xpp3/xpp3/1.1.4c/jars/xpp3.jar
> [ivy:retrieve] ::
> [ivy:retrieve]  ::  UNRESOLVED DEPENDENCIES ::
> [ivy:retrieve] ::
> [ivy:retrieve]  :: xpp3#xpp3;1.1.4c: not found
> [ivy:retrieve] ::
> [ivy:retrieve]  ERRORS
> [ivy:retrieve]  SERVER ERROR: HTTPS Required
> url=http://repo1.maven.org/maven2/xpp3/xpp3/1.1.4c/xpp3-1.1.4c.pom
> [ivy:retrieve]  SERVER ERROR: HTTPS Required
> url=http://repo1.maven.org/maven2/xpp3/xpp3/1.1.4c/xpp3-1.1.4c.jar
> [ivy:retrieve]
> [ivy:retrieve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
>
> BUILD FAILED
> /home/miscellaneous/Maps/tile-splitter/splitter/build.xml:113:
> impossible to resolve dependencies:
> resolve failed - see output for details
>
> Thanks
>
> Paul
>
> On 30/01/2020 13:10, Gerd Petermann wrote:
>> Hi Paul,
>>
>> please update, see
>> http://www.mkgmap.org.uk/websvn/revision.php?repname=splitter=596
>>
>> Gerd
>>
>> 
>> Von: mkgmap-dev  im Auftrag
>> von News 
>> Gesendet: Donnerstag, 30. Januar 2020 13:05
>> An: 'Development list for mkgmap'
>> Betreff: [mkgmap-dev] Splitter Build Failure
>>
>> I recently clean reinstalled my PC (openSuSE Leap 15.1) and today was
>> the first time I tried to build splitter and received the following
>> errors
>>
>> BUILD FAILED
>> /home/miscellaneous/Maps/tile-splitter/splitter/build.xml:95: Can't get
>> http://repo1.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar to
>> /home/miscellaneous/Maps/tile-splitter/splitter/lib/build/ivy-2.2.0.jar
>>
>> I changed the line in build.xml to use https instead of http and I then
>> got past this error and received the following error snippet
>>
>> [ivy:retrieve] ::
>> [ivy:retrieve]  ::  UNRESOLVED DEPENDENCIES ::
>> [ivy:retrieve] ::
>> [ivy:retrieve]  :: xpp3#xpp3;1.1.4c: not found
>> [ivy:retrieve] ::
>> [ivy:retrieve]  ERRORS
>> [ivy:retrieve]  SERVER ERROR: HTTPS Required
>> url=http://repo1.maven.org/maven2/xpp3/xpp3/1.1.4c/xpp3-1.1.4

Re: [mkgmap-dev] Splitter Build Failure

2020-01-30 Thread DD8KQ


I have a short look to your failures. For me, it looks like you have to
change the url-requests form http to https. But i don't know, where you
have to change it. 'repo1.maven.org' don't want to communicate with
unsecure http anymore.

Am 30.01.2020 um 14:04 schrieb News:

Hi Gerd

Sorry. This still fails. Full info below

[ivy:configure] impossible to define new type: class not found:
org.apache.ivy.plugins.signer.bouncycastle.OpenPGPSignatureGenerator
in [] nor Ivy classloader
[ivy:configure] :: Apache Ivy 2.3.0-local-2017090712 -
2017090712 :: http://ant.apache.org/ivy/ ::
[ivy:configure] :: loading settings :: file =
/home/miscellaneous/Maps/tile-splitter/splitter/ivysettings.xml

resolve-compile:
[ivy:retrieve]
[ivy:retrieve] :: problems summary ::
[ivy:retrieve]  WARNINGS
[ivy:retrieve]  module not found: xpp3#xpp3;1.1.4c
[ivy:retrieve]   local: tried
[ivy:retrieve] /home/user/.ivy2/local/xpp3/xpp3/1.1.4c/ivys/ivy.xml
[ivy:retrieve]    -- artifact xpp3#xpp3;1.1.4c!xpp3.jar:
[ivy:retrieve] /home/user/.ivy2/local/xpp3/xpp3/1.1.4c/jars/xpp3.jar
[ivy:retrieve]   shared: tried
[ivy:retrieve] /home/user/.ivy2/shared/xpp3/xpp3/1.1.4c/ivys/ivy.xml
[ivy:retrieve]    -- artifact xpp3#xpp3;1.1.4c!xpp3.jar:
[ivy:retrieve] /home/user/.ivy2/shared/xpp3/xpp3/1.1.4c/jars/xpp3.jar
[ivy:retrieve]   public: tried
[ivy:retrieve]
http://repo1.maven.org/maven2/xpp3/xpp3/1.1.4c/xpp3-1.1.4c.pom
[ivy:retrieve]    -- artifact xpp3#xpp3;1.1.4c!xpp3.jar:
[ivy:retrieve]
http://repo1.maven.org/maven2/xpp3/xpp3/1.1.4c/xpp3-1.1.4c.jar
[ivy:retrieve]   mkgmap: tried
[ivy:retrieve]
http://ivy.mkgmap.org.uk/repo/xpp3/xpp3/1.1.4c/ivys/ivy.xml
[ivy:retrieve]    -- artifact xpp3#xpp3;1.1.4c!xpp3.jar:
[ivy:retrieve]
http://ivy.mkgmap.org.uk/repo/xpp3/xpp3/1.1.4c/jars/xpp3.jar
[ivy:retrieve] ::
[ivy:retrieve]  ::  UNRESOLVED DEPENDENCIES ::
[ivy:retrieve] ::
[ivy:retrieve]  :: xpp3#xpp3;1.1.4c: not found
[ivy:retrieve] ::
[ivy:retrieve]  ERRORS
[ivy:retrieve]  SERVER ERROR: HTTPS Required
url=http://repo1.maven.org/maven2/xpp3/xpp3/1.1.4c/xpp3-1.1.4c.pom
[ivy:retrieve]  SERVER ERROR: HTTPS Required
url=http://repo1.maven.org/maven2/xpp3/xpp3/1.1.4c/xpp3-1.1.4c.jar
[ivy:retrieve]
[ivy:retrieve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS

BUILD FAILED
/home/miscellaneous/Maps/tile-splitter/splitter/build.xml:113:
impossible to resolve dependencies:
    resolve failed - see output for details

Thanks

Paul

On 30/01/2020 13:10, Gerd Petermann wrote:

Hi Paul,

please update, see
http://www.mkgmap.org.uk/websvn/revision.php?repname=splitter=596

Gerd


Von: mkgmap-dev  im Auftrag
von News 
Gesendet: Donnerstag, 30. Januar 2020 13:05
An: 'Development list for mkgmap'
Betreff: [mkgmap-dev] Splitter Build Failure

I recently clean reinstalled my PC (openSuSE Leap 15.1) and today was
the first time I tried to build splitter and received the following
errors

BUILD FAILED
/home/miscellaneous/Maps/tile-splitter/splitter/build.xml:95: Can't get
http://repo1.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar to
/home/miscellaneous/Maps/tile-splitter/splitter/lib/build/ivy-2.2.0.jar

I changed the line in build.xml to use https instead of http and I then
got past this error and received the following error snippet

[ivy:retrieve] ::
[ivy:retrieve]  ::  UNRESOLVED DEPENDENCIES ::
[ivy:retrieve] ::
[ivy:retrieve]  :: xpp3#xpp3;1.1.4c: not found
[ivy:retrieve] ::
[ivy:retrieve]  ERRORS
[ivy:retrieve]  SERVER ERROR: HTTPS Required
url=http://repo1.maven.org/maven2/xpp3/xpp3/1.1.4c/xpp3-1.1.4c.pom
[ivy:retrieve]  SERVER ERROR: HTTPS Required
url=http://repo1.maven.org/maven2/xpp3/xpp3/1.1.4c/xpp3-1.1.4c.jar

So it seems that maven.org is only accepting https connections but I
can't find where to correct the above error

Any suggestions?

Thanks

Paul
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev



___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


--

#

 Viele Grüße und 73 de Manfred Haiduk, DD8KQ
 e-mail mhai...@t-online.de dd...@gmx.de

#

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk

Re: [mkgmap-dev] Splitter Build Failure

2020-01-30 Thread News


Hi Gerd

Sorry. This still fails. Full info below

[ivy:configure] impossible to define new type: class not found: 
org.apache.ivy.plugins.signer.bouncycastle.OpenPGPSignatureGenerator in 
[] nor Ivy classloader
[ivy:configure] :: Apache Ivy 2.3.0-local-2017090712 - 
2017090712 :: http://ant.apache.org/ivy/ ::
[ivy:configure] :: loading settings :: file = 
/home/miscellaneous/Maps/tile-splitter/splitter/ivysettings.xml


resolve-compile:
[ivy:retrieve]
[ivy:retrieve] :: problems summary ::
[ivy:retrieve]  WARNINGS
[ivy:retrieve]  module not found: xpp3#xpp3;1.1.4c
[ivy:retrieve]   local: tried
[ivy:retrieve]/home/user/.ivy2/local/xpp3/xpp3/1.1.4c/ivys/ivy.xml
[ivy:retrieve]-- artifact xpp3#xpp3;1.1.4c!xpp3.jar:
[ivy:retrieve]/home/user/.ivy2/local/xpp3/xpp3/1.1.4c/jars/xpp3.jar
[ivy:retrieve]   shared: tried
[ivy:retrieve]/home/user/.ivy2/shared/xpp3/xpp3/1.1.4c/ivys/ivy.xml
[ivy:retrieve]-- artifact xpp3#xpp3;1.1.4c!xpp3.jar:
[ivy:retrieve]/home/user/.ivy2/shared/xpp3/xpp3/1.1.4c/jars/xpp3.jar
[ivy:retrieve]   public: tried
[ivy:retrieve] 
http://repo1.maven.org/maven2/xpp3/xpp3/1.1.4c/xpp3-1.1.4c.pom

[ivy:retrieve]-- artifact xpp3#xpp3;1.1.4c!xpp3.jar:
[ivy:retrieve] 
http://repo1.maven.org/maven2/xpp3/xpp3/1.1.4c/xpp3-1.1.4c.jar

[ivy:retrieve]   mkgmap: tried
[ivy:retrieve] http://ivy.mkgmap.org.uk/repo/xpp3/xpp3/1.1.4c/ivys/ivy.xml
[ivy:retrieve]-- artifact xpp3#xpp3;1.1.4c!xpp3.jar:
[ivy:retrieve] http://ivy.mkgmap.org.uk/repo/xpp3/xpp3/1.1.4c/jars/xpp3.jar
[ivy:retrieve]  ::
[ivy:retrieve]  ::  UNRESOLVED DEPENDENCIES ::
[ivy:retrieve]  ::
[ivy:retrieve]  :: xpp3#xpp3;1.1.4c: not found
[ivy:retrieve]  ::
[ivy:retrieve]  ERRORS
[ivy:retrieve]  SERVER ERROR: HTTPS Required 
url=http://repo1.maven.org/maven2/xpp3/xpp3/1.1.4c/xpp3-1.1.4c.pom
[ivy:retrieve]  SERVER ERROR: HTTPS Required 
url=http://repo1.maven.org/maven2/xpp3/xpp3/1.1.4c/xpp3-1.1.4c.jar

[ivy:retrieve]
[ivy:retrieve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS

BUILD FAILED
/home/miscellaneous/Maps/tile-splitter/splitter/build.xml:113: 
impossible to resolve dependencies:

resolve failed - see output for details

Thanks

Paul

On 30/01/2020 13:10, Gerd Petermann wrote:

Hi Paul,

please update, see 
http://www.mkgmap.org.uk/websvn/revision.php?repname=splitter=596

Gerd


Von: mkgmap-dev  im Auftrag von News 

Gesendet: Donnerstag, 30. Januar 2020 13:05
An: 'Development list for mkgmap'
Betreff: [mkgmap-dev] Splitter Build Failure

I recently clean reinstalled my PC (openSuSE Leap 15.1) and today was
the first time I tried to build splitter and received the following errors

BUILD FAILED
/home/miscellaneous/Maps/tile-splitter/splitter/build.xml:95: Can't get
http://repo1.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar to
/home/miscellaneous/Maps/tile-splitter/splitter/lib/build/ivy-2.2.0.jar

I changed the line in build.xml to use https instead of http and I then
got past this error and received the following error snippet

[ivy:retrieve]  ::
[ivy:retrieve]  ::  UNRESOLVED DEPENDENCIES ::
[ivy:retrieve]  ::
[ivy:retrieve]  :: xpp3#xpp3;1.1.4c: not found
[ivy:retrieve]  ::
[ivy:retrieve]  ERRORS
[ivy:retrieve]  SERVER ERROR: HTTPS Required
url=http://repo1.maven.org/maven2/xpp3/xpp3/1.1.4c/xpp3-1.1.4c.pom
[ivy:retrieve]  SERVER ERROR: HTTPS Required
url=http://repo1.maven.org/maven2/xpp3/xpp3/1.1.4c/xpp3-1.1.4c.jar

So it seems that maven.org is only accepting https connections but I
can't find where to correct the above error

Any suggestions?

Thanks

Paul
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev



___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Splitter Build Failure

2020-01-30 Thread Gerd Petermann

Hi Paul,

please update, see 
http://www.mkgmap.org.uk/websvn/revision.php?repname=splitter=596

Gerd


Von: mkgmap-dev  im Auftrag von News 

Gesendet: Donnerstag, 30. Januar 2020 13:05
An: 'Development list for mkgmap'
Betreff: [mkgmap-dev] Splitter Build Failure

I recently clean reinstalled my PC (openSuSE Leap 15.1) and today was
the first time I tried to build splitter and received the following errors

BUILD FAILED
/home/miscellaneous/Maps/tile-splitter/splitter/build.xml:95: Can't get
http://repo1.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar to
/home/miscellaneous/Maps/tile-splitter/splitter/lib/build/ivy-2.2.0.jar

I changed the line in build.xml to use https instead of http and I then
got past this error and received the following error snippet

[ivy:retrieve]  ::
[ivy:retrieve]  ::  UNRESOLVED DEPENDENCIES ::
[ivy:retrieve]  ::
[ivy:retrieve]  :: xpp3#xpp3;1.1.4c: not found
[ivy:retrieve]  ::
[ivy:retrieve]  ERRORS
[ivy:retrieve]  SERVER ERROR: HTTPS Required
url=http://repo1.maven.org/maven2/xpp3/xpp3/1.1.4c/xpp3-1.1.4c.pom
[ivy:retrieve]  SERVER ERROR: HTTPS Required
url=http://repo1.maven.org/maven2/xpp3/xpp3/1.1.4c/xpp3-1.1.4c.jar

So it seems that maven.org is only accepting https connections but I
can't find where to correct the above error

Any suggestions?

Thanks

Paul
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter option? (or new request)

2019-02-23 Thread Gerd Petermann

Hi Dave,

no, pbf output compression is hardcoded at the moment. When I introduced o5m 
output format in splitter it was much faster than pbf but I think the 
difference is very small now after an update of the pbf libs, esp. the protobuf 
lib.
Anyway, try output=o5m to find out.

Gerd


Von: mkgmap-dev  im Auftrag von dave 
pitney 
Gesendet: Samstag, 23. Februar 2019 18:37
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: [mkgmap-dev] splitter option? (or new request)

Hello -

Is there is an option to not compress pbf output files when using splitter?
I use the split files as input to mkgmap.

Osmium has an option "pbf_compression=none" which is placed in the
json file as an option. This increases file size (appx double) but allows
faster file reading/writing.

Given the low cost of storage and the (relatively) high cost of
processing/time this may be a useful splitter option.

I am current using r591.

Thank you,
--
pitney

- end of message 
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter, mapid

2019-01-03 Thread Andrzej Popowski


Hi Gerd,

we can use any formula to assign mapid. If this formula would be widely 
accepted, there wouldn't be much conflicts between maps created by 
different developers.


If we use the original Garmin's formula, then we would avoid conflicts 
with many existing Garmin's maps too. The only requirement would be to 
avoid FIDs already used by Garmin.


As for my other divagation, it was about how to implement Garmin's 
formula. For full implementation, we would need some changes to mkgmap 
too. Without changes to mkgmap, we can implement forumla only for FIDs 
up to some limit. I have suggested, how this partial implementation 
could look like.


--
Best regards,
Andrzej
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter, mapid

2019-01-02 Thread Gerd Petermann

Hi Andrzej,

looked at this again now. What I don't understand is why you want to use
exactly the same formular
as Garmin. I think my approach would be to make sure that we DON'T use their
numbers to avoid conflicts.

Gerd


popej wrote
> Hi Gerd,
> 
> yes, I got the formula after examining numerous Garmin's maps. I know, 
> that City Navigator is different, my guess is that next CN map gets next 
> free mapid. I'm not sure about marine maps and other new maps from Garmin.
> 
> It looks like the formula is easy to support up to FID 45744. For bigger 
> values you could take different strategy. The rules I know are following:
> 
> Mapid inside img data is 4-byte value, no problem here.
> 
> File names are independent of mapid, could be any arbitrary 8-digit 
> number (number from file name could be even written in TRE header, at 
> offset 0xCF).
> 
> Names of subfiles like 12345678.TRE can be written in HEX, in this case 
> you should use prefix 'I'. So 12345678.TRE is equivalent of 
> I0BC614E.TRE. Mapid 140.531.712 could correspond to I8605800.TRE.
> 
> My first suggestion is to limit --fid to max 45744. For bigger FID 
> option --mapid should be used.
> 
> Second proposition is to overlap mapid values for big FIDs, but with an 
> offset of 1024. So for FID > 45744 formula could be like this:
> 
> mapid = (fid - 19792)*2048 + 6316032 + 1024
> 
> -- 
> Best regards,
> Andrzej
> ___
> mkgmap-dev mailing list

> mkgmap-dev@.org

> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev





--
Sent from: http://gis.19327.n8.nabble.com/Mkgmap-Development-f5324443.html
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter, mapid

2018-12-18 Thread Gerd Petermann

Hi Andrzej,

sorry, you totally lost me here. I fight with a cold since 8 days ;-)
I'll look at this again once I feel good. again.

Gerd


Von: mkgmap-dev  im Auftrag von Andrzej 
Popowski 
Gesendet: Dienstag, 18. Dezember 2018 14:14
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: Re: [mkgmap-dev] splitter, mapid

Hi Gerd,

yes, I got the formula after examining numerous Garmin's maps. I know,
that City Navigator is different, my guess is that next CN map gets next
free mapid. I'm not sure about marine maps and other new maps from Garmin.

It looks like the formula is easy to support up to FID 45744. For bigger
values you could take different strategy. The rules I know are following:

Mapid inside img data is 4-byte value, no problem here.

File names are independent of mapid, could be any arbitrary 8-digit
number (number from file name could be even written in TRE header, at
offset 0xCF).

Names of subfiles like 12345678.TRE can be written in HEX, in this case
you should use prefix 'I'. So 12345678.TRE is equivalent of
I0BC614E.TRE. Mapid 140.531.712 could correspond to I8605800.TRE.

My first suggestion is to limit --fid to max 45744. For bigger FID
option --mapid should be used.

Second proposition is to overlap mapid values for big FIDs, but with an
offset of 1024. So for FID > 45744 formula could be like this:

mapid = (fid - 19792)*2048 + 6316032 + 1024

--
Best regards,
Andrzej
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter, mapid

2018-12-18 Thread Andrzej Popowski


Hi Gerd,

yes, I got the formula after examining numerous Garmin's maps. I know, 
that City Navigator is different, my guess is that next CN map gets next 
free mapid. I'm not sure about marine maps and other new maps from Garmin.


It looks like the formula is easy to support up to FID 45744. For bigger 
values you could take different strategy. The rules I know are following:


Mapid inside img data is 4-byte value, no problem here.

File names are independent of mapid, could be any arbitrary 8-digit 
number (number from file name could be even written in TRE header, at 
offset 0xCF).


Names of subfiles like 12345678.TRE can be written in HEX, in this case 
you should use prefix 'I'. So 12345678.TRE is equivalent of 
I0BC614E.TRE. Mapid 140.531.712 could correspond to I8605800.TRE.


My first suggestion is to limit --fid to max 45744. For bigger FID 
option --mapid should be used.


Second proposition is to overlap mapid values for big FIDs, but with an 
offset of 1024. So for FID > 45744 formula could be like this:


mapid = (fid - 19792)*2048 + 6316032 + 1024

--
Best regards,
Andrzej
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter, mapid

2018-12-17 Thread Gerd Petermann

Hi Andrzej,

thanks for that formula. I like the idea to use the fid as option for splitter.
I'm not yet sure if I understand. Up to now we use values for mapid that are up 
to 99.999.999 so that the tile names can use the number.
Presuming that fid can be any value up to 65535 we get 2048 * 65535 + 6316032 = 
140.531.712. Should splitter change to hex in this case, e.g. 08605800.osm?
Can you explain the odd number 6316032? Did you find it by looking at the 
numbers used in Garmin maps?

Gerd


Von: mkgmap-dev  im Auftrag von Andrzej 
Popowski 
Gesendet: Montag, 17. Dezember 2018 15:17
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: Re: [mkgmap-dev] splitter, mapid

Hi,

basically map ID should be unique for all tiles, regardless of FID. If
you load 2 tiles with the same map ID to a GPS, it will silently ignore
one of these tiles. Most probably GPS assume, that these are the same
tiles and there is no reason to process both.

Search index contains map ID. If you use 2 maps with repeated map ID,
then most probably search won't work correctly.

There is no problem with map ID in Basecamp or Mapsource, these programs
don't support multiple maps.

I suggest to link map ID with FID using following formula:

First map ID in a mapset = FID*2048 + 6316032

Or in hex:
First map ID in a mapset = FID*800h + 606000h

This formula assigns 2048 IDs for each FID. It should be true also for
original Garmin's maps, except for City Navigator series.

Gerd, maybe you could include these formula into splitter? Something like:
;--fid=27795
:  Calculate mapid basing on a FID value. Can be used instead of
--mapid. Splitter sets first mapid value as 2048*fid + 6316032

--
Best regards,
Andrzej
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter, mapid

2018-12-17 Thread brad


65535,  0x

On 12/17/18 4:41 PM, brad wrote:
What is the max famid?   the docs say , but I know there are 
mapid's larger.  maybe 9?


On 12/17/18 7:17 AM, Andrzej Popowski wrote:

Hi,

basically map ID should be unique for all tiles, regardless of FID. 
If you load 2 tiles with the same map ID to a GPS, it will silently 
ignore one of these tiles. Most probably GPS assume, that these are 
the same tiles and there is no reason to process both.


Search index contains map ID. If you use 2 maps with repeated map ID, 
then most probably search won't work correctly.


There is no problem with map ID in Basecamp or Mapsource, these 
programs don't support multiple maps.


I suggest to link map ID with FID using following formula:

First map ID in a mapset = FID*2048 + 6316032

Or in hex:
First map ID in a mapset = FID*800h + 606000h

This formula assigns 2048 IDs for each FID. It should be true also 
for original Garmin's maps, except for City Navigator series.


Gerd, maybe you could include these formula into splitter? Something 
like:

;--fid=27795
:  Calculate mapid basing on a FID value. Can be used instead of 
--mapid. Splitter sets first mapid value as 2048*fid + 6316032




___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter, mapid

2018-12-17 Thread brad

What is the max famid?   the docs say , but I know there are mapid's 
larger.  maybe 9?


On 12/17/18 7:17 AM, Andrzej Popowski wrote:

Hi,

basically map ID should be unique for all tiles, regardless of FID. If 
you load 2 tiles with the same map ID to a GPS, it will silently 
ignore one of these tiles. Most probably GPS assume, that these are 
the same tiles and there is no reason to process both.


Search index contains map ID. If you use 2 maps with repeated map ID, 
then most probably search won't work correctly.


There is no problem with map ID in Basecamp or Mapsource, these 
programs don't support multiple maps.


I suggest to link map ID with FID using following formula:

First map ID in a mapset = FID*2048 + 6316032

Or in hex:
First map ID in a mapset = FID*800h + 606000h

This formula assigns 2048 IDs for each FID. It should be true also for 
original Garmin's maps, except for City Navigator series.


Gerd, maybe you could include these formula into splitter? Something 
like:

;--fid=27795
:  Calculate mapid basing on a FID value. Can be used instead of 
--mapid. Splitter sets first mapid value as 2048*fid + 6316032




___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter, mapid

2018-12-17 Thread Gerd Petermann

Hi Brad,

I think you should start with the sources in splitter\doc
You can get the source package from 
http://www.mkgmap.org.uk/download/splitter.html
or via svn with
svn co https://svn.mkgmap.org.uk/svn/splitter

Gerd


Von: brad 
Gesendet: Montag, 17. Dezember 2018 17:59
An: Gerd Petermann; Development list for mkgmap
Betreff: Re: AW: [mkgmap-dev] splitter, mapid

I'd be happy to take a stab at updating some docs.It looks like the
docs are the same on mkgmap.org and wiki.openstreetmap.org. Do the
changes on mkgmap.org flow to OSM, or the other way? Should I post
updates here?

http://www.mkgmap.org.uk/doc/splitter.html

On 12/16/18 11:57 PM, Gerd Petermann wrote:
> Hi all,
>
> I've never tried it, but my understanding was that you can have the same tile 
> (mapid) in two different gmapsupp files.
> E.g. you might create a tile containing an area around Hamburg and put that 
> tile into a gmapsupp for Germany as well as another
> one for Lower Saxony.
> If that is right it would explain why you have to make sure that the content 
> of two tiles with the same id doesn't differ.
>
> @Brad: I think the current documentation about the meaning of all those names 
> and ids is not perfect because the authors of mkgmap
> were still learning when the docu was created, and it's an ongoing process ;-)
> Feel free to improve the docu (post a patch or maybe a new version of one or 
> more files).
>
> Gerd
>
> 
> Von: mkgmap-dev  im Auftrag von brad 
> 
> Gesendet: Montag, 17. Dezember 2018 01:47
> An: mkgmap-dev@lists.mkgmap.org.uk
> Betreff: [mkgmap-dev] splitter, mapid
>
> One thing I think I missed, is that each tile on a garmin device needs a
> unique mapid.   A different family-id and mapname for each gmap is not
> sufficient.   Does this sound correct?
>
> I'd been downloading US states from Geofabrik, creating maps with unique
> family-id, mapname, every name & id that seemed important, (with
> mkgmap)  but when I put them on the Garmin they all wouldn't show up.
> Only when I went back to the splitter step & used a different mapid did
> they all show up.
>
> Sometimes I'm not too good at reading directions,  did I overlook this
> someplace?
>
> Brad
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter, mapid

2018-12-17 Thread brad

I'd be happy to take a stab at updating some docs.    It looks like the 
docs are the same on mkgmap.org and wiki.openstreetmap.org. Do the 
changes on mkgmap.org flow to OSM, or the other way? Should I post 
updates here?


http://www.mkgmap.org.uk/doc/splitter.html

On 12/16/18 11:57 PM, Gerd Petermann wrote:

Hi all,

I've never tried it, but my understanding was that you can have the same tile 
(mapid) in two different gmapsupp files.
E.g. you might create a tile containing an area around Hamburg and put that 
tile into a gmapsupp for Germany as well as another
one for Lower Saxony.
If that is right it would explain why you have to make sure that the content of 
two tiles with the same id doesn't differ.

@Brad: I think the current documentation about the meaning of all those names 
and ids is not perfect because the authors of mkgmap
were still learning when the docu was created, and it's an ongoing process ;-)
Feel free to improve the docu (post a patch or maybe a new version of one or 
more files).

Gerd


Von: mkgmap-dev  im Auftrag von brad 

Gesendet: Montag, 17. Dezember 2018 01:47
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: [mkgmap-dev] splitter, mapid

One thing I think I missed, is that each tile on a garmin device needs a
unique mapid.   A different family-id and mapname for each gmap is not
sufficient.   Does this sound correct?

I'd been downloading US states from Geofabrik, creating maps with unique
family-id, mapname, every name & id that seemed important, (with
mkgmap)  but when I put them on the Garmin they all wouldn't show up.
Only when I went back to the splitter step & used a different mapid did
they all show up.

Sometimes I'm not too good at reading directions,  did I overlook this
someplace?

Brad
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter, mapid

2018-12-17 Thread Andrzej Popowski


Hi,

basically map ID should be unique for all tiles, regardless of FID. If 
you load 2 tiles with the same map ID to a GPS, it will silently ignore 
one of these tiles. Most probably GPS assume, that these are the same 
tiles and there is no reason to process both.


Search index contains map ID. If you use 2 maps with repeated map ID, 
then most probably search won't work correctly.


There is no problem with map ID in Basecamp or Mapsource, these programs 
don't support multiple maps.


I suggest to link map ID with FID using following formula:

First map ID in a mapset = FID*2048 + 6316032

Or in hex:
First map ID in a mapset = FID*800h + 606000h

This formula assigns 2048 IDs for each FID. It should be true also for 
original Garmin's maps, except for City Navigator series.


Gerd, maybe you could include these formula into splitter? Something like:
;--fid=27795
:  Calculate mapid basing on a FID value. Can be used instead of 
--mapid. Splitter sets first mapid value as 2048*fid + 6316032


--
Best regards,
Andrzej
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter, mapid

2018-12-17 Thread Thorsten Kukuk



Hi,

On Mon, Dec 17, Gerd Petermann wrote:

> Hi all,
> 
> I've never tried it, but my understanding was that you can have the same tile 
> (mapid) in two different gmapsupp files.
> E.g. you might create a tile containing an area around Hamburg and put that 
> tile into a gmapsupp for Germany as well as another
> one for Lower Saxony.
> If that is right it would explain why you have to make sure that the content 
> of two tiles with the same id doesn't differ.

Correct, this works, but only if the tile is 100% identical for both maps.
Or in other words: create a big map and select tiles from this map to build 
smaller
maps. So the same as if you select tiles in mapsource from a bigger map and 
transfer
them to the GPS device.

  Thorsten

> @Brad: I think the current documentation about the meaning of all those names 
> and ids is not perfect because the authors of mkgmap
> were still learning when the docu was created, and it's an ongoing process ;-)
> Feel free to improve the docu (post a patch or maybe a new version of one or 
> more files).
> 
> Gerd
> 
> 
> Von: mkgmap-dev  im Auftrag von brad 
> 
> Gesendet: Montag, 17. Dezember 2018 01:47
> An: mkgmap-dev@lists.mkgmap.org.uk
> Betreff: [mkgmap-dev] splitter, mapid
> 
> One thing I think I missed, is that each tile on a garmin device needs a
> unique mapid.   A different family-id and mapname for each gmap is not
> sufficient.   Does this sound correct?
> 
> I'd been downloading US states from Geofabrik, creating maps with unique
> family-id, mapname, every name & id that seemed important, (with
> mkgmap)  but when I put them on the Garmin they all wouldn't show up.
> Only when I went back to the splitter step & used a different mapid did
> they all show up.
> 
> Sometimes I'm not too good at reading directions,  did I overlook this
> someplace?
> 
> Brad
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

-- 
Thorsten Kukuk, Distinguished Engineer, Senior Architect SLES & MicroOS
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany
GF: Felix Imendoerffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nuernberg)
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter, mapid

2018-12-16 Thread osm@pinns


Hi Gerd

I've learnt this lesson the hard way, walking through france with 2 maps 
sharing the same map id :( None of them worked!


Similarly,if you want your Recent Findings to work, make sure you use 
the same map ids else you get the following message:


'This route does not match the available maps'

I would also add that more than likely the FID has to be the same .

It appears Garmin attaches the tile number (and FID) to each POI search 
result.


r

Nick

On 17/12/2018 06:57, Gerd Petermann wrote:

Hi all,

I've never tried it, but my understanding was that you can have the same tile 
(mapid) in two different gmapsupp files.
E.g. you might create a tile containing an area around Hamburg and put that 
tile into a gmapsupp for Germany as well as another
one for Lower Saxony.
If that is right it would explain why you have to make sure that the content of 
two tiles with the same id doesn't differ.

@Brad: I think the current documentation about the meaning of all those names 
and ids is not perfect because the authors of mkgmap
were still learning when the docu was created, and it's an ongoing process ;-)
Feel free to improve the docu (post a patch or maybe a new version of one or 
more files).

Gerd


Von: mkgmap-dev  im Auftrag von brad 

Gesendet: Montag, 17. Dezember 2018 01:47
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: [mkgmap-dev] splitter, mapid

One thing I think I missed, is that each tile on a garmin device needs a
unique mapid.   A different family-id and mapname for each gmap is not
sufficient.   Does this sound correct?

I'd been downloading US states from Geofabrik, creating maps with unique
family-id, mapname, every name & id that seemed important, (with
mkgmap)  but when I put them on the Garmin they all wouldn't show up.
Only when I went back to the splitter step & used a different mapid did
they all show up.

Sometimes I'm not too good at reading directions,  did I overlook this
someplace?

Brad
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter, mapid

2018-12-16 Thread Gerd Petermann

Hi all,

I've never tried it, but my understanding was that you can have the same tile 
(mapid) in two different gmapsupp files.
E.g. you might create a tile containing an area around Hamburg and put that 
tile into a gmapsupp for Germany as well as another
one for Lower Saxony.
If that is right it would explain why you have to make sure that the content of 
two tiles with the same id doesn't differ.

@Brad: I think the current documentation about the meaning of all those names 
and ids is not perfect because the authors of mkgmap
were still learning when the docu was created, and it's an ongoing process ;-)
Feel free to improve the docu (post a patch or maybe a new version of one or 
more files).

Gerd


Von: mkgmap-dev  im Auftrag von brad 

Gesendet: Montag, 17. Dezember 2018 01:47
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: [mkgmap-dev] splitter, mapid

One thing I think I missed, is that each tile on a garmin device needs a
unique mapid.   A different family-id and mapname for each gmap is not
sufficient.   Does this sound correct?

I'd been downloading US states from Geofabrik, creating maps with unique
family-id, mapname, every name & id that seemed important, (with
mkgmap)  but when I put them on the Garmin they all wouldn't show up.
Only when I went back to the splitter step & used a different mapid did
they all show up.

Sometimes I'm not too good at reading directions,  did I overlook this
someplace?

Brad
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter: option for maximum tile area?

2018-11-06 Thread Gerd Petermann

Hi Bernhard,

thanks for updating the Srtm2OSM wiki. I think it could still be improved, for 
example I did not find the -maxwaynodes option
when I just looked for it, only found it using the search function of the 
browser.

Reg. your other findings:
I know these pros and cons for a max-nodes value different to the default:

A larger --max-nodes value can help when you want to split a very large file (a 
large continent extract or planet) because
- splitter might need less memory because it reduces the number of tiles and 
thus the number of ways crossing tile boundaries
which are handled by the --keep-complete processing
- the smaller number of tiles allows to avoid a limit in the gmapsupp, if I 
remember correctly max 4000 tiles are allowed in one mapset
- the overall amount of data in the split files is smaller because less data is 
repeated in the areas outside the bounding boxes

A lower --max-nodes value can help because
- it avoids to reach several limits of the img format, esp. this one: "There is 
not enough room in a single garmin map for all the input data."
- it reduces the amount of memory needed in mkgmap to process one tile.
This can be important when multiple threads start to steal memory from each 
other and thus decrease throughput
- it allows to open and check the tile in JOSM (presuming you have the pbf or 
o5m plugin installed)
So, finding a perfet value depends on style and hardware and what mapset you 
plan to create.

The expected effect of the --precomp-sea option in splitter is that it produces 
smaller tiles in areas with many islands because it will add extra weight
to the coastline nodes.

Gerd




Von: mkgmap-dev  im Auftrag von 
Bernhard Hiller 
Gesendet: Dienstag, 6. November 2018 14:54
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?

Thanks all,
meanwhile I could generate a map which looks fine in QLandkarte and on
my Oregon.
I'd like to summarize the findings for those who try to create contour
lines with srtm2osm and feed them into a Garmin map. I updated the wiki
page https://wiki.openstreetmap.org/wiki/Srtm2Osm

First of all, it is possible to use srtm2osm with elevation data
downloaded from Viewfinderpanoramas and unzipped into its cache folder.
In contrast to what one might expect, the folder where the *.hgt files
must be placed is not the folder indicated with the -d option, but a
folder "SrtmCache" below it (that's hard coded in the source code of
srtm2osm).
Next, the default start ids for nodes and ways generated by the program
is long.MaxValue, which is a reserved value in splitter, thus causing an
exception (see a previous thread:
http://www.mkgmap.org.uk/pipermail/mkgmap-dev/2018q4/029073.html ). The
options -firstnodeid 9023372036854775807 -firstwayid 9023372036854775807
(or other high values less than long.MaxValue) remove this issue.
It is important to note that a misspelled option will not cause an error
message by srtm2osm - that option is merely ignored. So do not feel
surprised when a problem later on a step later on just happened again,
though you thought that you changed something before...

Another issue is the unlimited size of ways. In OSM data, ways longer
than 2000 nodes do likely not exist because the API prevents the upload
of such long ways. But srtm2osm produced ways with more than a hundred
thousand nodes. splitter tries to keep ways complete. The long ways lead
to a slow processing by splitter, and to an extremely inflated size of
the tiles created by splitter (if the original file and the tiles have
the same format, the sum of file sizes of the tiles should not be much
more than the file size of the original file): upto 9 times the original
size.
There after, mkgmap crashes with an OutOfMemory error.
The solution is to limit the file size with option -maxwaynodes 5000 (or
an even lower value: the files were still inflated by about a half).

Adding the option --precomp-sea to the splitter options is irrelevant
for the OutOfMemory exception in mkgmap. Also, the value of the
--max-nodes splitter option proofed irrelevant for the exception.
Running mkgmap with --max-jobs=3 instead of 4 was also irrelevant.

But running splitter with a high value of --max-nodes in combination
with --precomp-sea lead to a different exception in mkgmap:
(MapFailedException): 47120100.osm.pbf: (thrown in
RoadDef.writeRgnOffsets()) Overflow of the NET1. The tile
(47120100.osm.pbf) must be split so that there are fewer roads in it
Then, that tile was left empty. I could see this issue with big cities
(Bangkok, Kuala Lumpur) now, but never experienced it before (I created
a map of Central Europe with a value of 3 million without precomp-sea -
absolutely no problem).

I hope this information won't be outdated when some else tries to follow
these steps.

Kind regards,
Bernhard

Am 31.10.2018 um 22:03 schrieb Bernhard Hiller:
> Hi all,
> currently a

Re: [mkgmap-dev] splitter: option for maximum tile area?

2018-11-06 Thread Bernhard Hiller


Thanks all,
meanwhile I could generate a map which looks fine in QLandkarte and on 
my Oregon.
I'd like to summarize the findings for those who try to create contour 
lines with srtm2osm and feed them into a Garmin map. I updated the wiki 
page https://wiki.openstreetmap.org/wiki/Srtm2Osm


First of all, it is possible to use srtm2osm with elevation data 
downloaded from Viewfinderpanoramas and unzipped into its cache folder.
In contrast to what one might expect, the folder where the *.hgt files 
must be placed is not the folder indicated with the -d option, but a 
folder "SrtmCache" below it (that's hard coded in the source code of 
srtm2osm).
Next, the default start ids for nodes and ways generated by the program 
is long.MaxValue, which is a reserved value in splitter, thus causing an 
exception (see a previous thread: 
http://www.mkgmap.org.uk/pipermail/mkgmap-dev/2018q4/029073.html ). The 
options -firstnodeid 9023372036854775807 -firstwayid 9023372036854775807

(or other high values less than long.MaxValue) remove this issue.
It is important to note that a misspelled option will not cause an error 
message by srtm2osm - that option is merely ignored. So do not feel 
surprised when a problem later on a step later on just happened again, 
though you thought that you changed something before...


Another issue is the unlimited size of ways. In OSM data, ways longer 
than 2000 nodes do likely not exist because the API prevents the upload 
of such long ways. But srtm2osm produced ways with more than a hundred 
thousand nodes. splitter tries to keep ways complete. The long ways lead 
to a slow processing by splitter, and to an extremely inflated size of 
the tiles created by splitter (if the original file and the tiles have 
the same format, the sum of file sizes of the tiles should not be much 
more than the file size of the original file): upto 9 times the original 
size.

There after, mkgmap crashes with an OutOfMemory error.
The solution is to limit the file size with option -maxwaynodes 5000 (or 
an even lower value: the files were still inflated by about a half).


Adding the option --precomp-sea to the splitter options is irrelevant 
for the OutOfMemory exception in mkgmap. Also, the value of the 
--max-nodes splitter option proofed irrelevant for the exception. 
Running mkgmap with --max-jobs=3 instead of 4 was also irrelevant.


But running splitter with a high value of --max-nodes in combination 
with --precomp-sea lead to a different exception in mkgmap:
(MapFailedException): 47120100.osm.pbf: (thrown in 
RoadDef.writeRgnOffsets()) Overflow of the NET1. The tile 
(47120100.osm.pbf) must be split so that there are fewer roads in it
Then, that tile was left empty. I could see this issue with big cities 
(Bangkok, Kuala Lumpur) now, but never experienced it before (I created 
a map of Central Europe with a value of 3 million without precomp-sea - 
absolutely no problem).


I hope this information won't be outdated when some else tries to follow 
these steps.


Kind regards,
Bernhard

Am 31.10.2018 um 22:03 schrieb Bernhard Hiller:

Hi all,
currently a Java OutOfMemory exception prevents me from creating a 
map. I already use option --max-jobs=3 (the machine has 4 physical 
cores) and -Xmx5G (of 8 GB installed). Beyond OSM data, the map 
contains DEM and elevation contour lines.
From the tiles finished and those with a new timestamp but about 0 
bytes length, I can see that mkgmap was rendering tiles 47120005, 
47120006, 47120007 at the time of crash.
Tile 47120005 is extremely large by physical area - some 6° x 5.5° 
(see attached file), covering a lot of the south chinese sea, i.e. 
there are not many actual data in that area.
I guess that the problem arises with that tile. I remember some case 
in the past where a single tile covering such a large area of mainly 
sea caused mkgmap to take an enormous amount of time for rendering - 
also here, mkgmap already spent about 1 hour before crashing.
So I'd like to ask: is there some possibility for limiting the area of 
a tile among the splitter options?

Kind regards,
Bernhard


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter: option for maximum tile area?

2018-11-04 Thread Gerd Petermann

Ansonsten ist es so, wie ich vermutet habe. Srtm2OSM erzeugt elend lange Wege 
mit z.B. 184.962 Punkten. Die laufen dann natürlich weit über die Bounding Box 
hinaus und machen es wohl auch mkgmap schwer, weil die Wege erst recht spät 
abgeschnitten werden.
Fazit:
Das Problem ist in Srtm2OSM, versuch es mal mit der Option  -maxwaynodes=5000 
oder noch kleiner.

Gerd


Von: mkgmap-dev  im Auftrag von Gerd 
Petermann 
Gesendet: Sonntag, 4. November 2018 11:26
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?

Hallo Bernhard,

wolltest Du nicht max-nodes verringern? Jetzt ist es sogar 3.000.000 anstatt 
1.400.000

Gerd


Von: mkgmap-dev  im Auftrag von 
Bernhard Hiller 
Gesendet: Sonntag, 4. November 2018 11:19
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?

Hi Gerd,
I've uploaded some files:
splitter log:
https://drive.google.com/open?id=12wsnncNkfwKruEw-4UIgmvddK3E_TxtE
kml: https://drive.google.com/open?id=1UOv_FTtl_pK_Nj126WDFnirhaswnvyfZ
smallest tile:
https://drive.google.com/open?id=1_yDKKpALKSfiRTiicCrEOYgH1vKmnm4w
largest tile:
https://drive.google.com/open?id=1aidryuJuhixrJIHHou3sibbBNBpRWJeN
Kind regards,
Bernhard

Am 03.11.2018 um 11:38 schrieb Gerd Petermann:
> Hi Bernhard,
>
> please post a link to one of the files produced by splitter. If the sum of 
> file sizes is much larger than the input file that means that each file
> contains a lot of data that was written to keep ways complete. Maybe srtm2osm 
> creates lots of very long ways ?
>
> Gerd
>
> 
> Von: mkgmap-dev  im Auftrag von 
> Bernhard Hiller 
> Gesendet: Samstag, 3. November 2018 11:25
> An: mkgmap-dev@lists.mkgmap.org.uk
> Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?
>
> Hi Gerd,
> contour lines were created with srtm2osm  -step 25 -cat 500 100 25
> -large with 3" elevation data downloaded from viewfinderpanoramas.
> The mkgmap parameters contain "--add-pois-to-areas", but no
> "--add-pois-to-lines" (the splitter parameters contain neither - as Ralf
> said).
> The style for contour lines is simple:
> contour=evation & ele<=0 { delete contour; }
> contour=evation & contour_ext=elevation_major  { name
> '${ele|conv:m=t}'; } [0x22 level 4]
> contour=evation & contour_ext=elevation_medium { name
> '${ele|conv:m=t}'; } [0x21 level 2]
> contour=evation & contour_ext=elevation_minor  { name
> '${ele|conv:m=t}'; } [0x20 level 0]
> There are no rules for contours in the points and polygons file. The
> options file defines the levels only.
>
> Running splitter on the contour lines file only, creates several pdb
> files summing up to 4.5 GB
> =the problem is with the contour lines.
>
> Next, I'll try to cut Laos from the contour lines file and create a
> contour lines map of Laos only...
>
> Kind regards,
> Bernhard
>
> Am 02.11.2018 um 19:31 schrieb Gerd Petermann:
>> Hi Bernhard,
>>
>> just a guess: If you use option --add-pois-to-lines and you have a rule in 
>> the points file which processes ele to add a POI
>> you may see something like that.
>>
>> Gerd
>>
>> 
>> Von: mkgmap-dev  im Auftrag von Gerd 
>> Petermann 
>> Gesendet: Freitag, 2. November 2018 19:24
>> An: Bernhard Hiller; Development list for mkgmap
>> Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?
>>
>> Hi Bernhard,
>>
>> what are your rules for the contour lines?
>>
>> Gerd
>>
>> 
>> Von: mkgmap-dev  im Auftrag von 
>> Bernhard Hiller 
>> Gesendet: Freitag, 2. November 2018 19:18
>> An: mkgmap-dev@lists.mkgmap.org.uk
>> Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?
>>
>> Hi all,
>> still struggling with that strange issue. But I guess I found some hint
>> to the cause: inconsistent file sizes.
>> - extracted OSM data: 400 MB pbf
>> - elevation contour lines: 600 MB pbf
>> - merged file: 1 GB pbf
>> So far, sizes are consistent.
>>
>> When I run splitter on the OSM data only file, it produces many tiles
>> summing up to some 400 MB. Consistent, too. When I run mkgmap on this
>> output, I get a map within about half an hour, and it looks OK (tested
>> with QLandkarte).
>>
>> When I run splitter on the merged file, the tiles sum up to some 9 GB.
>> That is 9 times the size I'd expect. mkgmap can render several of those
>> tiles (but very slowly); and then crashes on one of them with an
>

Re: [mkgmap-dev] splitter: option for maximum tile area?

2018-11-04 Thread Gerd Petermann

Hallo Bernhard,

wolltest Du nicht max-nodes verringern? Jetzt ist es sogar 3.000.000 anstatt 
1.400.000

Gerd


Von: mkgmap-dev  im Auftrag von 
Bernhard Hiller 
Gesendet: Sonntag, 4. November 2018 11:19
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?

Hi Gerd,
I've uploaded some files:
splitter log:
https://drive.google.com/open?id=12wsnncNkfwKruEw-4UIgmvddK3E_TxtE
kml: https://drive.google.com/open?id=1UOv_FTtl_pK_Nj126WDFnirhaswnvyfZ
smallest tile:
https://drive.google.com/open?id=1_yDKKpALKSfiRTiicCrEOYgH1vKmnm4w
largest tile:
https://drive.google.com/open?id=1aidryuJuhixrJIHHou3sibbBNBpRWJeN
Kind regards,
Bernhard

Am 03.11.2018 um 11:38 schrieb Gerd Petermann:
> Hi Bernhard,
>
> please post a link to one of the files produced by splitter. If the sum of 
> file sizes is much larger than the input file that means that each file
> contains a lot of data that was written to keep ways complete. Maybe srtm2osm 
> creates lots of very long ways ?
>
> Gerd
>
> 
> Von: mkgmap-dev  im Auftrag von 
> Bernhard Hiller 
> Gesendet: Samstag, 3. November 2018 11:25
> An: mkgmap-dev@lists.mkgmap.org.uk
> Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?
>
> Hi Gerd,
> contour lines were created with srtm2osm  -step 25 -cat 500 100 25
> -large with 3" elevation data downloaded from viewfinderpanoramas.
> The mkgmap parameters contain "--add-pois-to-areas", but no
> "--add-pois-to-lines" (the splitter parameters contain neither - as Ralf
> said).
> The style for contour lines is simple:
> contour=evation & ele<=0 { delete contour; }
> contour=evation & contour_ext=elevation_major  { name
> '${ele|conv:m=t}'; } [0x22 level 4]
> contour=evation & contour_ext=elevation_medium { name
> '${ele|conv:m=t}'; } [0x21 level 2]
> contour=evation & contour_ext=elevation_minor  { name
> '${ele|conv:m=t}'; } [0x20 level 0]
> There are no rules for contours in the points and polygons file. The
> options file defines the levels only.
>
> Running splitter on the contour lines file only, creates several pdb
> files summing up to 4.5 GB
> =the problem is with the contour lines.
>
> Next, I'll try to cut Laos from the contour lines file and create a
> contour lines map of Laos only...
>
> Kind regards,
> Bernhard
>
> Am 02.11.2018 um 19:31 schrieb Gerd Petermann:
>> Hi Bernhard,
>>
>> just a guess: If you use option --add-pois-to-lines and you have a rule in 
>> the points file which processes ele to add a POI
>> you may see something like that.
>>
>> Gerd
>>
>> 
>> Von: mkgmap-dev  im Auftrag von Gerd 
>> Petermann 
>> Gesendet: Freitag, 2. November 2018 19:24
>> An: Bernhard Hiller; Development list for mkgmap
>> Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?
>>
>> Hi Bernhard,
>>
>> what are your rules for the contour lines?
>>
>> Gerd
>>
>> 
>> Von: mkgmap-dev  im Auftrag von 
>> Bernhard Hiller 
>> Gesendet: Freitag, 2. November 2018 19:18
>> An: mkgmap-dev@lists.mkgmap.org.uk
>> Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?
>>
>> Hi all,
>> still struggling with that strange issue. But I guess I found some hint
>> to the cause: inconsistent file sizes.
>> - extracted OSM data: 400 MB pbf
>> - elevation contour lines: 600 MB pbf
>> - merged file: 1 GB pbf
>> So far, sizes are consistent.
>>
>> When I run splitter on the OSM data only file, it produces many tiles
>> summing up to some 400 MB. Consistent, too. When I run mkgmap on this
>> output, I get a map within about half an hour, and it looks OK (tested
>> with QLandkarte).
>>
>> When I run splitter on the merged file, the tiles sum up to some 9 GB.
>> That is 9 times the size I'd expect. mkgmap can render several of those
>> tiles (but very slowly); and then crashes on one of them with an
>> OutOfMemory exception.
>>
>> So I think that the problem is somewhere in those contour lines, either
>> when merged or alone.
>> I'll try to create a contour lines only map as the next step to test
>> this hypothesis.
>>
>> Kind regards,
>> Bernhard
>>
>> Am 01.11.2018 um 09:32 schrieb Gerd Petermann:
>>> Hi Bernhard,
>>>
>>> I tried to reproduce the memory problems with tile 47120005. I don't think 
>>> that the sea itself is a problem here, at least not when you use the 
>>> --pr

Re: [mkgmap-dev] splitter: option for maximum tile area?

2018-11-04 Thread Bernhard Hiller


Hi Gerd,
I've uploaded some files:
splitter log: 
https://drive.google.com/open?id=12wsnncNkfwKruEw-4UIgmvddK3E_TxtE

kml: https://drive.google.com/open?id=1UOv_FTtl_pK_Nj126WDFnirhaswnvyfZ
smallest tile: 
https://drive.google.com/open?id=1_yDKKpALKSfiRTiicCrEOYgH1vKmnm4w
largest tile: 
https://drive.google.com/open?id=1aidryuJuhixrJIHHou3sibbBNBpRWJeN

Kind regards,
Bernhard

Am 03.11.2018 um 11:38 schrieb Gerd Petermann:

Hi Bernhard,

please post a link to one of the files produced by splitter. If the sum of file 
sizes is much larger than the input file that means that each file
contains a lot of data that was written to keep ways complete. Maybe srtm2osm 
creates lots of very long ways ?

Gerd


Von: mkgmap-dev  im Auftrag von Bernhard 
Hiller 
Gesendet: Samstag, 3. November 2018 11:25
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?

Hi Gerd,
contour lines were created with srtm2osm  -step 25 -cat 500 100 25
-large with 3" elevation data downloaded from viewfinderpanoramas.
The mkgmap parameters contain "--add-pois-to-areas", but no
"--add-pois-to-lines" (the splitter parameters contain neither - as Ralf
said).
The style for contour lines is simple:
contour=evation & ele<=0 { delete contour; }
contour=evation & contour_ext=elevation_major  { name
'${ele|conv:m=t}'; } [0x22 level 4]
contour=evation & contour_ext=elevation_medium { name
'${ele|conv:m=t}'; } [0x21 level 2]
contour=evation & contour_ext=elevation_minor  { name
'${ele|conv:m=t}'; } [0x20 level 0]
There are no rules for contours in the points and polygons file. The
options file defines the levels only.

Running splitter on the contour lines file only, creates several pdb
files summing up to 4.5 GB
=the problem is with the contour lines.

Next, I'll try to cut Laos from the contour lines file and create a
contour lines map of Laos only...

Kind regards,
Bernhard

Am 02.11.2018 um 19:31 schrieb Gerd Petermann:

Hi Bernhard,

just a guess: If you use option --add-pois-to-lines and you have a rule in the 
points file which processes ele to add a POI
you may see something like that.

Gerd


Von: mkgmap-dev  im Auftrag von Gerd 
Petermann 
Gesendet: Freitag, 2. November 2018 19:24
An: Bernhard Hiller; Development list for mkgmap
Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?

Hi Bernhard,

what are your rules for the contour lines?

Gerd


Von: mkgmap-dev  im Auftrag von Bernhard 
Hiller 
Gesendet: Freitag, 2. November 2018 19:18
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?

Hi all,
still struggling with that strange issue. But I guess I found some hint
to the cause: inconsistent file sizes.
- extracted OSM data: 400 MB pbf
- elevation contour lines: 600 MB pbf
- merged file: 1 GB pbf
So far, sizes are consistent.

When I run splitter on the OSM data only file, it produces many tiles
summing up to some 400 MB. Consistent, too. When I run mkgmap on this
output, I get a map within about half an hour, and it looks OK (tested
with QLandkarte).

When I run splitter on the merged file, the tiles sum up to some 9 GB.
That is 9 times the size I'd expect. mkgmap can render several of those
tiles (but very slowly); and then crashes on one of them with an
OutOfMemory exception.

So I think that the problem is somewhere in those contour lines, either
when merged or alone.
I'll try to create a contour lines only map as the next step to test
this hypothesis.

Kind regards,
Bernhard

Am 01.11.2018 um 09:32 schrieb Gerd Petermann:

Hi Bernhard,

I tried to reproduce the memory problems with tile 47120005. I don't think that 
the sea itself is a problem here, at least not when you use the --precomp-sea 
option. It took only 15 secs to process that tile with asia data from August 
with the default style and typical options for routing etc.
My input file didn't contain SRTM data so I assume this is the reason. Maybe 
you have many contour lines with ele=our data?

Gerd


Von: Gerd Petermann 
Gesendet: Mittwoch, 31. Oktober 2018 22:49
An: Gerd Petermann; Development list for mkgmap
Betreff: AW: [mkgmap-dev] splitter: option for maximum tile area?

Hi Bernhard,

looked again at the splitter command in your last post. You also use a rather 
high max-nodes value.
Such a high value means that you get rather large tiles and that mkgmap needs 
more memory for each tile
compared to the default 140. Many users use a value near 120.

Gerd


Von: mkgmap-dev  im Auftrag von Gerd 
Petermann 
Gesendet: Mittwoch, 31. Oktober 2018 22:20
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?

Hi Bernhard,
there is no option for this. Do you use option --preco

Re: [mkgmap-dev] splitter: option for maximum tile area?

2018-11-03 Thread Gerd Petermann

Hi Bernhard,

please post a link to one of the files produced by splitter. If the sum of file 
sizes is much larger than the input file that means that each file
contains a lot of data that was written to keep ways complete. Maybe srtm2osm 
creates lots of very long ways ?

Gerd


Von: mkgmap-dev  im Auftrag von 
Bernhard Hiller 
Gesendet: Samstag, 3. November 2018 11:25
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?

Hi Gerd,
contour lines were created with srtm2osm  -step 25 -cat 500 100 25
-large with 3" elevation data downloaded from viewfinderpanoramas.
The mkgmap parameters contain "--add-pois-to-areas", but no
"--add-pois-to-lines" (the splitter parameters contain neither - as Ralf
said).
The style for contour lines is simple:
contour=elevation & ele<=0 { delete contour; }
contour=elevation & contour_ext=elevation_major  { name
'${ele|conv:m=>ft}'; } [0x22 level 4]
contour=elevation & contour_ext=elevation_medium { name
'${ele|conv:m=>ft}'; } [0x21 level 2]
contour=elevation & contour_ext=elevation_minor  { name
'${ele|conv:m=>ft}'; } [0x20 level 0]
There are no rules for contours in the points and polygons file. The
options file defines the levels only.

Running splitter on the contour lines file only, creates several pdb
files summing up to 4.5 GB
=> the problem is with the contour lines.

Next, I'll try to cut Laos from the contour lines file and create a
contour lines map of Laos only...

Kind regards,
Bernhard

Am 02.11.2018 um 19:31 schrieb Gerd Petermann:
> Hi Bernhard,
>
> just a guess: If you use option --add-pois-to-lines and you have a rule in 
> the points file which processes ele to add a POI
> you may see something like that.
>
> Gerd
>
> 
> Von: mkgmap-dev  im Auftrag von Gerd 
> Petermann 
> Gesendet: Freitag, 2. November 2018 19:24
> An: Bernhard Hiller; Development list for mkgmap
> Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?
>
> Hi Bernhard,
>
> what are your rules for the contour lines?
>
> Gerd
>
> 
> Von: mkgmap-dev  im Auftrag von 
> Bernhard Hiller 
> Gesendet: Freitag, 2. November 2018 19:18
> An: mkgmap-dev@lists.mkgmap.org.uk
> Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?
>
> Hi all,
> still struggling with that strange issue. But I guess I found some hint
> to the cause: inconsistent file sizes.
> - extracted OSM data: 400 MB pbf
> - elevation contour lines: 600 MB pbf
> - merged file: 1 GB pbf
> So far, sizes are consistent.
>
> When I run splitter on the OSM data only file, it produces many tiles
> summing up to some 400 MB. Consistent, too. When I run mkgmap on this
> output, I get a map within about half an hour, and it looks OK (tested
> with QLandkarte).
>
> When I run splitter on the merged file, the tiles sum up to some 9 GB.
> That is 9 times the size I'd expect. mkgmap can render several of those
> tiles (but very slowly); and then crashes on one of them with an
> OutOfMemory exception.
>
> So I think that the problem is somewhere in those contour lines, either
> when merged or alone.
> I'll try to create a contour lines only map as the next step to test
> this hypothesis.
>
> Kind regards,
> Bernhard
>
> Am 01.11.2018 um 09:32 schrieb Gerd Petermann:
>> Hi Bernhard,
>>
>> I tried to reproduce the memory problems with tile 47120005. I don't think 
>> that the sea itself is a problem here, at least not when you use the 
>> --precomp-sea option. It took only 15 secs to process that tile with asia 
>> data from August with the default style and typical options for routing etc.
>> My input file didn't contain SRTM data so I assume this is the reason. Maybe 
>> you have many contour lines with ele= your data?
>>
>> Gerd
>>
>> 
>> Von: Gerd Petermann 
>> Gesendet: Mittwoch, 31. Oktober 2018 22:49
>> An: Gerd Petermann; Development list for mkgmap
>> Betreff: AW: [mkgmap-dev] splitter: option for maximum tile area?
>>
>> Hi Bernhard,
>>
>> looked again at the splitter command in your last post. You also use a 
>> rather high max-nodes value.
>> Such a high value means that you get rather large tiles and that mkgmap 
>> needs more memory for each tile
>> compared to the default 140. Many users use a value near 120.
>>
>> Gerd
>>
>> 
>> Von: mkgmap-dev  im Auftrag von Gerd 
>> Petermann 
>> Gesendet: Mittwoch, 31. Oktober 2018 22:20
>> An: Development list for mkg

Re: [mkgmap-dev] splitter: option for maximum tile area?

2018-11-03 Thread Bernhard Hiller


Hi Gerd,
contour lines were created with srtm2osm  -step 25 -cat 500 100 25 
-large with 3" elevation data downloaded from viewfinderpanoramas.
The mkgmap parameters contain "--add-pois-to-areas", but no 
"--add-pois-to-lines" (the splitter parameters contain neither - as Ralf 
said).

The style for contour lines is simple:
contour=elevation & ele<=0 { delete contour; }
contour=elevation & contour_ext=elevation_major  { name 
'${ele|conv:m=>ft}'; } [0x22 level 4]
contour=elevation & contour_ext=elevation_medium { name 
'${ele|conv:m=>ft}'; } [0x21 level 2]
contour=elevation & contour_ext=elevation_minor  { name 
'${ele|conv:m=>ft}'; } [0x20 level 0]
There are no rules for contours in the points and polygons file. The 
options file defines the levels only.


Running splitter on the contour lines file only, creates several pdb 
files summing up to 4.5 GB

=> the problem is with the contour lines.

Next, I'll try to cut Laos from the contour lines file and create a 
contour lines map of Laos only...


Kind regards,
Bernhard

Am 02.11.2018 um 19:31 schrieb Gerd Petermann:

Hi Bernhard,

just a guess: If you use option --add-pois-to-lines and you have a rule in the 
points file which processes ele to add a POI
you may see something like that.

Gerd


Von: mkgmap-dev  im Auftrag von Gerd 
Petermann 
Gesendet: Freitag, 2. November 2018 19:24
An: Bernhard Hiller; Development list for mkgmap
Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?

Hi Bernhard,

what are your rules for the contour lines?

Gerd


Von: mkgmap-dev  im Auftrag von Bernhard 
Hiller 
Gesendet: Freitag, 2. November 2018 19:18
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?

Hi all,
still struggling with that strange issue. But I guess I found some hint
to the cause: inconsistent file sizes.
- extracted OSM data: 400 MB pbf
- elevation contour lines: 600 MB pbf
- merged file: 1 GB pbf
So far, sizes are consistent.

When I run splitter on the OSM data only file, it produces many tiles
summing up to some 400 MB. Consistent, too. When I run mkgmap on this
output, I get a map within about half an hour, and it looks OK (tested
with QLandkarte).

When I run splitter on the merged file, the tiles sum up to some 9 GB.
That is 9 times the size I'd expect. mkgmap can render several of those
tiles (but very slowly); and then crashes on one of them with an
OutOfMemory exception.

So I think that the problem is somewhere in those contour lines, either
when merged or alone.
I'll try to create a contour lines only map as the next step to test
this hypothesis.

Kind regards,
Bernhard

Am 01.11.2018 um 09:32 schrieb Gerd Petermann:

Hi Bernhard,

I tried to reproduce the memory problems with tile 47120005. I don't think that 
the sea itself is a problem here, at least not when you use the --precomp-sea 
option. It took only 15 secs to process that tile with asia data from August 
with the default style and typical options for routing etc.
My input file didn't contain SRTM data so I assume this is the reason. Maybe 
you have many contour lines with ele= your data?

Gerd


Von: Gerd Petermann 
Gesendet: Mittwoch, 31. Oktober 2018 22:49
An: Gerd Petermann; Development list for mkgmap
Betreff: AW: [mkgmap-dev] splitter: option for maximum tile area?

Hi Bernhard,

looked again at the splitter command in your last post. You also use a rather 
high max-nodes value.
Such a high value means that you get rather large tiles and that mkgmap needs 
more memory for each tile
compared to the default 140. Many users use a value near 120.

Gerd


Von: mkgmap-dev  im Auftrag von Gerd 
Petermann 
Gesendet: Mittwoch, 31. Oktober 2018 22:20
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?

Hi Bernhard,
there is no option for this. Do you use option --precomp-sea in splitter? Maybe 
you use --no-trim ?
If that doesn't help you can try to change the values for
private static final int MAX_LAT_DEGREES =
private static final int MAX_LON_DEGREES =
in SplittableDensityArea.java and compile your own version of splitter.

Gerd


Von: mkgmap-dev  im Auftrag von Bernhard 
Hiller 
Gesendet: Mittwoch, 31. Oktober 2018 22:03
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: [mkgmap-dev] splitter: option for maximum tile area?

Hi all,
currently a Java OutOfMemory exception prevents me from creating a map.
I already use option --max-jobs=he machine has 4 physical cores) and
-Xmx5G (of 8 GB installed). Beyond OSM data, the map contains DEM and
elevation contour lines.
   From the tiles finished and those with a new timestamp but about 0
bytes length, I can see that mkgmap was rendering tiles 47120005,
47120006, 4712000

Re: [mkgmap-dev] splitter: option for maximum tile area?

2018-11-02 Thread Gerd Petermann

Hi Ralf,

you got it wrong, it is about mkgmap because the problem is in this stage. The 
subject might be misleading ;-)

Gerd

Von: mkgmap-dev  im Auftrag von Ralf 
Kleineisel 
Gesendet: Freitag, 2. November 2018 21:48
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?

On 11/02/2018 07:31 PM, Gerd Petermann wrote:

> just a guess: If you use option --add-pois-to-lines and you have a rule in 
> the points file which processes ele to add a POI
> you may see something like that.

I may have understood this wrong, but wasn't the file size comparison
after the splitter stage? Splitter doesn't use any style rules, does it?
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter: option for maximum tile area?

2018-11-02 Thread Ralf Kleineisel

On 11/02/2018 07:31 PM, Gerd Petermann wrote:

> just a guess: If you use option --add-pois-to-lines and you have a rule in 
> the points file which processes ele to add a POI
> you may see something like that.

I may have understood this wrong, but wasn't the file size comparison
after the splitter stage? Splitter doesn't use any style rules, does it?
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter: option for maximum tile area?

2018-11-02 Thread Gerd Petermann

Hi Bernhard,

just a guess: If you use option --add-pois-to-lines and you have a rule in the 
points file which processes ele to add a POI
you may see something like that.

Gerd


Von: mkgmap-dev  im Auftrag von Gerd 
Petermann 
Gesendet: Freitag, 2. November 2018 19:24
An: Bernhard Hiller; Development list for mkgmap
Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?

Hi Bernhard,

what are your rules for the contour lines?

Gerd


Von: mkgmap-dev  im Auftrag von 
Bernhard Hiller 
Gesendet: Freitag, 2. November 2018 19:18
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?

Hi all,
still struggling with that strange issue. But I guess I found some hint
to the cause: inconsistent file sizes.
- extracted OSM data: 400 MB pbf
- elevation contour lines: 600 MB pbf
- merged file: 1 GB pbf
So far, sizes are consistent.

When I run splitter on the OSM data only file, it produces many tiles
summing up to some 400 MB. Consistent, too. When I run mkgmap on this
output, I get a map within about half an hour, and it looks OK (tested
with QLandkarte).

When I run splitter on the merged file, the tiles sum up to some 9 GB.
That is 9 times the size I'd expect. mkgmap can render several of those
tiles (but very slowly); and then crashes on one of them with an
OutOfMemory exception.

So I think that the problem is somewhere in those contour lines, either
when merged or alone.
I'll try to create a contour lines only map as the next step to test
this hypothesis.

Kind regards,
Bernhard

Am 01.11.2018 um 09:32 schrieb Gerd Petermann:
> Hi Bernhard,
>
> I tried to reproduce the memory problems with tile 47120005. I don't think 
> that the sea itself is a problem here, at least not when you use the 
> --precomp-sea option. It took only 15 secs to process that tile with asia 
> data from August with the default style and typical options for routing etc.
> My input file didn't contain SRTM data so I assume this is the reason. Maybe 
> you have many contour lines with ele=in your data?
>
> Gerd
>
> 
> Von: Gerd Petermann 
> Gesendet: Mittwoch, 31. Oktober 2018 22:49
> An: Gerd Petermann; Development list for mkgmap
> Betreff: AW: [mkgmap-dev] splitter: option for maximum tile area?
>
> Hi Bernhard,
>
> looked again at the splitter command in your last post. You also use a rather 
> high max-nodes value.
> Such a high value means that you get rather large tiles and that mkgmap needs 
> more memory for each tile
> compared to the default 140. Many users use a value near 120.
>
> Gerd
>
> 
> Von: mkgmap-dev  im Auftrag von Gerd 
> Petermann 
> Gesendet: Mittwoch, 31. Oktober 2018 22:20
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?
>
> Hi Bernhard,
> there is no option for this. Do you use option --precomp-sea in splitter? 
> Maybe you use --no-trim ?
> If that doesn't help you can try to change the values for
> private static final int MAX_LAT_DEGREES =5;
> private static final int MAX_LON_DEGREES =0;
> in SplittableDensityArea.java and compile your own version of splitter.
>
> Gerd
>
> 
> Von: mkgmap-dev  im Auftrag von 
> Bernhard Hiller 
> Gesendet: Mittwoch, 31. Oktober 2018 22:03
> An: mkgmap-dev@lists.mkgmap.org.uk
> Betreff: [mkgmap-dev] splitter: option for maximum tile area?
>
> Hi all,
> currently a Java OutOfMemory exception prevents me from creating a map.
> I already use option --max-jobs=(the machine has 4 physical cores) and
> -Xmx5G (of 8 GB installed). Beyond OSM data, the map contains DEM and
> elevation contour lines.
>   From the tiles finished and those with a new timestamp but about 0
> bytes length, I can see that mkgmap was rendering tiles 47120005,
> 47120006, 47120007 at the time of crash.
> Tile 47120005 is extremely large by physical area - some 6° x 5.5° (see
> attached file), covering a lot of the south chinese sea, i.e. there are
> not many actual data in that area.
> I guess that the problem arises with that tile. I remember some case in
> the past where a single tile covering such a large area of mainly sea
> caused mkgmap to take an enormous amount of time for rendering - also
> here, mkgmap already spent about 1 hour before crashing.
> So I'd like to ask: is there some possibility for limiting the area of a
> tile among the splitter options?
> Kind regards,
> Bernhard
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>

_

Re: [mkgmap-dev] splitter: option for maximum tile area?

2018-11-02 Thread Gerd Petermann

Hi Bernhard,

what are your rules for the contour lines?

Gerd


Von: mkgmap-dev  im Auftrag von 
Bernhard Hiller 
Gesendet: Freitag, 2. November 2018 19:18
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?

Hi all,
still struggling with that strange issue. But I guess I found some hint
to the cause: inconsistent file sizes.
- extracted OSM data: 400 MB pbf
- elevation contour lines: 600 MB pbf
- merged file: 1 GB pbf
So far, sizes are consistent.

When I run splitter on the OSM data only file, it produces many tiles
summing up to some 400 MB. Consistent, too. When I run mkgmap on this
output, I get a map within about half an hour, and it looks OK (tested
with QLandkarte).

When I run splitter on the merged file, the tiles sum up to some 9 GB.
That is 9 times the size I'd expect. mkgmap can render several of those
tiles (but very slowly); and then crashes on one of them with an
OutOfMemory exception.

So I think that the problem is somewhere in those contour lines, either
when merged or alone.
I'll try to create a contour lines only map as the next step to test
this hypothesis.

Kind regards,
Bernhard

Am 01.11.2018 um 09:32 schrieb Gerd Petermann:
> Hi Bernhard,
>
> I tried to reproduce the memory problems with tile 47120005. I don't think 
> that the sea itself is a problem here, at least not when you use the 
> --precomp-sea option. It took only 15 secs to process that tile with asia 
> data from August with the default style and typical options for routing etc.
> My input file didn't contain SRTM data so I assume this is the reason. Maybe 
> you have many contour lines with ele=in your data?
>
> Gerd
>
> 
> Von: Gerd Petermann 
> Gesendet: Mittwoch, 31. Oktober 2018 22:49
> An: Gerd Petermann; Development list for mkgmap
> Betreff: AW: [mkgmap-dev] splitter: option for maximum tile area?
>
> Hi Bernhard,
>
> looked again at the splitter command in your last post. You also use a rather 
> high max-nodes value.
> Such a high value means that you get rather large tiles and that mkgmap needs 
> more memory for each tile
> compared to the default 140. Many users use a value near 120.
>
> Gerd
>
> 
> Von: mkgmap-dev  im Auftrag von Gerd 
> Petermann 
> Gesendet: Mittwoch, 31. Oktober 2018 22:20
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?
>
> Hi Bernhard,
> there is no option for this. Do you use option --precomp-sea in splitter? 
> Maybe you use --no-trim ?
> If that doesn't help you can try to change the values for
> private static final int MAX_LAT_DEGREES =5;
> private static final int MAX_LON_DEGREES =0;
> in SplittableDensityArea.java and compile your own version of splitter.
>
> Gerd
>
> 
> Von: mkgmap-dev  im Auftrag von 
> Bernhard Hiller 
> Gesendet: Mittwoch, 31. Oktober 2018 22:03
> An: mkgmap-dev@lists.mkgmap.org.uk
> Betreff: [mkgmap-dev] splitter: option for maximum tile area?
>
> Hi all,
> currently a Java OutOfMemory exception prevents me from creating a map.
> I already use option --max-jobs=(the machine has 4 physical cores) and
> -Xmx5G (of 8 GB installed). Beyond OSM data, the map contains DEM and
> elevation contour lines.
>   From the tiles finished and those with a new timestamp but about 0
> bytes length, I can see that mkgmap was rendering tiles 47120005,
> 47120006, 47120007 at the time of crash.
> Tile 47120005 is extremely large by physical area - some 6° x 5.5° (see
> attached file), covering a lot of the south chinese sea, i.e. there are
> not many actual data in that area.
> I guess that the problem arises with that tile. I remember some case in
> the past where a single tile covering such a large area of mainly sea
> caused mkgmap to take an enormous amount of time for rendering - also
> here, mkgmap already spent about 1 hour before crashing.
> So I'd like to ask: is there some possibility for limiting the area of a
> tile among the splitter options?
> Kind regards,
> Bernhard
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter: option for maximum tile area?

2018-11-02 Thread Bernhard Hiller


Hi all,
still struggling with that strange issue. But I guess I found some hint 
to the cause: inconsistent file sizes.

- extracted OSM data: 400 MB pbf
- elevation contour lines: 600 MB pbf
- merged file: 1 GB pbf
So far, sizes are consistent.

When I run splitter on the OSM data only file, it produces many tiles 
summing up to some 400 MB. Consistent, too. When I run mkgmap on this 
output, I get a map within about half an hour, and it looks OK (tested 
with QLandkarte).


When I run splitter on the merged file, the tiles sum up to some 9 GB. 
That is 9 times the size I'd expect. mkgmap can render several of those 
tiles (but very slowly); and then crashes on one of them with an 
OutOfMemory exception.


So I think that the problem is somewhere in those contour lines, either 
when merged or alone.
I'll try to create a contour lines only map as the next step to test 
this hypothesis.


Kind regards,
Bernhard

Am 01.11.2018 um 09:32 schrieb Gerd Petermann:

Hi Bernhard,

I tried to reproduce the memory problems with tile 47120005. I don't think that 
the sea itself is a problem here, at least not when you use the --precomp-sea 
option. It took only 15 secs to process that tile with asia data from August 
with the default style and typical options for routing etc.
My input file didn't contain SRTM data so I assume this is the reason. Maybe 
you have many contour lines with ele=in your data?

Gerd


Von: Gerd Petermann 
Gesendet: Mittwoch, 31. Oktober 2018 22:49
An: Gerd Petermann; Development list for mkgmap
Betreff: AW: [mkgmap-dev] splitter: option for maximum tile area?

Hi Bernhard,

looked again at the splitter command in your last post. You also use a rather 
high max-nodes value.
Such a high value means that you get rather large tiles and that mkgmap needs 
more memory for each tile
compared to the default 140. Many users use a value near 120.

Gerd


Von: mkgmap-dev  im Auftrag von Gerd 
Petermann 
Gesendet: Mittwoch, 31. Oktober 2018 22:20
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?

Hi Bernhard,
there is no option for this. Do you use option --precomp-sea in splitter? Maybe 
you use --no-trim ?
If that doesn't help you can try to change the values for
private static final int MAX_LAT_DEGREES =5;
private static final int MAX_LON_DEGREES =0;
in SplittableDensityArea.java and compile your own version of splitter.

Gerd


Von: mkgmap-dev  im Auftrag von Bernhard 
Hiller 
Gesendet: Mittwoch, 31. Oktober 2018 22:03
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: [mkgmap-dev] splitter: option for maximum tile area?

Hi all,
currently a Java OutOfMemory exception prevents me from creating a map.
I already use option --max-jobs=(the machine has 4 physical cores) and
-Xmx5G (of 8 GB installed). Beyond OSM data, the map contains DEM and
elevation contour lines.
  From the tiles finished and those with a new timestamp but about 0
bytes length, I can see that mkgmap was rendering tiles 47120005,
47120006, 47120007 at the time of crash.
Tile 47120005 is extremely large by physical area - some 6° x 5.5° (see
attached file), covering a lot of the south chinese sea, i.e. there are
not many actual data in that area.
I guess that the problem arises with that tile. I remember some case in
the past where a single tile covering such a large area of mainly sea
caused mkgmap to take an enormous amount of time for rendering - also
here, mkgmap already spent about 1 hour before crashing.
So I'd like to ask: is there some possibility for limiting the area of a
tile among the splitter options?
Kind regards,
Bernhard
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev



___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter: option for maximum tile area?

2018-11-01 Thread Gerd Petermann

Hi all,

validator would not find ways like these two:
https://www.openstreetmap.org/changeset/64081765

They are in the middle of South America. I found them only because they 
visually stuck out in JOSM. I would not find such an island when it is close to 
the real coastline
but on land.

Gerd

Von: mkgmap-dev  im Auftrag von Gerd 
Petermann 
Gesendet: Donnerstag, 1. November 2018 15:15
An: Thorsten Kukuk; Development list for mkgmap
Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?

Hi Thorsten,

thanks for the info. What is it that makes the correction so complicated?
My approach would be this:
1) download planet
2) filter ways with natural=coastline
3) split the file with splitter into e.g. 100 parts
4) for each part:
- open in JOSM
- run validator
- If an error is found, download that area to have current data and fix it

This should allow to find and correct unconnected coastline ways within a few 
hours.
Am I missing something?

Gerd

Von: mkgmap-dev  im Auftrag von 
Thorsten Kukuk 
Gesendet: Donnerstag, 1. November 2018 12:38
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?

On Thu, Nov 01, Gerd Petermann wrote:

> Hi Bernhard,
>
> my understanding is that you should never see broken precompiled sea data, 
> because the data source is not directly OSM
> but http://openstreetmapdata.com/data/land-polygons where Jochen Topf invests 
> time to check that the data is OK before updating it.
> So, sea.zip might not be up to date but should always be OK.

Not really, as he writes on his web page, his tools tries to repair the
coastlines. And as I use them, too, I know that this automatic "repair"
can break the result even more. I have added some additional
post-precessing checks and patched osmcoastline to bail out on more
errors to make sure that the result is for me really useable.

Currently the coastlines are so heavy broken, that osmcoastline only
seg.faults. So there will be no update with the broken costlines.
I assume that why geofabrik does not report coastlines errors anymore,
too.

  Thorsten

--
Thorsten Kukuk, Distinguished Engineer, Senior Architect SLES & MicroOS
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany
GF: Felix Imendoerffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nuernberg)
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter: option for maximum tile area?

2018-11-01 Thread Gerd Petermann

Hi Thorsten,

thanks for the info. What is it that makes the correction so complicated?
My approach would be this:
1) download planet
2) filter ways with natural=coastline
3) split the file with splitter into e.g. 100 parts
4) for each part:
- open in JOSM
- run validator
- If an error is found, download that area to have current data and fix it

This should allow to find and correct unconnected coastline ways within a few 
hours.
Am I missing something?

Gerd

Von: mkgmap-dev  im Auftrag von 
Thorsten Kukuk 
Gesendet: Donnerstag, 1. November 2018 12:38
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?

On Thu, Nov 01, Gerd Petermann wrote:

> Hi Bernhard,
>
> my understanding is that you should never see broken precompiled sea data, 
> because the data source is not directly OSM
> but http://openstreetmapdata.com/data/land-polygons where Jochen Topf invests 
> time to check that the data is OK before updating it.
> So, sea.zip might not be up to date but should always be OK.

Not really, as he writes on his web page, his tools tries to repair the
coastlines. And as I use them, too, I know that this automatic "repair"
can break the result even more. I have added some additional
post-precessing checks and patched osmcoastline to bail out on more
errors to make sure that the result is for me really useable.

Currently the coastlines are so heavy broken, that osmcoastline only
seg.faults. So there will be no update with the broken costlines.
I assume that why geofabrik does not report coastlines errors anymore,
too.

  Thorsten

--
Thorsten Kukuk, Distinguished Engineer, Senior Architect SLES & MicroOS
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany
GF: Felix Imendoerffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nuernberg)
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter: option for maximum tile area?

2018-11-01 Thread Thorsten Kukuk

On Thu, Nov 01, Gerd Petermann wrote:

> Hi Bernhard,
> 
> my understanding is that you should never see broken precompiled sea data, 
> because the data source is not directly OSM
> but http://openstreetmapdata.com/data/land-polygons where Jochen Topf invests 
> time to check that the data is OK before updating it.
> So, sea.zip might not be up to date but should always be OK.

Not really, as he writes on his web page, his tools tries to repair the
coastlines. And as I use them, too, I know that this automatic "repair"
can break the result even more. I have added some additional 
post-precessing checks and patched osmcoastline to bail out on more
errors to make sure that the result is for me really useable.

Currently the coastlines are so heavy broken, that osmcoastline only
seg.faults. So there will be no update with the broken costlines.
I assume that why geofabrik does not report coastlines errors anymore,
too.

  Thorsten

-- 
Thorsten Kukuk, Distinguished Engineer, Senior Architect SLES & MicroOS
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany
GF: Felix Imendoerffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nuernberg)
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter: option for maximum tile area?

2018-11-01 Thread Gerd Petermann

Hi Bernhard,

my understanding is that you should never see broken precompiled sea data, 
because the data source is not directly OSM
but http://openstreetmapdata.com/data/land-polygons where Jochen Topf invests 
time to check that the data is OK before updating it.
So, sea.zip might not be up to date but should always be OK.

Gerd


Von: mkgmap-dev  im Auftrag von 
Bernhard Hiller 
Gesendet: Donnerstag, 1. November 2018 11:23
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?

Hi Gerd,
thanks a lot for your many hints.
The file contains contour lines with ele=0, but I do not know how many.
Anyway, the style ignores contour lines at an elevation of 0.
Currently, I do not use the --precomp-sea option, but
"--generate-sea=multipolygon,extend-sea-sectors,floodblocker". Are the
precompiled coast line files OK again (I remember that they were broken
just recently)?
Changing the source code of splitter is not an option for me: I have no
Java IDE installed on my machine.
I'll try now with a severely reduced max-nodes value.
Kind regards,
Bernhard

Am 01.11.2018 um 09:32 schrieb Gerd Petermann:
> Hi Bernhard,
>
> I tried to reproduce the memory problems with tile 47120005. I don't think 
> that the sea itself is a problem here, at least not when you use the 
> --precomp-sea option. It took only 15 secs to process that tile with asia 
> data from August with the default style and typical options for routing etc.
> My input file didn't contain SRTM data so I assume this is the reason. Maybe 
> you have many contour lines with ele=in your data?
>
> Gerd
>
> 
> Von: Gerd Petermann 
> Gesendet: Mittwoch, 31. Oktober 2018 22:49
> An: Gerd Petermann; Development list for mkgmap
> Betreff: AW: [mkgmap-dev] splitter: option for maximum tile area?
>
> Hi Bernhard,
>
> looked again at the splitter command in your last post. You also use a rather 
> high max-nodes value.
> Such a high value means that you get rather large tiles and that mkgmap needs 
> more memory for each tile
> compared to the default 140. Many users use a value near 120.
>
> Gerd
>
> 
> Von: mkgmap-dev  im Auftrag von Gerd 
> Petermann 
> Gesendet: Mittwoch, 31. Oktober 2018 22:20
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?
>
> Hi Bernhard,
> there is no option for this. Do you use option --precomp-sea in splitter? 
> Maybe you use --no-trim ?
> If that doesn't help you can try to change the values for
> private static final int MAX_LAT_DEGREES =5;
> private static final int MAX_LON_DEGREES =0;
> in SplittableDensityArea.java and compile your own version of splitter.
>
> Gerd
>
> 
> Von: mkgmap-dev  im Auftrag von 
> Bernhard Hiller 
> Gesendet: Mittwoch, 31. Oktober 2018 22:03
> An: mkgmap-dev@lists.mkgmap.org.uk
> Betreff: [mkgmap-dev] splitter: option for maximum tile area?
>
> Hi all,
> currently a Java OutOfMemory exception prevents me from creating a map.
> I already use option --max-jobs=(the machine has 4 physical cores) and
> -Xmx5G (of 8 GB installed). Beyond OSM data, the map contains DEM and
> elevation contour lines.
>   From the tiles finished and those with a new timestamp but about 0
> bytes length, I can see that mkgmap was rendering tiles 47120005,
> 47120006, 47120007 at the time of crash.
> Tile 47120005 is extremely large by physical area - some 6° x 5.5° (see
> attached file), covering a lot of the south chinese sea, i.e. there are
> not many actual data in that area.
> I guess that the problem arises with that tile. I remember some case in
> the past where a single tile covering such a large area of mainly sea
> caused mkgmap to take an enormous amount of time for rendering - also
> here, mkgmap already spent about 1 hour before crashing.
> So I'd like to ask: is there some possibility for limiting the area of a
> tile among the splitter options?
> Kind regards,
> Bernhard
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter: option for maximum tile area?

2018-11-01 Thread Bernhard Hiller


Hi Gerd,
thanks a lot for your many hints.
The file contains contour lines with ele=0, but I do not know how many. 
Anyway, the style ignores contour lines at an elevation of 0.
Currently, I do not use the --precomp-sea option, but 
"--generate-sea=multipolygon,extend-sea-sectors,floodblocker". Are the 
precompiled coast line files OK again (I remember that they were broken 
just recently)?
Changing the source code of splitter is not an option for me: I have no 
Java IDE installed on my machine.

I'll try now with a severely reduced max-nodes value.
Kind regards,
Bernhard

Am 01.11.2018 um 09:32 schrieb Gerd Petermann:

Hi Bernhard,

I tried to reproduce the memory problems with tile 47120005. I don't think that 
the sea itself is a problem here, at least not when you use the --precomp-sea 
option. It took only 15 secs to process that tile with asia data from August 
with the default style and typical options for routing etc.
My input file didn't contain SRTM data so I assume this is the reason. Maybe 
you have many contour lines with ele=in your data?

Gerd


Von: Gerd Petermann 
Gesendet: Mittwoch, 31. Oktober 2018 22:49
An: Gerd Petermann; Development list for mkgmap
Betreff: AW: [mkgmap-dev] splitter: option for maximum tile area?

Hi Bernhard,

looked again at the splitter command in your last post. You also use a rather 
high max-nodes value.
Such a high value means that you get rather large tiles and that mkgmap needs 
more memory for each tile
compared to the default 140. Many users use a value near 120.

Gerd


Von: mkgmap-dev  im Auftrag von Gerd 
Petermann 
Gesendet: Mittwoch, 31. Oktober 2018 22:20
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?

Hi Bernhard,
there is no option for this. Do you use option --precomp-sea in splitter? Maybe 
you use --no-trim ?
If that doesn't help you can try to change the values for
private static final int MAX_LAT_DEGREES =5;
private static final int MAX_LON_DEGREES =0;
in SplittableDensityArea.java and compile your own version of splitter.

Gerd


Von: mkgmap-dev  im Auftrag von Bernhard 
Hiller 
Gesendet: Mittwoch, 31. Oktober 2018 22:03
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: [mkgmap-dev] splitter: option for maximum tile area?

Hi all,
currently a Java OutOfMemory exception prevents me from creating a map.
I already use option --max-jobs=(the machine has 4 physical cores) and
-Xmx5G (of 8 GB installed). Beyond OSM data, the map contains DEM and
elevation contour lines.
  From the tiles finished and those with a new timestamp but about 0
bytes length, I can see that mkgmap was rendering tiles 47120005,
47120006, 47120007 at the time of crash.
Tile 47120005 is extremely large by physical area - some 6° x 5.5° (see
attached file), covering a lot of the south chinese sea, i.e. there are
not many actual data in that area.
I guess that the problem arises with that tile. I remember some case in
the past where a single tile covering such a large area of mainly sea
caused mkgmap to take an enormous amount of time for rendering - also
here, mkgmap already spent about 1 hour before crashing.
So I'd like to ask: is there some possibility for limiting the area of a
tile among the splitter options?
Kind regards,
Bernhard
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev



___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter: option for maximum tile area?

2018-11-01 Thread Gerd Petermann

Hi Bernhard,

I tried to reproduce the memory problems with tile 47120005. I don't think that 
the sea itself is a problem here, at least not when you use the --precomp-sea 
option. It took only 15 secs to process that tile with asia data from August 
with the default style and typical options for routing etc.
My input file didn't contain SRTM data so I assume this is the reason. Maybe 
you have many contour lines with ele=0 in your data?

Gerd


Von: Gerd Petermann 
Gesendet: Mittwoch, 31. Oktober 2018 22:49
An: Gerd Petermann; Development list for mkgmap
Betreff: AW: [mkgmap-dev] splitter: option for maximum tile area?

Hi Bernhard,

looked again at the splitter command in your last post. You also use a rather 
high max-nodes value.
Such a high value means that you get rather large tiles and that mkgmap needs 
more memory for each tile
compared to the default 140. Many users use a value near 120.

Gerd


Von: mkgmap-dev  im Auftrag von Gerd 
Petermann 
Gesendet: Mittwoch, 31. Oktober 2018 22:20
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?

Hi Bernhard,
there is no option for this. Do you use option --precomp-sea in splitter? Maybe 
you use --no-trim ?
If that doesn't help you can try to change the values for
private static final int MAX_LAT_DEGREES = 85;
private static final int MAX_LON_DEGREES = 90;
in SplittableDensityArea.java and compile your own version of splitter.

Gerd


Von: mkgmap-dev  im Auftrag von 
Bernhard Hiller 
Gesendet: Mittwoch, 31. Oktober 2018 22:03
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: [mkgmap-dev] splitter: option for maximum tile area?

Hi all,
currently a Java OutOfMemory exception prevents me from creating a map.
I already use option --max-jobs=3 (the machine has 4 physical cores) and
-Xmx5G (of 8 GB installed). Beyond OSM data, the map contains DEM and
elevation contour lines.
 From the tiles finished and those with a new timestamp but about 0
bytes length, I can see that mkgmap was rendering tiles 47120005,
47120006, 47120007 at the time of crash.
Tile 47120005 is extremely large by physical area - some 6° x 5.5° (see
attached file), covering a lot of the south chinese sea, i.e. there are
not many actual data in that area.
I guess that the problem arises with that tile. I remember some case in
the past where a single tile covering such a large area of mainly sea
caused mkgmap to take an enormous amount of time for rendering - also
here, mkgmap already spent about 1 hour before crashing.
So I'd like to ask: is there some possibility for limiting the area of a
tile among the splitter options?
Kind regards,
Bernhard
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter: option for maximum tile area?

2018-10-31 Thread Gerd Petermann

Hi Bernhard,

looked again at the splitter command in your last post. You also use a rather 
high max-nodes value.
Such a high value means that you get rather large tiles and that mkgmap needs 
more memory for each tile
compared to the default 140. Many users use a value near 120.

Gerd


Von: mkgmap-dev  im Auftrag von Gerd 
Petermann 
Gesendet: Mittwoch, 31. Oktober 2018 22:20
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?

Hi Bernhard,
there is no option for this. Do you use option --precomp-sea in splitter? Maybe 
you use --no-trim ?
If that doesn't help you can try to change the values for
private static final int MAX_LAT_DEGREES = 85;
private static final int MAX_LON_DEGREES = 90;
in SplittableDensityArea.java and compile your own version of splitter.

Gerd


Von: mkgmap-dev  im Auftrag von 
Bernhard Hiller 
Gesendet: Mittwoch, 31. Oktober 2018 22:03
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: [mkgmap-dev] splitter: option for maximum tile area?

Hi all,
currently a Java OutOfMemory exception prevents me from creating a map.
I already use option --max-jobs=3 (the machine has 4 physical cores) and
-Xmx5G (of 8 GB installed). Beyond OSM data, the map contains DEM and
elevation contour lines.
 From the tiles finished and those with a new timestamp but about 0
bytes length, I can see that mkgmap was rendering tiles 47120005,
47120006, 47120007 at the time of crash.
Tile 47120005 is extremely large by physical area - some 6° x 5.5° (see
attached file), covering a lot of the south chinese sea, i.e. there are
not many actual data in that area.
I guess that the problem arises with that tile. I remember some case in
the past where a single tile covering such a large area of mainly sea
caused mkgmap to take an enormous amount of time for rendering - also
here, mkgmap already spent about 1 hour before crashing.
So I'd like to ask: is there some possibility for limiting the area of a
tile among the splitter options?
Kind regards,
Bernhard
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter: option for maximum tile area?

2018-10-31 Thread Gerd Petermann

Hi Bernhard,
there is no option for this. Do you use option --precomp-sea in splitter? Maybe 
you use --no-trim ?
If that doesn't help you can try to change the values for
private static final int MAX_LAT_DEGREES = 85;
private static final int MAX_LON_DEGREES = 90;
in SplittableDensityArea.java and compile your own version of splitter.

Gerd


Von: mkgmap-dev  im Auftrag von 
Bernhard Hiller 
Gesendet: Mittwoch, 31. Oktober 2018 22:03
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: [mkgmap-dev] splitter: option for maximum tile area?

Hi all,
currently a Java OutOfMemory exception prevents me from creating a map.
I already use option --max-jobs=3 (the machine has 4 physical cores) and
-Xmx5G (of 8 GB installed). Beyond OSM data, the map contains DEM and
elevation contour lines.
 From the tiles finished and those with a new timestamp but about 0
bytes length, I can see that mkgmap was rendering tiles 47120005,
47120006, 47120007 at the time of crash.
Tile 47120005 is extremely large by physical area - some 6° x 5.5° (see
attached file), covering a lot of the south chinese sea, i.e. there are
not many actual data in that area.
I guess that the problem arises with that tile. I remember some case in
the past where a single tile covering such a large area of mainly sea
caused mkgmap to take an enormous amount of time for rendering - also
here, mkgmap already spent about 1 hour before crashing.
So I'd like to ask: is there some possibility for limiting the area of a
tile among the splitter options?
Kind regards,
Bernhard
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter: "Cannot store node id 9223372036854775807, this value is reserved"

2018-10-29 Thread Bernhard Hiller


Hi Gerd,
Please find enclosed file "splitter.log" which shows node ids in that 
region:

10.000.000 nodes parsed... id=9223372036671913720
splitter crashes after the entry
190.000.000 nodes parsed... id=9223372036851913720

The source code of srtm2osm at 
https://svn.openstreetmap.org/applications/utils/Srtm2Osm/trunk/Srtm2Osm/Srtm2OsmCommand.cs 
shows that


private long firstNodeId = long.MaxValue;

is the default value, the source of the IdCounter shows that it does 
decrement for the further values by default.

=> The value comes from the generation of contour lines via srtm2osm.
By the way, also firstWayId is initialized to long.MaxValue. Would that 
also cause problems with splitter?


Strange that this fact did not cause problems in February, when I 
created a map of Taiwan with DEM and elevation contour lines. Perhaps a 
bounding box in an osmosis command happened to remove this node id by 
chance.


So, I guess, we have a solution. The bad thing is: I'll have to re-run 
the generation of the contourlines file with srtm2osm (with additional 
parameters for first node and way ids), which takes several hours, plus 
1 hour of sorting, 


Kind regards,
Bernhard

Am 29.10.2018 um 08:04 schrieb Gerd Petermann:

Hi Bernhard,

splitter cannot store the id values 0 and 9223372036854775807 (which is 
Long.MAX_VALUE)
No idea why your process whould create an id with such a value, looks like an 
error to me.
The following message about the sort is misleading :-(

Do you have an idea where this id comes from?

Gerd


Von: mkgmap-dev  im Auftrag von Bernhard 
Hiller 
Gesendet: Montag, 29. Oktober 2018 07:48:02
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: [mkgmap-dev] splitter: "Cannot store node id 9223372036854775807, this 
value is reserved"

Hi all,
I tried to create a map of south east asia with contourlines. DEM data
are from Viewfinderpanoramas, and were processed with srtm2osm. It took
7 h to generate a 40GB *.osm file, and another 1 hour to sort the data
with osmosis into a 600 MB *.pbf file. A region from 0-23N and 94-109E
was cut from the asia extract from Geofabrik with osmosis (some 400 MB).
These two files were combined with osmosis.

When I call splitter on that file, I get the error message:

Error: Cannot store node id 9223372036854775807, this value is reserved.
uk.me.parabola.splitter.SplitFailedException: Maybe the IDs are not
sorted. This is not supported with keep-complete=ue or --problem-list
  at
uk.me.parabola.splitter.MultiTileProcessor.storeCoord(MultiTileProcessor.java:508)
  at
uk.me.parabola.splitter.MultiTileProcessor.processNode(MultiTileProcessor.java:127)
  at
uk.me.parabola.splitter.AbstractMapProcessor.consume(AbstractMapProcessor.java:62)
  at
uk.me.parabola.splitter.OSMFileHandler.execute(OSMFileHandler.java:159)
  at
uk.me.parabola.splitter.ProblemLists.calcMultiTileElements(ProblemLists.java:255)
  at uk.me.parabola.splitter.Main.useProblemLists(Main.java:487)
  at uk.me.parabola.splitter.Main.start(Main.java:125)
  at uk.me.parabola.splitter.Main.main(Main.java:79)

The splitter command is:
E:\Maps\Development\Thai.1>"C:\Program
Files\Java\jre1.8.0_121\bin\java.exe" -Xmx6G -jar  "C:\Program Files
(x86)\OpenStreetMap\splitter-r590\splitter.jar"
E:\Maps\Raw\SouthEastAsia_Complete.osm.pbf --description=ernieMap Bike
Thai" --mapidG120001 --max-nodes 0 --max-threads=4
--write-kml=slitter.kml--no-trim  1>splitter.log

Do I need to sort again after merging? Or is that a different issue?
Thanks a lot for your hints.

Kind regards,
Bernhard
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev



Splitter version 590 compiled 2018-01-27T09:06:04+
boundary-tags=use-exclude-list
cache=
description=BernieMap Bike Thai
geonames-file=
handle-element-version=remove
ignore-osm-bounds=false
keep-complete=true
mapid=47120001
max-areas=2048
max-nodes=300
max-threads=4
mixed=false
no-trim=true
num-tiles=
output=pbf
output-dir=
overlap=auto
polygon-desc-file=
polygon-file=
precomp-sea=
problem-file=
problem-report=
resolution=13
search-limit=20
split-file=
status-freq=120
stop-after=dist
wanted-admin-level=5
write-kml=splitter.kml
Elapsed time: 0s   Memory: Current 123MB (5MB used, 118MB free) Max 5461MB
Time started: Mon Oct 29 19:06:16 CET 2018
Map is being split for resolution 13:
 - area boundaries are aligned to 0x800 map units (0.0439453125 degrees)
 - areas are multiples of 0x800 map units wide and high
Processing E:\Maps\Raw\Thai.DEM\SouthEastAsia_Ele.osm.pbf
10.000.000 nodes parsed... id=9223372036671913720
20.000.000 nodes parsed... id=9223372036681913720
30.000.000 nodes parsed... id=9223372036691913720
40.000.000 nodes parsed... id=9223372036701913720
50.000.000 nodes parsed... id=9223372036711913720
60.000.000 nodes parsed...

Re: [mkgmap-dev] splitter: "Cannot store node id 9223372036854775807, this value is reserved"

2018-10-29 Thread Gerd Petermann

Hi Bernhard,

I would not say that the error cannot be in the file produced by srtm2osm. The 
bad id might come after splitter complains
about the wrong order. You may try osmconvert with --out-statistics to check 
the file.
But be careful, current 64 Bit Windows version of osmconvert seems to be buggy, 
make sure to use the one from the english wiki page with large file support.

Gerd


Von: mkgmap-dev  im Auftrag von 
Bernhard Hiller 
Gesendet: Montag, 29. Oktober 2018 19:16:31
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: Re: [mkgmap-dev] splitter: "Cannot store node id 9223372036854775807, 
this value is reserved"

Hi Gerd,
I am investigation the origin of that id.
I tried to run splitter on the file generated by srtm2osm. After half an
hour, I got following exception:
E:\Maps\Development\Thai.1>"C:\Program
Files\Java\jre1.8.0_121\bin\java.exe" -Xmx6G -jar  "C:\Program Files
(x86)\OpenStreetMap\splitter-r590\splitter.jar"
E:\Maps\Raw\Thai.DEM\SouthEastAsia_Ele_Unsorted.osm
--description="BernieMap  Bike Thai" --mapid=47120001
--max-nodes=300 --max-threads=4 --write-kml=splitter.kml
--no-trim  1>splitter.log
Error: Node ids are not sorted. Use e.g. osmosis to sort the input data.
This is not supported with keep-complete=true or --problem-list
uk.me.parabola.splitter.SplitFailedException: Node ids are not sorted
 at
uk.me.parabola.splitter.MultiTileProcessor.storeCoord(MultiTileProcessor.java:501)
 at
uk.me.parabola.splitter.MultiTileProcessor.processNode(MultiTileProcessor.java:127)
 at
uk.me.parabola.splitter.AbstractMapProcessor.consume(AbstractMapProcessor.java:62)
 at
uk.me.parabola.splitter.OSMFileHandler.execute(OSMFileHandler.java:159)
 at
uk.me.parabola.splitter.ProblemLists.calcMultiTileElements(ProblemLists.java:255)
 at uk.me.parabola.splitter.Main.useProblemLists(Main.java:487)
 at uk.me.parabola.splitter.Main.start(Main.java:125)
 at uk.me.parabola.splitter.Main.main(Main.java:79)
which is OK, because srtm2osm does not write its output in some sorted way.

Next, I tried to run splitter on the file produced by osmosis after
sorting. Now, it did not take long to get the exception:
E:\Maps\Development\Thai.1>"C:\Program
Files\Java\jre1.8.0_121\bin\java.exe" -Xmx6G -jar  "C:\Program Files
(x86)\OpenStreetMap\splitter-r590\splitter.jar"
E:\Maps\Raw\Thai.DEM\SouthEastAsia_Ele.osm.pbf --description="BernieMap
Bike Thai" --mapid=47120001 --max-nodes=300 --max-threads=4
--write-kml=splitter.kml--no-trim  1>splitter.log
Error: Cannot store node id 9223372036854775807, this value is reserved.
uk.me.parabola.splitter.SplitFailedException: Maybe the IDs are not
sorted. This is not supported with keep-complete=true or --problem-list
 at
uk.me.parabola.splitter.MultiTileProcessor.storeCoord(MultiTileProcessor.java:508)
 at
uk.me.parabola.splitter.MultiTileProcessor.processNode(MultiTileProcessor.java:127)
 at
uk.me.parabola.splitter.AbstractMapProcessor.consume(AbstractMapProcessor.java:62)
 at
uk.me.parabola.splitter.OSMFileHandler.execute(OSMFileHandler.java:159)
 at
uk.me.parabola.splitter.ProblemLists.calcMultiTileElements(ProblemLists.java:255)
 at uk.me.parabola.splitter.Main.useProblemLists(Main.java:487)
 at uk.me.parabola.splitter.Main.start(Main.java:125)
 at uk.me.parabola.splitter.Main.main(Main.java:79)

That is, the wrong node id exists in the file of sorted elevation data.
I cannot find out if osmosis or srtm2osm produced that node id.

Kind regards,
Bernhard

Am 29.10.2018 um 08:04 schrieb Gerd Petermann:
> Hi Bernhard,
>
> splitter cannot store the id values 0 and 9223372036854775807 (which is 
> Long.MAX_VALUE)
> No idea why your process whould create an id with such a value, looks like an 
> error to me.
> The following message about the sort is misleading :-(
>
> Do you have an idea where this id comes from?
>
> Gerd
>
> 
> Von: mkgmap-dev  im Auftrag von 
> Bernhard Hiller 
> Gesendet: Montag, 29. Oktober 2018 07:48:02
> An: mkgmap-dev@lists.mkgmap.org.uk
> Betreff: [mkgmap-dev] splitter: "Cannot store node id 9223372036854775807, 
> this value is reserved"
>
> Hi all,
> I tried to create a map of south east asia with contourlines. DEM data
> are from Viewfinderpanoramas, and were processed with srtm2osm. It took
> 7 h to generate a 40GB *.osm file, and another 1 hour to sort the data
> with osmosis into a 600 MB *.pbf file. A region from 0-23N and 94-109E
> was cut from the asia extract from Geofabrik with osmosis (some 400 MB).
> These two files were combined with osmosis.
>
> When I call splitter on that file, I get the error message:
>
> Error: Cannot store node

Re: [mkgmap-dev] splitter: "Cannot store node id 9223372036854775807, this value is reserved"

2018-10-29 Thread Thorsten Kukuk

On Mon, Oct 29, Bernhard Hiller wrote:

> Hi Gerd,
> I am investigation the origin of that id.
> I tried to run splitter on the file generated by srtm2osm.

Try phyghtmap. In my experience, that works much better than srtm2osm.

  Thorsten

-- 
Thorsten Kukuk, Distinguished Engineer, Senior Architect SLES & MicroOS
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany
GF: Felix Imendoerffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nuernberg)
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter: "Cannot store node id 9223372036854775807, this value is reserved"

2018-10-29 Thread Bernhard Hiller


Hi Gerd,
I am investigation the origin of that id.
I tried to run splitter on the file generated by srtm2osm. After half an 
hour, I got following exception:
E:\Maps\Development\Thai.1>"C:\Program 
Files\Java\jre1.8.0_121\bin\java.exe" -Xmx6G -jar  "C:\Program Files 
(x86)\OpenStreetMap\splitter-r590\splitter.jar" 
E:\Maps\Raw\Thai.DEM\SouthEastAsia_Ele_Unsorted.osm 
--description="BernieMap  Bike Thai" --mapid=47120001 
--max-nodes=300 --max-threads=4 --write-kml=splitter.kml 
--no-trim  1>splitter.log

Error: Node ids are not sorted. Use e.g. osmosis to sort the input data.
This is not supported with keep-complete=true or --problem-list
uk.me.parabola.splitter.SplitFailedException: Node ids are not sorted
at 
uk.me.parabola.splitter.MultiTileProcessor.storeCoord(MultiTileProcessor.java:501)
at 
uk.me.parabola.splitter.MultiTileProcessor.processNode(MultiTileProcessor.java:127)
at 
uk.me.parabola.splitter.AbstractMapProcessor.consume(AbstractMapProcessor.java:62)
at 
uk.me.parabola.splitter.OSMFileHandler.execute(OSMFileHandler.java:159)
at 
uk.me.parabola.splitter.ProblemLists.calcMultiTileElements(ProblemLists.java:255)

at uk.me.parabola.splitter.Main.useProblemLists(Main.java:487)
at uk.me.parabola.splitter.Main.start(Main.java:125)
at uk.me.parabola.splitter.Main.main(Main.java:79)
which is OK, because srtm2osm does not write its output in some sorted way.

Next, I tried to run splitter on the file produced by osmosis after 
sorting. Now, it did not take long to get the exception:
E:\Maps\Development\Thai.1>"C:\Program 
Files\Java\jre1.8.0_121\bin\java.exe" -Xmx6G -jar  "C:\Program Files 
(x86)\OpenStreetMap\splitter-r590\splitter.jar" 
E:\Maps\Raw\Thai.DEM\SouthEastAsia_Ele.osm.pbf --description="BernieMap 
Bike Thai" --mapid=47120001 --max-nodes=300 --max-threads=4 
--write-kml=splitter.kml--no-trim  1>splitter.log

Error: Cannot store node id 9223372036854775807, this value is reserved.
uk.me.parabola.splitter.SplitFailedException: Maybe the IDs are not 
sorted. This is not supported with keep-complete=true or --problem-list
at 
uk.me.parabola.splitter.MultiTileProcessor.storeCoord(MultiTileProcessor.java:508)
at 
uk.me.parabola.splitter.MultiTileProcessor.processNode(MultiTileProcessor.java:127)
at 
uk.me.parabola.splitter.AbstractMapProcessor.consume(AbstractMapProcessor.java:62)
at 
uk.me.parabola.splitter.OSMFileHandler.execute(OSMFileHandler.java:159)
at 
uk.me.parabola.splitter.ProblemLists.calcMultiTileElements(ProblemLists.java:255)

at uk.me.parabola.splitter.Main.useProblemLists(Main.java:487)
at uk.me.parabola.splitter.Main.start(Main.java:125)
at uk.me.parabola.splitter.Main.main(Main.java:79)

That is, the wrong node id exists in the file of sorted elevation data. 
I cannot find out if osmosis or srtm2osm produced that node id.


Kind regards,
Bernhard

Am 29.10.2018 um 08:04 schrieb Gerd Petermann:

Hi Bernhard,

splitter cannot store the id values 0 and 9223372036854775807 (which is 
Long.MAX_VALUE)
No idea why your process whould create an id with such a value, looks like an 
error to me.
The following message about the sort is misleading :-(

Do you have an idea where this id comes from?

Gerd


Von: mkgmap-dev  im Auftrag von Bernhard 
Hiller 
Gesendet: Montag, 29. Oktober 2018 07:48:02
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: [mkgmap-dev] splitter: "Cannot store node id 9223372036854775807, this 
value is reserved"

Hi all,
I tried to create a map of south east asia with contourlines. DEM data
are from Viewfinderpanoramas, and were processed with srtm2osm. It took
7 h to generate a 40GB *.osm file, and another 1 hour to sort the data
with osmosis into a 600 MB *.pbf file. A region from 0-23N and 94-109E
was cut from the asia extract from Geofabrik with osmosis (some 400 MB).
These two files were combined with osmosis.

When I call splitter on that file, I get the error message:

Error: Cannot store node id 9223372036854775807, this value is reserved.
uk.me.parabola.splitter.SplitFailedException: Maybe the IDs are not
sorted. This is not supported with keep-complete=ue or --problem-list
  at
uk.me.parabola.splitter.MultiTileProcessor.storeCoord(MultiTileProcessor.java:508)
  at
uk.me.parabola.splitter.MultiTileProcessor.processNode(MultiTileProcessor.java:127)
  at
uk.me.parabola.splitter.AbstractMapProcessor.consume(AbstractMapProcessor.java:62)
  at
uk.me.parabola.splitter.OSMFileHandler.execute(OSMFileHandler.java:159)
  at
uk.me.parabola.splitter.ProblemLists.calcMultiTileElements(ProblemLists.java:255)
  at uk.me.parabola.splitter.Main.useProblemLists(Main.java:487)
  at uk.me.parabola.splitter.Main.start(Main.java:125)
  at uk.me.parabola.splitter.Main.main(Main.java:79)

The splitter command is:

Re: [mkgmap-dev] splitter: "Cannot store node id 9223372036854775807, this value is reserved"

2018-10-29 Thread Gerd Petermann

Hi Bernhard,

splitter cannot store the id values 0 and 9223372036854775807 (which is 
Long.MAX_VALUE)
No idea why your process whould create an id with such a value, looks like an 
error to me.
The following message about the sort is misleading :-(

Do you have an idea where this id comes from?

Gerd


Von: mkgmap-dev  im Auftrag von 
Bernhard Hiller 
Gesendet: Montag, 29. Oktober 2018 07:48:02
An: mkgmap-dev@lists.mkgmap.org.uk
Betreff: [mkgmap-dev] splitter: "Cannot store node id 9223372036854775807, this 
value is reserved"

Hi all,
I tried to create a map of south east asia with contourlines. DEM data
are from Viewfinderpanoramas, and were processed with srtm2osm. It took
7 h to generate a 40GB *.osm file, and another 1 hour to sort the data
with osmosis into a 600 MB *.pbf file. A region from 0-23N and 94-109E
was cut from the asia extract from Geofabrik with osmosis (some 400 MB).
These two files were combined with osmosis.

When I call splitter on that file, I get the error message:

Error: Cannot store node id 9223372036854775807, this value is reserved.
uk.me.parabola.splitter.SplitFailedException: Maybe the IDs are not
sorted. This is not supported with keep-complete=true or --problem-list
 at
uk.me.parabola.splitter.MultiTileProcessor.storeCoord(MultiTileProcessor.java:508)
 at
uk.me.parabola.splitter.MultiTileProcessor.processNode(MultiTileProcessor.java:127)
 at
uk.me.parabola.splitter.AbstractMapProcessor.consume(AbstractMapProcessor.java:62)
 at
uk.me.parabola.splitter.OSMFileHandler.execute(OSMFileHandler.java:159)
 at
uk.me.parabola.splitter.ProblemLists.calcMultiTileElements(ProblemLists.java:255)
 at uk.me.parabola.splitter.Main.useProblemLists(Main.java:487)
 at uk.me.parabola.splitter.Main.start(Main.java:125)
 at uk.me.parabola.splitter.Main.main(Main.java:79)

The splitter command is:
E:\Maps\Development\Thai.1>"C:\Program
Files\Java\jre1.8.0_121\bin\java.exe" -Xmx6G -jar  "C:\Program Files
(x86)\OpenStreetMap\splitter-r590\splitter.jar"
E:\Maps\Raw\SouthEastAsia_Complete.osm.pbf --description="BernieMap Bike
Thai" --mapid=47120001 --max-nodes=200 --max-threads=4
--write-kml=splitter.kml--no-trim  1>splitter.log

Do I need to sort again after merging? Or is that a different issue?
Thanks a lot for your hints.

Kind regards,
Bernhard
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter r590,wrong handling of relations

2018-02-07 Thread Thomas Morgenstern

Thank you :-).  Yesterday I had send a message to the Author Javier Sanchez 
.

thomas

--
Von: "Gerd Petermann" <gpetermann_muenc...@hotmail.com>
Datum: Mittwoch, 7. Februar 2018 06:24
An: "Development list for mkgmap" <mkgmap-dev@lists.mkgmap.org.uk>
Betreff: Re: [mkgmap-dev] splitter r590,wrong handling of relations


Hi Thomas,

I've just fixed the relation [1], so please try again with a fresh 
download tomorrow.


Gerd
[1] https://www.openstreetmap.org/changeset/56139020


Von: mkgmap-dev <mkgmap-dev-boun...@lists.mkgmap.org.uk> im Auftrag von 
Gerd Petermann <gpetermann_muenc...@hotmail.com>

Gesendet: Dienstag, 6. Februar 2018 14:47
An: Thomas Morgenstern; Development list for mkgmap
Betreff: Re: [mkgmap-dev] splitter r590,wrong handling of relations

Hi Thomas,

the relation is broken. Use JOSM validator and it will report 2 errors.
I think this way
https://www.openstreetmap.org/way/556302821
should not belong to it and a few nodes should be merged to repiar it:
node 4298102908
node 4298102907
node 655141348

Gerd


Von: mkgmap-dev <mkgmap-dev-boun...@lists.mkgmap.org.uk> im Auftrag von 
Thomas Morgenstern <webmas...@img2ms.de>

Gesendet: Dienstag, 6. Februar 2018 14:35
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] splitter r590,wrong handling of relations

Hi Gerd, i have now made a different split, so that 1 tile contains the
whole isle. In this case it looks good. I assume, that the relation is 
okay.
No idea what should be changed in data. Also i have looked before with 
JOSM

in the old split and  confirm, that in the left tile the relation also
exists, but not rendered from mkgmap. I can't judge, if the relation is
broken or not. Therefore the problem is in mkgmap ? I
thomas

--
Von: "Gerd Petermann" <gpetermann_muenc...@hotmail.com>
Datum: Dienstag, 6. Februar 2018 12:15
An: "Thomas Morgenstern" <webmas...@img2ms.de>; "Development list for
mkgmap" <mkgmap-dev@lists.mkgmap.org.uk>
Betreff: Re: [mkgmap-dev] splitter r590,wrong handling of relations


Hi Thomas,

I think the problem is that the relation is broken. Splitter writes it to
both tiles, but
mkgmap seems to have problems with it.
Anyway, the problem is in the data and should be fixed in OSM.

Gerd



Von: mkgmap-dev <mkgmap-dev-boun...@lists.mkgmap.org.uk> im Auftrag von
Thomas Morgenstern <webmas...@img2ms.de>
Gesendet: Dienstag, 6. Februar 2018 12:50
An: Development list for mkgmap
Betreff: [mkgmap-dev] splitter r590,wrong handling of relations

Hi Gerd, i found out, that splitter r590 can not proper handle the
relation
2.691.138. I can reproduce the problem. It makes no different, if I 
splitt

the canary-islands-latest.osm.pbf with the default-option or with my own
splitt.list. If I move the tileborder by changing the nord-south border,
also the problem is moved. See picture at
http://img2ms.de/OSM/ErrorInSplitter-r590.jpg. At the tile-boundary
splitter
creates only for the rightsite tile the polygon for  tag
boundary=protected_area , respektive leisure=nature_reserve. In the
leftsite-tile this tags are missing.
maybee you can have a look at that ?

regards thomas



___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter r590,wrong handling of relations

2018-02-06 Thread Gerd Petermann

Hi Thomas,

I've just fixed the relation [1], so please try again with a fresh download 
tomorrow.

Gerd
[1] https://www.openstreetmap.org/changeset/56139020


Von: mkgmap-dev <mkgmap-dev-boun...@lists.mkgmap.org.uk> im Auftrag von Gerd 
Petermann <gpetermann_muenc...@hotmail.com>
Gesendet: Dienstag, 6. Februar 2018 14:47
An: Thomas Morgenstern; Development list for mkgmap
Betreff: Re: [mkgmap-dev] splitter r590,wrong handling of relations

Hi Thomas,

the relation is broken. Use JOSM validator and it will report 2 errors.
I think this way
https://www.openstreetmap.org/way/556302821
should not belong to it and a few nodes should be merged to repiar it:
node 4298102908
node 4298102907
node 655141348

Gerd


Von: mkgmap-dev <mkgmap-dev-boun...@lists.mkgmap.org.uk> im Auftrag von Thomas 
Morgenstern <webmas...@img2ms.de>
Gesendet: Dienstag, 6. Februar 2018 14:35
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] splitter r590,wrong handling of relations

Hi Gerd, i have now made a different split, so that 1 tile contains the
whole isle. In this case it looks good. I assume, that the relation is okay.
No idea what should be changed in data. Also i have looked before with JOSM
in the old split and  confirm, that in the left tile the relation also
exists, but not rendered from mkgmap. I can't judge, if the relation is
broken or not. Therefore the problem is in mkgmap ? I
thomas

--
Von: "Gerd Petermann" <gpetermann_muenc...@hotmail.com>
Datum: Dienstag, 6. Februar 2018 12:15
An: "Thomas Morgenstern" <webmas...@img2ms.de>; "Development list for
mkgmap" <mkgmap-dev@lists.mkgmap.org.uk>
Betreff: Re: [mkgmap-dev] splitter r590,wrong handling of relations

> Hi Thomas,
>
> I think the problem is that the relation is broken. Splitter writes it to
> both tiles, but
> mkgmap seems to have problems with it.
> Anyway, the problem is in the data and should be fixed in OSM.
>
> Gerd
>
>
> 
> Von: mkgmap-dev <mkgmap-dev-boun...@lists.mkgmap.org.uk> im Auftrag von
> Thomas Morgenstern <webmas...@img2ms.de>
> Gesendet: Dienstag, 6. Februar 2018 12:50
> An: Development list for mkgmap
> Betreff: [mkgmap-dev] splitter r590,wrong handling of relations
>
> Hi Gerd, i found out, that splitter r590 can not proper handle the
> relation
> 2.691.138. I can reproduce the problem. It makes no different, if I splitt
> the canary-islands-latest.osm.pbf with the default-option or with my own
> splitt.list. If I move the tileborder by changing the nord-south border,
> also the problem is moved. See picture at
> http://img2ms.de/OSM/ErrorInSplitter-r590.jpg. At the tile-boundary
> splitter
> creates only for the rightsite tile the polygon for  tag
> boundary=protected_area , respektive leisure=nature_reserve. In the
> leftsite-tile this tags are missing.
> maybee you can have a look at that ?
>
> regards thomas
>
>
>
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter r590,wrong handling of relations

2018-02-06 Thread Gerd Petermann

Hi Thomas,

the relation is broken. Use JOSM validator and it will report 2 errors.
I think this way
https://www.openstreetmap.org/way/556302821
should not belong to it and a few nodes should be merged to repiar it:
node 4298102908
node 4298102907
node 655141348

Gerd


Von: mkgmap-dev <mkgmap-dev-boun...@lists.mkgmap.org.uk> im Auftrag von Thomas 
Morgenstern <webmas...@img2ms.de>
Gesendet: Dienstag, 6. Februar 2018 14:35
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] splitter r590,wrong handling of relations

Hi Gerd, i have now made a different split, so that 1 tile contains the
whole isle. In this case it looks good. I assume, that the relation is okay.
No idea what should be changed in data. Also i have looked before with JOSM
in the old split and  confirm, that in the left tile the relation also
exists, but not rendered from mkgmap. I can't judge, if the relation is
broken or not. Therefore the problem is in mkgmap ? I
thomas

--
Von: "Gerd Petermann" <gpetermann_muenc...@hotmail.com>
Datum: Dienstag, 6. Februar 2018 12:15
An: "Thomas Morgenstern" <webmas...@img2ms.de>; "Development list for
mkgmap" <mkgmap-dev@lists.mkgmap.org.uk>
Betreff: Re: [mkgmap-dev] splitter r590,wrong handling of relations

> Hi Thomas,
>
> I think the problem is that the relation is broken. Splitter writes it to
> both tiles, but
> mkgmap seems to have problems with it.
> Anyway, the problem is in the data and should be fixed in OSM.
>
> Gerd
>
>
> 
> Von: mkgmap-dev <mkgmap-dev-boun...@lists.mkgmap.org.uk> im Auftrag von
> Thomas Morgenstern <webmas...@img2ms.de>
> Gesendet: Dienstag, 6. Februar 2018 12:50
> An: Development list for mkgmap
> Betreff: [mkgmap-dev] splitter r590,wrong handling of relations
>
> Hi Gerd, i found out, that splitter r590 can not proper handle the
> relation
> 2.691.138. I can reproduce the problem. It makes no different, if I splitt
> the canary-islands-latest.osm.pbf with the default-option or with my own
> splitt.list. If I move the tileborder by changing the nord-south border,
> also the problem is moved. See picture at
> http://img2ms.de/OSM/ErrorInSplitter-r590.jpg. At the tile-boundary
> splitter
> creates only for the rightsite tile the polygon for  tag
> boundary=protected_area , respektive leisure=nature_reserve. In the
> leftsite-tile this tags are missing.
> maybee you can have a look at that ?
>
> regards thomas
>
>
>
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter r590,wrong handling of relations

2018-02-06 Thread Thomas Morgenstern

Hi Gerd, i have now made a different split, so that 1 tile contains the 
whole isle. In this case it looks good. I assume, that the relation is okay. 
No idea what should be changed in data. Also i have looked before with JOSM 
in the old split and  confirm, that in the left tile the relation also 
exists, but not rendered from mkgmap. I can't judge, if the relation is 
broken or not. Therefore the problem is in mkgmap ? I

thomas

--
Von: "Gerd Petermann" <gpetermann_muenc...@hotmail.com>
Datum: Dienstag, 6. Februar 2018 12:15
An: "Thomas Morgenstern" <webmas...@img2ms.de>; "Development list for 
mkgmap" <mkgmap-dev@lists.mkgmap.org.uk>

Betreff: Re: [mkgmap-dev] splitter r590,wrong handling of relations


Hi Thomas,

I think the problem is that the relation is broken. Splitter writes it to 
both tiles, but

mkgmap seems to have problems with it.
Anyway, the problem is in the data and should be fixed in OSM.

Gerd



Von: mkgmap-dev <mkgmap-dev-boun...@lists.mkgmap.org.uk> im Auftrag von 
Thomas Morgenstern <webmas...@img2ms.de>

Gesendet: Dienstag, 6. Februar 2018 12:50
An: Development list for mkgmap
Betreff: [mkgmap-dev] splitter r590,wrong handling of relations

Hi Gerd, i found out, that splitter r590 can not proper handle the 
relation

2.691.138. I can reproduce the problem. It makes no different, if I splitt
the canary-islands-latest.osm.pbf with the default-option or with my own
splitt.list. If I move the tileborder by changing the nord-south border,
also the problem is moved. See picture at
http://img2ms.de/OSM/ErrorInSplitter-r590.jpg. At the tile-boundary 
splitter

creates only for the rightsite tile the polygon for  tag
boundary=protected_area , respektive leisure=nature_reserve. In the
leftsite-tile this tags are missing.
maybee you can have a look at that ?

regards thomas



___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter r590,wrong handling of relations

2018-02-06 Thread Gerd Petermann

Hi Thomas,

I think the problem is that the relation is broken. Splitter writes it to both 
tiles, but
mkgmap seems to have problems with it.
Anyway, the problem is in the data and should be fixed in OSM.

Gerd



Von: mkgmap-dev  im Auftrag von Thomas 
Morgenstern 
Gesendet: Dienstag, 6. Februar 2018 12:50
An: Development list for mkgmap
Betreff: [mkgmap-dev] splitter r590,wrong handling of relations

Hi Gerd, i found out, that splitter r590 can not proper handle the relation
2.691.138. I can reproduce the problem. It makes no different, if I splitt
the canary-islands-latest.osm.pbf with the default-option or with my own
splitt.list. If I move the tileborder by changing the nord-south border,
also the problem is moved. See picture at
http://img2ms.de/OSM/ErrorInSplitter-r590.jpg. At the tile-boundary splitter
creates only for the rightsite tile the polygon for  tag
boundary=protected_area , respektive leisure=nature_reserve. In the
leftsite-tile this tags are missing.
maybee you can have a look at that ?

regards thomas



___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

1 2 3 4 5 6 7 8 9 >

1 - 100 of 876 matches

Mail list logo