Re: [mkgmap-dev] MapFailedException when generating map for Italy

2017-08-16 Thread Walter Schlögl 

Hi Gerd,

I think I should have looked at the logfiles earlier,
but since I do not really understand the splitter log, I tend not to read it 
carefully.


I found out, that the problem was gone even with the old parameter for 
max-nodes.

The solution was just to splitt the area again.

I checked the original europe.o5m, it was correct,
I checked the old 50100031.o5m and it was wrong as you said.
I compared the 2 splitter logfiles that have generated the wrong and the 
correct version

(I have also copied them for your reference)
and I found the first importand difference at line 564

correct run:
dictionary contains now 1.000 entries
...
Number of stored area combis for nodes: 154.673.904
Number of stored area combis for ways: 16.877.672
Number of stored Integers for rels: 0
Number of stored combis in dictionary: 1.449
Number of detected problem ways: 64.970
Number of detected problem rels: 6.311
JVM Memory Info: Current 2558MB (1865MB used, 693MB free) Max 5461MB


wrong run:
dictionary contains now 1.000 entries
dictionary contains now 2.000 entries
...
dictionary contains now 34.000 entries
...
Number of stored area combis for nodes: 154.672.668
Number of stored area combis for ways: 11.580.442
Number of stored Integers for rels: 0
Number of stored combis in dictionary: 34.620
Number of detected problem ways: 608.980
Number of detected problem rels: 25.075
JVM Memory Info: Current 2555MB (1954MB used, 601MB free) Max 5461MB

Of course I don't know what this means.

Since I am executing all this via Scripts that I did not change between 
these 2 runs,
my only explanation is, that splitter (or my computer) did not had the best 
day last saturday.


Walter


-Ursprüngliche Nachricht- 
From: Gerd Petermann

Sent: Wednesday, August 16, 2017 10:11 AM
To: Development list for mkgmap
Subject: Re: [mkgmap-dev] MapFailedException when generating map for Italy

Hi Walter,

another possible way to verify the data in your europe.o5m:
extract a node that is corrupted in 50100031.o5m, e.g.
osmfilter europe.o5m --keep="@id=1965123604" > test.osm

When I do this with 50100031.o5m I see

   

while the correct data should look like this:
   version="1"/>


Gerd

Von: mkgmap-dev  im Auftrag von Gerd 
Petermann 

Gesendet: Mittwoch, 16. August 2017 08:12:18
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] MapFailedException when generating map for Italy

Hi Walter,

The data in 50100031.o5m  looks very strange, maybe there is a special case 
in splitter.
So if you still have the same europe.o5m file, it would be great if you 
could try to reproduce the

problem. If you can, please send the splitter log and the areas.list.

Gerd

Von: mkgmap-dev  im Auftrag von 
Walter Schlögl 

Gesendet: Dienstag, 15. August 2017 22:29:33
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] MapFailedException when generating map for Italy

Hi Gerd,

I did not try to reproduce the corrupted file,
but I did not update the original italy.o5m file
that I used for splitting it with the lower maxnodes value.

My workflow has 4 steps.

1. update europe.o5m with daily diffs
2. cropping a country or region out of europe
3. splitter
4. mkgmap

Since I only repeated step 3 and 4,
I think that my splitter run has generated the corruption.
But since I remember also a corruption in step 1 some weeks ago,
this assumption could be wrong.

Walter

-Ursprüngliche Nachricht-
From: Gerd Petermann
Sent: Tuesday, August 15, 2017 7:19 PM
To: Development list for mkgmap
Subject: Re: [mkgmap-dev] MapFailedException when generating map for Italy

Hi Walter,

does that mean that you can still reproduce the corrupted file when using
the old maxnodes value?

Gerd


Von: mkgmap-dev  im Auftrag von
Walter Schlögl 
Gesendet: Dienstag, 15. August 2017 18:31:44
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] MapFailedException when generating map for Italy

Hi Gerd,

thanks for this info.
This could happen by merging the daily diffs into europe.o5m
But since the working version was built out of the same file,
I could also happen during the splitting.

Very strange, but now I know what to look at if it happens again.

Walter

-Ursprüngliche Nachricht-
From: Gerd Petermann
Sent: Tuesday, August 15, 2017 3:07 PM
To: Development list for mkgmap
Subject: Re: [mkgmap-dev] MapFailedException when generating map for Italy

Hi Walter,

something is wrong with the data in 50100031.o5m
It contains a lot of very long lines which do not appear in OSM. I guess you
have merged different data sources
and they use similar OSM ids for different objects (typically this happens
with SRTM data). Please

[mkgmap-dev] Commit r3993: don't create shop POI for shop=no or shop=none

2017-08-16 Thread svn commit 
Version mkgmap-r3993 was committed by gerd on Wed, 16 Aug 2017

don't create shop POI for shop=no or shop=none

http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap=3993
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

[mkgmap-dev] "shop=no"

2017-08-16 Thread Bernhard Hiller 
In the forum, there is a discussion on the enormous variety of values 
for some tags, see 
https://forum.openstreetmap.org/viewtopic.php?id=55541 (in Dutch).
There is also a link to a list showing the different values for the 
"shop" tag: 
http://mijndev.openstreetmap.nl/~marczoutendijk/taginfo-shop.html


You can see there that the combination "shop=no" exists 2372 times, 
"shop=none" 4 times.


With our current default style, they'd be summed up by the line
 shop=* [0x2e0c resolution 24]
which just would make them into a shop.

I guess we should better ignore the shop=no/none combination (note: a 
value containing "no" may still be correct), and provide a PoI based on 
the other tags for that node.

What do you think?

Regards,
Bernhard
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] MapFailedException when generating map for Italy

2017-08-16 Thread Gerd Petermann 
Hi Walter,

another possible way to verify the data in your europe.o5m:
extract a node that is corrupted in 50100031.o5m, e.g.
osmfilter europe.o5m --keep="@id=1965123604" > test.osm

When I do this with 50100031.o5m I see



while the correct data should look like this:


Gerd

Von: mkgmap-dev  im Auftrag von Gerd 
Petermann 
Gesendet: Mittwoch, 16. August 2017 08:12:18
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] MapFailedException when generating map for Italy

Hi Walter,

The data in 50100031.o5m  looks very strange, maybe there is a special case in 
splitter.
So if you still have the same europe.o5m file, it would be great if you could 
try to reproduce the
problem. If you can, please send the splitter log and the areas.list.

Gerd

Von: mkgmap-dev  im Auftrag von Walter 
Schlögl 
Gesendet: Dienstag, 15. August 2017 22:29:33
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] MapFailedException when generating map for Italy

Hi Gerd,

I did not try to reproduce the corrupted file,
but I did not update the original italy.o5m file
that I used for splitting it with the lower maxnodes value.

My workflow has 4 steps.

1. update europe.o5m with daily diffs
2. cropping a country or region out of europe
3. splitter
4. mkgmap

Since I only repeated step 3 and 4,
I think that my splitter run has generated the corruption.
But since I remember also a corruption in step 1 some weeks ago,
this assumption could be wrong.

Walter

-Ursprüngliche Nachricht-
From: Gerd Petermann
Sent: Tuesday, August 15, 2017 7:19 PM
To: Development list for mkgmap
Subject: Re: [mkgmap-dev] MapFailedException when generating map for Italy

Hi Walter,

does that mean that you can still reproduce the corrupted file when using
the old maxnodes value?

Gerd


Von: mkgmap-dev  im Auftrag von
Walter Schlögl 
Gesendet: Dienstag, 15. August 2017 18:31:44
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] MapFailedException when generating map for Italy

Hi Gerd,

thanks for this info.
This could happen by merging the daily diffs into europe.o5m
But since the working version was built out of the same file,
I could also happen during the splitting.

Very strange, but now I know what to look at if it happens again.

Walter

-Ursprüngliche Nachricht-
From: Gerd Petermann
Sent: Tuesday, August 15, 2017 3:07 PM
To: Development list for mkgmap
Subject: Re: [mkgmap-dev] MapFailedException when generating map for Italy

Hi Walter,

something is wrong with the data in 50100031.o5m
It contains a lot of very long lines which do not appear in OSM. I guess you
have merged different data sources
and they use similar OSM ids for different objects (typically this happens
with SRTM data). Please check your tool chain
and make sure that generated ids are much higher than those in real OSM
data. The node ids recently jumped over 5.000.000.000, see
https://osmstats.neis-one.org/?item=nodes

The tile in 50100025.o5m confirms that you have a tile that contains data
from italy and from Malta (which is a drive-on-left country).

Gerd




Von: mkgmap-dev  im Auftrag von Gerd
Petermann 
Gesendet: Dienstag, 15. August 2017 14:01:15
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] MapFailedException when generating map for Italy

Hi Walter,

I'll see if I can reproduce the crash. Reg. the remaining message:
I guess the tile contains also Malta?

Gerd

Von: mkgmap-dev  im Auftrag von
Walter Schlögl 
Gesendet: Dienstag, 15. August 2017 12:36:31
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] MapFailedException when generating map for Italy

Hi Gerd,

many thanks for your hints, with this I could solve all issues.

I have changed the splitter parameter from --max-nodes=80 to 60
but I have not updated the osm data to see what happens.
The message "There is not enough room" is gone now as expected,
but also the NullPointerException is gone.

I have copied the tile anyway if you still see a need for analysis.

https://1drv.ms/f/s!AqiYxUeSl56qgnCes1X0pcV4NIQ9

The only message that I still get for Italy is this.
SCHWERWIEGEND (StyledConverter): 50100025.o5m: Attention: Tile contains both
drive-on-left (22469) and drive-on-right roads (25540)

I would expect such a message for other countries, but not for Italy.
If this message is harmless, I will just ignore it.

Walter


-Ursprüngliche Nachricht-
From: Gerd Petermann
Sent: Monday,

Re: [mkgmap-dev] MapFailedException when generating map for Italy

2017-08-16 Thread Gerd Petermann 
Hi Walter,

The data in 50100031.o5m  looks very strange, maybe there is a special case in 
splitter.
So if you still have the same europe.o5m file, it would be great if you could 
try to reproduce the
problem. If you can, please send the splitter log and the areas.list.

Gerd

Von: mkgmap-dev  im Auftrag von Walter 
Schlögl 
Gesendet: Dienstag, 15. August 2017 22:29:33
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] MapFailedException when generating map for Italy

Hi Gerd,

I did not try to reproduce the corrupted file,
but I did not update the original italy.o5m file
that I used for splitting it with the lower maxnodes value.

My workflow has 4 steps.

1. update europe.o5m with daily diffs
2. cropping a country or region out of europe
3. splitter
4. mkgmap

Since I only repeated step 3 and 4,
I think that my splitter run has generated the corruption.
But since I remember also a corruption in step 1 some weeks ago,
this assumption could be wrong.

Walter

-Ursprüngliche Nachricht-
From: Gerd Petermann
Sent: Tuesday, August 15, 2017 7:19 PM
To: Development list for mkgmap
Subject: Re: [mkgmap-dev] MapFailedException when generating map for Italy

Hi Walter,

does that mean that you can still reproduce the corrupted file when using
the old maxnodes value?

Gerd


Von: mkgmap-dev  im Auftrag von
Walter Schlögl 
Gesendet: Dienstag, 15. August 2017 18:31:44
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] MapFailedException when generating map for Italy

Hi Gerd,

thanks for this info.
This could happen by merging the daily diffs into europe.o5m
But since the working version was built out of the same file,
I could also happen during the splitting.

Very strange, but now I know what to look at if it happens again.

Walter

-Ursprüngliche Nachricht-
From: Gerd Petermann
Sent: Tuesday, August 15, 2017 3:07 PM
To: Development list for mkgmap
Subject: Re: [mkgmap-dev] MapFailedException when generating map for Italy

Hi Walter,

something is wrong with the data in 50100031.o5m
It contains a lot of very long lines which do not appear in OSM. I guess you
have merged different data sources
and they use similar OSM ids for different objects (typically this happens
with SRTM data). Please check your tool chain
and make sure that generated ids are much higher than those in real OSM
data. The node ids recently jumped over 5.000.000.000, see
https://osmstats.neis-one.org/?item=nodes

The tile in 50100025.o5m confirms that you have a tile that contains data
from italy and from Malta (which is a drive-on-left country).

Gerd




Von: mkgmap-dev  im Auftrag von Gerd
Petermann 
Gesendet: Dienstag, 15. August 2017 14:01:15
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] MapFailedException when generating map for Italy

Hi Walter,

I'll see if I can reproduce the crash. Reg. the remaining message:
I guess the tile contains also Malta?

Gerd

Von: mkgmap-dev  im Auftrag von
Walter Schlögl 
Gesendet: Dienstag, 15. August 2017 12:36:31
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] MapFailedException when generating map for Italy

Hi Gerd,

many thanks for your hints, with this I could solve all issues.

I have changed the splitter parameter from --max-nodes=80 to 60
but I have not updated the osm data to see what happens.
The message "There is not enough room" is gone now as expected,
but also the NullPointerException is gone.

I have copied the tile anyway if you still see a need for analysis.

https://1drv.ms/f/s!AqiYxUeSl56qgnCes1X0pcV4NIQ9

The only message that I still get for Italy is this.
SCHWERWIEGEND (StyledConverter): 50100025.o5m: Attention: Tile contains both
drive-on-left (22469) and drive-on-right roads (25540)

I would expect such a message for other countries, but not for Italy.
If this message is harmless, I will just ignore it.

Walter


-Ursprüngliche Nachricht-
From: Gerd Petermann
Sent: Monday, August 14, 2017 10:02 AM
To: Development list for mkgmap
Subject: Re: [mkgmap-dev] MapFailedException when generating map for Italy

Hi Walter,

I think you have several problems here.
1) The message "There is not enough room in a single garmin map for all the
input data. The .osm file should be split into smaller pieces first."
means that you should use a smaller maxnodes value  in splitter (or change
the style to add fewer data)
2) in 50100031.o5m there seems to be a data error. If I got that right the
error was already fixed in the OSM database (way 195550142 was changed
recently),
so I probably cannot