Re: [mkgmap-dev] It's clearly a bug in the splitter

2012-10-17 Thread Felix Hartmann 
Well, I tried to render luxembourg using the osm.pbf file from 
geofabrik, without splitting it, and than the map is good. As soon as I 
split it (also if the output is only one single osm.pbf tile),

I have missing portions in the map.

I use the following commands for splitting (newest splitter):
java -Xms4000m -Xmx6800m -jar splitter.jar --max-nodes=125 
--output=pbf --overlap=5000  --geonames-file=cities15000 
--description=luxembourg --mapid=6423 luxembourg.osm.pbf



And this is the splitter output:
cache=
description=luxembourg
geonames-file=cities15000
legacy-mode=false
mapid=6423
max-areas=255
max-nodes=125
max-threads=8 (auto)
mixed=false
no-trim=false
output=pbf
output-dir=
overlap=5000
resolution=13
split-file=
status-freq=120
write-kml=
Elapsed time: 0s   Memory: Current 3833MB (20MB used, 3813MB free) Max 
6044MB

Time started: Wed Oct 17 13:18:08 CEST 2012
Map is being split for resolution 13:
 - area boundaries are aligned to 0x800 map units
 - areas are multiples of 0x1000 map units wide and high
Processing c:\openmtbmap\osmpbf_geofabrik\luxembourg.osm.pbf
Bounding box 5.7330331 49.4455305 6.532249 
50.1849604

in 1 file
Time: Wed Oct 17 13:18:09 CEST 2012
Exact map coverage is (49.44551467895508,5.733017921447754) to 
(50.184946060180664,6.532230377197266)
Trimmed and rounded map coverage is (49.482421875,5.712890625) to 
(50.185546875,6.50390625)

Splitting nodes into areas containing a maximum of 1'250'000 nodes each...
Area (49.482421875,5.712890625) to (50.185546875,6.50390625) contains 
997'119 nodes. DONE!

1 areas:
Area 6423 covers (0x233000,0x41000) to (0x23b000,0x4a000) LU-Luxembourg
Writing out split osm files Wed Oct 17 13:18:09 CEST 2012
Processing 1 areas in a single pass
(49.482421875,5.712890625) to (50.185546875,6.50390625)
Starting pass 1 of 1, processing 1 areas (6423 to 6423)
Grid [512][512] for grid area (49.3751335144043,5.605602264404297) to 
(50.2928352355957,6.611194610595703) requires max. 1 checks for each no

Grid was created in 62 ms
Allocating three-tier structure to save area info 
(HashMap->vector->chunkvector)
Allocating three-tier structure to save area info 
(HashMap->vector->chunkvector)

Processing c:\openmtbmap\osmpbf_geofabrik\luxembourg.osm.pbf
Bounding box 5.7330331 49.4455305 6.532249 
50.1849604

MAP occupancy: 1'000'000, number of area dictionary entries: 1 of 65535
Map details: HashMap -> 7 vectors for 88'816 chunks(vector usage < 1%)
Writing ways Wed Oct 17 13:18:11 CEST 2012
Writing relations Wed Oct 17 13:18:13 CEST 2012
***
Final statistics
***
Needed dictionary entries: 1 of 65535
coords occupancy
MAP occupancy: 1'036'326
Length-6 chunks: 91'338 (Bytes: 2'192'112)
Length-10 chunks: 0 (Bytes: 0)
Length-14 chunks: 0 (Bytes: 0)
Length-18 chunks: 0 (Bytes: 0)
Length-22 chunks: 0 (Bytes: 0)
Length-26 chunks: 0 (Bytes: 0)
Length-30 chunks: 0 (Bytes: 0)
Length-34 chunks: 0 (Bytes: 0)
Length-38 chunks: 0 (Bytes: 0)
Length-42 chunks: 0 (Bytes: 0)
Length-46 chunks: 0 (Bytes: 0)
Length-50 chunks: 0 (Bytes: 0)
Length-54 chunks: 0 (Bytes: 0)
Length-58 chunks: 0 (Bytes: 0)
Length-62 chunks: 0 (Bytes: 0)
Length-66 chunks: 0 (Bytes: 0)
Length-68 chunks: 0 (Bytes: 0)
RLE compresion info: compressed / uncompressed size / ratio: 548'028 / 
1'526'290 / 65%

Map details: HashMap -> 8 vectors for 91'338 chunks(vector usage < 1%)
ways occupancy
MAP occupancy: 103'579
Length-6 chunks: 29'433 (Bytes: 706'392)
Length-10 chunks: 0 (Bytes: 0)
Length-14 chunks: 0 (Bytes: 0)
Length-18 chunks: 0 (Bytes: 0)
Length-22 chunks: 0 (Bytes: 0)
Length-26 chunks: 0 (Bytes: 0)
Length-30 chunks: 0 (Bytes: 0)
Length-34 chunks: 0 (Bytes: 0)
Length-38 chunks: 0 (Bytes: 0)
Length-42 chunks: 0 (Bytes: 0)
Length-46 chunks: 0 (Bytes: 0)
Length-50 chunks: 0 (Bytes: 0)
Length-54 chunks: 0 (Bytes: 0)
Length-58 chunks: 0 (Bytes: 0)
Length-62 chunks: 0 (Bytes: 0)
Length-66 chunks: 0 (Bytes: 0)
Length-68 chunks: 0 (Bytes: 0)
RLE compresion info: compressed / uncompressed size / ratio: 176'598 / 
254'588 / 31%

Map details: HashMap -> 1 vectors for 29'433 chunks(vector usage < 2%)

Thread worker-3 has finished
Thread worker-2 has finished
Thread worker-5 has finished
Thread worker-4 has finished
Thread worker-6 has finished
Thread worker-0 has finished
Thread worker-1 has finished
Time finished: Wed Oct 17 13:18:13 CEST 2012
Total time taken: 5s

On 17.10.2012 12:54, Felix Hartmann wrote:
See the following screenshots - map vs josm - look at the highway 
junction both times on the top as a reference. There is about 1km 
missing - I really cannot understand how and why this happens.. (and 
not there is no related error message or missing tile), in the east of 
luxembourg, mkgmap also cuts away a bit.:




--
keep on biking and discovering new trails

Felix
openmtbmap.org &www.velomap.org


--
keep on biking and

[mkgmap-dev] [Patch V1]Re: Still problems with lakes

2012-10-17 Thread GerdP 
Hi,

here is a first try to fix this issue.
splitter_problem_list.patch
  

A sample list of problem polygons:
problem_polygons.txt
  

Specify it in the new parameter problem-file, e.g. 
--problem-file=./problem_polygons.txt

It is recommended to use o5m format as input. To convert an existing osm.pbf
file to o5m use e.g.
osmconvert --drop-version --out-o5m europe.osm.pbf > europe.o5m

Sorry, the new code is not well commented yet, but I think it works quite
well now. 
Any feedback is welcome.

Ciao,
GerdP


RheinSkipper wrote
>> Thanks for the feedback. I'll try coding a few changes (e.g. filtering by
> tags), if 
>> that doesn't work out, I'll code the handling of a external list and
>> we'll
> see what it helps.
>> 
>> Gerd
> 
> 
> Thank you so much!
> 
> ___
> mkgmap-dev mailing list

> mkgmap-dev@.org

> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev





--
View this message in context: 
http://gis.19327.n5.nabble.com/Still-problems-with-lakes-tp5725668p5731258.html
Sent from the Mkgmap Development mailing list archive at Nabble.com.
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] mkgmap splitter or mkgmap leave out information on luxembourg.osm.pbf from geofabrik

2012-10-17 Thread n Willink 
Hi Felix
I have encountered similar problems when splitter only produces one pbf file
- see previous posting about tdb & mdr issues

Have you tried reducing the max-nodes to obtain 2 pbfs - that seems to do
the trick with me

/nick



--
View this message in context: 
http://gis.19327.n5.nabble.com/mkgmap-splitter-or-mkgmap-leave-out-information-on-luxembourg-osm-pbf-from-geofabrik-tp5731208p5731262.html
Sent from the Mkgmap Development mailing list archive at Nabble.com.
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] mkgmap splitter or mkgmap leave out information on luxembourg.osm.pbf from geofabrik

2012-10-17 Thread Felix Hartmann 

On 17.10.2012 17:10, n Willink wrote:
> Hi Felix
> I have encountered similar problems when splitter only produces one pbf file
> - see previous posting about tdb & mdr issues
>
> Have you tried reducing the max-nodes to obtain 2 pbfs - that seems to do
> the trick with me
>
> /nick
yes doesn't change anything. On Luxembourg the output is exactly the 
same. There is some really serious bug in the splitter. Also --no-trim 
changes absolutely nothing.
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] It's clearly a bug in the splitter

2012-10-17 Thread GerdP 
Hi Felix,


> Exact map coverage is (49.44551467895508,5.733017921447754) to
> (50.184946060180664,6.532230377197266)
> Trimmed and rounded map coverage is (49.482421875,5.712890625) to
> (50.185546875,6.50390625)

I think the reason is that splitter writes a different bounding box to the
file. I am not sure, but it seems to be an error in RoundingUtils.java:

int roundedMinLon = roundDown(b.getMinLong(), shift);
int roundedMaxLon = roundDown(b.getMaxLong(), shift);

Both the min and the max value are rounded down. This looks wrong.
Similar problem with the latitude values:
int roundedMinLat = roundUp(minLat, shift);
int roundedMaxLat = roundUp(maxLat, shift);


Ciao,
Gerd




--
View this message in context: 
http://gis.19327.n5.nabble.com/mkgmap-splitter-or-mkgmap-leave-out-information-on-luxembourg-osm-pbf-from-geofabrik-tp5731208p5731274.html
Sent from the Mkgmap Development mailing list archive at Nabble.com.
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] [Patch V1]Re: Still problems with lakes

2012-10-17 Thread Jaromír Mikeš 
> Od: GerdP 

Hi Gerd,

> here is a first try to fix this issue.
> splitter_problem_list.patch
>   

I would like to try your patch as I have problem with glacier polygon when I 
build map of Greenland.
I patched r200 splitter directory ... how to build splitter.jar file now?

> A sample list of problem polygons:
> problem_polygons.txt
>   
> 
> Specify it in the new parameter problem-file, e.g. 
> --problem-file=./problem_polygons.txt

> It is recommended to use o5m format as input. To convert an existing osm.pbf
> file to o5m use e.g.
> osmconvert --drop-version --out-o5m europe.osm.pbf > europe.o5m

Why not osm.pbf format?

Thanks for your work

mira
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] It's clearly a bug in the splitter

2012-10-17 Thread WanMil 
I think this was discussed some time ago:
http://www.mkgmap.org.uk/pipermail/mkgmap-dev/2011q3/012193.html

WanMil

> Hi Felix,
>
>
>> Exact map coverage is (49.44551467895508,5.733017921447754) to
>> (50.184946060180664,6.532230377197266)
>> Trimmed and rounded map coverage is (49.482421875,5.712890625) to
>> (50.185546875,6.50390625)
>
> I think the reason is that splitter writes a different bounding box to the
> file. I am not sure, but it seems to be an error in RoundingUtils.java:
>
>   int roundedMinLon = roundDown(b.getMinLong(), shift);
>   int roundedMaxLon = roundDown(b.getMaxLong(), shift);
>
> Both the min and the max value are rounded down. This looks wrong.
> Similar problem with the latitude values:
>   int roundedMinLat = roundUp(minLat, shift);
>   int roundedMaxLat = roundUp(maxLat, shift);
>
>
> Ciao,
> Gerd
>
>
>
>
> --
> View this message in context: 
> http://gis.19327.n5.nabble.com/mkgmap-splitter-or-mkgmap-leave-out-information-on-luxembourg-osm-pbf-from-geofabrik-tp5731208p5731274.html
> Sent from the Mkgmap Development mailing list archive at Nabble.com.
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] It's clearly a bug in the splitter

2012-10-17 Thread Felix Hartmann 


On 17.10.2012 18:01, GerdP wrote:

int roundedMinLon = roundDown(b.getMinLong(), shift);
int roundedMaxLon = roundDown(b.getMaxLong(), shift);

Both the min and the max value are rounded down. This looks wrong.
Similar problem with the latitude values:
int roundedMinLat = roundUp(minLat, shift);
int roundedMaxLat = roundUp(maxLat, shift);
thanks, I changed it to the following - and now it works. I think 
someone should have a look at it, and check it in, as it makes much more 
sense


int roundedMinLon = roundDown(b.getMinLong(), shift);
int roundedMaxLon = round*Up*(b.getMaxLong(), shift);


int roundedMinLat = round*Down*(minLat, shift);
int roundedMaxLat = roundUp(maxLat, shift);


--
keep on biking and discovering new trails

Felix
openmtbmap.org & www.velomap.org

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] It's clearly a bug in the splitter

2012-10-17 Thread Felix Hartmann 
Actually the way I put the rounding now (to include everything) - is the 
way it has been up to rev 104, then came rev105 and the change 
(28.1.2010) - the patch did the same thing as I did.
However actually now that I think about it - I can only come up with one 
explication why rev 105 on splitter was done - that is so that tiles 
don't overlap. Because the current hack means all tiles will be expanded.

What we really need is to have the splitter identify ALL outside edges. 
Only on outside edges it shall do as I wrote lower down, on edges to the 
inside rev 105 is the way to go.
So can anyone code that in?
On 17.10.2012 20:23, WanMil wrote:
> I think this was discussed some time ago:
> http://www.mkgmap.org.uk/pipermail/mkgmap-dev/2011q3/012193.html
>
> WanMil
>
>> Hi Felix,
>>
>>
>>> Exact map coverage is (49.44551467895508,5.733017921447754) to
>>> (50.184946060180664,6.532230377197266)
>>> Trimmed and rounded map coverage is (49.482421875,5.712890625) to
>>> (50.185546875,6.50390625)
>> I think the reason is that splitter writes a different bounding box to the
>> file. I am not sure, but it seems to be an error in RoundingUtils.java:
>>
>>  int roundedMinLon = roundDown(b.getMinLong(), shift);
>>  int roundedMaxLon = roundDown(b.getMaxLong(), shift);
>>
>> Both the min and the max value are rounded down. This looks wrong.
>> Similar problem with the latitude values:
>>  int roundedMinLat = roundUp(minLat, shift);
>>  int roundedMaxLat = roundUp(maxLat, shift);
>>
>>
>> Ciao,
>> Gerd
>>
>>
>>
>>
>> --
>> View this message in context: 
>> http://gis.19327.n5.nabble.com/mkgmap-splitter-or-mkgmap-leave-out-information-on-luxembourg-osm-pbf-from-geofabrik-tp5731208p5731274.html
>> Sent from the Mkgmap Development mailing list archive at Nabble.com.
>> ___
>> mkgmap-dev mailing list
>> mkgmap-dev@lists.mkgmap.org.uk
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

-- 
keep on biking and discovering new trails

Felix
openmtbmap.org & www.velomap.org

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] [Patch V1]Re: Still problems with lakes

2012-10-17 Thread Jaromír Mikeš 
> Od: Gerd Petermann 

Hi Gerd,

> why o5m: o5m is much faster to read compared to pbf, and the new algorithm
> requires a few  more reads

As Greenland map hasn't too much data I stayed with pbf ... not really longer 
splitting.

> > > Specify it in the new parameter problem-file, e.g. 
> > > --problem-file=./problem_polygons.txt

I added " rel:1279614 # Greenland glacier " to my problem_polygons.txt

Now I nearly whole glacier! Amazing improvement!
I still missing southern tip of glacier.

I will try add some way values to fix it.

Many thanks

mira
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] [Patch V1]Re: Still problems with lakes

2012-10-17 Thread Henning Scholland 
Hi Gerd,
is it possible to use a normal splitter r200 with o5m input-data? What 
about mkgmap?

Henning

Am 17.10.2012 16:36, schrieb GerdP:
> Hi,
>
> here is a first try to fix this issue.
> splitter_problem_list.patch
> 
>
> A sample list of problem polygons:
> problem_polygons.txt
> 
>
> Specify it in the new parameter problem-file, e.g.
> --problem-file=./problem_polygons.txt
>
> It is recommended to use o5m format as input. To convert an existing osm.pbf
> file to o5m use e.g.
> osmconvert --drop-version --out-o5m europe.osm.pbf > europe.o5m
>
> Sorry, the new code is not well commented yet, but I think it works quite
> well now.
> Any feedback is welcome.
>
> Ciao,
> GerdP

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] It's clearly a bug in the splitter

2012-10-17 Thread Steve Ratcliffe 
On 17/10/12 19:44, Felix Hartmann wrote:
> However actually now that I think about it - I can only come up with one
> explication why rev 105 on splitter was done - that is so that tiles
> don't overlap. Because the current hack means all tiles will be expanded.

Yes, that is exactly right, the tiles will overlap and routing will fail 
across boundaries.

> What we really need is to have the splitter identify ALL outside edges.

Yes.

I'm not really sure how splitter works now, but it looks to me like all 
we need to do is alter

SplittableArea splittableArea = nodes.getRoundedArea(resolution);

in Main.calculateAreas() so that it doesn't round at all.

..Steve
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev