Re: [mkgmap-dev] Infinite loop splitter

2014-03-10 Thread Gerd Petermann
Hi Bernd,

the error is splitter is not dependent on the System.
I don't know if osmupdate tries to make sure that ways are 
kept complete if you use it with a bbox or polygon file.

Gerd

> From: weigelt.be...@web.de
> To: mkgmap-dev@lists.mkgmap.org.uk
> Date: Mon, 10 Mar 2014 16:08:19 +0100
> Subject: Re: [mkgmap-dev] Infinite loop splitter
> 
> Am Samstag, 8. März 2014, 23:49:18 schrieb GerdP:
> > So, if you use osmconvert/osmfilter to update your files
> > it is probably better to use a 64 bit Linux installation.
> 
> BTW
> Hmmh, i saw this error on a 64 bit linux system with 16 GiB RAM and i use 
> osmupdate to update an extract from a local planet
> 
> 
> 
> This is the splitter.log from march, 1st 2014
> 
> splitter version 317 compiled 2014-02-27T07:09:08+
> boundary-tags=use-exclude-list
> cache=
> 
> Elapsed time: 10m 0s   Memory: Current 2278MB (1176MB used, 1102MB free) Max 
> 5461MB
> Elapsed time: 12m 0s   Memory: Current 2278MB (1178MB used, 1100MB free) Max 
> 5461MB
> Elapsed time: 14m 0s   Memory: Current 2278MB (1178MB used, 1100MB free) Max 
> 5461MB
> Elapsed time: 16m 0s   Memory: Current 2278MB (1178MB used, 1100MB free) Max 
> 5461MB
> * Full GC *
> 
> Elapsed time: 11h 4m 7s   Memory: Current 2386MB (743MB used, 1643MB free) 
> Max 
> 5461MB
> Elapsed time: 11h 6m 7s   Memory: Current 2386MB (743MB used, 1643MB free) 
> Max 
> 5461MB
> Elapsed time: 11h 8m 7s   Memory: Current 2386MB (743MB used, 1643MB free) 
> Max 
> 5461MB
> Elapsed time: 11h 10m 7s   Memory: Current 2386MB (743MB used, 1643MB free) 
> Max 5461MB
> Elapsed time: 11h 12m 7s   Memory: Current 2386MB (743MB used, 1643MB free) 
> Max 5461MB
> Elapsed time: 11h 14m 7s   Memory: Current 2386MB (743MB used, 1643MB free) 
> Max 5461MB
> Elapsed time: 11h 16m 7s   Memory: Current 2386MB (743MB used, 1643MB free) 
> Max 5461MB
> 
> Bernd
> 
> 
> -- 
> amarok2 now playing:
> artist: Sade
> title: The Sweetest Taboo
> album: The Best of Sade
> 
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
  ___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Infinite loop splitter

2014-03-10 Thread Bernd Weigelt
Am Samstag, 8. März 2014, 23:49:18 schrieb GerdP:
> So, if you use osmconvert/osmfilter to update your files
> it is probably better to use a 64 bit Linux installation.

BTW
Hmmh, i saw this error on a 64 bit linux system with 16 GiB RAM and i use 
osmupdate to update an extract from a local planet



This is the splitter.log from march, 1st 2014

splitter version 317 compiled 2014-02-27T07:09:08+
boundary-tags=use-exclude-list
cache=

Elapsed time: 10m 0s   Memory: Current 2278MB (1176MB used, 1102MB free) Max 
5461MB
Elapsed time: 12m 0s   Memory: Current 2278MB (1178MB used, 1100MB free) Max 
5461MB
Elapsed time: 14m 0s   Memory: Current 2278MB (1178MB used, 1100MB free) Max 
5461MB
Elapsed time: 16m 0s   Memory: Current 2278MB (1178MB used, 1100MB free) Max 
5461MB
* Full GC *

Elapsed time: 11h 4m 7s   Memory: Current 2386MB (743MB used, 1643MB free) Max 
5461MB
Elapsed time: 11h 6m 7s   Memory: Current 2386MB (743MB used, 1643MB free) Max 
5461MB
Elapsed time: 11h 8m 7s   Memory: Current 2386MB (743MB used, 1643MB free) Max 
5461MB
Elapsed time: 11h 10m 7s   Memory: Current 2386MB (743MB used, 1643MB free) 
Max 5461MB
Elapsed time: 11h 12m 7s   Memory: Current 2386MB (743MB used, 1643MB free) 
Max 5461MB
Elapsed time: 11h 14m 7s   Memory: Current 2386MB (743MB used, 1643MB free) 
Max 5461MB
Elapsed time: 11h 16m 7s   Memory: Current 2386MB (743MB used, 1643MB free) 
Max 5461MB

Bernd


-- 
amarok2 now playing:
artist: Sade
title: The Sweetest Taboo
album: The Best of Sade

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


Re: [mkgmap-dev] Infinite loop splitter

2014-03-09 Thread Gerd Petermann
Hi Colin,

thanks for the feedback. It was an obvious bug (once identified),
but did not occur for me as I always used downloads
from geofabrik.
I don't  say you can't use osmconvert, but I would be careful
using the Windows version in combinaton with osmupdate and
files > 4GB.

Gerd

Date: Sun, 9 Mar 2014 16:17:58 +0100
From: colin.sm...@xs4all.nl
To: mkgmap-dev@lists.mkgmap.org.uk
Subject: Re: [mkgmap-dev] Infinite loop splitter



R320 certainly seems to have fixed it for me. If there is still a problem with 
osmconvert etc then the new splitter appears to cope with it somehow. I will 
keep an eye on the maps coming out of mkgmap in case there any anomalies, but 
so far it's looking good.

Thanks for the fix!

Colin

 

 
On 2014-03-09 08:49, GerdP wrote:


Hi all,

I've contacted Markus Weber, the author of osmconvert and osmfilter.
It seems that the problem with complex-ways occurs only
in the Windows version, maybe because something goes wrong with
files > 4GB. Markus doesn't have access to a Win64 system.
I use a mingw-64 installation to compile my own
binary for 64 bit, maybe an update will help, but I doubt it,
and unfortunately I forgot how the installation works,
I just remember it wasn't simple.

So, if you use osmconvert/osmfilter to update your files
it is probably better to use a 64 bit Linux installation.

Gerd



GerdP wrote
Hi Minko, fine:-) I think the only good reason to use splitter with an input 
file like europe.o5m or planet.o5m in combination with a poly file that 
extracts a small part is that it is able to keep ferry lines complete. 
osmconvert --complete-ways --complex-ways -B=bnl_01.poly europe.o5m 
-o=benelux.o5m fails for me as well. I'll ask the author about it. Gerd 
ligfietser wrote
Gerd wrote:
the order of input files would be java ... splitter.jar [options] europe.o5m 
adres.o5m vlaanderen-adres.o5m
Yes I did that too. Anyway, with r320 it seems to work, mkgmap is running now 
thanks! ___ mkgmap-dev mailing list


mkgmap-dev@.org


http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev



--
View this message in context: 
http://gis.19327.n5.nabble.com/Infinite-loop-splitter-tp5799076p5799142.html
Sent from the Mkgmap Development mailing list archive at Nabble.com.
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev




___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
  ___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Infinite loop splitter

2014-03-09 Thread Colin Smale
 

R320 certainly seems to have fixed it for me. If there is still a
problem with osmconvert etc then the new splitter appears to cope with
it somehow. I will keep an eye on the maps coming out of mkgmap in case
there any anomalies, but so far it's looking good. 

Thanks for the fix! 

Colin 

On 2014-03-09 08:49, GerdP wrote: 

> Hi all,
> 
> I've contacted Markus Weber, the author of osmconvert and osmfilter.
> It seems that the problem with complex-ways occurs only
> in the Windows version, maybe because something goes wrong with
> files > 4GB. Markus doesn't have access to a Win64 system.
> I use a mingw-64 installation to compile my own
> binary for 64 bit, maybe an update will help, but I doubt it,
> and unfortunately I forgot how the installation works,
> I just remember it wasn't simple.
> 
> So, if you use osmconvert/osmfilter to update your files
> it is probably better to use a 64 bit Linux installation.
> 
> Gerd
> 
> GerdP wrote
> Hi Minko, fine:-) I think the only good reason to use splitter with an input 
> file like europe.o5m or planet.o5m in combination with a poly file that 
> extracts a small part is that it is able to keep ferry lines complete. 
> osmconvert --complete-ways --complex-ways -B=bnl_01.poly europe.o5m 
> -o=benelux.o5m fails for me as well. I'll ask the author about it. Gerd 
> ligfietser wrote Gerd wrote: the order of input files would be java ... 
> splitter.jar [options] europe.o5m adres.o5m vlaanderen-adres.o5m Yes I did 
> that too. Anyway, with r320 it seems to work, mkgmap is running now thanks! 
> ___ mkgmap-dev mailing list

>> mkgmap-dev@.org

>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev [1]

--
View this message in context:
http://gis.19327.n5.nabble.com/Infinite-loop-splitter-tp5799076p5799142.html
[2]
Sent from the Mkgmap Development mailing list archive at Nabble.com.
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev [1]

 

Links:
--
[1] http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
[2]
http://gis.19327.n5.nabble.com/Infinite-loop-splitter-tp5799076p5799142.html___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Infinite loop splitter

2014-03-08 Thread GerdP
Hi all,

I've contacted Markus Weber, the author of osmconvert and osmfilter.
It seems that the problem with complex-ways occurs only
in the Windows version, maybe because something goes wrong with
files > 4GB. Markus doesn't have access to a Win64 system.
I use a mingw-64 installation to compile my own
binary for 64 bit, maybe an update will help, but I doubt it,
and unfortunately I forgot how the installation works,
I just remember it wasn't simple.

So, if you use osmconvert/osmfilter to update your files
it is probably better to use a 64 bit Linux installation.

Gerd



GerdP wrote
> Hi Minko,
> 
> fine:-)
> 
> I think the only good reason to use splitter with an input file like
> europe.o5m or planet.o5m in combination with a poly file
> that extracts a small part is that it is able to keep ferry lines
> complete.
> 
> osmconvert --complete-ways --complex-ways -B=bnl_01.poly europe.o5m
> -o=benelux.o5m
> fails for me as well. I'll ask the author about it.
> 
> Gerd
> ligfietser wrote
>> Gerd wrote:
>>> the order of input files would be
>>> java ... splitter.jar [options] europe.o5m adres.o5m
>>> vlaanderen-adres.o5m
>> 
>> Yes I did that too.
>> Anyway, with r320 it seems to work, mkgmap is running now thanks!
>> 
>> ___
>> mkgmap-dev mailing list

>> mkgmap-dev@.org

>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev





--
View this message in context: 
http://gis.19327.n5.nabble.com/Infinite-loop-splitter-tp5799076p5799142.html
Sent from the Mkgmap Development mailing list archive at Nabble.com.
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


Re: [mkgmap-dev] Infinite loop splitter

2014-03-08 Thread GerdP
Hi Minko,

fine:-)

I think the only good reason to use splitter with an input file like
europe.o5m or planet.o5m in combination with a poly file
that extracts a small part is that it is able to keep ferry lines complete.

osmconvert --complete-ways --complex-ways -B=bnl_01.poly europe.o5m
-o=benelux.o5m
fails for me as well. I'll ask the author about it.

Gerd


ligfietser wrote
> Gerd wrote:
>> the order of input files would be
>> java ... splitter.jar [options] europe.o5m adres.o5m
>> vlaanderen-adres.o5m
> 
> Yes I did that too.
> Anyway, with r320 it seems to work, mkgmap is running now thanks!
> 
> ___
> mkgmap-dev mailing list

> mkgmap-dev@.org

> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev





--
View this message in context: 
http://gis.19327.n5.nabble.com/Infinite-loop-splitter-tp5799076p5799112.html
Sent from the Mkgmap Development mailing list archive at Nabble.com.
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


Re: [mkgmap-dev] Infinite loop splitter

2014-03-08 Thread Minko
Gerd wrote:
> the order of input files would be
> java ... splitter.jar [options] europe.o5m adres.o5m
> vlaanderen-adres.o5m

Yes I did that too.
Anyway, with r320 it seems to work, mkgmap is running now thanks!

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


Re: [mkgmap-dev] Infinite loop splitter

2014-03-08 Thread GerdP
Hi Minko,

the order of input files would be 
java ... splitter.jar [options] europe.o5m  adres.o5m vlaanderen-adres.o5m

Your approach with osmconvert is probably faster because you have to use 
osmconvert anyway to convert to o5m.

Gerd


ligfietser wrote
> Hi Gerd,
> Thanks for the fix, I will test it out.
> I tried Hennings approach but splitting seemed to last much longer because
> it analyses the whole EU extract and I only need the Benelux plus a small
> area of France and Germany.
> 
> Splitting wasn't completed and threw out this error:
> 
> warning: --keep-complete is only used for the first input file.
> New way id 195314 is not higher than last id 264955883
> Maybe the IDs are not sorted. This is not supported with
> keep-complete=true or --problem-list
> 
> I'll try now  r320.
> 
> 
>> I've committed r320 to fix the loop problem.
>> 
>> You can specify multiplie input files, but you have to make
>> sure that each file contains ids which are higher than all
>> previous ones. I think this is the case with your address files.
>> 
>> The reason for the loop was a way in a mp-relation
>> for which no nodes were found.
>> I guess this is caused by using osmconvert without the parameters
>> --complete-ways --complex-ways
>> 
>> I am not sure what's faster: Using osmconvert with polygon first or
>> only splitter with multiple input files. When all files are in o5m
>> format,
>> single run with splitter should be quite fast.
>> 
>> Gerd
> ___
> mkgmap-dev mailing list

> mkgmap-dev@.org

> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev





--
View this message in context: 
http://gis.19327.n5.nabble.com/Infinite-loop-splitter-tp5799076p5799107.html
Sent from the Mkgmap Development mailing list archive at Nabble.com.
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


Re: [mkgmap-dev] Infinite loop splitter

2014-03-08 Thread Minko

Gerd wrote:
> > The reason for the loop was a way in a mp-relation
> > for which no nodes were found.
> > I guess this is caused by using osmconvert without the parameters
> > --complete-ways --complex-ways


If I use --complete-ways or --complex-ways with osmconvert, I didnt get any 
output at all so I always skip this.
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


Re: [mkgmap-dev] Infinite loop splitter

2014-03-08 Thread Minko
Hi Gerd,
Thanks for the fix, I will test it out.
I tried Hennings approach but splitting seemed to last much longer because it 
analyses the whole EU extract and I only need the Benelux plus a small area of 
France and Germany.

Splitting wasn't completed and threw out this error:

warning: --keep-complete is only used for the first input file.
New way id 195314 is not higher than last id 264955883
Maybe the IDs are not sorted. This is not supported with keep-complete=true or 
--problem-list

I'll try now  r320.


> I've committed r320 to fix the loop problem.
> 
> You can specify multiplie input files, but you have to make
> sure that each file contains ids which are higher than all
> previous ones. I think this is the case with your address files.
> 
> The reason for the loop was a way in a mp-relation
> for which no nodes were found.
> I guess this is caused by using osmconvert without the parameters
> --complete-ways --complex-ways
> 
> I am not sure what's faster: Using osmconvert with polygon first or
> only splitter with multiple input files. When all files are in o5m
> format,
> single run with splitter should be quite fast.
> 
> Gerd
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


Re: [mkgmap-dev] Infinite loop splitter

2014-03-08 Thread GerdP
Hi Minko,

I've committed r320 to fix the loop problem.
 
You can specify multiplie input files, but you have to make
sure that each file contains ids which are higher than all
previous ones. I think this is the case with your address files.

The reason for the loop was a way in a mp-relation
for which no nodes were found.
I guess this is caused by using osmconvert without the parameters
--complete-ways --complex-ways

I am not sure what's faster: Using osmconvert with polygon first or 
only splitter with multiple input files. When all files are in o5m format,
single run with splitter should be quite fast.

Gerd




--
View this message in context: 
http://gis.19327.n5.nabble.com/Infinite-loop-splitter-tp5799076p5799103.html
Sent from the Mkgmap Development mailing list archive at Nabble.com.
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


Re: [mkgmap-dev] Infinite loop splitter

2014-03-08 Thread GerdP
Hi Minko,

thanks, I can reproduce the problem with this file.
Looking for a fix now.

Gerd


ligfietser wrote
> Gerd,
> If I skip merging the Vlaanderen address data then splitting goes well.
> I have uploaded my 'corrupt' Benelux data which goes wrong (incl the
> Vlaanderen addresses) here:
> http://mijndev.openstreetmap.nl/~ligfietser/test/benelux.o5m
> 
> 
> ___
> mkgmap-dev mailing list

> mkgmap-dev@.org

> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev





--
View this message in context: 
http://gis.19327.n5.nabble.com/Infinite-loop-splitter-tp5799076p5799099.html
Sent from the Mkgmap Development mailing list archive at Nabble.com.
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


Re: [mkgmap-dev] Infinite loop splitter

2014-03-08 Thread Minko
Hi Henning,
I didnt know that you could feed multiple files to the splitter?
How do you do this?

> Hi Minko,
> is there a reason for merging the files before splitting? Did you try
> to
> just pass all the files to splitter? I'm doing so with osm and
> srtm-data. Don't know, if this helps in this case, but in general I
> think it should be much faster.
> 
> Henning
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


Re: [mkgmap-dev] Infinite loop splitter

2014-03-08 Thread Henning Scholland

Hi Minko,
is there a reason for merging the files before splitting? Did you try to 
just pass all the files to splitter? I'm doing so with osm and 
srtm-data. Don't know, if this helps in this case, but in general I 
think it should be much faster.


Henning

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


Re: [mkgmap-dev] Infinite loop splitter

2014-03-08 Thread Minko
Gerd,
If I skip merging the Vlaanderen address data then splitting goes well.
I have uploaded my 'corrupt' Benelux data which goes wrong (incl the Vlaanderen 
addresses) here:
http://mijndev.openstreetmap.nl/~ligfietser/test/benelux.o5m


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


Re: [mkgmap-dev] Infinite loop splitter

2014-03-08 Thread GerdP
Colin,

thanks for the hint.
No matter what other tools do: infinite loop in splitter is the worst case
as a reaction on errors ;-)

Gerd


Colin Smale wrote
> I see this very often as well, when I use osmupdate on an earlier
> download. The only solution I have found is to download a new pbf, then it
> works again. I suspect therefore that osmupdate or one of the underlying
> tools is corrupting the data.
> 
> Colin
> 
> On 8 March 2014 10:44:16 CET, Minko <

> ligfietser@

> > wrote:
>>Hi,
>>I was trying to update my Benelux map but the splitter refused to
>>finish the process.
>>Normally, splitter is finished in 2 minutes, now it gets in an infinite
>>loop with Full GC.
>>What can be the cause for this, a corrupt download of the europe
>>extract?
>>
>>
>>
>>
>>
>>___
>>mkgmap-dev mailing list
>>

> mkgmap-dev@.org

>>http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> 
> ___
> mkgmap-dev mailing list

> mkgmap-dev@.org

> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev





--
View this message in context: 
http://gis.19327.n5.nabble.com/Infinite-loop-splitter-tp5799076p5799086.html
Sent from the Mkgmap Development mailing list archive at Nabble.com.
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


Re: [mkgmap-dev] Infinite loop splitter

2014-03-08 Thread Colin Smale
I see this very often as well, when I use osmupdate on an earlier download. The 
only solution I have found is to download a new pbf, then it works again. I 
suspect therefore that osmupdate or one of the underlying tools is corrupting 
the data.

Colin

On 8 March 2014 10:44:16 CET, Minko  wrote:
>Hi,
>I was trying to update my Benelux map but the splitter refused to
>finish the process.
>Normally, splitter is finished in 2 minutes, now it gets in an infinite
>loop with Full GC.
>What can be the cause for this, a corrupt download of the europe
>extract?
>
>
>
>
>
>___
>mkgmap-dev mailing list
>mkgmap-dev@lists.mkgmap.org.uk
>http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Infinite loop splitter

2014-03-08 Thread GerdP
Hi Minko,

ligfietser wrote
> Now if I skip the first step and split it without merging the address
> nodes it goes well without problems. Could it be that the splitter creates
> fake ID's that overlap my address data nodes?
> Or is there an overlap in osm id's with the fake address nodes?

I have no idea. Download of Europe will take a while...

Gerd



--
View this message in context: 
http://gis.19327.n5.nabble.com/Infinite-loop-splitter-tp5799076p5799084.html
Sent from the Mkgmap Development mailing list archive at Nabble.com.
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


Re: [mkgmap-dev] Infinite loop splitter

2014-03-08 Thread Minko
BTW Here are the links to the files in case you need them:
https://www.dropbox.com/sh/d3ssenipw2vx6qs/ucrj2cNiFM/adres.o5m
https://www.dropbox.com/sh/d3ssenipw2vx6qs/bgWpOCpfwK/vlaanderen-adres.o5m
https://www.dropbox.com/sh/d3ssenipw2vx6qs/zqzksFWZcp/bnl_01.poly
https://www.dropbox.com/sh/d3ssenipw2vx6qs/2N8CdpLXQc/benelux.poly
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


Re: [mkgmap-dev] Infinite loop splitter

2014-03-08 Thread Minko
Hi Gerd,
I always mix the osm data with address nodes for Vlaanderen and the Netherlands 
with osmconvert.
This part of my batch file looks like this:

:osmconvert
%DIR%\Tools\osmconvert --drop-version %DIR%\%date%\Splitter\%extract% 
%DIR%\resources\adres.o5m %DIR%\resources\vlaanderen-adres.o5m 
-B=%DIR%\resources\bnl_01.poly -o=%DIR%\%date%\Splitter\benelux.o5m

%extract% is europe.osm.pbf from today

NL adres.o5m:
lon min: 3.3590433
lon max: 7.2259690
lat min: 50.7510383
lat max: 53.4972667
nodes: 8551053
ways: 0
relations: 0
node id min: 1000
node id max: 18551052
keyval pairs max: 2
keyval pairs max object: node 1000 

Vlaanderen vlaanderen-adres.o5m

lon min: 2.3066895
lon max: 5.8980601
lat min: 49.2933351
lat max: 51.5047683
nodes: 2449726
ways: 0
relations: 0
node id min: 2001
node id max: 22449726
keyval pairs max: 4
keyval pairs max object: node 2001 


After this I split the resulting file as follows

:Splitter
java -Xmx8000m -XX:+UseParallelGC -XX:+HeapDumpOnOutOfMemoryError 
-XX:+PrintGCTimeStamps -XX:+PrintGCDetails -jar  %SPLITTER% 
--output-dir=%date%\Splitter --max-threads=8 
--problem-file=%DIR%\resources\problem_areas.txt --precomp-sea=%seafile% 
--keep-complete --overlap=0 --mapid=10010001 
--polygon-file=%DIR%\resources\benelux.poly --max-nodes=175 
--write-kml=%areas%.kml --output=o5m 
--geonames-file=%DIR%\resources\cities5000.txt --description=OFM_BNL 
%DIR%\%date%\Splitter\benelux.o5m > splitter2.log 2>err2.log

Normally it all goes well until this morning.
I've tried the latest splitter version too but it didnt help.
I also tried to make the first polygon bnl_01.poly much bigger ( it is a little 
bigger than the second benelux.poly in the splitter step 2). This also didnt 
help.

Now if I skip the first step and split it without merging the address nodes it 
goes well without problems. Could it be that the splitter creates fake ID's 
that overlap my address data nodes?
Or is there an overlap in osm id's with the fake address nodes?

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


Re: [mkgmap-dev] Infinite loop splitter

2014-03-08 Thread Gerd Petermann
Hi Minko,

please keep the files that produce this problem.
I'd like to have them to be able to reproduce it.

Gerd

> Date: Sat, 8 Mar 2014 11:09:03 +0100
> From: ligfiet...@online.nl
> To: mkgmap-dev@lists.mkgmap.org.uk
> Subject: Re: [mkgmap-dev] Infinite loop splitter
> 
> What have you done to solve this Bernd, should I just wait for another 
> extract tomorrow?
> 
> > I have seen this error last week, too.
> > 
> > I had to recreate my dach extract.
> > 
> > Bernd
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
  ___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Infinite loop splitter

2014-03-08 Thread Minko
What have you done to solve this Bernd, should I just wait for another extract 
tomorrow?

> I have seen this error last week, too.
> 
> I had to recreate my dach extract.
> 
> Bernd
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


Re: [mkgmap-dev] Infinite loop splitter

2014-03-08 Thread Bernd Weigelt
Am Samstag, 8. März 2014, 10:44:16 schrieb Minko:
> I was trying to update my Benelux map but the splitter refused to finish the
> process. Normally, splitter is finished in 2 minutes, now it gets in an
> infinite loop with Full GC. What can be the cause for this, a corrupt
> download of the europe extract?


I have seen this error last week, too.

I had to recreate my dach extract.

Bernd


-- 
amarok2 now playing:




___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev