from:"Lambertus"

Multithreading the tile rendering for a single map is indeed a bit 
difficult and I gave it up because you need to keep track which image 
id's are already in use. Since I provide multiple maps the work-around 
is running a few scripts parallel, which is also a crude form of 
multithreading.


The script language is PHP and it doesn't run on Windows without some 
changes ('/' vs '\' in paths, 'rm -rf', that sort of stuff). Never tried it.


To get a better optimum in file size, using the process I described 
earlier, you could start off with a huge --max-nodes setting and then 
'search' for the highest --max-nodes that works for each specific area.


On 30/04/2014 11:49, Felix Hartmann wrote:

I would love if there was a possibility that you pass the used max-nodes
value to mkgmap.
When mkgmap is compiling the maps, then after the .img is created it
should check
a) did it crash due to too many max-nodes
b) for me not important - but for others with very old GPS, etrex 10,
--- is tile bigger than X (usually 8) MB.

if a) or b) true, then pass the file back to splitter and split with 60%
of maxnodes - and compile the resulting .img files again. Should it fail
again, use 40%, again 25%... Sometimes there are awful tiles, that need
supersmall max-nodes till they compile, however lately (last 1-2 years)
I never encountered them anymore. I think that happened rather due to a
but in splitter/mgkmap that is fixed now.

okay, you could also do this with a script, but it gets rather
complicated to multithread it (you need to wait till mgkmap finished
compiling all .img files - and run mkgmap first without address index to
save time) and do some clever routines on making sure that the map id
(e.g. 6340.img)  stay correct. Even more complicated to have
consequent map id...

For europe with a fixed max-node I get tiles from 1.9MB up to 18MB.
That's factor 9 - so it's huge...
If I could narrows that down easily to 8-18MB - without getting tiles
crashing due to too high max-nodes values, that would be sweet.



As for the scripts - would they run on windows too? - What programming
language are they in?


On 29.04.2014 21:39, o...@na1400.info wrote:

Oh, and ofcourse anyone interested can get my scripts, send an email.
They'll be on Github someday anyway.


On 2014-04-29 20:37, Gerd Petermann wrote:

Hi Lambertus,

okay, if I got that right you finally get *.img files with a size
near (but below) 8MB, so maybe Henning can use that script, too.

If you do that for e.g. Germany, how small is tpically the smallest
*.img file ?
Is it probably near 4 MB?

Gerd


To: mkgmap-dev@lists.mkgmap.org.uk
Date: Tue, 29 Apr 2014 20:30:27 +0200
From: o...@na1400.info
Subject: Re: [mkgmap-dev] mkgmap ToDo list

These are the direct results from Splitter. The format is o5m, both
input as output. Splitter version is: r321.

For this test I split the original source with --max-nodes=800.

Then

I render the initial tiles, when the result is larger than 8MB it's
subsplit again with --max-nodes=(800-(attempt*10)). The

initial

source files are ~70MB (o5m) and after several subsplits the two

*.img

are  8MB. During this process --max-nodes has been reduced to e.g.
750 and the source file is split up in two o5m files of about

37MB.


I can upload an example source file and it's two subsplit siblings

if

you want.


On 2014-04-29 19:38, GerdP wrote:
 Hi Lambertus,

 that's interesting. Are these the img file sizes or the osm file

sizes?


 Gerd


 Lambertus wrote
 Unfortunately I cannot confirm that. Below is a bit of logging

from my

 script:
 Original: 9720 (70551453), New: 0 (35684445), New: 1

(36852845)

 Original: 9701 (74621042), New: 0 (37522992), New: 1

(37222739)

 Original: 9702 (73391358), New: 0 (37679505), New: 1

(38098627)

 Original: 9703 (77862567), New: 0 (39075311), New: 1

(39261197)


 The original files above contain contour data, the filesize is

between

 brackets. As you can see both resulting file are approximately

the

 same
 size.

 On 2014-04-29 15:39, Gerd Petermann wrote:
 Hi Lambertus,

 and I guess that even after this optimization you will
 see a factor 3 or higher between the largest tile and the

smallest.

 Can you confirm that?

 Gerd

 Date: Tue, 29 Apr 2014 15:32:38 +0200
 From:

 osm@

 To:

 mkgmap-dev@.org

 Subject: Re: [mkgmap-dev] mkgmap ToDo list

 Num-tiles=x would indeed be better for this specific need.

 It is my experience that it regularly takes multiple calls to
 Splitter
 to get 2+ sub-tiles when you reduce the max-nodes by 100k for

each

 sub-split attempt. This is what I currently do to get an

optimum in

 tile-size vs total number of tiles.



 On 29/04/2014 15:09, Gerd Petermann wrote:
  Hi Lambertus,
 
  that sounds like a possible change in splitter:
  Instead of specifying max-nodes you may specify --num-tiles=x
  and splitter will try to find a split that produces excactly

x

 tiles
  which are not too narrow and have a node number

Re: [mkgmap-dev] mkgmap ToDo list

You apparently do manually over a long timespan what my script 
essentially does on each map update run. Attached is my script 
(build.php), it will not run it but it might give you an example of how 
I implemented this.


The interesting bits are the two functions render() and subsplit(). They 
call each other recursively until an image has been rendered 
successfully within the specified size limit or that subsplitting any 
further is not possible.


Please excuse the sloppy code :p

On 30/04/2014 13:47, Felix Hartmann wrote:


On 30.04.2014 13:36, Lambertus wrote:



To get a better optimum in file size, using the process I described
earlier, you could start off with a huge --max-nodes setting and then
'search' for the highest --max-nodes that works for each specific area.


Well that's basically what I do. I have for all areas a rather high
max-nodes, and If I see it crashes on a tile, the next week I decrease
it. However in reality I should also increase it sometimes - but that
would be even more work.
Some nice interaction between mkgmap calling splitter in order to keep
track of crashing tiles due to too high max-nodes would be great. I
cannot run multiple mkgmap.jar threads, as my server only has 12GB of
RAM - and that works out to exactly running a single mkgmap.jar instance
using max-jobs=8 --- plus multithreading the nsis.exe in the background
as nsis packing is super slow... (damn, when will they ever update nsis
to lzma2 and a rather recent .7z version - also getting rid of 2GB limit)..
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


 $threadinfo) {
$thread = $threadinfo["thread"];
if( ! $thread->isAlive() ) {
	echo "Thread $index is done\n";
	unset( $threads[$index] );
	$threads = array_values($threads);
}
			}
			// Check if there is room to start a new thread
			if (count($threads) < $maxThreads && !empty($files)) {
echo "**\n**Starting thread with file ".$files[0]." and maxid $maxid**\n**\n";
// Start the render thread
$thread = new Thread("render");
$thread->start($render_dir, $files[0], $maxid, $initial_nodes, 0);
$threads[] = array("thread" => $thread, "dir" => $thread_dir);
$maxid+=100;	// Assume maximum of 100 splitted tiles per initial tile

// Remove this tile from the todo list
unset($files[0]);
$files = array_values($files);
			}
			// let the CPU do its work
			sleep( 1 );
			die();
		}
	} else {
		// Use non-threaded recursive method for rendering
		$maxid = getMaxSourceId($type);
		$files = getSourceIds("",$type);
		foreach($files as $file) {
			echo spaces(0)."Render top level file $file\n";
			$maxid = render($render_dir, $file, $maxid, $initial_nodes, 0, false);
		}
	}
	
	// Delete any file that has 0 bytes length (has to be done before the updateKml function)
	echo spaces(0)."Cleanup after rendering\n";
	$files = getSourceIds("","img");
	foreach($files as $file) {
		if (filesize("$file.img") == 0) {
			echo spaces(0)."Deleting file $file.img because it has 0 bytes length\n";
			unlink("$file.img");
		}
	}

	// Update the initial KML split result file that lists all the tiles with the actual tiles produced by the recursive process
	echo spaces(0)."Update the KML file\n";
	updateKml("initial.kml", "world.kml");

	// Create a list of countries and their associated tiles
	echo spaces(0)."Match the countries with their associated tiles\n";
	// Matchit is a custom made script to match country bboxes to tile bboxes.
	$matchit = new Matchit($date, "/home/lambertus/garmin/utils/matchit/countries", $render_dir, "world.kml");
	$matchit->Run();

	// Move the results to the final directory
	// Create the web directory where the tiles are stored
	echo spaces(0)."Create the web dir: $web_dir/$date\n";
	if (!is_dir("$web_dir")) {
		mkdir("$web_dir");
	}
	if (!is_dir("$web_dir/$date")) {
		mkdir("$web_dir/$date");
	}

	// Move all files to the new directory
	echo spaces(0)."Move the images to the webdir\n";
	exec("mv *.img $web_dir/$date");
	exec("mv world.kml $web_dir/$date");
	exec("cp countries.js $web_dir/$date");
	exec("cp countries.xml $web_dir/$date");

	echo "Distribute the version to the other server(s)\n";
	exec("$script_dir/distribute.sh $date");

	echo spaces(0)."Finalize by writing a new version\n";
	file_put_contents("$web_dir/version", $date);

	echo spaces(0)."Done!\n";
}

// Recursive functions that splits and renders until success (or no solution)
function render($thread_dir, $sourceid, $maxid, $nodes, $level, $n

Re: [mkgmap-dev] Bounds and Sea Data on mkgmap.org.uk -- still old creation process.

What is needed to get the bounds and sea data in high-precision format? 
I use a reasonably new Mkgmap for this process: r3200. Should I use a 
different version?


On 04/30/2014 09:51 PM, Felix Hartmann wrote:
Just a question - is it planned to change the bounds and sea data that 
is available via mkgmap.org.uk to be update to the high precision format?

or maybe put a link to Thorstens site if not?
http://osm.thkukuk.de/data/




___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Bounds and Sea Data on mkgmap.org.uk -- still old creation process.

Well Mkgmap.org.uk points to one of my servers, that's why I responded. 
Looking through the Mkgmap options doesn't offer obvious hints, so maybe 
Thorsten can elaborate on this?



On 04/30/2014 10:00 PM, Felix Hartmann wrote:
Well I don't know actually - I would have assumed r3081 or later is 
fine. However the data by Thorsten is significantly bigger size than 
the one on mkgmap.org.uk - hence I assumed here the used version is 
still older?

On 30.04.2014 21:58, Lambertus wrote:
What is needed to get the bounds and sea data in high-precision 
format? I use a reasonably new Mkgmap for this process: r3200. Should 
I use a different version?


On 04/30/2014 09:51 PM, Felix Hartmann wrote:
Just a question - is it planned to change the bounds and sea data 
that is available via mkgmap.org.uk to be update to the high 
precision format?

or maybe put a link to Thorstens site if not?
http://osm.thkukuk.de/data/




___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev




___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] mkgmap ToDo list

2014-04-29 Thread Lambertus

While this possibly can be solved in Splitter or Mkgmap, it could also 
be solved by your build-script when you add a maximum tile size check 
and re-split (with a lower number of max-nodes) until you get two or 
more sub-tiles. Granted, this adds complexity to the script but it works 
well for me.


On 25/04/2014 21:54, Henning Scholland wrote:

Hi Gerd,

I would like to have img-tiles which have globally nearly the same
filesize, so that they use the space of devices like eTrex 10.

With my actual map I use globally the same value for max-nodes. But the
size of the img-tiles differ more then factor 2. Eg. a tile in Germany
is between 2 and 5 mb where a tile in China is about 10 mb. If I remove
details, this difference will increase, because in Germany more objects
will be removed from the img-tile then in China.

Henning

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] mkgmap ToDo list

2014-04-29 Thread Lambertus


Num-tiles=x would indeed be better for this specific need.

It is my experience that it regularly takes multiple calls to Splitter 
to get 2+ sub-tiles when you reduce the max-nodes by 100k for each 
sub-split attempt. This is what I currently do to get an optimum in 
tile-size vs total number of tiles.




On 29/04/2014 15:09, Gerd Petermann wrote:

Hi Lambertus,

that sounds like a possible change in splitter:
Instead of specifying max-nodes you may specify --num-tiles=x
and splitter will try to find a split that produces excactly x tiles
which are not too narrow and have a node number which is not
too far from the average (but still aligned to a multiple of map units
as now).
So, for your script that means you don't have to find the max-nodes
value.

I'll think about this again...

Gerd

  Date: Tue, 29 Apr 2014 14:59:36 +0200
  From: o...@na1400.info
  To: mkgmap-dev@lists.mkgmap.org.uk
  Subject: Re: [mkgmap-dev] mkgmap ToDo list
 
  While this possibly can be solved in Splitter or Mkgmap, it could also
  be solved by your build-script when you add a maximum tile size check
  and re-split (with a lower number of max-nodes) until you get two or
  more sub-tiles. Granted, this adds complexity to the script but it works
  well for me.
 
  On 25/04/2014 21:54, Henning Scholland wrote:
   Hi Gerd,
  
   I would like to have img-tiles which have globally nearly the same
   filesize, so that they use the space of devices like eTrex 10.
  
   With my actual map I use globally the same value for max-nodes. But the
   size of the img-tiles differ more then factor 2. Eg. a tile in Germany
   is between 2 and 5 mb where a tile in China is about 10 mb. If I remove
   details, this difference will increase, because in Germany more objects
   will be removed from the img-tile then in China.
  
   Henning
  
   ___
   mkgmap-dev mailing list
   mkgmap-dev@lists.mkgmap.org.uk
   http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
 
  ___
  mkgmap-dev mailing list
  mkgmap-dev@lists.mkgmap.org.uk
  http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev



___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] The --route option

2014-04-22 Thread Lambertus

The basic version could be usable for the old Etrex Legend with 8 Mb or 
Vista with 24 Mb storage space?


On 22/04/2014 11:23, Andrzej Popowski wrote:

Hi Gerd,

Garmin img consist of subfiles. There are some workable set of subfiles:

basic map: TRE+RGN+LBL
advanced: TRE+RGN+LBL+NET
routable: TRE+RGN+LBL+NET+NOD

IMO mkgmap should allow for creating any of these 3 types of maps.

NOD subfile can be removed from img after creating, but I would prefer
to have option for advanced map too. There is even obsolete option
net, which suggest this kind of map.

As far as I know, you can't remove NET subfile from img. It won't be
possible to get basic map without support for it in mkgmap.



___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] bounds and sea on pleiades

2014-02-11 Thread Lambertus


Sorry for the late response.

Looking at the logging it appears that the process crashed. I've added 
some checks to (hopefully) prevent uploading such files in the future. 
The current map update has to finish before I can start another attempt. 
In the meantime the latest is replaced by a previous version.


On 09-02-14 16:09, Minko wrote:

Lambertus,
Is the latest bounds file correct?
It is much smaller (72 Mb) than the previous ones (412 Mb)

  

On 04-02-14 18:55, Patrik Brunner wrote:


ok, thanks sorry to be a pain... ;-)


On 04.02.2014 18:52, Lambertus wrote:



Yes something went wrong ... I tried to optimize my build chain by
parallelizing things, but I managed to schedule two processes that
want all available RAM at the same time. That didn't go well. :p

So I killed the sea generator and will run that one later.

On 04-02-14 18:42, Patrik Brunner wrote:


Lambertus,

I've seen that the 'bounds' is already handled with the new concept:


http://osm2.pleiades.uni-wuppertal.de/bounds/latest/bounds.zip
But the sea boundaries are not yet done the same way even though there
is a new version of that file in the date specific directory:


http://osm2.pleiades.uni-wuppertal.de/sea/latest/sea_20140127.zip
Will this be done later or did something go 'wrong' with the new build
process of the sea boundaries ?
... don't want to be stressing you, it's just a question.

Thanks
Patrik


On 31.01.2014 19:18, Lambertus wrote:


This is not hard to achieve and I'll add an easier link with the next
update.


On 31-01-14 15:37, Patrik Brunner wrote:


Lambertus,

Actually you have a directory ./latest in both your sea and bounds
directory in which we can find the latest version of the sea and and
the bounds file.

Unfortunately these files still have the date tag in the name which
makes an automatic 'fix' download of the latest boundaries quite
hard. there's no need to change these files, but wouldn't it be
possible to have a file just called 'bounds.zip' and 'sea.zip' (and
respective for bz2 files) being a link to the actually latest file ?

So one could always download the latest file from the paths
./bounds/latest/bounds.zip and ./sea/latest/sea.zip

Not sure how complex it is to achieve this during your automated
generation/preparation/publishing, but guessing from my scripting
experience it shouldn't be that hard.

Thanks for having a look at this.
Cheers
Patrik
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev



___
mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


___
mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


___
mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] bounds and sea on pleiades

2014-02-04 Thread Lambertus

Yes something went wrong ... I tried to optimize my build chain by 
parallelizing things, but I managed to schedule two processes that want 
all available RAM at the same time. That didn't go well. :p


So I killed the sea generator and will run that one later.

On 04-02-14 18:42, Patrik Brunner wrote:

Lambertus,

I've seen that the 'bounds' is already handled with the new concept:

http://osm2.pleiades.uni-wuppertal.de/bounds/latest/bounds.zip

But the sea boundaries are not yet done the same way even though there 
is a new version of that file in the date specific directory:


http://osm2.pleiades.uni-wuppertal.de/sea/latest/sea_20140127.zip

Will this be done later or did something go 'wrong' with the new build 
process of the sea boundaries ?

... don't want to be stressing you, it's just a question.

Thanks
Patrik

On 31.01.2014 19:18, Lambertus wrote:
This is not hard to achieve and I'll add an easier link with the next 
update.



On 31-01-14 15:37, Patrik Brunner wrote:

Lambertus,

Actually you have a directory ./latest in both your sea and bounds 
directory in which we can find the latest version of the sea and and 
the bounds file.


Unfortunately these files still have the date tag in the name which 
makes an automatic 'fix' download of the latest boundaries quite 
hard. there's no need to change these files, but wouldn't it be 
possible to have a file just called 'bounds.zip' and 'sea.zip' (and 
respective for bz2 files) being a link to the actually latest file ?


So one could always download the latest file from the paths 
./bounds/latest/bounds.zip and ./sea/latest/sea.zip


Not sure how complex it is to achieve this during your automated 
generation/preparation/publishing, but guessing from my scripting 
experience it shouldn't be that hard.


Thanks for having a look at this.
Cheers
Patrik
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev




___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] bounds and sea on pleiades

2014-02-04 Thread Lambertus


No worries! :)

On 04-02-14 18:55, Patrik Brunner wrote:

ok, thanks sorry to be a pain... ;-)

On 04.02.2014 18:52, Lambertus wrote:
Yes something went wrong ... I tried to optimize my build chain by 
parallelizing things, but I managed to schedule two processes that 
want all available RAM at the same time. That didn't go well. :p


So I killed the sea generator and will run that one later.

On 04-02-14 18:42, Patrik Brunner wrote:

Lambertus,

I've seen that the 'bounds' is already handled with the new concept:

http://osm2.pleiades.uni-wuppertal.de/bounds/latest/bounds.zip

But the sea boundaries are not yet done the same way even though 
there is a new version of that file in the date specific directory:


http://osm2.pleiades.uni-wuppertal.de/sea/latest/sea_20140127.zip

Will this be done later or did something go 'wrong' with the new 
build process of the sea boundaries ?

... don't want to be stressing you, it's just a question.

Thanks
Patrik

On 31.01.2014 19:18, Lambertus wrote:
This is not hard to achieve and I'll add an easier link with the 
next update.



On 31-01-14 15:37, Patrik Brunner wrote:

Lambertus,

Actually you have a directory ./latest in both your sea and bounds 
directory in which we can find the latest version of the sea and 
and the bounds file.


Unfortunately these files still have the date tag in the name 
which makes an automatic 'fix' download of the latest boundaries 
quite hard. there's no need to change these files, but 
wouldn't it be possible to have a file just called 'bounds.zip' 
and 'sea.zip' (and respective for bz2 files) being a link to the 
actually latest file ?


So one could always download the latest file from the paths 
./bounds/latest/bounds.zip and ./sea/latest/sea.zip


Not sure how complex it is to achieve this during your automated 
generation/preparation/publishing, but guessing from my scripting 
experience it shouldn't be that hard.


Thanks for having a look at this.
Cheers
Patrik
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev




___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev




___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev




___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] bounds and sea on pleiades

2014-01-31 Thread Lambertus

This is not hard to achieve and I'll add an easier link with the next 
update.



On 31-01-14 15:37, Patrik Brunner wrote:

Lambertus,

Actually you have a directory ./latest in both your sea and bounds 
directory in which we can find the latest version of the sea and and 
the bounds file.


Unfortunately these files still have the date tag in the name which 
makes an automatic 'fix' download of the latest boundaries quite 
hard. there's no need to change these files, but wouldn't it be 
possible to have a file just called 'bounds.zip' and 'sea.zip' (and 
respective for bz2 files) being a link to the actually latest file ?


So one could always download the latest file from the paths 
./bounds/latest/bounds.zip and ./sea/latest/sea.zip


Not sure how complex it is to achieve this during your automated 
generation/preparation/publishing, but guessing from my scripting 
experience it shouldn't be that hard.


Thanks for having a look at this.
Cheers
Patrik
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] boundary France broken?

2014-01-26 Thread Lambertus

Thanks for the report Minko, the script should be fixed now. The sea 
polygons too.


On 25-01-14 22:01, Minko wrote:

Thanks Lambertus for updating those files!

There is however one issue. You packed the files in a subdirectory which mkgmap 
cannot process:

SEVERE (BoundaryUtil): splitter\10010048.o5m: boundary zip file contains 
directories. Files in directories will be 
ignored.G:\mkgmap\Boundaries\bounds_20140125.zip

I always point it directly to the zip file without unpacking it.
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] boundary France broken?


On 24-01-14 20:49, WanMil wrote:

Hi Lambertus, hi Thorsten,

Please post a link if you have installed your service so that we can 
link to it.

Thanks!

WanMil


Thanks for the example script. The bounds can be found at the following 
link:


http://osm2.pleiades.uni-wuppertal.de/bounds/

I try to update weekly, together with the map update for 
http://garmin.openstreetmap.nl

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] boundary France broken?


On 25-01-14 13:03, Patrik Brunner wrote:

Lambertus,
File properly downloadable, thanks for your efforts it seems to 
contain properly both France and Norway, allthough that's sort of 
unclear to me, but never mind.


Question:
will you also provide the precompiled sea boundaries in a similar way ?

Regards
Patrik


Yes, working on that currently.

Note: the bounds aren't fully up to date yet because I used the 'old' 
planet file that was created during the last map update (20140120). 
Bounds created with mkgmap r2982.

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] boundary France broken?


On 24-01-14 20:49, WanMil wrote:
Regarding the sea files: I can post a similar shell script within the 
next days. But that's also not a big trick:

* Install mkgmap including the extra jar files (the ones in lib/optional)
* Download the new data (land polygons) from openstreetmapdata.com
* Unzip the file
* java -cp mkgmap.jar:extra libs 
uk.me.parabola.mkgmap.sea.optional.PrecompSeaGenerator 
land_polygons.shp WGS84 sea

* Zip the new directory sea containing the new sea files.

Working on the sea now. Is a version of mkgmap available for download 
including the optional libs or do I have to compile one myself? I really 
would prefer the former option...

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] boundary France broken?


On 25-01-14 13:18, Lambertus wrote:

On 24-01-14 20:49, WanMil wrote:
Regarding the sea files: I can post a similar shell script within the 
next days. But that's also not a big trick:
* Install mkgmap including the extra jar files (the ones in 
lib/optional)

* Download the new data (land polygons) from openstreetmapdata.com
* Unzip the file
* java -cp mkgmap.jar:extra libs 
uk.me.parabola.mkgmap.sea.optional.PrecompSeaGenerator 
land_polygons.shp WGS84 sea

* Zip the new directory sea containing the new sea files.

Working on the sea now. Is a version of mkgmap available for download 
including the optional libs or do I have to compile one myself? I 
really would prefer the former option...

___

Just to show that I don't understand how it's done and what is needed :P

I've downloaded Geotools 2.7.5 then call mkgmap (replacing path_... 
with the actual path without the ''):


java -cp path_to_mkgmap_dir/mkgmap.jar:path_to_geotools_dir 
uk.me.parabola.mkgmap.sea.optional.PrecompSeaGenerator 
path_to_shape_dir/land_polygons.shp WGS84 sea 


But then I get the error:

Error: Could not find or load main class 
uk.me.parabola.mkgmap.sea.optional.PrecompSeaGenerator


I'm using mkgmap-r2982 as downloaded from the snapshots page @ mkgmap.org.uk

Any hints are appreciated...

This is the actual command in the script:
java -cp ${MKGMAPDIR}/mkgmap.jar:${GEOTOOLSDIR} 
uk.me.parabola.mkgmap.sea.optional.PrecompSeaGenerator 
land-polygons-complete-4326/land_polygons.shp WGS84 ${TODAY}


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] boundary France broken?

Many thanks Steve, the sea generator is running for 20 minutes now. It 
constantly outputs:

131072 tiles remaining

Guess this will take a while :)

I've downloaded the large tile land polygons file. While memory usage 
remains low for now, should I have chosen the alternative download with 
smaller tiles from osmd.com or add -Xmx8g to the commandline?



On 25-01-14 15:23, Steve Ratcliffe wrote:


Hi Lambertus,


java -cp path_to_mkgmap_dir/mkgmap.jar:path_to_geotools_dir
uk.me.parabola.mkgmap.sea.optional.PrecompSeaGenerator
path_to_shape_dir/land_polygons.shp WGS84 sea


But then I get the error:


Error: Could not find or load main class
uk.me.parabola.mkgmap.sea.optional.PrecompSeaGenerator


I've compiled everything and put it here: 
http://files.mkgmap.org.uk/detail/174


If you unpack the file then it should just need

  java -cp mkgmap.jar 
uk.me.parabola.mkgmap.sea.optional.PrecompSeaGenerator 
path_to_shape_dir/land_polygons.shp WGS84 sea


..Steve
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] boundary France broken?

Thanks Steve. I've added -Xmx8g but it doesn't appear to be faster nor 
use more memory. It's hitting about 350 MB with the large polygons shape.



On 25-01-14 15:57, Steve Ratcliffe wrote:

On 25/01/14 14:51, Lambertus wrote:

Many thanks Steve, the sea generator is running for 20 minutes now. It
constantly outputs:

131072 tiles remaining


I've not run it before, so don't know how long it will take or
it I am doing it right.  After a while the number did change.

Now I've got an out of memory error, so now trying it with
-Xmx8g

.. and it appears to be working much faster with the extra memory, 
even though it did not appear to be using much when I first checked.



..Steve

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] boundary France broken?

Yes, I finally saw some other output but memory usage was very high 
indeed (and swapping). So now it's working on the pre-split polygons.


I get quite a lot of the following messages, are those expected?


Compressed buffers are too short, causing extra copy


Processing won't finish in 6 minutes by looking at the progress, but I 
guess that's expected from this older cpu and memory architecture. It 
doesn't matter.



On 25-01-14 17:19, WanMil wrote:

Hi Lambertus,

you should download the split polygons. If you download the polygons 
not split you need *very* much memory. With split polygons the 
generator runs fine with -Xmx2g


After starting you should also see messages like
Worked out ... polygons

WanMil


Thanks Steve. I've added -Xmx8g but it doesn't appear to be faster nor
use more memory. It's hitting about 350 MB with the large polygons 
shape.



On 25-01-14 15:57, Steve Ratcliffe wrote:

On 25/01/14 14:51, Lambertus wrote:

Many thanks Steve, the sea generator is running for 20 minutes now. It
constantly outputs:

131072 tiles remaining


I've not run it before, so don't know how long it will take or
it I am doing it right.  After a while the number did change.

Now I've got an out of memory error, so now trying it with
-Xmx8g

.. and it appears to be working much faster with the extra memory,
even though it did not appear to be using much when I first checked.


..Steve

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] boundary France broken?


On 24-01-14 20:49, WanMil wrote:

Hi Lambertus, hi Thorsten,

Please post a link if you have installed your service so that we can 
link to it.

Thanks!

WanMil


The sea polygons can be found at the following link:

http://osm2.pleiades.uni-wuppertal.de/sea

I try to update weekly, together with the map update for 
http://garmin.openstreetmap.nl

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] boundary France broken?

2014-01-24 Thread Lambertus


Hi WanMil, all,

'My' servers have enough memory, cpu cycles, storage and bandwidth to 
host these files. Can you email me with some details, e.g. the shell 
scripts you're using?


Lambertus

On 24-01-14 10:45, WanMil wrote:

I think we can add such a link.

Anyhow is there anybody who has some server capacity and is able to 
update the bounds and sea files regularly?


For the bounds you need:
A planet file (o5m format requires around 33GB)
The osmfilter/osmupdate/osmconvert toolchain
mkgmap
Additional download of 50-60MB / day to update the planet file
Around 8GB main memory (less is possible but requires some more steps 
- I am using 3GB and therefore have to split the planet file into 3 
regions)

Some CPU power

For the sea files you need:
Download of 350GB per update
Some CPU power


I can help to create a tool chain which creates these files.

WanMil


Is it not better to put the boundary and sea files on
http://www.mkgmap.org.uk/download/mkgmap.html (at least a link)?
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev



___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] boundary France broken?

2014-01-24 Thread Lambertus


On 24-01-14 21:23, Thorsten Kukuk wrote:

Thanks, but I have my own scripts for Linux already.

Lambertus, if you want to have them or look at them, please tell me.
I don't know what you are running.

   Thorsten

Thanks Thorsten, but I've already slightly adapted WanMil's script to 
run within my environment and are creating the bounds now.


I'll post the link tomorrow when everything went fine.
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] boundary France broken?

2014-01-24 Thread Lambertus


No worries, the version file will be included in the zip. :)

On 24-01-14 22:22, Patrik Brunner wrote:

Lambertus,

I hope you also create somehow a version tag inside the two 
directories like it was done in the script of WanMil... this allows 
users to easily check which version of the boundaries is 
downloaded/used, even if the files were copied and lost therefore the 
creation date.


The relevant part of the script is:

# someone asked for a timestamp within the bounds.zip file
# so pipe the current date to file named version.txt
echo ${TODAY}  ${PLANETDIR}/${TODAY}/bounds/version.txt

Actually the mentioned 'someone' was me... ;-)

Regards
Patrik

On 24.01.2014 21:36, Lambertus wrote:

On 24-01-14 21:23, Thorsten Kukuk wrote:

Thanks, but I have my own scripts for Linux already.

Lambertus, if you want to have them or look at them, please tell me.
I don't know what you are running.

   Thorsten

Thanks Thorsten, but I've already slightly adapted WanMil's script to 
run within my environment and are creating the bounds now.


I'll post the link tomorrow when everything went fine.
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev




___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter maximum tile size

2012-12-04 Thread Lambertus

On 03/12/2012 22:24, WanMil wrote:

 But such large tiles are not displayed by Mapsource and Basecamp. They
 display mkgmap generated tiles only if the longitude and latitude tile
 size is below 179.9x°. I didn't measured the exact x (179.9° works,
 179.99° does not work).

 So a tile with the bounds
 bounds minlat=-89.0 minlon=-10.0 maxlat=90.0 maxlon=169.9/
 can be compiled and displayed with MapSource and Basecamp.

 A tile with the bounds
 bounds minlat=-89.0 minlon=-10.0 maxlat=90.0 maxlon=170.0/
 can be compiled with mkgmap but no data is displayed in MapSource and
 Basecamp.

 So splitter could use a max limit of 179° for lat and lon but I don't
 know if that makes sense at all.


Perhaps this explains why my map of New Zealand is not being shown at 
certain zoomlevels in MapSource when then 180° is in view.

See: http://forum.openstreetmap.org/viewtopic.php?id=9809
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://lists.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter r247

2012-12-02 Thread Lambertus

Please note that the message you replied to is a couple of days old and 
was stuck in the mailinglist server.


On 02-12-12 12:08, Gerd Petermann wrote:

Hi Lambertus,

okay, regarding performance I think it makes no sense to use a small 
max-areas value. The rule seems to be simple: The more passes, the 
longer the run time.
On the other hand, it would be good to know that the results are 
exactly the same  (same problem-list, same *.osm.gz, same areas.list). 
I've never tested that with planet, only

with small areas.

Gerd


Date: Fri, 30 Nov 2012 19:57:47 +0100
From: o...@na1400.info
To: mkgmap-dev@lists.mkgmap.org.uk
Subject: Re: [mkgmap-dev] splitter r247

Maybe because I forgot to change the -max-areas setting the split took 
a lot longer then expected. It even did not finish yet. CPU usage is 
around 1 to 4% now, this doesn't look good. I've attached the log.


The commandline I used was:

java -agentlib:hprof=cpu=samples,depth=20 
-XX:-HeapDumpOnOutOfMemoryError -XX:+PrintGCTimeStamps 
-XX:+PrintGCDetails -Xmx6000m -jar splitter.jar  --output=xml 
--keep-complete --overlap=0 --no-trim --mapid=1 --max-nodes=150 
--max-areas=512 --write-kml=initial.kml 
--geonames-file=cities15000.zip planet-latest.osm.pbf


I'll start the split again using --max-areas=2048

On 30-11-12 01:53, GerdP wrote:

 The results were ok, only runtime and memory usage was too high in
 your case. r250 corrects this.

 If you want to provide new logs, please run this version with java
 -agentlib:hprof=cpu=samples,depth=20 -XX:-HeapDumpOnOutOfMemoryError
 -XX:+PrintGCTimeStamps +PrintGCDetails -Xmx6000m -jar splitter.jar
  --max-areas=2048

 This should result in max. 2 hours runtime.

 If you want to compare with the previous results, try also with
 --max-areas=255

 Please send me the splitter log and the java.hprof .

 Gerd



___ mkgmap-dev mailing 
list mkgmap-dev@lists.mkgmap.org.uk 
http://lists.mkgmap.org.uk/mailman/listinfo/mkgmap-dev



___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://lists.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://lists.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter r247

2012-11-30 Thread Lambertus

whoops, I noticed a bit late that there is an IndexOutOfBounds exception 
a bit earlier in the logfile.


On 30-11-12 01:53, GerdP wrote:
 GerdP wrote
 Lambertus wrote
 Great to see that a simple log can be so helpful. :)
 Would be nice for splitter to be much faster with only a single pass on
 a moderate pc (i.e. 8 gb ram), but I'm not complaining as it is...

 Running splitter r247 with the added logging parameters on my dev pc
 now, will send the results tomorrow.
 Right now I don't even know if the result is correct. I think I will not
 need the log from r247 as I already know that it's wrong.

 Gerd
 The results were ok, only runtime and memory usage was too high in your
 case.
 r250 corrects this.

 If you want to provide new logs, please run this version with
 java -agentlib:hprof=cpu=samples,depth=20 -XX:-HeapDumpOnOutOfMemoryError
 -XX:+PrintGCTimeStamps +PrintGCDetails -Xmx6000m -jar splitter.jar 
 --max-areas=2048

 This should result in max. 2 hours runtime.

 If you want to compare with the previous results, try also with
 --max-areas=255

 Please send me the splitter log and the java.hprof .

 Gerd



 --
 View this message in context: 
 http://gis.19327.n5.nabble.com/splitter-r247-tp5737673p5738315.html
 Sent from the Mkgmap Development mailing list archive at Nabble.com.
 ___
 mkgmap-dev mailing list
 mkgmap-dev@lists.mkgmap.org.uk
 http://lists.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://lists.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter r247

2012-11-29 Thread Lambertus

Some results for r247, I hope it is of any use:

This version is able to split the planet (r246 could not) but needs more 
memory then r202. Using -Xmx4000m an OutOfMemoryException occurs but not 
with -Xmx6000m.

Number of stored ids: 19,330,031 require ca. 2.04 bytes per pair. 
1224726 chunks are used, the avg. number of values in one 64-chunk is 15.
Map details: bytes/overhead 22,288,890 / 17,251,542, overhead includes 2 
arrays with 8 MB
RLE compresion info: compressed / uncompressed size / ratio: 6,245,541 / 
19,330,024 / 68%
   JVM Memory Info: Current 5969MB (1422MB used, 4547MB free) Max 5969MB
Full Node tests:  334,886,250
Quick Node tests: 114,029,611
Thread worker-0 has finished
Thread worker-1 has finished
Thread worker-2 has finished
Distribution pass(es) took 15727941 ms (4h 20m)

$ java -version
java version 1.7.0_09
OpenJDK Runtime Environment (IcedTea7 2.3.3) (7u9-2.3.3-0ubuntu1~12.04.1)
OpenJDK 64-Bit Server VM (build 23.2-b09, mixed mode)

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://lists.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] mkgmap birthday

2012-11-29 Thread Lambertus

Congratulations Steve and anyone on the list who contributed to this 
great project! Mkgmap has really become a remarkable application.

On 26/11/2012 21:46, Steve Ratcliffe wrote:

 Hi

 On this day, 6 years ago, revision 1 was committed to svn.

 $ svn log -r1
 
 r1 | steve | 2006-11-26 20:46:08 + (Sun, 26 Nov 2006) | 1 line

 Created mkosmgmap project
 

 As you can see it originally had a slightly different, but equally
 hard to pronounce, name.

 It took less than a month to get the first working map, primitive as
 it was, and it has grown massively from there.  I never have imagined
 at the time that it would still be getting bigger and better all this
 time later.

 I'd like to thank everyone on this list who uses mkgmap and especially
 to those that have helped to develop it in the past and those that are
 doing so now.

 To celebrate I thought that I would break everything ;)

 I've updated the www.mkgmap.org.uk website to make it easier to add
 content, documentation, short news items about new features etc. in
 the future.

 You can now also sign up and get svn access to your own branch on any
 of the projects. If you already have svn access that you've used
 in the last couple of years, then you already have an account with
 the same login details as your svn account.

 There may be problems with things missing from the web site,
 subversion commit permissions, commit emails to the mailing lists etc.
 I'll probably notice from the error logs, but if you notice something
 wrong persisting, then let me know.

 Best wishes to all.

 ..Steve
 ___
 mkgmap-dev mailing list
 mkgmap-dev@lists.mkgmap.org.uk
 http://lists.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://lists.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter r247

2012-11-29 Thread Lambertus

Great to see that a simple log can be so helpful. :)
Would be nice for splitter to be much faster with only a single pass on 
a moderate pc (i.e. 8 gb ram), but I'm not complaining as it is...

Running splitter r247 with the added logging parameters on my dev pc 
now, will send the results tomorrow.

On 29-11-12 22:16, GerdP wrote:
 Oh oh!
 I looked again at this logs and wondered why some numbers did not change
 during the different passes.
 I found out that the 7 passes in the ProblemListProcessor are
 more or less nonsense. The intention was to save memory by processing only a
 set of writers
 in each pass, but in fact all 7 passes store the node informations for all
 writers.
 In short: splitter requires either much less memory whhen this bug is
 corrected,
 or you can run everything in one pass if you specify --max-areas=2048
 I hope I can fix this soon.

 Gerd




 --
 View this message in context: 
 http://gis.19327.n5.nabble.com/splitter-r247-tp5737673p5738287.html
 Sent from the Mkgmap Development mailing list archive at Nabble.com.
 ___
 mkgmap-dev mailing list
 mkgmap-dev@lists.mkgmap.org.uk
 http://lists.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://lists.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] splitter-problem-list-r242 index out of bounds exception

2012-11-24 Thread Lambertus

On 24-11-12 00:26, GerdP wrote:
 Yes, today I found that this problem is likely to occur with r242, or 
 to be precise, with splitter and my bounding box patch. It is related 
 to my question regarding alignment. I hope I can fix it soon. Gerd

Thanks for the confirmation Gerd, do you think that this is specific to 
splitting planet-size area's or will it happen when splitting much 
smaller areas as well?
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

[mkgmap-dev] splitter-problem-list-r242 index out of bounds exception

2012-11-23 Thread Lambertus

Tried to split the whole planet using splitter-problem-list-r242.jar and 
got the following error:

Elapsed time: 16m 0s   Memory: Current 1076MB (991MB used, 85MB free) 
Max 3555MB
in 1 file
Time: Fri Nov 23 23:03:14 CET 2012
Exact map coverage is (-90.0,-180.0) to (90.0,180.0)
Trimmed and rounded map coverage is (-84.990234375,-180.0) to 
(85.078125,180.0)
Splitting nodes into areas containing a maximum of 1,500,000 nodes each...
Exception in thread main java.lang.ArrayIndexOutOfBoundsException: 3869
 at uk.me.parabola.splitter.DensityMap.getNodeCount(DensityMap.java:85)
 at 
uk.me.parabola.splitter.SplittableDensityArea.getSplitHoriz(SplittableDensityArea.java:132)
 at 
uk.me.parabola.splitter.SplittableDensityArea.split(SplittableDensityArea.java:75)
 at uk.me.parabola.splitter.Main.calculateAreas(Main.java:344)
 at uk.me.parabola.splitter.Main.split(Main.java:190)
 at uk.me.parabola.splitter.Main.start(Main.java:145)
 at uk.me.parabola.splitter.Main.main(Main.java:134)

The commandline was:

java -Xmx4000m -ea -jar splitter.jar --output=xml --keep-complete 
--overlap=0 --no-trim --mapid=1 --max-nodes=150 
--write-kml=initial.kml --geonames-file=cities15000.zip 
planet-latest.osm.pbf

Is this a known problem? Searching the list and websearch did not result 
in a hit...
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

[mkgmap-dev] Feature request for Splitter

2012-08-13 Thread Lambertus

This is a request for a new Splitter method.

It would greatly speed-up my map generation if Splitter could use a 
split-count parameter that tells Splitter to output N tiles instead of 
having to manipulate the max-nodes and max-ways parameters to get the 
required amount of split results.

Background:
My scripts initially split the world using a high max-nodes parameter in 
order to create the tiles as large as possible.

When rendering of a split result fails (or the resulting Garmin image is 
larger then 8MB) then Splitter is called with a decreasing amount of 
max-nodes until the split action results in two or more tiles. For each 
split action the max-nodes count is reduced with 100k nodes. For some 
areas, like Paris in France, the initial tile is split about 15-16 times 
before the result is successfully rendered and passes all tests.

This takes a lot of time while most sub-splits result in only a single 
tile until the max-nodes is reduced from 1.400.000 to less then 100.000. 
So the amount of split actions would be greatly reduced if Splitter 
could simply be told to return 2 (or more) tiles.
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] WARNING: input files have differing sort orders

2012-07-17 Thread Lambertus

Thanks Steve and Hermann, the description helps.
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] WARNING: input files have differing sort orders

2012-07-12 Thread Lambertus

I get this warning too. From the previous posts it is not clear to me 
what the effect of this warning on is the functionality of the generated 
maps. Is it harmful or can it be ignored?



On 4-7-2012 17:21, Steve Ratcliffe wrote:
 Hi

 It's a bug that occurs when you have type file.  I don't think that
 the type file has a sort order field (though I must check since I'd
 not considered that untilled just now).

 I guess that it would be good if the file name could be printed when
 this happens.

 I'll post a patch later

 Patch attached which fixes this and will properly warn if the
 TYP file has the wrong code page.

 ..Steve


 ___
 mkgmap-dev mailing list
 mkgmap-dev@lists.mkgmap.org.uk
 http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev



___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Discussion in Talk-DE about Garmin-maps and their consumers

2012-07-05 Thread Lambertus

Hello Henning,

I tried to follow the discussion but my German isn't good enough to 
understand all the details and Google translate is awkward. I don't know 
if I can really contribute to the discussion.

Do I understand correctly that some commentators fear that e.g. my 
service stops because it's mostly a one-man operation? What are the 
proposed solutions to that?

Thanks.

On 4-7-2012 11:41, aighes wrote:
 Hi,
 yesterday started a interesting discussion about How should/could OSM
 (de) present Garmin-maps.

 I think many mapdevelopers read this list. So if you are interested...it
 would be nice to participate. There are already some interesting
 thoughts, but maybe you have also some ideas.

 The problem is, that many users doesn't know how many maps exist and
 how to find the map they need. Of course there is a wiki-list...but this
 list is quiet ugly and confusing.


 http://gis.19327.n5.nabble.com/Feedback-vom-OpenStreetMap-Stand-auf-dem-GeoGames-Leipzig-Event-td5714855.html
 http://www.mail-archive.com/talk-de@openstreetmap.org/msg94992.html

 Discussion is till now in German, but I think you could also post in
 english, if you like or if many are interested, we should move dicussion
 to talk.

 Henning


 ___
 mkgmap-dev mailing list
 mkgmap-dev@lists.mkgmap.org.uk
 http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev



___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Discussion in Talk-DE about Garmin-maps and their consumers

2012-07-05 Thread Lambertus

On 5-7-2012 13:39, aighes wrote:
 The main issue is, that many users doesn't find our maps, because they
 are hidden. There is only the wiki-list, but this list isn't quite
 usable to normal people. The fear is only a minor problem.

When I type in Google: Free Garmin maps I find my website listed first 
and the OSM Garmin/Download wiki page listed in 4th place.

Similarly searching for OpenStreetMap Garmin map lists my website 
first and the OSM Garmin/Download wiki page in 5th place.

Similar results are returned using Bing search.

So, I don't really understand why people can't find the maps.

 So there were two ideas how to fix this issue.

 1)
 Have a prominent website (linked e.g. by openstreetmap.de), which
 contains information to all available maps and guide the user to the
 map, he is looking for. User could watch a screenshot an read the main
 explanations to the map and then will be linked to download-website of
 the map or directly to the download.

Ofcourse this is a good idea. E.g. The Dutch OSM page has such a link.

 2)
 Set up a powerful renderserver, who renders several maps OnDemand
 (inclusive caching of course) and deliver several mapstylings to the
 user directly. This service will also be linked e.g. by openstreetmap.de.

 So you are offering such a OnDemand-Service. So it would be great to
 have your opinion how powerful such a server has to be and how is your
 user-feedback.

The current server is a few generations old dual CPU dualcore Xeon 2.0 
GHz (4 cores total), 8 GB ram, 900 GB cache and 100 Mbit internet 
connection.

The server generates 3 maps simultaneously at a rate up to about 500 
unique maps per 24 hours. It could be more but at this pace the cache is 
usually full within 24 hours and map generation is throttled. Also the 
internet connection is sometimes fully saturated which takes it's toll 
on disk performance (i.e. disk heads are moving as fast as they can 
while the system is starved for data), this slows the generation of maps.

About 1500 unique users download one or more maps per day at a rate of 
around 300GB per day. Caching levels are about 30% I would guess 
(unfortunately I don't have good numbers on that), it helps tremendously.

So if you plan on setting up a similar on-demand system that provides 
more map types to more people then use at least:
- a modern quadcore CPU
- 2 TB fast disk cache
- Good 100 MBit internet
- 8 GB ram
- unlimited bandwidth usage

I also have experience with on-demand rendering using a renderfarm in 
which case the CPU and disk cache per box is less important, but enough 
ram and fast internet connection remain important. Configuring all the 
details and distributing map updates is a pain in the ass though.

Ofcourse there is still room for smarter caching and reducing the 
cache/CPU usage by producing maps more efficiently (i.e. less versions 
of the same map), but it requires a lot of effort to improve significantly.

Regarding user feedback, I get a lot of it. Lots of thank you's 
ofcourse, but there is also a large group of internet users for which 
you can't make things simple enough and well-documented enough for them 
to manage without help. Picture your typical computer-illiterate 
mother-in-law who decides that she needs a free update for her satnav 
using the babysitter on this side of the screen. Or freshly pensioned 
guy who decides he's going to pick up the geocaching hobby but never 
used a pc before. :p

Those WILL contact you and expect to be helped, for free.

So, if you plan on setting up a similar system like I did: plan for a 
lot of your time to perform helpdesk duties answering silly questions, 
figuring out what the h*ll people are trying to tell you and find ways 
around individual firewall/virusscanner/InternetExplorer problems, etc. :)
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] java.lang.IllegalArgumentException and max-nodes size --- currently impossible for me to create Asia map without a) breaking or b) missing tiles

2012-05-25 Thread Lambertus

Splitter could be integrated into Mkgmap, but it has proven not too 
difficult to script a subsplit if an Mkgmap render attempt failed. The 
scripts is described in normal language below, implementing it in Perl, 
Python, bash or even PHP (as I have) is easy. It might keep Mkgmap 
slightly ;-) simpler to maintain without splitter incorporated.

In my script:
When rendering an osm.gz file I check the return value of Mkgmap, the 
existence of a resulting .img file and that the .img file is larger then 
0 bytes. If any of these checks fail then splitter is invoked using a 
lower node count (e.g. 100k nodes less) until the subsplit results in 
two or more osm.gz files. These files are then rendered again and for 
each the process described above is performed again.

The only challenge I encountered using this process is keeping tabs on 
the tile numbers and providing the correct tile number as commandline 
parameters to Mkgmap and Splitter. I'll post the code of my scripts if 
there is a need for it.

On 25-05-12 17:04, Felix Hartmann wrote:
 It is not really nice currently to try to compile Asia from Geofabrik.
 If I set a high max-nodes value, then it goes through (but about 30
 tiles missing), if on the other hand I choose a lower max-nodes value,
 then the following bug appears.
 Splitter integration into mkgmap would really be helpful...


 Exception in thread main java.lang.IllegalArgumentException
   at java.nio.Buffer.position(Unknown Source)
   at uk.me.parabola.imgfmt.Utils.bytesToString(Utils.java:80)
   at
 uk.me.parabola.imgfmt.sys.ImgHeader.setHeader(ImgHeader.java:255)
   at uk.me.parabola.imgfmt.sys.ImgFS.readInitFS(ImgFS.java:313)
   at uk.me.parabola.imgfmt.sys.ImgFS.openFs(ImgFS.java:131)
   at uk.me.parabola.imgfmt.sys.ImgFS.openFs(ImgFS.java:124)
   at
 uk.me.parabola.mkgmap.combiners.FileInfo.imgInfo(FileInfo.java:184)
   at
 uk.me.parabola.mkgmap.combiners.FileInfo.getFileInfo(FileInfo.java:144)
   at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:429)
   at
 uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:126)
   at uk.me.parabola.mkgmap.main.Main.main(Main.java:114)

 ___
 mkgmap-dev mailing list
 mkgmap-dev@lists.mkgmap.org.uk
 http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Commit: r2109: Refactor the default style a little.

2012-02-07 Thread Lambertus

Thanks GerdP and Marko!

On 7-2-2012 21:50, Marko Mäkelä wrote:
 On Tue, Feb 07, 2012 at 12:22:15PM -0800, GerdP wrote:
 Hi Lambertur,

 I think it is possible and probably not intended. The file default/info is
 missing the line
 base-style=landuse

 Sorry, you are right. I will add it. I guess I did not notice, because
 forests are being tagged natural=wood around here.

   Marko
 ___
 mkgmap-dev mailing list
 mkgmap-dev@lists.mkgmap.org.uk
 http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

[mkgmap-dev] Stackoverflow while loading style files

2012-01-14 Thread Lambertus

Hopefully someone can shed some light on what I'm doing wrong?

java -Xmx1000M -ea -jar ~/garmin/utils/mkgmap/mkgmap.jar 
--style-file=/home/lambertus/garmin/world/styles/default/ 
--input-file=63240001.osm.gz

java.lang.StackOverflowError
 at sun.nio.cs.UTF_8.updatePositions(UTF_8.java:58)
 at sun.nio.cs.UTF_8$Encoder.encodeArrayLoop(UTF_8.java:392)
 at sun.nio.cs.UTF_8$Encoder.encodeLoop(UTF_8.java:447)
 at java.nio.charset.CharsetEncoder.encode(CharsetEncoder.java:544)
 at java.lang.StringCoding$StringEncoder.encode(StringCoding.java:240)
 at java.lang.StringCoding.encode(StringCoding.java:272)
 at java.lang.String.getBytes(String.java:946)
 at java.io.UnixFileSystem.getBooleanAttributes0(Native Method)
 at java.io.UnixFileSystem.getBooleanAttributes(UnixFileSystem.java:228)
 at java.io.File.isDirectory(File.java:754)
 at 
uk.me.parabola.mkgmap.osmstyle.DirectoryFileLoader.init(DirectoryFileLoader.java:47)
 at 
uk.me.parabola.mkgmap.osmstyle.StyleFileLoader.createStyleLoader(StyleFileLoader.java:64)
 at uk.me.parabola.mkgmap.osmstyle.StyleImpl.init(StyleImpl.java:132)
 at 
uk.me.parabola.mkgmap.osmstyle.StyleImpl.readBaseStyle(StyleImpl.java:518)
 at uk.me.parabola.mkgmap.osmstyle.StyleImpl.init(StyleImpl.java:140)
 at 
uk.me.parabola.mkgmap.osmstyle.StyleImpl.readBaseStyle(StyleImpl.java:518)
 at uk.me.parabola.mkgmap.osmstyle.StyleImpl.init(StyleImpl.java:140)
 at 
uk.me.parabola.mkgmap.osmstyle.StyleImpl.readBaseStyle(StyleImpl.java:518)
(Lot's of the same errors like the last 6 lines snipped)

The styles directory is an svn export of the styles section in the 
Mkgmap SVN repository, svn info reports version 2160. Mkgmap version is 
r2162.
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Stackoverflow while loading style files

2012-01-14 Thread Lambertus


Great find Minko! When the commandline is changed to:

java -Xmx1000M -ea -jar ~/garmin/utils/mkgmap/mkgmap.jar 
--style-file=/home/lambertus/garmin/world/styles/ --style=default 
--input-file=63240001.osm.gz


Then it works.

If there is more then one style in the styles directory then set the 
--style-file to the styles directory and select the style using the 
--style option.
If there is only one style in the styles directory then the --style-file 
option can be set tot the style directory and the --style option can be 
omitted.


Thanks!

On 14-01-12 13:18, Minko wrote:

Lambertus, maybe this link helps?
http://www.mail-archive.com/mkgmap-dev@lists.mkgmap.org.uk/msg11034.html
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Commit: r2162: If the global address index option is given alongside the --gmapsupp

2012-01-06 Thread Lambertus

Awsome guys! A big thank you all for this and all the other tweaks, 
improvements and new features.

Steve, do I understand it correctly that if I want to create a gmapsupp 
and also a tdb then I better run Mkgmap twice? Once with --gmappsupp and 
once with --tdb?

Also, would you be so kind to put a compiled version of r2162 on the 
download page? Thanks in advance.

On 06-01-12 17:58, svn commit wrote:
 Version 2162 was commited by steve on 2012-01-06 16:58:50 + (Fri, 06 Jan 
 2012)

 If the global address index option is given alongside the --gmapsupp
 option, then the index will be created directly within the
 gmapsupp.img file, so that address search will work on devices without
 having to install via MapSource.

 If in addition the tdbfile option is given, then the existing
 MapSource style index will also be created.  This will take about
 twice the amount of memory, so it best to avoid doing this.
 ___
 mkgmap-dev mailing list
 mkgmap-dev@lists.mkgmap.org.uk
 http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

[mkgmap-dev] A few default stylesheet tweaks

2011-07-18 Thread Lambertus

Hello list, Marko,

Would it be possible that the default stylesheet does not show:

- Admin levels 8, 9, 10 (admin_level=)
- Communication (GSM/Broadband) towers (type=communication)
- Powerlines (power=line)

These do not add much functionality but clutter the map and confuse people.

What do you think?
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] A few default stylesheet tweaks

2011-07-18 Thread Lambertus

On 18-07-11 18:56, Rich wrote:
 On 07/18/11 19:46, Lambertus wrote:
 Hello list, Marko,

 Would it be possible that the default stylesheet does not show:

 - Admin levels 8, 9, 10 (admin_level=)
 - Communication (GSM/Broadband) towers (type=communication)
 - Powerlines (power=line)
 latter two are fairly good for navigating on the countryside

Hmm, isn't that what the GPS is for? :p

In urban areas the communication towers totally overrule everything else 
on the map. Personally this more then nullifies the benefit in the 
countryside. The powerlines distract from the rest of the map but to a 
lesser extent so I can live with that. I received multiple reports from 
people arguing that the higher admin levels confuse them as they are 
prominently displayed, but I'll propose a compromise to not showing only 
9 and 10 if 8 is deemed useful. :-)

Currently I'm pre-filtering these things before feeding the data into 
splitter but the admin levels are used in the new address search version 
and some of these polygons are used for roads as well and disappear from 
the map when the polygon is pre-filtered out. Perhaps there is another 
solution for this? E.g. renaming admin levels to something else, or 
duplicate ways which have other tags?
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Denmark: SEVERE (MapBuilder): ./63240348.osm.gz: FIXME - too many POIs in group

2011-05-19 Thread Lambertus

On 19-05-11 21:21, Daniela Duerbeck wrote:
 Hi!

 When I try to build a map of Denmark, I get a lot of SEVERE
 (MapBuilder): ./63240348.osm.gz: FIXME - too many POIs in group
 messages. Then the mkgmap processes exits without generating a map.
 What can I do?

I use a script that re-splits the tile for situations like where Mkgmap 
does not produce a map from initial Splitter results. The script reduces 
node count until splitter produces 2 or more 'sub'tiles, then tries to 
render those subtiles and repeat the process until all subtiles are 
rendered if necessary or until the node count reaches a lower limit.

So the process goes like this:
1- Split your osm.pbf/bz2 file with an initial max-nodes setting (e.g. 
1.2 million)
2- Attempt to render the tiles with Mkgmap into Garmin images
3- See if any of the new Garmin images failed (i.e. do not exist or 0 
bytes in size)
4- Re-split the failed tiles with ever lower max-node counts until 
splitter produces 2 or more subtiles
5- Goto 2 (but render only the subtiles ofcourse)

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] [locator] Simplifying the check line in polygon

2011-05-16 Thread Lambertus

This sounds like a performance problem I had matching all Garmin tiles 
to country polygons. Some country definitions from Cloudmade contain 
more then 1000 polygons and there are almost 2000 tiles covering the 
world so you can imagine that this takes quite some processing time.

This is solved this by first generating a simple bounding box for all 
the country polygons. Then each tile is first pre-filtered by matching 
against the bbox (most will fall outside the bbox so this is where the 
computational speedup is made). Tiles that pass this pre-filtering are 
then matched against the real polygons.

Matching against polygons and bboxes is done using a the inside/outside 
functions of a 2D library (similar to what Jon Burgess is linking).

Speedup using such a technique is (in my case) _at least_ a factor 10.

This '1-box' tile matching algorithm has one caveat: it doesn't provide 
any speedup when you're matching against a bbox that crosses the 
-180/+180 degrees line, because all tiles will match the pre-filtering. 
When such a condition is detected the algorithm is changed to a '2-box' 
variant. This variant is a bit slower because it uses one bbox for 180 
degrees and one bbox for -180 degrees, assuming that no boundary will 
cover the full 380 degrees.

Applying this to your problem:
- First create a simple bbox for each polygon
- Pre-filter lines/polygons against the simple bbox
- If passed the pre-filter then exact-filter lines/polygons against the 
polygon.

My code is in PHP which I can provide if you're interested in seeing an 
example, but the idea should be clear right?
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Splitter output files are nearly empty

2011-05-04 Thread Lambertus

On 2011-05-04 07:44, Marko Mäkelä wrote:
 The processing time probably won't be reduced if the machine starts
 swapping. Like Thorsten pointed out, it is good to leave some breathing
 room for the computer.

Sure, but the machine has 8GB ram so it won't have to swap while doing 
multiple things at the same time and swapping did not take place 
(according to Munin). I'll run the script again with 5 or 6 GB heap, but 
I'm not convinced it will change the output or processing time 
significantly.

 I am not sure if the --cache option still works. It was sort of
 superceded or made unnecessary by the Protobuf input format support.

 Oh, if it's not working anymore then I assume it will ignore the
 option. Or is this too simple thinking?

 I do not know. I never used the --cache option myself. One more
 possibility (a wild guess) is that the format of the cache directory
 changed and you had some old-format stuff there that was being
 misinterpreted.

The script does a full cleanup before starting the processing. There is 
no cache directory lingering around. But I try to remember if I saw a 
cache directory while processing and I don't think I did, so maybe the 
option is already ignored.

Anyway, I've generated a pbf planet file from the bz2 XML last night, so 
can test if splitting pbf will be successful this evening.
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Splitter output files are nearly empty

2011-05-04 Thread Lambertus

On 2011-05-04 11:51, Steve Ratcliffe wrote:
 * Full GC *
 Elapsed time: 18m 0s   Memory: Current 135MB (106MB used, 29MB free) Max 
 MB

 The 'Full GC' line is not a problem, it is a message printed by
 splitter every so often before it attempts to force a garbage collect.
 This is not usually a useful thing to do, but it can help make the
 printed memory usages more accurate, I suppose.

Thanks for clearing that up Steve. The newer splitter apparently needs 
only a fraction of the memory it needed before. Great work devs!

 Splitter can still read .osm.gz from stdin as long as there is a
 --split-file
 option, and as long as it doesn't have to use more than one pass
 through the data.

Alright. I don't think splitter is able to only use one pass on a planet 
file so it's another reason to move to the pbf format.

 You should just provide no file name on the command line though.  (ie
 do not give /dev/stdin) However /dev/stdin should work as long as the
 input is uncompressed.

Thanks, the /dev/stdin ended up there because of other problems I had 
and it worked. I can't confirm or deny that it works without it ;-)

 Previously there was a --cache option which would copy and convert the
 input to a cache file, which would then be re-read if --split-file was
 not given or if more than one pass was needed.

 Since the newer pbf format is as or more efficient than the cache file
 format and osmosis is capable of producing it directly it seems best
 to just use osmosis to write it. Having splitter convert to pbf will
 not save any disk space and will probably be slower than overall
 than osmosis producing it directly.

Again: thanks. It's clear I'll better migrate to pbf.


___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Splitter output files are nearly empty

2011-05-03 Thread Lambertus

On 03-05-11 20:01, Marko Mäkelä wrote:
 On Tue, May 03, 2011 at 05:48:55PM +0200, Lambertus wrote:
 I assume that GC means garbage collection. Which JVM are you using? An
 educated guess is that the memory runs out and a full garbage
 collection cycle is started as a last resort.

Ok, Garbage Collection sounds reasonable, but I gave Java 7.5 GB heap 
space while it used only a few hundred megabytes. I've got no idea why 
it would run out of memory.

java version 1.6.0_24
Java(TM) SE Runtime Environment (build 1.6.0_24-b07)
Java HotSpot(TM) 64-Bit Server VM (build 19.1-b02, mixed mode)

 - Is input from stdin still functional/supported for splitter?
 I am not sure. I am using pbf output.

Yeah, I'm slow in moving to pbf, but is OSM XML really not 
functional/supported anymore? Splitter seems to accept the XML in the 
first phase without complaints. It's only in the step where it starts to 
write-out the tiles that is starts to complain. It would be better if 
Splitter rejects the input if it can't handle it properly later on?
 Why not use

 osmosis --rb world.osm.pbf ... --wb filtered-world.osm.pbf

 and

 ... splitter.jar filtered-world.osm.pbf

Well I wanted to run multiple processes at the same time to utilize the 
quadcore CPU better and reduce processing time (but, then I shouldn't 
have used bz2 XML in the first place). Ok, I'll try this next.
 I am not sure if the --cache option still works. It was sort of
 superceded or made unnecessary by the Protobuf input format support.

Oh, if it's not working anymore then I assume it will ignore the option. 
Or is this too simple thinking?

Thanks for the response Marko. I'll start a new run using Osmosis pbf 
output.
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Splitter output files are nearly empty

2011-05-03 Thread Lambertus

On 03-05-11 23:25, Thorsten Kukuk wrote:
 On Tue, May 03, Lambertus wrote:
 Ok, Garbage Collection sounds reasonable, but I gave Java 7.5 GB heap
 space while it used only a few hundred megabytes. I've got no idea why
 it would run out of memory.
 Do you really have so much free memory available? If you tell
 Java it should use 7.5GB heap space, it will use that, even if not
 available, and fail. I run Java with 4GB heap space (machine had more),
 but since other processes allocated memory, too, it did fail sometimes
 with similar symptoms. After I reduced the memory to 2.5 GB heap space,
 it runs fine.

The machine's got 8 GB ram and 8 GB swap. There were other processes 
running as well, like bunzip and Osmosis but those don't use much ram.

Even so, the logs say it's using only 250 MB ram, that should not cause 
problems right? If the logs said it was using more then 6GB then, ok, I 
could accept such an explanation, but 250 MB

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] [locator] Separate boundary files

2011-04-29 Thread Lambertus

On 2011-04-29 14:37, Martin wrote:
 Has anybody already uploaded the file?

 Martin

An older (november 2010) coastline extract from the Planet dump is here: 
http://planetosm.oxilion.nl/~lambertus/coastline.osm.gz
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] junction=roundabout

2011-04-23 Thread Lambertus

Op 23-04-11 10:42, Henning Scholland schreef:
 Hi Minko
 Might it be an fault in OSM-data? In Germany junction=roundabout imply
 oneway=yes. I don't know exactly how this is handled in other countries,
 but I won't be surprised, if this is globally implied. So if there is an
 roundabout, which isn't an oneway, you should add an oneway=no.

 Henning

 ___
 mkgmap-dev mailing list
 mkgmap-dev@lists.mkgmap.org.uk
 http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
I might add that I suspect Ligfietser (=recumbent cyclist) is talking 
about cycleway roundabouts (or cycleways that circle a roundabout). Many 
of these (at least in the Netherlands) allow cyclists to take the 
roundabout in both directions.

So, yes, for cars roundabouts will be oneways mostly everywhere, but not 
necessarily for cyclists or pedestrians. I also note that the wiki 
tagging page suggests that oneway=yes is implied, so it follows that 
every non-oneway roundabout should be tagged oneway=no?
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Address search and index.

2011-02-14 Thread Lambertus

On 2011-02-13 23:52, Steve Ratcliffe wrote:
 There is a big problem with address search with poor and inconsistent map 
 data.  With my map of England I get the choice of four different variations 
 on the name of the country to choose from. One must be chosen and doing so 
 means that you can only find streets that have that particular country name.

 Now there is no way for me to know since after all everything is really in 
 the same country.

 So to make address search really useful the country names have to be cleaned 
 up, (other countries may be in a better state).

 Does anyone have any good ideas about this?

 ..Steve
 ___

Don't know if it's a *good idea*, but, anyway..

Define a list of countries for Mkgmap and ignore all other countries. 
That will stimulate users to fix the data because they can't find a place.
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Index branch - success!

2011-02-11 Thread Lambertus

Sounds terrific! Congratulations :D

Op 11-02-11 20:11, Steve Ratcliffe schreef:
 Hi

 Some progress on the index branch.

 I found that the flags at the end of mdr7 trigger the acceptance of
 the 20-29 sections.

 I can now download the maps to my Legend.

 Now when you try address search, it is in a completely different mode.
 There is a country field (although labelled region) and as you make
 selections in the country/city fields it reduces the options available
 in the streets field

 Addresses can be found in other tiles and not just the closest one.

 ..Steve
 ___
 mkgmap-dev mailing list
 mkgmap-dev@lists.mkgmap.org.uk
 http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Splitter question

2010-11-26 Thread Lambertus


On 2010-11-25 19:45, Danny Backx wrote:

On Thu, 2010-11-25 at 14:17 +0100, Lambertus wrote:

If there is interest in my script (PHP) then I can clean it up a bit and
post it here.


Yes please.

Danny


The script is attached. I've added some inline comments and hope all is 
clear. Don't hesitate to ask if you have any questions.
= 10) {
		echo "Splitting $tileid with nodes=$nodes, starting at $mapid. Max id was $maxid, level = $level\n";

		$command = "java -Xmx1792M -jar ~/garmin/utils/splitter/splitter.jar --no-trim --mapid=".$mapid." --max-nodes=".$nodes." --write-kml=".$tileid.".kml --geonames-file=$cities $tileid.osm.gz";
		passthru($command);
		$newtileid = $maxid;

		// Determine the maximum id produced by the split
		$newmaxid = getMaxSourceId();
		
		// require at least two sub files
		if (($newmaxid - $maxid) < 2) {
			echo "The split did not result in at least two subtiles, try again with less nodes per tile\n";
			$level++;
			$newmaxid = subsplit($tileid, $maxid, $level);
		} else {
			// Render the new source files
			$maxid++;
			$tiles = $newmaxid - $maxid;
			for ($i = 0; $i <= $tiles; $i++) {
echo "Start a new recursive render with $maxid, $newmaxid, $level, $nodes\n";
$newmaxid = render($maxid, $newmaxid, $level);
$maxid++;
			}
			echo "Done with subsplit at level $level\n";
		}
	} else {
		echo "Subsplitting passed the minimum node count. Giving up\n";
		$newmaxid = $maxid;
	}	
	return $newmaxid;
}

// Return the number of the file with the highest filenumber
function getMaxSourceId() {
	$maxid = 0;
	$ids = getSourceIds();
	foreach ($ids as $id) {
		if ($id > $maxid)
			$maxid = $id;
	}
	return $maxid;
}

// Return the available filenumbers
function getSourceIds() {
	$ids = array();
	foreach (glob("*.osm.gz") as $filename) {
		$parts = explode(".", $filename);
		$ids[] = $parts[0];
	}
	return $ids;
}

// Delete a directory tree. Please ensure $dir ends with a slash
function delTree($dir) {
$files = glob( $dir . '*', GLOB_MARK );
foreach( $files as $file ){
if( substr( $file, -1 ) == '/' )
delTree( $file );
else
unlink( $file );
}
rmdir( $dir );
} 

/* 
 * Below are some functions to update the list of available tiles from the initial split with the subsplit results
 */
function simplexml_append(SimpleXMLElement $parent, SimpleXMLElement $new_child){
$node1 = dom_import_simplexml($parent);
$dom_sxe = dom_import_simplexml($new_child);
$node2 = $node1->ownerDocument->importNode($dom_sxe, true);
$node1->appendChild($node2);
} 

function simplexml_replace(SimpleXMLElement $parent, SimpleXMLElement $new_child){
   $node1 = dom_import_simplexml($parent);
   $dom_sxe = dom_import_simplexml($new_child);
   $node2 = $node1->ownerDocument->importNode($dom_sxe, true);
   $node1->parentNode->replaceChild($node2,$node1);
}

function updateKml($name) {
	$kmlRed = "
  ";
	$kmlBlue = "
  ";

	// Load the original KML file
	if (file_exists("$name.kml")) {
		$xml = simplexml_load_file("$name.kml");
	} else {
		exit("Failed to open $name.kml.");
	}
	
	// loop through all KML files to update the original KML file
	foreach (glob("*.kml") as $replacement) {
		if (strpos($replacement, $name) === false) {
			$id = explode(".", $replacement);
		
			// Load each new KML file
			$newxml = simplexml_load_file($replacement);
			foreach ($xml->Document->Placemark as $oldPlacemark) {
if (strpos($oldPlacemark->name, $id[0]) !== false) {
	// This KML file contains subtiles -> delete the tile they replace

	$
	$oNode->parentNode->removeChild($oNode);
	break;
}
			}
		
			// Add the subtiles
			foreach ($newxml->Document->Placemark as $newPlacemark) {
simplexml_append($xml->Document, $newPlacemark);
			}
		}
	}

	// Determine which tiles actually exist (update KML rendering)
	$red = simplexml_load_string($kmlRed);
	$blue = simplexml_load_string($kmlBlue);
	simplexml_replace($xml->Document->Style, $red);
	simplexml_append($xml->Document, $blue);
	
	foreach ($xml->Document->Placemark as $tile) {
		$id = $tile->name;
		if (file_exists($id.".img") == true) {
			$tile->styleUrl = "#Blue";
		} else {
			$tile->styleUrl = "#Red";
		}
	}

	// Write out the new version of the KML file
	$fh = fopen("$name.kml.1", "w+");
	if ($fh) {
		fwrite($fh, $xml->asXML());
		fclose($fh);
	}
	// rename original KML file to old
	rename("$name.kml", "$name.kml.old");
	
	// rename new KML file to original
	rename("$name.kml.1", "$name.kml");
}

/* The original Mkgmap commandline from the render() function:
 * $output = passthru("

Re: [mkgmap-dev] Splitter question

2010-11-25 Thread Lambertus

I think I've been successfully working around this 'problem' without the 
need to hack Splitter, but Chris Miller did changed some things based on 
off-list communication (Details elude me at this moment though) to make 
this possible.

My process in splitting the planet is essentially a 2-step method:

1. Initial split to generate large tiles, some of which might be 
processed successfully. The ones that don't pass go into the second step.

2. A recursive process in which:
2a. Split the tile with a diminishing --max-nodes setting until Splitter 
returns at least two subtiles.
2b. The resulting subtiles can be processed by Mkgmap, or RoadMap in the 
Danny's case. If a subtile can't be processed then goto 2.

This way I can create tiles as large as possible but with varying 
amounts of nodes in an automated way, without the need for manually 
defined tile definitions.

The only prerequisite for this process is the ability to 'know' when the 
processing of a splitted tile is successful.

It is essentially the same process as the 'doit' and 'deeper' scripts 
but in an recursive way so that the depth of the process is variable 
depending on the actual needs instead of fixed 2 levels.

If there is interest in my script (PHP) then I can clean it up a bit and 
post it here.

On 2010-11-22 18:02, Scott Crosby wrote:
 On Sun, Nov 21, 2010 at 1:04 PM, Danny Backx danny.ba...@scarlet.be
 mailto:danny.ba...@scarlet.be wrote:

 I've taken some time and played with the suggestions given.

 I'm including two scripts. One (doit.nl http://doit.nl) is a
 sample script to split the
 OSM file for the Netherlands into manageable pieces, and then to convert
 the OSM files into RoadMap format. This mostly works :
 - I get 339 map files
 - 9 maps turn out to be too big still for RoadMap

 So I created another script (deeper.nl http://deeper.nl, used in a
 subdirectory, you'll
 notice that in the script) that'll take the files I put in the
 subdirectory, and divide them in four smaller chunks.

 I used deeper.nl http://deeper.nl and still had problems with 3 of
 the smaller chunks so
 I reran deeper.nl http://deeper.nl one more time, producing maps
 that were (again) four
 times smaller than the offending OSM files.



 I realize that I'm working around something splitter is leaving
 unsolved, and I'm partially automating this, my question is whether all
 this is a good idea.


 If you're willing to do some work on the splitter to improve it, I think
 that yes, you could improve it in a few days of work to the point where
 this becomes a non-issue for you, and the splitter generates excellent
 and well-balanced tiles.

 The core problem is that the splitter needs to track the point density
 around the world very densely. It, divide the world into, say 8192 tiles
 horizontally and 8192 vertically. It goes through the planet it counts
 how many nodes there are for each tile, and then uses that summary
 'density map' table to derive the areas for splitting, by, say. Your
 problem is that you want regions smaller than 1*1, because even a 1*1
 has too much data for your program. Increasing the density that stuff is
 tracked, to 32k*32ktiles across the world increases memory by 16x and
 might not solve your problem because the the minimum sized tile of 1*1
 may still have too many nodes, even if it is smaller in geographic size.

 The fix for you is that dividing Europe into, say 16k*16k tiles
 generates tiles with a smaller geographic extent than dividing the world
 into tiles of 32k*32k. You should be able to feed the geographic bounds
 data from the bound tag to configure the density map to only cover
 part of the world. There are a few catches in that you'll have to obey
 the alignment constraints that are required by mkgmap. (Which, alas, I
 do not know.). But that would solve your problem and might make the
 splitter more useful to others..

 There are also a few smaller-scale things that may help. For instance,
 the minimum are size in the splitter is 2*2 tiles, not 1*1 tiles.

 Scott



 ___
 mkgmap-dev mailing list
 mkgmap-dev@lists.mkgmap.org.uk
 http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Sea generation

2010-11-05 Thread Lambertus

On 2010-11-04 23:04, WanMil wrote:
 Some loud thinking:
 Would it be helpful if we copy the Mapnik behaviour of well defined
 coastlines? One could define one separate file that contains all
 coastline data (from europe, from the world?). This file could be
 maintained and improved in a better way than it is done in the OSM data.
 natural=coastline would be ignored/removed from the OSM input.
 As a second step the separate coastline file would build a basis and the
 data from the OSM could be compared to this. Differences in the OSM
 files are only accepted if they don't invert the sea polygons and if
 they don't build new gaps.

I think it would. With so many inexperienced users one should expect 
that the coastline is broken at points.

According to the coastline wiki page:
At high zoom levels the coast polygons used are generated from the 
natural=coastline tag -- the data is made available to the Mapnik 
renderer as a large shapefile (processed_p) which is generated every few 
weeks from planet dumps

I gather that this file should contain a good coastline:
http://tile.openstreetmap.org/processed_p.tar.bz2 (357 MB)

 Question:
 Has anyone extracted coastline data for the whole world/europe/asia/...?
 How many data is this?

I've not done it yet, but will do...

 Yeah. The mp code throws all polygons away that don't touch the bounding
 box. Additionally it closes polygons automatically if their endpoints
 lie outside the bounding box.

Would that mean that a polygon is closed inside the map instead than at 
the tileborder if there aren't nodes within the overlap area?

That would be unfortunate because processing time greatly increases if 
the overlap area is increased in an attempt to include more nodes.

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Sea generation

2010-11-04 Thread Lambertus

Op 04-11-10 20:24, Marko Mäkelä schreef:
 On Thu, Nov 04, 2010 at 08:06:18PM +0100, Lambertus wrote:
 So I'm looking forward to any improvements that work without
 hand-picking tile borders.
 You are cutting a planet extract yourself, aren't you?
Yes, with Splitter in automatic mode (no areas.list)
   My problems are
 different, because I am relying on an area extract that is not
 rectangular and lacks information of the bounding polygon.
Ah, ok, I wasn't aware of that.
   You could be
 suffering from bad data. You know, I have been told that the sea areas
 on the Mapnik rendition are based on some fixed data and the
 natural=coastline lines are ignored.
That rings a bell, but I hadn't connected the dots...
   It is relatively easy to keep a
 small area free from mkgmap-reported issues. Usually there are fewer
 than 5 errors when I do a test build, and after fixing the issues, the
 next day I typically get an error-free map. For any bigger area, I guess
 you can never get a mkgmap-complaint-free map, not even if you had the
 resources to fix all the reported errors by the time the next day's
 extract is cloned off.
Indeed, I see lots of errors in Mkgmap output. I don't think a single 
person can fix them in time before the next planet dump (which probably 
will have new errors).
 I have been toying with the idea of hand-picking tile borders for
 Estonia as well, but I don't know if it is worth my trouble. I'd guess
 that the country extract (as opposed to an Europe or planet extract) is
 most useful for islands. Finland is an island in some sense: you can't
 cross the Russian border without a passport and visa, the land borders
 to Sweden and Norway in the north are very far away from most of the
 population, and you will have plenty of time to switch maps when
 crossing the sea borders. Sweden, Norway and Estonia are less of
 islands, because they have easily crossable land borders. The map would
 have to lump several countries together in order to be practical, and I
 don't want to start downloading and cutting the Europe extract.
Ok it dawns on me that Finland can be a special island case. But my 
ultimate wish (which many of you will probably share) is that it all can 
be done in hands-off automatic mode. Like I said before hand-picking 
tile-borders in a quickly changing OSM environment is a daunting task 
and I rather just not use functionality then make my life overly 
complex. :-) But anyway, displaying the sea in the maps would still be 
awesome.
 When it comes to incomplete multipolygons that are severed by the tile
 borders, I guess that mkgmap should just ignore incomplete polygons that
 are totally outside the bounding box. Maybe this has been implemented
 already, because I am seeing very few multipolygon warnings nowadays. I
 should check if there is anything wrong about the two multipolygons that
 mkgmap now complains about near the tile borders.

Would it be possible to close the polygons in Mkgmap? I realise such 
would not be easy, but maybe someone has an idea?

Thanks for your clarifications!

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Missing ways part 1

2010-11-02 Thread Lambertus

On 2010-11-02 09:25, Markus_g wrote:
 I agree it prob won't work world wide. I think I tried it on another country
 recently where it didn't work.

Well I'm not really sure but even --overlap=5000 seems to severely slow 
things down (although I'm not seeing the error messages this time).


 -Original Message-
 From: mkgmap-dev-boun...@lists.mkgmap.org.uk
 [mailto:mkgmap-dev-boun...@lists.mkgmap.org.uk] On Behalf Of Lambertus
 Sent: Tuesday, 2 November 2010 4:56 AM
 To: mkgmap-dev@lists.mkgmap.org.uk
 Subject: Re: [mkgmap-dev] Missing ways part 1

 Op 01-11-10 09:42, Lambertus schreef:
 A new update is running with --overlap=5 and
 Well, that proved to be not working. Splitter has been crunching all day
 on the initial split and is constantly printing out messages like this:

 Node 388947686 in too many areas. Already in areas 0x86665608, trying to
 add area 0x96

 This is the command line I used:

 java -Xmx6000m -ea -jar ~/garmin/utils/splitter/splitter.jar
 --overlap=5 --no-trim --max-areas=255 --cache=cache --mapid=6324000
 --max-nodes=120 etc.

 So this might work in specific areas but not worldwide.

 ___
 mkgmap-dev mailing list
 mkgmap-dev@lists.mkgmap.org.uk
 http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev


 ___
 mkgmap-dev mailing list
 mkgmap-dev@lists.mkgmap.org.uk
 http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

[mkgmap-dev] Feature request: accept osm data from stdin

2010-11-02 Thread Lambertus

Would it be possible that Mkgmap supports reading the OSM data from 
stdin? Splitter is capable of doing so and it would eliminate the need 
for a temporary file if Mkgmap could do so too.

E.g. --input-file=- or assume stdin if no filename is given?
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Feature request: accept osm data from stdin

2010-11-02 Thread Lambertus

On 2010-11-02 10:28, Marko Mäkelä wrote:
 On Tue, Nov 02, 2010 at 10:22:33AM +0100, Lambertus wrote:
 Would it be possible that Mkgmap supports reading the OSM data from
 stdin? Splitter is capable of doing so and it would eliminate the need
 for a temporary file if Mkgmap could do so too.

 If you are using a unix-like system, --input-file=/dev/stdin or
 /dev/fd/0 could be a feasible workaround. It generally works for
 programs that do not attempt to seek (or rewind) the input file.

Yes, I use Linux, so that might work then. Thanks!
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Missing ways part 1

2010-11-01 Thread Lambertus

Op 01-11-10 09:42, Lambertus schreef:
 A new update is running with --overlap=5 and
Well, that proved to be not working. Splitter has been crunching all day 
on the initial split and is constantly printing out messages like this:

Node 388947686 in too many areas. Already in areas 0x86665608, trying to 
add area 0x96

This is the command line I used:

java -Xmx6000m -ea -jar ~/garmin/utils/splitter/splitter.jar 
--overlap=5 --no-trim --max-areas=255 --cache=cache --mapid=6324000 
--max-nodes=120 etc.

So this might work in specific areas but not worldwide.

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Fuel POI label corrupted

2010-10-14 Thread Lambertus

On 2010-10-14 06:10, Charlie Ferrero wrote:
 Peter Hendricks wrote:
 Hi all,

 I would like to report what looks like a style sheet problem. I'm using
 Lambertus' Garmin map from http://garmin.na1400.info/routable.php. This
 node http://www.openstreetmap.org/browse/node/418611927 shows up on the
 Garmin map with label ptth: Ptt. The default style sheet is used, so I
 am told.

Yes, I'm using the default stylesheet.

  It looks like the POI is being labelled operator: name:en where
  operator is being transliterated from the Thai (?) script.  This is
  expected behaviour from mkgmap if Lambertus is using the default style
  combined with mkgmap's --name-tag-list option (i.e.
  --name-tag-list=name:en, name).
 
I'm using the follow Mkgmap option:
--name-tag-list=name:en,int_name,name:zh_py,name:engels,name
Where name:engels is the transliterated name.

I had no idea that the name-tag option interacts with the stylesheet. 
Does this mean that I need to have an adapted stylesheet because of the 
name-tag-list? Any other options?

 Unfortunately, as I understand it Lambertus is keen to stick with the
 default style.

Correct, it seems to me that maintaining a stylesheet takes quite some 
time, time which is a scarce commodity. Creating my own stylesheet is 
just very low on my priority list.

That said, if anyone knows of a good public *generic* stylesheet that is 
maintained or actively developed and is better then the default 
stylesheet then I will consider to switch.

I know there are some specialist stylesheets around (e.g. for MTB of 
cycling) but I'm not aware of a generic stylesheet other then the Mkgmap 
default.
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] New, faster splitter

2010-06-08 Thread Lambertus

Just to say: thanks WanMil and Chris!

(now I need to upgrade to a quadcore to benefit from all these goodies...)
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Africa Extract from Geofabrik using --route

2010-04-12 Thread Lambertus

I may be having the same problems with Dakar but I'm using the planet 
dump. I'll re-render Dakar this evening and post the results.

Felix Hartmann wrote:
 Can anyone find out why mkgmap is throwing this error since quite a few 
 time when one tries to compile the Africa extract from Geofabrik?
 This happens using both default style, as my own style-file.
 
 I don't think it's related to --max-nodes=30 as this is already 
 quite low.
 
 I'm using the following options:
 mkgmap.jar --max-jobs=3 %generate-sea% --reduce-point-density=5.4 
 --reduce-point-density-polygon=8 
 --suppress-dead-end-nodes--delete-tags-file=deletetags --transparent 
 --blacklist-tags-file=deletetags --adjust-turn-headings 
 --add-pois-to-areas --ignore-maxspeeds --ignore-turn-restrictions 
 --remove-short-arcs=4 --description=openmtbmap_%abr% --route 
 --country-abbr=%abr% --country-name=%country% --mapname=%FID% 
 --family-id=%FID% --product-id=1 --series-name=openmtbmap_%abr%_%date% 
 --family-name=mtbmap_%abr%_%date% --tdbfile --overview-mapname=mapset 
 --area-name=%country%_%date%_openmtbmap.org -c 
 d:\garmin\mkgmap_680\maps\template.%country%
 
 Complete output below:
 END DOCUMENT
 SCHWERWIEGEND (Polyline): 6536.osm.gz: Problem writing line (class 
 uk.me.parabola.imgfmt.app.trergn.Polyline) of type 0x1e containing 3 
 points and starting at 
 http://www.openstreetmap.org/?mlat=1.02299mlon=-7.53589z
 oom=17
 SCHWERWIEGEND (Polyline): 6536.osm.gz:   Subdivision shift is 0 and 
 its centre is at 
 http://www.openstreetmap.org/?mlat=2.59361mlon=-7.69180zoom=17
 SCHWERWIEGEND (Polyline): 6536.osm.gz:   deltaLat = -73196
 END DOCUMENT
 END DOCUMENT
 END DOCUMENT
 END DOCUMENT
 END DOCUMENT
 END DOCUMENT
 END DOCUMENT
 END DOCUMENT
 SCHWERWIEGEND (MapSplitter): 65360007.osm.gz: Area too small to split at 
 http://www.openstreetmap.org/?mlat=14.71812mlon=-17.46835zoom=17 
 (reduce the density of points, length of lines, etc.)
 SCHWERWIEGEND (MapSplitter): 65360007.osm.gz: Area too small to split at 
 http://www.openstreetmap.org/?mlat=14.72337mlon=-17.47039zoom=17 
 (reduce the density of points, length of lines, etc.)
 SCHWERWIEGEND (MapSplitter): 65360007.osm.gz: Area too small to split at 
 http://www.openstreetmap.org/?mlat=14.71812mlon=-17.46835zoom=17 
 (reduce the density of points, length of lines, etc.)
 SCHWERWIEGEND (MapSplitter): 65360007.osm.gz: Area too small to split at 
 http://www.openstreetmap.org/?mlat=14.72337mlon=-17.47039zoom=17 
 (reduce the density of points, length of lines, etc.)
 SCHWERWIEGEND (NOD1Part): 65360007.osm.gz: Region contains too many 
 nodes/arcs (discarding 1 nodes to be able to continue)
 SCHWERWIEGEND (NOD1Part): 65360007.osm.gz:   Expect the routing to be 
 broken near BBox[14,72290/-17,47047, 14,72292/-17,47045]
 SCHWERWIEGEND (NOD1Part): 65360007.osm.gz: Region contains too many 
 nodes/arcs (discarding 1 nodes to be able to continue)
 SCHWERWIEGEND (NOD1Part): 65360007.osm.gz:   Expect the routing to be 
 broken near BBox[14,72395/-17,47054, 14,72398/-17,47051]
 SCHWERWIEGEND (NOD1Part): 65360007.osm.gz: Region contains too many 
 nodes/arcs (discarding 1 nodes to be able to continue)
 SCHWERWIEGEND (NOD1Part): 65360007.osm.gz:   Expect the routing to be 
 broken near BBox[14,71773/-17,46873, 14,71775/-17,46871]
 SCHWERWIEGEND (NOD1Part): 65360007.osm.gz: Region contains too many 
 nodes/arcs (discarding 1 nodes to be able to continue)
 SCHWERWIEGEND (NOD1Part): 65360007.osm.gz:   Expect the routing to be 
 broken near BBox[14,71855/-17,46800, 14,71857/-17,46798]
 java.lang.AssertionError
  at uk.me.parabola.imgfmt.app.net.RoadIndex.write(RoadIndex.java:45)
  at 
 uk.me.parabola.imgfmt.app.net.RoadDef.writeLevelDivs(RoadDef.java:218)
  at 
 uk.me.parabola.imgfmt.app.net.RoadDef.writeNet1(RoadDef.java:142)
  at uk.me.parabola.imgfmt.app.net.NETFile.write(NETFile.java:58)
  at 
 uk.me.parabola.mkgmap.build.MapBuilder.makeMap(MapBuilder.java:222)
  at uk.me.parabola.mkgmap.main.MapMaker.makeMap(MapMaker.java:96)
  at uk.me.parabola.mkgmap.main.MapMaker.makeMap(MapMaker.java:61)
  at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:189)
  at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:186)
  at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
  at java.util.concurrent.FutureTask.run(Unknown Source)
  at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown 
 Source)
  at java.lang.Thread.run(Unknown Source)
 Exiting - if you want to carry on regardless, use the --keep-going option
 
 ___
 mkgmap-dev mailing list
 mkgmap-dev@lists.mkgmap.org.uk
 http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk

Re: [mkgmap-dev] Highway name or ref

2010-04-06 Thread Lambertus

Someone on the forum seems to have found the solution:

Quote by Vclaw:

  I noticed this same thing on some Garmin maps I was making. I think
  its caused by the highway shields, as the Garmin can't display those
  as well as the name at the same time. I fixed it by setting the
  display_name for the highways. eg a style rule like this:

  highway=primary {name '${ref|highway-symbol:box} ${name}' |
  '${ref|highway-symbol:box}' | '${name}'; add display_name = '$

It's aparently the second part of this patch of which only the first 
part has been committed: 
http://www.mkgmap.org.uk/pipermail/mkgmap-dev/2009q4/005056.html

Marko Mäkelä wrote:
 On Wed, Mar 31, 2010 at 09:52:57AM +0200, Lambertus wrote:
 There's a question on the OSM forum about the naming of highways in the 
 default stylesheet which comes down to this: if a ref is available then 
 that seems always used even if a name is also available, can this be 
 changed to show the name as well?

 http://forum.openstreetmap.org/viewtopic.php?pid=68439#p68439
 
 I can't repeat this as described on my Edge 705 and today's map
 from http://www.polkupyoraily.net/osm/.  I do see the mouseover label
 ref name, but if I hit Enter on it, the POI window displays either
 ref or ref name, i.e., 152 or 1521 Jokivarrentie.  I tried it here:
 
 http://www.openstreetmap.org/?lat=60.3470937lon=25.1346522zoom=18
 http://www.openstreetmap.org/browse/node/374125267
 
 The 1521 is a lesser-class road that lacks the highway shield.  It says
 1521 Jokivarrentie on the map without a shield.  The 152 (when I zoom in)
 displays the ref=152 in an oval shield and the name=Sipoontie below that.
 
 Also, if I select the motorway a few hundred meters west from that point,
 it says just 4 (the ref), but the mouseover tooltip says
 4 Lahdenväylä (E) [E=etelä=south].  When I zoom in, the map displays the
 ref in a box and the name below that.
 
 It would be nice if the POI window displayed the name in addition to the ref
 for roads that carry a highway shield.
 Could this be arranged with display_name?
 
 For some reason, QLandkarteGT is not showing me any street names or labels;
 only POI names.
 
 Best regards,
 
   Marko
 ___
 mkgmap-dev mailing list
 mkgmap-dev@lists.mkgmap.org.uk
 http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

[mkgmap-dev] Highway name or ref

2010-03-31 Thread Lambertus

There's a question on the OSM forum about the naming of highways in the 
default stylesheet which comes down to this: if a ref is available then 
that seems always used even if a name is also available, can this be 
changed to show the name as well?

http://forum.openstreetmap.org/viewtopic.php?pid=68439#p68439
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Maps crashing Mapsource in big cities

2010-02-12 Thread Lambertus

Carlos Dávila wrote:
 Mark Burton escribió:
 I don't understand why people make such big tiles, what's the benefit?
   
 I use as big tiles as I can to avoid inter tile routing problems. Is it
 justified?
 ___
 mkgmap-dev mailing list
 mkgmap-dev@lists.mkgmap.org.uk
 http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

The same problem about crashing MapSource and RoadTrip has also been 
reported for my maps (e.g. eastern Australia, north and south of 
Brisbane). The mapbuilding script tries to make those maps as large as 
possible. Perhaps related is that I  upped the max-nodes from 1.400.000 
to 1.600.000 for the current map version.

I'll reduce the max-nodes for the next update but, ideally, Mkgmap 
should complain and quit if there is too much information in the source 
data (This is certainly no complaint, ofcourse I know this all depends 
on the understanding of the Garmin format).
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Maps crashing Mapsource in big cities

2010-02-12 Thread Lambertus

Mark Burton wrote:
 OK - you tell us what the limit is and we'll make sure that mkgmap
 gripes if you bust the limit.
 
 Seriously, if you can provide any info as what works and what doesn't
 in terms of maps sizes, numbers of lines, polys, nodes, etc. that would
 be useful and we can code accordingly.
 
I don't know exactly if you were kinda mad at me writing this or if this 
is a language barrier thing, but please know that I have high respect 
your and others work on Mkgmap and Splitter. My writing certainly wasn't 
meant as critique. I'm truly sorry if it was felt that way.

That said, is there a tool that gives the statistics that you're looking 
for from an OSM extract? I would certainly try to provide more useful 
feedback if possible.
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Maps crashing Mapsource in big cities

2010-02-12 Thread Lambertus

Mark Burton wrote:
 Hi Lambertus,
 
 Mark Burton wrote:
 OK - you tell us what the limit is and we'll make sure that mkgmap
 gripes if you bust the limit.

 Seriously, if you can provide any info as what works and what doesn't
 in terms of maps sizes, numbers of lines, polys, nodes, etc. that would
 be useful and we can code accordingly.

 I don't know exactly if you were kinda mad at me writing this or if this 
 is a language barrier thing, but please know that I have high respect 
 your and others work on Mkgmap and Splitter. My writing certainly wasn't 
 meant as critique. I'm truly sorry if it was felt that way.
 
 Don't panic, I'm not mad at all - sorry if that's what you thought. The
 point I was trying to make was simply that we can't add warnings about
 size limits being busted until we know what the limits are and that
 info really needs to come from the people who are trying to make
 big/complex maps.
 
Great, I hoped so :-)

 That said, is there a tool that gives the statistics that you're looking 
 for from an OSM extract? I would certainly try to provide more useful 
 feedback if possible.
 
 I do not know of such a tool but what we could do is get mkgmap to
 report a bunch of statistics (number of nodes/ways/relations,
 subdivisions, routing table sizes, etc.) and that could be correlated
 against good and bad maps.
 
If that is thought to be useful then I can upload the statistics to the 
website for all tiles on each weekly update and do some extensive 
testing  when problems are reported. For that I'd apreciate an option to 
save the statistics to a file named something like {tilenumber}.stat or 
whatever.
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Bad routing error here in Rome...

2010-02-10 Thread Lambertus

I don't know if more information is still needed, but I've received some 
additional information:

Dear Lambertus,

I'm referring to the map in Colombo Sri Lanka,

I tried from several paces in Colombo roads, and selected, place as
destination to route calculation, but none of the time it worked,

Then just to double check, I selected, very close 2 places on the same road
as the start point and as the end point , but the same problem was there, in
my mobile Garmin, and also in my Garmin unit,

Eg:  2 places in Union place Road, Colombo ( when I'm standing on union
place Road, I selected another point on the same road as destination, but
same problem came up,

Hope that you can recognize my problem

Thank you

Diluk  

-Original Message-
From: Lambertus
Sent: Monday, February 08, 2010 3:10 PM
To: Diluk
Subject: Re: Your routable Garmin map request

Hello Diluk,

Good to see that your roads are showing up. Regarding the errors you 
get: can you provide detailed information on the routes that you have 
tried, so that I and others can reproduce them (so we can try to find 
the source of the problem)?

Thanks in advance.

Regards, Lambertus

Diluk wrote:
  

  Dear friend,
  
  Today morning I received the new map set on my email after few hours 
  from my request,
  
  All the roads, I added recently are showing in the new map,
  
  But when I try to navigate, it  says  ROUTE CALCULATION EROR
  
  Why is that, please help,
  
  Thank you,


The tile in question is: 
http://planetosm.oxilion.nl/~lambertus/garmin/routable/03-02-2010/63240212.img
Unfortunately I've deleted the OSM source file earlier (I needed the 
diskspace). Mkgmap version is r1557.

Lambertus wrote:
 Just fyi: I've also received questions about bad routing errors in 
 MapSource after my latest update as well. I'm asking for additional 
 information in order to make them reproducible. Mkgmap is latest 
 snapshot from last Thursday, splitter is latest (afaik r105). I'll 
 report back later.

 Marco Certelli wrote:
   
 Hello.

 I'll re-test the situation every week. Compiled italy.osm.bz2 from 
 download.geofabrik.de

 1) latest mkgmap + latest splitter: Routing Error in Mapsource and Nuvi 255

 2) mkgmap 1188 + oldest splitter (51kB): No Error in Mapsource and No error 
 in Nuvi 255

 Ciao, Marco.



 --- Dom 31/1/10, Marco Certelli marco_certe...@yahoo.it ha scritto:

 
 Da: Marco Certelli marco_certe...@yahoo.it
 Oggetto: Re: [mkgmap-dev] Bad routing error here in Rome...
 A: Development list for mkgmap mkgmap-dev@lists.mkgmap.org.uk
 Data: Domenica 31 gennaio 2010, 16:43
 Hi Mark.

 No way. I reduced maxnode to 50 (Italy is splitted in
 42 tiles)

 I tested latest splitter and latest mkgmap: exactly same
 error as before in mapsource routing.

 Ciao, Marco.


 --- Sab 30/1/10, Mark Burton ma...@ordern.com
 ha scritto:

   
 Da: Mark Burton ma...@ordern.com
 Oggetto: Re: [mkgmap-dev] Bad routing error here in
 
 Rome...
   
 A: mkgmap-dev@lists.mkgmap.org.uk
 Data: Sabato 30 gennaio 2010, 17:00

 Hi Marco,

 
 Here is the part of the cmd script doing the
   
 actual
   
 job:
 
 java -enableassertions -Xmx1000m -jar
   
 ..\bin\splitter.jar --mapid=66%FID%001
 
 --max-nodes=100
   
 ..\OSM-Data\%osmfile%
 
 java -enableassertions -Xmx1000m -jar
   
 ..\bin\mkgmap.jar --code-page=1252
 --country-name=%country% --latin1 --family-id=%FID%
 --mapname=66%FID%001 --overview-mapname=66%FID%000
 
 --tdbfile
   
 --series-name=OSM-%country%
 
 --family-name=OpenStreetMap:
   
 %country% --road-name-pois --add-pois-to-areas
 --no-poi-address --ignore-maxspeeds
 
 --remove-short-arcs
   
 --preserve-element-order
 --style-file=..\bin\resources\styles\ --style=%style%
 --description=%country% --route --net --gmapsupp -c
 template.args

 OK - thanks.

 Have you tried using smaller tile sizes? i.e. does the
 
 same
   
 problem
 exist if you use, say, 50 for max nodes?

 Also, does the map show any other problems in that
 
 area
   
 apart from
 routing not working?

 Mark
 ___
 mkgmap-dev mailing list
 mkgmap-dev@lists.mkgmap.org.uk
 http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

 

   
   
 ___
 mkgmap-dev mailing list
 mkgmap-dev@lists.mkgmap.org.uk
 http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
 

 ___
 mkgmap-dev mailing list
 mkgmap-dev@lists.mkgmap.org.uk
 http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
   

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Gmapibuilder: Searching for new site to host source code

2010-02-10 Thread Lambertus

If Gmapibuilder is not suitable for the Mkgmap SVN repository, then you 
could try the OSM SVN repository... I guess many of us already have an 
account for the OSM repository.

As a sidenote, I got a mail today about Gmapibuilder results claiming 
that the MapInstaller gives an error with my latest map release. Since I 
haven't changed gmapibuilder it might be something in Mkgmap (r1557) 
that triggers this. Is someone with access to a Mac able to check? 
Unfortunately I cannot find the email report with the exact description 
anymore... 8-? The gmapibuilder version I have is dated Nov 16, 2009 (I 
think Clinton mailed that one to me).


You could try Clinton Gladstone wrote:
 Hi All,

 I, and others, have noticed that the original hosting server for Gmapibuilder 
 appears to be no longer available. Attempts to contact the original author 
 have also failed.

 Fortunately the source is BSD licensed. I still have the source (which I 
 patched to convert .mdr and .mdx files as well), and would like to find a new 
 permanent host for it. Are there any disadvantages to using Google Source, or 
 is there a better place to host the code?

 (For those of you who do not know/remember Gmapibuilder converts maps created 
 by mkgmap to the Gmapi format used on Mac OS X.)

 Any advice is appreciated.

 Cheers.
 ___
 mkgmap-dev mailing list
 mkgmap-dev@lists.mkgmap.org.uk
 http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
   

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Gmapibuilder: Searching for new site to host source code

2010-02-10 Thread Lambertus

Ah sorry, it wasn't a mail afterall but a forum post... Well, here it 
is: http://forum.openstreetmap.org/viewtopic.php?pid=59086#p59086

Hi Lambertus,

Has anything been changed with the system that generates RoadTrip
maps? I downloaded a map and MapInstall gives an error: Alert.
There is a problem with the OSM World Routable installation. Please
re-install OSM World Routable and start MapInstall again

I tried re-installing the map to no avail. I reinstalled the
previous OSM routable map and it works fine with MapInstall.

Interestingly the problem map is installed to RaodTrip with
MapManager no problem. Its just MapInstall finds an error with the map.

Thanks

Lambertus wrote:
 If Gmapibuilder is not suitable for the Mkgmap SVN repository, then you 
 could try the OSM SVN repository... I guess many of us already have an 
 account for the OSM repository.

 As a sidenote, I got a mail today about Gmapibuilder results claiming 
 that the MapInstaller gives an error with my latest map release. Since I 
 haven't changed gmapibuilder it might be something in Mkgmap (r1557) 
 that triggers this. Is someone with access to a Mac able to check? 
 Unfortunately I cannot find the email report with the exact description 
 anymore... 8-? The gmapibuilder version I have is dated Nov 16, 2009 (I 
 think Clinton mailed that one to me).


 You could try Clinton Gladstone wrote:
   
 Hi All,

 I, and others, have noticed that the original hosting server for 
 Gmapibuilder appears to be no longer available. Attempts to contact the 
 original author have also failed.

 Fortunately the source is BSD licensed. I still have the source (which I 
 patched to convert .mdr and .mdx files as well), and would like to find a 
 new permanent host for it. Are there any disadvantages to using Google 
 Source, or is there a better place to host the code?

 (For those of you who do not know/remember Gmapibuilder converts maps 
 created by mkgmap to the Gmapi format used on Mac OS X.)

 Any advice is appreciated.

 Cheers.
 ___
 mkgmap-dev mailing list
 mkgmap-dev@lists.mkgmap.org.uk
 http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
   
 

 ___
 mkgmap-dev mailing list
 mkgmap-dev@lists.mkgmap.org.uk
 http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
   

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Making splitter and MultiPolygon code play together

2010-02-03 Thread Lambertus

Chris Miller wrote:
 I'm thinking the best thing to do is to make the cache compulsory (which 
 in turn would make --mixed redundant) and once the cache is generated and 
 all the multipolygons have been found, an additional pass can be made over 
 the ways cache file to determine which nodes fall in which multipolygons 
 and dealt with accordingly. Without a compulsory cache in place this would 
 be very expensive.
 
 The upside to a compulsory cache is that the code doesn't get too messy and 
 performance doesn't suffer much, plus there will likely be other benefits 
 in the future too. The downside is that a chunk of disk space will always 
 be required by the splitter for writing the cache.
 
 Does anyone have any objections to this? If not I'll take a look sometime 
 in the next few days. I'll also look at fixing the lack of support in the 
 splitter for relations containing other relations.
 

No objections here. Disk space due to cache isn't really a problem, even 
when processing the entire planet file.
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Mkgmap complains about '' in role id

2010-01-30 Thread Lambertus

Your assertion was correct. I've updated splitter and the the problem is 
gone. Thanks!

BTW, this is the first time I was able to render the whole planet 
without showstopping errors :D

Christoph Wagner wrote:
 Which Version of splitter do you use? The bug was fixed in version 103. Do 
 you use an older version?

   
 

 ___
 mkgmap-dev mailing list
 mkgmap-dev@lists.mkgmap.org.uk
 http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Mkgmap complains about '' in role id

2010-01-24 Thread Lambertus

I'm using a development version from Chris Miller that - I think - was 
relabeled to r103 later. But maybe this isn't the case. I'll download 
the latest version and test that. Thanks for the tip!

 Christoph Wagner wrote:
 Lambertus schrieb:
   
 Thanks for having a look at it.

 Well ofcourse it's a crappy mapped way and the busroute is mapped wrong, 
 but that should not make Mkgmap complain though ;-)

 Now that you've posted the XML from the API, it shows that this is 
 actually caused either by Splitter or Translit because the source osm 
 file used by Mkgmap shows the '' only (not 'amp;').

 I'll do some tests to see where the 'amp;' turns into '' in my 
 toolchain, but that'll have to wait until tomorrow evening.
 

 Which Version of splitter do you use? The bug was fixed in version 103. Do 
 you use an older version?

   
 

 ___
 mkgmap-dev mailing list
 mkgmap-dev@lists.mkgmap.org.uk
 http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

[mkgmap-dev] Mkgmap complains about '' in role id

2010-01-23 Thread Lambertus

There are a few tiles that fail to render due to 'Bad file format: 
63240029.osm.gz' on every update and I've tracked it down to Mkgmap 
complaining about a '' character in the role id of a way in a relation.

Example:
http://www.openstreetmap.org/browse/relation/359107

Can Mkgmap be made to accept this? Or is this an error in the planet 
dump procedure?

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Mkgmap complains about '' in role id

2010-01-23 Thread Lambertus

Thanks for having a look at it.

Well ofcourse it's a crappy mapped way and the busroute is mapped wrong,
but that should not make Mkgmap complain though ;-)

Now that you've posted the XML from the API, it shows that this is
actually caused either by Splitter or Translit because the source osm
file used by Mkgmap shows the '' only (not 'amp;').

I'll do some tests to see where the 'amp;' turns into '' in my
toolchain, but that'll have to wait until tomorrow evening.

Marko Mäkelä wrote:
Hi Lambertus,

On Sat, Jan 23, 2010 at 06:11:56PM +0100, Lambertus wrote:

There are a few tiles that fail to render due to 'Bad file format:
63240029.osm.gz' on every update and I've tracked it down to Mkgmap
complaining about a '' character in the role id of a way in a relation.

Example:
http://www.openstreetmap.org/browse/relation/359107

Can Mkgmap be made to accept this? Or is this an error in the planet
dump procedure?

It is obviously crappy map data, but mkgmap should not complain on
valid XML data structure anyway.

I started by downloading the following in JOSM:

http://www.openstreetmap.org/?lat=29.52352lon=-98.39367zoom=18

Then I downloaded the remaining relation members in JOSM's relation editor
and saved the result to a file. But mkgmap did not croak for me on this:

relation id='359107' timestamp='2010-01-22T11:39:42Z' uid='110263'
user='wern
er2101' visible='true' version='5' changeset='3682358'
member type='way' ref='45878370' role='632' /
member type='way' ref='40995493' role='630' /
member type='way' ref='33181726' role='17 amp;94' /
tag k='route' v='bus' /
tag k='type' v='route' /
/relation

This looks like well-formed XML to me. I double-checked by downloading
the URL reported by JOSM:
http://www.openstreetmap.org/api/0.6/map?bbox=-98.39435664550781,29.523176677246095,-98.39298335449219,29.523863322753908

I downloaded it both with Debian Iceweasel (aka Mozilla Firefox)
and wget. In both copies, it was correctly encoded as role='17 amp;94'.

Side note: The node that I downloaded is part of the way
http://www.openstreetmap.org/browse/way/45878370
which looks like it was directly converted from a GPS track,
not connected to other roads. The relation is utter nonsense too.
It seems to combine four different bus lines to one relation.

I suspect that your map splitter is at error: it does not encode
role='' values properly.

Marko
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] New (?) error message

2009-12-14 Thread Lambertus

I see this behavior as well with quite a few tiles. I work around this 
by using: ulimit -t 300 java -jar  for rendering. Then check if an 
image file is created afterwards. If none is found the tile is split in 
half and then rendered again. This usually resolves it.


Nop wrote:
 Hi!
 
 Can anybody tell me the meaning of this error?
 
 What's remarkable: It seems that error was triggered by an oversize OSM 
 file. I had 200 files that processed fine, then the oversize one 
 triggered the error. The message was output about 14 times and mkgmap 
 apparently went into an endless loop, I had to kill it after several 
 hours of CPU without any progress/output.
 
 bye
   Nop
 
 
 
 Am 13.12.2009 08:44, schrieb Nop:
 Hi!

 Running mkgmap r1404 on a large map, I have encountered the following
 error message that I have not seen before:

 SCHWERWIEGEND (MapSplitter): Area too small to split at
 http://www.openstreetmap.org/?lat=46.36971lon=13.83734zoom=17 (reduce
 the density of points, length of lines, etc.)

 Is this the same as the previous error too many objects for map file
 or something else?

 bye
  Nop

 ___
 mkgmap-dev mailing list
 mkgmap-dev@lists.mkgmap.org.uk
 http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

 
 ___
 mkgmap-dev mailing list
 mkgmap-dev@lists.mkgmap.org.uk
 http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] --index crashes on an otherwise successfully rendered image

2009-12-09 Thread Lambertus

Hi Steve,

The image I linked to on the server was created with an r1387 of Mkgmap, 
but I tested with r1404 as well. Same result.

The patch from Clinton's email from 2009-11-28 20:59 works, but Mkgmap 
now reports a lot of 'bad index' warnings which I reported back on 
2009-12-02 21:36. Maybe the report got lost in the noise, as I sent a 
few mails that day (about my Java JDK environment problems).


Steve Ratcliffe wrote:
 Hi Lambertus
 
 Mkgmap sometimes crashes with when trying to build a gmapsupp including
 index using an existing (prerenderd) image.

 Exception in thread main java.lang.IndexOutOfBoundsException: Index:
 32803, Size: 488
   at java.util.ArrayList.rangeCheck(ArrayList.java:571)
   at java.util.ArrayList.get(ArrayList.java:349)
 
 This happens when trying to make a gmapsupp with the following Garmin
 image:
 http://planetosm.oxilion.nl/~lambertus/garmin/routable/18-11-2009/63240210.img
 
 I don't know the Mkgmap verion used to render the source image, but it's
 pretty recent. If this version is needed then I'll look it up this evening.
 
 It was created with 1387.  I believe that the problem is one that was 
 fixed in 1391.  It is creating the map that needs to be redone.
 
 Although we should stop the --index failing when encountering a bad 
 file, it has been quite useful to fix all the bugs in writing the POI 
 information.
 
 ..Steve
 ___
 mkgmap-dev mailing list
 mkgmap-dev@lists.mkgmap.org.uk
 http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Compiling Mkgmap failes

Marko Mäkelä wrote:
 Can you invoke javac from the command line?  I do not have any Eclipse
 stuff installed on my Debian Lenny system.  I am using the sun-java6-jdk
 package (Version 6-12-1, http://packages.debian.org/lenny/sun-java6-jdk).

   
Yes, javac runs fine from the commandline. Can you clarify? I don't 
understand what you want me to do... If it is the version number you're 
looking for, then that's given in my first mail.
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Compiling Mkgmap failes

Valentijn Sessink wrote:
 Lambertus schreef:
   
 I'm having problems compiling Mkgmap, maybe someone here is able to help 
 
 [...]

 What does
 sudo update-alternatives --config java

 tell you?
   
There are 2 choices for the alternative java (providing /usr/bin/java).

  SelectionPath  Priority   Status

* 0/usr/bin/gij-4.4   1044  auto mode
  1/usr/bin/gij-4.4   1044  manual mode
  2/usr/lib/jvm/java-6-sun/jre/bin/java   63manual mode

Press enter to keep the current choice[*], or type selection number:

Which one should I choose? Number 2 I suspect?
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Compiling Mkgmap failes

Valentijn Sessink wrote:
 ... and you might want to check
 dpkg --status sun-java6-jdk|grep Version
 ... too: 6-17-0ubuntu1.8.04 is what should be your current version on
 hardy; as there is a java problem when you have only the package from
 hardy. You need hardy-proposed in your package list. (At least that is
 what I experienced).

   
  dpkg --status sun-java6-jdk|grep Version
Version: 6-15-1


My ubuntu version is 9.10 (Karmic Koala)
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] Compiling Mkgmap failes

Ok, I finally got the correct search phrase and a suggestion popped-up 
suggesting to install ecj. Well, who would have thought of that... =-O


So, now the compiling starts but prints a whole lot of warnings and one 
error then exits. Output is attached.


The single error is:

  [javac] 7. ERROR in 
/home/lambertus/garmin/test/mkgmap/src/uk/me/parabola/imgfmt/app/labelenc/DecodedText.ja

   [javac] va (at line 42)
   [javac] text = new String(ba, 0, ba.length, charSet);
   [javac]^
   [javac] The constructor String(byte[], int, int, Charset) is undefined

Any ideas?
ant dist
Buildfile: build.xml

prepare:

compile:
[javac] Compiling 317 source files to 
/home/lambertus/garmin/test/mkgmap/build/classes
[javac] --
[javac] 1. WARNING in 
/home/lambertus/garmin/test/mkgmap/src/uk/me/parabola/imgfmt/ExitException.java 
(at line 25)
[javac] public class ExitException extends RuntimeException {
[javac]  ^
[javac] The serializable class ExitException does not declare a static 
final serialVersionUID field of type long
[javac] --
[javac] --
[javac] 2. WARNING in 
/home/lambertus/garmin/test/mkgmap/src/uk/me/parabola/imgfmt/FileExistsException.java
[javac]  (at line 26)
[javac] public class FileExistsException extends IOException {
[javac]  ^^^
[javac] The serializable class FileExistsException does not declare a 
static final serialVersionUID field of type long
[javac] --
[javac] --
[javac] 3. WARNING in 
/home/lambertus/garmin/test/mkgmap/src/uk/me/parabola/imgfmt/FileNotWritableException.java
 (at line 25)
[javac] public class FileNotWritableException extends IOException {
[javac]  
[javac] The serializable class FileNotWritableException does not declare a 
static final serialVersionUID field of type long
[javac] --
[javac] --
[javac] 4. WARNING in 
/home/lambertus/garmin/test/mkgmap/src/uk/me/parabola/imgfmt/FormatException.java
[javac]  (at line 24)
[javac] public class FormatException extends RuntimeException {
[javac]  ^^^
[javac] The serializable class FormatException does not declare a static 
final serialVersionUID field of type long
[javac] --
[javac] --
[javac] 5. WARNING in 
/home/lambertus/garmin/test/mkgmap/src/uk/me/parabola/imgfmt/ReadFailedException.java
 (at line 22)
[javac] public class ReadFailedException extends RuntimeException {
[javac]  ^^^
[javac] The serializable class ReadFailedException does not declare a 
static final serialVersionUID field of type long
[javac] --
[javac] --
[javac] 6. WARNING in 
/home/lambertus/garmin/test/mkgmap/src/uk/me/parabola/imgfmt/app/Exit.java
[javac]  (at line 37)
[javac] private Label label;
[javac]   ^
[javac] The field Exit.label is never read locally
[javac] --
[javac] --
[javac] 7. ERROR in 
/home/lambertus/garmin/test/mkgmap/src/uk/me/parabola/imgfmt/app/labelenc/DecodedText.ja
[javac] va (at line 42)
[javac] text = new String(ba, 0, ba.length, charSet);
[javac]^
[javac] The constructor String(byte[], int, int, Charset) is undefined
[javac] --
[javac] --
[javac] 8. WARNING in 
/home/lambertus/garmin/test/mkgmap/src/uk/me/parabola/imgfmt/app/lbl/ExitFacility.java
 (at line 21)
[javac] import uk.me.parabola.imgfmt.app.trergn.Subdivision;
[javac]
[javac] The import uk.me.parabola.imgfmt.app.trergn.Subdivision is never 
used
[javac] --
[javac] --
[javac] 9. WARNING in 
/home/lambertus/garmin/test/mkgmap/src/uk/me/parabola/imgfmt/app/lbl/Highway.java
[javac]  (at line 33)
[javac] class ExitPoint implements Comparable {
[javac]^^
[javac] Comparable is a raw type. References to generic type ComparableT 
should be parameterized
[javac] --
[javac] 10. WARNING in 
/home/lambertus/garmin/test/mkgmap/src/uk/me/parabola/imgfmt/app/lbl/Highway.java
 (at line 67)
[javac] java.util.Collections.sort(exits);
[javac] ^
[javac] Type safety: Unchecked invocation sort(ListHighway.ExitPoint) of 
the generic method sort(ListT) of type Collections
[javac] --
[javac] --
[javac] 11. WARNING in 
/home/lambertus/garmin/test/mkgmap/src/uk/me/parabola/imgfmt/app/lbl/LBLFileReader.ja
[javac] va (at line 302)
[javac] boolean hasTides = false;
[javac] 
[javac] The local variable

Re: [mkgmap-dev] Compiling Mkgmap failes

Valentijn Sessink wrote:
 Lambertus,

   
 * 0/usr/bin/gij-4.4   1044  auto mode
   1/usr/bin/gij-4.4   1044  manual mode
 Which one should I choose? Number 2 I suspect?
 

 Yep. Numbers 0 and 1 are the Gnu Java runtime. You probably have the Gnu
 java compiler as well.

 You did not completely remove all java before installing sun's java.
 Please remove gij-4.4 and related: probably gcj-4.4-jdk, gcj,
 gcj-4.4-jre and/or other gcj-related stuff. And you might as well check
 update-alternatives --config javac (which should be the Sun version as
 well).
   
Thanks a lot Valentijn, removing libgcj10 and it's dependent packages 
solved all compiling problems for me.
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

Re: [mkgmap-dev] --index crashes on an otherwise successfully rendered image