Re: [mkgmap-dev] possible bug? mkgmap freezes with lines only routing style
> i'm trying to create a lines-only routable map. from a certain amount of > lines in the style mkgmap starts turning out 2 empty img files (empty > means 0 bytes, not empty maps), then memory usage freezes pretty much, > cpu usage oscillates wildly and nothing more happens. > > this problem is tied to --route, leaving the option out turns things > back to normal. > > a certain amount of poi's in the style solve the problem too - with some > lines for cities in the points file everything works as expected. it > almost feels as if mkgmap "needs something to do" between all the roads. > > any ideas? Can you please send the problematic style and your mkgmap parameters so that we can try to reproduce the problem? Thanks WanMil ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] possible bug? mkgmap freezes with lines only routing style
here: www.geocaching-dresden.de/mkgmap/style.zip i found a workaround just now: the style contains a file called "lines workaround", which does the naming of the roads in a separate step - no freezing. micha Am 20.10.2011 17:58, schrieb WanMil: i'm trying to create a lines-only routable map. from a certain amount of lines in the style mkgmap starts turning out 2 empty img files (empty means 0 bytes, not empty maps), then memory usage freezes pretty much, cpu usage oscillates wildly and nothing more happens. this problem is tied to --route, leaving the option out turns things back to normal. a certain amount of poi's in the style solve the problem too - with some lines for cities in the points file everything works as expected. it almost feels as if mkgmap "needs something to do" between all the roads. any ideas? Can you please send the problematic style and your mkgmap parameters so that we can try to reproduce the problem? Thanks WanMil ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] possible bug? mkgmap freezes with lines only routing style
I can confirm that behaviour. Although in real mkgmap does not freeze (endless loop) but is working very very hard. Possibly the mkgmap code is not very optimal for the special test case. @Steve, the code requires a very long time in the NETFile.simplifySortedRoads method because I think there are many roads with the same name. I think I remember a similar discussion some time ago? The method seems to perform a brute force method to detect connected roads with the same name. Can you have a look on it? WanMil > here: www.geocaching-dresden.de/mkgmap/style.zip > > i found a workaround just now: the style contains a file called "lines > workaround", which does the naming of the roads in a separate step - no > freezing. > > micha > > > Am 20.10.2011 17:58, schrieb WanMil: >>> i'm trying to create a lines-only routable map. from a certain amount of >>> lines in the style mkgmap starts turning out 2 empty img files (empty >>> means 0 bytes, not empty maps), then memory usage freezes pretty much, >>> cpu usage oscillates wildly and nothing more happens. >>> >>> this problem is tied to --route, leaving the option out turns things >>> back to normal. >>> >>> a certain amount of poi's in the style solve the problem too - with some >>> lines for cities in the points file everything works as expected. it >>> almost feels as if mkgmap "needs something to do" between all the roads. >>> >>> any ideas? >> Can you please send the problematic style and your mkgmap parameters so >> that we can try to reproduce the problem? >> >> Thanks >> WanMil >> ___ >> mkgmap-dev mailing list >> mkgmap-dev@lists.mkgmap.org.uk >> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev >> > > > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] possible bug? mkgmap freezes with lines only routing style
On 20/10/11 21:36, WanMil wrote: > I can confirm that behaviour. Although in real mkgmap does not freeze > (endless loop) but is working very very hard. Possibly the mkgmap code > is not very optimal for the special test case. > > @Steve, the code requires a very long time in the > NETFile.simplifySortedRoads method because I think there are many roads > with the same name. I think I remember a similar discussion some time > ago? The method seems to perform a brute force method to detect > connected roads with the same name. Can you have a look on it? Yes that routine is a disaster when there are many roads with the same name. I'm not sure what it is supposed to do and how important it is. I'll have a go at fixing it. ..Steve ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] possible bug? mkgmap freezes with lines only routing style
Hi
I would like some opinions please from those that make maps.
If you have a style that say sets the name of every path to 'Path'
eg: highway=path | highway=footway | highway=track {name 'Path' }
or anything similar that results in thousands of roads with the same
name, then mkgmap appears to freeze.
The code that is slow is creating a list of roads that will be used
when you search for an address in MapSource or on the device.
I can fix this in various ways, but I want to ask how useful it is to
have thousands of roads with the same name appear as a result of a
search?
For a start in mapsource, you only see a few of the results anyway.
Should I drop the name completely from the index when there are more
than say 500 roads with the same name in the same city?
There is a branch called simplify-sorted-roads where I am trying out
different approaches to the problem, if anyone else wants to try it out
or look at it.
> I can confirm that behaviour. Although in real mkgmap does not freeze
> (endless loop) but is working very very hard. Possibly the mkgmap code
> is not very optimal for the special test case.
>
> @Steve, the code requires a very long time in the
> NETFile.simplifySortedRoads method because I think there are many roads
> with the same name. I think I remember a similar discussion some time
> ago? The method seems to perform a brute force method to detect
> connected roads with the same name. Can you have a look on it?
..Steve
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] possible bug? mkgmap freezes with lines only routing style
Am 29.10.2011 15:57, schrieb Steve Ratcliffe:
Hi
I would like some opinions please from those that make maps.
If you have a style that say sets the name of every path to 'Path'
eg: highway=path | highway=footway | highway=track {name 'Path' }
or anything similar that results in thousands of roads with the same
name, then mkgmap appears to freeze.
The code that is slow is creating a list of roads that will be used
when you search for an address in MapSource or on the device.
I can fix this in various ways, but I want to ask how useful it is to
have thousands of roads with the same name appear as a result of a
search?
For a start in mapsource, you only see a few of the results anyway.
Should I drop the name completely from the index when there are more
than say 500 roads with the same name in the same city?
There is a branch called simplify-sorted-roads where I am trying out
different approaches to the problem, if anyone else wants to try it out
or look at it.
I think it would be better to control it via style-file. Eg. you should
add a tagg like mkgmap:nomerge=yes for such ways.
Henning
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] possible bug? mkgmap freezes with lines only routing style
Hi,
On Sat, Oct 29, Steve Ratcliffe wrote:
> If you have a style that say sets the name of every path to 'Path'
>
>eg: highway=path | highway=footway | highway=track {name 'Path' }
>
> or anything similar that results in thousands of roads with the same
> name, then mkgmap appears to freeze.
>
> The code that is slow is creating a list of roads that will be used
> when you search for an address in MapSource or on the device.
>
> I can fix this in various ways, but I want to ask how useful it is to
> have thousands of roads with the same name appear as a result of a
> search?
In all cases where I set a name for a highway it is because the way
does not have a name. So I set something like "unpaved road" or
something similar. Which means, no, it doesn't make any sense to
have this as a search result. "upnaved road" will not help anybody
at all to find a street.
> Should I drop the name completely from the index when there are more
> than say 500 roads with the same name in the same city?
I think that's a good idea. But I have no experience what a good
limit is.
Thorsten
--
Thorsten Kukuk, Project Manager/Release Manager SLES
SUSE LINUX Products GmbH, Maxfeldstr. 5, D-90409 Nuernberg
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg)
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] possible bug? mkgmap freezes with lines only routing style
it would make a lot of sense if such names do not find their way into
the index. to control that behaviour flagging those ways via the style
file feels like the most flexible method.
Am 29.10.2011 15:57, schrieb Steve Ratcliffe:
Hi
I would like some opinions please from those that make maps.
If you have a style that say sets the name of every path to 'Path'
eg: highway=path | highway=footway | highway=track {name 'Path' }
or anything similar that results in thousands of roads with the same
name, then mkgmap appears to freeze.
The code that is slow is creating a list of roads that will be used
when you search for an address in MapSource or on the device.
I can fix this in various ways, but I want to ask how useful it is to
have thousands of roads with the same name appear as a result of a
search?
For a start in mapsource, you only see a few of the results anyway.
Should I drop the name completely from the index when there are more
than say 500 roads with the same name in the same city?
There is a branch called simplify-sorted-roads where I am trying out
different approaches to the problem, if anyone else wants to try it out
or look at it.
I can confirm that behaviour. Although in real mkgmap does not freeze
(endless loop) but is working very very hard. Possibly the mkgmap code
is not very optimal for the special test case.
@Steve, the code requires a very long time in the
NETFile.simplifySortedRoads method because I think there are many roads
with the same name. I think I remember a similar discussion some time
ago? The method seems to perform a brute force method to detect
connected roads with the same name. Can you have a look on it?
..Steve
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] possible bug? mkgmap freezes with lines only routing style
I think it's a good idea to use a flag like 'mkgmap:noindex=true' which
can be set in the style file. That provides the best flexibility.
WanMil
>
> Hi
>
> I would like some opinions please from those that make maps.
>
> If you have a style that say sets the name of every path to 'Path'
>
> eg: highway=path | highway=footway | highway=track {name 'Path' }
>
> or anything similar that results in thousands of roads with the same
> name, then mkgmap appears to freeze.
>
> The code that is slow is creating a list of roads that will be used
> when you search for an address in MapSource or on the device.
>
> I can fix this in various ways, but I want to ask how useful it is to
> have thousands of roads with the same name appear as a result of a
> search?
>
> For a start in mapsource, you only see a few of the results anyway.
>
> Should I drop the name completely from the index when there are more
> than say 500 roads with the same name in the same city?
>
> There is a branch called simplify-sorted-roads where I am trying out
> different approaches to the problem, if anyone else wants to try it out
> or look at it.
>
>
>> I can confirm that behaviour. Although in real mkgmap does not freeze
>> (endless loop) but is working very very hard. Possibly the mkgmap code
>> is not very optimal for the special test case.
>>
>> @Steve, the code requires a very long time in the
>> NETFile.simplifySortedRoads method because I think there are many roads
>> with the same name. I think I remember a similar discussion some time
>> ago? The method seems to perform a brute force method to detect
>> connected roads with the same name. Can you have a look on it?
>
> ..Steve
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] possible bug? mkgmap freezes with lines only routing style
Yes that is probably best.
I set names pretty extensively, and even I would only need about 80
additional lines of code in my lines style with mkgmap:noindex=true (as
I have to duplicate all lines where names are set to have one with
name!=* set mkgmap:nopreviousname=yes and then check that condition on
any future setting of names, the other option of doing a (highway=* &
name!=* & ref!=* & route_name!=*) set mkgmap:nopreviousname=yes wouldn't
mean less lines of new code).
On 31.10.2011 11:40, WanMil wrote:
> I think it's a good idea to use a flag like 'mkgmap:noindex=true' which
> can be set in the style file. That provides the best flexibility.
>
> WanMil
>
>> Hi
>>
>> I would like some opinions please from those that make maps.
>>
>> If you have a style that say sets the name of every path to 'Path'
>>
>> eg: highway=path | highway=footway | highway=track {name 'Path' }
>>
>> or anything similar that results in thousands of roads with the same
>> name, then mkgmap appears to freeze.
>>
>> The code that is slow is creating a list of roads that will be used
>> when you search for an address in MapSource or on the device.
>>
>> I can fix this in various ways, but I want to ask how useful it is to
>> have thousands of roads with the same name appear as a result of a
>> search?
>>
>> For a start in mapsource, you only see a few of the results anyway.
>>
>> Should I drop the name completely from the index when there are more
>> than say 500 roads with the same name in the same city?
>>
>> There is a branch called simplify-sorted-roads where I am trying out
>> different approaches to the problem, if anyone else wants to try it out
>> or look at it.
>>
>>
>>> I can confirm that behaviour. Although in real mkgmap does not freeze
>>> (endless loop) but is working very very hard. Possibly the mkgmap code
>>> is not very optimal for the special test case.
>>>
>>> @Steve, the code requires a very long time in the
>>> NETFile.simplifySortedRoads method because I think there are many roads
>>> with the same name. I think I remember a similar discussion some time
>>> ago? The method seems to perform a brute force method to detect
>>> connected roads with the same name. Can you have a look on it?
>> ..Steve
>> ___
>> mkgmap-dev mailing list
>> mkgmap-dev@lists.mkgmap.org.uk
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> ___
> mkgmap-dev mailing list
> mkgmap-dev@lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] possible bug? mkgmap freezes with lines only routing style
Perhaps if the name is set to a constant by a rule, that should default mkgmap:no-index to true. pgpFoxIITDr9v.pgp Description: PGP signature ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] possible bug? mkgmap freezes with lines only routing style
maybe it would be a good idea to have a command line switch *and* a
style rule.
the command line switch would have to default to "no-index=false",
otherwise indexing in all older styles would break. a 2nd optinon
"no-index=true" could automatically add a no-index tag to all lines
which then can be set to false in the style whenever we want a line indexed.
Am 29.10.2011 15:57, schrieb Steve Ratcliffe:
Hi
I would like some opinions please from those that make maps.
If you have a style that say sets the name of every path to 'Path'
eg: highway=path | highway=footway | highway=track {name 'Path' }
or anything similar that results in thousands of roads with the same
name, then mkgmap appears to freeze.
The code that is slow is creating a list of roads that will be used
when you search for an address in MapSource or on the device.
I can fix this in various ways, but I want to ask how useful it is to
have thousands of roads with the same name appear as a result of a
search?
For a start in mapsource, you only see a few of the results anyway.
Should I drop the name completely from the index when there are more
than say 500 roads with the same name in the same city?
There is a branch called simplify-sorted-roads where I am trying out
different approaches to the problem, if anyone else wants to try it out
or look at it.
I can confirm that behaviour. Although in real mkgmap does not freeze
(endless loop) but is working very very hard. Possibly the mkgmap code
is not very optimal for the special test case.
@Steve, the code requires a very long time in the
NETFile.simplifySortedRoads method because I think there are many roads
with the same name. I think I remember a similar discussion some time
ago? The method seems to perform a brute force method to detect
connected roads with the same name. Can you have a look on it?
..Steve
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] possible bug? mkgmap freezes with lines only routing style
On 31/10/11 10:40, WanMil wrote: > I think it's a good idea to use a flag like 'mkgmap:noindex=true' which > can be set in the style file. That provides the best flexibility. The option appears a popular choice! It might be a good addition, although I think it would have to work with all element types and not just roads. So I will take it that there is no-one in favour of just removing often repeated names automatically, and really as long as they don't cause any harm I don't see any problem with them. So I will merge later the current simplify-sorted-roads branch which now only uses the modified algorithm when there is a large number of same named roads. It never takes more than a fraction of a second. ..Steve ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] possible bug? mkgmap freezes with lines only routing style
Actually I just thought about one possible problem, as I don't know how the mkgmap:noindex=true will be implemented. What happens when a user simply rebuilds the address index of existing .img files without using a style-file when mkgmap:noindex=true had been set for problematic cases? a) will it then slow down if many roads have the same name b) doesn't matter, the names are now in such a position that mkgmap will not use them for address index? If a) is the case, then I think there would also need to be a switch to enable say 300 roads same name, don't index. If b) is the case then the mkgmap:noindex=true setup is sufficient. On 10.11.2011 13:25, Steve Ratcliffe wrote: > On 31/10/11 10:40, WanMil wrote: >> I think it's a good idea to use a flag like 'mkgmap:noindex=true' which >> can be set in the style file. That provides the best flexibility. > The option appears a popular choice! > It might be a good addition, although I think it would have > to work with all element types and not just roads. > > So I will take it that there is no-one in favour of just removing > often repeated names automatically, and really as long as they > don't cause any harm I don't see any problem with them. > > So I will merge later the current simplify-sorted-roads branch which > now only uses the modified algorithm when there is a large number of > same named roads. It never takes more than a fraction of a second. > > ..Steve > > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] possible bug? mkgmap freezes with lines only routing style
Hi > What happens when a user simply rebuilds the address index of existing > .img files without using a style-file when mkgmap:noindex=true had been > set for problematic cases? > > a) will it then slow down if many roads have the same name > b) doesn't matter, the names are now in such a position that mkgmap will > not use them for address index? It doesn't make much difference for the global index. There is only ever one copy of a name for a map/city/region/country combination in the global index. It is only the individual map tiles that contain an entry for each road with the same name. The global index only says that a particular map has at least one road with a given name. If Mapsource or a device crashes because of the large number of names, then we will just have to limit the number, it would not be optional. ..Steve > ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] possible bug? mkgmap freezes with lines only routing style
On 11.11.2011 11:34, Steve Ratcliffe wrote:
>
> Hi
>
>> What happens when a user simply rebuilds the address index of existing
>> .img files without using a style-file when mkgmap:noindex=true had been
>> set for problematic cases?
>>
>> a) will it then slow down if many roads have the same name
>> b) doesn't matter, the names are now in such a position that mkgmap will
>> not use them for address index?
>
> It doesn't make much difference for the global index. There is only
> ever one copy of a name for a map/city/region/country combination
> in the global index. It is only the individual map tiles that contain an
> entry for each road with the same name. The global index only says
> that a particular map has at least one road with a given name.
>
> If Mapsource or a device crashes because of the large number of names,
> then we will just have to limit the number, it would not be optional.
>
> ..Steve
>>
>
I think you did not understand what I meant.
1. I compile maps using a style-file that does highway=track & name!=*
{set name 'track'; mkgmap:noindex=true}
2. A user/me wants to add contourlines to the map and uses mkgmap to
recompile the maps including the contourlines into a new mapset.
tdb/mdr.img needs to be written from scratch, however the user uses the
default style-file that does not have mkgmap:noindex=true information
for highway=track.
So I don't want that the index in case 1. is correct (omits those
thousands of roads with name "track", but for step 2. the index is
incorrect and actually has thousands of "track" roads in the index. If
this is how it works then we would need (additionally to
mkgmap:noindex=true) an option like mkgmap:noindex:same=XXX for the
command line.
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] possible bug? mkgmap freezes with lines only routing style
Hi > I think you did not understand what I meant. Maybe, that's the problem with talking about options that don't exist :) There are two indexes that we might be talking about. 1. A road index in the .img. If there are 2000 "Path" roads then there will be 2000 entries "Path" in this index. 2. The global index (mdr.img). There will be only one entry in this index for those 2000 "Path" roads. The mdr road index is created by reading the road index from .img road index. The style has no effect on img->mdr creation, but only in the osm->img conversion. The only way I can imagine an option mkgmap:no-index in the style can work is to prevent road being placed into the .img road index. It therefore will not be in the mdr road index either, and that will be true even if the mdr index is created later without using any style options with that same .img tile. > So I don't want that the index in case 1. is correct (omits those > thousands of roads with name "track", but for step 2. the index is > incorrect and actually has thousands of "track" roads in the index. If > this is how it works then we would need (additionally to So as far as I can tell, what you are asking will never happen in any case, even now without the no-index option. There will never be thousands of Path entries in the index, there will be one per map/city, no matter if there are 5 roads called "Path" or 2000 in the same map/city. > mkgmap:noindex=true) an option like mkgmap:noindex:same=XXX for the > command line. You could have a command line option to exclude some names from the global index that are really in the tiles, but I don't really see the point. ..Steve ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] possible bug? mkgmap freezes with lines only routing style
thanks for clearing that up. I didn't know enough about the indexes. So style-file command is enough for anything that I can imagine. ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
