Re: [mkgmap-dev] possible bug? mkgmap freezes with lines only routing style
Hi
I would like some opinions please from those that make maps.
If you have a style that say sets the name of every path to 'Path'
eg: highway=path | highway=footway | highway=track {name 'Path' }
or anything similar that results in thousands of roads with the same
name, then mkgmap appears to freeze.
The code that is slow is creating a list of roads that will be used
when you search for an address in MapSource or on the device.
I can fix this in various ways, but I want to ask how useful it is to
have thousands of roads with the same name appear as a result of a
search?
For a start in mapsource, you only see a few of the results anyway.
Should I drop the name completely from the index when there are more
than say 500 roads with the same name in the same city?
There is a branch called simplify-sorted-roads where I am trying out
different approaches to the problem, if anyone else wants to try it out
or look at it.
I can confirm that behaviour. Although in real mkgmap does not freeze
(endless loop) but is working very very hard. Possibly the mkgmap code
is not very optimal for the special test case.
@Steve, the code requires a very long time in the
NETFile.simplifySortedRoads method because I think there are many roads
with the same name. I think I remember a similar discussion some time
ago? The method seems to perform a brute force method to detect
connected roads with the same name. Can you have a look on it?
..Steve
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] possible bug? mkgmap freezes with lines only routing style
Am 29.10.2011 15:57, schrieb Steve Ratcliffe:
Hi
I would like some opinions please from those that make maps.
If you have a style that say sets the name of every path to 'Path'
eg: highway=path | highway=footway | highway=track {name 'Path' }
or anything similar that results in thousands of roads with the same
name, then mkgmap appears to freeze.
The code that is slow is creating a list of roads that will be used
when you search for an address in MapSource or on the device.
I can fix this in various ways, but I want to ask how useful it is to
have thousands of roads with the same name appear as a result of a
search?
For a start in mapsource, you only see a few of the results anyway.
Should I drop the name completely from the index when there are more
than say 500 roads with the same name in the same city?
There is a branch called simplify-sorted-roads where I am trying out
different approaches to the problem, if anyone else wants to try it out
or look at it.
I think it would be better to control it via style-file. Eg. you should
add a tagg like mkgmap:nomerge=yes for such ways.
Henning
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
[mkgmap-dev] mkgmap ignores --country-abbr?
Hi WanMil, In the Geofabrik finland.osm.pbf there are a few Russian place=*. Some of these carry addr:country=RUS. I am invoking mkgmap (among others) with the following options: --country-abbr=FIN --country-name=Finland --location-autofill=nearest I am not using any bounds information. You can see the full invocation in http://www.polkupyoraily.net/osm/files/osm2img.sh In the tile for southeast Finland (63240007.osm.pbf), many if not all places are now displaying as RUS instead of FIN. I added addr:country=FI to Rääkkylä, and it now correctly shows FIN. Before that change, all place=* in that tile had either no addr:country or had addr:country=RU. Is it so that --location-autofill=nearest somehow overrides --country-abbr and --country-name? I would expect all place=* that lack addr:country to get an implicit addr:country=FI when I specify --country-abbr=FIN. In that way, the worst thing that would happen is that my map of Finland would have some Russian places (which are located very close to the border and are lacking addr:country=RU) incorrectly displaying as FIN instead of RUS. Best regards, Marko ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] possible bug? mkgmap freezes with lines only routing style
Hi,
On Sat, Oct 29, Steve Ratcliffe wrote:
If you have a style that say sets the name of every path to 'Path'
eg: highway=path | highway=footway | highway=track {name 'Path' }
or anything similar that results in thousands of roads with the same
name, then mkgmap appears to freeze.
The code that is slow is creating a list of roads that will be used
when you search for an address in MapSource or on the device.
I can fix this in various ways, but I want to ask how useful it is to
have thousands of roads with the same name appear as a result of a
search?
In all cases where I set a name for a highway it is because the way
does not have a name. So I set something like unpaved road or
something similar. Which means, no, it doesn't make any sense to
have this as a search result. upnaved road will not help anybody
at all to find a street.
Should I drop the name completely from the index when there are more
than say 500 roads with the same name in the same city?
I think that's a good idea. But I have no experience what a good
limit is.
Thorsten
--
Thorsten Kukuk, Project Manager/Release Manager SLES
SUSE LINUX Products GmbH, Maxfeldstr. 5, D-90409 Nuernberg
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg)
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] mkgmap ignores --country-abbr?
Hi WanMil,
In the Geofabrik finland.osm.pbf there are a few Russian place=*. Some
of these carry addr:country=RUS.
I am invoking mkgmap (among others) with the following options:
--country-abbr=FIN --country-name=Finland --location-autofill=nearest
I am not using any bounds information. You can see the full invocation
in http://www.polkupyoraily.net/osm/files/osm2img.sh
In the tile for southeast Finland (63240007.osm.pbf), many if not all
places are now displaying as RUS instead of FIN. I added addr:country=FI
to Rääkkylä, and it now correctly shows FIN. Before that change, all
place=* in that tile had either no addr:country or had addr:country=RU.
Is it so that --location-autofill=nearest somehow overrides
--country-abbr and --country-name? I would expect all place=* that lack
addr:country to get an implicit addr:country=FI when I specify
--country-abbr=FIN. In that way, the worst thing that would happen is
that my map of Finland would have some Russian places (which are located
very close to the border and are lacking addr:country=RU) incorrectly
displaying as FIN instead of RUS.
Best regards,
Marko
Hi Marko,
the following happens:
1. the style rules are applied, so all elements (places in this case)
with addr:country set are applied with the country information.
2. the nearest search starts which means:
2.1 all places with complete country/region information are put to a
known places list
2.2 for all places with incomplete country/region information the
nearest place from the known places list is retrieved and its
information is copied.
The nearest search does not have a max distance. So if one place in the
tile is tagged with addr:country=RU and no other country information is
provided all places in the tile will be tagged with country RU. Maybe it
would make sense to define such a maximum distance?
While looking into the source code to write how it works I've probably
found a problem so that some places are not put to the known places list.
Could you try it if it improves the behaviour for you?
WanMil
Index: src/uk/me/parabola/mkgmap/build/Locator.java
===
--- src/uk/me/parabola/mkgmap/build/Locator.java (revision 2063)
+++ src/uk/me/parabola/mkgmap/build/Locator.java (working copy)
@@ -63,6 +63,11 @@
if(p.getCountry() != null)
p.setCountry(normalizeCountry(p.getCountry()));
+ if(p.getCountry() != null p.getRegion() != null p.getCity() == null)
+ {
+ p.setCity(p.getName());
+ }
+
resolveIsInInfo(p); // Pre-process the is_in field
if(p.getCity() != null)
@@ -172,12 +177,6 @@
return;
}
- if(p.getCountry() != null p.getRegion() != null p.getCity() == null)
- {
- p.setCity(p.getName());
- return;
- }
-
if(p.getIsIn() != null)
{
String[] cityList = p.getIsIn().split(,);
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] possible bug? mkgmap freezes with lines only routing style
it would make a lot of sense if such names do not find their way into
the index. to control that behaviour flagging those ways via the style
file feels like the most flexible method.
Am 29.10.2011 15:57, schrieb Steve Ratcliffe:
Hi
I would like some opinions please from those that make maps.
If you have a style that say sets the name of every path to 'Path'
eg: highway=path | highway=footway | highway=track {name 'Path' }
or anything similar that results in thousands of roads with the same
name, then mkgmap appears to freeze.
The code that is slow is creating a list of roads that will be used
when you search for an address in MapSource or on the device.
I can fix this in various ways, but I want to ask how useful it is to
have thousands of roads with the same name appear as a result of a
search?
For a start in mapsource, you only see a few of the results anyway.
Should I drop the name completely from the index when there are more
than say 500 roads with the same name in the same city?
There is a branch called simplify-sorted-roads where I am trying out
different approaches to the problem, if anyone else wants to try it out
or look at it.
I can confirm that behaviour. Although in real mkgmap does not freeze
(endless loop) but is working very very hard. Possibly the mkgmap code
is not very optimal for the special test case.
@Steve, the code requires a very long time in the
NETFile.simplifySortedRoads method because I think there are many roads
with the same name. I think I remember a similar discussion some time
ago? The method seems to perform a brute force method to detect
connected roads with the same name. Can you have a look on it?
..Steve
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] mkgmap ignores --country-abbr?
Hi WanMil, Thank you for the explanation. Your patch did not make any difference: as far as I can see, all places in the problematic tile were still being assigned to Russia, except Rääkkylä, to which I added addr:country=FI. I did not add any addr:region or addr:city information. Some places in Russia do carry all of addr:city, addr:country, addr:region. Do all these have to be set in order to have a place copied to the known places list? The default style points file does not refer to addr:region, but only to mkgmap:region. There are no mkgmap:* tags in the source data. A last resort fix might be to tweak the style file, to drop anything tagged with another addr:country than --country-abbr. Regards, Marko ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
[mkgmap-dev] [PATCH] remove macrons from Japanese maps
This strips macrons (mostly ō and ū) from maps created with code page
ms932 (extended Shift-JIS, Japanese chars) because they do not exist in
that code page.
Rather than butcher TableTransliterator.java, I created
SparseTransliterator.java for the few char conversions I needed.
Index: src/uk/me/parabola/imgfmt/app/labelenc/CodeFunctions.java
===
--- src/uk/me/parabola/imgfmt/app/labelenc/CodeFunctions.java (revision 2053)
+++ src/uk/me/parabola/imgfmt/app/labelenc/CodeFunctions.java (working copy)
@@ -102,10 +102,8 @@
funcs.setEncodingType(ENCODING_FORMAT10);
funcs.setEncoder(new AnyCharsetEncoder(ms932));
funcs.setDecoder(new AnyCharsetDecoder(ms932));
- Transliterator transliterator = new
NullTransliterator();
- funcs.setTransliterator(transliterator);
+ funcs.setTransliterator(new
SparseTransliterator(nomacron));
funcs.setCodepage(932);
-
} else {
funcs.setEncodingType(ENCODING_FORMAT9);
funcs.setDecoder(new AnyCharsetDecoder(charset));
Index: src/uk/me/parabola/imgfmt/app/labelenc/SparseTransliterator.java
===
--- src/uk/me/parabola/imgfmt/app/labelenc/SparseTransliterator.java
(revision 0)
+++ src/uk/me/parabola/imgfmt/app/labelenc/SparseTransliterator.java
(revision 0)
@@ -0,0 +1,72 @@
+/*
+ * Copyright (C) 2011 mkgmap authors
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 3 or
+ * version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ */
+
+package uk.me.parabola.imgfmt.app.labelenc;
+
+import java.util.Locale;
+
+import uk.me.parabola.log.Logger;
+
+/**
+ * A sparse character-based transliterator that leaves most characters
unchanged.
+ *
+ */
+public class SparseTransliterator implements Transliterator {
+ private static final Logger log =
Logger.getLogger(SparseTransliterator.class);
+
+ private final boolean useNoMacron;
+ private boolean forceUppercase;
+
+ public SparseTransliterator(String targetCharset) {
+ useNoMacron = (targetCharset.equals(nomacron)) ? true : false;
+ }
+
+ /**
+* Convert a string into a string that uses only acceptable characters.
+*
+* @param s The original string. It can use any unicode character. Can
be null in which case null will
+* be returned.
+* @return A string that uses only acceptable characters.
+*/
+ public String transliterate(String s) {
+ if (s == null)
+ return null;
+
+ StringBuilder sb = new StringBuilder(s.length() + 5);
+ for (char c : s.toCharArray()) {
+ if (useNoMacron) {
+ // Only macrons are modified, all other chars
(including non-ascii) are left unchanged
+ if (c == 0x101) // Unicode Character 'LATIN
SMALL LETTER A WITH MACRON' (U+0101)
+ c = 'a';
+ if (c == 0x113) // Unicode Character 'LATIN
SMALL LETTER E WITH MACRON' (U+0113)
+ c = 'e';
+ if (c == 0x12b) // Unicode Character 'LATIN
SMALL LETTER I WITH MACRON' (U+012B)
+ c = 'i';
+ if (c == 0x14d) // Unicode Character 'LATIN
SMALL LETTER O WITH MACRON' (U+014D)
+ c = 'o';
+ if (c == 0x16b) // Unicode Character 'LATIN
SMALL LETTER U WITH MACRON' (U+016B)
+ c = 'u';
+ }
+ sb.append(c);
+ }
+
+ String text = sb.toString();
+ if (forceUppercase)
+ text = text.toUpperCase(Locale.ENGLISH);
+ return text;
+ }
+
+ public void forceUppercase(boolean uc) {
+ forceUppercase = uc;
+ }
+}
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
[mkgmap-dev] [PATCH] avoid redundant warnings for same cause
This reduces the warnings for FIXME - too many POIs in group and
point number too big to one per source map.
Kimon
Index: src/uk/me/parabola/mkgmap/combiners/MdrBuilder.java
===
--- src/uk/me/parabola/mkgmap/combiners/MdrBuilder.java (revision 2053)
+++ src/uk/me/parabola/mkgmap/combiners/MdrBuilder.java (working copy)
@@ -50,6 +50,7 @@
import uk.me.parabola.imgfmt.fs.FileSystem;
import uk.me.parabola.imgfmt.fs.ImgChannel;
import uk.me.parabola.imgfmt.sys.ImgFS;
+import uk.me.parabola.log.Logger;
import uk.me.parabola.mkgmap.CommandArgs;
import uk.me.parabola.mkgmap.srt.SrtTextReader;
@@ -69,6 +70,8 @@
private File tmpName;
private String outputName;
+ private static final Logger log = Logger.getLogger(MdrBuilder.class);
+
/**
* Create the mdr file and initialise.
* It has a name that is based on the overview-mapname option, as does
@@ -274,12 +277,16 @@
* @param maps Maps of regions, cities countries etc.
*/
private void addPoints(MapReader mr, AreaMaps maps) {
+ boolean warnedPointNumberTooBig = false;
ListPoint list = mr.pointsForLevel(0);
for (Point p : list) {
Label label = p.getLabel();
if (p.getNumber() 256) {
// I think we limit the number of
points+ind-points, but just in case
- System.out.println(point number too big);
+ if (!warnedPointNumberTooBig) {
+ log.warn(point number too big);
+ warnedPointNumberTooBig = true;
+ }
continue;
}
Index: src/uk/me/parabola/mkgmap/main/Main.java
===
--- src/uk/me/parabola/mkgmap/build/MapBuilder.java (revision 2053)
+++ src/uk/me/parabola/mkgmap/build/MapBuilder.java (working copy)
@@ -741,6 +741,8 @@
boolean haveIndPoints = false;
int pointIndex = 1;
+
+ boolean warnedTooManyPOIs = false;
// although the non-indexed points are output first,
// pointIndex must be initialized to the number of indexed
@@ -794,8 +796,12 @@
map.addMapObject(p);
if(!point.hasExtendedType()) {
if(name != null div.getZoom().getLevel() ==
0) {
- if(pointIndex 255)
- log.error(FIXME - too many
POIs in group);
+ if(pointIndex 255) {
+ if (!warnedTooManyPOIs) {
+ log.error(FIXME - too
many POIs in group);
+ warnedTooManyPOIs =
true;
+ }
+ }
else if(point.isExit()) {
Exit e =
((MapExitPoint)point).getExit();
if(e != null)
___
mkgmap-dev mailing list
mkgmap-dev@lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
