The 2nd patch fixes the remaining problems:
1. Ways without tags but referenced by relations are no longer removed
(they might be tagged by the relation style file and could therefore
appear in the map)
2. Intersection of ways with the tile bounding box is now checked
instead of only checking that at least one point is contained in the
bounding box. Only very few ways are affected by this but otherwise
routing problems are possible.
The performance improvement seems to be good (measured with my test map):
r2159: ~250s
patched: ~225s
I also expect (although I haven't tested) that the max memory
requirement of mkgmap is decreased. I think max mem is used when the
style file is just processed. At this stage all raw OSM elements and all
style file processed elements are kept in memory. With the patch the
number of OSM elements is noticeably reduced.
WanMil
Hi,
this is another performance improvement:
Usually the mkgmap input tiles are larger than the processed bounding
box (splitter parameter overlap). So there are much many elements which
are processed but thrown away at a late step in mkgmap.
The patch tries to remove them much earlier before the style files are
processed and before the LocationHook starts (which ignores them but
that must also be calculated).
The patch contains one drawback:
Ways which have all its points outside the bounding box of the tile but
which cross the tile are also removed. If that's a point the patch must
be improved.
Have fun!
WanMil
_______________________________________________
mkgmap-dev mailing list
[email protected]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Index: src/uk/me/parabola/mkgmap/reader/osm/UnusedElementsRemoverHook.java
===================================================================
--- src/uk/me/parabola/mkgmap/reader/osm/UnusedElementsRemoverHook.java (revision 0)
+++ src/uk/me/parabola/mkgmap/reader/osm/UnusedElementsRemoverHook.java (revision 0)
@@ -0,0 +1,101 @@
+package uk.me.parabola.mkgmap.reader.osm;
+
+import java.awt.Rectangle;
+import java.util.ArrayList;
+import java.util.HashSet;
+import java.util.Map.Entry;
+import java.util.Set;
+
+import uk.me.parabola.imgfmt.app.Area;
+import uk.me.parabola.imgfmt.app.Coord;
+import uk.me.parabola.log.Logger;
+import uk.me.parabola.util.EnhancedProperties;
+
+public class UnusedElementsRemoverHook extends OsmReadingHooksAdaptor {
+ private static final Logger log = Logger.getLogger(UnusedElementsRemoverHook.class);
+
+ private ElementSaver saver;
+
+ public UnusedElementsRemoverHook() {
+ }
+
+ public boolean init(ElementSaver saver, EnhancedProperties props) {
+ this.saver = saver;
+ return true;
+ }
+
+ public void end() {
+ long t1 = System.currentTimeMillis();
+ log.error("Removing unused elements");
+
+ final Area bbox = saver.getBoundingBox();
+
+ long nodes = saver.getNodes().size();
+ for (Node node : new ArrayList<Node>(saver.getNodes().values())) {
+ if (node.getTagCount() == 0) {
+ saver.getNodes().remove(node.getId());
+ continue;
+ }
+ if (bbox.contains(node.getLocation()) == false) {
+ saver.getNodes().remove(node.getId());
+ }
+ }
+
+ long tr1 = System.currentTimeMillis();
+ // store all way ids that are referenced by a relation
+ Set<Long> relationWays = new HashSet<Long>();
+ for (Relation rel : saver.getRelations().values()) {
+ for (Entry<String, Element> relEntry : rel.getElements()) {
+ if (relEntry.getValue() instanceof Way) {
+ relationWays.add(relEntry.getValue().getId());
+ }
+ }
+ }
+ log.error("Collecting way ids from relations took "+(System.currentTimeMillis()-tr1)+" ms");
+
+ Rectangle bboxRect = new Rectangle(bbox.getMinLong(), bbox.getMinLat(), bbox.getWidth(), bbox.getHeight());
+ long relWays = 0;
+ long ways = saver.getWays().size();
+ for (Way way : new ArrayList<Way>(saver.getWays().values())) {
+ // check if a way has no tags and is not a member of a relation
+ // a relation might be used to add tags to the way using the style file
+ if (way.getTagCount() == 0) {
+ if (relationWays.contains(way.getId())) {
+ relWays++;
+ } else {
+ saver.getWays().remove(way.getId());
+ continue;
+ }
+ }
+
+ // check if the way is completely outside the tile bounding box
+ boolean coordInBbox = false;
+ Coord prevC = null;
+ for (Coord c : way.getPoints()) {
+ if (bbox.contains(c)) {
+ coordInBbox = true;
+ break;
+ } else if (prevC != null) {
+ if (bboxRect.intersectsLine(prevC.getLongitude(), prevC.getLatitude(), c.getLongitude(), c.getLatitude())) {
+ log.error("Intersection!");
+ log.error(bbox.getMinLong()+" "+bbox.getMinLat()+" "+bbox.getMaxLat()+" "+bbox.getMaxLong());
+ log.error(prevC.getLongitude()+" "+prevC.getLatitude()+" "+c.getLongitude()+" "+c.getLatitude());
+ coordInBbox = true;
+ break;
+ }
+ }
+ prevC = c;
+ }
+ if (coordInBbox==false) {
+ saver.getWays().remove(way.getId());
+ }
+ }
+
+ log.error("Relation referenced ways: "+relationWays.size()+ " Used: "+relWays);
+
+ log.error("Removing unused elements took "+(System.currentTimeMillis()-t1)+" ms");
+ log.error("Nodes: before "+nodes+" now "+saver.getNodes().size());
+ log.error("Ways: before "+ways+" now "+saver.getWays().size());
+ }
+
+}
Index: src/uk/me/parabola/mkgmap/reader/osm/OsmMapDataSource.java
===================================================================
--- src/uk/me/parabola/mkgmap/reader/osm/OsmMapDataSource.java (revision 2159)
+++ src/uk/me/parabola/mkgmap/reader/osm/OsmMapDataSource.java (working copy)
@@ -53,6 +53,7 @@
private final OsmReadingHooks[] POSSIBLE_HOOKS = {
new SeaGenerator(),
new MultiPolygonFinishHook(),
+ new UnusedElementsRemoverHook(),
new RoutingHook(),
new HighwayHooks(),
new LocationHook(),
_______________________________________________
mkgmap-dev mailing list
[email protected]
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
