Re: [mkgmap-dev] problematic_polygons file
Hi Carlos, I've seen problematic_polygons file in growing quite fast in the wiki. May it affect splitter performance? If so, would it make sense to split the file by continents or countries? regarding performance: a) if you specify the --problem-file parm, splitter has to do a lot more work, so that affects performance, no matter how many ids you put ino your list. b) I don't expect a measurable performance impact for any id that does NOT occur in your input OSM file(s) as long as this list doesn't contain millions of ids. The information is stored in HashMaps, so the only negative impact is a higher possibility of hash collisions and a slightly higher memory usage for these HashMaps. Most of the additional time that is required to handle the list is caused by the fact that splitter has to read the input file more often (3 times). With o5m and pbf format, only parts of the file(s) are read, with XML input the complete file is processed. Of course, those ids that occor in your input data will require more heap and probably produce more output data, so that will affect performance. OK? The bigger problem that I see is this: The list now contains some relations like rel:52822 # Border Sweden rel:1059668 # Border Norway If you use the complete list to split e.g. finland.osm.pbf, the input file will only contain parts of the needed data for these polygons. I wonder what splitter should do in these cases? Currently it might print a few messages regarding missing nodes or ways, but it will use the incomplete data and it might write the incomplete relation to more tiles than r202, thus mgkmap might produce even more error messages. I think it would be better to change this so that splitter returns to the default handling for every relation or way that is not complete, with a corresponding message. Would that be better? Ciao, Gerd ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] problematic_polygons file
El 23/10/12 09:59, Gerd Petermann escribió: Hi Carlos, I've seen problematic_polygons file in growing quite fast in the wiki. May it affect splitter performance? If so, would it make sense to split the file by continents or countries? regarding performance: a) if you specify the --problem-file parm, splitter has to do a lot more work, so that affects performance, no matter how many ids you put ino your list. b) I don't expect a measurable performance impact for any id that does NOT occur in your input OSM file(s) as long as this list doesn't contain millions of ids. The information is stored in HashMaps, so the only negative impact is a higher possibility of hash collisions and a slightly higher memory usage for these HashMaps. Most of the additional time that is required to handle the list is caused by the fact that splitter has to read the input file more often (3 times). With o5m and pbf format, only parts of the file(s) are read, with XML input the complete file is processed. Of course, those ids that occor in your input data will require more heap and probably produce more output data, so that will affect performance. OK? Yes, thank you for the explanation The bigger problem that I see is this: The list now contains some relations like rel:52822 # Border Sweden rel:1059668 # Border Norway If you use the complete list to split e.g. finland.osm.pbf, the input file will only contain parts of the needed data for these polygons. I wonder what splitter should do in these cases? Currently it might print a few messages regarding missing nodes or ways, but it will use the incomplete data and it might write the incomplete relation to more tiles than r202, thus mgkmap might produce even more error messages. I think it would be better to change this so that splitter returns to the default handling for every relation or way that is not complete, with a corresponding message. Would that be better? If a multipolygon is not complete I think mkgmap won't use it, so there's no sense adding it to more tiles. I agree with you to return to default handling it those cases. ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] problematic_polygons file
Am 23.10.2012 09:59, schrieb Gerd Petermann: Hi Carlos, I've seen problematic_polygons file in growing quite fast in the wiki. May it affect splitter performance? If so, would it make sense to split the file by continents or countries? regarding performance: a) if you specify the --problem-file parm, splitter has to do a lot more work, so that affects performance, no matter how many ids you put ino your list. b) I don't expect a measurable performance impact for any id that does NOT occur in your input OSM file(s) as long as this list doesn't contain millions of ids. The information is stored in HashMaps, so the only negative impact is a higher possibility of hash collisions and a slightly higher memory usage for these HashMaps. Most of the additional time that is required to handle the list is caused by the fact that splitter has to read the input file more often (3 times). With o5m and pbf format, only parts of the file(s) are read, with XML input the complete file is processed. Of course, those ids that occor in your input data will require more heap and probably produce more output data, so that will affect performance. OK? The bigger problem that I see is this: The list now contains some relations like rel:52822 # Border Sweden rel:1059668 # Border Norway If you use the complete list to split e.g. finland.osm.pbf, the input file will only contain parts of the needed data for these polygons. I wonder what splitter should do in these cases? Currently it might print a few messages regarding missing nodes or ways, but it will use the incomplete data and it might write the incomplete relation to more tiles than r202, thus mgkmap might produce even more error messages. I think it would be better to change this so that splitter returns to the default handling for every relation or way that is not complete, with a corresponding message. Would that be better? I think splitter should write everything he could find in input-data to each tile. The intention is a complete boundary. Of course if you only create a map of Finland, border of Sweden would only partly in the map, but these lines shouldn't be broken. But is this a new problem? I think the relation of eg. Sweden would be written to a tile containing a part of Swedish boundary and there are also missing ways and nodes. @Carlos: Maybe it would be better to separate the wiki-list in several parts like ferry, boundary, lakes, forrest. So it is easier to find out, if the ID is already in the list. Henning ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] problematic_polygons file
El 23/10/12 10:32, Henning Scholland escribió: @Carlos: Maybe it would be better to separate the wiki-list in several parts like ferry, boundary, lakes, forrest. So it is easier to find out, if the ID is already in the list. I've already partially ordered it by ID which I think is the easiest way to know if a way/relation is already in the list. What do you think? ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] problematic_polygons file
Hi I separated now ferry ways and boundary-relations and everything else and ordered each group by ID. I think the rest depends on the length of the list. Boundaries and ferries are separated, because they are just a side effect and are dominating actual. Also maybe they could be removed if a general rule for them is possible. Henning ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
