Re: [mkgmap-dev] splitter: "Cannot store node id 9223372036854775807, this value is reserved"
Hi Gerd, Please find enclosed file "splitter.log" which shows node ids in that region: 10.000.000 nodes parsed... id=9223372036671913720 splitter crashes after the entry 190.000.000 nodes parsed... id=9223372036851913720 The source code of srtm2osm at https://svn.openstreetmap.org/applications/utils/Srtm2Osm/trunk/Srtm2Osm/Srtm2OsmCommand.cs shows that private long firstNodeId = long.MaxValue; is the default value, the source of the IdCounter shows that it does decrement for the further values by default. => The value comes from the generation of contour lines via srtm2osm. By the way, also firstWayId is initialized to long.MaxValue. Would that also cause problems with splitter? Strange that this fact did not cause problems in February, when I created a map of Taiwan with DEM and elevation contour lines. Perhaps a bounding box in an osmosis command happened to remove this node id by chance. So, I guess, we have a solution. The bad thing is: I'll have to re-run the generation of the contourlines file with srtm2osm (with additional parameters for first node and way ids), which takes several hours, plus 1 hour of sorting, Kind regards, Bernhard Am 29.10.2018 um 08:04 schrieb Gerd Petermann: Hi Bernhard, splitter cannot store the id values 0 and 9223372036854775807 (which is Long.MAX_VALUE) No idea why your process whould create an id with such a value, looks like an error to me. The following message about the sort is misleading :-( Do you have an idea where this id comes from? Gerd Von: mkgmap-dev im Auftrag von Bernhard Hiller Gesendet: Montag, 29. Oktober 2018 07:48:02 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: [mkgmap-dev] splitter: "Cannot store node id 9223372036854775807, this value is reserved" Hi all, I tried to create a map of south east asia with contourlines. DEM data are from Viewfinderpanoramas, and were processed with srtm2osm. It took 7 h to generate a 40GB *.osm file, and another 1 hour to sort the data with osmosis into a 600 MB *.pbf file. A region from 0-23N and 94-109E was cut from the asia extract from Geofabrik with osmosis (some 400 MB). These two files were combined with osmosis. When I call splitter on that file, I get the error message: Error: Cannot store node id 9223372036854775807, this value is reserved. uk.me.parabola.splitter.SplitFailedException: Maybe the IDs are not sorted. This is not supported with keep-complete=ue or --problem-list at uk.me.parabola.splitter.MultiTileProcessor.storeCoord(MultiTileProcessor.java:508) at uk.me.parabola.splitter.MultiTileProcessor.processNode(MultiTileProcessor.java:127) at uk.me.parabola.splitter.AbstractMapProcessor.consume(AbstractMapProcessor.java:62) at uk.me.parabola.splitter.OSMFileHandler.execute(OSMFileHandler.java:159) at uk.me.parabola.splitter.ProblemLists.calcMultiTileElements(ProblemLists.java:255) at uk.me.parabola.splitter.Main.useProblemLists(Main.java:487) at uk.me.parabola.splitter.Main.start(Main.java:125) at uk.me.parabola.splitter.Main.main(Main.java:79) The splitter command is: E:\Maps\Development\Thai.1>"C:\Program Files\Java\jre1.8.0_121\bin\java.exe" -Xmx6G -jar "C:\Program Files (x86)\OpenStreetMap\splitter-r590\splitter.jar" E:\Maps\Raw\SouthEastAsia_Complete.osm.pbf --description=ernieMap Bike Thai" --mapidG120001 --max-nodes 0 --max-threads=4 --write-kml=slitter.kml--no-trim 1>splitter.log Do I need to sort again after merging? Or is that a different issue? Thanks a lot for your hints. Kind regards, Bernhard ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev Splitter version 590 compiled 2018-01-27T09:06:04+ boundary-tags=use-exclude-list cache= description=BernieMap Bike Thai geonames-file= handle-element-version=remove ignore-osm-bounds=false keep-complete=true mapid=47120001 max-areas=2048 max-nodes=300 max-threads=4 mixed=false no-trim=true num-tiles= output=pbf output-dir= overlap=auto polygon-desc-file= polygon-file= precomp-sea= problem-file= problem-report= resolution=13 search-limit=20 split-file= status-freq=120 stop-after=dist wanted-admin-level=5 write-kml=splitter.kml Elapsed time: 0s Memory: Current 123MB (5MB used, 118MB free) Max 5461MB Time started: Mon Oct 29 19:06:16 CET 2018 Map is being split for resolution 13: - area boundaries are aligned to 0x800 map units (0.0439453125 degrees) - areas are multiples of 0x800 map units wide and high Processing E:\Maps\Raw\Thai.DEM\SouthEastAsia_Ele.osm.pbf 10.000.000 nodes parsed... id=9223372036671913720 20.000.000 nodes parsed... id=9223372036681913720 30.000.000 nodes parsed... id=9223372036691913720 40.000.000 nodes parsed... id=9223372036701913720 50.000.000 nodes parsed... id=9223372036711913720 60.000.000 nodes parsed... id=92233720367219
Re: [mkgmap-dev] splitter: "Cannot store node id 9223372036854775807, this value is reserved"
Hi Bernhard, I would not say that the error cannot be in the file produced by srtm2osm. The bad id might come after splitter complains about the wrong order. You may try osmconvert with --out-statistics to check the file. But be careful, current 64 Bit Windows version of osmconvert seems to be buggy, make sure to use the one from the english wiki page with large file support. Gerd Von: mkgmap-dev im Auftrag von Bernhard Hiller Gesendet: Montag, 29. Oktober 2018 19:16:31 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] splitter: "Cannot store node id 9223372036854775807, this value is reserved" Hi Gerd, I am investigation the origin of that id. I tried to run splitter on the file generated by srtm2osm. After half an hour, I got following exception: E:\Maps\Development\Thai.1>"C:\Program Files\Java\jre1.8.0_121\bin\java.exe" -Xmx6G -jar "C:\Program Files (x86)\OpenStreetMap\splitter-r590\splitter.jar" E:\Maps\Raw\Thai.DEM\SouthEastAsia_Ele_Unsorted.osm --description="BernieMap Bike Thai" --mapid=47120001 --max-nodes=300 --max-threads=4 --write-kml=splitter.kml --no-trim 1>splitter.log Error: Node ids are not sorted. Use e.g. osmosis to sort the input data. This is not supported with keep-complete=true or --problem-list uk.me.parabola.splitter.SplitFailedException: Node ids are not sorted at uk.me.parabola.splitter.MultiTileProcessor.storeCoord(MultiTileProcessor.java:501) at uk.me.parabola.splitter.MultiTileProcessor.processNode(MultiTileProcessor.java:127) at uk.me.parabola.splitter.AbstractMapProcessor.consume(AbstractMapProcessor.java:62) at uk.me.parabola.splitter.OSMFileHandler.execute(OSMFileHandler.java:159) at uk.me.parabola.splitter.ProblemLists.calcMultiTileElements(ProblemLists.java:255) at uk.me.parabola.splitter.Main.useProblemLists(Main.java:487) at uk.me.parabola.splitter.Main.start(Main.java:125) at uk.me.parabola.splitter.Main.main(Main.java:79) which is OK, because srtm2osm does not write its output in some sorted way. Next, I tried to run splitter on the file produced by osmosis after sorting. Now, it did not take long to get the exception: E:\Maps\Development\Thai.1>"C:\Program Files\Java\jre1.8.0_121\bin\java.exe" -Xmx6G -jar "C:\Program Files (x86)\OpenStreetMap\splitter-r590\splitter.jar" E:\Maps\Raw\Thai.DEM\SouthEastAsia_Ele.osm.pbf --description="BernieMap Bike Thai" --mapid=47120001 --max-nodes=300 --max-threads=4 --write-kml=splitter.kml--no-trim 1>splitter.log Error: Cannot store node id 9223372036854775807, this value is reserved. uk.me.parabola.splitter.SplitFailedException: Maybe the IDs are not sorted. This is not supported with keep-complete=true or --problem-list at uk.me.parabola.splitter.MultiTileProcessor.storeCoord(MultiTileProcessor.java:508) at uk.me.parabola.splitter.MultiTileProcessor.processNode(MultiTileProcessor.java:127) at uk.me.parabola.splitter.AbstractMapProcessor.consume(AbstractMapProcessor.java:62) at uk.me.parabola.splitter.OSMFileHandler.execute(OSMFileHandler.java:159) at uk.me.parabola.splitter.ProblemLists.calcMultiTileElements(ProblemLists.java:255) at uk.me.parabola.splitter.Main.useProblemLists(Main.java:487) at uk.me.parabola.splitter.Main.start(Main.java:125) at uk.me.parabola.splitter.Main.main(Main.java:79) That is, the wrong node id exists in the file of sorted elevation data. I cannot find out if osmosis or srtm2osm produced that node id. Kind regards, Bernhard Am 29.10.2018 um 08:04 schrieb Gerd Petermann: > Hi Bernhard, > > splitter cannot store the id values 0 and 9223372036854775807 (which is > Long.MAX_VALUE) > No idea why your process whould create an id with such a value, looks like an > error to me. > The following message about the sort is misleading :-( > > Do you have an idea where this id comes from? > > Gerd > > > Von: mkgmap-dev im Auftrag von > Bernhard Hiller > Gesendet: Montag, 29. Oktober 2018 07:48:02 > An: mkgmap-dev@lists.mkgmap.org.uk > Betreff: [mkgmap-dev] splitter: "Cannot store node id 9223372036854775807, > this value is reserved" > > Hi all, > I tried to create a map of south east asia with contourlines. DEM data > are from Viewfinderpanoramas, and were processed with srtm2osm. It took > 7 h to generate a 40GB *.osm file, and another 1 hour to sort the data > with osmosis into a 600 MB *.pbf file. A region from 0-23N and 94-109E > was cut from the asia extract from Geofabrik with osmosis (some 400 MB). > These two files were combined with osmosis. > > When I call splitter on that file, I get the error message: > > Error: Cannot store node id 9223372036854775807, this value is reserved. > uk.me.parabola.splitter.SplitFailedException: Maybe the IDs are not > sorted. This is not supported with keep-complete=ue or --problem-lis
Re: [mkgmap-dev] splitter: "Cannot store node id 9223372036854775807, this value is reserved"
On Mon, Oct 29, Bernhard Hiller wrote: > Hi Gerd, > I am investigation the origin of that id. > I tried to run splitter on the file generated by srtm2osm. Try phyghtmap. In my experience, that works much better than srtm2osm. Thorsten -- Thorsten Kukuk, Distinguished Engineer, Senior Architect SLES & MicroOS SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany GF: Felix Imendoerffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nuernberg) ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] splitter: "Cannot store node id 9223372036854775807, this value is reserved"
Hi Gerd, I am investigation the origin of that id. I tried to run splitter on the file generated by srtm2osm. After half an hour, I got following exception: E:\Maps\Development\Thai.1>"C:\Program Files\Java\jre1.8.0_121\bin\java.exe" -Xmx6G -jar "C:\Program Files (x86)\OpenStreetMap\splitter-r590\splitter.jar" E:\Maps\Raw\Thai.DEM\SouthEastAsia_Ele_Unsorted.osm --description="BernieMap Bike Thai" --mapid=47120001 --max-nodes=300 --max-threads=4 --write-kml=splitter.kml --no-trim 1>splitter.log Error: Node ids are not sorted. Use e.g. osmosis to sort the input data. This is not supported with keep-complete=true or --problem-list uk.me.parabola.splitter.SplitFailedException: Node ids are not sorted at uk.me.parabola.splitter.MultiTileProcessor.storeCoord(MultiTileProcessor.java:501) at uk.me.parabola.splitter.MultiTileProcessor.processNode(MultiTileProcessor.java:127) at uk.me.parabola.splitter.AbstractMapProcessor.consume(AbstractMapProcessor.java:62) at uk.me.parabola.splitter.OSMFileHandler.execute(OSMFileHandler.java:159) at uk.me.parabola.splitter.ProblemLists.calcMultiTileElements(ProblemLists.java:255) at uk.me.parabola.splitter.Main.useProblemLists(Main.java:487) at uk.me.parabola.splitter.Main.start(Main.java:125) at uk.me.parabola.splitter.Main.main(Main.java:79) which is OK, because srtm2osm does not write its output in some sorted way. Next, I tried to run splitter on the file produced by osmosis after sorting. Now, it did not take long to get the exception: E:\Maps\Development\Thai.1>"C:\Program Files\Java\jre1.8.0_121\bin\java.exe" -Xmx6G -jar "C:\Program Files (x86)\OpenStreetMap\splitter-r590\splitter.jar" E:\Maps\Raw\Thai.DEM\SouthEastAsia_Ele.osm.pbf --description="BernieMap Bike Thai" --mapid=47120001 --max-nodes=300 --max-threads=4 --write-kml=splitter.kml--no-trim 1>splitter.log Error: Cannot store node id 9223372036854775807, this value is reserved. uk.me.parabola.splitter.SplitFailedException: Maybe the IDs are not sorted. This is not supported with keep-complete=true or --problem-list at uk.me.parabola.splitter.MultiTileProcessor.storeCoord(MultiTileProcessor.java:508) at uk.me.parabola.splitter.MultiTileProcessor.processNode(MultiTileProcessor.java:127) at uk.me.parabola.splitter.AbstractMapProcessor.consume(AbstractMapProcessor.java:62) at uk.me.parabola.splitter.OSMFileHandler.execute(OSMFileHandler.java:159) at uk.me.parabola.splitter.ProblemLists.calcMultiTileElements(ProblemLists.java:255) at uk.me.parabola.splitter.Main.useProblemLists(Main.java:487) at uk.me.parabola.splitter.Main.start(Main.java:125) at uk.me.parabola.splitter.Main.main(Main.java:79) That is, the wrong node id exists in the file of sorted elevation data. I cannot find out if osmosis or srtm2osm produced that node id. Kind regards, Bernhard Am 29.10.2018 um 08:04 schrieb Gerd Petermann: Hi Bernhard, splitter cannot store the id values 0 and 9223372036854775807 (which is Long.MAX_VALUE) No idea why your process whould create an id with such a value, looks like an error to me. The following message about the sort is misleading :-( Do you have an idea where this id comes from? Gerd Von: mkgmap-dev im Auftrag von Bernhard Hiller Gesendet: Montag, 29. Oktober 2018 07:48:02 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: [mkgmap-dev] splitter: "Cannot store node id 9223372036854775807, this value is reserved" Hi all, I tried to create a map of south east asia with contourlines. DEM data are from Viewfinderpanoramas, and were processed with srtm2osm. It took 7 h to generate a 40GB *.osm file, and another 1 hour to sort the data with osmosis into a 600 MB *.pbf file. A region from 0-23N and 94-109E was cut from the asia extract from Geofabrik with osmosis (some 400 MB). These two files were combined with osmosis. When I call splitter on that file, I get the error message: Error: Cannot store node id 9223372036854775807, this value is reserved. uk.me.parabola.splitter.SplitFailedException: Maybe the IDs are not sorted. This is not supported with keep-complete=ue or --problem-list at uk.me.parabola.splitter.MultiTileProcessor.storeCoord(MultiTileProcessor.java:508) at uk.me.parabola.splitter.MultiTileProcessor.processNode(MultiTileProcessor.java:127) at uk.me.parabola.splitter.AbstractMapProcessor.consume(AbstractMapProcessor.java:62) at uk.me.parabola.splitter.OSMFileHandler.execute(OSMFileHandler.java:159) at uk.me.parabola.splitter.ProblemLists.calcMultiTileElements(ProblemLists.java:255) at uk.me.parabola.splitter.Main.useProblemLists(Main.java:487) at uk.me.parabola.splitter.Main.start(Main.java:125) at uk.me.parabola.splitter.Main.main(Main.java:79) The splitter command is: E:\Maps\Develo
Re: [mkgmap-dev] splitter: "Cannot store node id 9223372036854775807, this value is reserved"
Hi Bernhard, splitter cannot store the id values 0 and 9223372036854775807 (which is Long.MAX_VALUE) No idea why your process whould create an id with such a value, looks like an error to me. The following message about the sort is misleading :-( Do you have an idea where this id comes from? Gerd Von: mkgmap-dev im Auftrag von Bernhard Hiller Gesendet: Montag, 29. Oktober 2018 07:48:02 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: [mkgmap-dev] splitter: "Cannot store node id 9223372036854775807, this value is reserved" Hi all, I tried to create a map of south east asia with contourlines. DEM data are from Viewfinderpanoramas, and were processed with srtm2osm. It took 7 h to generate a 40GB *.osm file, and another 1 hour to sort the data with osmosis into a 600 MB *.pbf file. A region from 0-23N and 94-109E was cut from the asia extract from Geofabrik with osmosis (some 400 MB). These two files were combined with osmosis. When I call splitter on that file, I get the error message: Error: Cannot store node id 9223372036854775807, this value is reserved. uk.me.parabola.splitter.SplitFailedException: Maybe the IDs are not sorted. This is not supported with keep-complete=true or --problem-list at uk.me.parabola.splitter.MultiTileProcessor.storeCoord(MultiTileProcessor.java:508) at uk.me.parabola.splitter.MultiTileProcessor.processNode(MultiTileProcessor.java:127) at uk.me.parabola.splitter.AbstractMapProcessor.consume(AbstractMapProcessor.java:62) at uk.me.parabola.splitter.OSMFileHandler.execute(OSMFileHandler.java:159) at uk.me.parabola.splitter.ProblemLists.calcMultiTileElements(ProblemLists.java:255) at uk.me.parabola.splitter.Main.useProblemLists(Main.java:487) at uk.me.parabola.splitter.Main.start(Main.java:125) at uk.me.parabola.splitter.Main.main(Main.java:79) The splitter command is: E:\Maps\Development\Thai.1>"C:\Program Files\Java\jre1.8.0_121\bin\java.exe" -Xmx6G -jar "C:\Program Files (x86)\OpenStreetMap\splitter-r590\splitter.jar" E:\Maps\Raw\SouthEastAsia_Complete.osm.pbf --description="BernieMap Bike Thai" --mapid=47120001 --max-nodes=200 --max-threads=4 --write-kml=splitter.kml--no-trim 1>splitter.log Do I need to sort again after merging? Or is that a different issue? Thanks a lot for your hints. Kind regards, Bernhard ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
