Re: [mkgmap-dev] Error in memory calculation
Ah, now i understand Am 05.02.2020 um 22:33 schrieb Gerd Petermann: Hi Manfred, the calculation is only done when you don't use the --max-jobs option and when more than two input files are processed. Gerd Von: mkgmap-dev im Auftrag von DD8KQ Gesendet: Mittwoch, 5. Februar 2020 21:49 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] Error in memory calculation Hi Gerd just did a small map job but on the console there is no print about number of max jobs. Is there a special log flag to get this print out ? What i've seen in the task manager while building the map, all 8 cpu's are used, but i don't know anything about the number of jobs ... Am 05.02.2020 um 21:35 schrieb Gerd Petermann: Hi Manfred, mkgmap contains code to estimate a suitable value for max-jobs if this parameter isn't set. This code fails and the result is that it believes that a first thread needed all available memory. This doesn't happen when the -Xms parameter in my sample call is removed, and it also doesn't happen with JRE 1.8 (both from OpenJDK and Oracle). I've not yet started to look at the details returned in class java.lang.management.MemoryPoolMXBean Gerd Von: mkgmap-dev im Auftrag von DD8KQ Gesendet: Mittwoch, 5. Februar 2020 21:16 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] Error in memory calculation Hi Gerd i'm not sure what you are looking for. In my build environment i'm using java 1.8.0_241 (64 bit) and as my laptop have 32 Gb of memory availiable my command line to build is like this : java -Xmx3m -XX:-UseGCOverheadLimit -ea -jar -Dlog.config=logging.properties "c:\Program Files (x86)\KartenTools\mkgmap\mkgmap.jar" and depending on which map i want to build, the occupied memory runs up nearly to 30Gb Am 05.02.2020 um 14:05 schrieb Ticker Berkin: Hi Gerd I don't have a 64bit environment or that much memory. In my normal environment, I get: $ java -Xms1500M -Xmx1540M -jar ../mkgmap.trunk/mkgmap.jar \ dummy.osm dummy.osm dummy.osm Time started: Wed Feb 05 12:56:28 GMT 2020 Setting max-jobs to 4 Number of MapFailedExceptions: 0 Number of ExitExceptions: 0 Time finished: Wed Feb 05 12:56:29 GMT 2020 Total time taken: 1 second -Xmx1540M is the maximum value that allows Java to start up on my machine. With a newer version I loaded in October, but it didn't have any obvious advantages: $ /opt/jre1.8.0_231/bin/java -Xms1500M -Xmx1540M \ -jar ../mkgmap.trunk/mkgmap.jar \ dummy.osm dummy.osm dummy.osm Time started: Wed Feb 05 12:58:29 GMT 2020 Setting max-jobs to 4 Number of MapFailedExceptions: 0 Number of ExitExceptions: 0 Time finished: Wed Feb 05 12:58:30 GMT 2020 Total time taken: 1 second Ticker On Fri, 2020-01-31 at 16:15 +, Gerd Petermann wrote: please try this command with openjdk 11 or 13 64Bit: java -Xms4G -Xmx10G -jar mkgmap.jar f:\osm\dummy.osm f:\osm\dummy.osm f:\osm\dummy.osm This is what I get: Time started: Fri Jan 31 17:12:53 CET 2020 Setting max-jobs to 1 Number of MapFailedExceptions: 0 To reduce the run time, consider increasing the amnount of memory available for use by mkgmap by using the Java -Xmx flag to set the memory to more than 41000 MB, providing this is less than the amount of physical memory installed. Number of ExitExceptions: 0 Time finished: Fri Jan 31 17:12:54 CET 2020 Total time taken: 1 second ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev -- # Viele Grüße und 73 de Manfred Haiduk, DD8KQ e-mail mhai...@t-online.de dd...@gmx.de # ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev -- # Viele Grüße und 73 de Manfred Haiduk, DD8KQ e-mail mhai...@t-online.de dd...@gmx.de # ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev -- # Viele Grüße und 73 de Manfred Haiduk, DD8KQ e-mail mhai...@t-online.de dd...@gmx.de # ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Error in memory calculation
Hi Manfred, the calculation is only done when you don't use the --max-jobs option and when more than two input files are processed. Gerd Von: mkgmap-dev im Auftrag von DD8KQ Gesendet: Mittwoch, 5. Februar 2020 21:49 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] Error in memory calculation Hi Gerd just did a small map job but on the console there is no print about number of max jobs. Is there a special log flag to get this print out ? What i've seen in the task manager while building the map, all 8 cpu's are used, but i don't know anything about the number of jobs ... Am 05.02.2020 um 21:35 schrieb Gerd Petermann: > Hi Manfred, > > mkgmap contains code to estimate a suitable value for max-jobs if this > parameter isn't set. This code fails and the result is that it believes that > a first thread needed all available memory. This doesn't happen when the -Xms > parameter in my sample call is removed, and it also doesn't happen with JRE > 1.8 (both from OpenJDK and Oracle). I've not yet started to look at the > details returned in class java.lang.management.MemoryPoolMXBean > > Gerd > > > Von: mkgmap-dev im Auftrag von DD8KQ > > Gesendet: Mittwoch, 5. Februar 2020 21:16 > An: mkgmap-dev@lists.mkgmap.org.uk > Betreff: Re: [mkgmap-dev] Error in memory calculation > > Hi Gerd > > i'm not sure what you are looking for. In my build environment i'm using > java 1.8.0_241 (64 bit) and as my laptop have 32 Gb of memory availiable > my command line to build is like this : > > java -Xmx3m -XX:-UseGCOverheadLimit -ea -jar > -Dlog.config=logging.properties "c:\Program Files > (x86)\KartenTools\mkgmap\mkgmap.jar" > > and depending on which map i want to build, the occupied memory runs up > nearly to 30Gb > > Am 05.02.2020 um 14:05 schrieb Ticker Berkin: >> Hi Gerd >> >> I don't have a 64bit environment or that much memory. In my normal >> environment, I get: >> >> $ java -Xms1500M -Xmx1540M -jar ../mkgmap.trunk/mkgmap.jar \ >> dummy.osm dummy.osm dummy.osm >> Time started: Wed Feb 05 12:56:28 GMT 2020 >> Setting max-jobs to 4 >> Number of MapFailedExceptions: 0 >> Number of ExitExceptions: 0 >> Time finished: Wed Feb 05 12:56:29 GMT 2020 >> Total time taken: 1 second >> >> -Xmx1540M is the maximum value that allows Java to start up on my >> machine. >> >> With a newer version I loaded in October, but it didn't have any >> obvious advantages: >> >> $ /opt/jre1.8.0_231/bin/java -Xms1500M -Xmx1540M \ >> -jar ../mkgmap.trunk/mkgmap.jar \ >> dummy.osm dummy.osm dummy.osm >> Time started: Wed Feb 05 12:58:29 GMT 2020 >> Setting max-jobs to 4 >> Number of MapFailedExceptions: 0 >> Number of ExitExceptions: 0 >> Time finished: Wed Feb 05 12:58:30 GMT 2020 >> Total time taken: 1 second >> >> Ticker >> >> On Fri, 2020-01-31 at 16:15 +, Gerd Petermann wrote: >>> please try this command with openjdk 11 or 13 64Bit: >>> java -Xms4G -Xmx10G -jar mkgmap.jar f:\osm\dummy.osm f:\osm\dummy.osm >>> f:\osm\dummy.osm >>> This is what I get: >>> Time started: Fri Jan 31 17:12:53 CET 2020 >>> Setting max-jobs to 1 >>> Number of MapFailedExceptions: 0 >>> To reduce the run time, consider increasing the amnount of memory >>> available for use by mkgmap by using the Java -Xmx flag to set the >>> memory to more than 41000 MB, providing this is less than the amount >>> of physical memory installed. >>> Number of ExitExceptions: 0 >>> Time finished: Fri Jan 31 17:12:54 CET 2020 >>> Total time taken: 1 second >> ___ >> mkgmap-dev mailing list >> mkgmap-dev@lists.mkgmap.org.uk >> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > -- > > # > >Viele Grüße und 73 de Manfred Haiduk, DD8KQ >e-mail mhai...@t-online.de dd...@gmx.de > > # > > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev -- # Viele Grüße und 73 de Manfred Haiduk, DD8KQ e-mail mhai...@t-online.de dd...@gmx.de # ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Error in memory calculation
Hi Gerd just did a small map job but on the console there is no print about number of max jobs. Is there a special log flag to get this print out ? What i've seen in the task manager while building the map, all 8 cpu's are used, but i don't know anything about the number of jobs ... Am 05.02.2020 um 21:35 schrieb Gerd Petermann: Hi Manfred, mkgmap contains code to estimate a suitable value for max-jobs if this parameter isn't set. This code fails and the result is that it believes that a first thread needed all available memory. This doesn't happen when the -Xms parameter in my sample call is removed, and it also doesn't happen with JRE 1.8 (both from OpenJDK and Oracle). I've not yet started to look at the details returned in class java.lang.management.MemoryPoolMXBean Gerd Von: mkgmap-dev im Auftrag von DD8KQ Gesendet: Mittwoch, 5. Februar 2020 21:16 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] Error in memory calculation Hi Gerd i'm not sure what you are looking for. In my build environment i'm using java 1.8.0_241 (64 bit) and as my laptop have 32 Gb of memory availiable my command line to build is like this : java -Xmx3m -XX:-UseGCOverheadLimit -ea -jar -Dlog.config=logging.properties "c:\Program Files (x86)\KartenTools\mkgmap\mkgmap.jar" and depending on which map i want to build, the occupied memory runs up nearly to 30Gb Am 05.02.2020 um 14:05 schrieb Ticker Berkin: Hi Gerd I don't have a 64bit environment or that much memory. In my normal environment, I get: $ java -Xms1500M -Xmx1540M -jar ../mkgmap.trunk/mkgmap.jar \ dummy.osm dummy.osm dummy.osm Time started: Wed Feb 05 12:56:28 GMT 2020 Setting max-jobs to 4 Number of MapFailedExceptions: 0 Number of ExitExceptions: 0 Time finished: Wed Feb 05 12:56:29 GMT 2020 Total time taken: 1 second -Xmx1540M is the maximum value that allows Java to start up on my machine. With a newer version I loaded in October, but it didn't have any obvious advantages: $ /opt/jre1.8.0_231/bin/java -Xms1500M -Xmx1540M \ -jar ../mkgmap.trunk/mkgmap.jar \ dummy.osm dummy.osm dummy.osm Time started: Wed Feb 05 12:58:29 GMT 2020 Setting max-jobs to 4 Number of MapFailedExceptions: 0 Number of ExitExceptions: 0 Time finished: Wed Feb 05 12:58:30 GMT 2020 Total time taken: 1 second Ticker On Fri, 2020-01-31 at 16:15 +, Gerd Petermann wrote: please try this command with openjdk 11 or 13 64Bit: java -Xms4G -Xmx10G -jar mkgmap.jar f:\osm\dummy.osm f:\osm\dummy.osm f:\osm\dummy.osm This is what I get: Time started: Fri Jan 31 17:12:53 CET 2020 Setting max-jobs to 1 Number of MapFailedExceptions: 0 To reduce the run time, consider increasing the amnount of memory available for use by mkgmap by using the Java -Xmx flag to set the memory to more than 41000 MB, providing this is less than the amount of physical memory installed. Number of ExitExceptions: 0 Time finished: Fri Jan 31 17:12:54 CET 2020 Total time taken: 1 second ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev -- # Viele Grüße und 73 de Manfred Haiduk, DD8KQ e-mail mhai...@t-online.de dd...@gmx.de # ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev -- # Viele Grüße und 73 de Manfred Haiduk, DD8KQ e-mail mhai...@t-online.de dd...@gmx.de # ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Error in memory calculation
Hi Manfred, mkgmap contains code to estimate a suitable value for max-jobs if this parameter isn't set. This code fails and the result is that it believes that a first thread needed all available memory. This doesn't happen when the -Xms parameter in my sample call is removed, and it also doesn't happen with JRE 1.8 (both from OpenJDK and Oracle). I've not yet started to look at the details returned in class java.lang.management.MemoryPoolMXBean Gerd Von: mkgmap-dev im Auftrag von DD8KQ Gesendet: Mittwoch, 5. Februar 2020 21:16 An: mkgmap-dev@lists.mkgmap.org.uk Betreff: Re: [mkgmap-dev] Error in memory calculation Hi Gerd i'm not sure what you are looking for. In my build environment i'm using java 1.8.0_241 (64 bit) and as my laptop have 32 Gb of memory availiable my command line to build is like this : java -Xmx3m -XX:-UseGCOverheadLimit -ea -jar -Dlog.config=logging.properties "c:\Program Files (x86)\KartenTools\mkgmap\mkgmap.jar" and depending on which map i want to build, the occupied memory runs up nearly to 30Gb Am 05.02.2020 um 14:05 schrieb Ticker Berkin: > Hi Gerd > > I don't have a 64bit environment or that much memory. In my normal > environment, I get: > > $ java -Xms1500M -Xmx1540M -jar ../mkgmap.trunk/mkgmap.jar \ >dummy.osm dummy.osm dummy.osm > Time started: Wed Feb 05 12:56:28 GMT 2020 > Setting max-jobs to 4 > Number of MapFailedExceptions: 0 > Number of ExitExceptions: 0 > Time finished: Wed Feb 05 12:56:29 GMT 2020 > Total time taken: 1 second > > -Xmx1540M is the maximum value that allows Java to start up on my > machine. > > With a newer version I loaded in October, but it didn't have any > obvious advantages: > > $ /opt/jre1.8.0_231/bin/java -Xms1500M -Xmx1540M \ >-jar ../mkgmap.trunk/mkgmap.jar \ >dummy.osm dummy.osm dummy.osm > Time started: Wed Feb 05 12:58:29 GMT 2020 > Setting max-jobs to 4 > Number of MapFailedExceptions: 0 > Number of ExitExceptions: 0 > Time finished: Wed Feb 05 12:58:30 GMT 2020 > Total time taken: 1 second > > Ticker > > On Fri, 2020-01-31 at 16:15 +, Gerd Petermann wrote: >> please try this command with openjdk 11 or 13 64Bit: >> java -Xms4G -Xmx10G -jar mkgmap.jar f:\osm\dummy.osm f:\osm\dummy.osm >> f:\osm\dummy.osm >> This is what I get: >> Time started: Fri Jan 31 17:12:53 CET 2020 >> Setting max-jobs to 1 >> Number of MapFailedExceptions: 0 >> To reduce the run time, consider increasing the amnount of memory >> available for use by mkgmap by using the Java -Xmx flag to set the >> memory to more than 41000 MB, providing this is less than the amount >> of physical memory installed. >> Number of ExitExceptions: 0 >> Time finished: Fri Jan 31 17:12:54 CET 2020 >> Total time taken: 1 second > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev -- # Viele Grüße und 73 de Manfred Haiduk, DD8KQ e-mail mhai...@t-online.de dd...@gmx.de # ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Error in memory calculation
Hi Gerd i'm not sure what you are looking for. In my build environment i'm using java 1.8.0_241 (64 bit) and as my laptop have 32 Gb of memory availiable my command line to build is like this : java -Xmx3m -XX:-UseGCOverheadLimit -ea -jar -Dlog.config=logging.properties "c:\Program Files (x86)\KartenTools\mkgmap\mkgmap.jar" and depending on which map i want to build, the occupied memory runs up nearly to 30Gb Am 05.02.2020 um 14:05 schrieb Ticker Berkin: Hi Gerd I don't have a 64bit environment or that much memory. In my normal environment, I get: $ java -Xms1500M -Xmx1540M -jar ../mkgmap.trunk/mkgmap.jar \ dummy.osm dummy.osm dummy.osm Time started: Wed Feb 05 12:56:28 GMT 2020 Setting max-jobs to 4 Number of MapFailedExceptions: 0 Number of ExitExceptions: 0 Time finished: Wed Feb 05 12:56:29 GMT 2020 Total time taken: 1 second -Xmx1540M is the maximum value that allows Java to start up on my machine. With a newer version I loaded in October, but it didn't have any obvious advantages: $ /opt/jre1.8.0_231/bin/java -Xms1500M -Xmx1540M \ -jar ../mkgmap.trunk/mkgmap.jar \ dummy.osm dummy.osm dummy.osm Time started: Wed Feb 05 12:58:29 GMT 2020 Setting max-jobs to 4 Number of MapFailedExceptions: 0 Number of ExitExceptions: 0 Time finished: Wed Feb 05 12:58:30 GMT 2020 Total time taken: 1 second Ticker On Fri, 2020-01-31 at 16:15 +, Gerd Petermann wrote: please try this command with openjdk 11 or 13 64Bit: java -Xms4G -Xmx10G -jar mkgmap.jar f:\osm\dummy.osm f:\osm\dummy.osm f:\osm\dummy.osm This is what I get: Time started: Fri Jan 31 17:12:53 CET 2020 Setting max-jobs to 1 Number of MapFailedExceptions: 0 To reduce the run time, consider increasing the amnount of memory available for use by mkgmap by using the Java -Xmx flag to set the memory to more than 41000 MB, providing this is less than the amount of physical memory installed. Number of ExitExceptions: 0 Time finished: Fri Jan 31 17:12:54 CET 2020 Total time taken: 1 second ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev -- # Viele Grüße und 73 de Manfred Haiduk, DD8KQ e-mail mhai...@t-online.de dd...@gmx.de # ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Branch is_in ready for a first test
Hi, a path in a forest should be presented thik: __ | forest | ||= || | | The hole way will be draw thick. If you set a point at the polygoneborder and cut the way there, only the part is_in the forest will be draw thick. __ | forest | ||--00--- || | | greets Arndt Gerd Petermann < gpetermann_muenc...@hotmail.com> hat am 5. Februar 2020 um 09:51 geschrieben: Hi Ticker, I am lost in alternatives. I don't like the current solution and I also don't like my "three bit flags" solution. With the current code we can only distinguish 4 cases, your original list contained 6 cases, the newer 5. What should be changed in current code (r4228)? More or other method parameters? Please ignore the tuning idea. Most of this is only true in theory as rounding problems don't allow stop early unless a a real crossing (in-out or out-in) is detected. In reality the internal rounding of coordinates create something like a halo along the polygon edges. Results are unpredictable when a tested point is inside that halo unless it is exactly at the same position of a polygon node. Same position means it is either the identical node or has identical OSM coordinates. With polygons we have the special case shown with b13 and b14 in my example file is-in-hook-samples-v4.osm. I forgot to add cases where the target polygon is inside the tested polygon. Typically those would be tagging errors, e.g. landuse=residential inside a building=yes or a closed way with both tags. If I got you right you suggest to introduce more methods? As a reminder, these are the 6 different cases for a line: L1: all of the line is outside the polygon L2: some of the line is outside and the rest touches or runs along the polygon edge L3: all of the line runs along the polygon edge L4: some of the line is inside and the rest touches or runs along. L5: all of the line is inside the polygon L6: some is inside and some outside the polygon. Obviously some point is on the polygon edge but we don't care if runs along the edge. For case L3 the results of is_in(x,y,any) and is_in(x,y,all) are rather unpredictable, but more likely "false" is returned. For points we have - in theory - just three states: P1: point is inside polygon P2: point is on the edge of a polygon P3 point is outside The current code ignores the method parameter and the halo problem and returns true for P1 and P2 and false for P3. The current code treats polygons like lines but case L3 may return different results as the code tries to find out if the tested polygon is inside a hole. Gerd Von: mkgmap-dev < mkgmap-dev-boun...@lists.mkgmap.org.uk> im Auftrag von Ticker Berkin < rwb-mkg...@jagit.co.uk> Gesendet: Mittwoch, 5. Februar 2020 07:58 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Branch is_in ready for a first test Hi all I'm still of the opinion that it is better to specify a 'method' parameter rather than return 3 flags for the following reasons: - for polygons, it is only meaningful to need to know if ANY or ALL of the rule polygon is in the target. - for lines, it was thought better for the 'ALL' case to allow/ignore the line touching the edge, as long as the rest was IN. This tuning ability is lost unless keywords are used. - for points, I agree that returning one of the 3 flags seems to make sense, but I still maintain it is clearer to have methods that ask in/in-or-on/on rather than the equivalent test with a regexp for the on -or-on case. - for many methods, optimisation is possible, eg. 1/ the processing can stop as soon an element is found that determines the result. 2/ The target polygons can be processed one-by-one instead of joined together. - to express the line question any-in-or-on with a regexp is messy and
Re: [mkgmap-dev] Error in memory calculation
Sorry, wring thread... I'll try to find a better solution for 64 Bit OpenJDK. Gerd Von: mkgmap-dev im Auftrag von Gerd Petermann Gesendet: Mittwoch, 5. Februar 2020 16:26 An: Ticker Berkin; Development list for mkgmap Betreff: Re: [mkgmap-dev] Error in memory calculation Hi Ticker, OK, I think it's best when I change the code so that we have a method that computes the flags IN,ON,OUT, and maybe something else about polygons in holes. Input parameters are ElementQuadTree qt and the Element el. This is required anyway to allow unit tests. The code in IsInFunction.java should only implement the "user interface" and evaluates the result. I'd be happy when you create that part. Gerd Von: mkgmap-dev im Auftrag von Ticker Berkin Gesendet: Mittwoch, 5. Februar 2020 14:05 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Error in memory calculation Hi Gerd I don't have a 64bit environment or that much memory. In my normal environment, I get: $ java -Xms1500M -Xmx1540M -jar ../mkgmap.trunk/mkgmap.jar \ dummy.osm dummy.osm dummy.osm Time started: Wed Feb 05 12:56:28 GMT 2020 Setting max-jobs to 4 Number of MapFailedExceptions: 0 Number of ExitExceptions: 0 Time finished: Wed Feb 05 12:56:29 GMT 2020 Total time taken: 1 second -Xmx1540M is the maximum value that allows Java to start up on my machine. With a newer version I loaded in October, but it didn't have any obvious advantages: $ /opt/jre1.8.0_231/bin/java -Xms1500M -Xmx1540M \ -jar ../mkgmap.trunk/mkgmap.jar \ dummy.osm dummy.osm dummy.osm Time started: Wed Feb 05 12:58:29 GMT 2020 Setting max-jobs to 4 Number of MapFailedExceptions: 0 Number of ExitExceptions: 0 Time finished: Wed Feb 05 12:58:30 GMT 2020 Total time taken: 1 second Ticker On Fri, 2020-01-31 at 16:15 +, Gerd Petermann wrote: > please try this command with openjdk 11 or 13 64Bit: > java -Xms4G -Xmx10G -jar mkgmap.jar f:\osm\dummy.osm f:\osm\dummy.osm > f:\osm\dummy.osm > This is what I get: > Time started: Fri Jan 31 17:12:53 CET 2020 > Setting max-jobs to 1 > Number of MapFailedExceptions: 0 > To reduce the run time, consider increasing the amnount of memory > available for use by mkgmap by using the Java -Xmx flag to set the > memory to more than 41000 MB, providing this is less than the amount > of physical memory installed. > Number of ExitExceptions: 0 > Time finished: Fri Jan 31 17:12:54 CET 2020 > Total time taken: 1 second ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Error in memory calculation
Hi Ticker, OK, I think it's best when I change the code so that we have a method that computes the flags IN,ON,OUT, and maybe something else about polygons in holes. Input parameters are ElementQuadTree qt and the Element el. This is required anyway to allow unit tests. The code in IsInFunction.java should only implement the "user interface" and evaluates the result. I'd be happy when you create that part. Gerd Von: mkgmap-dev im Auftrag von Ticker Berkin Gesendet: Mittwoch, 5. Februar 2020 14:05 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Error in memory calculation Hi Gerd I don't have a 64bit environment or that much memory. In my normal environment, I get: $ java -Xms1500M -Xmx1540M -jar ../mkgmap.trunk/mkgmap.jar \ dummy.osm dummy.osm dummy.osm Time started: Wed Feb 05 12:56:28 GMT 2020 Setting max-jobs to 4 Number of MapFailedExceptions: 0 Number of ExitExceptions: 0 Time finished: Wed Feb 05 12:56:29 GMT 2020 Total time taken: 1 second -Xmx1540M is the maximum value that allows Java to start up on my machine. With a newer version I loaded in October, but it didn't have any obvious advantages: $ /opt/jre1.8.0_231/bin/java -Xms1500M -Xmx1540M \ -jar ../mkgmap.trunk/mkgmap.jar \ dummy.osm dummy.osm dummy.osm Time started: Wed Feb 05 12:58:29 GMT 2020 Setting max-jobs to 4 Number of MapFailedExceptions: 0 Number of ExitExceptions: 0 Time finished: Wed Feb 05 12:58:30 GMT 2020 Total time taken: 1 second Ticker On Fri, 2020-01-31 at 16:15 +, Gerd Petermann wrote: > please try this command with openjdk 11 or 13 64Bit: > java -Xms4G -Xmx10G -jar mkgmap.jar f:\osm\dummy.osm f:\osm\dummy.osm > f:\osm\dummy.osm > This is what I get: > Time started: Fri Jan 31 17:12:53 CET 2020 > Setting max-jobs to 1 > Number of MapFailedExceptions: 0 > To reduce the run time, consider increasing the amnount of memory > available for use by mkgmap by using the Java -Xmx flag to set the > memory to more than 41000 MB, providing this is less than the amount > of physical memory installed. > Number of ExitExceptions: 0 > Time finished: Fri Jan 31 17:12:54 CET 2020 > Total time taken: 1 second ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Error in memory calculation
Hi Gerd I don't have a 64bit environment or that much memory. In my normal environment, I get: $ java -Xms1500M -Xmx1540M -jar ../mkgmap.trunk/mkgmap.jar \ dummy.osm dummy.osm dummy.osm Time started: Wed Feb 05 12:56:28 GMT 2020 Setting max-jobs to 4 Number of MapFailedExceptions: 0 Number of ExitExceptions: 0 Time finished: Wed Feb 05 12:56:29 GMT 2020 Total time taken: 1 second -Xmx1540M is the maximum value that allows Java to start up on my machine. With a newer version I loaded in October, but it didn't have any obvious advantages: $ /opt/jre1.8.0_231/bin/java -Xms1500M -Xmx1540M \ -jar ../mkgmap.trunk/mkgmap.jar \ dummy.osm dummy.osm dummy.osm Time started: Wed Feb 05 12:58:29 GMT 2020 Setting max-jobs to 4 Number of MapFailedExceptions: 0 Number of ExitExceptions: 0 Time finished: Wed Feb 05 12:58:30 GMT 2020 Total time taken: 1 second Ticker On Fri, 2020-01-31 at 16:15 +, Gerd Petermann wrote: > please try this command with openjdk 11 or 13 64Bit: > java -Xms4G -Xmx10G -jar mkgmap.jar f:\osm\dummy.osm f:\osm\dummy.osm > f:\osm\dummy.osm > This is what I get: > Time started: Fri Jan 31 17:12:53 CET 2020 > Setting max-jobs to 1 > Number of MapFailedExceptions: 0 > To reduce the run time, consider increasing the amnount of memory > available for use by mkgmap by using the Java -Xmx flag to set the > memory to more than 41000 MB, providing this is less than the amount > of physical memory installed. > Number of ExitExceptions: 0 > Time finished: Fri Jan 31 17:12:54 CET 2020 > Total time taken: 1 second ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Branch is_in ready for a first test
Hi Gerd My idea would be have a list of methods per rule-context, eg: POLYGON: any, all LINE: all, all_in_or_on, all_on, any, any_in_or_on POINT: in, in_or_on, on until someone comes up with clearer names. These should be an enumeration in Java so they can be changed, aliased etc without needing to change the main code and checked as the function is parsed. The main logic operates as it does now, except LINE and POLYGON handling probably need to diverge, setting the 3 internal booleans IN/ON/OUT. Then, at the end, these are converted to a single boolean result according to requested method and the table a)..e) for LINEs in my previous mail POLYGON results should be based on: is ALL/ANY of the area of rule polygon in matching polygon(s). The issue of polygons with holes, esp. if the rule polygon is the hole in the target polygon are impossible to decide. I think one of your examples had a residential area where some of the major buildings where cut out from this area with a multi-polygon relation. Is the building in the residential area? Logical answer is no - there is no area shared between the two. Expected answer - yes if you know what a residential area and a building are. Optimisation can be performed for, eg, the any... methods, stopping as soon as the relevant IN or ON flag is set. Regarding cases L1-L6 for lines: L1 OUT L2 OUT ON L3 ON L4 ON IN L5IN L6 OUT ON IN My intention was to phrase the query, via the method, in terms like: Is any part of this line in the matching polygons? Is all of this line in matching polygons? Is this line an edge of matching polygons? rather than the 3 flags defining the relationship the particular line to the polygon, so is_in(any) matches L4/L5/L6 and is_in(all) matches L4/L5, etc (here, for LINEs, method 'any' is 'any-in' and 'all' is 'some-in-none-out') Also, at the same time, ignore the cases where an inner line touches the edge or an outer line touches the edge. This gives the 5 cases a) to e) Handling the question this way gets around problems like: > For case L3 the results of is_in(x,y,any) and is_in(x,y,all) are > rather unpredictable, but more likely "false" is returned. If the rule wants to do something with a line that bounds the polygon, it asks for is_in(all_on) For POINTs, the same rational applies, the rule asks for what it want to handle, eg is_in(in_or_on)=false to match a POI that outside the polygon. Ticker On Wed, 2020-02-05 at 08:51 +, Gerd Petermann wrote: > Hi Ticker, > > I am lost in alternatives. I don't like the current solution and I > also don't like my "three bit flags" solution. > With the current code we can only distinguish 4 cases, your original > list contained 6 cases, the newer 5. > What should be changed in current code (r4228)? More or other method > parameters? > > Please ignore the tuning idea. Most of this is only true in theory as > rounding problems don't allow stop early unless a a real crossing (in > -out or out-in) is detected. In reality the internal rounding of > coordinates create something like a halo along the polygon edges. > Results are unpredictable when a tested point is inside that halo > unless it is exactly at the same position of a polygon node. Same > position means it is either the identical node or has identical OSM > coordinates. > > With polygons we have the special case shown with b13 and b14 in my > example file is-in-hook-samples-v4.osm. I forgot to add cases where > the target polygon is inside the tested polygon. Typically those > would be tagging errors, e.g. landuse=residential inside a > building=yes or a closed way with both tags. > > If I got you right you suggest to introduce more methods? As a > reminder, these are the 6 different cases for a line: > L1: all of the line is outside the polygon > L2: some of the line is outside and the rest touches or runs along > the polygon edge > L3: all of the line runs along the polygon edge > L4: some of the line is inside and the rest touches or runs along. > L5: all of the line is inside the polygon > L6: some is inside and some outside the polygon. Obviously some point > is on the polygon edge but we don't care if runs along the edge. > > For case L3 the results of is_in(x,y,any) and is_in(x,y,all) are > rather unpredictable, but more likely "false" is returned. > > For points we have - in theory - just three states: > P1: point is inside polygon > P2: point is on the edge of a polygon > P3 point is outside > The current code ignores the method parameter and the halo problem > and returns true for P1 and P2 and false for P3. > > The current code treats polygons like lines but case L3 may return > different results as the code tries to find out if the tested polygon > is inside a hole. > > Gerd > > > Von: mkgmap-dev im Auftrag > von Ticker Berkin > Gesendet: Mittwoch, 5. Februar 2020 07:58 > An: Development list for mkgmap > Betreff: Re:
Re: [mkgmap-dev] mapnik typ file and barrier=wall
Hi all I've also just checked this. With a typ-file that ensures all lines in range 0x01..0x3f have a representation, Basecamp shows nothing for: 0x17 0x2c 0x30-3f whereas my eTrex devices show them all. I can re-assign barrier to 0x2d, and I'd planned to use 0x2c for "Course of old Railway", but, for these 2, Joris says: "breaks car-routing". @joris what are the circumstances where this happens? These will be non -routing lines - will this cause the problem? The other possibilities for these 2 are 0x2a/b, which gpsMapEdit call Marine Boundary/Marine Hazard. Then we've used up lines that Basecamp can show in what I hope is the non-routing range. There are still some unused between 0x0d .. 0x13. Concerning the range 0x40-0xFF: 0x51... look the same as 0x01... but with a direction label (eg NW/S/...) on the line on my eTrex. However code in mkgmap uses the top 2 bits of the typ in some circumstances (for direction and long -encoding) and forcing them to be set crashes mkgmap. I'm not sure if this is a defined Garmin usage and I gave up investigating. Ticker On Mon, 2020-02-03 at 19:23 +, Gerd Petermann wrote: > OK, > > let's wait for Tickers response. IIGTR the rendering of barriers was > introduced with r4279: > http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap=4279 > > Gerd > > > Von: mkgmap-dev im Auftrag > von Joris Bo > Gesendet: Montag, 3. Februar 2020 19:23 > An: Development list for mkgmap > Betreff: Re: [mkgmap-dev] mapnik typ file and barrier=wall > > Hi > > To share other (i don't know why) issues I once encountered. > > 0x17Does not show a line. Don't use : maybe only be used as > transparent overlay feature > 0x2cIf used: Lines are visible but breaks car-routing > 0x2dIf used: Lines are visible but breaks car-routing > 0x10201 Don't use > 0x10202 Don't use > 0x10b05 to 0x10b1f No lines visible > > > > Kind regards > Joris > > > -Oorspronkelijk bericht- > Van: mkgmap-dev Namens Joris > Bo > Verzonden: maandag 3 februari 2020 19:16 > Aan: Development list for mkgmap > Onderwerp: Re: [mkgmap-dev] mapnik typ file and barrier=wall > > Hi > > A comment line in my sources says: 20181219 Don't know why but lines > are not visible. Do not use > > > > -Oorspronkelijk bericht- > Van: mkgmap-dev Namens Gerd > Petermann > Verzonden: zondag 2 februari 2020 09:51 > Aan: mkgmap-dev@lists.mkgmap.org.uk > Onderwerp: [mkgmap-dev] mapnik typ file and barrier=wall > > Hi TYP experts, > > it seems that line type 0x17 (barrier=wall) is not visible in > Basecamp? > > Gerd > > > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev > ___ > mkgmap-dev mailing list > mkgmap-dev@lists.mkgmap.org.uk > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Branch is_in ready for a first test
Hi Ticker, I am lost in alternatives. I don't like the current solution and I also don't like my "three bit flags" solution. With the current code we can only distinguish 4 cases, your original list contained 6 cases, the newer 5. What should be changed in current code (r4228)? More or other method parameters? Please ignore the tuning idea. Most of this is only true in theory as rounding problems don't allow stop early unless a a real crossing (in-out or out-in) is detected. In reality the internal rounding of coordinates create something like a halo along the polygon edges. Results are unpredictable when a tested point is inside that halo unless it is exactly at the same position of a polygon node. Same position means it is either the identical node or has identical OSM coordinates. With polygons we have the special case shown with b13 and b14 in my example file is-in-hook-samples-v4.osm. I forgot to add cases where the target polygon is inside the tested polygon. Typically those would be tagging errors, e.g. landuse=residential inside a building=yes or a closed way with both tags. If I got you right you suggest to introduce more methods? As a reminder, these are the 6 different cases for a line: L1: all of the line is outside the polygon L2: some of the line is outside and the rest touches or runs along the polygon edge L3: all of the line runs along the polygon edge L4: some of the line is inside and the rest touches or runs along. L5: all of the line is inside the polygon L6: some is inside and some outside the polygon. Obviously some point is on the polygon edge but we don't care if runs along the edge. For case L3 the results of is_in(x,y,any) and is_in(x,y,all) are rather unpredictable, but more likely "false" is returned. For points we have - in theory - just three states: P1: point is inside polygon P2: point is on the edge of a polygon P3 point is outside The current code ignores the method parameter and the halo problem and returns true for P1 and P2 and false for P3. The current code treats polygons like lines but case L3 may return different results as the code tries to find out if the tested polygon is inside a hole. Gerd Von: mkgmap-dev im Auftrag von Ticker Berkin Gesendet: Mittwoch, 5. Februar 2020 07:58 An: Development list for mkgmap Betreff: Re: [mkgmap-dev] Branch is_in ready for a first test Hi all I'm still of the opinion that it is better to specify a 'method' parameter rather than return 3 flags for the following reasons: - for polygons, it is only meaningful to need to know if ANY or ALL of the rule polygon is in the target. - for lines, it was thought better for the 'ALL' case to allow/ignore the line touching the edge, as long as the rest was IN. This tuning ability is lost unless keywords are used. - for points, I agree that returning one of the 3 flags seems to make sense, but I still maintain it is clearer to have methods that ask in/in-or-on/on rather than the equivalent test with a regexp for the on -or-on case. - for many methods, optimisation is possible, eg. 1/ the processing can stop as soon an element is found that determines the result. 2/ The target polygons can be processed one-by-one instead of joined together. - to express the line question any-in-or-on with a regexp is messy and obscure. In the java coding of the function, I expect it to use flags like IN/ON/OUT, and it is trivial for the Java to convert these, in conjunction with the 'method', to a boolean result that is easily handled at the rule level. - negation of the function is trivial when it returns a boolean. - method keyword is more readable (and writeable) than bitstring regex test. - the method keyword allows extendability, eg 1/ different accuracy requirements, 2/ magic keywords that could split the rule object into the parts that return true and the parts that return false. - Mike's idea of 'coincident' - see later. I don't thing we should consider line/polygon splitting at the moment. @jan, With my table: simplified a) some-in-none-out(all) IN and not OUT b) all-in-or-on (IN or ON) and not OUT not OUT c) all-onON and not (OUT or IN) not (OUT or IN) d) any-in(any) IN e) any-in-or-on IN or ON I was attempting to show precisely the meaningful line cases in terms of the flags, which I hoped to remain hidden. Without the method keyword, you'd have to implement the equivalent for the cases you required with a regexp to test the flags. Mike Baggaley, on 16th Jan, suggested the following keywords; I've added a transliteration of his description of how these correspond to the flags: all_inside IN and not (ON or OUT) a) touchingIN and not OUT b) all_touchingnot OUT d) some_inside IN e) some_touching IN or ON coincident all points of rule object match the target polygon I think the use of
