Re: [OSM-talk] Serious JOSM performance degradation
On Sat, 2017-11-11 at 17:21 -0500, James wrote: > have you tried with wireframe mode(ctrl+w)? Hi, it appears to be an autocomplete issue so this shouldn't have an impact. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Serious JOSM performance degradation
2017-11-11 23:09 GMT+01:00 Jan Martinec : > You can download somewhat-recent JOSM versions from > https://josm.openstreetmap.de/download/ , I do see 12921 there. Also > josm-latest, which is the testing version (currently 13101). > I can recommend this script to download and start JOSM, particularly if you are working with "latest" versions, because it caches a configurable number of older versions, and makes it easy to use one of those in case you experience problems with latest: https://wiki.openstreetmap.org/wiki/User:Cobra/JOSM-script Cheers, Martin ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Serious JOSM performance degradation
On Mon, 2017-11-13 at 20:53 +, Bob Hawkins wrote: > I posted a topic on this matter in OpenStreetMap Forum>Editors on the > very same day as this thread was started, by coincidence, and > directed to this mailing list by SomeoneElse. I received helpful > replies and believe I have succeeded in overcoming the slow responses > we were experiencing as a result. My reply is here: https://forum.op > enstreetmap.org/viewtopic.php?id=60403. I should be interested to > learn if it helps others. Hi, it appears the issue I reported was fixed in the development version: https://josm.openstreetmap.de/ticket/15547 The forum post you've linked to seems to mention memory issues. I don't recall experiencing those. I wonder if there are two separate issues, or if I simply haven't noticed memory issues. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Serious JOSM performance degradation
I posted a topic on this matter in OpenStreetMap Forum>Editors on the very same day as this thread was started, by coincidence, and directed to this mailing list by SomeoneElse. I received helpful replies and believe I have succeeded in overcoming the slow responses we were experiencing as a result. My reply is here: https://forum.openstreetmap.org/viewtopic.php?id=60403. I should be interested to learn if it helps others. --- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Serious JOSM performance degradation
Ticket: https://josm.openstreetmap.de/ticket/15547 ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Serious JOSM performance degradation
> MB can be vague nowadays, sometimes people use it when they mean > 1000^2, and sometimes it means 1024^2. MiB is explicitly 1024^2. > (lowercase mb is even more vague as it could also mean Mega-bit, > which > is 256 bytes). > Sorry, 131072 bytes. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Serious JOSM performance degradation
It has something to do with the difference between 10**3 and 2**10 2017-11-11 23:21 GMT+01:00 john whelan : > >but it is not widely acknowledged within the industry or media. > > That probably explains why in fifty years of working with computers I > hadn't come across the term. > > Cheerio John > > On 11 Nov 2017 5:03 pm, "Daniel Koć" wrote: > >> W dniu 11.11.2017 o 22:55, john whelan pisze: >> >> >17MiB >> >> Interesting file size. What is a MiB? >> >> >> https://en.wikipedia.org/wiki/Mebibyte >> >> -- >> "My method is uncertain/ It's a mess but it's working" [F. Apple] >> >> >> ___ >> talk mailing list >> talk@openstreetmap.org >> https://lists.openstreetmap.org/listinfo/talk >> >> > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk > > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Serious JOSM performance degradation
On Sat, 2017-11-11 at 22:57 +0100, mmd wrote: > I can reproduce this locally on JOSM 13106. According to > cpu jvisualvm > profiling (on Oracle JDK), most time is spent in method > org.openstreetmap.josm.data.tagging.ac.AutoCompletionSet.add. > > Best bet would be to create an issue on > https://josm.openstreetmap.de/newticket Thank you! So it's autocomplete as I suspected. I will file a ticket. Is there a way to disable autocomplete in the meantime? On Sat, 2017-11-11 at 22:27 +0100, Jo wrote: > When working with public transport, the PT_Assistant plugin might be > interesting for you. On the other hand, it might cause additional > overhead. Seems useful! On Sat, 2017-11-11 at 23:09 +0100, Jan Martinec wrote: > You can download somewhat-recent JOSM versions from https://josm. > openstreetmap.de/download/ , I do see 12921 there. Also josm-latest, > which > is the testing version (currently 13101). Thank you. > Interesting file size. What is a MiB? > > If you are working with large file sizes and using the results > elsewhere > you should be using ECC memory. Anything else and you stand a chance > of > corruption. > > How much memory does your laptop have? MiB is practically the same as MB in for the needs of this conversation. MB can be vague nowadays, sometimes people use it when they mean 1000^2, and sometimes it means 1024^2. MiB is explicitly 1024^2. (lowercase mb is even more vague as it could also mean Mega-bit, which is 256 bytes). I am not working with astronomically huge memory, just a mundane Overpass fetch-edit*-upload cycle. But perhaps it's all been smooth because the dataset is entirely nodes, and no ways or relations. The laptop is 4 GiB. Pretty average hardware. *The edit is actually several custom GTFS scripts followed by some manual touches but that's irrelevant for the performance issue. For anyone interested in what I do with 30k stops: https://wiki.openstreetmap.org/wiki/User:SafwatHalaby/scripts/gtfs ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Serious JOSM performance degradation
>but it is not widely acknowledged within the industry or media. That probably explains why in fifty years of working with computers I hadn't come across the term. Cheerio John On 11 Nov 2017 5:03 pm, "Daniel Koć" wrote: > W dniu 11.11.2017 o 22:55, john whelan pisze: > > >17MiB > > Interesting file size. What is a MiB? > > > https://en.wikipedia.org/wiki/Mebibyte > > -- > "My method is uncertain/ It's a mess but it's working" [F. Apple] > > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk > > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Serious JOSM performance degradation
have you tried with wireframe mode(ctrl+w)? On Nov 11, 2017 5:11 PM, "Jan Martinec" wrote: > You can download somewhat-recent JOSM versions from > https://josm.openstreetmap.de/download/ , I do see 12921 there. Also > josm-latest, which is the testing version (currently 13101). > > Cheers, > Honza "Piškvor" Martinec > > Dne 11. 11. 2017 22:15 napsal uživatel "john whelan" < > jwhelan0...@gmail.com>: > > If you give it more memory sometimes that compensates on the CPU side by > not requiring so much disk reading and writing. Also there are almost > certainly internal tables which work better in memory. JAVA is a strange > world of its own and with so many contributors and plugins I'm not sure > anyone has a clear idea of exactly how JOSM works with in it. > > Not guaranteed but worth a try. > > Cheerio John > > On 11 November 2017 at 16:03, Safwat Halaby wrote: > >> More details: >> >> JOSM 12712 has no issues. >> Latest JOSM 13053 has issues. >> Not sure how to download the intermediate version (12921). >> It's a CPU starvation issue and not a memory overuse issue. (Core at >> 100% for 10-30 seconds). >> >> Test case: >> >> Open up the attached file, try editing the ref or name of a stop. >> >> 12712 works flawlessly. >> 13053 hangs for 10-30 seconds. > > > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk > > > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk > > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Serious JOSM performance degradation
You can download somewhat-recent JOSM versions from https://josm. openstreetmap.de/download/ , I do see 12921 there. Also josm-latest, which is the testing version (currently 13101). Cheers, Honza "Piškvor" Martinec Dne 11. 11. 2017 22:15 napsal uživatel "john whelan" : If you give it more memory sometimes that compensates on the CPU side by not requiring so much disk reading and writing. Also there are almost certainly internal tables which work better in memory. JAVA is a strange world of its own and with so many contributors and plugins I'm not sure anyone has a clear idea of exactly how JOSM works with in it. Not guaranteed but worth a try. Cheerio John On 11 November 2017 at 16:03, Safwat Halaby wrote: > More details: > > JOSM 12712 has no issues. > Latest JOSM 13053 has issues. > Not sure how to download the intermediate version (12921). > It's a CPU starvation issue and not a memory overuse issue. (Core at > 100% for 10-30 seconds). > > Test case: > > Open up the attached file, try editing the ref or name of a stop. > > 12712 works flawlessly. > 13053 hangs for 10-30 seconds. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Serious JOSM performance degradation
W dniu 11.11.2017 o 22:55, john whelan pisze: >17MiB Interesting file size. What is a MiB? https://en.wikipedia.org/wiki/Mebibyte -- "My method is uncertain/ It's a mess but it's working" [F. Apple] ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Serious JOSM performance degradation
Am 11.11.2017 um 22:25 schrieb Safwat Halaby: > Apparently one of my messages did not get through due to a huge file. > Here it is once more: > > More details: > > JOSM 12712 has no issues. > Latest JOSM 13053 has issues. > Not sure how to download the intermediate version (12921). > It's a CPU starvation issue and not a memory overuse issue. (Core at > 100% for 10-30 seconds). > > Test case: > Open up the attached file, try editing the ref or name of a stop. > 12712 works flawlessly. > 13053 hangs for 10-30 seconds. > I can reproduce this locally on JOSM 13106. According to jvisualvm cpu profiling (on Oracle JDK), most time is spent in method org.openstreetmap.josm.data.tagging.ac.AutoCompletionSet.add. Best bet would be to create an issue on https://josm.openstreetmap.de/newticket -- ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Serious JOSM performance degradation
>17MiB Interesting file size. What is a MiB? If you are working with large file sizes and using the results elsewhere you should be using ECC memory. Anything else and you stand a chance of corruption. How much memory does your laptop have? Thanks John On 11 Nov 2017 4:48 pm, "Safwat Halaby" wrote: > That's interestingly low. I'm talking about a 17MiB file here, and it > was never slugish before this bug. My laptop is an average thinkpad... > > On Sat, 2017-11-11 at 15:52 -0500, john whelan wrote: > > java -Xmx821000 -jar G:\josm\josm-tested.jar > > > > in a .bat file seems to be happy with 2Mb files. It can be sluggish > > though > > so I try to keep it below 250kb for best performance. > > > > Cheerio John > > > > On 11 November 2017 at 15:43, Safwat Halaby > > wrote: > > > > > I deal with a huge dataset (about 30k bus stops nodes) almost > > > daily. > > > This was never a problem before, but since recently (I cannot > > > pinpoint > > > the exact version), JOSM often hangs when I edit the bus stop tags. > > > Waiting for a few seconds (can be up to ~30) always resolves this. > > > > > > I suspect it has to do with JOSM's autocomplete, because it seems > > > to > > > happen when editing massively used keys. For instance, all stops > > > have a > > > "ref" key, and when I click "edit" on a single stop's "ref", JOSM > > > stalls. A similar issue occurs when adding "description" to > > > changeset > > > uploads. Description is also used by all stops. > > > > > > Any tips or pointers? > > > > > > ___ > > > talk mailing list > > > talk@openstreetmap.org > > > https://lists.openstreetmap.org/listinfo/talk > > > > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Serious JOSM performance degradation
That's interestingly low. I'm talking about a 17MiB file here, and it was never slugish before this bug. My laptop is an average thinkpad... On Sat, 2017-11-11 at 15:52 -0500, john whelan wrote: > java -Xmx821000 -jar G:\josm\josm-tested.jar > > in a .bat file seems to be happy with 2Mb files. It can be sluggish > though > so I try to keep it below 250kb for best performance. > > Cheerio John > > On 11 November 2017 at 15:43, Safwat Halaby > wrote: > > > I deal with a huge dataset (about 30k bus stops nodes) almost > > daily. > > This was never a problem before, but since recently (I cannot > > pinpoint > > the exact version), JOSM often hangs when I edit the bus stop tags. > > Waiting for a few seconds (can be up to ~30) always resolves this. > > > > I suspect it has to do with JOSM's autocomplete, because it seems > > to > > happen when editing massively used keys. For instance, all stops > > have a > > "ref" key, and when I click "edit" on a single stop's "ref", JOSM > > stalls. A similar issue occurs when adding "description" to > > changeset > > uploads. Description is also used by all stops. > > > > Any tips or pointers? > > > > ___ > > talk mailing list > > talk@openstreetmap.org > > https://lists.openstreetmap.org/listinfo/talk > > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Serious JOSM performance degradation
When working with public transport, the PT_Assistant plugin might be interesting for you. On the other hand, it might cause additional overhead. Jo 2017-11-11 22:23 GMT+01:00 Safwat Halaby : > I don't know what these are. > > The test described below was done without plugins, by the way. > > On Sat, 2017-11-11 at 22:14 +0100, Jo wrote: > > Did you start using PT_Assistant? Or a MapCSS style dedicated to PT? > > > > Polyglot > > > > 2017-11-11 22:10 GMT+01:00 john whelan : > > > > > If you give it more memory sometimes that compensates on the CPU > > > side by > > > not requiring so much disk reading and writing. Also there are > > > almost > > > certainly internal tables which work better in memory. JAVA is a > > > strange > > > world of its own and with so many contributors and plugins I'm not > > > sure > > > anyone has a clear idea of exactly how JOSM works with in it. > > > > > > Not guaranteed but worth a try. > > > > > > Cheerio John > > > > > > On 11 November 2017 at 16:03, Safwat Halaby > > > wrote: > > > > > > > More details: > > > > > > > > JOSM 12712 has no issues. > > > > Latest JOSM 13053 has issues. > > > > Not sure how to download the intermediate version (12921). > > > > It's a CPU starvation issue and not a memory overuse issue. (Core > > > > at > > > > 100% for 10-30 seconds). > > > > > > > > Test case: > > > > > > > > Open up the attached file, try editing the ref or name of a stop. > > > > > > > > 12712 works flawlessly. > > > > 13053 hangs for 10-30 seconds. > > > > > > > > > > > > ___ > > > talk mailing list > > > talk@openstreetmap.org > > > https://lists.openstreetmap.org/listinfo/talk > > > > > > > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Serious JOSM performance degradation
Apparently one of my messages did not get through due to a huge file. Here it is once more: More details: JOSM 12712 has no issues. Latest JOSM 13053 has issues. Not sure how to download the intermediate version (12921). It's a CPU starvation issue and not a memory overuse issue. (Core at 100% for 10-30 seconds). Test case: Open up the attached file, try editing the ref or name of a stop. 12712 works flawlessly. 13053 hangs for 10-30 seconds. original message ends here. As for the file, I cannot attach it, but use this Overpass query to obtain the same dataset: [out:xml][timeout:90][bbox:29.4013195,33.8818359,33.4131022,36.0791016] ; ( area(3601473946); // Israel area(3601803010); // Judea and Samaria district )->.a; ( node["highway"="bus_stop"](area.a); way["highway"="bus_stop"](area.a); )->.b; (rel(bw.b);rel(bn.b))->.routes; (.b;way(bn.b);)->.b; (.b;node(w.b);)->.b; (.b;.routes;); out meta; ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Serious JOSM performance degradation
I don't know what these are. The test described below was done without plugins, by the way. On Sat, 2017-11-11 at 22:14 +0100, Jo wrote: > Did you start using PT_Assistant? Or a MapCSS style dedicated to PT? > > Polyglot > > 2017-11-11 22:10 GMT+01:00 john whelan : > > > If you give it more memory sometimes that compensates on the CPU > > side by > > not requiring so much disk reading and writing. Also there are > > almost > > certainly internal tables which work better in memory. JAVA is a > > strange > > world of its own and with so many contributors and plugins I'm not > > sure > > anyone has a clear idea of exactly how JOSM works with in it. > > > > Not guaranteed but worth a try. > > > > Cheerio John > > > > On 11 November 2017 at 16:03, Safwat Halaby > > wrote: > > > > > More details: > > > > > > JOSM 12712 has no issues. > > > Latest JOSM 13053 has issues. > > > Not sure how to download the intermediate version (12921). > > > It's a CPU starvation issue and not a memory overuse issue. (Core > > > at > > > 100% for 10-30 seconds). > > > > > > Test case: > > > > > > Open up the attached file, try editing the ref or name of a stop. > > > > > > 12712 works flawlessly. > > > 13053 hangs for 10-30 seconds. > > > > > > > > ___ > > talk mailing list > > talk@openstreetmap.org > > https://lists.openstreetmap.org/listinfo/talk > > > > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Serious JOSM performance degradation
Did you start using PT_Assistant? Or a MapCSS style dedicated to PT? Polyglot 2017-11-11 22:10 GMT+01:00 john whelan : > If you give it more memory sometimes that compensates on the CPU side by > not requiring so much disk reading and writing. Also there are almost > certainly internal tables which work better in memory. JAVA is a strange > world of its own and with so many contributors and plugins I'm not sure > anyone has a clear idea of exactly how JOSM works with in it. > > Not guaranteed but worth a try. > > Cheerio John > > On 11 November 2017 at 16:03, Safwat Halaby wrote: > >> More details: >> >> JOSM 12712 has no issues. >> Latest JOSM 13053 has issues. >> Not sure how to download the intermediate version (12921). >> It's a CPU starvation issue and not a memory overuse issue. (Core at >> 100% for 10-30 seconds). >> >> Test case: >> >> Open up the attached file, try editing the ref or name of a stop. >> >> 12712 works flawlessly. >> 13053 hangs for 10-30 seconds. > > > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk > > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Serious JOSM performance degradation
If you give it more memory sometimes that compensates on the CPU side by not requiring so much disk reading and writing. Also there are almost certainly internal tables which work better in memory. JAVA is a strange world of its own and with so many contributors and plugins I'm not sure anyone has a clear idea of exactly how JOSM works with in it. Not guaranteed but worth a try. Cheerio John On 11 November 2017 at 16:03, Safwat Halaby wrote: > More details: > > JOSM 12712 has no issues. > Latest JOSM 13053 has issues. > Not sure how to download the intermediate version (12921). > It's a CPU starvation issue and not a memory overuse issue. (Core at > 100% for 10-30 seconds). > > Test case: > > Open up the attached file, try editing the ref or name of a stop. > > 12712 works flawlessly. > 13053 hangs for 10-30 seconds. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Serious JOSM performance degradation
java -Xmx821000 -jar G:\josm\josm-tested.jar in a .bat file seems to be happy with 2Mb files. It can be sluggish though so I try to keep it below 250kb for best performance. Cheerio John On 11 November 2017 at 15:43, Safwat Halaby wrote: > I deal with a huge dataset (about 30k bus stops nodes) almost daily. > This was never a problem before, but since recently (I cannot pinpoint > the exact version), JOSM often hangs when I edit the bus stop tags. > Waiting for a few seconds (can be up to ~30) always resolves this. > > I suspect it has to do with JOSM's autocomplete, because it seems to > happen when editing massively used keys. For instance, all stops have a > "ref" key, and when I click "edit" on a single stop's "ref", JOSM > stalls. A similar issue occurs when adding "description" to changeset > uploads. Description is also used by all stops. > > Any tips or pointers? > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Serious JOSM performance degradation
On Sat, 2017-11-11 at 22:43 +0200, Safwat Halaby wrote: > I deal with a huge dataset (about 30k bus stops nodes) almost daily. > This was never a problem before, but since recently (I cannot > pinpoint > the exact version), JOSM often hangs when I edit the bus stop tags. > Waiting for a few seconds (can be up to ~30) always resolves this. > > I suspect it has to do with JOSM's autocomplete, because it seems to > happen when editing massively used keys. For instance, all stops have > a > "ref" key, and when I click "edit" on a single stop's "ref", JOSM > stalls. A similar issue occurs when adding "description" to changeset > uploads. Description is also used by all stops. > > Any tips or pointers? > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk I will soon test this out with various versions. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk