Re: [Talk-il] Hebrew maps for Garmin
On Tue, 2018-02-06 at 01:51 +0300, Dov Ber Ackerman wrote: > Hello I am trying to find an updated map of Israel for a Garmin > device for > a friend, but they are all in English so the hebrew search does not > work.!! > Are there any recently (last 2 years etc.) updated maps for Israel > that > would work or can be converted to an .img file? > > Thanks in advance, > > Dov Ber > ___ > Talk-il mailing list > Talk-il@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk-il I would advice using the forum, it is much more active and you are more likely to get a reply: https://forum.openstreetmap.org/viewforum.php?id =33 ___ Talk-il mailing list Talk-il@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-il
Re: [OSM-talk] Serious JOSM performance degradation
On Sat, 2017-11-11 at 17:21 -0500, James wrote: > have you tried with wireframe mode(ctrl+w)? Hi, it appears to be an autocomplete issue so this shouldn't have an impact. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Serious JOSM performance degradation
On Mon, 2017-11-13 at 20:53 +, Bob Hawkins wrote: > I posted a topic on this matter in OpenStreetMap Forum>Editors on the > very same day as this thread was started, by coincidence, and > directed to this mailing list by SomeoneElse. I received helpful > replies and believe I have succeeded in overcoming the slow responses > we were experiencing as a result. My reply is here: https://forum.op > enstreetmap.org/viewtopic.php?id=60403. I should be interested to > learn if it helps others. Hi, it appears the issue I reported was fixed in the development version: https://josm.openstreetmap.de/ticket/15547 The forum post you've linked to seems to mention memory issues. I don't recall experiencing those. I wonder if there are two separate issues, or if I simply haven't noticed memory issues. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Serious JOSM performance degradation
Ticket: https://josm.openstreetmap.de/ticket/15547 ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Serious JOSM performance degradation
> MB can be vague nowadays, sometimes people use it when they mean > 1000^2, and sometimes it means 1024^2. MiB is explicitly 1024^2. > (lowercase mb is even more vague as it could also mean Mega-bit, > which > is 256 bytes). > Sorry, 131072 bytes. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Serious JOSM performance degradation
On Sat, 2017-11-11 at 22:57 +0100, mmd wrote: > I can reproduce this locally on JOSM 13106. According to > cpu jvisualvm > profiling (on Oracle JDK), most time is spent in method > org.openstreetmap.josm.data.tagging.ac.AutoCompletionSet.add. > > Best bet would be to create an issue on > https://josm.openstreetmap.de/newticket Thank you! So it's autocomplete as I suspected. I will file a ticket. Is there a way to disable autocomplete in the meantime? On Sat, 2017-11-11 at 22:27 +0100, Jo wrote: > When working with public transport, the PT_Assistant plugin might be > interesting for you. On the other hand, it might cause additional > overhead. Seems useful! On Sat, 2017-11-11 at 23:09 +0100, Jan Martinec wrote: > You can download somewhat-recent JOSM versions from https://josm. > openstreetmap.de/download/ , I do see 12921 there. Also josm-latest, > which > is the testing version (currently 13101). Thank you. > Interesting file size. What is a MiB? > > If you are working with large file sizes and using the results > elsewhere > you should be using ECC memory. Anything else and you stand a chance > of > corruption. > > How much memory does your laptop have? MiB is practically the same as MB in for the needs of this conversation. MB can be vague nowadays, sometimes people use it when they mean 1000^2, and sometimes it means 1024^2. MiB is explicitly 1024^2. (lowercase mb is even more vague as it could also mean Mega-bit, which is 256 bytes). I am not working with astronomically huge memory, just a mundane Overpass fetch-edit*-upload cycle. But perhaps it's all been smooth because the dataset is entirely nodes, and no ways or relations. The laptop is 4 GiB. Pretty average hardware. *The edit is actually several custom GTFS scripts followed by some manual touches but that's irrelevant for the performance issue. For anyone interested in what I do with 30k stops: https://wiki.openstreetmap.org/wiki/User:SafwatHalaby/scripts/gtfs ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Serious JOSM performance degradation
That's interestingly low. I'm talking about a 17MiB file here, and it was never slugish before this bug. My laptop is an average thinkpad... On Sat, 2017-11-11 at 15:52 -0500, john whelan wrote: > java -Xmx821000 -jar G:\josm\josm-tested.jar > > in a .bat file seems to be happy with 2Mb files. It can be sluggish > though > so I try to keep it below 250kb for best performance. > > Cheerio John > > On 11 November 2017 at 15:43, Safwat Halaby <swiftf...@gmx.com> > wrote: > > > I deal with a huge dataset (about 30k bus stops nodes) almost > > daily. > > This was never a problem before, but since recently (I cannot > > pinpoint > > the exact version), JOSM often hangs when I edit the bus stop tags. > > Waiting for a few seconds (can be up to ~30) always resolves this. > > > > I suspect it has to do with JOSM's autocomplete, because it seems > > to > > happen when editing massively used keys. For instance, all stops > > have a > > "ref" key, and when I click "edit" on a single stop's "ref", JOSM > > stalls. A similar issue occurs when adding "description" to > > changeset > > uploads. Description is also used by all stops. > > > > Any tips or pointers? > > > > ___ > > talk mailing list > > talk@openstreetmap.org > > https://lists.openstreetmap.org/listinfo/talk > > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Serious JOSM performance degradation
Apparently one of my messages did not get through due to a huge file. Here it is once more: More details: JOSM 12712 has no issues. Latest JOSM 13053 has issues. Not sure how to download the intermediate version (12921). It's a CPU starvation issue and not a memory overuse issue. (Core at 100% for 10-30 seconds). Test case: Open up the attached file, try editing the ref or name of a stop. 12712 works flawlessly. 13053 hangs for 10-30 seconds. original message ends here. As for the file, I cannot attach it, but use this Overpass query to obtain the same dataset: [out:xml][timeout:90][bbox:29.4013195,33.8818359,33.4131022,36.0791016] ; ( area(3601473946); // Israel area(3601803010); // Judea and Samaria district )->.a; ( node["highway"="bus_stop"](area.a); way["highway"="bus_stop"](area.a); )->.b; (rel(bw.b);rel(bn.b))->.routes; (.b;way(bn.b);)->.b; (.b;node(w.b);)->.b; (.b;.routes;); out meta; ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Serious JOSM performance degradation
I don't know what these are. The test described below was done without plugins, by the way. On Sat, 2017-11-11 at 22:14 +0100, Jo wrote: > Did you start using PT_Assistant? Or a MapCSS style dedicated to PT? > > Polyglot > > 2017-11-11 22:10 GMT+01:00 john whelan <jwhelan0...@gmail.com>: > > > If you give it more memory sometimes that compensates on the CPU > > side by > > not requiring so much disk reading and writing. Also there are > > almost > > certainly internal tables which work better in memory. JAVA is a > > strange > > world of its own and with so many contributors and plugins I'm not > > sure > > anyone has a clear idea of exactly how JOSM works with in it. > > > > Not guaranteed but worth a try. > > > > Cheerio John > > > > On 11 November 2017 at 16:03, Safwat Halaby <swiftf...@gmx.com> > > wrote: > > > > > More details: > > > > > > JOSM 12712 has no issues. > > > Latest JOSM 13053 has issues. > > > Not sure how to download the intermediate version (12921). > > > It's a CPU starvation issue and not a memory overuse issue. (Core > > > at > > > 100% for 10-30 seconds). > > > > > > Test case: > > > > > > Open up the attached file, try editing the ref or name of a stop. > > > > > > 12712 works flawlessly. > > > 13053 hangs for 10-30 seconds. > > > > > > > > ___ > > talk mailing list > > talk@openstreetmap.org > > https://lists.openstreetmap.org/listinfo/talk > > > > ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Serious JOSM performance degradation
On Sat, 2017-11-11 at 22:43 +0200, Safwat Halaby wrote: > I deal with a huge dataset (about 30k bus stops nodes) almost daily. > This was never a problem before, but since recently (I cannot > pinpoint > the exact version), JOSM often hangs when I edit the bus stop tags. > Waiting for a few seconds (can be up to ~30) always resolves this. > > I suspect it has to do with JOSM's autocomplete, because it seems to > happen when editing massively used keys. For instance, all stops have > a > "ref" key, and when I click "edit" on a single stop's "ref", JOSM > stalls. A similar issue occurs when adding "description" to changeset > uploads. Description is also used by all stops. > > Any tips or pointers? > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk I will soon test this out with various versions. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
[OSM-talk] Serious JOSM performance degradation
I deal with a huge dataset (about 30k bus stops nodes) almost daily. This was never a problem before, but since recently (I cannot pinpoint the exact version), JOSM often hangs when I edit the bus stop tags. Waiting for a few seconds (can be up to ~30) always resolves this. I suspect it has to do with JOSM's autocomplete, because it seems to happen when editing massively used keys. For instance, all stops have a "ref" key, and when I click "edit" on a single stop's "ref", JOSM stalls. A similar issue occurs when adding "description" to changeset uploads. Description is also used by all stops. Any tips or pointers? ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Sudden influx of bad HOTosm edits in West Bank
I'm reviewing the fixes and adding changeset comments. The fixes are mostly good. It seems Hannah is occasionally adding area=yes on accident. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Sudden influx of bad HOTosm edits in West Bank
Hi, and thank you for the cleanup efforts! Please note that in some cases, people added buildings on top of buildings. Depending on your QA software, this can be hard to spot because it does not show up in diffs. > Have you already corrected or > reverted some of the errors that were noted? I reverted two serious cases: mdavenport1985(uid 6744070) was adding nonsense: https://www.openstreetmap.org/changeset/53501083 https://www.openstreetmap.org/changeset/53429550 The Data Working group stepped in to stop him: https://www.openstreetmap.org/user/mdavenport1985/blocks A_N_D_R_E_W(uid 6876174) was intentionally vandalizing: https://www.openstreetmap.org/changeset/53347778 I also reverted one non serious case of buildings on top of buildings: https://www.openstreetmap.org/changeset/53347357 ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Sudden influx of bad HOTosm edits in West Bank
On Sat, 2017-11-04 at 15:23 -0400, john whelan wrote: > You need to be given permission but having editted on OSM is not a > requirement. > > Cheerio John > > On 4 Nov 2017 3:20 pm, "Safwat Halaby" <swiftf...@gmx.com> wrote: > > > Can anyone create HOT OSM tasks? Is there an entry barrier? > > > > The creator of task http://tasks.hotosm.org/project/3441 only has 5 > > edits. > > > > ___ > > talk mailing list > > talk@openstreetmap.org > > https://lists.openstreetmap.org/listinfo/talk > > Thanks for the info! ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Sudden influx of bad HOTosm edits in West Bank
Can anyone create HOT OSM tasks? Is there an entry barrier? The creator of task http://tasks.hotosm.org/project/3441 only has 5 edits. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Sudden influx of bad HOTosm edits in West Bank
On Sun, 2017-10-29 at 15:47 -0700, Mark Wagner wrote: > On Sun, 29 Oct 2017 22:15:18 +0200 > Safwat Halaby <swiftf...@gmx.com> wrote: > > > - Adding closed ways with area=yes instead of building=yes, or with > > no > > tags at all > > A closed way with "area=yes" is a *very* common newbie mistake with > iD: > the user traced an area, then forgot to tag it, or didn't realize > they > needed to apply additional tags. > Yes but that's not the issue with those edits per se. For instance adding area=yes (or building=yes) on pre existing buildings en masse is not a very common newbie mistake. See my other messages for more details ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Sudden influx of bad HOTosm edits in West Bank
On Sun, 2017-10-29 at 20:13 +0100, Blake Girardot HOT/OSM wrote: > Greetings, > > I SomeoneElse mentioned this in our HOT IRC channel. > > I have already asked the project creator to take a look. > > But, are you sure they are bad edits? did you use the 2016 imagery > specified in the mapping projet? > > For example, change set 53330616, the first one i randomly looked at > from you list looks like bad editing until you use the 2016 imagery, > please see the linked to two images below, one with Bing/DG Premium > (they > look the same) and one with the imagery supplied with the project. > > So, please let us not rush to revert if you are not using the > proper imagery. > > As I said, I will or someone else from HOT will look into it further, > thank you very much for bringing it to our attention, as well as the > other issues you pointed out, which I have not had a chance to look > at > yet. > > https://screenpresso.com/=l5lhb (old bing) > > https://screenpresso.com/=TPYpg (new aerial) I apologize for spamming the list with multiple split messages. Had some technical issues. Last message in the series: I'm afraid I suspect you're not using QA tools properly. Specifically in JOSM, click the MIDDLE button to to reveal old buildings buried beneath the newer buildings. Also, JOSM will not show you when the users deleted buildings and then re-added them. You'll only see the new buildings. Also, these are not your typical newbie errors. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Sudden influx of bad HOTosm edits in West Bank
On Sun, 2017-10-29 at 20:10 +0100, Blake Girardot HOT/OSM wrote: > But, are you sure they are bad edits? did you use the 2016 imagery > specified in the mapping projet? > > For example, change set 53330616, the first one i randomly looked at > from you list looks like bad editing until you use the 2016 imagery, > please see the attached two images, one with Bing/DG Premium (they > look the same) and one with the imagery supplied with the project. This point has been repeated so I'll say one last time that this has nothing to do with imagery alignment. It has to do with destroying previous buildings and re-adding them, or adding new buildings on top of them without destroying them, or other various weird edits. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Sudden influx of bad HOTosm edits in West Bank
On Sun, 2017-10-29 at 22:24 +0100, Blake Girardot wrote: > Safwat, > > You are not helping things by unilaterally reverting things and > accusing people of vandalism after you asked HOT to look into it. > > At this point all I have seen from your first list was some newbie > edits with some newbie mistakes and some stuff that looked totally > fine when using the correct imagery. > > and before you started reverting all of the andrew users work, I saw > decent mapping and bad tagging from a new mapper. clearly not > vandalism. > > Granted I am not the best at reviewing things by changeset so I might > have missed something, that is totally possible. > > At this point I am going to stop looking into it since you have > decided to take matters into your own hands with your own solutions. > > Regards, > > Blake > > > On Sun, Oct 29, 2017 at 9:15 PM, Safwat Halaby <swiftf...@gmx.com> > wrote: > > > > Due to a mailing client config error, I'm unsure if I sent my > > messages privately or to the list (or at all). At the risk of > > repeating > > myself: > > > > The problems are not related to Imagery at all. They're blatantly > > obvious mapping issues like: > > - removing things and re-adding them for no apparent reason > > - Adding buildings on top of pre-existing buildings > > - Adding closed ways with area=yes instead of building=yes, or with > > no > > tags at all > > - Intentional vandalism in some cases. > > > > ___ > > talk mailing list > > talk@openstreetmap.org > > https://lists.openstreetmap.org/listinfo/talk > > > Dear blake, I'm afraid you're misunderstanding this. I've not touched anything since contacting the DWG. I've done exactly 2 reverts prior to contacting them. Revert 1: The user has added duplicate buildings on top of already existing buildings, except that those duplicate buildings had no building=yes tag. I have not accused that user of vandalism and simply reverted. I then realized this is not an isolated case, so left things for the DWG. https://www.openstreetmap.org/changeset/53347357 Revert 2: User "Andrew" added things like "world's biggest refrigerator", "WHEEL OF FORTUNE", "ANDREWS BIGGEST CAR", "Concrete Thing", etc. This is blatantly obvious, high priority, destructive vandalism. Sure, the user may have done some good changes in between, but why should I bother spending time bisecting the edits of a vandalist? In cases like this, you revert and ask questions later, and perhaps later extract the good bits if you've got the time to work through the junk. https://www.openstreetmap.org/changeset/53347778 ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] weeklyOSM - Mapbox updates
On Sun, 2017-10-29 at 18:22 +, Andy Townsend wrote: > On 29/10/2017 18:09, Safwat Halaby wrote: > > Hmm, around the same times the Mapbox updates > > stopped[1], OsmCHA[2] > > stopped showing new changesets and I reported[3] that. Perhaps it > > was > > an unintended side-effect of whatever Mapbox did? > > That's something at your end: I think https://osmcha.mapbox.com/ > shows > updates from "2 minutes ago" for me. The filters (e.g. for searching > by > HOT task number) have also been working. > > Best Regards, > > Andy > > > ___ > talk mailing list > talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk There were a couple of days of no updates, but that issue has since been acknowledged and fixed. It is all good in the present time. See: https://lists.openstreetmap.org/pipermail/talk/2017-October/079053.html ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Sudden influx of bad HOTosm edits in West Bank
Due to a mailing client config error, I'm unsure if I sent my messages privately or to the list (or at all). At the risk of repeating myself: The problems are not related to Imagery at all. They're blatantly obvious mapping issues like: - removing things and re-adding them for no apparent reason - Adding buildings on top of pre-existing buildings - Adding closed ways with area=yes instead of building=yes, or with no tags at all - Intentional vandalism in some cases. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] weeklyOSM - Mapbox updates
Hmm, around the same times the Mapbox updates stopped[1], OsmCHA[2] stopped showing new changesets and I reported[3] that. Perhaps it was an unintended side-effect of whatever Mapbox did? Just speculating. [1] https://help.openstreetmap.org/questions/59826 [2] https://osmcha.mapbox.com/ [3] https://lists.openstreetmap.org/pipermail/talk/2017-October/079036.html ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Sudden influx of bad HOTosm edits in West Bank
Here's some intentional vandalism (among the other horrible edits). https://www.openstreetmap.org/user/A_N_D_R_E_W There also seems to be a pattern of people removing and readding buildings or adding buildings on top of buildings. (Why are they doing that? Perhaps they have an assignment of a minimum additions and it's a way to cheaply meet some quota?) ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
[OSM-talk] Sudden influx of bad HOTosm edits in West Bank
There's a sudden influx of bad edits by different users related to Palestine HotOSM tasks. I don't know why. Perhaps it's a poorly-trained group? Incomplete list of relevant hotOSM tasks: 3441, 3447, 3759, 3768 Incomplete List of bad changesets: https://www.openstreetmap.org/changeset/53330583 (reverted) https://www.openstreetmap.org/changeset/53330562 https://www.openstreetmap.org/changeset/53330596 https://osmcha.mapbox.com/changesets/53330616 https://osmcha.mapbox.com/changesets/53330636 https://www.openstreetmap.org/changeset/53330694 https://www.openstreetmap.org/changeset/53330695 https://www.openstreetmap.org/changeset/53330707 (intersecting ways) https://www.openstreetmap.org/changeset/53330589 https://www.openstreetmap.org/changeset/53330728 ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Adding wikidata tags to the remaining objects with only wikipedia tag
>Wow, i had to read this twice to really believe what i was reading >here. >Seems you are still in deep denial about the fundamental differences >between OSM and Wikipedia. Christopher, A Wikidata tag is just as verifiable as Wikipedia tag: Both require visiting an external site. Yuri made no further claims about anything fundamental. Yuri's second argument is that Wikidata tags are more stable, which is objectively true. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Publishing bot code. GPL or AGPL?
Thank you everyone for the very informative replies. I've decided to use GPLv3. (And I think the difference between it and MIT is negligible in practice for this particular use case) ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
[OSM-talk] Publishing bot code. GPL or AGPL?
I understand that GPLv3 has a loophole in which someone could modify your GPL-licensed code, and then run it on a server which offers some service. Since a service is being sent over the wire, and not the executable itself, then they can keep their modified code private. AGPL prevents this loophole. Does the same logic apply for OSM bots? Would someone using a personally modified GPL'ed bot not have to publish it? Should I use AGPL instead if I wish to force any bot user to publish the code? ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] A thought on bot edits
Thank you. On Sat, 2017-10-07 at 10:55 +0200, Christoph Hormann wrote: > I would be careful interpreting the lack of objections to your > automated > edits in the local community as universal approval though. There > are > likely also locals who do not think this is a good idea but due to > the > low intensity and low volume of edits they don't see it necessary to > say anything. That's a good point. I'll keep it in mind. It's worth noting I always look at a small sample of changes before hitting the final "upload" button. The bot does not upload fully autonomously. I'm still uncomfortable with a fully automatic bot. This human supervision already proved helpful on several occasions. In one recent case, the bot would have conflicted with Yuri's recent Wiki edits, and the manual supervision caught this and we then coordinated properly: https://www.openstreetmap.org/changeset/51350214 If my bot were fully automatic, it would have annoyed Yuri and interfered with his project without discussion. Worst still, If the two edits were being made by fully automatic bots, a perpetual bot edit war would have possibly ensued. (Depending on the way the bots were coded) As for local particularities, I completely agree. For instance, Israel has a unique case where the "name" tag is always duplicated at "name:he" or "name:ar". A theoretical global bot which removes name/name:lang whenever the values are duplicated would cause local damage and wouldn't be aware of the local conventions, even though such a change may be welcome elsewhere. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Could we just pause any wikidata edits for a month or two?
On Tue, 2017-10-03 at 17:21 -0400, Yuri Astrakhan wrote: > While I have nothing against pausing bulk wikidata additions for a > month, > we should be very clear here: > * several communities use bots to maintain and inject these tags, > e.g. > Israel. Should they pause their bots? I am the Maintainer of the IL bot. I don't think it should be paused. Everyone here is fine with it, and it's not specific to wikidata tags. It maintains shop/brand/fuel chains and keeps some of their tags consistent. I documented[1] the bot transparently, and anyone can easily modify the bot's "injected" tags easily if it's ever needed. Whatever is eventually decided regarding the Wiki tags can easily be applied to the bot. I see no reason for the bot to cease working. It's not a Wiki bot. [1] https://wiki.openstreetmap.org/wiki/User:SafwatHalaby/scripts/brand ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] A thought on bot edits
Drudgery is evil, well written bots save us from drudgery, and allow us to use human time more productively, therefore well written bots are good. Why should a human clean up whitespace, or add the "cuisine" tag to a hundred "Burger King" branches? Shouldn't our creative brains invest their time elsewhere? I don't think a generic sweeping rule is a good idea. Each bot should be analyzed individually. Badly written bots, the ones that add work rather than save work, should be stopped. Namespace or database separation mean the bots cannot as effectively save us from drudgery. The point of good bots is to save the work that needs to be done on the actual OSM data. However, I do think the "burden of proof" lies on the bot owner; It is the owner's job to explain why the bot is needed, why it is "good", and to document it extensively and make its operations very transparent. I try *very hard* to do that with my scripts. Every single changeset and algorithm is logged at my talk page, all changesets have the bot=yes tag, a dedicated bot user is used, etc. Again, I think every case is to be handled individually, with BOP on the bot owner. bad/undocumented/undiscussed/non-transparent bots are the problem. https://wiki.openstreetmap.org/wiki/User:SafwatHalaby#SafwatHalaby_bot ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
[OSM-talk] OSMCha isn't showing new changesets
https://osmcha.mapbox.com/ hasn't been showing new changesets since 5 days. Does anyone know why? ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Downloading Version 3 of all bus stops in a country
On Wed, 2017-09-27 at 09:18 +0200, Jo wrote: > 2017-09-27 8:30 GMT+02:00 Safwat Halaby <swiftf...@gmx.com>: > > > On Tue, 2017-09-26 at 11:46 +0200, Jo wrote: > > > > > Then load that in PostGIS and create scripts to read GTFS into > > > PostGIS. > > > > > > Then compare the data in the DB and produce output and ideally a > > > UI. > > > > > > I started doing something like that here: > > > > > > https://github.com/osmbe/public_transport > > > > > > Let me know if you see ways of working on that or another way to > > > tackle the > > > problem together. > > > > > > Jo > > > > I will check the project out. Thanks for the link. Would you mind > > explaining what it is capable of? The readme is not so descriptive. > > > > For the time being not so much yet. Before the summer I made some > progress > migrating scripts I had created for an import of Belgian bus stops > and > lines, but during the summer I got a bit 'distracted' by mentoring > the > PT_Assistant plugin. > > The basic idea is to take operator data, either GTFS or a dump of > their > internal DB. > > Then compare all the stops regarding tags and position. For the stops > the > other tables routes, trips and segments, can be used to calculate > route_ref. > > Then create OSM route relations based on their data and compare to > what is > present in OSM. > > This is where it becomes trickier to keep track of their versions and > ours, > especially if the intent is to both give feedback to the operators > and have > a platform showing mappers which stops, routes and route_masters are > in > need of attention. > > Polyglot That's very interesting. My script is far less ambitious (no routes, only plain stops). ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Downloading Version 3 of all bus stops in a country
I think my last two replies never got through and were sent privately instead. Here's a rephrasing. (which is possibly better anyways). __ On Wed, 2017-09-27 at 10:53 +0200, Marc Gemis wrote: > isn't it possible that the 2017 contains data from e.g. 2014, which > has not been reviewed in the meantime ? > Which would then mean that the 2015 edit is more recent. > > Or is there a review date in the "database" ? There is no internal review date, and what you describe could happen. This is the only drawback I can think of. The first run would be imperfect in that regard, but the government publishes new GTFS dumps on a daily basis, so after the first run this wouldn't be an issue at all, and the "review date" would be "yesterday" or at worst "during the last few weeks". This is still better than any practically possible manual work considering our mapper density and the amount of stops. No one has been able to manually review or edit all those stops and it's been 5 years and the stops are horribly stale. Note that: 1. extra tags (shelter, wheelchair) will be preserved despite the drawback 2. the new database is considered pretty good quality, so issues will likely be negligible or nonexistent. __ On Wed, 2017-09-27 at 11:07 +0200, Jo wrote: > If there is a conflict regarding position or tags, they should be > resolved > by a human mapper. If I were to apply the newer is better approach, > we > would constantly be reverting back to the positions the operators > think > their stops are at. > > It's important to respect the mappers work, because without mappers > OSM > quickly dies off. > > It might be complex to put such a solution in place, but it should be > obvious that if data flows in 2 directions, too much simplification > doesn't > work. The script will not "constantly revert". 1. Government publishes database with bad edit X: script adds it 2. user fixes it with fix Y 3. Goverment publishes database which still has bad edit X: The script ignores 3, because "newer is better" and the user edit is newer. No constant reverts are involved. However, in the following scenario, the script would indeed introduce a bad edit to OSM: 1. Government publishes database with bad edit X: script adds it 2. user fixes it with fix Y 3. Goverment publishes database with a NEW BAD EDIT Z. So the script rests on the assumption that whenever a government updates a bus stop, it's because the bus stop has actually changed or because a newer better measurement was made, so Z is always expected to be better than X (and usually better than Y because it's newer in time) and this scenario shouldn't exist. Z is always assumed better. If a provider is unreliable such that it makes random edits in its new database that are LESS accurate than its older database, then this script is not suitable for dealing with such a provider and the complexity of your project is needed. This does not seem to be the case in Israel. To quote anonymous_gushdan_mapper from the forums: > Israel has something like 30,000 bus stops, > and they change daily all across the country. > There's no way human mappers could ever verify > the accuracy of all of them, unless you have > someone working full-time on > this. However, the data is considered extremely > accurate, and inaccuracies are quite rare. > We do have a system that announces the name > of the next stop, which uses this data. > I think people from other countries don't realize that this > is not a single, private operator data, nor it's a single > city data - it's government generated data that controls > the entire public transportation network in the country. > If a bus stop is not in this dataset, it doesn't exist. > There will never be anything more accurate for bus stops > in Israel than this dataset. > If accuracy is important to us, we *must* implement this > importing script, otherwise the data on OSM will get > stale quickly - just like the current data is stale > and shows a lot of bus stops that have been > since then moved or canceled. source: https://forum.openstreetmap.org/viewtopic.php?pid=665570#p665570 So, I could probably naively always trust the GTFS and it'll still be mostly fine. But I still want to give mappers the ability to fix coordinates and such, without overriding their edits the next run (unless they've been updated by the government to newer values during that time). Lastly, I'd like to point out that no local mappers are complaining about this and the feedback is so far positive. Thanks. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] The real face of MAPS.ME edits and notes - a short analysis
Note: I'm replying to an old mail. If you don't have it. You can find it here: https://lists.openstreetmap.org/pipermail/talk/2017-June/078181.html "The real face of MAPS.ME edits and notes - a short analysis" >The price to reproduce all the on-the-ground mappers' > contributions we have is > likely to be somewhere between 4 million and 150 million euros: Hello, These monetary considerations could be valid so long as the MAPS.ME bad edits do not exceed a certain threshold. Past that threshold, maintainers can no longer keep up with the noise and the value of the map essentially drops to zero. Take a look at the Middle East, the map was filled with POIs such as "my house", "my uncle's house". For a map consumer, it would be a poor choice to use OpenStreetMap in these regions, because it's junk. They'd prefer Google Maps or Bing Maps. And so, OpenStreetMap becomes valueless, regardless of much money would have been invested in surveying, etc. I've deleted thousands of personal bookmarks in that area but I can no longer keep up and have no desire to continue doing so. The "my house" style tags will probably re-accumulate with time, reducing the value of the map. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Downloading Version 3 of all bus stops in a country
On Wed, 2017-09-27 at 09:12 +0200, Jo wrote: > Deleting data on OpenStreetMap and replacing it by imported data is > obviously never the acceptable approach. > > What I don't understand is why you don't create something that > compares the > latest version of all the bus stops in OSM with the latest version of > the > GTFS data from upstream. > > Why compare with an inbetween version? I'm taking a "Newer is better" approach for conflict resolution. Let's say a user made a 2015 edit for a coordinate (We'll call this version X). And your 2017 database has a different coordinate (We'll call this version Y). How do you determine which coordinate to keep? If you had a 2012 database that would be easy. Case 1: 2012 database: Y 2015 user edit: X 2017 database: Y This means the Y has not been updated since 2012. Newer is better, so trust the user edit and set the coordinates to X. Case 2: 2012 database: T 2015 user edit: X 2017 database: Y This means Y is the newest value, and we should override the user edit. Without two database, we'd have to guess or resort to manual user intervention via fancy web services. > What I think is needed is a (web) service that stands between the > operator > data, be it GTFS or DB dumps and OpenStreetMap where comparison is > made and > which can be used by mappers to either improve OSM, or to send > feedback to > the operators that there are issues with the data they provide. Or > where > the operators can request flagged stops in bulk. > I am a huge advocate of simplicity. In my humble opinion, web services and SQL databases are overkill for what I'm trying to do. Your project spans dozens of files while mine is a single .js file for the JOSM scripting plugin, which reads two textual stops.txt files from the old and the newer GTFS databases. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Downloading Version 3 of all bus stops in a country
On Tue, 2017-09-26 at 11:46 +0200, Jo wrote: > Then load that in PostGIS and create scripts to read GTFS into > PostGIS. > > Then compare the data in the DB and produce output and ideally a UI. > > I started doing something like that here: > > https://github.com/osmbe/public_transport > > Let me know if you see ways of working on that or another way to > tackle the > problem together. > > Jo I will check the project out. Thanks for the link. Would you mind explaining what it is capable of? The readme is not so descriptive. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Downloading Version 3 of all bus stops in a country
Thanks for the info, mmd! ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Downloading Version 3 of all bus stops in a country
Hi John Whelan, As implied in the forum thread, not wanting to destroy user data is exactly why I'm building a relatively complex script. The naive approach is to destroy all-bus stops are re-import, everytime a GTFS update is released. But I don't want that. Instead of doing that, the script preserves all user-submitted data such as shelter, wheelchair, and GPS coordinate fixes and provides incremental updates rather than the destruction of all bus stops per import. The incremental updates do not destroy user changes. In order to achieve that, the older and the newer GTFS database need to be compared per update. Yesterday I didn't have the database which was used in the old 2012 import, so I couldn't locally test-run my script. which is what lead to my original question in this thread. But now I managed to reverse-build the old GTFS database from version 3 of the bus stops in Israel by downloading the bus stops through the relevant changesets. Here's some of the discussion: >Here's a merging idea. > >Problem: Dealing with conflicts between mapper edits and gtfs data. >Solution: "The most recent version is the correct version" philosophy. > >- The first gtfs update would update everything. Conflicts are > resolved by prioritizing the gtfs file's version. This is a > "necessary evil" but is >only needed once. (edit: I might be > able to mitigate this by tracing bus stop OSM history). >- Some time passes, and users update some of the bus stops. >- The ministry of transportation updates some bus stops in > its database and publishes a new gtfs file. >- The next gtfs update would inspect the difference between > the new gtfs file and the older gtfs file. Only bus stops > that have had their data (in >the gtfs file) changed since the > last file are updated. So, conflicts are resolved by prioritizing > the gtfs file version, but only for the bus stops that were changed > by the ministry since the last update. The rest of the bus stops > are left intact. (source: https://forum.openstreetmap.org/viewtopic.php?id=16738=3) Also, we know the data can be used in OSM. https://forum.openstreetmap.org/viewtopic.php?pid=244096#p244096 ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Downloading Version 3 of all bus stops in a country
On Tue, 2017-09-26 at 17:08 -0400, john whelan wrote: > Please do not do this globally. > Of course not. I wouldn't deploy anything globally without prior discussion. What I meant was the script is not Israel-specific, and anyone who knows their provider has some reasonable accuracy could use it. It allows incremental GTFS updates where mapper-submitted accuracy fixes are not overridden by the next update. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Fixing OSM wikipedia redirects
On Mon, 2017-09-25 at 23:11 -0400, Yuri Astrakhan wrote: > Fixing them by hand seems like a huge waste of the > community time I agree. This is purely mechanical work. Drudgery is evil. I'm willing to try writing a script once I finish my other scripting projects. This should be straightforward. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Downloading Version 3 of all bus stops in a country
On Tue, 2017-09-26 at 08:50 -0400, john whelan wrote: > I suggest pulling in the country file and if need be chop it up to > load > into JOSM. Load up the latest bus stops in a different layer then > use the > todo plugin and go through them one at a time cleaning them up that > way. > > Note that the GTFS file bus stop location may not be accurate, some > bus > stops are known to be 100 meters out, it depends on the provider. If > you > have a system that announces bus stops for partially sighted people > then > that is an indication they should be fairly good but check with the > provider. Even when they are accurate they can still be out by a few > meters, locally one was in the middle of a road junction so they do > need > verifying. > We are well aware that GTFS accuracy is not perfect. (Though nobody reported a 100-meter deviation so far). A potential solution, which could also work globally, is discussed below. I already have a (somewhat) working prototype. I'll look for other solutions too. https://forum.openstreetmap.org/viewtopic.php?id=16738=3 ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Downloading Version 3 of all bus stops in a country
link fix: https://forum.openstreetmap.org/viewtopic.php?id=16738 ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Downloading Version 3 of all bus stops in a country
I'd like to point out (as I pointed out in the sub-threads), that I solved this problem simply by fetching the changeset which introduced v3. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Downloading Version 3 of all bus stops in a country
On Tue, 2017-09-26 at 14:12 +0200, Frederik Ramm wrote: > > Are all these automatic edits at least discussed on the Israeli > mailing > list, or is this a free-for-all where anyone who knows how to write a > script runs over all bus stops in Israel again and again? > > Bye > Frederik > Why are you automatically assuming the worst? I have, and I am, extensively discussing fixing the import mess. Even the previously messy imports by the other users were discussed and were fairly coordinated. See this: https://forum.openstreetmap.org/viewtopic.php?id =16738 ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Downloading Version 3 of all bus stops in a country
On Tue, 2017-09-26 at 12:15 +0100, Tom Hughes wrote: > On 26/09/17 11:41, Safwat Halaby wrote: > > On Tue, 2017-09-26 at 11:31 +0200, Michael Reichert wrote: > > > Hi, > > > > > > Am 2017-09-26 um 09:19 schrieb SwiftFast: > > > > I need to download all version 3 bus stops in a country which > > > > has > > > > about > > > > 30,000 bus stops. Version 3 exists since a 2012 import. What's > > > > the > > > > recommended way to accomplish this? I have two ways in mind. > > > > > > Version 3 of what? > > > > > > Best regards > > > > > > > Version 3 of all Israeli bus stops that have > > "source=israel_gtfs_v1". > > See this: https://www.openstreetmap.org/node/1802982884/history and > > look at "version #3". > > But how do you know it's version 3 you want for every one? > > It sounds like you're making an assumption that all these objects > have > only been edited in a particular way by some automated process and > that > nobody has ever touched one by hand and hence added an extra version. > > Tom > V3 was an automatic import for all nodes. it was made pretty quickly after v1 and v2 and it's very unlikely someone edited during that time. But I solved my problems simply by fetching the responsible changeset. I was overcomplicating this problem. All I needed was: get https://api.openstreetmap.org/api/0.6/changeset/14265835/download Thanks for the help! ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Downloading Version 3 of all bus stops in a country
I think I've needlessly overcomplicated this. All I need to do was to do is fetch changeset #14265835, which introduced those historic nodes. ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Downloading Version 3 of all bus stops in a country
On Tue, 2017-09-26 at 11:31 +0200, Michael Reichert wrote: > Hi, > > Am 2017-09-26 um 09:19 schrieb SwiftFast: > > I need to download all version 3 bus stops in a country which has > > about > > 30,000 bus stops. Version 3 exists since a 2012 import. What's the > > recommended way to accomplish this? I have two ways in mind. > > Version 3 of what? > > Best regards > Version 3 of all Israeli bus stops that have "source=israel_gtfs_v1". See this: https://www.openstreetmap.org/node/1802982884/history and look at "version #3". ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Downloading Version 3 of all bus stops in a country
On Tue, 2017-09-26 at 11:46 +0200, Jo wrote: > Hi Safwat, > > Overpass API is definitely your friend. Version 3 doesn't mean much > though. > Do you mean all bus stops with a public_transport tag? Thanks for the reply. By version 3, I mean a particular historic version of the OSM element. For instance, see "Version #3" of this bus stop: https://www.openstreetmap.org/node/1802982884/history I need all "version #3" of all bus stops that have a "source=israel_gtfs_v1" in Israel. As far as I know. Overpass can only fetch the latest version. Can overpass directly fetch version #3? ___ talk mailing list talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk
