Re: [OSM-talk] Could/should editors detect/disallow huge changeset bboxes?

[Bryce Jasmer]

For those interested in knowing what a "typical" bounding box size is,
check out my diary entry where I sample changesets over the last 8 years
and generate a heatmap of bounding box sizes over time.
https://www.openstreetmap.org/user/b-jazz/diary/393858


On Sun, Jun 14, 2020 at 3:21 AM  wrote:

> On Sat, 13 Jun 2020 21:36:37 +0100, Alan Mackie 
> wrote:
>
> >I have no problem with big bounding boxes that result from editing large
> >objects. I get annoyed by the ones where somebody added twontiny houses on
> >opposite sides of the world.
>
>   And those with a normal work method of editing random items in their
> 'home location' and an 'away location' on the other side of the country,
> with a comment of 'updating'.
>
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [Talk-us] Request for review of plan for scripted edit

[Bryce Jasmer]

I’m really opposed to this idea of scaring people away from editing objects
with the “data freshness” boogie man argument. If someone really cares
about freshness, the entire history of an object is available to you.

On Thu, Aug 8, 2019 at 12:57 PM Kevin Broderick 
wrote:

> I'm of mixed feelings on the apparent freshness, but as long as the
> guidelines are followed so changesets are of reasonable size and easily
> identified as scripted, I don't see much of an issue.
>
> While having an automated script make assumptions caused me to twitch a
> little, the reality is that a human is going to make the same assumption.
> If I see a seven-digit number on a sign and want to dial it, I'm going to
> assume that the area code is 207; if I'm across the state line in New
> Hampshire, I'm going to assume 603. If that assumption isn't correct, the
> source data is bad anyhow, and adding the implicit area code isn't making
> it substantially worse. Have you been able to discern how many seven-digit
> numbers are in the system?
>
> On Thu, Aug 8, 2019 at 2:56 PM Jmapb  wrote:
>
>> On 8/8/2019 1:28 PM, Alex Hennings wrote:
>> > Community,
>> >
>> > I'm planning a scripted change and would like feedback. Plans are
>> > outlined here:
>> > https://wiki.openstreetmap.org/wiki/Automated_edits/blackboxlogic
>> >
>> > I'd appreciate feedback or questions in the 'Discussion' portion of
>> > that wiki page, or within this email list.
>>
>> Hi Alex!
>>
>> First, a possible typo: I think "Nodes, Ways and References" should be
>> "Nodes, Ways and Relations"?
>>
>> I'm a fan of the +1-xxx-xxx- format, since it's the only standard
>> format that's visually intuitive to North American users. I often switch
>> numbers to this format when I make updates to an existing POI.
>>
>> Personally, though, I've always felt a little uneasy about automated
>> updates like this because they give a false impression of the freshness
>> of the data. If it's been five years since any "real" updates to a POI,
>> I'd rather that the date of last update reflected that. It's hard to
>> gauge the community consensus on this issue, but IMO running this on
>> POIs that have been manually updated (ie not by a mass edit) in the last
>> 6 months would be fine.
>>
>> Regarding the single area code question... now that cell phones, VOIP,
>> and nationwide calling plans are ubiquitous, the idea that a certain
>> area code refers to a certain area is steadily eroding. I have started
>> to see a few businesses with out-of-state phone numbers on their
>> signs... but at this point it's still more likely that an out-of-state
>> area code is an error or SEO spam. I'd suggest that these would go into
>> your "Manually review or flag" category.
>>
>> Regardless, the idea that an area can have a single "traditional" area
>> code is still true. Personally I have no problem with prepending the
>> traditional area code onto 7-digit phone numbers. (I do it all the time
>> in manual mapping.)
>>
>> Finally, thanks for posting your tools... I see these are written in
>> CSharp, which I'm only tangentially familiar with. What sort of
>> environment would one need to build these?
>>
>> Thanks, Jason
>>
>>
>>
>> ___
>> Talk-us mailing list
>> Talk-us@openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/talk-us
>>
>
>
> --
> Kevin Broderick
> k...@kevinbroderick.com
> ___
> Talk-us mailing list
> Talk-us@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-us
>
___
Talk-us mailing list
Talk-us@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk-us

Re: [OSM-talk] iD influencing tagging

[Bryce Jasmer]

The wiki page for landuse=reservoir says:

"Description: Ambiguous and better alternatives exist, see water=reservoir"

So, is iD wrong to use this, or is the wiki incorrect?

On Sun, Apr 7, 2019 at 8:24 AM Tomas Straupis 
wrote:

> 2019-04-07, sk, 17:47 Bryce Jasmer rašė:
> > Can you give some examples of what the OSM normals are and how iD
> differs from them?
>
>   There is no way (other than writing tags directly) to tag reservoirs
> as landuse=reservoir (original and still wider used water tagging
> scheme), iD insists on natural=water+water=reservoir.
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] iD influencing tagging

[Bryce Jasmer]

Can you give some examples of what the OSM normals are and how iD differs
from them?

Is there a bug against iD not showing semi-detached house? Can you provide
the link(s) to the bug(s) so we can read what the discussion/rationale is
for not showing it?

As for a building tool, is there any evidence one would be rejected if
someone wrote and tested a feature and submitted a pull request?


On Sun, Apr 7, 2019, 5:44 AM John Whelan  wrote:

> I note that the matter has been raised in talk-de and mentioned in osm
> weekly.
>
> Tagging is not always easy, but I do have concerns when iD is so commonly
> used but the recommended tags do not align with OpenStreetMap I'll say
> normals.
>
> Specifically one of my concerns is a semi-detached house is not recognised
> in iD only the more general tag house.
>
> JOSM I think is much more open than iD but given the way OpenStreetMap
> functions I suspect ID is a much more closed environment.
>
> For example JOSM has the buildings_tool plugin, Africa has a large number
> of odd shaped buildings mapped in iD.
>
> Thoughts ladies and gentlemen?
>
> Thanks John
>
>
> --
> Sent from Postbox
> 
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] HTTPS all the Things (Automated Edit)

[Bryce Jasmer]

Is this a problem that only a few are concerned with? Can I get a
geographic area where I can run a larger number of changes in a larger
bounding box? I could easily make some one-off changes on a per country
basis if that would help. And would fewer changesets of, say, 100 objects
be a good middle-of-the-road number?


On Mon, Mar 25, 2019 at 6:03 PM Mateusz Konieczny 
wrote:

> And from my side - avoid making changesets with more than 1000 objects.
> Reverting changesets that went wrong, with tens of thousands modified
> objects
> is basically not making possible to review it.
>
> Mar 26, 2019, 12:41 AM by andrew.harv...@gmail.com:
>
> Any chance you could do more changes per changeset? At the moment this is
> flooding feeds in osmcha with many small changesets, it would be easier if
> you did one big changeset.
>
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Your thoughts on osm.org

[Bryce Jasmer]

The home page would be a curated list of the different maps and tools that
are built on OSM. Ones that are meant to be more for the end user, not a
reference implementation of what a map could look like. And it would detect
if you are on mobile and provide a list of apps that could be downloaded
for mobile devices depending on the type of app you are looking for.

On Tue, Mar 12, 2019 at 9:59 AM Martijn van Exel  wrote:

> Hi all,
>
> Here’s something I ask myself from time to time and would like to hear
> other people’s thoughts about.
>
> Imagine the openstreetmap.org home page, but without the map.
> What would the home page be about instead? What would be on it?
>
> Thanks for sharing,
> Martijn
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] How to save Overpass query results to a GeoJSON file with Python?

[Bryce Jasmer]

I use the overpass library (I think it's the one you mentioned). The
resulting dictionary can be coerced to a string and written out and it
should be valid geojson.

#!/usr/bin/python3
import overpass
api = overpass.API()
result = 
api.get('area[name="Granollers"][admin_level=8];(nwr["highway"](area);relation["highway"](area););')
with open('results.geojson', 'w') as fd:
fd.write(str(result))


On Mon, Mar 11, 2019 at 8:04 AM Carlos Cámara Menoyo via talk <
talk@openstreetmap.org> wrote:

> Thanks for your reply, Mateusz.
>
> I know (and use) Overpass turbo. However, what I want to do is to store a
> query and execute it in order to update results and store them as a file
> for further manipulation, so I can easily update results without the need
> to accessing to the interface, generate the query, run it and then download
> the results. I would like to achieve same functionallity but just by
> executing a file (which, additionally, may have more than one query).
>
> So far, I found overpy ,
> which seems to work fine in terms of querying but has two problems for me:
>
>1. It seems to be unmaintained (last commit dates from April 2017)
>2. I didn't succeed in storing the result into a geospatial file (or a
>geopandas data frame) (visit this jupyter lab with my current achievements:
>https://github.com/ccamara/osm-python/blob/master/overpy-demo.ipynb)
>
> I also have just found Overpass API wrapper
> , but I still have
> to explore it and understand how does it work and how can I tame it to my
> needs.
>
> You propose a third way, which is using overpass and writing a script. It
> is not clear to me what do you mean with that, since I thought that that
> was precisely what I was attempting. Could you be more specific, please? Do
> you have other insight?
>
> Regards,
> Carlos
>
>
> Carlos Cámara-Menoyo
> https://carloscamara.es
>
>
> ‐‐‐ Original Message ‐‐‐
> On Monday 11 March de 2019 a les 14:15, Mateusz Konieczny <
> matkoni...@tutanota.com> wrote:
>
>
>
>
> Mar 11, 2019, 11:18 AM by talk@openstreetmap.org:
>
> Dear colleagues,
>
> I have just started to use Python and I would like to make a query to
> Overpass and store the results in a geospatial format (e.g. GeoJSON) so I
> can programatically update the data.
>
> As far as I know, there is a library called overpy that should be what I
> am looking for. After reading its documentation
>  I came up
> with the following code:
>
> import overpy
>
> API = overpy.Overpass()
> # Fetch highways within Granollers' city.
>
> result = API.query("""
> area[name="Granollers"][admin_level=8];
> // query part for: “highway=*”
> (way["highway"](area);
> relation["highway"](area);
> );
> // print results
> out body;
> """)
>
> The thing is that I am not familiar at all with Python and I don't know
> how to store result as a GeoJSON file (Honestly, I don't even know what
> kind of thing result is -I tried with type(result) and I get overpy.Result
> as an output, which doesn't mean much to my poor knowledge). I have been
> reading overpy documentation but I haven't been able to figure it out.
> Can anyone give me a clue with this?
>
> Are you familiar with http://overpass-turbo.eu/
> https://wiki.openstreetmap.org/wiki/Overpass_turbo ?
>
> There are two parts here
>
> (1) using overpass itself
> (2) writing python script
>
> That are separate (you can call Overpass API from any program, not just
> python)
>
> "I have just started to use Python" - note that it means learning two
> separate languages at once:
>
> (1) Python itself
> (2) Overpass query syntax
>
> PS In case that Overpass is new for you my page may be of some use:
>
>
> https://mapsaregreat.com/geographic-data-mining-and-visualisation-for-beginners/overpass-turbo-tutorial.html
>
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] HTTPS all the Things (Automated Edit)

[Bryce Jasmer]

Thanks for pointing that out. I have fixed it by redirecting between the
two pages.

On Fri, Mar 8, 2019 at 4:28 AM Michael Reichert 
wrote:

> Hi Bryce,
>
> Am 22/02/2019 um 08.02 schrieb Bryce Jasmer:
> > The wiki page is
> https://wiki.openstreetmap.org/wiki/Automated_Edits/b-jazz
>
> I have seen that you started uploading. Could you please add a link to
> that wiki page to the profile page of b-jazz-bot or create
> https://wiki.openstreetmap.org/wiki/Automated_Edits/b-jazz-bot
> as a redirect page to
> https://wiki.openstreetmap.org/wiki/Automated_Edits/b-jazz
> in order to comply with the Automated Edits Code of Conduct.
>
> Best regards
>
> Michael
>
> --
> Per E-Mail kommuniziere ich bevorzugt GPG-verschlüsselt. (Mailinglisten
> ausgenommen)
> I prefer GPG encryption of emails. (does not apply on mailing lists)
>
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] HTTPS all the Things (Automated Edit)

[Bryce Jasmer]

James, I’m not following you. Can you expand on what changes you assume the
bot will be making, and what the “horribly wrong” event as a result of said
changes? I think you’re leaving out a piece of the puzzle and I’m not sure
what it is.

Thanks.

On Tue, Feb 26, 2019 at 6:46 AM James  wrote:

> I can give an example of this going horribly wrong:
>
> http://www.osmcanada.ca redirects to https://www.osmcanada.ca
>
> but I specifically disabled https on http://tasks.osmcanada.ca (hosted on
> same server) because josm doesnt play nice with https task manager
>
> Web admins will redirect their traffic if it needs to be. We shouldn't
> force https, worst case port 80 will redirect to 443 via http header.
> Semi-worst case they have HSTS header that tells browser to connect to 443
> until xyz(far in future) and best case web admin registered on hsts preload
> list, so modern browsers will do their job.
>
> On Tue., Feb. 26, 2019, 9:39 a.m. Bryce Jasmer,  wrote:
>
>> The HSTS discussion is completely orthogonal to what the stated goal is
>> and any further discussion on it is really just muddying the waters. HSTS
>> comes into play after the user is already visiting over https.
>>
>> If I’m mistaken, please help me understand.
>>
>> On Tue, Feb 26, 2019 at 6:30 AM Rory McCann  wrote:
>>
>>> On 26/02/2019 14:45, Joseph Reeves wrote:
>>> > As an aside, HSTS is interesting here because the website operator is
>>> > saying "only use this domain over https", but at that point, we don't
>>> > need to make changes to the database because the web client should be
>>> > aware of the HSTS preload list; the protocol listed in the referrer
>>> > is not relevant.
>>>
>>> I don't think we can rely totally on HSTS. I'm sure not all sites are on
>>> HSTS preload lists. I think OSM has more "website=http://*; tags
>>> (965k)¹
>>> than Firefox² & Chrome³ have in their HSTS preload lists...
>>>
>>> [1] https://taginfo.openstreetmap.org/keys/website#values
>>>
>>> [2]
>>>
>>> https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Strict-Transport-Security#Preloading_Strict_Transport_Security
>>>
>>> https://hg.mozilla.org/mozilla-central/raw-file/tip/security/manager/ssl/nsSTSPreloadList.inc
>>>
>>> [3]
>>> https://www.chromium.org/hsts
>>>
>>> https://cs.chromium.org/codesearch/f/chromium/src/net/http/transport_security_state_static.json?cl=5b2537d89ea5994d27bba5735961b0be1095c54c
>>>
>>> ___
>>> talk mailing list
>>> talk@openstreetmap.org
>>> https://lists.openstreetmap.org/listinfo/talk
>>>
>> ___
>> talk mailing list
>> talk@openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/talk
>>
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Other tags/values | Re: HTTPS all the Things (Automated Edit)

[Bryce Jasmer]

You have a good point. I’ll add code to skip that object if multiple
identical values are present.

On Tue, Feb 26, 2019 at 6:39 AM Rory McCann  wrote:

> Hi,
>
> As far as I remember, you're not planning on touching other tags, like
> `contact:website`. *But* what happens if an object has
> `website=http://example.com,contact:website=http://example.com`? You'd
> make it inconsistant!
>
> I suggest a simple solution: If any other tag has that same value, then
> skip that object. Then it won't be made "inconsistant". I have no idea
> how many objects this applies to, possibly none! Hopefully you're script
> will be able to tell us :)
>
> Rory
>
> On 22/02/2019 08:02, Bryce Jasmer wrote:
> > I have written a script that will search for OSM objects that have a
> > website tag that explicitly states "http://...; or implicitly uses http
> > by leaving of the protocol specification. The script will then loop
> > through all that it discovers and asks the http site if it will redirect
> > me to the secure version of the website over the https protocol. If it
> > does, I will update the database with the new value.
> >
> > This has a couple of advantages. From now through the end of time, any
> > user clicking on one of those links will be spared the time it takes to
> > establish the connection, ask if there is a secure version of the site,
> > and tear down the connection. It's on the order of 10-200 ms to do, but
> > over the life of the link and the number of objects that are clicked and
> > the population, this could save centuries of time :-)
> >
> > Another advantage is that it will make https more pervasive and
> > hopefully people will start thinking https and forgetting all about
> > http. A more secure internet is in all of our best interests.
> >
> > Anyway, I'd like to (slowly) run this across the planet. I've discussed
> > this on the US Slack channel and have performed the actions on the
> > United States already. I've addressed many questions and have heard no
> > strong objections. I'm seeking feedback from the larger community now
> > before proceeding.
> >
> > The wiki page is
> https://wiki.openstreetmap.org/wiki/Automated_Edits/b-jazz
> >
> > The Slack conversation is available, but has died down and the
> > transcript is available at the wiki page mentioned above.
> >
> > The diary entry with some more conversation is at the bot's page:
> > https://www.openstreetmap.org/user/b-jazz-bot/diary/47743
> >
> > The source code is available on GitLab for review:
> > https://gitlab.com/b-jazz/https_all_the_things
> >
> > Example changeset for a run over the "9yfd" geohash:
> > https://www.openstreetmap.org/changeset/67454775
> >
> > I welcome your input.
> >
> >
> >
> > ___
> > talk mailing list
> > talk@openstreetmap.org
> > https://lists.openstreetmap.org/listinfo/talk
> >
>
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] HTTPS all the Things (Automated Edit)

[Bryce Jasmer]

The HSTS discussion is completely orthogonal to what the stated goal is and
any further discussion on it is really just muddying the waters. HSTS comes
into play after the user is already visiting over https.

If I’m mistaken, please help me understand.

On Tue, Feb 26, 2019 at 6:30 AM Rory McCann  wrote:

> On 26/02/2019 14:45, Joseph Reeves wrote:
> > As an aside, HSTS is interesting here because the website operator is
> > saying "only use this domain over https", but at that point, we don't
> > need to make changes to the database because the web client should be
> > aware of the HSTS preload list; the protocol listed in the referrer
> > is not relevant.
>
> I don't think we can rely totally on HSTS. I'm sure not all sites are on
> HSTS preload lists. I think OSM has more "website=http://*; tags (965k)¹
> than Firefox² & Chrome³ have in their HSTS preload lists...
>
> [1] https://taginfo.openstreetmap.org/keys/website#values
>
> [2]
>
> https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Strict-Transport-Security#Preloading_Strict_Transport_Security
>
> https://hg.mozilla.org/mozilla-central/raw-file/tip/security/manager/ssl/nsSTSPreloadList.inc
>
> [3]
> https://www.chromium.org/hsts
>
> https://cs.chromium.org/codesearch/f/chromium/src/net/http/transport_security_state_static.json?cl=5b2537d89ea5994d27bba5735961b0be1095c54c
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] HTTPS all the Things (Automated Edit)

[Bryce Jasmer]

In that situation, the admin wouldn’t redirect all of their traffic to
their test site with a potentially broken cert. The bot will only modify
objects where the admin is specifically redirecting traffic already. It
makes no assumptions. The scope is very limited for this exact reason. It
will NOT guess that just because something is listening on 443 that it
should make changes.

On Tue, Feb 26, 2019 at 5:12 AM Andy Townsend  wrote:

> On 26/02/2019 12:34, Bryce Jasmer wrote:
> > Correct. No change will be made on anything other than the most
> > straightforward of redirects. So even http://example.com ->
> > https://example.com/home.aspx will be ignored.
>
> What about certificate checking?  Suppose someone primarily uses http://
> for accessing their server, but has either a self-signed certificate on
> https:// or an untrusted / expired one (perhaps they were testing).
> Presumably in that case you wouldn't change http:// to https:// ?
>
> Best Regards,
>
> Andy
>
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] HTTPS all the Things (Automated Edit)

[Bryce Jasmer]

How would you feel about bounding boxes that cross country borders but are
3 geohash digits or smaller? (Sorry I cant give you an example at the
moment, the power has been out so I can’t access tools on my computer.) I’m
not sure what your definition of enormous is and what would be an
acceptable size.

On Tue, Feb 26, 2019 at 4:39 AM Mateusz Konieczny 
wrote:

>
> In that case this mechanical edit makes sense for me (as long as edits
> will not create enormous bounding boxes due to grouping edits across
> country in one edit)
>
> Feb 26, 2019, 1:34 PM by br...@jasmer.com:
>
> Correct. No change will be made on anything other than the most
> straightforward of redirects. So even http://example.com ->
> https://example.com/home.aspx will be ignored.
>
> On Tue, Feb 26, 2019 at 4:23 AM Frederik Ramm  wrote:
>
> Hi,
>
> On 26.02.19 12:47, Mateusz Konieczny wrote:
> > So when http://domainname.com redirects to
> > https://some-other-domainname.com 
> > no edit will be made, right?
>
> The logic for this appears to be here
>
>
> https://gitlab.com/b-jazz/https_all_the_things/blob/master/src/httpsosm.py#L132-137
>
> which reads:
>
> if any((website.replace('http://', 'https://', 1) == new_location,
>website.replace('http://', 'https://', 1) + '/'
> == new_location, website.replace('http://',
> 'https://www.', 1) == new_location,
> website.replace('http://', 'https://www.', 1) + '/' == new_location,
>  website.replace('http://www.', 'https://', 1)
> == new_location,
> website.replace('http://www.', 'https://', 1) + '/' == new_location)):
>element['tags']['website'] = new_location
>
> Bye
> Frederik
>
> --
> Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09" E008°23'33"
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] HTTPS all the Things (Automated Edit)

[Bryce Jasmer]

Correct. No change will be made on anything other than the most
straightforward of redirects. So even http://example.com ->
https://example.com/home.aspx will be ignored.

On Tue, Feb 26, 2019 at 4:23 AM Frederik Ramm  wrote:

> Hi,
>
> On 26.02.19 12:47, Mateusz Konieczny wrote:
> > So when http://domainname.com redirects to
> > https://some-other-domainname.com 
> > no edit will be made, right?
>
> The logic for this appears to be here
>
>
> https://gitlab.com/b-jazz/https_all_the_things/blob/master/src/httpsosm.py#L132-137
>
> which reads:
>
> if any((website.replace('http://', 'https://', 1) == new_location,
>website.replace('http://', 'https://', 1) + '/'
> == new_location, website.replace('http://',
> 'https://www.', 1) == new_location,
> website.replace('http://', 'https://www.', 1) + '/' == new_location,
>  website.replace('http://www.', 'https://', 1)
> == new_location,
> website.replace('http://www.', 'https://', 1) + '/' == new_location)):
>element['tags']['website'] = new_location
>
> Bye
> Frederik
>
> --
> Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09" E008°23'33"
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] HTTPS all the Things (Automated Edit)

[Bryce Jasmer]

Good point Stephan about protocol-less urls being left to the "browser"
using the same protocol as it is currently using. But I think my approach
is pretty sound in that I'll only update the value if there is a redirect
from http to https. I did a sample of a dozen websites that don't redirect
and tried out the https version of their site. 100% of them were broken. So
I can't assume https, but trying http and looking for a published redirect
seems pretty sensible to me.
 Thanks for the feedback.


On Fri, Feb 22, 2019 at 12:55 AM Stephan Knauss 
wrote:

> Hi,
>
> Please be aware that protocol independent URLs do not mean that http is
> used. The client will simply continue using the protocol it used before.
>
> Real need for that is quite limited. So in most cases they are better
> written as https.
>
> But it then needs to be changed where the URL is used and not on the
> provider end.
>
> Stephan
>
>
> On February 22, 2019 8:02:20 AM GMT+01:00, Bryce Jasmer 
> wrote:
>>
>> I have written a script that will search for OSM objects that have a
>> website tag that explicitly states "http://...; or implicitly uses http
>> by leaving of the protocol specification. The script will then loop through
>> all that it discovers and asks the http site if it will redirect me to the
>> secure version of the website over the https protocol. If it does, I will
>> update the database with the new value.
>>
>> This has a couple of advantages. From now through the end of time, any
>> user clicking on one of those links will be spared the time it takes to
>> establish the connection, ask if there is a secure version of the site, and
>> tear down the connection. It's on the order of 10-200 ms to do, but over
>> the life of the link and the number of objects that are clicked and the
>> population, this could save centuries of time :-)
>>
>> Another advantage is that it will make https more pervasive and hopefully
>> people will start thinking https and forgetting all about http. A more
>> secure internet is in all of our best interests.
>>
>> Anyway, I'd like to (slowly) run this across the planet. I've discussed
>> this on the US Slack channel and have performed the actions on the United
>> States already. I've addressed many questions and have heard no strong
>> objections. I'm seeking feedback from the larger community now before
>> proceeding.
>>
>> The wiki page is
>> https://wiki.openstreetmap.org/wiki/Automated_Edits/b-jazz
>>
>> The Slack conversation is available, but has died down and the transcript
>> is available at the wiki page mentioned above.
>>
>> The diary entry with some more conversation is at the bot's page:
>> https://www.openstreetmap.org/user/b-jazz-bot/diary/47743
>>
>> The source code is available on GitLab for review:
>> https://gitlab.com/b-jazz/https_all_the_things
>>
>> Example changeset for a run over the "9yfd" geohash:
>> https://www.openstreetmap.org/changeset/67454775
>>
>> I welcome your input.
>>
>>
>>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

[OSM-talk] HTTPS all the Things (Automated Edit)

[Bryce Jasmer]

I have written a script that will search for OSM objects that have a
website tag that explicitly states "http://...; or implicitly uses http by
leaving of the protocol specification. The script will then loop through
all that it discovers and asks the http site if it will redirect me to the
secure version of the website over the https protocol. If it does, I will
update the database with the new value.

This has a couple of advantages. From now through the end of time, any user
clicking on one of those links will be spared the time it takes to
establish the connection, ask if there is a secure version of the site, and
tear down the connection. It's on the order of 10-200 ms to do, but over
the life of the link and the number of objects that are clicked and the
population, this could save centuries of time :-)

Another advantage is that it will make https more pervasive and hopefully
people will start thinking https and forgetting all about http. A more
secure internet is in all of our best interests.

Anyway, I'd like to (slowly) run this across the planet. I've discussed
this on the US Slack channel and have performed the actions on the United
States already. I've addressed many questions and have heard no strong
objections. I'm seeking feedback from the larger community now before
proceeding.

The wiki page is https://wiki.openstreetmap.org/wiki/Automated_Edits/b-jazz

The Slack conversation is available, but has died down and the transcript
is available at the wiki page mentioned above.

The diary entry with some more conversation is at the bot's page:
https://www.openstreetmap.org/user/b-jazz-bot/diary/47743

The source code is available on GitLab for review:
https://gitlab.com/b-jazz/https_all_the_things

Example changeset for a run over the "9yfd" geohash:
https://www.openstreetmap.org/changeset/67454775

I welcome your input.
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Deleting many old edits by a "bad actor"

[Bryce Jasmer]

Right. That's why I mentioned reverting if clean (meaning no other edits
have been done on the elements in question) and hand editing if the
elements have been modified since they made their bad edits.

Check out the spreadsheet for details. I've looked at every edit and have a
plan for how to correct them.

*https://tinyurl.com/y8cyh7n7 <https://tinyurl.com/y8cyh7n7>* is a shorter
URL for the spreadsheet that the mailing list probably won't break.

On Wed, Sep 5, 2018 at 3:38 PM, Martin Koppenhoefer 
wrote:

>
>
> sent from a phone
>
> > On 5. Sep 2018, at 22:54, Bryce Jasmer  wrote:
> >
> > Is there a reason I shouldn't do this cleanup?
>
>
> 3-4 years are a long time, you shouldn’t simply revert everything to the
> version before s/99he touched it, because we would loose all the edits that
> happened after that.
>
> For objects that are still last edited by this user, it seems a bit safer
> to revert, but also then it will potentially create duplicates, because the
> features s/he deleted may have been recreated from scratch in the meantime.
>
>
> cheers
> Martin
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Deleting many old edits by a "bad actor"

[Bryce Jasmer]

[I'm resending this since it might be stuck in the moderator queue as I
wasn't a list member when I first sent it.]


On Tue, Sep 4, 2018 at 3:40 PM, Bryce Jasmer  wrote:

> While doing a lot of edits in my area, I started to notice a pattern with
> a particular user's name coming up and having to fix their information. I
> dug into their edits a little more and had a hard time finding anything
> that was actually valid. Some of the things they had done:
>
> * Assigning the name of the nearby freeway offramp as the addr:street
> instead of the actual street the building is on
> * Most addr.housenumber values are three digit and look like they just
> mashed the keypad. Lot's of 789, 687, and so-on values.
> * Deleting roads that have been around and will be continue to be around
> for a long time
> * Deleting building ways for buildings that still exist
> * Assigning wrong addr:postcode values
> * Giving names like "George" to apartment buildings
> * Giving names like "Ikea" to businesses that clearly aren't Ikea
>
> The user hasn't been around for 3.5 years now, but they had done 88
> changesets in a short period (many in one day) and then thankfully
> disappeared from doing further edits. I guess they got bored. I tried
> reaching out to the user a few days ago, but highly doubt they will respond.
>
> Roughly half the edits are in my town, and the remainder are done in a
> town in Michigan. I've prepared a spreadsheet with a tab for each city and
> a list of the changesets along with what I'd like to do to them (revert if
> clean, hand edit if dirty, skip if they have organically been repaired over
> time). I'm going to see if I can find a local user to the Michigan
> changesets and have them help me. If I can't find someone, I'll take care
> of it myself.
>
> The spreadsheet can be found at https://docs.google.com/spreadsheets/d/
> 1xmi1giD5yuKh2d3vlmWM2bgYApVNRldSM-2i5--Ethk/edit?usp=sharing
>
> I'm not 100% certain that the edits are all bad, but judging from the
> history, I think it would be best to revert to versions of objects from
> before they touched them.
>
> Is there a reason I shouldn't do this cleanup?
>
> Thanks,
> Bryce (b-jazz)
>
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk