Re: [OSM-talk] New tool/API to find local OSM mailing lists by location

[Edward Betts]

Thanks for the feedback. It's obvious I need to make two changes:

- support other forums beyond the mailing lists hosted on
  http://lists.openstreetmap.org/
- return all the matches within the result hierarchy, not just the most
  specific, so searching for Glasgow should give talk-scotland and talk-GB

Best to avoid building anything against the existing API because I'm going to
change it. I'll post here when I've made these changes.
-- 
Edward.

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] finding overlapping buildings

[Mike Thompson]

John, Pierre,

I made some improvements to the select duplicate buildings script. It now
uses a spatial index, which makes it a lot faster on large datasets. It
also now uses the actual area of the buildings and their intersection, as
opposed to their bounding boxes. I will work on your other requests, but
may not have a lot of time for the next few weeks.

Mike

https://github.com/MikeTho16/JOSM-Scripts

Select Duplicate Buildings
Selects duplicate, or near duplicate, area buildings in JOSM's active
datalayer.
A "near duplicate" is a building whose area overlaps another building's
area by more than 50%. Only the first building encountered of an
overlapping pair is selected. This is done so the issue does not have to be
looked at twice. The selected buildings are added to the current
selection.
Currently only works with buildings that are ways (not multipolygons).

To Run:
* Install JOSM's Scripting Plugin (only necessary once)
* Place file in a convenient location on your system (only necessary once)
* Click "Scripting" (on top menu bar)
* Click "Run"
* Click "..." button and select the script file.
* Click "Run"
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] finding overlapping buildings

[Frédéric Rodrigo]

With Osmose you can also get only large building intersection by filter 
on severity


http://osmose.openstreetmap.fr/en/map/#zoom=11&lat=49.9788&lon=8.3169&layer=Mapnik&overlays=T&item=0%2C8300&level=1%2C2&tags=&fixable=

Or addressee the class 2 only.
http://osmose.openstreetmap.fr/en/map/#item=0&class=2


Le 22/11/2017 à 02:26, john whelan a écrit :

>Osmose has an 'overlapping building' option. Top of the list

I want something to feed into JOSM and not just any building that 
overlaps by 5%.


Thanks John

On 21 November 2017 at 19:49, Dave F > wrote:


Osmose has an 'overlapping building' option. Top of the list

Note: Some building are drawn on top of eachother to produce 3D
rendering of multi-storey buildings.

DaveF



On 21/11/2017 23:16, john whelan wrote:

Can someone describe a method I can locate these in JOSM.  I'm
not after crossing buildings but just those that are mapped twice
so two buildings with 50% or more overlap.

Straight duplicates aren't a problem but ones that are drawn
twice by two different mappers are.  Yes I know it shouldn't
happen but it does.

Thanks John


___
talk mailing list
talk@openstreetmap.org 
https://lists.openstreetmap.org/listinfo/talk






Virus-free. www.avast.com




<#m_3123707017832396152_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

___
talk mailing list
talk@openstreetmap.org 
https://lists.openstreetmap.org/listinfo/talk





___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk




___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] finding overlapping buildings

[john whelan]

Very nice.  The process I'm using is run it to find the rough area that
contains duplicate overlapping buildings.  Then redownload the area so its
fresh.  Then scroll through the buildings.  Sometimes one is beeter tagged
and more accurate than the other so you really do have to scroll through
them with the to do list.

Thanks John

On 28 November 2017 at 15:11, Mike Thompson  wrote:

> John, Pierre,
>
> I made some improvements to the select duplicate buildings script. It now
> uses a spatial index, which makes it a lot faster on large datasets. It
> also now uses the actual area of the buildings and their intersection, as
> opposed to their bounding boxes. I will work on your other requests, but
> may not have a lot of time for the next few weeks.
>
> Mike
>
> https://github.com/MikeTho16/JOSM-Scripts
>
> Select Duplicate Buildings
> Selects duplicate, or near duplicate, area buildings in JOSM's active
> datalayer.
> A "near duplicate" is a building whose area overlaps another building's
> area by more than 50%. Only the first building encountered of an
> overlapping pair is selected. This is done so the issue does not have to
> be
> looked at twice. The selected buildings are added to the current
> selection.
> Currently only works with buildings that are ways (not multipolygons).
>
> To Run:
> * Install JOSM's Scripting Plugin (only necessary once)
> * Place file in a convenient location on your system (only necessary once)
> * Click "Scripting" (on top menu bar)
> * Click "Run"
> * Click "..." button and select the script file.
> * Click "Run"
>
>
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] finding overlapping buildings

[john whelan]

The problem is how do you fix them?  Having something directly in JOSM is
useful. They tend to appear in clusters so step one is find the cluster.
Step two is sort the duplicates out.

There really is some very poor mapping of buildings and this at least
identifies the ones that there should be no disagreement about whether they
should be deleted or not.

One day we'll sort out what to do about the very badly mapped buildings
that at least two other mappers have referred to as junk but that's another
story.

Cheerio John

On 28 November 2017 at 15:46, Frédéric Rodrigo 
wrote:

> With Osmose you can also get only large building intersection by filter on
> severity
>
> http://osmose.openstreetmap.fr/en/map/#zoom=11&lat=49.9788&;
> lon=8.3169&layer=Mapnik&overlays=T&item=
> 0%2C8300&level=1%2C2&tags=&fixable=
>
> Or addressee the class 2 only.
> http://osmose.openstreetmap.fr/en/map/#item=0&class=2
>
>
> Le 22/11/2017 à 02:26, john whelan a écrit :
>
>> >Osmose has an 'overlapping building' option. Top of the list
>>
>> I want something to feed into JOSM and not just any building that
>> overlaps by 5%.
>>
>> Thanks John
>>
>> On 21 November 2017 at 19:49, Dave F > > wrote:
>>
>> Osmose has an 'overlapping building' option. Top of the list
>>
>> Note: Some building are drawn on top of eachother to produce 3D
>> rendering of multi-storey buildings.
>>
>> DaveF
>>
>>
>>
>> On 21/11/2017 23:16, john whelan wrote:
>>
>>> Can someone describe a method I can locate these in JOSM.  I'm
>>> not after crossing buildings but just those that are mapped twice
>>> so two buildings with 50% or more overlap.
>>>
>>> Straight duplicates aren't a problem but ones that are drawn
>>> twice by two different mappers are.  Yes I know it shouldn't
>>> happen but it does.
>>>
>>> Thanks John
>>>
>>>
>>> ___
>>> talk mailing list
>>> talk@openstreetmap.org 
>>> https://lists.openstreetmap.org/listinfo/talk
>>> 
>>>
>>
>>
>> > =link&utm_campaign=sig-email&utm_content=emailclient>
>> Virus-free. www.avast.com
>> > =link&utm_campaign=sig-email&utm_content=emailclient>
>>
>>
>> <#m_3123707017832396152_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
>>
>> ___
>> talk mailing list
>> talk@openstreetmap.org 
>> https://lists.openstreetmap.org/listinfo/talk
>> 
>>
>>
>>
>>
>> ___
>> talk mailing list
>> talk@openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/talk
>>
>
>
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] finding overlapping buildings

[Frédéric Rodrigo]

You can use the export menu to load all the pinned objects into JOSM, by 
using the remote command.



Le 28/11/2017 à 22:11, john whelan a écrit :
The problem is how do you fix them?  Having something directly in JOSM 
is useful. They tend to appear in clusters so step one is find the 
cluster.  Step two is sort the duplicates out.


There really is some very poor mapping of buildings and this at least 
identifies the ones that there should be no disagreement about whether 
they should be deleted or not.


One day we'll sort out what to do about the very badly mapped 
buildings that at least two other mappers have referred to as junk but 
that's another story.


Cheerio John

On 28 November 2017 at 15:46, Frédéric Rodrigo > wrote:


With Osmose you can also get only large building intersection by
filter on severity


http://osmose.openstreetmap.fr/en/map/#zoom=11&lat=49.9788&lon=8.3169&layer=Mapnik&overlays=T&item=0%2C8300&level=1%2C2&tags=&fixable=



Or addressee the class 2 only.
http://osmose.openstreetmap.fr/en/map/#item=0&class=2



Le 22/11/2017 à 02:26, john whelan a écrit :

>Osmose has an 'overlapping building' option. Top of the list

I want something to feed into JOSM and not just any building
that overlaps by 5%.

Thanks John

On 21 November 2017 at 19:49, Dave F
mailto:davefoxfa...@btinternet.com>
>> wrote:

    Osmose has an 'overlapping building' option. Top of the list

    Note: Some building are drawn on top of eachother to
produce 3D
    rendering of multi-storey buildings.

    DaveF



    On 21/11/2017 23:16, john whelan wrote:

    Can someone describe a method I can locate these in
JOSM.  I'm
    not after crossing buildings but just those that are
mapped twice
    so two buildings with 50% or more overlap.

    Straight duplicates aren't a problem but ones that are
drawn
    twice by two different mappers are.  Yes I know it
shouldn't
    happen but it does.

    Thanks John




___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] finding overlapping buildings

[Pierre Béland]

Wow  again, this time a lot more efficient.  With 29,000 ways, very fast 
Result. 
I did not believe it since only perfect duplicates of buildings were selected. 
This was done by a new contributor that did participate to a geoweek mapathon 
responding for the Volcano emergency in Bali. He did use JOSM and was probably 
not trained enough to look at Validations results from JOSM. At least, he 
should have spot these duplicates of nodes+ways. In less then two weeks this 
contributor edited almost 25,000 JOSM objects (ie, nodes, ways or relations). A 
good prospect but quite important to correct rapidly of new contributors 
working so intensely. 
see this zone 
https://www.openstreetmap.org/#map=17/-8.447237794027034/115.40394650097596
Looking at this Bali area, I saw many new contributors quite motivated and who 
added a lot of buildings. But it seems not enough monitoring from the 
organizers. This tool can surely help.
 An other interesting aspect with your scripts, you let us learn how to make 
such scripts.
regard

Pierre 
 

Le mardi 28 novembre 2017 15:13:22 HNE, Mike Thompson  
a écrit :  
 
 John, Pierre,
I made some improvements to the select duplicate buildings script. It now uses 
a spatial index, which makes it a lot faster on large datasets. It also now 
uses the actual area of the buildings and their intersection, as opposed to 
their bounding boxes. I will work on your other requests, but may not have a 
lot of time for the next few weeks.
Mike
https://github.com/MikeTho16/JOSM-Scripts

Select Duplicate Buildings 
Selects duplicate, or near duplicate, area buildings in JOSM's active 
datalayer.A "near duplicate" is a building whose area overlaps another 
building'sarea by more than 50%. Only the first building encountered of an 
overlapping pair is selected. This is done so the issue does not have to be 
looked at twice. The selected buildings are added to the current selection.  
Currently only works with buildings that are ways (not multipolygons).
To Run:* Install JOSM's Scripting Plugin (only necessary once)* Place file in a 
convenient location on your system (only necessary once)* Click "Scripting" (on 
top menu bar)* Click "Run"* Click "..." button and select the script file.* 
Click "Run"

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk
  ___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] finding overlapping buildings

[SandorS]

John,
yes, the buildings in Africa are really messed up. But yet, they are there in 
contrast to some largest digital map-makers. Just take a look on some examples 
like here https://goo.gl/CYVahc or here https://goo.gl/UWuBcf or here 
https://goo.gl/9iNYCm or here https://goo.gl/jUuqmS. There are several 10s of 
thousands similar errors like in the examples. But even though the absolute 
number is large, it is only a percent or two compared to the total number of 
buildings in Africa. And as I told you it is possible to repair almost all the 
above illustrated anomalies. It is challenging and I am working on it using the 
source data (XML dump) from some weeks ago.
So, if you are patient for several days, you will get the buildings for Africa 
probably with the best preparation and based on the potential in the source 
data. The data will be in shp format accompanied by some statistics, additional 
geometries for control and a short documentation.
Of course, if you don need these I will not bother you. But at least, you could 
use the stuff as an option to compare if you use other models. Anyway, the 
subject might not be of interest for other members so if you still want the 
buildings we could just switch to bilateral communication about some details 
(for instance Africa, the continent and Madagascar, eventually some small 
islands, in which projection(s) and so on).
Regards, Sandor.

Sent from Mail for Windows 10

From: john whelan
Sent: 28 November 2017 22:13
To: Frédéric Rodrigo
Cc: OpenStreetMap talk mailing list
Subject: Re: [OSM-talk] finding overlapping buildings

The problem is how do you fix them?  Having something directly in JOSM is 
useful. They tend to appear in clusters so step one is find the cluster.  Step 
two is sort the duplicates out.

There really is some very poor mapping of buildings and this at least 
identifies the ones that there should be no disagreement about whether they 
should be deleted or not.

One day we'll sort out what to do about the very badly mapped buildings that at 
least two other mappers have referred to as junk but that's another story.

Cheerio John


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] finding overlapping buildings

[Pierre Béland]

Hi Frederik, 
I tried with the area I provided as example in Bali. Interesting to learn that 
we can export a group like that to JOSM for editing / correcting.See this zone 
https://www.openstreetmap.org/#map=17/-8.447237794027034/115.40394650097596

As I said in a thread for Maproulette, QA tools such as MapRoulette or Osmose, 
while exporting data, should use a common  tag added to te changeset metadata 
to describe the Coordination QA tools used to spot problems and edit an area. I 
am not sure if only the comment is recognized actually. If so, a hashtag could 
refer to both osmose and the item with something like #Osmose-item-xxx where 
xxx describes the item.
See this JOSM ticket that address transfer of HOT TM tags 
https://josm.openstreetmap.de/ticket/10758and this HOT TM ticket 
https://github.com/hotosm/osm-tasking-manager2/issues/703

 
Pierre 
 

Le mardi 28 novembre 2017 16:17:55 HNE, Frédéric Rodrigo 
 a écrit :  
 
 You can use the export menu to load all the pinned objects into JOSM, by 
using the remote command.


Le 28/11/2017 à 22:11, john whelan a écrit :
> The problem is how do you fix them?  Having something directly in JOSM 
> is useful. They tend to appear in clusters so step one is find the 
> cluster.  Step two is sort the duplicates out.
>
> There really is some very poor mapping of buildings and this at least 
> identifies the ones that there should be no disagreement about whether 
> they should be deleted or not.
>
> One day we'll sort out what to do about the very badly mapped 
> buildings that at least two other mappers have referred to as junk but 
> that's another story.
>
> Cheerio John
>
> On 28 November 2017 at 15:46, Frédéric Rodrigo  > wrote:
>
>    With Osmose you can also get only large building intersection by
>    filter on severity
>
>    
>http://osmose.openstreetmap.fr/en/map/#zoom=11⪫=49.9788&lon=8.3169&layer=Mapnik&overlays=T&item=0%2C8300&level=1%2C2&tags=&fixable=
>    
>
>
>    Or addressee the class 2 only.
>    http://osmose.openstreetmap.fr/en/map/#item=0&class=2
>    
>
>
>    Le 22/11/2017 à 02:26, john whelan a écrit :
>
>        >Osmose has an 'overlapping building' option. Top of the list
>
>        I want something to feed into JOSM and not just any building
>        that overlaps by 5%.
>
>        Thanks John
>
>        On 21 November 2017 at 19:49, Dave F
>                
>                >> wrote:
>
>            Osmose has an 'overlapping building' option. Top of the list
>
>            Note: Some building are drawn on top of eachother to
>        produce 3D
>            rendering of multi-storey buildings.
>
>            DaveF
>
>
>
>            On 21/11/2017 23:16, john whelan wrote:
>
>                Can someone describe a method I can locate these in
>            JOSM.  I'm
>                not after crossing buildings but just those that are
>            mapped twice
>                so two buildings with 50% or more overlap.
>
>                Straight duplicates aren't a problem but ones that are
>            drawn
>                twice by two different mappers are.  Yes I know it
>            shouldn't
>                happen but it does.
>
>                Thanks John
>


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk
  ___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] finding overlapping buildings

[john whelan]

I don't have an interest in buildings as such what a lot of my mapping is
concerned with validating HOT projects.  I generally sit on three and check
the work as its done.  Working like this I can catch new mappers making
mistakes and give them gentle feedback to get the overall quality of
mapping up a little.  It also helps get the project completed.

We are talking very new mappers here doing things like labeling buildings
name=road.  building=unclassified, highway=yes.  area=yes has dropped off
recently from the new mappers but its still fairly common.

As a percentage the errors aren't that high but from a data quality point
of view every little helps and if you catch them young they'll add fewer
errors in future or at least that is the hope.

Maproulette I seem to recall will spot crossing highways with no node.
Currently there are about a thousand of these in Egypt looking at
unclassified and above.  JOSM validation does quite a nice job of detecting
these and its a lot quicker than running maproulette.  Occasionally I'd
come across a pocket of duplicated buildings where one mapper had drawn
over the buildings a second time.  Sometimes it was even the same mapper
and that's where the interest came from.

I am aware that the area and number of buildings is used to estimate
population the exact incantations needed to do this I'm not familiar with
but duplicated buildings throw these calculations off because the same
building is counted twice and that's why I'd like to get rid of them.

One problem with using an XML dump is they age very quickly.  I use country
dumps, for adding crossing nodes and untagged ways they are fine for
roughly four days.  After that you start to get conflicts.  When deleting
the last thing you want to do is delete one of a pair of buildings if the
other has already been deleted.  So for deleting duplicate buildings it's
very much a case of go to the area and freshly download it.

For the case of twenty buildings mapped as one.  I really don't know what
to do.  From a population count point of view the area is roughly right and
generally I leave them alone.

Again for badly mapped buildings where they aren't square what do you do?
If you square them its an approximation and my science background says
anytime you approximate something you are adding in errors.

For ones that are three times the size of the buildings and look like a
flag blowing in the wind?  Again what do you do?  I think these are close
to junk and should probably be deleted.  Others will say no they should be
cleaned up.  Fine except cleaning them up takes far more effort than it
would take to remap them with the building_tool plugin and they really
screw up the population estimates.  Many of these have a HOT project number
in the tag by the way.

Cheerio John

On 28 November 2017 at 17:57, SandorS  wrote:

> John,
>
> yes, the buildings in Africa are really messed up. But yet, they are there
> in contrast to some largest digital map-makers. Just take a look on some
> examples like here https://goo.gl/CYVahc or here https://goo.gl/UWuBcf or
> here https://goo.gl/9iNYCm or here https://goo.gl/jUuqmS. There are
> several 10s of thousands similar errors like in the examples. But even
> though the absolute number is large, it is only a percent or two compared
> to the total number of buildings in Africa. And as I told you it is
> possible to repair almost all the above illustrated anomalies. It is
> challenging and I am working on it using the source data (XML dump) from
> some weeks ago.
>
> So, if you are patient for several days, you will get the buildings for
> Africa probably with the best preparation and based on the potential in the
> source data. The data will be in shp format accompanied by some statistics,
> additional geometries for control and a short documentation.
>
> Of course, if you don need these I will not bother you. But at least, you
> could use the stuff as an option to compare if you use other models.
> Anyway, the subject might not be of interest for other members so if you
> still want the buildings we could just switch to bilateral communication
> about some details (for instance Africa, the continent and Madagascar,
> eventually some small islands, in which projection(s) and so on).
>
> Regards, Sandor.
>
>
>
> Sent from Mail  for
> Windows 10
>
>
>
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] finding overlapping buildings

[Mike Thompson]

On Tue, Nov 28, 2017 at 2:28 PM, Pierre Béland  wrote:

> Wow  again, this time a lot more efficient.  With 29,000 ways, very fast
> Result.
>
Glad it is fast on 29,000, I only tested on 2,500.  Any additional ideas or
suggestions can be entered on GitHub (or you can just send them to me
directly).


>
> An other interesting aspect with your scripts, you let us learn how to
> make such scripts.
>
Happy to share what I have learned, but I feel I am still in the student
mode myself. Next I would like to create a plugin.
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk