Re: [OSM-talk] An import in New Zealand, assistance requested

[Robin Paulson]

On 2 September 2017 at 05:07, Éric Gillet  wrote:
>> Well it seems as if exactly -that- wasn't happening which is why this
>> thread was started in the first place. Seems however that the brakes have
>> been put on now, see

the brakes have not been put on; every day, hundreds/thousands more
buildings are added. one mapper listened to what i said and stopped,
but still no change. no mention anywhere of the new data set, or any
published information on how to merge, etc.

> I do not see any reference to such problem in OP.
>
> If the original plan didn't include building, that's fine by me to stop
> because importing building is wholly different from roads.

correct. original plan was roads, plus some other data detailed in the
linz2osm tool. buildings is "only" one more set of data, but it is
*huge*.

-- 
robin

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] An import in New Zealand, assistance requested

[Robin Paulson]

On 1 September 2017 at 02:55, Simon Poole  wrote:
> Sorry for responding to this late.
>
> Just because a specific source has been legally "OK"ed doesn't imply
> that an import of all the data from a specific source is warranted and
> should continue on for all times. The import guidelines are silent on
> this, but I would suggest that revisiting and reviewing such undertaking
> now and then would really make sense.
>
> Not only because we've learnt lots of things in the many, I believe 9,
> years since the LINZ import started, but further because in those 9
> years the community has likely completely changed, a quick check
> indicated that 20 times more people have mapped in NZ than when the
> import started. As the complaint at hand nicely illustrates, maybe going
> back and checking what the community thinks is appropriate now would be
> a good idea, instead of trudging along on a course set by 100 people
> many years back.

yep, i was one of those 100 people (actually, there were originally 3 of us...)

the process was poor, little involvement of others or request for
comment. granted, this number has expended since, but only at a "how
do we" not at a "should we" level.

-- 
robin

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] An import in New Zealand, assistance requested

[Robin Paulson]

On 17 August 2017 at 01:29, Richard  wrote:
> soneone else has to answer that but I was under the impression that
> the LINZ has been looked at in detail. It seems to be an import effort
> that is underway since many years.

yes, but they keep adding more and more detail to the "LINZ" pile.
LINZ is an intermediary (although they also produce data, to make
things more complicated), they host data for several councils, central
government and other orgs in NZ. to claim that because some data
(roads) has been cleared, does not mean that all subsequent data
downloaded from LINZ is also cleared.

> Otoh my questions about LINZ imports were answered adequately. I would
> guess it is a country with a low number of mappers per square kilometer
> as you call it. Sometimes it happens they import the waterfalls long
> before the waterways belonging to them..

AKL is dense enough that there are hundreds of mappers (maybe not all active...)

-- 
robin

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] An import in New Zealand, assistance requested

[Robin Paulson]

On 16 August 2017 at 06:31, Oleksiy Muzalyev
 wrote:
> I see that these buildings have got one author who has on his statistics
> page about 6 thousand edits. How do you know that these buildings were
> actually imported and not drawn manually in an editor? Was there a clear
> acknowledgement about importing of these building? I have got about 9
> thousand edits on mine and I did not do a single import.

it's an import, the data is from auckland council. yes, there are some
members who are drawing by hand (including me). more than one of the
importers has talked to me about this.

-- 
robin

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] An import in New Zealand, assistance requested

[Éric Gillet]

2017-08-31 21:56 GMT+02:00 Simon Poole :
>
> If there is problems with the data or what's being entered on OSM, we can
> stop and think before continuing.
>
> Well it seems as if exactly -that- wasn't happening which is why this
> thread was started in the first place. Seems however that the brakes have
> been put on now, see https://lists.openstreetmap.
> org/pipermail/imports/2017-July/005118.html
>

I do not see any reference to such problem in OP.

If the original plan didn't include building, that's fine by me to stop
because importing building is wholly different from roads.
But if it did, and the data is valid, precise and license-compatible, it's
sad to stop because of minute details imo.
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] An import in New Zealand, assistance requested

[Simon Poole]

Am 31.08.2017 um 20:56 schrieb Éric Gillet:
> 2017-08-31 16:55 GMT+02:00 Simon Poole  >:
>
> Sorry for responding to this late.
>
> Just because a specific source has been legally "OK"ed doesn't imply
> that an import of all the data from a specific source is warranted
> and 
> should continue on for all times. The import guidelines are silent on
> this, but I would suggest that revisiting and reviewing such
> undertaking
> now and then would really make sense.
>
>
> Meh, If the import has been going on for years, and the only objection
> is about "process" or meta-data, I'd say they should continue.
I can guarantee you that they didn't start off with buildings (the
original import was road data it seems).

> If there is problems with the data or what's being entered on OSM, we
> can stop and think before continuing.
Well it seems as if exactly -that- wasn't happening which is why this
thread was started in the first place. Seems however that the brakes
have been put on now, see
https://lists.openstreetmap.org/pipermail/imports/2017-July/005118.html

Simon
>
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk



signature.asc
Description: OpenPGP digital signature
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] An import in New Zealand, assistance requested

[Éric Gillet]

2017-08-31 16:55 GMT+02:00 Simon Poole :

> Sorry for responding to this late.
>
> Just because a specific source has been legally "OK"ed doesn't imply
> that an import of all the data from a specific source is warranted and
> should continue on for all times. The import guidelines are silent on
> this, but I would suggest that revisiting and reviewing such undertaking
> now and then would really make sense.


Meh, If the import has been going on for years, and the only objection is
about "process" or meta-data, I'd say they should continue.
If there is problems with the data or what's being entered on OSM, we can
stop and think before continuing.
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] An import in New Zealand, assistance requested

[Simon Poole]

Sorry for responding to this late.

Just because a specific source has been legally "OK"ed doesn't imply
that an import of all the data from a specific source is warranted and 
should continue on for all times. The import guidelines are silent on
this, but I would suggest that revisiting and reviewing such undertaking
now and then would really make sense.

Not only because we've learnt lots of things in the many, I believe 9,
years since the LINZ import started, but further because in those 9
years the community has likely completely changed, a quick check
indicated that 20 times more people have mapped in NZ than when the
import started. As the complaint at hand nicely illustrates, maybe going
back and checking what the community thinks is appropriate now would be
a good idea, instead of trudging along on a course set by 100 people
many years back. 

Simon


Am 18.08.2017 um 03:24 schrieb Andrew Harvey:
> On 16 August 2017 at 23:29, Richard  wrote:
>> On Wed, Aug 16, 2017 at 07:44:00AM -0400, john whelan wrote:
 They have data for all of New Zealand
>>> 's roads,
>>> released under a license which is compatible with that of OSM.
>>>
>>> Has the Open Data license been cleared by the legal working group?
>> soneone else has to answer that but I was under the impression that
>> the LINZ has been looked at in detail. It seems to be an import effort
>> that is underway since many years.
> LINZ completed the OSMF CC BY waiver so it should be all okay.
>
> See http://wiki.openstreetmap.org/wiki/Contributors#LINZ and
> http://wiki.openstreetmap.org/wiki/File:OSM_waiver_-_LINZ.pdf
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk




signature.asc
Description: OpenPGP digital signature
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] An import in New Zealand, assistance requested

[Andrew Harvey]

On 16 August 2017 at 23:29, Richard  wrote:
> On Wed, Aug 16, 2017 at 07:44:00AM -0400, john whelan wrote:
>> >They have data for all of New Zealand
>> 's roads,
>> released under a license which is compatible with that of OSM.
>>
>> Has the Open Data license been cleared by the legal working group?
>
> soneone else has to answer that but I was under the impression that
> the LINZ has been looked at in detail. It seems to be an import effort
> that is underway since many years.

LINZ completed the OSMF CC BY waiver so it should be all okay.

See http://wiki.openstreetmap.org/wiki/Contributors#LINZ and
http://wiki.openstreetmap.org/wiki/File:OSM_waiver_-_LINZ.pdf

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] An import in New Zealand, assistance requested

[Richard]

On Wed, Aug 16, 2017 at 07:44:00AM -0400, john whelan wrote:
> >They have data for all of New Zealand
> 's roads,
> released under a license which is compatible with that of OSM.
> 
> Has the Open Data license been cleared by the legal working group?

soneone else has to answer that but I was under the impression that
the LINZ has been looked at in detail. It seems to be an import effort 
that is underway since many years.

> >Isn't a local group good enough for local imports?
> 
> Interesting question that was debated in talk-ca.  The local group of
> mappers had a number of meetings on the topic but they were physical
> meetings over coffee.  Some mappers in the rest of Canada felt excluded.
> 
> I'd recommend you jump through the loops with such a major data import.
> There have been problems in the past with the data quality of imports and
> there have been issues with the licensing plus there are issues around
> should any imports be accepted.  This view is held by many in countries
> that have a high density of mappers per square kilometre and not so many in
> countries with fewer mappers.

I know what you mean... some imports in Canada could have been better
and asking the responsible OSM accounts for information is a waste 
of time.

BTW, I was once subscribed to talk-ca and frustrated that noone of my
messages posted there went through (particularly asking about those 
problematic imports). What kind of strange ml policy is that?

Otoh my questions about LINZ imports were answered adequately. I would 
guess it is a country with a low number of mappers per square kilometer 
as you call it. Sometimes it happens they import the waterfalls long 
before the waterways belonging to them..

Richard


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] An import in New Zealand, assistance requested

[Oleksiy Muzalyev]

I would add - the objective problems. In the 20th century, when these 
legacy databases were created, there was no GPS, no satellite imagery as 
we know it.  Officials had to actually walk and use manual tools to map 
properties what is certainly prone to human errors. Databases themselves 
were unreliable, the data was often got corrupted or partially corrupted 
at the drop of a hat, the first versions of MySQL and PostgreSQL 
appeared by the end of 90s only.


Nowadays, when the e-commerce requires imagery processing on an 
industrial scale, a lot of automatic optical shape & patterns 
recognition software is being developed, because a product has to be 
extracted from a background to get a "flying on air" style. But in the 
product photography a photo-studio with the controlled conditions could 
be used, besides there are billions of new products each year, and this 
process will never end.


It is quite different with the Earth surface. We've got a finite amount 
of land to map. There will be no new land (neither on Earth nor on other 
planets, at least not in a foreseeable future). Besides new technologies 
arrive, like better satellite imagery quality, more GPS-style systems, 
affordable industrial grade RPAS, faster databases, better editing 
tools, better displays, etc.


There is place for legacy maps. I myself like to view historical maps, 
but I am not sure that it makes sense to import massively this 
historical data into the main OSM map.


Best regards,
Oleksiy

On 8/16/2017 1:44 PM, john whelan wrote:
...There have been problems in the past with the data quality of 
imports ...


Cheerio John


___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] An import in New Zealand, assistance requested

[john whelan]

>They have data for all of New Zealand
's roads,
released under a license which is compatible with that of OSM.

Has the Open Data license been cleared by the legal working group?

Fairly recently some data was imported from the City of Ottawa, Canada.  It
only took five years to resolve the Open Data licensing issues but they are
now resolved.  Note the City released the data under what they considered
to be an Open Data license around six years ago.  It's only recently that
they have amended their Open Data license and it took a fair chunk of work
to get it sorted out.  I understand Toronto's Open Data license which is
different is still under consideration.

>Isn't a local group good enough for local imports?

Interesting question that was debated in talk-ca.  The local group of
mappers had a number of meetings on the topic but they were physical
meetings over coffee.  Some mappers in the rest of Canada felt excluded.

I'd recommend you jump through the loops with such a major data import.
There have been problems in the past with the data quality of imports and
there have been issues with the licensing plus there are issues around
should any imports be accepted.  This view is held by many in countries
that have a high density of mappers per square kilometre and not so many in
countries with fewer mappers.

Cheerio John

On 16 August 2017 at 07:17, Richard  wrote:

> On Tue, Aug 15, 2017 at 11:10:56PM +, Robin Paulson wrote:
>
> > Before I ask for assistance, please note this: I've already brought
> > this up on osm-imports and got nowhere useful.
> >
> > There are a number of users in NZ doing mass imports of building data
> > in Auckland (I estimate 500,000 buildings), while wilfully ignoring
> > community guidelines.
> >
> > I have asked numerous times for those doing the imports to stop and
> > follow the notes on the imports page. There is no wiki page, no
> > discussion on osm-imports, only vague references to a tool called
> > linz2osm and responses of "Oh, I thought everything had been OK'd
> > already, we discussed it on a local (google groups) mailing list".
>
> Isn't a local group good enough for local imports?
>
> Regarding linz2osm, I have been directed to this pages:
>   http://linz2osm.openstreetmap.org.nz/
>   https://wiki.openstreetmap.org/wiki/LINZ
>
> I was in touch with http://www.openstreetmap.org/user/barnaclebarnes_linz
> regarding the waterways a while ago.
>
> Richard
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk
>
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] An import in New Zealand, assistance requested

[Richard]

On Tue, Aug 15, 2017 at 11:10:56PM +, Robin Paulson wrote:

> Before I ask for assistance, please note this: I've already brought
> this up on osm-imports and got nowhere useful.
> 
> There are a number of users in NZ doing mass imports of building data
> in Auckland (I estimate 500,000 buildings), while wilfully ignoring
> community guidelines.
> 
> I have asked numerous times for those doing the imports to stop and
> follow the notes on the imports page. There is no wiki page, no
> discussion on osm-imports, only vague references to a tool called
> linz2osm and responses of "Oh, I thought everything had been OK'd
> already, we discussed it on a local (google groups) mailing list".

Isn't a local group good enough for local imports? 

Regarding linz2osm, I have been directed to this pages:
  http://linz2osm.openstreetmap.org.nz/
  https://wiki.openstreetmap.org/wiki/LINZ

I was in touch with http://www.openstreetmap.org/user/barnaclebarnes_linz
regarding the waterways a while ago.

Richard

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] An import in New Zealand, assistance requested

[Oleksiy Muzalyev]

Dear Robin,

I see that these buildings have got one author who has on his statistics 
page about 6 thousand edits. How do you know that these buildings were 
actually imported and not drawn manually in an editor? Was there a clear 
acknowledgement about importing of these building? I have got about 9 
thousand edits on mine and I did not do a single import.


I am not a stranger to horrors of untangling Gordian knots of automated 
imports and ravages of optical shape recognition software on vast 
expanses. I am not in a Data Group, but I am curious how you realized 
that these are the imports? From where this data could be imported? Was 
it actually imported from a database or drawn automatically by some 
pattern & shape recognition software?


Who could have such a detailed database about hundreds of thousands of 
buildings? Who could enter such data into a legacy database? With what 
tools?


With best regards,
Oleksiy

On 16.08.17 01:10, Robin Paulson wrote:

Before I ask for assistance, please note this: I've already brought
this up on osm-imports and got nowhere useful.
http://osm.org/go/uuU13aiv
There are a number of users in NZ doing mass imports of building data
in Auckland (I estimate 500,000 buildings), while wilfully ignoring
community guidelines.

I have asked numerous times for those doing the imports to stop and
follow the notes on the imports page. There is no wiki page, no
discussion on osm-imports, only vague references to a tool called
linz2osm and responses of "Oh, I thought everything had been OK'd
already, we discussed it on a local (google groups) mailing list".
Sometimes they stop for a while and then restart some time later,
maybe hoping no-one will notice. I hear there are other imports too,
one of the importers offhand mentioned a mass import of waterways,
also with no wiki page, discussion or outside involvement.

I bring it up here, because nobody subscribed to osm-imports appears
willing to back up these requests, to say to the importers that they
should follow the guidelines. A number of the importers are subscribed
to the osm-imports and osm-talk lists, but are strangely quiet when I
mention this request to stop and do things with community involvement.

--
robin

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk




___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] An import in New Zealand, assistance requested

[Marc Gemis]

Isn't this something for the Data Working Group: d...@osmfoundation.org ?


regards

m

On Wed, Aug 16, 2017 at 1:10 AM, Robin Paulson  wrote:
> Before I ask for assistance, please note this: I've already brought
> this up on osm-imports and got nowhere useful.
>
> There are a number of users in NZ doing mass imports of building data
> in Auckland (I estimate 500,000 buildings), while wilfully ignoring
> community guidelines.
>
> I have asked numerous times for those doing the imports to stop and
> follow the notes on the imports page. There is no wiki page, no
> discussion on osm-imports, only vague references to a tool called
> linz2osm and responses of "Oh, I thought everything had been OK'd
> already, we discussed it on a local (google groups) mailing list".
> Sometimes they stop for a while and then restart some time later,
> maybe hoping no-one will notice. I hear there are other imports too,
> one of the importers offhand mentioned a mass import of waterways,
> also with no wiki page, discussion or outside involvement.
>
> I bring it up here, because nobody subscribed to osm-imports appears
> willing to back up these requests, to say to the importers that they
> should follow the guidelines. A number of the importers are subscribed
> to the osm-imports and osm-talk lists, but are strangely quiet when I
> mention this request to stop and do things with community involvement.
>
> --
> robin
>
> ___
> talk mailing list
> talk@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

[OSM-talk] An import in New Zealand, assistance requested

[Robin Paulson]

Before I ask for assistance, please note this: I've already brought
this up on osm-imports and got nowhere useful.

There are a number of users in NZ doing mass imports of building data
in Auckland (I estimate 500,000 buildings), while wilfully ignoring
community guidelines.

I have asked numerous times for those doing the imports to stop and
follow the notes on the imports page. There is no wiki page, no
discussion on osm-imports, only vague references to a tool called
linz2osm and responses of "Oh, I thought everything had been OK'd
already, we discussed it on a local (google groups) mailing list".
Sometimes they stop for a while and then restart some time later,
maybe hoping no-one will notice. I hear there are other imports too,
one of the importers offhand mentioned a mass import of waterways,
also with no wiki page, discussion or outside involvement.

I bring it up here, because nobody subscribed to osm-imports appears
willing to back up these requests, to say to the importers that they
should follow the guidelines. A number of the importers are subscribed
to the osm-imports and osm-talk lists, but are strangely quiet when I
mention this request to stop and do things with community involvement.

--
robin

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk