Re: [Talk-us] [Talk-us-newyork] Interested in importing address points in New York State
Fascinating read, thanks so much for the research and the write-up! -- Skyler On July 19, 2020 23:12:52 Kevin Kenny wrote: On Thu, Jul 16, 2020 at 12:46 PM Mateusz Konieczny via Talk-us wrote: Once you write this diary entry (or OSM Wiki page) please post it to the mailing list! Here you go: https://www.openstreetmap.org/user/ke9tv/diary/393684 Feel free to repost, wikify, share as appropriate! Skyler, thanks for trying again to reach out! I'd written, both email and paper, to several people listed as contacts on gis.ny.gov and never received a response. You'll notice that I made a copy of the mail on my personal site and referenced it as an exhibit in the diary entry. (PS: I was shouting at the screen as I read that self-congratulatory article at https://gis.ny.gov/outreach/gist/fall01.htm) -- 73 de ke9tv/2, Kevin ___ Talk-us-newyork mailing list talk-us-newy...@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-us-newyork ___ Talk-us mailing list Talk-us@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] [Talk-us-newyork] Interested in importing address points in New York State
On July 18, 2020 16:07:55 Russell Nelson wrote: Well there you go! You should make a page on the Wiki under Imports, and save this email there. So it doesn't get lost. Like my NYSDEC email did. :( I will make sure to do that when I get a chance! -- Skyler ___ Talk-us mailing list Talk-us@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] [Talk-us-newyork] Interested in importing address points in New York State
On July 18, 2020 14:53:26 Mateusz Konieczny wrote: Not a lawyer, but is it possible that "use the points for any lawful purpose" is nonanswer as a use invaloving copyright infringement would not be lawful? Also not a lawyer, but there is plenty of unambiguous context. I contacted him regarding the SAM Address Points in the NYS GIS Clearing House and he replied that he "would very much like the SAM address points to be included in Open Street Map." There's not much room for confusion or alternative interpretation. -- Skyler ___ Talk-us mailing list Talk-us@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] [Talk-us-newyork] Interested in importing address points in New York State
Well there you go! You should make a page on the Wiki under Imports, and save this email there. So it doesn't get lost. Like my NYSDEC email did. :( On 7/18/20 12:34 PM, Skyler Hawthorne wrote: Well, it turned it to be a lot easier than I was thinking it would be! I reached out to the contact listed on the Clearing House web site, using the template in the wiki page, and he replied confirming that we have permission to use the data. This is the text of the email exchange, and I've also attached the raw .eml file. ___ Talk-us mailing list Talk-us@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] [Talk-us-newyork] Interested in importing address points in New York State
Not a lawyer, but is it possible that "use the points for any lawful purpose" is nonanswer as a use invaloving copyright infringement would not be lawful? Jul 18, 2020, 18:34 by o...@dead10ck.com: > Well, it turned it to be a lot easier than I was thinking it would be! I > reached out to the contact listed on the Clearing House web site, using the > template in the wiki page, and he replied confirming that we have permission > to use the data. This is the text of the email exchange, and I've also > attached the raw .eml file. > > From: Winters, Frank (ITS) frank.wint...@its.ny.gov > Date: July 17, 2020 22:30:24 > Subject: RE: Interested in importing the Address Point data from the Clearing > House into OpenStreetMap > To: Skyler Hawthorne o...@dead10ck.com > CC: Coryell, Rodger (ITS) rodger.cory...@its.ny.gov, Fargione, Craig (ITS) > craig.fargi...@its.ny.gov > > Hi Skyler, nice to hear form you. We would very much like the SAM address > points to be included in Open Street Map. The permitted use of the points is > quite simple. You may use the points for any lawful purpose. While we do our > best to maintain a comprehensive and accurate set of address points with our > limited resources we know it has shortcomings. See the metadata for the > liability disclaimer. > > We generally post quarterly updates to the data set. > > > Frank Winters > Geographic Information Officer > > Office of Information Technology Services > W. Averell Harriman State Office Campus > Bldg. 5 - Floor 1 > Albany, NY 12226 > 518.242.5036 | 518.281.9140 m | frank.wint...@its.ny.gov > > > -Original Message- > From: Skyler Hawthorne Sent: Friday, July 17, 2020 6:30 PM > To: Winters, Frank (ITS) > Subject: Interested in importing the Address Point data from the Clearing > House into OpenStreetMap > > ATTENTION: This email came from an external source. Do not open attachments > or click on links from unknown senders or unexpected emails. > > Hello Mr Winters, > > Thank you for your part in making the GIS data for New York State available > to the public through the Clearing House project! > > I am a contributor to the OpenStreetMap project [1], a collaborative open > project to create a global geodata set freely usable by anyone [2]. > > We respect the IP rights of others and I write to ask if we can use this > data. There does not appear to be any explicit information about the license > under which the data sets in the Cleaning House web site are distributed. > It's unclear what the terms are for its use, and specifically whether or not > it is public domain, and if it is permitted to import into the OpenStreetMap > project and redistribute to the world under an open license. > > At the most simple, I would seek a statement like this: > > "The New York State GIS Program Office [or the relevant NYS department(s)] > has no objections to geodata derived in part from the GIS Clearing House data > sets being incorporated into the OpenStreetMap project geodata database and > released under a free and open license" [1] > > I also ask that whatever statement you are prepared to make can be made > public for information purposes. > > Below is a fact sheet. If you would like any more information, I will do my > best to help or can ask our project's License Working Group to get in touch > with you. > > Regards, > Skyler Hawthorne > > Fact Sheet > > [1] The OpenStreetMap project currently has over 750,000 registered > contributors worldwide. Our main website is https://www.openstreetmap.org > > [2] We are mandated to make our geodata available in perpetuity under a free > and open licence. We are not allowed to use a commercial license, but > commercial organisations are allowed to use our data under similar terms. > > [3] Our data is currently published under the Open Database License 1.0, > https://protect2.fireeye.com/v1/url?k=76761582-2a4eb33f-7674ecb7-000babd9fa3f-f71edf933744da0d=1=391ef603-5912-439f-b6e4-b8ac749598bd=http%3A%2F%2Fwww.opendatacommons.org%2Flicenses%2Fodbl%2F > > [4] Most of our geodata is contributed by individuals. However, we are very > grateful when able to incorporate or derive from other geo-data datasets > where license terms are compatible. > > [5] We formally attribute all such sources at > https://wiki.openstreetmap.org/wiki/Attribution, using any specific wording > if you request. We also try to provide a link to this page with any extract > of data from our database. However, for reasons of practicality, we do not > require end-users to repeat such attribution since it runs into hundreds. > > [6] We also keep a public track of third party data use at > https://wiki.openstreetmap.org/wiki/Import/Catalogue and usually have a > project page for each dataset, describing how we use it and whether there are > any license restrictions to be aware of. > > [7] If you have any specific legal questions, the OpenStreetMap Foundation's > License Working Group can be reached
Re: [Talk-us] [Talk-us-newyork] Interested in importing address points in New York State
Well, it turned it to be a lot easier than I was thinking it would be! I reached out to the contact listed on the Clearing House web site, using the template in the wiki page, and he replied confirming that we have permission to use the data. This is the text of the email exchange, and I've also attached the raw .eml file. From: Winters, Frank (ITS) frank.wint...@its.ny.gov Date: July 17, 2020 22:30:24 Subject: RE: Interested in importing the Address Point data from the Clearing House into OpenStreetMap To: Skyler Hawthorne o...@dead10ck.com CC: Coryell, Rodger (ITS) rodger.cory...@its.ny.gov, Fargione, Craig (ITS) craig.fargi...@its.ny.gov Hi Skyler, nice to hear form you. We would very much like the SAM address points to be included in Open Street Map. The permitted use of the points is quite simple. You may use the points for any lawful purpose. While we do our best to maintain a comprehensive and accurate set of address points with our limited resources we know it has shortcomings. See the metadata for the liability disclaimer. We generally post quarterly updates to the data set. Frank Winters Geographic Information Officer Office of Information Technology Services W. Averell Harriman State Office Campus Bldg. 5 - Floor 1 Albany, NY 12226 518.242.5036 | 518.281.9140 m | frank.wint...@its.ny.gov -Original Message- From: Skyler Hawthorne Sent: Friday, July 17, 2020 6:30 PM To: Winters, Frank (ITS) Subject: Interested in importing the Address Point data from the Clearing House into OpenStreetMap ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. Hello Mr Winters, Thank you for your part in making the GIS data for New York State available to the public through the Clearing House project! I am a contributor to the OpenStreetMap project [1], a collaborative open project to create a global geodata set freely usable by anyone [2]. We respect the IP rights of others and I write to ask if we can use this data. There does not appear to be any explicit information about the license under which the data sets in the Cleaning House web site are distributed. It's unclear what the terms are for its use, and specifically whether or not it is public domain, and if it is permitted to import into the OpenStreetMap project and redistribute to the world under an open license. At the most simple, I would seek a statement like this: "The New York State GIS Program Office [or the relevant NYS department(s)] has no objections to geodata derived in part from the GIS Clearing House data sets being incorporated into the OpenStreetMap project geodata database and released under a free and open license" [1] I also ask that whatever statement you are prepared to make can be made public for information purposes. Below is a fact sheet. If you would like any more information, I will do my best to help or can ask our project's License Working Group to get in touch with you. Regards, Skyler Hawthorne Fact Sheet [1] The OpenStreetMap project currently has over 750,000 registered contributors worldwide. Our main website is https://www.openstreetmap.org [2] We are mandated to make our geodata available in perpetuity under a free and open licence. We are not allowed to use a commercial license, but commercial organisations are allowed to use our data under similar terms. [3] Our data is currently published under the Open Database License 1.0, https://protect2.fireeye.com/v1/url?k=76761582-2a4eb33f-7674ecb7-000babd9fa3f-f71edf933744da0d=1=391ef603-5912-439f-b6e4-b8ac749598bd=http%3A%2F%2Fwww.opendatacommons.org%2Flicenses%2Fodbl%2F [4] Most of our geodata is contributed by individuals. However, we are very grateful when able to incorporate or derive from other geo-data datasets where license terms are compatible. [5] We formally attribute all such sources at https://wiki.openstreetmap.org/wiki/Attribution, using any specific wording if you request. We also try to provide a link to this page with any extract of data from our database. However, for reasons of practicality, we do not require end-users to repeat such attribution since it runs into hundreds. [6] We also keep a public track of third party data use at https://wiki.openstreetmap.org/wiki/Import/Catalogue and usually have a project page for each dataset, describing how we use it and whether there are any license restrictions to be aware of. [7] If you have any specific legal questions, the OpenStreetMap Foundation's License Working Group can be reached at le...@osmfoundation.org and will be glad to help. -- Skyler <> ___ Talk-us mailing list Talk-us@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] [Talk-us-newyork] Interested in importing address points in New York State
Well, it turned it to be a lot easier than I was thinking it would be! I reached out to the contact listed on the Clearing House web site, using the template in the wiki page, and he replied confirming that we have permission to use the data. This is the text of the email exchange, and I've also attached the raw .eml file. From: Winters, Frank (ITS) frank.wint...@its.ny.gov Date: July 17, 2020 22:30:24 Subject: RE: Interested in importing the Address Point data from the Clearing House into OpenStreetMap To: Skyler Hawthorne o...@dead10ck.com CC: Coryell, Rodger (ITS) rodger.cory...@its.ny.gov, Fargione, Craig (ITS) craig.fargi...@its.ny.gov Hi Skyler, nice to hear form you. We would very much like the SAM address points to be included in Open Street Map. The permitted use of the points is quite simple. You may use the points for any lawful purpose. While we do our best to maintain a comprehensive and accurate set of address points with our limited resources we know it has shortcomings. See the metadata for the liability disclaimer. We generally post quarterly updates to the data set. Frank Winters Geographic Information Officer Office of Information Technology Services W. Averell Harriman State Office Campus Bldg. 5 - Floor 1 Albany, NY 12226 518.242.5036 | 518.281.9140 m | frank.wint...@its.ny.gov -Original Message- From: Skyler Hawthorne Sent: Friday, July 17, 2020 6:30 PM To: Winters, Frank (ITS) Subject: Interested in importing the Address Point data from the Clearing House into OpenStreetMap ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. Hello Mr Winters, Thank you for your part in making the GIS data for New York State available to the public through the Clearing House project! I am a contributor to the OpenStreetMap project [1], a collaborative open project to create a global geodata set freely usable by anyone [2]. We respect the IP rights of others and I write to ask if we can use this data. There does not appear to be any explicit information about the license under which the data sets in the Cleaning House web site are distributed. It's unclear what the terms are for its use, and specifically whether or not it is public domain, and if it is permitted to import into the OpenStreetMap project and redistribute to the world under an open license. At the most simple, I would seek a statement like this: "The New York State GIS Program Office [or the relevant NYS department(s)] has no objections to geodata derived in part from the GIS Clearing House data sets being incorporated into the OpenStreetMap project geodata database and released under a free and open license" [1] I also ask that whatever statement you are prepared to make can be made public for information purposes. Below is a fact sheet. If you would like any more information, I will do my best to help or can ask our project's License Working Group to get in touch with you. Regards, Skyler Hawthorne Fact Sheet [1] The OpenStreetMap project currently has over 750,000 registered contributors worldwide. Our main website is https://www.openstreetmap.org [2] We are mandated to make our geodata available in perpetuity under a free and open licence. We are not allowed to use a commercial license, but commercial organisations are allowed to use our data under similar terms. [3] Our data is currently published under the Open Database License 1.0, https://protect2.fireeye.com/v1/url?k=76761582-2a4eb33f-7674ecb7-000babd9fa3f-f71edf933744da0d=1=391ef603-5912-439f-b6e4-b8ac749598bd=http%3A%2F%2Fwww.opendatacommons.org%2Flicenses%2Fodbl%2F [4] Most of our geodata is contributed by individuals. However, we are very grateful when able to incorporate or derive from other geo-data datasets where license terms are compatible. [5] We formally attribute all such sources at https://wiki.openstreetmap.org/wiki/Attribution, using any specific wording if you request. We also try to provide a link to this page with any extract of data from our database. However, for reasons of practicality, we do not require end-users to repeat such attribution since it runs into hundreds. [6] We also keep a public track of third party data use at https://wiki.openstreetmap.org/wiki/Import/Catalogue and usually have a project page for each dataset, describing how we use it and whether there are any license restrictions to be aware of. [7] If you have any specific legal questions, the OpenStreetMap Foundation's License Working Group can be reached at le...@osmfoundation.org and will be glad to help. On July 16, 2020 12:16:19 Kevin Kenny wrote: (By the way, hi, Skyler, and welcome! You've stepped into a difficult area - most programmers don't realize just how difficult until they've waded in. The legal situation in New York is _very_ complicated, because the key court case that governs GIS data
Re: [Talk-us] [Talk-us-newyork] Interested in importing address points in New York State
Thank you so much for your reply! That's exactly the kind of insight I was hoping for by posting here. On July 16, 2020 12:16:19 Kevin Kenny wrote: I'm less sanguine than Skyler is about the data quality. I suspect s/he (the given name doesn't clearly identify a preferred pronoun) has been looking at urban or suburban areas in counties whose GIS departments have relatively stable funding. In those situations, yes, the data are fairly good. There is still a serious conflation issue that isn't addressed, with respect to buildings whose footprints are already mapped but do not bear addresses, where the address point may or may not be in the building footprint. Many address points, too, get clustered at the entrance of a private or shared driveway, rather than being on the indivdual dwellings. I seem to recall that at least one or two of the apartment and townhouse complexes in the general area of https://www.openstreetmap.org/#map=18/42.83211/-73.89931 had to have their house numbers collected on foot, because the E911 data showed all the address points in a single cluster. In the rural areas, particularly in the counties with tiny populations, the situation is grimmer. I'm not certain that Schuyler or Wyoming Counties even would _have_ dedicated GIS departments! Until relatively recently, when grant money was available to have this information in GIS systems for E911 use, they mostly were still using paper maps, often referenced to an unknown datum. (The first job in dealing with any scanned tax plat is figuring out what coordinate frame it's using - around here, NAD27 differs from NAD83 by a few tens of metres.) The address points may be parcel centroids, or building centroids, or the point where the driveway meets the road, or even just something that was digitized from a pencil sketch made by an assessor. Import of this sort of data could well prove to be a short-term gain but impose a heavy long-term burden; consider the love-hate relationship that we all have with TIGER. (The import means that we've got a nearly-filled-in map, a lot of which is of halfway-decent quality, and we don't have the mappers to have done it nearly as quickly any other way. Nevertheless, for some years we've been paying the price in bad data and worse conflation.) So, my advice for both legal and technical reasons would be to use caution, and recognize that mechanical import is likely to be a disaster - the data will need to be eyeballed by human beings and corrected. I certainly did not do an extensive check of the quality, so this is a super useful perspective. (I wanted more clarity on the legal aspect before investing more time in that, since, after all, if it's a definite no go from a legal perspective, why waste any time at all?) It's unfortunate that there's such a big variation in quality, although not unexpected, since they come from the counties themselves. However, at least the examples you gave would not necessarily make me consider the data unusable without extensive correction. The way I look at this is: if the point is close enough that were a person to stand right at the exact spot, could they find the place they are looking for? If the answer is yes for the vast majority of the data, then I would call that a net gain for OSM. Furthermore, if the data were never manually reviewed and corrected, would it still be valuable enough to import? You obviously have extensive experience with this data set, so I would trust your judgment on this, but if the worst problems we see are mostly the ones you described, it would sound to me like the pros outweigh the cons, even if the points were never corrected. For example, I've personally seen many roads from TIGER imports that are way way off, or even nonexistent, especially long driveways in deeply rural areas. But the fact that the main named roads are there at all is a huge benefit to OSM, even if not every road is perfectly accurate, and many will simply never be reviewed. (With that said, obviously I would want the data to be as accurate as possible, and I'm not making a case to import all the data as is with no review or correction, but simply thinking through the practical reality of the task of making all the data completely accurate. We don't want perfect to be the enemy of good.) For the issue of conflation with existing buildings with no address tags, that might be too difficult of a case to address without reviewing each and every case by hand, which might be practically infeasible. I've seen a lot of cases where there is a house and a detached garage, or in-law right next to the house. It might be possible to detect if there is only one point that is inside of a building, but for the other cases you mentioned, where it might instead be the centroid of the parcel, or at the intersection of the driveway and the street, I don't think there would be a way around fixing these by hand, which indeed would be
