Re: [Talk-us] Address Node Import for San Francisco
Just wanna say that addressing in SF would be awesome :-) Steve stevecoast.com On Dec 10, 2010, at 1:29 AM, Katie Filbert filbe...@gmail.com wrote: On Thu, Dec 9, 2010 at 6:20 PM, Serge Wroclawski emac...@gmail.com wrote: On Thu, Dec 9, 2010 at 6:00 PM, Gregory Arenius greg...@arenius.com wrote: I've been working on an import of San Francisco address node data. I have several thoughts and questions and would appreciate any feedback. The Wiki page doesn't mention the original dataset url. I have a few concerns: 1) Without seeing the dataset url, it's hard to know anything about the dataset (its age, accuracy, etc.) This is a real problem with imports- knowing the original quality of the dataset before it's imported. The project has had to remove or correct so many bad datasets, it's incredibly annoying. About the data. Its in a shapefile format containing about 230,000 individual nodes. The data is really high quality and all of the addresses I have checked are correct. It has pretty complete coverage of the entire city. MHO is that individual node addresses are pretty awful. If you can import the building outlines, and then attach the addresses to them, great (and you'll need to consider what's to be done with any existing data), but otherwise, IMHO, this dataset just appears as noise. Also, there are a large number of places where there are multiple nodes in one location if there is more than one address at that location. One example would be a house broken into five apartments. Sometimes they keep one address and use apartment numbers and sometimes each apartment gets its own house number. In the latter cases there will be five nodes with different addr:housenumber fields but identical addr:street and lat/long coordinates. Should I keep the individual nodes or should I combine them? Honestly, I think this is a very cart-before-horse. Please consider making a test of your dataset somewhere people can check out, and then solicit feedback on the process. I haven't yet looked into how I plan to do the actual uploading but I'll take care to make sure its easily reversible if anything goes wrong and doesn't hammer any servers. There are people who've spent years with the project and not gotten imports right, I think this is a less trivial problem than you might expect. I've also made a wiki page for the import. Feedback welcome here or on the wiki page. This really belongs on the imports list as well, but my feedback would be: 1) Where's the shapefile? (if for nothing else, than the licnese, but also for feedback) 2) Can you attach the addresses to real objects (rather than standalone nodes)? 3) What metadata will you keep from the other dataset? 4) How will you handle internally conflicting data? 5) How will you handle conflicts with existing OSM data? - Serge A few comments... 1) San Francisco explicitly says they do not have building outline data. :( So, I suppose we get to add buildings ourselves. I do see that SF does have parcels. For DC, we are attaching addresses to buildings when there is a one-to-one relation between them. When there are multiple address nodes for a single building, then we keep them as nodes. In vast majority of cases, we do not have apartment numbers but in some cases we have things like 1120a, 1120b, 1120c that can be imported. Obviously, without a buildings dataset, our approach won't quite apply for SF. 2) I don't consider the addresses as noise. The data is very helpful for geocoding. If the renderer does a sloppy job making noise out of addresses, the renderings should be improved. 3) Having looked at the data catalogue page, I do have concerns about the terms of use and think it's best to get SF to explicitly agree to allow OSM to use the data. http://gispub02.sfgov.org/website/sfshare/index2.asp 4) If you can get explicit permission, then I suggest breaking up the address nodes into smaller chunks (e.g. by census block group), convert them to osm format with Ian's shp-to-osm tool, and check them for quality and against existing OSM data (e.g. existing pois w/ addresses) in JOSM before importing. QGIS and/or PostGIS can be useful for chopping up the data into geographic chunks. This approach gives opportunity to apply due diligence, to check things, and keep chunks small enough that it's reasonably possible to deal with any mistakes or glitches. -Katie ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us -- Katie Filbert filbe...@gmail.com @filbertkm ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us ___ Talk-us mailing list
[Talk-us] High Priority Corridors?
It appears that User:NE2 has added a tag NHS_High_Priority_Corridor to hundred of ways around the country. Has anyone seen such an automated edit proposal anywhere on the mailing lists? Not only is this tag named inconsistantly with the rest of the tags we've used in the past (I'd rather see it show up as nhs:high priority corridor or something, but the tag should be on the route relation, not the ways. Any opinions? Otherwise I will probably end up reverting the changes in my area. ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] High Priority Corridors?
On 12/12/10 10:30 AM, Ian Dees wrote: It appears that User:NE2 has added a tag NHS_High_Priority_Corridor to hundred of ways around the country. Has anyone seen such an automated edit proposal anywhere on the mailing lists? Not only is this tag named inconsistantly with the rest of the tags we've used in the past (I'd rather see it show up as nhs:high priority corridor or something, but the tag should be on the route relation, not the ways. Any opinions? Otherwise I will probably end up reverting the changes in my area. i concur, the NHS tagging should be done as route relations. richard ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Address Node Import for San Francisco
On Dec 9, 2010, at 3:00 PM, Gregory Arenius wrote: About the data. Its in a shapefile format containing about 230,000 individual nodes. The data is really high quality and all of the addresses I have checked are correct. It has pretty complete coverage of the entire city. I've worked with this file before. When I matched it to OSM data two years ago, I found that the SF data had numerous errors, so I wrote this mapping script: http://mike.teczno.com/img/sf-addresses/mapping.py Usage: mapping.py [osm streets csv] [sf streets csv] [street names csv] Here are all the street names in the shapefile: http://mike.teczno.com/img/sf-addresses/sfaddresses.csv Here are all the street names in OSM at the time I did the comparison (may have changed since): http://mike.teczno.com/img/sf-addresses/osm_streets.csv And this is the mapping result I got: http://mike.teczno.com/img/sf-addresses/street_names.csv Hopefully this is helpful, as you'll want to import street names that actually match those in OSM's view of San Francisco. I found some other weird burrs in the data as well, in terms of how it arranges addresses stacked on top of one another in tall buildings. Nothing that can't be dealt with in an import. I also did a bunch of geometry work to match those address points to nearby street segments in order to break up the street grid into addresses segments, but that code is a bit of a rat's nest. The idea was to build up the little block numbers you see rendered here: http://www.flickr.com/photos/mmigurski/5229627985/sizes/l/ Katie's suggestion of breaking the data into smaller chunks is a good one. -mike. michal migurski- m...@stamen.com 415.558.1610 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] High Priority Corridors?
On Sun, Dec 12, 2010 at 10:40 AM, Richard Welty rwe...@averillpark.net wrote: i concur, the NHS tagging should be done as route relations. They don't work as relations, since they're not always routes that can be followed from start to end. ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] High Priority Corridors?
Oops - meant to send this to the list. On Sun, Dec 12, 2010 at 4:29 PM, Nathan Edgars II nerou...@gmail.com wrote: On Sun, Dec 12, 2010 at 10:40 AM, Richard Welty rwe...@averillpark.net wrote: i concur, the NHS tagging should be done as route relations. They don't work as relations, since they're not always routes that can be followed from start to end. Though I guess I could see them as relations, and am willing to convert them. In any case you'd still have an NHS=yes (or NHS=STRAHNET or whatever) tag on the way. ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] High Priority Corridors?
On Sun, Dec 12, 2010 at 3:29 PM, Nathan Edgars II nerou...@gmail.comwrote: On Sun, Dec 12, 2010 at 10:40 AM, Richard Welty rwe...@averillpark.net wrote: i concur, the NHS tagging should be done as route relations. They don't work as relations, since they're not always routes that can be followed from start to end. Then create a new relation that contains the ways that are part of the system? ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] High Priority Corridors?
On Sun, Dec 12, 2010 at 4:50 PM, Ian Dees ian.d...@gmail.com wrote: On Sun, Dec 12, 2010 at 3:29 PM, Nathan Edgars II nerou...@gmail.com wrote: On Sun, Dec 12, 2010 at 10:40 AM, Richard Welty rwe...@averillpark.net wrote: i concur, the NHS tagging should be done as route relations. They don't work as relations, since they're not always routes that can be followed from start to end. Then create a new relation that contains the ways that are part of the system? Wouldn't this be using a relation as a category? I would be willing to change the numbered high-priority corridors to relations, but they won't be strictly routes (then again neither are some state highways that have gaps or spurs). ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] High Priority Corridors?
On Sun, Dec 12, 2010 at 4:11 PM, Nathan Edgars II nerou...@gmail.comwrote: OK, how's this for a relation? http://www.openstreetmap.org/browse/relation/1319303 While I'm happy to see them in a relation, I still question having such data in OSM. If there is no possible way for OSM mappers to improve the data (borders of national and state parks is a good example), I don't think it should be included ... but that's a different argument. ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] High Priority Corridors?
On Sun, Dec 12, 2010 at 5:15 PM, Ian Dees ian.d...@gmail.com wrote: On Sun, Dec 12, 2010 at 4:11 PM, Nathan Edgars II nerou...@gmail.com wrote: OK, how's this for a relation? http://www.openstreetmap.org/browse/relation/1319303 While I'm happy to see them in a relation, I still question having such data in OSM. If there is no possible way for OSM mappers to improve the data (borders of national and state parks is a good example), I don't think it should be included ... but that's a different argument. It does get improved as new alignments are built. The NHS is another data point for determining which roads are the most major, as are intrastate systems like Florida has. ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] High Priority Corridors?
On Sun, Dec 12, 2010 at 10:30 AM, Ian Dees ian.d...@gmail.com wrote: It appears that User:NE2 has added a tag NHS_High_Priority_Corridor to hundred of ways around the country. Has anyone seen such an automated edit proposal anywhere on the mailing lists? Not only is this tag named inconsistantly with the rest of the tags we've used in the past (I'd rather see it show up as nhs:high priority corridor or something, but the tag should be on the route relation, not the ways. Any opinions? Otherwise I will probably end up reverting the changes in my area. Sure would be nice to see people following more / all of the import guidelines. http://wiki.openstreetmap.org/wiki/Import/Guidelines There seem to be many steps missing from this import, not least of all, the discuss it first, document it on the wiki, check the license, and use an import-specific account guidelines. ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] High Priority Corridors?
On Sun, Dec 12, 2010 at 5:25 PM, Richard Weait rich...@weait.com wrote: On Sun, Dec 12, 2010 at 10:30 AM, Ian Dees ian.d...@gmail.com wrote: It appears that User:NE2 has added a tag NHS_High_Priority_Corridor to hundred of ways around the country. Has anyone seen such an automated edit proposal anywhere on the mailing lists? Not only is this tag named inconsistantly with the rest of the tags we've used in the past (I'd rather see it show up as nhs:high priority corridor or something, but the tag should be on the route relation, not the ways. Any opinions? Otherwise I will probably end up reverting the changes in my area. Sure would be nice to see people following more / all of the import guidelines. http://wiki.openstreetmap.org/wiki/Import/Guidelines There seem to be many steps missing from this import, not least of all, the discuss it first, document it on the wiki, check the license, and use an import-specific account guidelines. Perhaps that would be nice, but I don't see what it has to do with this. ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] High Priority Corridors?
On 12/12/10 4:50 PM, Ian Dees wrote: On Sun, Dec 12, 2010 at 3:29 PM, Nathan Edgars II nerou...@gmail.com mailto:nerou...@gmail.com wrote: On Sun, Dec 12, 2010 at 10:40 AM, Richard Welty rwe...@averillpark.net mailto:rwe...@averillpark.net wrote: i concur, the NHS tagging should be done as route relations. They don't work as relations, since they're not always routes that can be followed from start to end. Then create a new relation that contains the ways that are part of the system? i don't think relations containing ways are required to be completely connected, they're just things that are related. richard ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] High Priority Corridors?
On 12/12/10 5:19 PM, Nathan Edgars II wrote: On Sun, Dec 12, 2010 at 5:15 PM, Ian Deesian.d...@gmail.com wrote: On Sun, Dec 12, 2010 at 4:11 PM, Nathan Edgars IInerou...@gmail.com wrote: OK, how's this for a relation? http://www.openstreetmap.org/browse/relation/1319303 While I'm happy to see them in a relation, I still question having such data in OSM. If there is no possible way for OSM mappers to improve the data (borders of national and state parks is a good example), I don't think it should be included ... but that's a different argument. It does get improved as new alignments are built. The NHS is another data point for determining which roads are the most major, as are intrastate systems like Florida has. NHS is a datapoint for what roads the US DOD thinks are important in a military crisis. while i don't necessarily object to documenting it in OSM, it's not clear how much OSM really gains from that documentation, which is why i never pursued trying to get the NHS stuff into OSM myself. richard ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] High Priority Corridors?
On Sun, Dec 12, 2010 at 5:53 PM, Richard Welty rwe...@averillpark.net wrote: NHS is a datapoint for what roads the US DOD thinks are important in a military crisis. You're thinking of the Strategic Highway Network, which, together with the Interstates, is less than half of the entire NHS mileage. The NHS is a much larger system of main roads, as well as intermodal connectors (inventoried separately). The advantage of NHS is that it encourages states to focus on a limited number of high-priority routes and to concentrate on improving them with federal-aid funds. http://www.fhwa.dot.gov/publications/publicroads/96spring/p96sp2.cfm [the use of high-priority routes here is confusing, since it's not the same as the High Priority Corridors defined by Congress] ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] High Priority Corridors?
On 12/12/2010 04:31 PM, Nathan Edgars II wrote: On Sun, Dec 12, 2010 at 5:25 PM, Richard Weait richard-gnthur35lhcavxtiumw...@public.gmane.org wrote: On Sun, Dec 12, 2010 at 10:30 AM, Ian Dees ian.dees-re5jqeeqqe8avxtiumw...@public.gmane.org wrote: It appears that User:NE2 has added a tag NHS_High_Priority_Corridor to hundred of ways around the country. Has anyone seen such an automated edit proposal anywhere on the mailing lists? Not only is this tag named inconsistantly with the rest of the tags we've used in the past (I'd rather see it show up as nhs:high priority corridor or something, but the tag should be on the route relation, not the ways. Any opinions? Otherwise I will probably end up reverting the changes in my area. Sure would be nice to see people following more / all of the import guidelines. http://wiki.openstreetmap.org/wiki/Import/Guidelines There seem to be many steps missing from this import, not least of all, the discuss it first, document it on the wiki, check the license, and use an import-specific account guidelines. Perhaps that would be nice, but I don't see what it has to do with this. You mean, other than what appears to be a large import of data from another source? signature.asc Description: OpenPGP digital signature ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] High Priority Corridors?
On Sun, Dec 12, 2010 at 10:28 PM, Ian Dees ian.d...@gmail.com wrote: While it may not be an import in the sense of convert a shapefile to OSM format and send it to the API, it is a mass inclusion of external data. The import guidelines could still be followed in such situations. It's no more a mass inclusion of external data than tagging a long segment of motorway as highway=motorway. ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
