Re: [Talk-us] Import of addresses for San Francisco, CA
One building can definitely have more than a single address. Addresses provide postal delivery information and don't necessarily correspond 1:1 with other concepts like physical buildings or legal parcels. -mike. michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html > On Jun 18, 2020, at 11:08 AM, Joseph Eisenberg > wrote: > > Legally in San Francisco does the address refer to the property, the > building, or the entrance? > > While I'm from California originally, I admit that I don't know the > definition of an address there. > > If one building can have more than one address (based on separate entrances), > then it would be best to use nodes. > > - Joseph Eisenberg > > On Thu, Jun 18, 2020 at 9:36 AM Yury Yatsynovich <mailto:yury.yatsynov...@gmail.com>> wrote: > I saw addresses in SF mapped both ways -- some are mapped as nodes inside the > buildings or on contours of buildings (as entrances); others are added > directly on the ways/relations of buildings. > > On Thu, Jun 18, 2020 at 12:20 PM Joseph Eisenberg <mailto:joseph.eisenb...@gmail.com>> wrote: > > matching buildings to address points > > I support the idea of adding missing addresses. However, is it best practice > to match the address with the building area? > > Currently in SF are almost all addresses matched with buildings, or are some > mapped as nodes or separate area features? > > -- Joseph Eisenberg > > On Thu, Jun 18, 2020 at 8:30 AM Yury Yatsynovich <mailto:yury.yatsynov...@gmail.com>> wrote: > Greetings! > > I've been recently thinking about importing addresses for San Francisco, CA. > It looks like there has been interest in this kind of import (the page > devoted to it was created in 2010, > https://wiki.openstreetmap.org/wiki/San_Francisco_Address_Import > <https://wiki.openstreetmap.org/wiki/San_Francisco_Address_Import>). > > So, please, consider this message as my "community buy-in": does anyone have > any objections related to this possible import? > By now I've obtained a permit from the data owner > (https://data.sfgov.org/Geographic-Locations-and-Boundaries/Addresses-Enterprise-Addressing-System/3mea-di5p > > <https://data.sfgov.org/Geographic-Locations-and-Boundaries/Addresses-Enterprise-Addressing-System/3mea-di5p>) > and almost finished writing my code for matching buildings to address points. > > If there are no objections, I'll go on with organising all documentation and > sharing the code/resulting .osm-files for review by OSM community. > > Looking forward to receiving your feedback! > > p.s. I have some experience in importing addresses (e.g., see > https://wiki.openstreetmap.org/wiki/Import/Catalogue/MassGIS_Addresses > <https://wiki.openstreetmap.org/wiki/Import/Catalogue/MassGIS_Addresses>) > > > > > -- > Yury Yatsynovich > ___ > Talk-us mailing list > Talk-us@openstreetmap.org <mailto:Talk-us@openstreetmap.org> > https://lists.openstreetmap.org/listinfo/talk-us > <https://lists.openstreetmap.org/listinfo/talk-us> > > > -- > Yury Yatsynovich > ___ > Talk-us mailing list > Talk-us@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk-us ___ Talk-us mailing list Talk-us@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Import of addresses for San Francisco, CA
I support this import. I would also support the import of addresses for neighboring Oakland, CA. -mike. michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html > On Jun 18, 2020, at 8:23 AM, Yury Yatsynovich > wrote: > > Greetings! > > I've been recently thinking about importing addresses for San Francisco, CA. > It looks like there has been interest in this kind of import (the page > devoted to it was created in 2010, > https://wiki.openstreetmap.org/wiki/San_Francisco_Address_Import > <https://wiki.openstreetmap.org/wiki/San_Francisco_Address_Import>). > > So, please, consider this message as my "community buy-in": does anyone have > any objections related to this possible import? > By now I've obtained a permit from the data owner > (https://data.sfgov.org/Geographic-Locations-and-Boundaries/Addresses-Enterprise-Addressing-System/3mea-di5p > > <https://data.sfgov.org/Geographic-Locations-and-Boundaries/Addresses-Enterprise-Addressing-System/3mea-di5p>) > and almost finished writing my code for matching buildings to address points. > > If there are no objections, I'll go on with organising all documentation and > sharing the code/resulting .osm-files for review by OSM community. > > Looking forward to receiving your feedback! > > p.s. I have some experience in importing addresses (e.g., see > https://wiki.openstreetmap.org/wiki/Import/Catalogue/MassGIS_Addresses > <https://wiki.openstreetmap.org/wiki/Import/Catalogue/MassGIS_Addresses>) > > > > > -- > Yury Yatsynovich > ___ > Talk-us mailing list > Talk-us@openstreetmap.org > https://lists.openstreetmap.org/listinfo/talk-us ___ Talk-us mailing list Talk-us@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Burning Man old data, publicity opportunity
Black Rock City moves slightly each year, to minimize impact on any particular spot in the playa. The streets are also given new names, and the city expands to accommodate additional population growth. It's effectively a totally new geography each time. Mikel has been involved in the project in the past, and been given advance private access to survey shapefiles. -mike. On Aug 5, 2013, at 11:36 AM, Kathleen Danielson wrote: I only know a little about Burning Man (seriously, just what I read in Cory Doctorow's Homeland), but mapping BRC makes sense to me. Is it on the exact same location every year? In that case it seems like it would make sense to update the map annually. If they are on different parts of the dessert each year, it would probably make sense to map each one, but once the city is torn down, modify the tags to indicate a past structure. The Historical OSM folks would probably have better guidance on the best way to do this. Either way, it seems like this could be a really neat way to preserve Black Rock City. Are any OSM folks going to be at Burning Man this year? On Mon, Aug 5, 2013 at 2:25 PM, Clay Smalley claysmal...@gmail.com wrote: So Black Rock City is mapped on OpenStreetMap... twice. And both are old versions (the city as it was in 2008 and 2009). Anyone know why this is? Should the two old cities be deleted and replaced with the 2013 city? I have a feeling that this could be a fun publicity opportunity for OSM, if we're the first ones to map out Black Rock City during Burning Man. Not to mention that the kind of people who go to Burning Man would probably rather support OSM over Google Maps, given someone tells them about OSM. It brings in people from everywhere, so ideally they could go back to their respective communities and possibly get more involved in local OSM mapping. Just ranting a pipe dream. Does this sound realistic? ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Burning Man old data, publicity opportunity
I'd love to help with this, but I'm not much of a burner (first last: 2001) so I won't be able to follow up. Sorry! -mike. On Aug 5, 2013, at 11:54 AM, Jeffrey Johnson wrote: He was certainly involved back then, but like me, his interest in BM/BRC has dropped off a bit. I've connected Migurski with the people in the Bay Area that have all the data and hoping he can help get everything together so it can go into the map. Jeff On Mon, Aug 5, 2013 at 11:49 AM, Steven Johnson sejohns...@gmail.com wrote: I'm almost certain that Mikel was involved in one, or both of those '08/'09 efforts to map Black Rock City. Worth contacting him about what it would take to re-do it for 2013. -- SEJ -- twitter: @geomantic -- skype: sejohnson8 There are two types of people in the world. Those that can extrapolate from incomplete data. On Mon, Aug 5, 2013 at 2:25 PM, Clay Smalley claysmal...@gmail.com wrote: So Black Rock City is mapped on OpenStreetMap... twice. And both are old versions (the city as it was in 2008 and 2009). Anyone know why this is? Should the two old cities be deleted and replaced with the 2013 city? I have a feeling that this could be a fun publicity opportunity for OSM, if we're the first ones to map out Black Rock City during Burning Man. Not to mention that the kind of people who go to Burning Man would probably rather support OSM over Google Maps, given someone tells them about OSM. It brings in people from everywhere, so ideally they could go back to their respective communities and possibly get more involved in local OSM mapping. Just ranting a pipe dream. Does this sound realistic? ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] tile.openstreetmap.us down tomorrow
I need to bring the configurations for the vector datasets up to date, now that the Postgres tables are all back. http://openstreetmap.us/~migurski/vector-datasource/ -mike. On Jul 20, 2013, at 4:28 PM, Ian Dees wrote: Hi all, After some confusion about getting our new memory working, we should be back to normal now. Most of the tile.openstreetmap.us layers are working again (including the USGS Large Scale and TIGER 2012 Roads layer). Let me know if you see otherwise. -Ian On Wed, Jul 17, 2013 at 3:43 PM, Ian Dees ian.d...@gmail.com wrote: Yep, there was a hiccup when the data center folks restored the 500GB Postgres partition onto the 20GB root partition, so it's happening again. We should get it back up this evening. On Wed, Jul 17, 2013 at 3:35 PM, Ben Miller bborkmil...@gmail.com wrote: Is the hardware upgrade still in progress? I'm not seeing the TIGER or USGS topo layers currently. Thanks for the upgrades, BTW, whatever they may be! On Mon, Jul 15, 2013 at 10:07 AM, Ian Dees ian.d...@gmail.com wrote: This downtime didn't happen last week. I expect that it will happen today and hopefully be back up by some time tomorrow. On Thu, Jul 11, 2013 at 7:47 PM, Ian Dees ian.d...@gmail.com wrote: Hi folks, We're installing some hardware upgrades in the server running tile.openstreetmap.us, so some of the tile layers in the editors will be unavailable for the day. The layers that might not work: - TIGER 2012 Road Names - USGS Topo Maps - USGS Large Scale Aerials - some other layers that are used less frequently Please let me know if you have any questions, Ian ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Steady increase in the number of mappers in the US
On Jul 19, 2013, at 9:36 AM, Clifford Snow wrote: On Fri, Jul 19, 2013 at 9:03 AM, Alex Barth a...@mapbox.com wrote: The local dimension of OpenStreetMap was exactly why OSM US decided to do #editathons. There's one happening this weekend, join us. http://openstreetmap.us/2013/07/july-summer-editathon/ Our group is signed up. (Seattle) Well I've been promoting this event quite enough on this list so I'm sure everyone of you already has this on the radar :) But share it in your networks if you haven't yet. It's not too late. We had excellent turnout yesterday in San Francisco with almost 20 people in Code for America's Ben Franklin room. We got a lot of newcomers who had attended the June SOTM and were interested in contributing, a near 50/50 gender balance (!!!), and a handful of traditional GIS education people who were looking for connections to the OSM world. Attendees worked on a bunch of projects: cycling route relations in Kansas City, building import process for SF, highly-detailed parking lot models for San Ramon, addresses and business in San Jose, and automated detection of unmapped suburbs from Telenav probe data (not public). Photo: https://twitter.com/michalmigurski/status/358695759769632768/photo/1 Partial attendee list: http://www.openstreetmap.org/user/Alan http://www.openstreetmap.org/user/Allison%20Carpio http://www.openstreetmap.org/user/bdiscoe http://www.openstreetmap.org/user/chachasikes http://www.openstreetmap.org/user/curiousscholar http://www.openstreetmap.org/user/dirtysouth http://www.openstreetmap.org/user/EmilyWask http://www.openstreetmap.org/user/jfire http://www.openstreetmap.org/user/matthieun http://www.openstreetmap.org/user/mdrigo http://www.openstreetmap.org/user/migurski http://www.openstreetmap.org/user/MonoSim http://www.openstreetmap.org/user/Ondrae http://www.openstreetmap.org/user/rduecyg http://www.openstreetmap.org/user/robertstack http://www.openstreetmap.org/user/Thomas%20Hervey Also, here is a JSON API to see the last 25 edits for the #editathon hashtag: http://osm-tags.teczno.com/tag/editathon?callback=do_stuff …and all the #editathon changesets as of two minutes ago: http://mike.teczno.com/img/editathon-2013-07-21-17-43-00Z.csv -mike. michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Turn restriction dispute
I don't agree. NE2’s edits, most of all the route relations, are enormously valuable to OSM in the US. I'm not aware of any precedent for banning a user like this, and I'm not eager to see one set. -mike. --- michal migurski http://mike.teczno.com On Feb 9, 2013, at 9:30 PM, stevea stevea...@softworkers.com wrote: Russ, I second your vote/motion, not that anybody called for a second, or even that I am able to offer it. What I AM able to do is be civil and use the talk-us list, as it is our nationwide forum to discuss. There are plenty of other consensus understandings that might be loosely called rules which make up the fabric of OSM as a community. NE2 has again proven that he is either unwilling or unable to abide by those. Consequently, I think we should inform him that serious discussion of permanently banning him from OSM (this thread) is underway, and his behavior can either change for the better, or he can count on eventually being permanently banned. He has had plenty of opportunities to do so, and so I am not optimistic he will be around much longer. But if the community wants him, that can emerge as a consensus as well. His better (than nothing) edits are in a clear minority compared to the usual messes he makes. He DOES, for better or worse, stir controversy, which is why we discuss, which is part of the community. If, for that reason alone (that he is controversial), there are those who do not wish to ban him, speak up now, as you may (may) be able to make the case that we need somebody like him as an example of what to do with difficult contributors. I think it is unanimous that he is that, at least. I wouldn't miss him if he were gone, either. SteveA California He's banned from (at least) this list. Consequently, you cannot expect him to discuss this issue here. We had a discussion of whether to ban him from editing in the past, which never really got resolved. It just died out. Yes, he's done a lot of editing, and yes, some of his edits have been fruitful, but no, some of his edits have been less than helpful. I wouldn't miss him if he were gone. I vote, not that anybody called for a vote, to ask him to leave. -russ ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Turn restriction dispute
I'm familiar with his work and have run afoul of his views in the past, most recently when I performed a large scale edit to US route relation tags, some of which he did not agree with. I don't know if any were reverted. Nevertheless, I don't see the value in running him out on a rail without more actual evidence of malice, detailed precedents from other bans, and some expectation that the OSMF could help here. These days I'm happier with NE2 than I am with the foundation, believe it or not. -mike. --- michal migurski http://mike.teczno.com On Feb 10, 2013, at 8:53 AM, Serge Wroclawski emac...@gmail.com wrote: On Sun, Feb 10, 2013 at 8:56 AM, Michal Migurski m...@teczno.com wrote: I don't agree. NE2’s edits, most of all the route relations, are enormously valuable to OSM in the US. I'm not aware of any precedent for banning a user like this, and I'm not eager to see one set. Mike, Your information on NE2 is grossly inaccurate. NE2 makes very few positive edits, and many, many destructive ones, as well as previous threats to make more edits that conform with his (and only his) vision of the world. I realize that from a numerical standpoint, it may seem like he is a positive contributor, but this is due to our general acceptance of people even in the face of disagreement. But in NE2's case, he is a bully, and having a bully does not serve the community well. Regarding precedent, this would not be the first person that the OSMF has had to take action on. Others have been banned, but NE2's particular brand of edit has always ridden the line, as he's not explicitly doing anything illegal (ie not copyright violation). But OSM is not his personal playground, and his view that this project is his sandbox to impose his will on (reality and community consensus be damned) is just unacceptable. It's understandable that if you are not familiar with NE2's behavior first hand, that you would see this as a a misunderstanding, but NE2's behavior has been damaging to the project for so long that we simply have no choice but to take actions to protect the project's cohesiveness. - Serge ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] More on TIGER: Where it's likely safe to import
Okay, so I didn't do the density thing yet, but I've made some other additions to the TIGER green map that I think are worth mentioning. I squashed the greens and inverted the blacks so that the map is more legible, potentially easier to print, and less confusing in the difference between the no-editors and yes-editors areas. The URLs are all still the same and the map is here: http://www.openstreetmap.us/~migurski/green-means-go/ If you see any black tiles, clear your cache. The old version is now here: http://www.openstreetmap.us/~migurski/green-means-go-2013-01-03/ I'm starting to add SEO-friendly, single-geography views. I'm just getting started, using counties at first. Every county in the lower 48 will be accessible via a URL like one of these: http://www.openstreetmap.us/~migurski/green-means-go/Iowa/O'Brien-County.html http://www.openstreetmap.us/~migurski/green-means-go/California/Alameda-County.html http://www.openstreetmap.us/~migurski/green-means-go/Illinois/Cook-County.html http://www.openstreetmap.us/~migurski/green-means-go/California/San-Bernardino-County.html http://www.openstreetmap.us/~migurski/green-means-go/California/Riverside-County.html http://www.openstreetmap.us/~migurski/green-means-go/Arizona/Coconino-County.html http://www.openstreetmap.us/~migurski/green-means-go/Louisiana/East-Baton-Rouge-Parish.html http://www.openstreetmap.us/~migurski/green-means-go/New-York/New-York-County.html http://www.openstreetmap.us/~migurski/green-means-go/Virginia/Falls-Church-city.html The county views are open to suggestions, and quite low on detail for the moment. What should go on there? -mike. On Jan 6, 2013, at 5:51 PM, Michal Migurski wrote: Thanks Steve! I think that would be possible, yeah. I already use object density as one of the inputs here, and could use it to tweak the greens somewhat. The colors need a second pass. -mike. On Jan 4, 2013, at 3:00 PM, Steve Coast wrote: Mike Nice. Could you use relative object density per little square as a proxy for population density? I ask because all the bright green areas near me are forests or old milk farms. I don’t care. I want to see high density areas and attack those as a priority since that’s where people are and where they go... Steve From: Michal Migurski m...@teczno.com Sent: January 3, 2013 7:03 PM To: OpenStreetMap US Talk talk-us@openstreetmap.org Subject: Re: [Talk-us] More on TIGER: Where it's likely safe to import On Dec 17, 2012, at 12:40 PM, Michal Migurski wrote: On Dec 16, 2012, at 8:32 PM, Martijn van Exel wrote: Have you looked into full history planet parsing to get a fuller picture of editing history? I took a stab at full history user metrics some time ago using osmjs; https://github.com/mvexel/OSMQualityMetrics/blob/master/UserStats.js - this produces one set of metrics for the entire .osh file you feed it but it may prove useful for future work. I haven't touched this in a while but it should still work :/ I have downloaded a copy and given it a beginning look. I'm new to parsing things of that magnitude; my first thought was to use the full history file for creations/modifications/deletions on nodes and add that to what I'm doing already for ways on the osm2pgsql tables. Does that sound reasonable? Over the holiday break, I've been grinding through the full history file. I'll write more about the process and publish some raw data, but the short version is that I've replaced the Green Means Go tiles with new versions that incorporate edits to the underlying nodes, address some of the feedback I've received, and show a slightly different map of the US. New map: http://www.openstreetmap.us/~migurski/green-means-go/ Old map: http://www.openstreetmap.us/~migurski/green-means-go-2012-12-16/ Charlotte Wolter and NE2 both pointed out that Florida should see a lot more post-import editing that I had originally shown, and in fact that's what the map now shows: http://www.openstreetmap.us/~migurski/green-means-go/#9/28.3213/-81.6257 http://www.openstreetmap.us/~migurski/green-means-go-2012-12-16/#9/28.3213/-81.6257 Urban fringe areas also show more edits, making them less attractive targets for blind imports: http://www.openstreetmap.us/~migurski/green-means-go/#10/37.7707/-122.3451 http://www.openstreetmap.us/~migurski/green-means-go-2012-12-16/#10/37.7707/-122.3451 I've also stripped away the US coastal territory based on NLCD water designation, per SteveC's suggestion. These massive edits to counties in Pennsylvania are interesting to me: http://www.openstreetmap.us/~migurski/green-means-go/#8/40.918/-77.146 -mike. michal migurski- contact info and pgp key: sf/cahttp
[Talk-us] US Shields Development Server (was: paying a debt and making connections)
On Jan 17, 2013, at 1:04 PM, Mike N wrote: On 1/17/2013 10:10 AM, Richard Weait wrote: I did ask, what it would take to get you to hold a local Mappy Hour in your town? That was never answered. So, I ask again, What would it take? More mappers. I've tried everything I could think of to get others interested, but have given up recently - it's all an uphill swim. But.one new mapper in my area was highly motivated to contribute by the US Shields Development Server. He seems to have lost interest in more contributions while waiting for a possible real US Shields Server. This sounds interesting, what is it? -mike. michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] identifying TIGER deserts
On Jan 6, 2013, at 4:30 AM, Alex Barth wrote: On Jan 4, 2013, at 3:33 PM, Richard Welty rwe...@averillpark.net wrote: On 1/4/13 1:31 PM, Michal Migurski wrote: Interesting—how would you characterize bad roads? One characteristic of crappy TIGER data is road wiggliness, is that what you mean? the tiger 2010/11/12 data is much better for many of the bad tiger areas. it'd be a bit of work to do a comparison of the data, but it might be able to yield the desired information. This is exactly the direction I'm thinking. What would it take to create a TIGER 07 vs 12 comparison map and use this as a basis for identifying the areas that need most attention? I've been talking to Ian over here and we were thinking of intersecting OSM highway data with TIGER 12. This could be achieved e. g. by overlaying a light, opaque OSM highway layer with a contrasting TIGER layer, only exposing TIGER 12 geometry where it differs from OSM. Thoughts? This approach makes sense. You're thinking to come up with a metric for mismatched area between the two datasets? Would it be enough to total it up into 1x1 km buckets or would you want it to be substantially more detailed? -mike. michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] More on TIGER: Where it's likely safe to import
Thanks Steve! I think that would be possible, yeah. I already use object density as one of the inputs here, and could use it to tweak the greens somewhat. The colors need a second pass. -mike. On Jan 4, 2013, at 3:00 PM, Steve Coast wrote: Mike Nice. Could you use relative object density per little square as a proxy for population density? I ask because all the bright green areas near me are forests or old milk farms. I don’t care. I want to see high density areas and attack those as a priority since that’s where people are and where they go... Steve From: Michal Migurski m...@teczno.com Sent: January 3, 2013 7:03 PM To: OpenStreetMap US Talk talk-us@openstreetmap.org Subject: Re: [Talk-us] More on TIGER: Where it's likely safe to import On Dec 17, 2012, at 12:40 PM, Michal Migurski wrote: On Dec 16, 2012, at 8:32 PM, Martijn van Exel wrote: Have you looked into full history planet parsing to get a fuller picture of editing history? I took a stab at full history user metrics some time ago using osmjs; https://github.com/mvexel/OSMQualityMetrics/blob/master/UserStats.js - this produces one set of metrics for the entire .osh file you feed it but it may prove useful for future work. I haven't touched this in a while but it should still work :/ I have downloaded a copy and given it a beginning look. I'm new to parsing things of that magnitude; my first thought was to use the full history file for creations/modifications/deletions on nodes and add that to what I'm doing already for ways on the osm2pgsql tables. Does that sound reasonable? Over the holiday break, I've been grinding through the full history file. I'll write more about the process and publish some raw data, but the short version is that I've replaced the Green Means Go tiles with new versions that incorporate edits to the underlying nodes, address some of the feedback I've received, and show a slightly different map of the US. New map: http://www.openstreetmap.us/~migurski/green-means-go/ Old map: http://www.openstreetmap.us/~migurski/green-means-go-2012-12-16/ Charlotte Wolter and NE2 both pointed out that Florida should see a lot more post-import editing that I had originally shown, and in fact that's what the map now shows: http://www.openstreetmap.us/~migurski/green-means-go/#9/28.3213/-81.6257 http://www.openstreetmap.us/~migurski/green-means-go-2012-12-16/#9/28.3213/-81.6257 Urban fringe areas also show more edits, making them less attractive targets for blind imports: http://www.openstreetmap.us/~migurski/green-means-go/#10/37.7707/-122.3451 http://www.openstreetmap.us/~migurski/green-means-go-2012-12-16/#10/37.7707/-122.3451 I've also stripped away the US coastal territory based on NLCD water designation, per SteveC's suggestion. These massive edits to counties in Pennsylvania are interesting to me: http://www.openstreetmap.us/~migurski/green-means-go/#8/40.918/-77.146 -mike. michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] identifying TIGER deserts
On Jan 4, 2013, at 1:33 AM, Alex Barth wrote: On Jan 3, 2013, at 7:06 PM, Michal Migurski m...@teczno.com wrote: Sounds great, I'm actually in the middle of publishing a followup to the green-means-go map after ploughing through the full planet history dump. Great Tue 1/8, 5PM Eastern. My handle is lx_barth, I'll be on #osm-dev at the same time (alexb). Anybody who's interested here in identifying TIGER deserts is welcome to join. I just looked at your new green-means-go map. There is the gap that I'm seeing between what you're doing and what Ruben, Ian and I'd like to do: we're particularly interested in places where the TIGER geography is off. I. e. where roads are not where they should be. After a first test here with Virginia [1] we saw that TIGER deserts do not necessarily equal bad road geography. So we're right now particularly interested in methods for identifying bad TIGER road geography. I'm thinking such a Bad TIGER Roads Map could be a valuable community resource to see where stuff is worst and needs fixing that we could maintain over time. Interesting—how would you characterize bad roads? One characteristic of crappy TIGER data is road wiggliness, is that what you mean? -mike. michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] More on TIGER: Where it's likely safe to import
On Dec 17, 2012, at 12:40 PM, Michal Migurski wrote: On Dec 16, 2012, at 8:32 PM, Martijn van Exel wrote: Have you looked into full history planet parsing to get a fuller picture of editing history? I took a stab at full history user metrics some time ago using osmjs; https://github.com/mvexel/OSMQualityMetrics/blob/master/UserStats.js - this produces one set of metrics for the entire .osh file you feed it but it may prove useful for future work. I haven't touched this in a while but it should still work :/ I have downloaded a copy and given it a beginning look. I'm new to parsing things of that magnitude; my first thought was to use the full history file for creations/modifications/deletions on nodes and add that to what I'm doing already for ways on the osm2pgsql tables. Does that sound reasonable? Over the holiday break, I've been grinding through the full history file. I'll write more about the process and publish some raw data, but the short version is that I've replaced the Green Means Go tiles with new versions that incorporate edits to the underlying nodes, address some of the feedback I've received, and show a slightly different map of the US. New map: http://www.openstreetmap.us/~migurski/green-means-go/ Old map: http://www.openstreetmap.us/~migurski/green-means-go-2012-12-16/ Charlotte Wolter and NE2 both pointed out that Florida should see a lot more post-import editing that I had originally shown, and in fact that's what the map now shows: http://www.openstreetmap.us/~migurski/green-means-go/#9/28.3213/-81.6257 http://www.openstreetmap.us/~migurski/green-means-go-2012-12-16/#9/28.3213/-81.6257 Urban fringe areas also show more edits, making them less attractive targets for blind imports: http://www.openstreetmap.us/~migurski/green-means-go/#10/37.7707/-122.3451 http://www.openstreetmap.us/~migurski/green-means-go-2012-12-16/#10/37.7707/-122.3451 I've also stripped away the US coastal territory based on NLCD water designation, per SteveC's suggestion. These massive edits to counties in Pennsylvania are interesting to me: http://www.openstreetmap.us/~migurski/green-means-go/#8/40.918/-77.146 -mike. michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] identifying TIGER deserts
On Dec 20, 2012, at 11:19 AM, Michal Migurski wrote: next steps A few obvious steps stand out, including rendering a national version of these maps. I'd also love to figure out whether it makes sense to join forces with Mike Migurski's Green Means Go map. It'd also be interesting to use a smarter Osmosis command (and the full history file) to limit the input data to just those ways created by the 3 primary user accounts associated with the TIGER import:https://github.com/MapQuest/TIGER-Edited-map/blob/master/inc/layer-tiger.xml.inc#L45 (h/t Migurski) Joining forces would be fun! For the past two days, I've been processing the Full History file into a form that's easier to cope with for a big categorization run. I want to look at the individual nodes, using their associated changesets to pick up on non-import edits that fell through the cracks when I looked only at OSM ways. Here's where I'm at now: http://www.openstreetmap.us/~migurski/TIGER-Raster/nodes/ I've updated the page above with a preliminary rendering showing non-TIGER node density, preview and full GeoTIFF here: http://www.openstreetmap.us/~migurski/TIGER-Raster/nodes/non-tiger-uids.jpg http://www.openstreetmap.us/~migurski/TIGER-Raster/nodes/out-full-history-uids-no.tif.bz2 I'm processing the full history planet file a second time now, paying attention to just the nodes that make up ways with highway tags, since I'm noticing a lot of large non-road imports in this dataset. Also, what's the deal with the Massachusetts TIGER import? When did it happen? The state is conspicuously missing when I use the Mapquest TIGER edits case statement: (case when osm_uid = '7168' -- DaveHansenTiger and osm_timestamp::timestamp = '2007-08-03'::timestamp and osm_timestamp::timestamp = '2008-05-04'::timestamp then 0 when osm_uid = '15169' -- Milenko and osm_timestamp::timestamp = '2007-10-29'::timestamp and osm_timestamp::timestamp = '2007-12-12'::timestamp then 0 when osm_uid = '20587' -- balrog-kun and osm_timestamp::timestamp = '2010-03-21'::timestamp and osm_timestamp::timestamp = '2010-04-08'::timestamp and osm_version::int 3 -- maybe someone else edited between import and name expansion then 0 else 1 end) as is_touched (Github currently dead but it's from https://github.com/MapQuest/TIGER-Edited-map/blob/master/inc/layer-tiger.xml.inc) -mike. michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] identifying TIGER deserts
On Dec 22, 2012, at 7:01 PM, Russ Nelson wrote: Michal Migurski writes: Also, what's the deal with the Massachusetts TIGER import? Massachusetts had already made an improved version of the TIGER data, so the decision was made to import that instead. Ah, good to know. Any idea what the approximate date and importing account were? -mike. michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] identifying TIGER deserts
On Dec 19, 2012, at 5:39 PM, Ruben Lopez Mendoza wrote: We've worked out some kinks of our first attempt to identify TIGER deserts, and produced maps a couple maps along the way. … I've captured all of this work - the postgres functions, shapefiles, and tilemill projects in the a repo here:https://github.com/Rub21/tiger-deserts +1. next steps A few obvious steps stand out, including rendering a national version of these maps. I'd also love to figure out whether it makes sense to join forces with Mike Migurski's Green Means Go map. It'd also be interesting to use a smarter Osmosis command (and the full history file) to limit the input data to just those ways created by the 3 primary user accounts associated with the TIGER import:https://github.com/MapQuest/TIGER-Edited-map/blob/master/inc/layer-tiger.xml.inc#L45 (h/t Migurski) Joining forces would be fun! For the past two days, I've been processing the Full History file into a form that's easier to cope with for a big categorization run. I want to look at the individual nodes, using their associated changesets to pick up on non-import edits that fell through the cracks when I looked only at OSM ways. Here's where I'm at now: http://www.openstreetmap.us/~migurski/TIGER-Raster/nodes/ -mike. michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] More on TIGER: Where it's likely safe to import
On Dec 16, 2012, at 8:32 PM, Martijn van Exel wrote: OK this is plain awesome. Great work Mike. One note of caution though - the title may suggest that you can just go ahead and import away, but folks would still have to follow the import guidelines and contact the OSM community at large, come up with a solid proposal and discuss that, even if there is no local community. I know it says it on the tin, but it's kind of tucked away at the bottom. You're right, I've emphasized that up near the top, thanks. Have you looked into full history planet parsing to get a fuller picture of editing history? I took a stab at full history user metrics some time ago using osmjs; https://github.com/mvexel/OSMQualityMetrics/blob/master/UserStats.js - this produces one set of metrics for the entire .osh file you feed it but it may prove useful for future work. I haven't touched this in a while but it should still work :/ I have downloaded a copy and given it a beginning look. I'm new to parsing things of that magnitude; my first thought was to use the full history file for creations/modifications/deletions on nodes and add that to what I'm doing already for ways on the osm2pgsql tables. Does that sound reasonable? I'll check out your parser; what kinds of metrics does it produce? -mike. On Sun, Dec 16, 2012 at 8:02 PM, Michal Migurski m...@teczno.com wrote: I pulled together some of the notes and imagery I've been posting here recently: http://www.openstreetmap.us/~migurski/green-means-go/ It's a map of 1km×1km squares covering the continental United States. Green squares show places where data imports are unlikely to interfere with community mapping. Raw data is linked at the bottom. Three things that would make this better: - Regular updates with archived older versions. - Renders for specific counties, intended for local GIS communities. - Some awareness of full planet history. The OSM-US server has data for regular updates. -mike. michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us -- Martijn van Exel http://oegeo.wordpress.com/ http://openstreetmap.us/ ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] More on TIGER: Where it's likely safe to import
On Dec 17, 2012, at 1:58 AM, Frederik Ramm wrote: Hi, On 12/17/12 04:02, Michal Migurski wrote: I pulled together some of the notes and imagery I've been posting here recently: http://www.openstreetmap.us/~migurski/green-means-go/ I take offense at your wording (on the page): Where in the United States could government imports improve OpenStreetMap? - you might add data to OSM but will you improve OSM? It's not the same, and equating the two is a mistake that insiders should not make. The wording Green squares show places where data imports are unlikely to interfere with community mapping. is also misleading; it has been shown that imports can very well interfere with *future* community mapping of which you would, naturally, not find traces in the data you analysed. The correct wording is: Green squares show places where little or no community mapping has taken place in the past. How about something like this? Green squares show places where data imports are unlikely to conflict with past community mapping. I think in the case of the US, the previous government data is so bad relative to what's currently out there that a fresh import will necessarily improve OSM, if I can make the green areas more reflective of the true state of edited places. Full history is a means to this; I've got some off-list responses from people who don't think that their own mapping efforts are accurately reflected in the green squares. -mike. michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] More on TIGER: Where it's likely safe to import
Information density: maybe a different grid for lower zoom levels, e.g. 5km, 10km, etc.? It would have the opposite effect of what's there now I think, which is look at all that green! -mike. On Dec 17, 2012, at 12:22 PM, Steve Coast wrote: Nice. Suggestions; - kill water somehow - Information density at low zoom levels implies that basically everywhere is green. But you zoom to the bay area and see this isn't the case. So, change the coloring? Modulate it by population density? Steve On Dec 16, 2012, at 8:32 PM, Martijn van Exel m...@rtijn.org wrote: OK this is plain awesome. Great work Mike. One note of caution though - the title may suggest that you can just go ahead and import away, but folks would still have to follow the import guidelines and contact the OSM community at large, come up with a solid proposal and discuss that, even if there is no local community. I know it says it on the tin, but it's kind of tucked away at the bottom. Have you looked into full history planet parsing to get a fuller picture of editing history? I took a stab at full history user metrics some time ago using osmjs; https://github.com/mvexel/OSMQualityMetrics/blob/master/UserStats.js - this produces one set of metrics for the entire .osh file you feed it but it may prove useful for future work. I haven't touched this in a while but it should still work :/ On Sun, Dec 16, 2012 at 8:02 PM, Michal Migurski m...@teczno.com wrote: I pulled together some of the notes and imagery I've been posting here recently: http://www.openstreetmap.us/~migurski/green-means-go/ It's a map of 1km×1km squares covering the continental United States. Green squares show places where data imports are unlikely to interfere with community mapping. Raw data is linked at the bottom. Three things that would make this better: - Regular updates with archived older versions. - Renders for specific counties, intended for local GIS communities. - Some awareness of full planet history. The OSM-US server has data for regular updates. -mike. michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us -- Martijn van Exel http://oegeo.wordpress.com/ http://openstreetmap.us/ ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] More on TIGER: Where it's likely safe to import
Agreed. What I *really* want is a version of this map that's tailored to meaningful jurisdictions, Census places and counties. It's one thing to see an all-over view but if you're a city GIS guy and you have a file of data that you want to input, it'd be useful for you to see just your specific area with some guidance on how to proceed: 1. How much is green vs. not green, and the likelihood of improvement. 2. OSM users who are responsible for existing work in your area. 3. Non-highway data in the area, e.g. POIs and such. Christine White from Esri, who attended SOTM-US in Portland, told me that at the annual user conference she gets a regular stream of these people approaching her with blobs of official data, a desire to donate it to OSM, and no knowledge about how to proceed or what effect it would have. We should help her and them! -mike. On Dec 17, 2012, at 12:39 PM, Charlotte Wolter wrote: While I like the idea of being able to identify and possibly do imports for one-kilometer-square (why not miles?) chunks of the map, I think it needs to be accompanied with lots of cautionary language about assessing the area thoroughly before taking any such action. We could give people examples of what to look for to see if the area really is a TIGER desert, and what to check before making a move. May be it would be better if a group of squares are identified using criteria set up by the Data group or someone similarly experienced. Then, the square kilometers could be presented in a Maproulette kind of format, but with a chance to choose which one you take on. That way, you could choose square kilometers that are near where you are working anyway or near areas with which you are familiar. Best, Charlotte At 12:22 PM 12/17/2012, you wrote: Nice. Suggestions; - kill water somehow - Information density at low zoom levels implies that basically everywhere is green. But you zoom to the bay area and see this isn't the case. So, change the coloring? Modulate it by population density? Steve On Dec 16, 2012, at 8:32 PM, Martijn van Exel m...@rtijn.org wrote: OK this is plain awesome. Great work Mike. One note of caution though - the title may suggest that you can just go ahead and import away, but folks would still have to follow the import guidelines and contact the OSM community at large, come up with a solid proposal and discuss that, even if there is no local community. I know it says it on the tin, but it's kind of tucked away at the bottom. Have you looked into full history planet parsing to get a fuller picture of editing history? I took a stab at full history user metrics some time ago using osmjs; https://github.com/mvexel/OSMQualityMetrics/blob/master/UserStats.js - this produces one set of metrics for the entire .osh file you feed it but it may prove useful for future work. I haven't touched this in a while but it should still work :/ On Sun, Dec 16, 2012 at 8:02 PM, Michal Migurski m...@teczno.com wrote: I pulled together some of the notes and imagery I've been posting here recently: http://www.openstreetmap.us/~migurski/green-means-go/ It's a map of 1km×1km squares covering the continental United States. Green squares show places where data imports are unlikely to interfere with community mapping. Raw data is linked at the bottom. Three things that would make this better: - Regular updates with archived older versions. - Renders for specific counties, intended for local GIS communities. - Some awareness of full planet history. The OSM-US server has data for regular updates. -mike. michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us -- Martijn van Exel http://oegeo.wordpress.com/ http://openstreetmap.us/ ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us Charlotte Wolter 927 18th Street Suite A Santa Monica, California 90403 +1-310-597-4040 techl...@techlady.com Skype: thetechlady The Four Internet Freedoms Freedom to visit any site on the Internet Freedom to access any content or service that is not illegal Freedom to attach any device that does not interfere with the network Freedom to know all the terms of a service, particularly any that would affect the first three freedoms. ___ Talk-us mailing list Talk-us@openstreetmap.org
Re: [Talk-us] Fwd: Re: More on TIGER: Where it's likely safe to import
On Dec 17, 2012, at 1:01 PM, Charlotte Wolter wrote: Hi Mike, I was looking at the web page yu created. One problem is that I'm not sure which green, light or dark or kinda light or kinda dark, means an untouched kilometer. Also, what does black mean? Then I checked an area I edited extensively: Chinle, Arizona. Again, I was puzzled because I have worked on most of the area, but some is dark green, some light green and some black. However, the basic idea seems great. It just needs a little more interpretation, I think, and maybe different colors? Green generally means untouched, based the user IDs attached to ways. I think I'm undercounting participation from people who edited nodes whose changes don't show up in the line table. -mike. michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] More on TIGER: Where it's likely safe to import
On Dec 17, 2012, at 12:50 PM, Steve Coast wrote: On Dec 17, 2012, at 12:45 PM, Michal Migurski m...@teczno.com wrote: Information density: maybe a different grid for lower zoom levels, e.g. 5km, 10km, etc.? It would have the opposite effect of what's there now I think, which is look at all that green! I prefer modulating by population, since a sea of green in Wyoming (with apologies to those in Wyoming) really doesn't matter since nobody lives there. The question is, where is the population / edits ration low, not the absolute edits numbers. Maybe ask people from CloudMade what they did 3 years ago. Also, I wouldn't ask the question(s) in a vacuum. I suspect, but cannot confirm, that if you did a similar analysis of NT or TA data in the US you'd see exactly the same thing; a natural economic bias in the metrics to mapping places with high population density. I suspect the same, it makes sense given driving patterns and economic demand. -mike. michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
[Talk-us] More on TIGER: Where it's likely safe to import
I pulled together some of the notes and imagery I've been posting here recently: http://www.openstreetmap.us/~migurski/green-means-go/ It's a map of 1km×1km squares covering the continental United States. Green squares show places where data imports are unlikely to interfere with community mapping. Raw data is linked at the bottom. Three things that would make this better: - Regular updates with archived older versions. - Renders for specific counties, intended for local GIS communities. - Some awareness of full planet history. The OSM-US server has data for regular updates. -mike. michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] identifying TIGER deserts
Are you guys using the way version number to determine TIGERness? This from Andy Allan might be more effective: -- -- TIGER edits case statement from -- https://github.com/MapQuest/TIGER-Edited-map/blob/master/inc/layer-tiger.xml.inc -- (case when osm_uid = '7168' -- DaveHansenTiger and osm_timestamp::timestamp = '2007-08-03'::timestamp and osm_timestamp::timestamp = '2008-05-04'::timestamp then 0 when osm_uid = '15169' -- Milenko and osm_timestamp::timestamp = '2007-10-29'::timestamp and osm_timestamp::timestamp = '2007-12-12'::timestamp then 0 when osm_uid = '20587' -- balrog-kun and osm_timestamp::timestamp = '2010-03-21'::timestamp and osm_timestamp::timestamp = '2010-04-08'::timestamp and osm_version::int 3 -- maybe someone else edited between import and name expansion then 0 else 1 end) as is_touched Number of distinct users who've edited in a given area would also be a useful metric, though incomplete without reference to the full history file. -mike. On Dec 14, 2012, at 7:45 PM, Alex Barth wrote: Before I log off for tonight, let me share this screenshot I just got from Ruben (cc'ed) showing his progress on creating a slippy TIGER deserts map, pretty much using Martijn's approach of binned average geometry versions. These are first steps and there are still some math problems to iron out, expect an update next week. http://cl.ly/image/0x3h0s1H1v0k On Dec 7, 2012, at 1:00 PM, Ian Villeda vill...@mapbox.com wrote: Hey team, Over here at MapBox we were inspired by Martjin's great post on identifying TIGER deserts[1], so we're attempting to effectively identify TIGER deserts for the rest of the continental U.S. (sorry Hawaii + Alaska, you're next). The goal is to create a slippy map from zoom level 2 to 15(?) such that it's easy to find the places in need of the most love. We have a osmium-genereated postigs database with all ways tagged as highway=* and their version number. The plan to run the analysis against a 5x5 km grid, although we will probably play with the size of the gridcell. Another option might be to conduct the analysis and varying levels of granularity. We're still consiering different methods of analysis - the goal being to normalize way density to display better identify TIGER deserts. We'd would love your thoughts / input / reactions. A few ideas we've tossed around / are activley pursuing: - percent of version 1 and/or 2 ways in each gridcell - average number of version 1/2 ways per gridcell - using a threshold value(s) similiar to Martjin's approach - [your ideas here] Ultimately we'd like showcase the rendered results in site that would make editing TIGER deserts easy. For now though this is just a heads up and an invitation if you have any thoughts on how better to identify TIGER deserts. thanks! [1]: http://oegeo.wordpress.com/2012/10/21/binders-full-of-tiger-deserts/ -- ian villeda mapbox | developmentseed https://twitter.com/ian_villeda ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us Alex Barth http://twitter.com/lxbarth tel (+1) 202 250 3633 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] identifying TIGER deserts
Some context for the query below: https://gist.github.com/4291608 I've been using this projection for everything, a spherical Albers to make it possible to generate a slippy map in a conic projection: +proj=aea +lat_1=29.5 +lat_2=45.5 +lat_0=23 +lon_0=-96 +x_0=0 +y_0=0 +a=6378137 +b=6378137 +nadgrids=@null +units=m +no_defs -mike. On Dec 14, 2012, at 9:47 PM, Michal Migurski wrote: Are you guys using the way version number to determine TIGERness? This from Andy Allan might be more effective: -- -- TIGER edits case statement from -- https://github.com/MapQuest/TIGER-Edited-map/blob/master/inc/layer-tiger.xml.inc -- (case when osm_uid = '7168' -- DaveHansenTiger and osm_timestamp::timestamp = '2007-08-03'::timestamp and osm_timestamp::timestamp = '2008-05-04'::timestamp then 0 when osm_uid = '15169' -- Milenko and osm_timestamp::timestamp = '2007-10-29'::timestamp and osm_timestamp::timestamp = '2007-12-12'::timestamp then 0 when osm_uid = '20587' -- balrog-kun and osm_timestamp::timestamp = '2010-03-21'::timestamp and osm_timestamp::timestamp = '2010-04-08'::timestamp and osm_version::int 3 -- maybe someone else edited between import and name expansion then 0 else 1 end) as is_touched Number of distinct users who've edited in a given area would also be a useful metric, though incomplete without reference to the full history file. -mike. On Dec 14, 2012, at 7:45 PM, Alex Barth wrote: Before I log off for tonight, let me share this screenshot I just got from Ruben (cc'ed) showing his progress on creating a slippy TIGER deserts map, pretty much using Martijn's approach of binned average geometry versions. These are first steps and there are still some math problems to iron out, expect an update next week. http://cl.ly/image/0x3h0s1H1v0k On Dec 7, 2012, at 1:00 PM, Ian Villeda vill...@mapbox.com wrote: Hey team, Over here at MapBox we were inspired by Martjin's great post on identifying TIGER deserts[1], so we're attempting to effectively identify TIGER deserts for the rest of the continental U.S. (sorry Hawaii + Alaska, you're next). The goal is to create a slippy map from zoom level 2 to 15(?) such that it's easy to find the places in need of the most love. We have a osmium-genereated postigs database with all ways tagged as highway=* and their version number. The plan to run the analysis against a 5x5 km grid, although we will probably play with the size of the gridcell. Another option might be to conduct the analysis and varying levels of granularity. We're still consiering different methods of analysis - the goal being to normalize way density to display better identify TIGER deserts. We'd would love your thoughts / input / reactions. A few ideas we've tossed around / are activley pursuing: - percent of version 1 and/or 2 ways in each gridcell - average number of version 1/2 ways per gridcell - using a threshold value(s) similiar to Martjin's approach - [your ideas here] Ultimately we'd like showcase the rendered results in site that would make editing TIGER deserts easy. For now though this is just a heads up and an invitation if you have any thoughts on how better to identify TIGER deserts. thanks! [1]: http://oegeo.wordpress.com/2012/10/21/binders-full-of-tiger-deserts/ -- ian villeda mapbox | developmentseed https://twitter.com/ian_villeda ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us Alex Barth http://twitter.com/lxbarth tel (+1) 202 250 3633 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] MapQuest Open Maps now has proper state shields
On Dec 9, 2012, at 7:37 PM, James Mast wrote: I was just looking at their tiles tonight and they've updated them to include state shields!! Saw shields for PA, SC, MD, VA, WV, CA, and NY for starters. Maybe now we should all agree to use the proper state abbreviation for the ref tags on the ways instead of not having them at all (Florida), or just SR. Cool! If anyone's curious, here's a shapefile with linework for all current (as of a few weeks ago) US route relations, generalized to appear at zoom 15. Coverage is great for US:I, US:US and US State routes, starts getting patchy down at the county level: http://benzene.openstreetmap.us/~migurski/US-Routes-2012-12-09.png -mike. michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] MapQuest Open Maps now has proper state shields
On Dec 9, 2012, at 8:12 PM, Michal Migurski wrote: I was just looking at their tiles tonight and they've updated them to include state shields!! Saw shields for PA, SC, MD, VA, WV, CA, and NY for starters. Maybe now we should all agree to use the proper state abbreviation for the ref tags on the ways instead of not having them at all (Florida), or just SR. Cool! If anyone's curious, here's a shapefile with linework for all current (as of a few weeks ago) US route relations, generalized to appear at zoom 15. Coverage is great for US:I, US:US and US State routes, starts getting patchy down at the county level: http://benzene.openstreetmap.us/~migurski/US-Routes-2012-12-09.png …aaand the actual data: http://benzene.openstreetmap.us/~migurski/US-Routes-2012-12-09.zip -mike. michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] MapQuest Open Maps now has proper state shields
On Dec 9, 2012, at 9:21 PM, Toby Murray wrote: On Sun, Dec 9, 2012 at 9:37 PM, James Mast rickmastfa...@hotmail.com wrote: I was just looking at their tiles tonight and they've updated them to include state shields!! Saw shields for PA, SC, MD, VA, WV, CA, and NY for starters. Maybe now we should all agree to use the proper state abbreviation for the ref tags on the ways instead of not having them at all (Florida), or just SR. Interesting. Anyone know when this happened? Last time I looked I seem to recall generic square shields with any state abbreviations stripped out. I also don't see any change to their stylesheet on github. I'm mostly curious if they switched to route relations or if they are still going off of the ref tag on ways. I would guess ref tags, based on the two mismatched 24’s here: http://www.openstreetmap.org/?lat=37.82443lon=-122.26695zoom=15layers=Q Likely due to inconsistent refs on ways like these: http://www.openstreetmap.org/browse/way/28183735 http://www.openstreetmap.org/browse/way/6348404 -mike. michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] identifying TIGER deserts
That's great! I've been thinking about some of the same things, inspired by Dennis Zielstra's talk at SOTM-US. I'm looking at road lengths for 1x1 kilometer squares, measuring OSM user count (post-import) vs. possibility for improvement between 2007 and 2012 TIGER/Line data: http://mike.teczno.com/img/osm-users-imports-2012-09.png - Blue means 2012 data would be an improvement - Red/Yellow means there are active OSM users there I used an Albers equal-area projection to ensure correct length and density regardless of latitude, and my intent for this is to produce county-by-county views that GIS managers in different jurisdictions can use to understand whether their data is worth important, and what OSM users they'd need to contact in order to be successful. I have a slippy map version of this but the projection registration is off by a few hundred meters; need to figure out why that is and fix it. -mike. On Dec 7, 2012, at 10:00 AM, Ian Villeda wrote: Hey team, Over here at MapBox we were inspired by Martjin's great post on identifying TIGER deserts[1], so we're attempting to effectively identify TIGER deserts for the rest of the continental U.S. (sorry Hawaii + Alaska, you're next). The goal is to create a slippy map from zoom level 2 to 15(?) such that it's easy to find the places in need of the most love. We have a osmium-genereated postigs database with all ways tagged as highway=* and their version number. The plan to run the analysis against a 5x5 km grid, although we will probably play with the size of the gridcell. Another option might be to conduct the analysis and varying levels of granularity. We're still consiering different methods of analysis - the goal being to normalize way density to display better identify TIGER deserts. We'd would love your thoughts / input / reactions. A few ideas we've tossed around / are activley pursuing: - percent of version 1 and/or 2 ways in each gridcell - average number of version 1/2 ways per gridcell - using a threshold value(s) similiar to Martjin's approach - [your ideas here] Ultimately we'd like showcase the rendered results in site that would make editing TIGER deserts easy. For now though this is just a heads up and an invitation if you have any thoughts on how better to identify TIGER deserts. thanks! [1]: http://oegeo.wordpress.com/2012/10/21/binders-full-of-tiger-deserts/ -- ian villeda mapbox | developmentseed https://twitter.com/ian_villeda ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] identifying TIGER deserts
GeoTIFF version of that data: http://mike.teczno.com/img/osm-users-imports-2012-09.tif On Dec 7, 2012, at 10:32 AM, Michal Migurski wrote: That's great! I've been thinking about some of the same things, inspired by Dennis Zielstra's talk at SOTM-US. I'm looking at road lengths for 1x1 kilometer squares, measuring OSM user count (post-import) vs. possibility for improvement between 2007 and 2012 TIGER/Line data: http://mike.teczno.com/img/osm-users-imports-2012-09.png - Blue means 2012 data would be an improvement - Red/Yellow means there are active OSM users there I used an Albers equal-area projection to ensure correct length and density regardless of latitude, and my intent for this is to produce county-by-county views that GIS managers in different jurisdictions can use to understand whether their data is worth important, and what OSM users they'd need to contact in order to be successful. I have a slippy map version of this but the projection registration is off by a few hundred meters; need to figure out why that is and fix it. -mike. On Dec 7, 2012, at 10:00 AM, Ian Villeda wrote: Hey team, Over here at MapBox we were inspired by Martjin's great post on identifying TIGER deserts[1], so we're attempting to effectively identify TIGER deserts for the rest of the continental U.S. (sorry Hawaii + Alaska, you're next). The goal is to create a slippy map from zoom level 2 to 15(?) such that it's easy to find the places in need of the most love. We have a osmium-genereated postigs database with all ways tagged as highway=* and their version number. The plan to run the analysis against a 5x5 km grid, although we will probably play with the size of the gridcell. Another option might be to conduct the analysis and varying levels of granularity. We're still consiering different methods of analysis - the goal being to normalize way density to display better identify TIGER deserts. We'd would love your thoughts / input / reactions. A few ideas we've tossed around / are activley pursuing: - percent of version 1 and/or 2 ways in each gridcell - average number of version 1/2 ways per gridcell - using a threshold value(s) similiar to Martjin's approach - [your ideas here] Ultimately we'd like showcase the rendered results in site that would make editing TIGER deserts easy. For now though this is just a heads up and an invitation if you have any thoughts on how better to identify TIGER deserts. thanks! [1]: http://oegeo.wordpress.com/2012/10/21/binders-full-of-tiger-deserts/ -- ian villeda mapbox | developmentseed https://twitter.com/ian_villeda ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] identifying TIGER deserts
Absolutely, yeah. I chose a raster approach to the data I was working with because I imagined it would be easiest to adapt to arbitrary jurisdictions, whether adopt-a-pixel or masked by a county boundary. Easy to track buckets of data over time that way, too. -mike. On Dec 7, 2012, at 10:38 AM, Martijn van Exel wrote: This would also fit in really well with the data steward notion. Part of the idea that we've been toying with for this is to have data dashboards for the stewarded areas to support targeted fixing / improving and to build a stronger local discussion and community around data information knowledge. On Fri, Dec 7, 2012 at 11:32 AM, Michal Migurski m...@teczno.com wrote: my intent for this is to produce county-by-county views that GIS managers in different jurisdictions can use to understand whether their data is worth -- Martijn van Exel http://oegeo.wordpress.com/ http://openstreetmap.us/ michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] identifying TIGER deserts
Thanks! I agree about a regularly-updated map. For this one, I used the up-to-date osm2pgsql database that Ian Dees is maintaining on the OSM-US server we have living in Oregon, an awesome resource for USian OSM things. =) I re-projected everything to spherical albers, and then iterated over every grid square in the raster and did a simple Postgres SUM(ST_Length) query for the roads in that area. The data was written out as a set of Float32 rasters, which I later combined with Numpy to make pretty pictures. -mike. On Dec 7, 2012, at 1:09 PM, Alex Barth wrote: Mike - AWESOME looking map and similar to what Ian and Ruben are working on. Would you be open to share how you built that map? How hard (time consuming) is it to regenerate? I'd love to get a regularly updated map out there showing TIGER deserts. We'd use that ourselves to start fixing some of the worst areas in terms of geographic accuracy. But obviously such a map would be a good thing to have for the wider community. On Dec 7, 2012, at 1:32 PM, Michal Migurski m...@teczno.com wrote: That's great! I've been thinking about some of the same things, inspired by Dennis Zielstra's talk at SOTM-US. I'm looking at road lengths for 1x1 kilometer squares, measuring OSM user count (post-import) vs. possibility for improvement between 2007 and 2012 TIGER/Line data: http://mike.teczno.com/img/osm-users-imports-2012-09.png - Blue means 2012 data would be an improvement - Red/Yellow means there are active OSM users there I used an Albers equal-area projection to ensure correct length and density regardless of latitude, and my intent for this is to produce county-by-county views that GIS managers in different jurisdictions can use to understand whether their data is worth important, and what OSM users they'd need to contact in order to be successful. I have a slippy map version of this but the projection registration is off by a few hundred meters; need to figure out why that is and fix it. -mike. On Dec 7, 2012, at 10:00 AM, Ian Villeda wrote: Hey team, Over here at MapBox we were inspired by Martjin's great post on identifying TIGER deserts[1], so we're attempting to effectively identify TIGER deserts for the rest of the continental U.S. (sorry Hawaii + Alaska, you're next). The goal is to create a slippy map from zoom level 2 to 15(?) such that it's easy to find the places in need of the most love. We have a osmium-genereated postigs database with all ways tagged as highway=* and their version number. The plan to run the analysis against a 5x5 km grid, although we will probably play with the size of the gridcell. Another option might be to conduct the analysis and varying levels of granularity. We're still consiering different methods of analysis - the goal being to normalize way density to display better identify TIGER deserts. We'd would love your thoughts / input / reactions. A few ideas we've tossed around / are activley pursuing: - percent of version 1 and/or 2 ways in each gridcell - average number of version 1/2 ways per gridcell - using a threshold value(s) similiar to Martjin's approach - [your ideas here] Ultimately we'd like showcase the rendered results in site that would make editing TIGER deserts easy. For now though this is just a heads up and an invitation if you have any thoughts on how better to identify TIGER deserts. thanks! [1]: http://oegeo.wordpress.com/2012/10/21/binders-full-of-tiger-deserts/ -- ian villeda mapbox | developmentseed https://twitter.com/ian_villeda ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us Alex Barth http://twitter.com/lxbarth tel (+1) 202 250 3633 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] What to do with unnamed NHD streams
On Oct 28, 2012, at 11:41 PM, Paul Norman wrote: From: Michal Migurski [mailto:m...@teczno.com] Subject: Re: [Talk-us] What to do with unnamed NHD streams On Oct 28, 2012, at 10:29 PM, Paul Norman wrote: The problem is you need to convert to .osm and *then* simplify. If you do this in the other order you have problems where one object intersects another (e.g. because they share a geometry for a portion of them). You end up simplifying away the intersection points and your resulting ways won't end up correctly sharing nodes. There are ways around this, by first de-duping the shared edges or nodes. Topology preservation is not terribly difficult if you prepare your data, for example by splitting lines and polygons at intersections (as in your lake example), simplifying only the parts and then reconstructing the original geometries. Do you have a link to an example of the PostGIS magic to do this? It's beyond what I could do. I implemented a version of this here: http://github.com/migurski/Bloch Without digging into the code too deeply, the short version is that you can use the intersection of two shapes to arrive at something like a topology. For a pair of polygons that share a border, the ST_Intersection() results in a linestring that forms the border, which you can ST_Difference from each border to get the remaining pieces. The expensive part is the gigantic pairwise comparison to come up with the full list of all feature pairs that touch each other or come really close. A slight complication I found is that you can't just go for intersections but you also have to go to near intersections - sometimes the NHD data is off by a couple cm. I don't know if this will pose a practical issue for the simplification. Possible to round every node position to 1e-7 degrees? Do you have any sample NHD extracts that might be usable for a test drive? All of NHD can be found at on the USGS FTP site, but you need to compile gdal with 3rd-party toolkits to be able to use them. This is quite often a pain. I clipped out a small part of Kansas I was testing with early, converted it to a set of shapefiles and posted it at http://took.paulnorman.ca/imports/NHDH0202_shp.zip I should note that some information is lost in the shapefile conversion (long field names). For space reasons I dropped the WBD_HU layers. The first step of my converting is to remove them anyways. Awesome, downloading! -mike. michal migurski- m...@stamen.com 415.558.1610 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] What to do with unnamed NHD streams
On Oct 29, 2012, at 12:03 AM, Michal Migurski wrote: Do you have any sample NHD extracts that might be usable for a test drive? All of NHD can be found at on the USGS FTP site, but you need to compile gdal with 3rd-party toolkits to be able to use them. This is quite often a pain. I clipped out a small part of Kansas I was testing with early, converted it to a set of shapefiles and posted it at http://took.paulnorman.ca/imports/NHDH0202_shp.zip I should note that some information is lost in the shapefile conversion (long field names). For space reasons I dropped the WBD_HU layers. The first step of my converting is to remove them anyways. Awesome, downloading! Yum: https://dl.dropbox.com/u/30204300/NHD.png I can see that the water areas are also represented as centerlines, how would you import these? -mike. michal migurski- m...@stamen.com 415.558.1610 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] What to do with unnamed NHD streams
On Oct 29, 2012, at 12:49 AM, Paul Norman wrote: There are ways around this, by first de-duping the shared edges or nodes. Topology preservation is not terribly difficult if you prepare your data, for example by splitting lines and polygons at intersections (as in your lake example), simplifying only the parts and then reconstructing the original geometries. Do you have a link to an example of the PostGIS magic to do this? It's beyond what I could do. I implemented a version of this here: http://github.com/migurski/Bloch Without digging into the code too deeply, the short version is that you can use the intersection of two shapes to arrive at something like a topology. For a pair of polygons that share a border, the ST_Intersection() results in a linestring that forms the border, which you can ST_Difference from each border to get the remaining pieces. The expensive part is the gigantic pairwise comparison to come up with the full list of all feature pairs that touch each other or come really close. I'll have a look tomorrow - I'm too tired to give it thought right now. This is a rough sketch for how the process could work on the NHD Areas file: https://gist.github.com/3978487 It's fairly primitive, but the general idea is there. First I converted to an Albers projection, then snapped all the geometries to a 1-meter grid. Each polygon is looked at in turn, with all the joined streams used as a locations along the point path to create anchors for the simplification process. I'm not catching enough edge corner cases for this to really work, yet, but the lines simplify down astonishingly well. A likely better way to do this would be to more aggressively convert the NHD into a truly topological dataset, simplify *that*, and then reform all the polygons. -mike. michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] What to do with unnamed NHD streams
On Oct 28, 2012, at 10:29 PM, Paul Norman wrote: From: Michal Migurski [mailto:m...@teczno.com] Sent: Sunday, October 28, 2012 8:58 PM To: OpenStreetMap US Talk Subject: Re: [Talk-us] What to do with unnamed NHD streams Does [NHD] all come in shapefile form? The simplification would be a relatively easy (though time-consuming) task for PostGIS on a server with sufficient storage, outputting new shapefiles for ogr2osm. I can help with this using one of our office servers that we use for such tasks. It comes in .mdb or .gdb form. These could be loaded into postgis and I've considered doing it. My home server has the storage and cores to do it, but I don't think it would help. The problem is you need to convert to .osm and *then* simplify. If you do this in the other order you have problems where one object intersects another (e.g. because they share a geometry for a portion of them). You end up simplifying away the intersection points and your resulting ways won't end up correctly sharing nodes. There are ways around this, by first de-duping the shared edges or nodes. Topology preservation is not terribly difficult if you prepare your data, for example by splitting lines and polygons at intersections (as in your lake example), simplifying only the parts and then reconstructing the original geometries. I would *really* like to be able to simplify prior to ogr2osm as it would dramatically decrease the size of the nodes data in-memory and decrease conversion time, I just can't see how to do it prior to processing. JOSM's simplify ways function works okay, although it doesn't deal with the case of two ways sharing nodes very well. Do you have any sample NHD extracts that might be usable for a test drive? -mike. michal migurski- m...@stamen.com 415.558.1610 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Scrubbing route relations (attn: Richard Welty, etc.)
On Oct 24, 2012, at 5:01 PM, Chris Lawrence wrote: On Wed, Oct 24, 2012 at 12:28 PM, Michal Migurski m...@stamen.com wrote: NE2 asked me to revert the changes, because he's unhappy with me moving the route variant information from the ref tags to the modifier tags, e.g. turning ref=80 Business into ref=80 modifier=Business. According to the supported tagging guidelines on Aperiodic, my interpretation should be correct: The value of the ref tag on the relation must contain just the route number, without any network information. http://elrond.aperiodic.net/shields/supported.html ... As far as what to do from here... damned if I know. I don't think anyone wants NE2 banned (he's a dedicated armchair mapper and contributes a lot to OSM, even if he's somewhat pigheaded in applying his own particular approaches in places where it's not as clear he has a lot of local knowledge), but at the same time he doesn't seem to be willing to concede on this point which IMO has become an obstacle to improving the map as-used by Skobbler, Mapbox, Stamen, Cloudmade, Mapquest etc downstream. I guess my advice would be to proceed but be cautious of blowback. That's what I've ultimately decided to do. I'm in semi-regular conversation with NE2 offlist, and while I'm happy with my own changes and stand by them, I also have no interest in an edit war or a situation where NE2's valuable contributions are put in jeopardy. If he feels strongly enough to revert them, I'd be okay with that. I have no opinion on the network vs. modifier question, but I do believe that ref tags should be usable as-is in a renderer, so that's the direction my changes have taken. I'll let this all sit for a while, to see where it winds up in a few weeks. -mike. michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] which boundaries are these?
On Oct 25, 2012, at 12:30 PM, Martijn van Exel wrote: Hi all, I want to do a more thorough look into TIGER ghost towns and am thinking about a town-by-town analysis. For that I need proper 'town' boundaries. I like the ones you get when you search for a place on Google Maps: https://maps.google.com/maps?q=princeton,+nj https://maps.google.com/maps?q=nederland,+co What are these boundaries? I am guessing it is some kind of Census division? It's likely to be a Census place, non-continuous areas that can cross other boundaries are intended to represent logical towns and cities. http://www2.census.gov/geo/tiger/TIGER2012/PLACE/ -mike. michal migurski- contact info and pgp key: sf/cahttp://mike.teczno.com/contact.html ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Scrubbing route relations (attn: Richard Welty, etc.)
On Oct 23, 2012, at 1:54 PM, Michal Migurski wrote: On Oct 21, 2012, at 8:54 PM, Michal Migurski wrote: I feel like this scrubbing process has revealed so much about the intricacies of different road networks that I'm going to take a slightly different approach, and focus my work on just the ref and modifier tags. I can standardize the US:US and US:I networks along with US:CA where I live, but I should hold off on attempting to overfit other states' network tags. Here's the newest: http://mike.teczno.com/img/OSM-Extracted-Routes-changes-2.csv.zip There are 5,828 changes now. I have left the network tags alone, generally. Most changes are focused on the ref and modifier tags. I'm looking for advice feedback. I applied these changes to OSM last night, in a series of five changesets: http://www.openstreetmap.org/browse/changeset/13611326 http://www.openstreetmap.org/browse/changeset/13612265 http://www.openstreetmap.org/browse/changeset/13612825 http://www.openstreetmap.org/browse/changeset/13612736 http://www.openstreetmap.org/browse/changeset/13613023 Offlist, I've been talking to NE2 about the edits, and he pointed out this morning that they negatively affect shield rendering on Aperiodic: http://elrond.aperiodic.net/shields/?zoom=15lat=38.7166lon=-77.79472layers=B Whereas formerly relations with network=US:US and the modifier in the ref failed somewhat gracefully if a bit pigheadedly (by not displaying shields at all), they now show up incorrectly as mainline routes. - NE2 NE2 asked me to revert the changes, because he's unhappy with me moving the route variant information from the ref tags to the modifier tags, e.g. turning ref=80 Business into ref=80 modifier=Business. According to the supported tagging guidelines on Aperiodic, my interpretation should be correct: The value of the ref tag on the relation must contain just the route number, without any network information. http://elrond.aperiodic.net/shields/supported.html I'm looking for guidance on this changeset, with the intent of making route relation information in the US internally consistent. I can simply revert it, but I wasn't happy with the state of relation tags before and I'll continue to look for ways to make them consistent nationally. I can apply a new changeset that moves or duplicates the variant information in the modifier tags to the ref tags, but this feels incorrect. I can apply an alternative changeset that moves or duplicates the variant information to the *network* tags (another recommendation from the Aperiodic tagging guideline), but previous conversations about this change led me to believe that messing with the network tags too much would be a Bad Idea. For those of you with an interest in the route relations, what do you think is the correct next move here? NE2, I've been talking to you offlist but I hope you jump in here. -mike. michal migurski- m...@stamen.com 415.558.1610 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Scrubbing route relations (attn: Richard Weait, etc.)
(Changing subject line to the Richard I originally meant) That was my understanding as well, but I got feedback that boiled down to don't mess with the network tags (too much). What do others think about this? -mike. On Oct 24, 2012, at 12:16 PM, Dale Puch wrote: That is my understanding as well based on previous discussions. For For US Business route 80: network = US:US:Business ref = 80 On Wed, Oct 24, 2012 at 2:13 PM, Alexander Jones happy5...@gmail.com wrote: Using your example, the network tag should say US:US:Business Alexander Michal Migurski wrote: On Oct 23, 2012, at 1:54 PM, Michal Migurski wrote: On Oct 21, 2012, at 8:54 PM, Michal Migurski wrote: I feel like this scrubbing process has revealed so much about the intricacies of different road networks that I'm going to take a slightly different approach, and focus my work on just the ref and modifier tags. I can standardize the US:US and US:I networks along with US:CA where I live, but I should hold off on attempting to overfit other states' network tags. Here's the newest: http://mike.teczno.com/img/OSM-Extracted-Routes-changes-2.csv.zip There are 5,828 changes now. I have left the network tags alone, generally. Most changes are focused on the ref and modifier tags. I'm looking for advice feedback. I applied these changes to OSM last night, in a series of five changesets: http://www.openstreetmap.org/browse/changeset/13611326 http://www.openstreetmap.org/browse/changeset/13612265 http://www.openstreetmap.org/browse/changeset/13612825 http://www.openstreetmap.org/browse/changeset/13612736 http://www.openstreetmap.org/browse/changeset/13613023 Offlist, I've been talking to NE2 about the edits, and he pointed out this morning that they negatively affect shield rendering on Aperiodic: http://elrond.aperiodic.net/shields/?zoom=15lat=38.7166lon=-77.79472layers=B Whereas formerly relations with network=US:US and the modifier in the ref failed somewhat gracefully if a bit pigheadedly (by not displaying shields at all), they now show up incorrectly as mainline routes. - NE2 NE2 asked me to revert the changes, because he's unhappy with me moving the route variant information from the ref tags to the modifier tags, e.g. turning ref=80 Business into ref=80 modifier=Business. According to the supported tagging guidelines on Aperiodic, my interpretation should be correct: The value of the ref tag on the relation must contain just the route number, without any network information. http://elrond.aperiodic.net/shields/supported.html I'm looking for guidance on this changeset, with the intent of making route relation information in the US internally consistent. I can simply revert it, but I wasn't happy with the state of relation tags before and I'll continue to look for ways to make them consistent nationally. I can apply a new changeset that moves or duplicates the variant information in the modifier tags to the ref tags, but this feels incorrect. I can apply an alternative changeset that moves or duplicates the variant information to the *network* tags (another recommendation from the Aperiodic tagging guideline), but previous conversations about this change led me to believe that messing with the network tags too much would be a Bad Idea. For those of you with an interest in the route relations, what do you think is the correct next move here? NE2, I've been talking to you offlist but I hope you jump in here. -mike. michal migurski- m...@stamen.com 415.558.1610 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us -- Dale Puch ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us michal migurski- m...@stamen.com 415.558.1610 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Scrubbing route relations
On Oct 21, 2012, at 8:54 PM, Michal Migurski wrote: I feel like this scrubbing process has revealed so much about the intricacies of different road networks that I'm going to take a slightly different approach, and focus my work on just the ref and modifier tags. I can standardize the US:US and US:I networks along with US:CA where I live, but I should hold off on attempting to overfit other states' network tags. Here's the newest: http://mike.teczno.com/img/OSM-Extracted-Routes-changes-2.csv.zip There are 5,828 changes now. I have left the network tags alone, generally. Most changes are focused on the ref and modifier tags. -mike. michal migurski- m...@stamen.com 415.558.1610 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Scrubbing route relations
On Oct 22, 2012, at 8:00 AM, Paul Johnson wrote: On Oct 22, 2012 9:57 AM, Alexander Jones happy5...@gmail.com wrote: Paul Johnson wrote: FM and RM are the same network...seems odd for them to show up twice here... I could've sworn that the general consensus from a previous argument was one network per shield type. They use the same shield, and even TXDOT signs Farm Road and Ranch Road interchangeably. They're definitely the same network. Should they both be classified under the :FM network, if that's the case? -mike. michal migurski- m...@stamen.com 415.558.1610 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] press from SOTM US
On Oct 22, 2012, at 1:12 PM, Alex Barth wrote: The data extracted by geocoding should just not lead to a substantial extract of the database, hence not producing a derivative database in the sense of the ODbL. I feel this would be within the spirit of why the ODbL was adopted (to encourage contribution) while clarifying an important use of OSM data that would create a huge incentive to improve data. Right now we largely don't have functioning municipal boundaries in OSM. Obviously, any data that is mixed into OSM data for _powering_ the geocoder would fall under share alike stipulations. MySociety is working on derived municipal boundaries from OSM data: http://global.mapit.mysociety.org/ E.g.: http://global.mapit.mysociety.org/area/168844.html There's data in there, and code out there that you could build on. The MapIt service itself is non-commercial, but the code that drives it is freely-available. http://code.mapit.mysociety.org/ You bring up the important problem of properly bounding the geocoding case. I'm thinking if all that can be extracted from OSM's database are names and addresses for lat/lon pairs or lat/lon pairs for names or addresses, it would be arguably impossible or at least impractically hard to recreate a functioning street network from it and the extracted data would be a narrow subset of OSM no matter how many locations are being geocoded. Thoughts? This seems to match the spirit of the license as far as I understand it. -mike. michal migurski- m...@stamen.com 415.558.1610 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Scrubbing route relations
On Oct 21, 2012, at 5:28 AM, Minh Nguyen wrote: On 2012-10-20 4:00 PM, Michal Migurski wrote: - Normalizing network names for all county routes with the :CR infix I'm not enthusiastic about sticking `:CR` in all the county route relations. I favor `US:[state]:[county]`, at least for the relations in Ohio, for the same reason we have `US:[state]` rather than `US:SR:[state]`. It doesn't look like you touched any Ohio county routes, but that's probably because you didn't realize that's what they are. :-) Ohio county route relations' `network`s conform to a simple pattern: `US:OH:[ABC]`, where [ABC] is the county's three-letter, all-caps ODOT code. Obviously, the codes aren't used as commonly as USPS state abbreviations, but many counties use them on signage, and they're quite handy for this purpose. That's right, I didn't understand those and they didn't look like county names, so I left them alone. The nice thing about the :CR part is that it helps explain what the word is after, for example county names. There are only 50 states so it seems easier to just say US:[state], but there are loads more counties and it's impractical to remember them all on sight. Having the extra `:CR` component might make sense in states like New Jersey and California that have consistent, statewide county route standards. But in Ohio, most counties that signpost their routes do it in different ways, in violation of the state MUTCD. There are so many variations that entire websites [1] are devoted to documenting them. (And as you'd imagine, some townships have their own unique route shields, too.) That's interesting and worth knowing, thanks. You mentioned that using `:CR` makes it possible to correctly interpret county routes without knowing the county names. I guess that depends on what we expect the relations to be used for. To a developer generating shields for display on a map, `:CR` would suggest standardizing on, say, the blue and gold pentagonal shield, when in fact that would be misleading in maybe three-quarters of the state. And I'd say the shields are the /only/ interesting thing about Ohio county routes. By the way, if anyone's interested in rendering these shields, I've started a collection of SVG templates at Wikimedia Commons [2]. One thing I learned while making these templates is that some counties include the township name in their county route shields. Presumably, a route that crosses township lines would have more than one shield variant. Should we have subrelations with the township name in `modifier`? The blue and gold shields are used in many places around the country, right? I think you're right, that using :CR where those are in place would make a lot of sense. I feel like this scrubbing process has revealed so much about the intricacies of different road networks that I'm going to take a slightly different approach, and focus my work on just the ref and modifier tags. I can standardize the US:US and US:I networks along with US:CA where I live, but I should hold off on attempting to overfit other states' network tags. -mike. michal migurski- m...@stamen.com 415.558.1610 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] SOTM-US geocoding/share-alike discussion
I'm not on legal-talk, so this mail is going out only to Talk-US. I'm happy to have it forwarded. We had a license BoF organized primarily by Mapbox (Eric Gunderson and Alex Barth) with participation from Foursquare (David Blackman), on the topic of the license and its effect on geocoding data. Steve C, Henk Hoff, Paul Norman, Richard Fairhurst and many others attended. My understanding of Mapbox's issue, paraphrased, is that they have potential clients with lawyers scared of the ODbL and license status of latitude and longitudes returned from addresses geocoded against OSM. As I understood it, the end result of the discussion was that the ODbL may or may not apply in this case and that Mapbox should submit some specific uses cases to the board to illustrate their specific concern so we can all stop blathering about whether the license is good or bad and move on to useful particulars. In other words, what the 20120522 LWG meeting notes say. -mike. On Oct 21, 2012, at 1:58 AM, Frederik Ramm wrote: Hi, on talk-us there was a mention of Carl Frantzen's recent three-part article with SOTM-US coverage, http://idealab.talkingpointsmemo.com/2012/10/openstreetmap-part-1-new-cartographers.php, and his mention of OSM moving away from his open-source roots. Apparently, this refers to some unfortunate statements at SOTM-US about share-alike being bad for business or something, and Frantzen mentions that a couple of businesses have set up an informal group to discuss which bits of our license they don't understand or want clarification on. As far as I know, nobody who knows anything about OSM seriously suggested that we move away from open source, it was just a phrase unfortunately reported. I am still rather surprised to hear about this as a side note of SOTM-US coverage instead of here on this list where license discussions should be at home. I would urge anyone who is unclear about anything with ODbL and/or who believes that any community norms we have must be refined, to discuss that here on this mailing list - whether it's for business or personal use. Looking through past discussions in the archives of minutes of our Licensing Working Group, it seems clear to me that OSM data under ODbL is unlikely to ever be available for no strings attached geocoding; we won't ask for your customer database just because you geocode with OSM, but you will have to adhere to some rules nonetheless. LWG has never actually made a decision on geocoding, and all mentions in their minutes carry big disclaimers (This is a summary of our discussion and should NOT be construed as a formal statement of position). Under that disclaimer, the 20120515 minutes contain the following: To be able to claim that the remainder of the record, (often proprietary business information or personal information such as a patient record) is not virally touched by geocoding against OSM ODbL data needs a distinction to be demonstrated. This distinction needs to be a clear and logical general rule or principle. It also needs to be acceptable to the OSM community. At the moment, we feel this does not exist. In the same notes there's a discussion of a like with like principle which means that Whatever is used in the (reverse)geocoding look-up is virally touched, but nothing else. The 20120522 meeting notes contain a link to a concept paper https://docs.google.com/document/pub?id=1Ag81OlT1TtnhYwVE-bBtL018SNoU_V-anG4wLdwMT4c and explicitly say: To improve it, and test the rationality of the ideas expressed, we need and welcome real-world cases of geocoding and reverse-geocoding. So I guess anyone who wants to use OSM in a geocoding scenario should read that and submit their opinion, here or to LWG. Personally, I've gone on record as an advocate of a non-share-alike (PD) license for OSM but the project as a whole has decided to have a share-alike license and I accept that; I don't think that geocode as much as you want without sharing any data is possible with the ODbL data set. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us michal migurski- m...@stamen.com 415.558.1610 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Scrubbing route relations
I did not know about the Aperiodic site, very cool! It sounds from the advice there that I should use a different strategy with the network tags, and duplicate the Loop, Business and other modifiers to that tag. -mike. On Oct 19, 2012, at 11:26 PM, Chris Lawrence wrote: My only concern with the scrubbing applies to bannered routes (routes with the modifier tag); we've been going back and forth on the proper tagging for seemingly years now, and while I think the proposed scrubbing conforms with the original intent under the tagging scheme, the generally agreed-upon (by everyone except NE2, at least) tagging seems to be as documented at: http://elrond.aperiodic.net/shields/supported.html The rendering expects data tagged according to the documentation on the wiki and further refined based on discussions on the talk-us mailing list. In particular, two things are worth knowing: * The value of the ref tag on the relation must contain just the route number, without any network information. The relation for Interstate 5 should be tagged with network=US:I, ref=5. * Variant routes (alternate, business, truck, etc.) must be tagged with the variant in the network tag, not the ref tag. (The variant text should also be in the modifier tag, but this rendering does not depend on its presence there.) Thus, US Alternate Route 1 would only be rendered if it were tagged network=US:US:Alternate, ref=1. Since this site (http://elrond.aperiodic.net/shields/) seems to be the only real consumer of route relations at the moment (we're not sure if Mapquest Open is or whether it's just running a bunch of ad-hoc data fixups to make its shields sensible), I think it makes sense to follow the guidance from that site at least as far as deprecating modifier in favor of making variants part of the network tag. However, the CR tagging changes suggested here seem fine to me. Chris ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us michal migurski- m...@stamen.com 415.558.1610 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Scrubbing route relations
Based on feedback from route relation mappers and people on this list, here's a list of 7,575 route relation changes I'd like to make: http://mike.teczno.com/img/OSM-Extracted-Routes-changes.csv.zip Some of the rules I've followed: - Shortening ref tags to just what goes on a shield, with prefixes and suffixes moved to the network and modifier tags - Treating Business as a special subnetwork, e.g. US:I:Business vs. US:I - Normalizing network names for all county routes with the :CR infix - Title-case and spaces for wordy networks If there aren't any objections, I'd like to start applying these to the OSM database this upcoming week. -mike. michal migurski- m...@stamen.com 415.558.1610 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Scrubbing route relations
On Oct 20, 2012, at 4:43 PM, Paul Norman wrote: From: Michal Migurski [mailto:m...@stamen.com] Sent: Saturday, October 20, 2012 4:01 PM To: OpenStreetMap U.S. Subject: Re: [Talk-us] Scrubbing route relations Based on feedback from route relation mappers and people on this list, here's a list of 7,575 route relation changes I'd like to make: http://mike.teczno.com/img/OSM-Extracted-Routes-changes.csv.zip Some of the rules I've followed: - Shortening ref tags to just what goes on a shield, with prefixes and suffixes moved to the network and modifier tags Perhaps I'm misunderstanding, but some of the lines seem to indicate the reverse. Line 3030, id=301986 seems to describe a change from network=US:I:Business to network=US:I and ref=69 to ref=69 Business. It's the opposite, looks like I misnamed the columns. I've posted a new version of the doc with the correct column names to the same address, thanks for noticing! Looking at id=172892 and id=1745481 is also confusing. They're both truck routes, but one gets Truck in the ref and the other doesn't? They both get Truck removed (see above with the switched column names). The change would be from these: network=US:OH:Truck,ref=38, modifier=Truck network=US:AL, ref=47 Truck, modifier= …to these: network=US:OH, ref=38, modifier=Truck network=US:AL, ref=47, modifier=Truck You might want to seek out the top few editors of relations who aren't on the list and point them at http://lists.openstreetmap.org/pipermail/talk-us/2012-October/009253.html and invite them to join in. I've been in touch with them via messages on the OSM.org site, not sure how many are subscribers here. -mike. michal migurski- m...@stamen.com 415.558.1610 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Scrubbing route relations
On Oct 18, 2012, at 10:08 PM, Toby Murray wrote: On Thu, Oct 18, 2012 at 6:26 PM, Michal Migurski m...@stamen.com wrote: Hi everyone, We're getting ready to do a major data update to the Stamen Terrain layer and I've been working on scrubbing the route relations data from OSM. I've linked to a before and after CSV, processed via Google Refine to normalize networks, refs and modifiers. I'm curious to get some feedback on it: http://mike.teczno.com/img/osm-scrubbed-routes-2012-09.zip The associated code: https://gist.github.com/3915267 Reviewing a diff, I see pretty much all of the relations I have created are unchanged soo... looks great! :) More seriously, I do like the idea of having the county name in there for county roads. But I haven't done any county routes myself yet so my opinions are not particularly strong. Looking at the diff visually I see a lot of either moving modifiers to their own tag or removing the duplicated modifier from the network tag. I guess the question there is, should the various loops, bypasses, truck routes, etc be considered as part of the same network as the unmodified highway. For example if I do a tag query for network=US:I and ref=376, should it return the business route as well as the base US 376 route? My understanding of the modifier tag is that the ref should be the content of the main part of the shield, and the modifier goes above e.g. these shields: http://en.wikipedia.org/wiki/List_of_business_routes_of_the_Interstate_Highway_System I'm not well versed enough in these to really get what the ref should contain, but it seems logical to me that it would be short and sweet like an actual sign on the side of a road. I've posted a new version of everything at the URL's above, with corrected county network names. -mike. michal migurski- m...@stamen.com 415.558.1610 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Scrubbing route relations
Hi Richard, thanks! The CR vs. County name thing is new to me. Another mapper suggested that I replace them with something like this so it's not necessary to know all the county names in order to correctly interpret something as a county route: network=US:NY:CR:Rensselaer Kosher? -mike. On Oct 18, 2012, at 5:16 PM, Richard Welty wrote: the scrubbing looks mostly ok, BUT... i am one of the folks who started out using US:state:CR for county route networks, which i now believe was a mistake. we really should be using county names, not CR in the network tag, e.g. network=US:NY:Rensselaer otherwise we can't really tell which county the route is in, and can't distinguish CR 1 in Columbia County from CR 1 in Rensselaer County. richard On 10/18/12 7:26 PM, Michal Migurski wrote: Hi everyone, We're getting ready to do a major data update to the Stamen Terrain layer and I've been working on scrubbing the route relations data from OSM. I've linked to a before and after CSV, processed via Google Refine to normalize networks, refs and modifiers. I'm curious to get some feedback on it: http://mike.teczno.com/img/osm-scrubbed-routes-2012-09.zip ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us michal migurski- m...@stamen.com 415.558.1610 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Network tag Re: Highway Shield Rendering
On Apr 8, 2012, at 7:49 AM, Nathan Edgars II wrote: On 4/8/2012 10:27 AM, Craig Hinners wrote: Chris Lawrencelordsu...@gmail.com: modifier=* would represent MUTCD-type banners attached to the shield This is the first I've heard of this tag. I don't recall it being discussed when we were hashing ideas around on this last summer. (Not that that is reason to discount it.) But what came out of that discussion was the following guidance: ref will store the unique identifier within a particular classification, where particular classification is stored wholly in the network tag. So, network=US:US:Business/ref=13 and network=US:US:Truck/ref=70 both conform to that definition. network=US:US/modifier=Business/ref=13 does not. On the other hand, network=US:US ref=13 Business does. For what it's worth, as a renderer I preferred the modifier tag when doing the Terrain shields: http://maps.stamen.com/terrain/#13/34.0510/-118.2146 Modifiers in the ref tag would be a close second; those typically need a lot of scrubbing and normalization anyway. In the network tag would be the biggest hassle of all. I'm working on a revision to the Terrain tiles, by the way, and I'm curious if I got anything obviously wrong with the highways that I should look at fixing. One thing I'll be attempting is to get the shields to behave a little less sloppily as they meander along routes, probably by staggering them more. -mike. michal migurski- m...@stamen.com 415.558.1610 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
[Talk-us] Printing out counties
I'm making print versions of OSM coverage in US counties: http://mike.teczno.com/notes/county-papers.html What additional data, whether OSM or Census or otherwise, should be on there? -mike. michal migurski- m...@stamen.com 415.558.1610 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Printing out counties
On Mar 11, 2012, at 12:24 PM, Ian Dees wrote: On Sun, Mar 11, 2012 at 2:07 PM, Michal Migurski m...@stamen.com wrote: I'm making print versions of OSM coverage in US counties: http://mike.teczno.com/notes/county-papers.html What additional data, whether OSM or Census or otherwise, should be on there? - Public spaces (parks, zoos, police/fire stations, etc.) - Addressing (usually the lack thereof) - Building outlines These are the sorts of things that counties tend to already have data for. They'll either be impressed with how great OSM is or they'll show off their data and we can get them to give it to us :). I like the contrast between OSM is good and help make OSM good - maybe there's a way to detect which is the case, and adapt the wording around the maps to highlight this. -mike. michal migurski- m...@stamen.com 415.558.1610 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] The vandalism has begun
On Mar 11, 2012, at 4:17 PM, Mike N wrote: I looked at the .CA lists and couldn't tell if this was due to removing data that isn't even CC-SA compliant, or making it easier to get a jump start on remapping. If the purpose is to make remapping easier, I think it is disrespectful of data consumers. Data consumers are aware of the 1 April date, after which time anything goes as far as removing non-ODBL data. They can prepare for 1 April by stopping real time updates, however this removal may have taken them by surprise. Agreed. It would be nice to archive the last or second to last planet of this month and not have giant chunks of data missing or broken. -mike. michal migurski- m...@stamen.com 415.558.1610 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Printing out counties
On Mar 11, 2012, at 1:31 PM, Frederik Ramm wrote: Hi, On 03/11/2012 08:37 PM, Michal Migurski wrote: These are the sorts of things that counties tend to already have data for. They'll either be impressed with how great OSM is or they'll show off their data and we can get them to give it to us :). I like the contrast between OSM is good and help make OSM good - maybe there's a way to detect which is the case, and adapt the wording around the maps to highlight this. It would be ideal if the wording makes clear that the primary way of making OSM good is getting an account, logging in, and mapping; doing a mapping party, meeting other mappers in a pub, do cool stuff. Not taking county data and importing it ;) For sure, lot's of Join Us, Hippies!-type verbiage. =) -mike. michal migurski- m...@stamen.com 415.558.1610 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Address improvement through imports?
On Nov 2, 2011, at 9:28 AM, Martijn van Exel wrote: On Wed, Nov 2, 2011 at 10:23 AM, Anthony o...@inbox.org wrote: On Wed, Nov 2, 2011 at 12:19 PM, Martijn van Exel m...@rtijn.org wrote: I'm all for not importing data where there's existing data people can use, but in the case of TIGER addresses you could actually make a point for importing: OSM could be a platform for improving that address data (like it should be for the GNIS points). 'All we need' is a suitable microtasking platform. I can't speak for other locations, but here in Florida there is much better address information available from the counties than that available from TIGER. So if you're going to build a suitable microtasking platform for Florida, use the county data, not TIGER :). Couldn't agree more - we have to make an effort to import the best address data available and that probably means looking at the local government level. That's why I made that page I started this thread with in the first place - to try and consolidate that effort. I like this approach. Ian and others in the thread have described address imports as a sort of computer-assisted manual editing, and I think for US addressing it's a great approach. The gulf between urban and rural parts of the U.S. is wide, and if we don't take advantage of the excellent local government data available for these purposes there are going to be regions of the country that *never* get mapped. I believe that we should develop an approach that is welcoming to local government representative who can help with the assisted importing, which may involve doing a few test imports in known places followed by the development of new tools or description of new procedures that newcomers to OSM who already steward local address databases can repeat for their own jurisdictions. First step would probably be a plan for shitty TIGER data - what should these local government folks do in cases where their local OSM coverage is pure TIGER 2007 drek? Just to name a local example, Mill Valley CA is right across the bay from SF and composed of mostly import junk. Maybe someone (heh) could do a purpose-built fork of Potlatch designed especially for pulling in address info without displaying any other road data to eliminate confusion, for use by owners of address info who know they have good, high-precision coverage. -mike. michal migurski- m...@stamen.com 415.558.1610 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] US local chapter board election results
Hey, cool! Thanks everyone. I'm excited to get started with Martijn, Randy, Jim, and Richard. According to the wiki page there is a monthly chapter meeting tomorrow, but the most recent one was six months ago. I'll dial the number tomorrow and see what happens. =) -mike. On Oct 12, 2011, at 9:49 AM, Richard Weait wrote: Dear All, The results of the US local chapter board election have been received and found to be valid. Fourteen valid ballots were received from 50% of the eligible members. I would like to thank the outgoing members of the board for their service to the community. I would also like to thank all of the candidates for offering to serve for the next year. Richard Welty, was re-elected. Martijn, Randy, Jim and Mike were each elected. You can find the detailed results on the wiki, http://wiki.openstreetmap.org/wiki/Foundation/Local_Chapters/United_States/Elections Best regards, Richard Weait, independent scrutineer, on behalf of, Jonathan Bennett, independent scrutineer, Ian Dees, member of outgoing board. ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us michal migurski- m...@stamen.com 415.558.1610 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] US local chapter board election results
Hand off agenda is a great idea. My computing habits mean that IRC is probably unrealistic for me. As far as goals for this year, I know that Ian has been building a server intended for hosting tile renders. I'd like to see that continue with a US-specific tile layer ready for public consumption six months from now. Another idea I'd love discuss are extracts designed to assess the quality of OSM data on a county by county basis. I think this can be done in an automated fashion borrowing some of the ideas introduced by Geofabrik's Inspector, ultimately resulting in a regularly-produced summary of data quality for each of the 3000+ counties in the US. Anyway, 'til tomorrow. -mike. On Oct 12, 2011, at 1:47 PM, Jim McAndrew wrote: I agree about the idea of the handoff agenda, but with or without, I will also be in the meeting tomorrow. We should at least go in with some goals on what we plan to accomplish this year, and discuss if those goals are practical and how we can work together to help each other with our goals, and how our goals fit with the goals of the greater OSM US. I'm going to try to be on the IRC channel more often as well. On Wed, Oct 12, 2011 at 12:39 PM, Richard Weait rich...@weait.com wrote: On Wed, Oct 12, 2011 at 2:32 PM, Michal Migurski m...@stamen.com wrote: Hey, cool! Thanks everyone. I'm excited to get started with Martijn, Randy, Jim, and Richard. According to the wiki page there is a monthly chapter meeting tomorrow, but the most recent one was six months ago. I'll dial the number tomorrow and see what happens. =) Perhaps the outgoing board can help you to put together an agenda, here on the list? I'm sure they'll have some thoughts on a smooth transition as well. Also #osm-us is a low traffic irc channel that might work for you. #osm-us is on irc.oftc.net, and available from the browser at http://irc.openstreetmap.org/ michal migurski- m...@stamen.com 415.558.1610 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
[Talk-us] Walking Papers is back and I'm sorry it was down.
Hi everyone, I fixed Walking Papers today. It was not processing new prints and scans for almost two weeks. It's been hit with a double-whammy of garbage input and not enough of my time to fix it. I apologize for the downtime, especially to those of you hoping to use Walking Papers in your mapping parties and everyone who sent e-mail. I’ve changed our polling script to hopefully be more resilient to future problems, and I’m expecting the full set of prints and scans from the affected time to be processed successfully. -mike. michal migurski- m...@stamen.com 415.558.1610 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Address Node Import for San Francisco
On Dec 9, 2010, at 3:00 PM, Gregory Arenius wrote: About the data. Its in a shapefile format containing about 230,000 individual nodes. The data is really high quality and all of the addresses I have checked are correct. It has pretty complete coverage of the entire city. I've worked with this file before. When I matched it to OSM data two years ago, I found that the SF data had numerous errors, so I wrote this mapping script: http://mike.teczno.com/img/sf-addresses/mapping.py Usage: mapping.py [osm streets csv] [sf streets csv] [street names csv] Here are all the street names in the shapefile: http://mike.teczno.com/img/sf-addresses/sfaddresses.csv Here are all the street names in OSM at the time I did the comparison (may have changed since): http://mike.teczno.com/img/sf-addresses/osm_streets.csv And this is the mapping result I got: http://mike.teczno.com/img/sf-addresses/street_names.csv Hopefully this is helpful, as you'll want to import street names that actually match those in OSM's view of San Francisco. I found some other weird burrs in the data as well, in terms of how it arranges addresses stacked on top of one another in tall buildings. Nothing that can't be dealt with in an import. I also did a bunch of geometry work to match those address points to nearby street segments in order to break up the street grid into addresses segments, but that code is a bit of a rat's nest. The idea was to build up the little block numbers you see rendered here: http://www.flickr.com/photos/mmigurski/5229627985/sizes/l/ Katie's suggestion of breaking the data into smaller chunks is a good one. -mike. michal migurski- m...@stamen.com 415.558.1610 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] San Francisco Geodata
On Aug 26, 2010, at 1:15 PM, Gregory Arenius wrote: San Francisco has a website at datasf.org that has a lot of good geodata on it. The current licence isn't acceptable for using any of the data but they might be willing to give us permission to use it. Some of the datasets that I thought were interesting were city lots, addresses, neighborhoods, landuse, public lands, street centerlines, speed limits and the bike network. I've worked with those centerlines, and I think that they are the same file that the county provided to TIGER last time around which was already imported into OSM a few years back. It's already been very well improved upon. HOWEVER, the address points data file is fully legit and I have a lot of personal experience working with it and OSM together. Nothing public so far but I'd be willing to chat about it here if I can remember the code I wrote last year. =) -mike. michal migurski- m...@stamen.com 415.558.1610 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] OSM presentations
I gave this keynote at the North American Cartographic Information Society late last year: http://mike.teczno.com/notes/slides/nacis.html It's mostly about the motivations for OSM and the end products that can be made from the data. Maybe applicable? -mike. On Mar 6, 2010, at 7:41 AM, Richard Welty wrote: is there a collection anywhere of presentations on OSM that are available for reference or reuse? if there isn't, anyone have presentations that they're willing to let me take a look at? now that i am on the verge of being employed again, i find much to my surprise that there will be some mapping involved and that the folks at GE Research i'll be working with are intrigued by OSM. thus, i will likely need a presentation to give spun towards experienced software types who are mostly a little light on mapping and GIS (they're working on a logistics/supply management system for internal use by GE manufacturing units). probably there is no existing presentation that does exactly what i want, but being able to look at a number of them might be a big aid in developing one. richard ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us michal migurski- m...@stamen.com 415.558.1610 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Historical locations
Certainly moving or cloning the Historic marker into the tags, the formal metadata, is better than it being only informal in the name. That would make them disappear from the common renderings, but would keep them in the database for use by a custom style or query? I would greatly prefer this approach. Although tagging for the renderer is greatly discouraged, right now a lot of renders look wrong. OpenCycleMap, for example, looks like the Great Guide To Biking To Church at certain zoom levels. -mike. michal migurski- m...@stamen.com 415.558.1610 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
[Talk-us] Bay Area trailer parks: hamlet ? Also neighborhoods cities
Hi, There are a large number of mobile home / trailer parks mapped in San Jose, Santa Clara, and other parts of the South Bay. They're tagged place=hamlet, and I'm wondering if there's a better way to identify them? Beej71, if you're on this list I think a lot of these came from you. Examples here: http://www.openstreetmap.org/?lat=37.3995lon=-122.01521zoom=15layers=B000FTFT http://www.openstreetmap.org/?lat=37.37096lon=-121.89402zoom=16layers=B000FTF Hamlet is supposed to be defined by national/state/provincial government yet these aren't really defined by anyone except their owners. I personally view them as generally equivalent to named apartment complexes, and therefore not a place. May I suggest that they be redrawn as landuse=residential areas, with names defined? On a related topic, I'm also wondering how to handle parts of cities that are places or neighborhoods yet not administratively distinct, e.g. West Oakland (Oakland), The Mission (SF), etc. Would place=neighborhood make sense here? This part of West Oakland with two named apartment complexes (all place=hamlet) illustrates what I mean: http://www.openstreetmap.org/?lat=37.8096lon=-122.29504zoom=16layers=B000FTF -mike. michal migurski- [EMAIL PROTECTED] 415.558.1610 ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
