Re: [Talk-us] [Imports] Uploads to City of Salisbury, MD
Nick, Josh In the meanwhile I have applied the changeset removing the duplicates and it seems to have finished just fine. I briefly downloaded the date again and the data now looks ok. Validator is not reporting any duplicates anymore. I also fixed the building=yes tags in the inner way of the multipolygons. Nick, you may go ahead now with checking the data and further improving it -one thing that surprised me was that there does not seem to be a place-node for the city itself. Cheers and happy mapping M. On 23.03.2012, at 03:09, Josh Doe wrote: > On Thu, Mar 22, 2012 at 8:49 PM, Josh Doe wrote: >> On Thu, Mar 22, 2012 at 10:52 AM, Nick Chamberlain >> wrote: >>> Mark, if you could commit the remove duplicates changeset, that'd be >>> great. I will do my best to check if the issues are resolved, and will >>> gladly accept any guidance on the best ways to do so. Thanks. >> >> I'm reverting a few of the changesets as we speak, so if Mark could >> hold off a few minutes, I'll update you all as I go. Since this only >> concerns the US, future messages will only be sent on the talk-us@ >> list. > > I've reverted a few of them (see [0]), but when I checked #10901301 > (2 nodes), it couldn't revert cleanly, since some nodes are > danglers, while others are used for ways in subsequent uploads. I'll > stop for now, but maybe Mark can take a look at things now? > > -Josh > > [0]: > http://wiki.openstreetmap.org/wiki/User_talk:Nick_SPW#Salisbury.2C_Maryland_import ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] [Imports] Uploads to City of Salisbury, MD
Nick and Josh thanks for the clarification on your upload strategy. With previous large uploads I have experience the same behaviour resulting in massive dupes. So I guess it is not a conversion issue. If you want me to commit the remove duplicates changeset, I can do so. But you will have to go through the data subsequently and check if the issues are resolved and no new ones emerged. M On 22.03.2012, at 14:12, Nick Chamberlain wrote: > Josh and Marc, > > Thank you! I apologize that I'm unable to speak the OSM language as > well as everyone, I'm working on it :) I posted on the Salisbury, > Maryland Import page that Josh created to give more detail about my > uploads. > > I didn't really think that I created so many duplicates, because I did a > lot of things in JOSM before I actually chose to upload. One thing I > know for sure is that I didn't I upload until I was actually able to - I > was getting a proxy error and the uploads were timing out when I > attempted to upload the entire batch. I assumed that these attempts > were unsuccessful, which I might be wrong about and might have resulted > in duplication. > > I assumed that my successful attempts started, maybe @ 10901673, when I > realized I needed to break the original shapefile up tabularly into > percentiles and upload 10 segments of the building footprint dataset, > one after the other. These were all definitely successful, and were > only done once per percentile. > > Josh, where are you finding the list of changesets in the format you > posted? I can only figure out how to list them in my editor profile > with my comments. > > If you believe that the method you mention that removes the 71,000 nodes > is the best approach, please feel free to do so. I will also gladly > manually fix the inner ring tagging issue as the data gets fixed. > Please let me know what I can do to help. I am also willing to share > the .osm files and/or shapefiles if that will help. Thanks. > > - Nick > > -Original Message- > From: joshthephysic...@gmail.com [mailto:joshthephysic...@gmail.com] On > Behalf Of Josh Doe > Sent: Thursday, March 22, 2012 8:51 AM > To: Marc Zoss > Cc: impo...@openstreetmap.org; talk-us@openstreetmap.org; Nick > Chamberlain > Subject: Re: [Imports] [Talk-us] Uploads to City of Salisbury, MD > > On Thu, Mar 22, 2012 at 8:04 AM, Marc Zoss wrote: >> I briefly downloaded all sby:bldgtype-tagged ways and relation of > Maryland through the overpass-api. Then removed the ones having only a > sby:bldgtype tag, run the validator and deleted the duplicated nodes and > ways. >> This would result in a changeset to remove the roughly 71'000 > duplicates nodes and ways. >> >> If the area was edited since the import and reverting gets tricky, > this might be the option to go, at least the result looks ok at the > first glance. >> >> Please also note that the conversion step seems to add a building=yes > tag on on inner ring of building polygons () which is certainly bad > tagging, despite the correct rendering (52 occurrences, so could be > fixed manually). > > Thanks for doing that, as that was the next step I was going to try. I > posted some regarding the changesets here: > http://wiki.openstreetmap.org/wiki/User_talk:Nick_SPW#Salisbury.2C_Maryl > and_import > > I think perhaps we should revert a subset of the changesets, such as the > dangling nodes, and then use your method to handle the rest. > > -Josh ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] [Imports] Uploads to City of Salisbury, MD
I briefly downloaded all sby:bldgtype-tagged ways and relation of Maryland through the overpass-api. Then removed the ones having only a sby:bldgtype tag, run the validator and deleted the duplicated nodes and ways. This would result in a changeset to remove the roughly 71'000 duplicates nodes and ways. If the area was edited since the import and reverting gets tricky, this might be the option to go, at least the result looks ok at the first glance. Please also note that the conversion step seems to add a building=yes tag on on inner ring of building polygons () which is certainly bad tagging, despite the correct rendering (52 occurrences, so could be fixed manually). M On 22.03.2012, at 12:02, Josh Doe wrote: > On Thu, Mar 22, 2012 at 6:40 AM, Josh Doe wrote: >> On Wed, Mar 21, 2012 at 10:50 PM, Paul Norman wrote: >>> If there are duplicated ways and nodes, perhaps reverting is the best >>> option? >> >> Unfortunately I'd agree that reverting these changesets will be the >> easiest and best course of action. Trust me that you'll spend a long >> time cleaning up dupe nodes and ways. Some of the buildings have up to >> four dupe nodes and 3 dupe ways, all from different changesets. > > This is what I can tell from the first few building changesets (not an > extensive investigation): > 10882159: 493 dangling nodes (REVERT) > 10884039: 3517 dangling nodes (REVERT) > 10885267: 1 building node (OK) > 10891857: 5 dangling nodes (REVERT) > 10891870: 8000 dangling nodes (REVERT) > 10893364: building nodes and ways (OK?) > > -Josh > > ___ > Imports mailing list > impo...@openstreetmap.org > http://lists.openstreetmap.org/listinfo/imports ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] [Imports] Uploads to City of Salisbury, MD
Dear Nick I would strongly advise to review and revise you conversion scripts. The move to select and delete these duplicated ways with JOSM seems a bit risky. However for the already uploaded data you possibly don't have much choices. Give it a go by using the following in JOSM's find function. tags:1 and closed and "sby:bldgtype"=* I have also noted that your import has generated quite some duplicate ways (and thus ton of nodes). JOSM's validator is your friend - check the data with it and remove automatically the dupes - better do it twice. Best regars Marc On 22.03.2012, at 00:08, AJ Ashton wrote: > Hi Nick, > > The quality of these building footprints looks great, and it's the > kind of thing I personally like to see imported because it's tedious > to trace them accurately and difficult/impossible to capture them with > GPS. And as you mentioned it can assist future placement of shops, > amenities, etc. > > However, from downloading and looking at a random area in JOSM, I'm > seeing some problems. You can't tell from the renders, but many > buildings are there more than once as separate overlapping objects. > Example: > > - http://www.openstreetmap.org/browse/way/153813132 > - http://www.openstreetmap.org/browse/way/153808317 > - http://www.openstreetmap.org/browse/way/153849919 > > It looks like the problem objects all have a 'sby:bldgtype' tag and no > other tags, so it should be a straight-forward fix. I not exactly sure > the best way to go about that, though. > > -- > AJ Ashton > > ___ > Imports mailing list > impo...@openstreetmap.org > http://lists.openstreetmap.org/listinfo/imports ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
