Re: [Talk-us] NHD Dataset
On Mon, Feb 16, 2009 at 10:37 AM, Theodore Book tb...@libero.it wrote: * In some places, a lot of work has been done manually entering water features. While I expect the NHD data is generally better, we have the tough issue of respecting other people's work (and not duplicating things in the database) I would be willing do merge my own basin or compare the data first before upload. http://openstreetmap.org/?lat=34.636lon=-82.845zoom=10layers=B000FTF Cheers, Adam * The upload speed seems to be a limiting factor - it is taking me some 70-80 hours to upload the one basis with bulk_upload.pl. I am not sure how many basins there are in the country, but it seems as though it could take a year if it were all done sequentially. * The scripts I used to convert the shapefiles to OSM only supported the features I found in this one basin. They should be easy to extend, (or someone else may have better ones), but they would need to be tested. Theodore Ian Dees wrote: On Mon, Feb 16, 2009 at 9:00 AM, Theodore Book tb...@libero.it mailto:tb...@libero.it wrote: If you haven't been following the wiki page, I have been doing some work on the NHD dataset, and feel that I have gotten a decent OSM conversion of the Etowah river watershed (north and northwest of Atlanta). I am going ahead and uploading that basin. If you are interested in the conversion, it would be great if you could take a look at it and give me any feedback you may have. Thanks. Excellent work! I think the next step is to list all of the subbasins and start converting/importing. What is everyone's opinion on waiting for the API 0.6 upgrade? I'll bet we could sneak in a complete import before... Also, do we want to have individuals make requests for data via the NHD website or should I make another DVD-data-dump request to the NHD folks and import the whole thing at once? ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] NHD Dataset
On Mon, Feb 16, 2009 at 10:37:05AM -0500, Theodore Book wrote: I think it would be great to get it all in as soon as possible. I do see a few limiting factors, though: * In some places, a lot of work has been done manually entering water features. While I expect the NHD data is generally better, we have the tough issue of respecting other people's work (and not duplicating things in the database) * The upload speed seems to be a limiting factor - it is taking me some 70-80 hours to upload the one basis with bulk_upload.pl. I am not sure how many basins there are in the country, but it seems as though it could take a year if it were all done sequentially. If you're not uploading from dev.openstreetmap.org, there's probably a significant speedbump to be gained from doing so. Additionally, once the 0.6 API is up, we can group a lot of changes into changesets: a *huge* chunk of the time spent when uploading remotely is spent on round trip to the server, nothing else. * The scripts I used to convert the shapefiles to OSM only supported the features I found in this one basin. They should be easy to extend, (or someone else may have better ones), but they would need to be tested. What script did you use for doing the polygons themselves? Did i t support splitting of large polygons into smaller ways? Regards, -- Christopher Schmidt MetaCarta ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] NHD Dataset
On Mon, Feb 16, 2009 at 10:24:21AM -0600, Ian Dees wrote: On Mon, Feb 16, 2009 at 9:56 AM, Christopher Schmidt crschm...@metacarta.com wrote: What script did you use for doing the polygons themselves? Did i t support splitting of large polygons into smaller ways? Chris, I was thinking about how to implement this in my shp-to-osm convertor thing. Is it enough to just split the ways and have shared nodes (with the next way) at either end? Does a relation need to be defined? I don't think either is enough. In order for things to render correctly, the standard way when tracing is to create smaller closed polygons -- the Charles River is a set of 10 or so, I believe. I don't know enough about water to know if other things will work, but I do know that the 0.6 API switch will 'split' any way over 2000 nodes; perhaps there is some documentation related to that talking about what will be one or how to wavoid problems... Regards, -- Christopher Schmidt MetaCarta ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] NHD Dataset
It looks like the solution is to use Advanced Multipolygons: http://wiki.openstreetmap.org/wiki/Talk:Relation:multipolygon#Advanced_multipolygons Does anyone want to start writing an algorithm to nicely slice up polygons into small enough chunks? On Mon, Feb 16, 2009 at 11:02 AM, Rev. Theodore Book tb...@libero.itwrote: I know that splitting large areas into smaller polygons makes some of the processing faster, but it seems to me that it introduces some problems, as well - by having multiple lakes with the same name, for example, it seems that it would be more difficult to render names properly, and the representation seems to become rather arbitrary. Is there a proposal for a relation to identify that the multiple areas are actually parts of the same polygon? Christopher Schmidt wrote: I don't think either is enough. In order for things to render correctly, the standard way when tracing is to create smaller closed polygons -- the Charles River is a set of 10 or so, I believe. I don't know enough about water to know if other things will work, but I do know that the 0.6 API switch will 'split' any way over 2000 nodes; perhaps there is some documentation related to that talking about what will be one or how to wavoid problems... Regards, ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] NHD Dataset
On Mon, Feb 16, 2009 at 12:55:51PM -0600, Ian Dees wrote: It looks like the solution is to use Advanced Multipolygons: http://wiki.openstreetmap.org/wiki/Talk:Relation:multipolygon#Advanced_multipolygons Does anyone want to start writing an algorithm to nicely slice up polygons into small enough chunks? So, to be clear, what this looks like to me is that for any polygon, you can take the 'one long way' that would previously have outlined it, and instead of having that 'one long way', you have as many little ways as it would take to make up that ring. So, for a 4,005 node way, assuming a 1,000 node limit, you could have: * 10 1,000 node ways * 1 5 node way and: relation id=1 tag k=type v=multipolygon / member type=way id=1 role=outer / !-- 1-1000 -- member type=way id=2 role=outer / !-- 1001-2000 -- member type=way id=3 role=outer / !-- etc. -- member type=way id=4 role=outer / member type=way id=5 role=outer / /relation However, it is not clear that that is what Frederik meant in: http://lists.openstreetmap.org/pipermail/talk/2009-February/034091.html He suggested that the ways should be split up into smaller 'ways of managable size', but from what I've seen, these 'ways of managble size' are often closed polygons on their own: http://openstreetmap.org/browse/way/24417625/ Doing the former is easy. Doing the latter is hard, because figuring out how far you can go before 'drawing a line' to close the polygon is something I can't understand how to do. Imagine a 'C' shaped lake: --- | | | | | | | | |-- Creating closed chunks of that polygon, keeping the edges mostly as they are in the original, is not something I can imagine a way to do programatically. (I think that tesslating it into a bunch of triangles would be possible, but then you're really creating something that doesn't look at all like the output polygon.) Maybe Frederik was saying that the former is okay. My curiousity in that case is just if renderers -- especially Mapnik -- will actually support these things. (I can't imagine it working for osmarender, but that is less concenring to me.) If they won't, then adding the data like this may not be a good idea, simply because people will start doing wacky things to get data to show up in the map, messing up data... Is my concern clear? If it's the former, it's pretty easy to add to polyshp2osm, but the latter is hard... CCing talk@, since I assume frederik isn't on this list, and he can probably quickly tell me that I'm wrong in thinkign he might mean the latter. :) Regards, -- Christopher Schmidt MetaCarta ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
