Re: [Talk-us] [Imports] NHD data skipped by nhd2osm

2010-06-28 Thread andrzej zaborowski
On 28 June 2010 04:36, Nakor nakor@gmail.com wrote:
 On 06/23/2010 01:07 PM, James U wrote:


 2.  Merge all the files in josm.  Make sure you have allocated a lot of
 memory
 to josm as this can take a lot of memory.  It will also take a lot of
 time.



 Would it be possible to add a parameter to specify the first id for the
 generated file?

 You can then generate the flowline file with id starting at -1, the flowline
 with id starting at -1,000,000 and so on and instead of waiting hours for
 JOSM to merge the files, you just concatenate all the files.

There's a dummy script at
http://repo.or.cz/w/ump2osm.git/blob/HEAD:/osm-merge to do that (would
need to be modified for 2 layers).  The bug mentioned there has been
fixed and I though josm now does something clever on merging, like
merging nodes.

BTW I have always strongly discouraged doing big uploads with josm as
it always dropped them on the floor on network errors in a way that
doesn't let you recover and continue from where it broke, being one of
the main causes of people needing to do reverts and spamming db
history with deleted nodes.  It may have gotten more clever recently,
not sure.

Cheers

___
Talk-us mailing list
Talk-us@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-us


Re: [Talk-us] [Imports] NHD data skipped by nhd2osm

2010-06-23 Thread Dylan Semler
On Tue, Jun 22, 2010 at 3:56 PM, Ian Dees ian.d...@gmail.com wrote:

 On Tue, Jun 22, 2010 at 2:36 PM, Dylan Semler dylan.sem...@gmail.comwrote:

 I've patched the nhd2osm scripts to print details about the NHD
 tag conversions[1].  I'm attaching the output of the scripts with my patch
 applied if anyone is interested in what it does.



 The shp-to-osm Java app I wrote has a more complete set of NHD conversion
 rules here:

 http://wiki.openstreetmap.org/wiki/NHD_Rules

 Information on the JAR is here:

 http://wiki.openstreetmap.org/wiki/Shp-to-osm.jar


Are the nhd2osm scripts still being maintained?  If so, I'll look into
adding those rules to the nhd2osm scripts.
___
Talk-us mailing list
Talk-us@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-us


Re: [Talk-us] [Imports] NHD data skipped by nhd2osm

2010-06-23 Thread James U
I've preferred Ian's java app to shp-to-osm.  

I just want to put a plug in for a workflow to potential NHD importers that I 
have arrived at after many previous mistakes.  

1.  Convert nhd shp for a subbasin files using shp-to-osm with max-nodes set to 
a very high number.  This makes one big file per nhd file (waterbody, area, 
flowline, point, line).

2.  Merge all the files in josm.  Make sure you have allocated a lot of memory 
to josm as this can take a lot of memory.  It will also take a lot of time.


3.  Use validator to identify duplicate nodes.  There are two main sources of 
duplicate nodes:
  a.  Flowline intersections--NHD has each reach as a separate vector.  
These should all be merged.
  b.  Flowline intersections with riverbanks and waterbodies.  These, too 
should be merged.
  c.  Shared boundaries between lakes, wetlands, riverbanks, dams and 
other features.  I think there is some dispute what to do here.  If there are 
two multipolygon relations involved, one shared way should be created that is 
a member of both relations.  If there aren't two relations, I think one could 
either create two or simple merge nodes.  I have been merging nodes.  

4. Quality control.  Keep in mind several things about the NHD.  They are not 
perfect and they overlap between basins.  Try your best to keep the import 
from duplicating others' and your own work.   Using the slippy map plugin, 
double check that features in the import do not already exist.  They could 
either be NHD data, which can be deleted from the new import, or another form 
of data--either an import such as PGS or a real human.  Use judgment to 
determine which is the best.  Based on up-to-date orthophotography I can tell 
that NHD is not always the best data.  However, many water features input by 
hand are not always better either-- they are frequently hastily sketched off of 
Yahoo pictures and not super accurate.

If you do multiple subbasins in the same basin, you will quickly notice that 
there is a ton of overlap in both the waterbodies and the riverbanks between 
different subbasins.

5.  Upload the entire subbasin using josm's ability to split large changesets.  
If your connection is at all flaky, you may want josm to split into smaller 
changesets than the maximum (5 changes).  

Sorry for the book, but I wanted to share some of my experiences.

James

On Tuesday, June 22, 2010 03:56:16 pm Ian Dees wrote:
 On Tue, Jun 22, 2010 at 2:36 PM, Dylan Semler dylan.sem...@gmail.comwrote:
  I've patched the nhd2osm scripts to print details about the NHD
  tag conversions[1].  I'm attaching the output of the scripts with my
  patch applied if anyone is interested in what it does.
 
 The shp-to-osm Java app I wrote has a more complete set of NHD conversion
 rules here:
 
 http://wiki.openstreetmap.org/wiki/NHD_Rules
 
 Information on the JAR is here:
 
 http://wiki.openstreetmap.org/wiki/Shp-to-osm.jar

___
Talk-us mailing list
Talk-us@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-us


Re: [Talk-us] [Imports] NHD data skipped by nhd2osm

2010-06-22 Thread Ian Dees
On Tue, Jun 22, 2010 at 2:36 PM, Dylan Semler dylan.sem...@gmail.comwrote:

 I've patched the nhd2osm scripts to print details about the NHD
 tag conversions[1].  I'm attaching the output of the scripts with my patch
 applied if anyone is interested in what it does.



The shp-to-osm Java app I wrote has a more complete set of NHD conversion
rules here:

http://wiki.openstreetmap.org/wiki/NHD_Rules

Information on the JAR is here:

http://wiki.openstreetmap.org/wiki/Shp-to-osm.jar
___
Talk-us mailing list
Talk-us@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-us