[Talk-us] Tiger 2007 Data
Has anyone looked at importing the TIGER 2007 data yet? I was going to start coding up the conversion utilities to get started. It appears that this shapefile format may have existing OSM converters out there. Anyone want to admit to having one? ;) -- Dave ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Tiger 2007 Data
On Sun, Oct 19, 2008 at 10:54 AM, Dave Hansen <[EMAIL PROTECTED]> wrote: > Has anyone looked at importing the TIGER 2007 data yet? I was going to > start coding up the conversion utilities to get started. It appears > that this shapefile format may have existing OSM converters out there. > Anyone want to admit to having one? ;) > > -- Dave > What were you looking to import? I thought your TIGER import was a one-shot deal. Karl ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Tiger 2007 Data
On Sun, 2008-10-19 at 11:10 -0700, Karl Newman wrote: > On Sun, Oct 19, 2008 at 10:54 AM, Dave Hansen <[EMAIL PROTECTED]> wrote: > Has anyone looked at importing the TIGER 2007 data yet? I was > going to > start coding up the conversion utilities to get started. It > appears > that this shapefile format may have existing OSM converters > out there. > Anyone want to admit to having one? ;) > > -- Dave > > What were you looking to import? I thought your TIGER import was a > one-shot deal. I *wish* it was a 1-shot deal. That darn Steve C. seems to think that maps should be both accurate *and* up to date. The nerve! Anyway, it seems that the new data is substantially better than the old in some places. What I'd like to do is twofold: 1. regenerate .osm files for all the new TIGER data 2. Compare new .osm files to old TIGER data plus stuff edited in OSM Then, decide how if/when it is appropriate to write over the old TIGER stuff with new. Or, to merge it somehow. -- Dave ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Tiger 2007 Data
On Sun, Oct 19, 2008 at 11:14 AM, Dave Hansen <[EMAIL PROTECTED]> wrote: > On Sun, 2008-10-19 at 11:10 -0700, Karl Newman wrote: > > On Sun, Oct 19, 2008 at 10:54 AM, Dave Hansen <[EMAIL PROTECTED]> wrote: > > Has anyone looked at importing the TIGER 2007 data yet? I was > > going to > > start coding up the conversion utilities to get started. It > > appears > > that this shapefile format may have existing OSM converters > > out there. > > Anyone want to admit to having one? ;) > > > > -- Dave > > > > What were you looking to import? I thought your TIGER import was a > > one-shot deal. > > I *wish* it was a 1-shot deal. That darn Steve C. seems to think that > maps should be both accurate *and* up to date. The nerve! > > Anyway, it seems that the new data is substantially better than the old > in some places. What I'd like to do is twofold: > > 1. regenerate .osm files for all the new TIGER data > 2. Compare new .osm files to old TIGER data plus stuff edited in OSM > > Then, decide how if/when it is appropriate to write over the old TIGER > stuff with new. Or, to merge it somehow. > > -- Dave > Ah. Quite a trick... I seem to recall that somebody had wanted to do that earlier, but I think they got discouraged by the difficulty. Karl ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Tiger 2007 Data
On Sun, 2008-10-19 at 11:25 -0700, Karl Newman wrote: > Ah. Quite a trick... I seem to recall that somebody had wanted to do > that earlier, but I think they got discouraged by the difficulty. Well, time may have made me forget what a pain the original import was. We'll see how far I get this time. :) -- Dave ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Tiger 2007 Data
Dave Hansen writes: > Then, decide how if/when it is appropriate to write over the old TIGER > stuff with new. Or, to merge it somehow. I've updated it in some cases. Perhaps only update if the data is unchecked? -- --my blog is athttp://blog.russnelson.com | Unregulation is a slippery Crynwr sells support for free software | PGPok | slope to prosperity and 521 Pleasant Valley Rd. | +1 315-323-1241 | freedom. Potsdam, NY 13676-3213 | Sheepdog | ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Tiger 2007 Data
> Then, decide how if/when it is appropriate to write over the old TIGER > stuff with new. Or, to merge it somehow. Be very, very careful here. Conflation is a difficult thing. I used to work at Tele Atlas, and there was a major project to conflate Tele Atlas North American data and GDT data (a company they just acquired). They had at least a hundred people committed full time to completing the task in a year (I don't know the exact number), with tens of millions in funding - and they failed in a big way. The head of the North American division got axed as a result. If there's a particular layer that has useful info, that's not already present - that could be imported successfully. Otherwise, the most useful thing would be some sort of set-up where someone could view changes to the new TIGER data simultaneously with the OSM data, and choose what to import. Or - if you can determine if a whole region was untouched - replace previously imported data with new data. What you don't want to mess up is the careful work people have done to improve the quality of data in a region manually. If you try to merge data automatically without careful oversight, you could destroy a lot of hard work. A one-time import - to create a starting point for people to edit - makes all the sense in the world. Subsequent updates are a much harder thing. -Alan ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Tiger 2007 Data
On Sun, 2008-10-19 at 11:33 -0700, Alan Brown wrote: > > Then, decide how if/when it is appropriate to write over the old > TIGER > > stuff with new. Or, to merge it somehow. > > Be very, very careful here. > > Conflation is a difficult thing. I used to work at Tele Atlas, and > there was a major project to conflate Tele Atlas North American data > and GDT data (a company they just acquired). They had at least a > hundred people committed full time to completing the task in a year (I > don't know the exact number), with tens of millions in funding - and > they failed in a big way. The head of the North American division got > axed as a result. It's not a difficult thing if nothing has been changed since the last import. My goal for now is simply to overwrite the things that haven't been updated by people since the last time I touched them. -- Dave ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Tiger 2007 Data
On Sun, 2008-10-19 at 14:29 -0400, Russ Nelson wrote: > Dave Hansen writes: > > Then, decide how if/when it is appropriate to write over the old TIGER > > stuff with new. Or, to merge it somehow. > > I've updated it in some cases. Perhaps only update if the data is > unchecked? Sure, that would be one reasonable thing. But, for now, I'm just trying to get an idea of the scale of the problem. That's going to be a challenge in and of itself. -- Dave ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Tiger 2007 Data
The improvement of tiger data accuracy I think was about 2/3 to 3/4 done in the 2007 set. They listed what had not been improved yet by county http://www.census.gov/geo/www/tiger/tgrshp2007/tgrshp07nomtaip.txt So skipping those this time for road data seems probable. I would guess another big step in eliminating places to worry about would be comparing the 2007 data to both the original import and the current OSM If the old tiger and current OSM match, but the 2007 data differs, then replace with the 2007 data. That is going to be a lot of spatial and meta tag comparisons though. The big idea being trying to limit data that needs to be uploaded and merged. I don't recall reading that anyone got beyond the conceptual stages of another Tiger import. I believe they changed from "tiger" format to shape files? So the shape to osm converters as well as your original script seem a place to start. I did some playing around with the MassToOSM python script. For what I used and where I go to see http://www.nabble.com/My-shape-file-import-steps-so-far-td19267077.html My intention was to get county level files and work from that one county at a time. Where I got hung up was I didn't know how to break up a county wide file in order to better work on it. There was also talk about importing better hydro data, but I'm not sure where that got to. If processing load instead of uploading to OSM is an issue, perhaps do your setup with the idea of maiking it into a VM. Have a master system tracking the jobs, perhaps uploading results back to it for import to OSM. On Sun, Oct 19, 2008 at 2:49 PM, Dave Hansen <[EMAIL PROTECTED]> wrote: > On Sun, 2008-10-19 at 14:29 -0400, Russ Nelson wrote: > > Dave Hansen writes: > > > Then, decide how if/when it is appropriate to write over the old TIGER > > > stuff with new. Or, to merge it somehow. > > > > I've updated it in some cases. Perhaps only update if the data is > > unchecked? > > Sure, that would be one reasonable thing. > > But, for now, I'm just trying to get an idea of the scale of the > problem. That's going to be a challenge in and of itself. > > -- Dave > > > ___ > Talk-us mailing list > Talk-us@openstreetmap.org > http://lists.openstreetmap.org/listinfo/talk-us > -- Dale Puch ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
