Hi
Would it be possible to get the geobaseOrthoimage imagery up as and alternative
to the Yahoo! Ariel imagery in Potlatch? In many rural areas the 10 metre
imagery has significantly better resolution.
Sam
________________________________
From: "talk-ca-requ...@openstreetmap.org" <talk-ca-requ...@openstreetmap.org>
To: talk-ca@openstreetmap.org
Sent: Monday, February 16, 2009 8:30:57 PM
Subject: Talk-ca Digest, Vol 12, Issue 10
Send Talk-ca mailing list submissions to
talk-ca@openstreetmap.org
To subscribe or unsubscribe via the World Wide Web, visit
http://lists.openstreetmap.org/listinfo/talk-ca
or, via email, send a message with subject or body 'help' to
talk-ca-requ...@openstreetmap.org
You can reach the person managing the list at
talk-ca-ow...@openstreetmap.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Talk-ca digest..."
Today's Topics:
1. CanVec /geobase NIDs (Sam Vekemans)
2. Re: OSM Geobase import: giving a try (Frank Steggink)
3. OSM GeoBase import: giving a try (Richard Degelder)
4. Re: OSM Geobase import: giving a try (Steve Singer)
5. Re: GeoBase Wiki edits (Steve Singer)
6. Re: GeoBase Wiki edits (Sam Vekemans)
7. Automatch for importing NHD (Sam Vekemans)
8. Re: OSM Geobase import: giving a try (Frank Steggink)
----------------------------------------------------------------------
Message: 1
Date: Sun, 15 Feb 2009 13:44:06 -0800
From: Sam Vekemans <acrosscanadatra...@gmail.com>
Subject: [Talk-ca] CanVec /geobase NIDs
To: stegg...@steggink.org, talk-ca@openstreetmap.org
Message-ID:
<9dbbf3b20902151344k4741f45fk1b346be3a5f83...@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Hi,
welcome to the team! I look forward to seeing the progress on the 021L07 area.
Your questions were already answered, and thank you, as it gives me a
better idea of how to make the wiki more clear :)
re: nids
i brought up the issue again as it needs to be further explained on
the wiki, (with respect to CanVec also) as there is always 2 sides to
decisions. -notion of 'expectancy' needs to be adressed in 1 page or
less, and clearly understood.
I am more in favor of using the 'automatch' for ALL map features.
Does anyone know that the python script line that adds the OSM tags
from the Database file.
Hopefully someone can help?
Trial & error is the other method :)
cheers,
Sam
------------------------------
Message: 2
Date: Sun, 15 Feb 2009 18:05:48 -0500
From: Frank Steggink <stegg...@steggink.org>
Subject: Re: [Talk-ca] OSM Geobase import: giving a try
To: Steve Singer <ssinger...@sympatico.ca>
Cc: talk-ca@openstreetmap.org
Message-ID: <49989fcc....@steggink.org>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Hi Steve,
Thanks for your response.
> Yes the wiki needs a cleanup, I've been hoping someone else would do it
> since that doesn't seem to be happening I will try to find sometime soon
> to delete all obsolete information from the wiki pages.
I'll try to do some cleanup, once I have a better understanding of th eefforts
of others.
> Yes please try to keep nids for data that actually comes from geobase.
> Not importing could limit us in the future.
In what way would it limit us? When we'll receive a new dataset from Geobase?
Or
do you hint towards other datasets which are linked to the NID? In that case
that additional data can't be linked to existing database, because that doesn't
contain the NID attributes.
>> b. How can we guarantee that the final import will be consistent?
>> (See also
>> my first question.)
>
> Depends what you mean by consitent.
> 1. Using consistent tags for things, the best way to ensure this is to
> look at what others are doing and share your scripts.
>
> 2. data consistency, ie roads line up and are joined between different
> imports or between the existing and geobase data. Right now we have no
> automated consistency tools, we are depending on people to manually line
> up/connect ways post import.
Both meanings were intended. I wonder if automatic consistency will work. It
seems to have a too high chance of failure.
> You are free to use your own processes and software. I don't think any
> two people are doing the exact same thing for importing. This is the
> process I follow
>
> You don't need to use roadmatcher, you just need to have some method of
> avoiding hundreds of duplicate roads.
I actually wanted to use PostgreSQL/PostGIS, but it seems that no OSM data can
be exported from it. At least not when OSM data was previously imported through
osm2pgsql. Maybe it will work to keep the Geobase data in Postgis, export as
OSM, and then apply some postprocessing (fixing in JOSM).
> For comparision purposes what I've done with the larger areas in Alberta:
>
> 1. Populate a postgis database with the NRN shapefile
> 2. Populate a postgis database with the OSM data using osm2pgsql
> 3. Generate a NRN and OSM shapefile for the area that I'm interested in
> using pgsql2shp and some SQL queries. I handle reprojection during this
> process as well (lately I've been reprojecting into meters)
> 4. Import both shapefiles into jump and automatch with roadmatcher
> 5. Export my results
> 6. Run geobase2osm on the NRN GML file(yes I had to download the NRN data
> twice) and produce a standalone file
> 7. Review the standalone file in JOSM
> 8. Upload the standalone file with bulk_upload.pl
> 9. Manual fixup in JOSM.
I'm going to try to generate buffers around the imported road data from OSM,
and
will use it to clip away any Geobase data (NRN) which is entirely within a
certain distance from the OSM data. And then export it to OSM. I'll need to
work
it out further, using my test area. I think that a buffer of 10 or 20 meter is
enough.
> Do not use any of the Ibycus stuff the import the licenses are not
> compatible. Do not use ibycus in the geobase import. Someone should
> have removed these references from the wiki a long time ago. I will try
> to do it soon.
OK. Makes sense to me. It's already postprocessed to some degree, rendering it
less suitable.
> I just viewed/posted my initial .osm files in JOSM and when people where
> happy with them I did an import to a small isolated area. Using a
> seperate userid is a good idea.
>
> You can get the latest version of geobase2osm.py here, a number of
> people have used this as the basis for their custom procedures.
>
> http://svn.openstreetmap.org/applications/utils/import/geobase2osm/geobase2osm.py
>
OK, will give that a try :) But it might be necessary if I use PostGIS as the
main repository for the Geobase data. I want to remove the number of steps as
much as possible, to make it more automatable and user friendly.
One final question: what is the accuracy of the Geobase data? Is it worse,
comparable, or better than typical GPS tracks? The roads I've drawn so far
match
the Yahoo imagery quite well, although usually there is a slight offset of a
few
meters.
Regards,
Frank
------------------------------
Message: 3
Date: Sun, 15 Feb 2009 20:13:42 -0500
From: Richard Degelder <rtdegel...@gmail.com>
Subject: [Talk-ca] OSM GeoBase import: giving a try
To: Talk-CA OpenStreetMap <talk-ca@openstreetmap.org>
Message-ID: <1234746822.6475.43.ca...@nomad>
Content-Type: text/plain
Frank,
Welcome to the project.
> Yes please try to keep nids for data that actually comes from geobase.
> Not importing could limit us in the future.
In what way would it limit us? When we'll receive a new dataset from Geobase?
Or
do you hint towards other datasets which are linked to the NID? In that case
that additional data can't be linked to existing database, because that doesn't
contain the NID attributes.
The reason for keeping the NIDs intact is with any future updates and imports
coming
from GeoBase, and apparently GeoGratis and CanVac. For OSM itself this data is
meaningless, the renderers have no idea how to deal with it and so ignore it.
Other
imports are going to, unless they have the same NIDs, also not use the data.
GeoBase regularly does updates to the data sets and are looking to complete the
current
data sets for all provinces. Their reference value is the NID. As long as we
also ensure that the NID is present we are going to be able to easily
incorporate those
updates when they appear.
>> b. How can we guarantee that the final import will be consistent?
>> (See also
>> my first question.)
>
> Depends what you mean by consitent.
> 1. Using consistent tags for things, the best way to ensure this is to
> look at what others are doing and share your scripts.
>
> 2. data consistency, ie roads line up and are joined between different
> imports or between the existing and geobase data. Right now we have no
> automated consistency tools, we are depending on people to manually line
> up/connect ways post import.
Both meanings were intended. I wonder if automatic consistency will work. It
seems to have a too high chance of failure.
Canada is too big, and there is too much data to incorporate everything
manually. If we
automate as much as possible the more effectively we can incorporate the
available data
as quickly as possible. The problem is finding an effective way to automate as
much
as possible without having too many errors incorporated. Not everything that
we are going
to want is going to be available from GeoBase, and, in many areas where we have
a number
of mappers, we are also able to incorporate data before it is available from
GeoBase, or
even available to GeoBase.
What we are currently having to do is to have people actually look at the data
to clean
it up after an import. If we can eliminate that to a large degree, possibly
through some
form of automation, we can speed up the import process. But until we get to
that point
we are going to have to have manual intervention. To some degree, however,
there is
always going to be the times where someone will edit, and modify, data
currently within
OSM and that is part of the process.
One final question: what is the accuracy of the Geobase data? Is it worse,
comparable, or better than typical GPS tracks? The roads I've drawn so far
match
the Yahoo imagery quite well, although usually there is a slight offset of a
few
meters.
I have been looking at a lot of the data produced for me by Steve and comparing
it to
the Yahoo! imagery. I have also found an offset, although it seems to be
variable and
inconsistent. The GeoBase data comes from a number of sources, each with
their own
accuracy. Looking at the data it almost seems that the people that entered it
have a
real influence into the quality. I have found some minor curves marked with a
lot of
data points while in other cases a more significant bend is marked with only a
few. But
then the same occurs within OSM and can also be credited to the style of the
contributor.
As for the overall quality of the data from GeoBase I am happy to incorporate
it within
OSM, it is certainly better than a lot I have seen, including from GPS traces.
But it
does depend on the source of the data and the effort of OSM users, when setting
up for
GPS tracks, can get close enough to the overall quality to match much of it.
If you are
very careful when creating your GPS tracks, and have a good signal and good
equipment,
you are going to exceed the quality of some of the GeoBase data but in some
cases you are
only going to match it with your best efforts.
Have fun importing the data. And if you can develop a best practises that can
import the
effectively and accurately, hopefully without overwriting the current OSM data,
then
let others know so that we all can incorporate those techniques. We are
learning as we
go and most of us are very willing to learn a better method of being able to
import, and
use, the tremendous quantity of data that we have available to us with GeoBase.
We were fortunate to have access to the data and now we are going to have to
find a way
effectively assimilate it within OSM. There are going to be minor issues but
hopefully
we can manage them easily enough.
Richard Degelder
------------------------------
Message: 4
Date: Sun, 15 Feb 2009 20:19:52 -0500 (EST)
From: Steve Singer <ssinger...@sympatico.ca>
Subject: Re: [Talk-ca] OSM Geobase import: giving a try
To: Frank Steggink <stegg...@steggink.org>
Cc: talk-ca@openstreetmap.org
Message-ID: <blu0-smtp69df86541737e3343502aaac...@phx.gbl>
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
On Sun, 15 Feb 2009, Frank Steggink wrote:
> Hi Steve,
>
> In what way would it limit us? When we'll receive a new dataset from Geobase?
> Or do you hint towards other datasets which are linked to the NID? In that
> case that additional data can't be linked to existing database, because that
> doesn't contain the NID attributes.
Mostly when we receive geobase updates.
Consider what should happen when as geobase releases road name data for
provinces that have already been imported. We will just need a script that
finds the OSM way that is tagged with a particular geobase:uuid and add in
the name tags from the geobase update. If we don't have the uuid in the OSM
data we'd need to figure out some other method.
Also we think about what happens when a road crosses two of the artificial
tiles we dividing the import process across, if each road as a unique NRN
id you can easily excluded the nids already imported in the adjacent tile
from the current one.
I only reason to not include it would be to save a few bytes of space.
> I actually wanted to use PostgreSQL/PostGIS, but it seems that no OSM data
> can be exported from it. At least not when OSM data was previously imported
> through osm2pgsql. Maybe it will work to keep the Geobase data in Postgis,
> export as OSM, and then apply some postprocessing (fixing in JOSM).
I've been exporting OSM data from PostGIS without issues.
I've added some more detailed steps at
http://wiki.openstreetmap.org/wiki/Geobase_NRN_-_OSM_Map_Feature
I export the OSM data from PostGIS as a shapefile, I only use this as an
input to road matcher (I only need a few fields). I don't think the data
from osm2pgsql can be used to regenerate .osm files though.
> I'm going to try to generate buffers around the imported road data from OSM,
> and will use it to clip away any Geobase data (NRN) which is entirely within
> a certain distance from the OSM data. And then export it to OSM. I'll need to
> work it out further, using my test area. I think that a buffer of 10 or 20
> meter is enough.
I thought about using buffers initially but then I came across road matcher.
10-20 meters tends to work for OSM data from GPS traces but the roads from
low-res imagery (often of highways in rural areas) often can be off by 100
meters.
> One final question: what is the accuracy of the Geobase data? Is it worse,
> comparable, or better than typical GPS tracks? The roads I've drawn so far
> match the Yahoo imagery quite well, although usually there is a slight offset
> of a few meters.
The Geobase data comes from different sources. Some of it is from GPS,
other data is from Vector files and some from imagery. My guess is that all
of it is more accurate than the low-res pictures Yahoo provides in Northern
Alberta.
My guess is that most of the geobase GPS data is better/comparable to the
typical OSM mapper GPS data, the vector data doesn't seem to be as good (and
I've found instances in Ontario where the geobase vector data shows
roads/intersections that don't exist). The geobase documentation lists
accuracy/precision somewhere but I can't recall what they claim.
>
> Regards,
>
> Frank
>
Steve
------------------------------
Message: 5
Date: Sun, 15 Feb 2009 20:32:11 -0500 (EST)
From: Steve Singer <ssinger...@sympatico.ca>
Subject: Re: [Talk-ca] GeoBase Wiki edits
To: Sam Vekemans <acrosscanadatra...@gmail.com>
Cc: talk-ca@openstreetmap.org
Message-ID: <blu0-smtp56d0ed0fb9190a53cf8858ac...@phx.gbl>
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
On Sun, 15 Feb 2009, Sam Vekemans wrote:
> Hi, great job on editing the page, thanks :)
> Wondering if you think it's a good idea if i make a page [[CanVec]] - to
> clearly describe what it is,
> and then another page [[CanVec Import]] to show the status.
> .. then on the [[Canada]] page, show the status of the whole import.
I would leave CanVec_OSM_Map_Features page as a top-level page for the
CanVec data. That should describe what it is, if the status gets
complicated/complete enough you could move it onto a separate page but I
don't think we are at that stage yet.
> Perhaps a chart with CanVec/GeoBase/Other at the top, then The provinces
> along the side?
The main GeoBase import page lists 9 different datasets and we have 13
provinces/territories. I wiki chart with 9 columns seems like it will be a
bit hard to read. I don't think you can condense all of the 8 geobase
datasets to 1 column since the imports will happen independently.
> Cheers,
> Sam
>
Steve
------------------------------
Message: 6
Date: Sun, 15 Feb 2009 20:47:24 -0800
From: Sam Vekemans <acrosscanadatra...@gmail.com>
Subject: Re: [Talk-ca] GeoBase Wiki edits
To: Steve Singer <ssinger...@sympatico.ca>
Cc: talk-ca@openstreetmap.org
Message-ID:
<9dbbf3b20902152047s1f9fdc72m6953179901cd6...@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
That makes sence, then i guess its just the wording at the top of the
page (canvec map features) that needs to be fixed.
When i have my script in sharable form & get those charts done -it should help.
i'll add a 'who's working on it section' as that might make it more clear.
Cheers,
Sam
On 2/15/09, Steve Singer <ssinger...@sympatico.ca> wrote:
> On Sun, 15 Feb 2009, Sam Vekemans wrote:
>
>> Hi, great job on editing the page, thanks :)
>> Wondering if you think it's a good idea if i make a page [[CanVec]] - to
>> clearly describe what it is,
>> and then another page [[CanVec Import]] to show the status.
>> .. then on the [[Canada]] page, show the status of the whole import.
>
> I would leave CanVec_OSM_Map_Features page as a top-level page for the
> CanVec data. That should describe what it is, if the status gets
> complicated/complete enough you could move it onto a separate page but I
> don't think we are at that stage yet.
>
>> Perhaps a chart with CanVec/GeoBase/Other at the top, then The provinces
>> along the side?
>
> The main GeoBase import page lists 9 different datasets and we have 13
> provinces/territories. I wiki chart with 9 columns seems like it will be a
> bit hard to read. I don't think you can condense all of the 8 geobase
> datasets to 1 column since the imports will happen independently.
>
>
>
>> Cheers,
>> Sam
>>
>
> Steve
>
------------------------------
Message: 7
Date: Mon, 16 Feb 2009 18:18:56 -0800
From: Sam Vekemans <acrosscanadatra...@gmail.com>
Subject: [Talk-ca] Automatch for importing NHD
To: talk...@openstreetmap.org, Ian Dees <ian.d...@gmail.com>,
talk-ca@openstreetmap.org, Michel Gilbert <michc...@gmail.com>, Steve
Singer <ssinger...@sympatico.ca>
Message-ID:
<9dbbf3b20902161818u38da3cds5e911dd993164...@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Hi,
fyi, the automatch script for importing canada's NRN datasets is
available to copy from - hopefully it can help. (to deal with
duplicates)
I still havent figured out how to get the .txt file that converts the
database file to osm tags. -the shp2osm script.
Thanks,
Sam
ps hopefully the talk-ca can learn from the talk-us list.
------------------------------
Message: 8
Date: Mon, 16 Feb 2009 21:31:32 -0500
From: Frank Steggink <stegg...@steggink.org>
Subject: Re: [Talk-ca] OSM Geobase import: giving a try
To: talk-ca@openstreetmap.org
Message-ID: <499a2184.1080...@steggink.org>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Hi Richard, Sam, Steve,
Thanks for your feedback. I'll continue using PostGIS (good for my own skills
as
well), and look at the other suggestions you've made. I'll also leave the NIDs
on the data which I will eventually upload. They don't hurt, although there is
the theoretical possibility that due to subsequent edits they get shifted
(splitting and joining).
I'll also investigate PostGIS buffering a bit more. 10 to 20 meters should be
enough. In cases of roads which have been shifted 100 meters, they need better
investigation anyways. In case they turn out to be drawn from low-res imagery,
they can better be replaced by the Geobase data. Hopefully the buffering won't
cause data to be removed accidentally. For areas where the OSM density is quite
small (as in my test areas) it is no big deal to remove duplicate Geobase roads
manually. Better to be safe than sorry. How would RoadMatcher deal with a case
where two roads are aligned, but one of them has been shifted?
With regard of this I also think a manual step should always take place. (Maybe
even to evaluate discarded data.) We must only make sure that it covers those
parts which can't be done automatically, so the required amount of time spent
of
it is as small as possible. Our brains can still make judgments better than the
computer. I don't think anyone has very sophisticated algorithms which
eliminate
human input in the process ;)
I hope to really start with the Geobase import soon. I came across the Global
Administrative Areas (GADM) database, which is part of the BioGeo project
mentioned at the potential datasources. I've looked more closely at the data,
and exchanged some ideas about the import of this data as administrative
boundaries with the guys on #osm-nl. (Sorry, sometimes it's necessary to chat
in
my mother language ;)) I also inquired after use of the data by OSM, because of
their copyright (CC-BY-NC-SA 3.0), and because they aggregated data from
various
other sources.
Regards,
Frank
------------------------------
_______________________________________________
Talk-ca mailing list
Talk-ca@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-ca
End of Talk-ca Digest, Vol 12, Issue 10
***************************************
__________________________________________________________________
Looking for the perfect gift? Give the gift of Flickr!
http://www.flickr.com/gift/_______________________________________________
Talk-ca mailing list
Talk-ca@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-ca
