you are not a bozo, you have been a great help. the problems with the versions of openjump was apparent to me and that is why I hesitated to even attempt to use it. it is not clear at all from the webpages what version from what source does what. happy easter all, thanks, mike
On Fri, Mar 19, 2010 at 6:58 PM, Adam Dunn <dunna...@gmail.com> wrote: > Well, I'm a complete bozo! The whole open source philosophy is to always be > running the latest (stable) version of your software. After all, why not, > considering it's open source and all. That's precisely what I *wasn't* doing > with OpenJUMP. So it turns out that the latest version of openjump can > successfully read all the property tags that I need it to. My sincere > apologies for cluttering up this mailing list. > > I also realize that much of this Albanian mapping stuff belongs on a mailing > list set up for Albania, and not on the Canadian list, but this whole > process, and much of the software was made for/by Canada. (Besides, there's > no talk-al set up yet). > > Now I have to try and convert geobase2osm.py to work for the Albanian data. > You can be sure I'll be back with more inane questions! > > Adam > > On Fri, Mar 19, 2010 at 10:25 AM, Adam Dunn <dunna...@gmail.com> wrote: >> >> I opened up my shapefile in QGIS after it had passed through PostGIS, and >> it turns out that it does have all the tags. So it appears there is nothing >> technically wrong with my SQL functions. It is OpenJUMP that's not reading >> the tags properly. I'm thinking maybe OpenJUMP isn't reading the smallint or >> integer types properly. Does anyone here know more about programming/opening >> ESRI files in OpenJUMP? Do I have to cast int4 and int2 to some other type >> in order for OpenJUMP to read them? Anybody know how I would do this without >> getting casting errors? >> >> Adam >> >> On Fri, Mar 19, 2010 at 12:03 AM, jamesmikedup...@googlemail.com >> <jamesmikedup...@googlemail.com> wrote: >>> >>> >>> On Fri, Mar 19, 2010 at 2:46 AM, Adam Dunn <dunna...@gmail.com> wrote: >>>> >>>> The reason why it is functionally useless at this point is because the >>>> id/et_id is not being transferred through the process correctly. Perhaps an >>>> explanation of the process is in order: >>>> The SQL stage is basically for reprojection and selecting a bounding >>>> box. If you already had both the government and the OSM roads converted >>>> into >>>> a Shapefile format, and they both had the same projection, and you were >>>> willing to work on the entire country at the same time, then you wouldn't >>>> even need SQL. >>> >>> Here are here: >>> http://xhema.flossk.org:8080/mapdata/03/MapAction/tpginc/LL_Roads_OSM.shp >>> >>> Here are the osm data (from cloudmade) >>> http://xhema.flossk.org:8080/mapdata/03/MapAction/tpginc/LL_Roads_OSM.shp >>> >>>> You generally don't want to do the whole country at a time (trust me, >>>> it's not something you do in OpenJUMP), and you generally need to do some >>>> reprojection/format conversion, so we need SQL. What gets spit out of the >>>> SQL step is two files, OSM.shp and GOV.shp, and they each have geometry >>>> (where the roads are) and id numbers (internal numbers used by each). >>>> These two files get loaded into OpenJUMP Roadmatcher, where you find the >>>> matches, find the standalones, then output the result. This result file has >>>> a list of the id numbers for the roads that are standalone in the >>>> government >>>> database. >>>> Then (for Canada) geobase2osm.py will use the Roadmatcher result file as >>>> a mask to see which roads to copy from the government gml file over to an >>>> osm file. It does this by looking at the standalone id numbers, and then >>>> copying only the roads in the gml that have those ids. >>>> I kind of think of it like a transistor. A transistor takes some voltage >>>> input and connects it to the output, depending on whether the gate voltage >>>> is on or not. The input/output voltages can be large or small, but the gate >>>> voltage only needs to be tiny. The gate voltage itself doesn't get passed >>>> through to the output. Something like this: >>>> gate >>>> | >>>> input -- transistor -- output >>>> >>>> Equivalently, geobase2osm.py connects the Government.gml geometry to the >>>> Standalone.osm geometry, but only if the Standalone Result ID is on. The >>>> Result ID itself doesn't get passed through, it's only used as a gate mask. >>>> So we get something like: >>>> Result ID# >>>> | >>>> GovGeometry.gml -- geobase2osm.py -- StandaloneGeometry.osm >>>> >>>> Since I haven't yet gotten the ID numbers to pass through, we can't use >>>> them as a mask for geobase2osm.py, rendering it useless. We can't generate >>>> osm files that have no duplicates with what's already on OSM. >>> >>> I see, so the ids are the problem. I can assign them new ids. We can >>> rename the column? >>> >>>> >>>> Let's say you want to do things more manually. You could run the match >>>> process in Roadmatcher, then visually look and see what roads need to be >>>> copied over, then go over to JOSM and manually pick out those roads. This >>>> is >>>> silly, as you could do this faster by skipping Roadmatcher, and just >>>> matching by eye within JOSM. >>> >>> I see, well that is what i hoped roadmatcher would help with. >>>> >>>> Even if I do get ID numbers to transfer through, I don't know how >>>> portable geobase2osm.py will be to the Albanian data. What tools have other >>>> imports used? Maybe there's something else that uses a slightly different >>>> method? >>>> >>> >>> OK, well we need to look into this in detail. >>> I was working on buffering the roads to remove duplicate. >>> >>> http://osmopenlayers.blogspot.com/2010/03/comparison-of-maps-using-buffering.html >>> >>> http://osmopenlayers.blogspot.com/2010/03/maps-of-albania-comparison-of-roads.html >>> >>> Thanks >>> mike >>>> >>>> Adam >>>> >>>> On Thu, Mar 18, 2010 at 1:38 PM, jamesmikedup...@googlemail.com >>>> <jamesmikedup...@googlemail.com> wrote: >>>>> >>>>> The roads shp files are all the same,, that is true. >>>>> >>>>> >>>>> On Thu, Mar 18, 2010 at 9:19 PM, Adam Dunn <dunna...@gmail.com> wrote: >>>>>> >>>>>> So far, I've got the LL_Roads2 files imported in PostgreSQL, and I've >>>>>> written some functions to export them (modified from the NRN process on >>>>>> wiki), but it's not working properly. Using pgAdmin on Linux, I can see >>>>>> what >>>>>> data values there are, and their types: >>>>>> gid serial NOT NULL, >>>>>> id numeric, >>>>>> "type" character varying(50), >>>>>> category smallint, >>>>>> cat2 smallint, >>>>>> shape_leng numeric, >>>>>> "name" character varying(30), >>>>>> shape_le_1 numeric, >>>>>> et_id integer, >>>>>> the_geom geometry, >>>>>> >>>>>> When I export this to shp and open in OpenJUMP, I can see that some of >>>>>> the values aren't getting exported properly. gid doesn't seem to be >>>>>> exported >>>>>> at all. id, type, shape_leng, shape_le_1, and the_geom all seem to be >>>>>> fine. >>>>>> Unfortunately, category, cat2, and et_id are all coming up with the >>>>>> value 0, >>>>>> even though they had other values in the original dataset. Name I can't >>>>>> test, since it seems to be null in the original data. >>>>>> Here's the function I'm using: >>>>>> >>>>>> >>>>>> DROP TYPE albania_gov_data; >>>>>> CREATE TYPE albania_gov_data AS ( >>>>>> --Included all Albanian keys, since I don't know what they >>>>>> mean/which are important >>>>>> gid integer, >>>>>> the_geom geometry, >>>>>> id numeric, >>>>>> way_type text, >>>>>> category smallint, >>>>>> cat2 smallint, >>>>>> shape_leng numeric, >>>>>> way_name text, >>>>>> shape_le_1 numeric, >>>>>> et_id integer >>>>>> >>>>>> ); >>>>>> CREATE OR REPLACE FUNCTION select_albania_gov_roadtile(coordinates >>>>>> text) >>>>>> RETURNS SETOF albania_gov_data >>>>>> AS $$ >>>>>> SELECT gid,ST_Transform(the_geom,4326),id,"type", --might need >>>>>> 4191 rather than 4326 >>>>>> category,cat2,shape_leng,"name",shape_le_1,et_id FROM >>>>>> albania_roadseg >>>>>> WHERE >>>>>> ST_Intersects(ST_Transform(the_geom,4326), >>>>>> ST_GeomFromEWKT($1)) >>>>>> >>>>>> >>>>>> $$ >>>>>> LANGUAGE SQL; >>>>>> >>>>>> CREATE TYPE albania_osm_data AS ( >>>>>> way geometry, >>>>>> osm_id integer, >>>>>> name text >>>>>> ); >>>>>> >>>>>> CREATE OR REPLACE FUNCTION select_albania_osm_roadtile(coordinates >>>>>> text) >>>>>> RETURNS SETOF albania_osm_data >>>>>> AS $$ >>>>>> SELECT ST_Transform(way,4326),osm_id,substr(name,0,80) --might >>>>>> need 4191 rather than 4326 >>>>>> FROM planet_osm_line WHERE >>>>>> ST_Intersects(way,ST_GeomFromEWKT($1)) >>>>>> AND ((highway is not null) >>>>>> OR (railway is not null)) --Albania appears to have >>>>>> railways in gov_data >>>>>> $$ >>>>>> LANGUAGE SQL; >>>>>> >>>>>> >>>>>> Anybody got any ideas? >>>>> >>>>> But the shapes are working? >>>>> >>>>>> >>>>>> On the data itself: >>>>>> What is the difference between all the files on the ftp? There is >>>>>> LL_Roads, LL_Roads2, LL_Roads_OSM, but a quick check in OpenJUMP, and >>>>>> they >>>>>> all look the same. >>>>> >>>>> no difference >>>>> >>>>>> >>>>>> For my test, I exported an area around Gjirokastra >>>>>> [http://www.openstreetmap.org/index.html?minlat=40&minlon=20&maxlat=40.2&maxlon=20.2&box=yes&layers=B000FTF]. >>>>>> In this area, the id tag was only partially useful: In downtown >>>>>> Gjirokastra >>>>>> (the densely roaded area that is not currently visible on OSM, but is >>>>>> approx >>>>>> bounded by [http://www.openstreetmap.org/browse/way/30455602] and >>>>>> [http://www.openstreetmap.org/browse/way/36777258]) the id tag seemed to >>>>>> be >>>>>> missing for roads. The id tag was only available in the outer (rural?) >>>>>> areas. Shape_leng was mostly 0.0, except for about 20 roads at the south >>>>>> end >>>>>> of Gjirokastra (out of 2479 roads), in which case it matched up with >>>>>> shape_le_1. This data also includes Footpath as a type, along with the >>>>>> other >>>>>> types I mentioned in my last email. A listing of all the different >>>>>> ways/types/categories in this data would be nice to have. >>>>> >>>>> I can tag them afterwards. >>>>> I have a tag tool as well to do a report. >>>>> the most important thing is to get rid of the duplicates >>>>> >>>>> >>>>>> >>>>>> The data is missing a gml file. This is necessary to run the >>>>>> geobase2osm.py step. You could probably use ogr2ogr to do it, but you'd >>>>>> need >>>>>> proper id or et_id, whichever would work better as a unique identifier >>>>>> (probably not id, since it's not available in urban areas!). >>>>> >>>>> hmmm... >>>>>> >>>>>> I did get both the OSM data and the (I assume) government data into >>>>>> openjump roadmatcher, so it would be possible to roadmatch with what I >>>>>> have, >>>>>> but not very useful. >>>>> >>>>> why not? >>>>> thank you for your efforts to you irc? come to irc.freenode.net #flossk >>>>> or add me on skype : h4ck3rm1k3 >>>>> >>>>> thanks, >>>>> mike >>>>> >>>>>> >>>>>> Adam >>>>>> >>>>>> On Wed, Mar 17, 2010 at 2:47 PM, Adam Dunn <dunna...@gmail.com> wrote: >>>>>> > >>>>>> > Hi, I was kind of hoping someone more knowledgeable in GIS >>>>>> > programming would've responded first for the following reasons: >>>>>> > geobase2osm.py is an integral part of the whole roadmatching process >>>>>> > for >>>>>> > Canada, and it's a very Canadian-specific script (there are chunks of >>>>>> > code >>>>>> > that deal with naming of major trunks on a province-by-province basis >>>>>> > using >>>>>> > if-else programming). >>>>>> > >>>>>> > Also, I don't have much experience programming GIS stuff. For >>>>>> > example, the Canadian way of doing this uses EPSG 3348 for projection >>>>>> > [1]. I >>>>>> > did some Google searching, and it looks like you would want to use >>>>>> > EPSG 4191 >>>>>> > for the Albania area (see [2]), but 2462 might also the one you want >>>>>> > to use >>>>>> > ([3], although it looks like you get weird projected bounds with it). I >>>>>> > don't know why this reprojection is really even necessary. When you >>>>>> > look at >>>>>> > EPSG 3348 (the Canadian one), the projected bounds are really weird >>>>>> > there as >>>>>> > well, so maybe it has to be done just to match up with NRN. If the >>>>>> > Albania >>>>>> > dataset is already in Lat/Long and 4191 is in Lat/Long and OSM is in >>>>>> > Lat/Long, maybe you don't need to reproject at all? Someone with more >>>>>> > GIS >>>>>> > knowledge should know. >>>>>> > >>>>>> > You'll also want a script to automatically convert various >>>>>> > attributes in the shp file to tags that are used by OSM. For example, I >>>>>> > opened up the Roads_OSM.shp file in OpenJUMP (just the way it is, no >>>>>> > changes >>>>>> > or reprojections or anything), and I see that one road has the >>>>>> > following >>>>>> > properties: >>>>>> > ID: 175879.0 >>>>>> > Type: Well-Kept Gravel Road >>>>>> > Category: 2 >>>>>> > Cat2: 20 >>>>>> > Shape_Leng: 0.0 >>>>>> > Name: >>>>>> > Shape_Le_1: 774.851 >>>>>> > ET_ID: 167693 >>>>>> > >>>>>> > Most of these you probably won't care about, but Type: Well-Kept >>>>>> > Gravel Road should be converted to >>>>>> > surface=gravel; >>>>>> > and highway=unclassified or residential or track=* depending on what >>>>>> > the Category: 2 and Cat2: 20 mean. >>>>>> > >>>>>> > Another road has Type: Dwelling Area Road, which would probably be >>>>>> > highway=residential; >>>>>> > >>>>>> > I've also seen Type: Village Road, Type: Railway, Type: Seasonal >>>>>> > Road, and Type: National Asphalted Road. >>>>>> > >>>>>> > You can see where geobase2osm.py makes similar mappings for Canada, >>>>>> > by looking at lines 63 to 77 in geobase2osm.py [4]. >>>>>> > >>>>>> > None of the roads I sampled had names associated with them (even the >>>>>> > national highway, the name was blank). You'll probably have to match >>>>>> > up road >>>>>> > names to the roads using one of the other data sets? >>>>>> > >>>>>> > I would like to help out more, but I'm afraid I don't have the >>>>>> > experience. I could lend a hand in modifying the SQL tables and >>>>>> > geobase2osm.py, but I would be of limited help there, and I haven't a >>>>>> > clue >>>>>> > how you would get road names imported. You'll want more expertise help >>>>>> > in >>>>>> > getting the proper toolchain. >>>>>> > >>>>>> > [1] http://spatialreference.org/ref/epsg/3348/ >>>>>> > [2] http://spatialreference.org/ref/epsg/4191/ >>>>>> > [3] http://spatialreference.org/ref/epsg/2462/ >>>>>> > [4] >>>>>> > http://svn.openstreetmap.org/applications/utils/import/geobase2osm/geobase2osm.py >>>>>> > >>>>>> > Adam >>>>>> > >>>>>> > On Wed, Mar 17, 2010 at 12:31 PM, Sam Vekemans >>>>>> > <acrosscanadatra...@gmail.com> wrote: >>>>>> >> >>>>>> >> Hi Adam, >>>>>> >> James is looking to use RoadMatcher, i know its on the wiki, but >>>>>> >> that >>>>>> >> page needs to be fixed. >>>>>> >> Would you be able to explain it? >>>>>> >> >>>>>> >> Personally, i now favour tracing WMS, but for massive areas >>>>>> >> RoadMatcher is the way to go :-) >>>>>> >> >>>>>> >> Also, Robert (Bob) Shand from PEI is currently Itching to also >>>>>> >> learn >>>>>> >> how its done. >>>>>> >> >>>>>> >> Cheers, >>>>>> >> Sam >>>>>> >> >>>>>> >> ---------- Forwarded message ---------- >>>>>> >> From: Michael Barabanov <michael.baraba...@gmail.com> >>>>>> >> Date: Wed, 17 Mar 2010 12:13:39 -0700 >>>>>> >> Subject: Re: [Talk-ca] call for help, importing roads >>>>>> >> To: "jamesmikedup...@googlemail.com" >>>>>> >> <jamesmikedup...@googlemail.com> >>>>>> >> Cc: talk-ca@openstreetmap.org >>>>>> >> >>>>>> >> Sounds like you just need to convert SHP to OSM. Then you can >>>>>> >> upload from >>>>>> >> JOSM. There's a script here (though I haven't tried it): >>>>>> >> http://redmine.yellowbkpk.com/projects/list_files/geo >>>>>> >> >>>>>> >> On Mon, Mar 15, 2010 at 12:48 PM, jamesmikedup...@googlemail.com < >>>>>> >> jamesmikedup...@googlemail.com> wrote: >>>>>> >> >>>>>> >> > Hi, >>>>>> >> > I have been talking to Sam V. who mentioned that you guys are >>>>>> >> > experts in >>>>>> >> > openjump for road importing. >>>>>> >> > before I dig too deep into this myself, let me ask if anyone can >>>>>> >> > help me >>>>>> >> > with my current problem: >>>>>> >> > >>>>>> >> > I have these files from a public domain source, projected into >>>>>> >> > Lat/Lon >>>>>> >> > >>>>>> >> > http://xhema.flossk.org:8080/mapdata/03/MapAction/tpginc/ >>>>>> >> > >>>>>> >> > [image: [ ]] >>>>>> >> > LL_Roads_OSM.dbf<http://xhema.flossk.org:8080/mapdata/03/MapAction/tpginc/LL_Roads_OSM.dbf>05-Mar-2010 >>>>>> >> > 07:48 53M [image: [ >>>>>> >> > ]]LL_Roads_OSM.prj<http://xhema.flossk.org:8080/mapdata/03/MapAction/tpginc/LL_Roads_OSM.prj>05-Mar-2010 >>>>>> >> > 07:48 143 >>>>>> >> > [image: [ >>>>>> >> > ]]LL_Roads_OSM.shp<http://xhema.flossk.org:8080/mapdata/03/MapAction/tpginc/LL_Roads_OSM.shp>05-Mar-2010 >>>>>> >> > 07:48 >>>>>> >> > 35M [image: [ >>>>>> >> > ]]LL_Roads_OSM.shx<http://xhema.flossk.org:8080/mapdata/03/MapAction/tpginc/LL_Roads_OSM.shx>05-Mar-2010 >>>>>> >> > 07:48 >>>>>> >> > 1.5M >>>>>> >> > They are from http://tpginc.net/gis/albania/albania.php >>>>>> >> > >>>>>> >> > we would like to find the new roads that are not in OSM and >>>>>> >> > import them. >>>>>> >> > Can anyone help? can you tell me exactly what software to >>>>>> >> > install, I am a >>>>>> >> > bit confused my the many pages and broken links I found for jump. >>>>>> >> > >>>>>> >> > thanks, >>>>>> >> > mike >>>>>> >> > >>>>>> >> > _______________________________________________ >>>>>> >> > Talk-ca mailing list >>>>>> >> > Talk-ca@openstreetmap.org >>>>>> >> > http://lists.openstreetmap.org/listinfo/talk-ca >>>>>> >> > >>>>>> >> > >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> -- >>>>>> >> Twitter: @Acrosscanada >>>>>> >> Blog: http://Acrosscanadatrails.blogspot.com >>>>>> >> Facebook: http://www.facebook.com/sam.vekemans >>>>>> >> Skype: samvekemans >>>>>> >> OpenStreetMap IRC: http://irc.openstreetmap.org >>>>>> >> @Acrosscanadatrails >>>>>> > >>>>>> >>>>> >>>> >>> >> > > > _______________________________________________ > Talk-ca mailing list > Talk-ca@openstreetmap.org > http://lists.openstreetmap.org/listinfo/talk-ca > > _______________________________________________ Talk-ca mailing list Talk-ca@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-ca
