Re: [OSM-talk] GeoBase nodes import
Sam Vekemans wrote: I wanted to float the idea (to the general talk list) about importing the full data base that is available, not as shapes/ways/lines, but as nodes which show what map features the node represents. So for example, importing a park (in a mapped area) we would just show the outline dots of the park, and the user can connect the dots and show it the 'right' way osm-style. Huh?!? So nothing from the Geobase import would actually appear on the map until somebody had gone and traced over it manually? Have I actually understood that correctly? Only it sounds bonkers to me... Tom -- Tom Hughes (t...@compton.nu) http://www.compton.nu/ ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] GeoBase nodes import
Thanks :)Bonkers? ... perhaps :@) Actually, that would be the just the 1st step. meaning that; On those mapped areas where say - if the actual shapes were imported, you would see 'ghost lines'. .. this is not preferred. So, by just seeing the reference points, then these shapes can be drawn in by hand (where needed). Then, on the areas where it has been just OSM roads done, the rest of the import can happen, so the connect-the-dots would just be for the roads. Going this way, it would avoid the number 1 objection to the import, which is importing data showing up as ghost lines, because the change-detect script wasn't able to detect that someone already drew in the map feature. and going this way, would also solve the problem of how to deal with updates. they would get imported as nodes instead of ways. (so not to interfere with any mappers work) So far, in discussions, what came up was the importance of not undermining the work of mappers who have spent the last couple years trying to make the map an accurate representation of the real world. (doing it this way, it would give the command back to the OSM mappers, rather than 1 big mapper going onto openstreetmap and trampling over everything ... it's like a newbie, messing up on potlatch or JOSM... but 10,000 times worse) So far, it looks like the majority of the country would be able to accept a full import of everything... except the roads. .. it's only 10 or so tiles that nothing would be imported (just nodes), and less than 100 where only roads would be omitted. (there are 12,539 tiles in Canada) (.5x.25 degrees) - something like that... the actual breakdown will (should) be posted on the wiki. I hope that makes sense, Thanks for the input :) Sam Vekemans Across Canada Trails On Thu, Dec 18, 2008 at 12:34 AM, Tom Hughes t...@compton.nu wrote: Sam Vekemans wrote: I wanted to float the idea (to the general talk list) about importing the full data base that is available, not as shapes/ways/lines, but as nodes which show what map features the node represents. So for example, importing a park (in a mapped area) we would just show the outline dots of the park, and the user can connect the dots and show it the 'right' way osm-style. Huh?!? So nothing from the Geobase import would actually appear on the map until somebody had gone and traced over it manually? Have I actually understood that correctly? Only it sounds bonkers to me... Tom -- Tom Hughes (t...@compton.nu) http://www.compton.nu/ ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] GeoBase nodes import
Sam Vekemans wrote: Bonkers? ... perhaps :@) Well if I've understood correctly then what you're suggesting is that we add what amounts to presumably some tens of millions of nodes each of which would presumably have a large number of highly duplicative tags. If that is what you're saying then I stand by my comment that it seems completely and utterly mad. Actually, that would be the just the 1st step. meaning that; On those mapped areas where say - if the actual shapes were imported, you would see 'ghost lines'. .. this is not preferred. So, by just seeing the reference points, then these shapes can be drawn in by hand (where needed). If everything is going to be drawn by hand then why bother importing stuff at all? Why not just create a tool that can generate an OSM file for an area from Geobase and then let people generate extracts for their area and load them into JOSM to do the hand addition of the lines before uploading? Then, on the areas where it has been just OSM roads done, the rest of the import can happen, so the connect-the-dots would just be for the roads. Ah right, so you're just suggesting this for everything, just for some subsets of the data in each area? How will you decide which features to import fully and which to import partially in each area? Going this way, it would avoid the number 1 objection to the import, which is importing data showing up as ghost lines, because the change-detect script wasn't able to detect that someone already drew in the map feature. What are ghost lines? What is this change-detect script and what is it going to be doing? and going this way, would also solve the problem of how to deal with updates. they would get imported as nodes instead of ways. (so not to interfere with any mappers work) To be honest I think you're probably living in a dream world if you think there is any real chance of being apply to apply updates from the base data automatically. So far, it looks like the majority of the country would be able to accept a full import of everything... except the roads. .. it's only 10 or so tiles that nothing would be imported (just nodes), and less than 100 where only roads would be omitted. (there are 12,539 tiles in Canada) (.5x.25 degrees) - something like that... the actual breakdown will (should) be posted on the wiki. So this is mostly only about the roads, but in a lot of ways people would probably consider them the most important feature, so if we don't get any roads imported without them having to be traced over then we've only gained a fairly limited amount. To be honest most of this is obviously up to you guys as you're the ones that will have to work with it. My main concern is what impact whatever scheme you choose will have on the server infrastructure and data storage requirements, which is why I have concerns about creating tens of millions of nodes which have many duplicated tags or which will wind up never being used and will just be deleted again. Tom -- Tom Hughes (t...@compton.nu) http://www.compton.nu/ ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] GeoBase nodes import
2008/12/18 Tom Hughes t...@compton.nu So far, it looks like the majority of the country would be able to accept a full import of everything... except the roads. .. it's only 10 or so tiles that nothing would be imported (just nodes), and less than 100 where only roads would be omitted. (there are 12,539 tiles in Canada) (.5x.25 degrees) - something like that... the actual breakdown will (should) be posted on the wiki. So this is mostly only about the roads, but in a lot of ways people would probably consider them the most important feature, so if we don't get any roads imported without them having to be traced over then we've only gained a fairly limited amount. To be honest most of this is obviously up to you guys as you're the ones that will have to work with it. My main concern is what impact whatever scheme you choose will have on the server infrastructure and data storage requirements, which is why I have concerns about creating tens of millions of nodes which have many duplicated tags or which will wind up never being used and will just be deleted again. If it's only 10 tiles of 12539, why bother with importing anything at all for those 10 tiles automatically and instead have those areas manually merged in? For the 100 where roads would be omitted, surely again a manual merge of that road data for those 100 would be the best solution... Then, as Tom says, you only have to share the OSM files for the data to the people who are taking responsibility for massaging the data in rather than putting in lots of nodes with all the way names attached, which appart from using up lots of space, would be a pain in the bum to connect up and in the event that roads are already there, would you want a stack of unconnected nodes all over the place that need someone to manually hunt down and delete? I could see having a copy object down to lower layer option in JOSM would be very useful for the manual merging... I know that you can already copy between layers, but, I think it's either merge the entire layer into the lower one or copy an object to another layer but not preserving it's position... d ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] GeoBase nodes import
Thanks Tom D, storage requirements, which is why I have concerns about creating tens of millions of nodes which have many duplicated tags or which will wind up never being used and will just be deleted again. If it's only 10 tiles of 12539, why bother with importing anything at all for those 10 tiles automatically and instead have those areas manually merged in? What about just importing the nodes for these 10 tiles? This way, the mappers in these areas, can go down to the next level of detail, and add in more features like building outlines, grass areas, things that were missed. .. but having them as nodes so they don't mess up the map. For the 100 where roads would be omitted, surely again a manual merge of that road data for those 100 would be the best solution... Then, as Tom says, you only have to share the OSM files for the data to the people who are taking responsibility for massaging the data in rather than putting in This happened for the AND import... but for Canada, i think we'd like something better. (if possible) And so, im not sure that the local are mappers would want to have a manual merge (wiping out what they did, importing all the roads, then slowely bringing back the OSM roads if any are needed) By having the roads which were not mapped available as nodes to be traced, we are not wiping out what the mapper did. The mapper takes priority :) And yes, for the 10-100 tiles, there is no need to import just the nodes for everything else... as this everything else will be imported as shape/line files. ... and so having just the nodes for the roads to be traced would make sense here. And for the 100 - 12,539, your both right. there would be no need to have a big Canada node-dump. For the updates- These are available as a much smaller file size, so having the nodes available, so as mappers see them, they can be manually flipped to an OSM map feature. (flipped=most of the GeoBase reference removed, accept the created by:tag) For nodes on nodes- i think this would only be in the few cases where the OSM mapper was identical to the import. Thanks for the help in re-defining what the idea is :) Now just to translate it for the wiki (but i think i'll wait a day or so for more feed-back from others. Hopefully this makes a little more sense, :) Sam Vekemans Across Canada Trails ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] GeoBase nodes import
Doesn't get my vote. If it's a feature already why would we want to have to recreate it manually. Also by just importing nodes how can we visually/easily tell what nodes relate to what feature, especially if features overlap or cross. I can understand the difficulty in adding new data into already mapped areas, but if this import process is done by tile for instance then the local user can specifically ask for the import and can work on clean-up tile by tile (or other logical area) or decide that the data for that area is already good enough and doesn't need the addition of the geobase data. Just because we have another source of data doesn't mean to say that our data isn't as good or better for specific locations. Cheers Andy -Original Message- From: talk-boun...@openstreetmap.org [mailto:talk- boun...@openstreetmap.org] On Behalf Of Sam Vekemans Sent: 17 December 2008 8:09 PM To: talk@openstreetmap.org; talk...@openstreetmap.org Subject: [OSM-talk] GeoBase nodes import Hi all, On the talk-ca list we are comparing the differences similarities between this project the TIGER import. I wanted to float the idea (to the general talk list) about importing the full data base that is available, not as shapes/ways/lines, but as nodes which show what map features the node represents. So for example, importing a park (in a mapped area) we would just show the outline dots of the park, and the user can connect the dots and show it the 'right' way osm-style. This would ensure that accuracy is there as well as alow updates as these updates would be directly ontop of the old one. These imported nodes would be then used as reference (like colour coded snap points) Is there a way that josm is able to allow the user to click on the space where the node is and select which node they want to use to attach the line/area/relation they are working with? Is there a way to make sure that these nodes dont get rendered? In the case of address range, what if these lines are shown as 1 side of the property outline? Thanks, Sam Vekemans Across Canada Trails ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk No virus found in this incoming message. Checked by AVG - http://www.avg.com Version: 8.0.176 / Virus Database: 270.9.19/1854 - Release Date: 17/12/2008 7:21 PM ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] GeoBase nodes import
2008/12/17 Sam Vekemans acrosscanadatra...@gmail.com: So for example, importing a park (in a mapped area) we would just show the outline dots of the park, and the user can connect the dots and show it the 'right' way osm-style. So we would discard the knowledge of the sequence in which the nodes should be added to the ways? This seems like a bad idea. I've encountered irregular shapes before where having the nodes does not show clearly in which order to connect them. And that's leaving aside the fact that there's an awful lot of work involved in this. Far better to push forward with some of the JOSM enhancements proposed to allow easier management (through hiding) of clutter. Doing this, you could happily namespace the imported ways and users not actively involved with the de-duping task could simply hide them. Dermot -- -- Iren sind menschlich ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] GeoBase nodes import
Hi, Sam Vekemans wrote: And so, im not sure that the local are mappers would want to have a manual merge (wiping out what they did, importing all the roads, then slowely bringing back the OSM roads if any are needed) By having the roads which were not mapped available as nodes to be traced, we are not wiping out what the mapper did. There are many other options you could consider. For example, we have just received data for all tertiary and bigger roads for one of the largest German Länder, North Rhine-Westphalia. We have already got a lot of mapping done in that area; my guess is that of the 20.000 km of roads they have given us, probably only about 10% are actually still missing in OSM, and another 20-30% can be very effectively used to improve existing geometry. (The import is described, in German, on http://wiki.openstreetmap.org/wiki/Import/Catalogue/Strassen_NRW). This is not a high-profile project like the GeoBase import is, so we wanted to do something with little effort. We settled on loading all data into a WMS server which can be accessed through the OSM Inspector tool (http://tools.geofabrik.de/osmi/?view=strassennrw) or directly as a background layer in JOSM so if there are small corrections or omissions, people can just trace them off the WMS in JOSM. (It would also be relatively easy to provide a tile server with these data in case one wanted to have it in Potlatch.) In addition, for those cases where somebody spots a larger item that is missing in our data, we have provided simple .osm files for every section of road, and these can be downloaded individually. So if you spot a certain road segment on the map and find it's not in OSM yet, you can simply download the corresponding .osm file and merge it into JOSM. The OSM inspector view will also directly give you the URL for the download if you click on an item. This was relatively easy to set up (partly because we're dealing with roads which are fairly uniform compared to a whole set of different features). It would probably not be too difficult to enhance JOSM to auto-download such an .osm snippet on request (i.e. click on something in the inspector view, then click add to my running JOSM session or so), or to change the whole system to use data tiles instead of road fragments or so. The whole thing is designed to be computer assisted manual import; there's no automatic import here because our map is simply not empty enough for that. - If the situation in Canada is similar then you might consider something along these lines. If you have many empty spots then maybe you need more automatism though. Bye Frederik ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] GeoBase nodes import
Hi, Sam Vekemans wrote: So for example, importing a park (in a mapped area) we would just show the outline dots of the park, and the user can connect the dots and show it the 'right' way osm-style. Sounds very esoteric, and it will also create a huge inflation of node tags (something that has been done with TIGER as well - TIGER comprises 80% of OSM data volume but that's not beause the US is so big, just because every node carries a kilobyte of tags...). Is there a way that josm is able to allow the user to click on the space where the node is and select which node they want to use to attach the line/area/relation they are working with? No, but I'm sure it could be arranged. Is there a way to make sure that these nodes dont get rendered? Most renderers don't render stuff they do not understand, so you'd just have to make sure your tags are out of this world ;-) Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] GeoBase nodes import
Thanks, Esoteric: i had to wiki that one :) Anyway, i dont think anyone has a problem with having Canada take the number 1 spot for nodes :) would just mean that planet file gets bigger. If it meant that importing just the nodes (with extra unique ref tags) over existing well mapped areas, (so not compromizing other user data) and at the same time makes it easy for the any of the few agencies to use the data (if they want) - keeping it a 2-way street. -would it be worth it? - i think it would :) I got word back that geobase is held in shapes/lines data sets. -where it is technically possable to break it down to nodes, just means the files get bigger. Fortunatly we have no limit on file size, just would need to get a bigger server back-bone, a university donated one? Perhaps its possable to have it as part of the import script? Like the land mass import, without the lines connecting it? And josm having the option to select different nodes which are ontop of eachother? Good to hear its technically possable. Cheers, Sam Vekemans Across Canada Trails On 12/17/08, Frederik Ramm frede...@remote.org wrote: Hi, Sam Vekemans wrote: So for example, importing a park (in a mapped area) we would just show the outline dots of the park, and the user can connect the dots and show it the 'right' way osm-style. Sounds very esoteric, and it will also create a huge inflation of node tags (something that has been done with TIGER as well - TIGER comprises 80% of OSM data volume but that's not beause the US is so big, just because every node carries a kilobyte of tags...). Is there a way that josm is able to allow the user to click on the space where the node is and select which node they want to use to attach the line/area/relation they are working with? No, but I'm sure it could be arranged. Is there a way to make sure that these nodes dont get rendered? Most renderers don't render stuff they do not understand, so you'd just have to make sure your tags are out of this ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
