Re: [OSM-talk-be] CRAB Import Tool
On 2013-10-22 20:53, Kurt Roeckx wrote: On Mon, Oct 21, 2013 at 10:45:22PM +0200, Kurt Roeckx wrote: On Mon, Oct 21, 2013 at 10:06:03PM +0200, Kurt Roeckx wrote: I really see no good reason not to add those IDs at this point. I don't see the harm in them. I can only see them being useful. I would actually want to propose a different import strategy: - Add the CRAB IDs to all existing addresses in Flanders - Import the rest or large parts of CRAB in one big import So after feedback on this, I want to propose that instead of actually importing this that we provide the data that this import tool would generate in such a way that it's easy for people to take the data and import it themself, potentially after fixing things. This would make it easier to improve the import tool after getting feedback of what it generates wrong. If you could export to OSM format , that would be awesome. Like in the way Overpass does this. In pseudo: - get data from osm (assuming here , the data is partial, so lets say, everything with an 'addr' tag in your field of view.) , the same effect you have when exporting a certain key using overpass. - get data from crab, craft is as such (preparse it) to facilitate merging with osm data set. - Make the diff, but create an OSM compliant xml (with meta data, otherwise you won't be able to create a changeset from it) - open the changeset with JOSM, verify, correct, validate and push. So, truthfully, I think a tool like you envision is still interesting and the more we do, the better and less manual JOSM work to do. But we need to do chunks of it, we should do this for small area's.it's also easier to (later on) fix things that went wrong yet unnoticed, that way you don't have to deal with huge changesets finding that single node on page 450 (ever tried paging through changesets using the site ? ;-) . Even a perfect full import in one go would give us headaches later. It keeps things managable I think it's great you want to do this, I'm just not too positive about the success and it's not that I doubt your skills, it's that I doubt we'll be able to cover all exceptions that you usually run into in a decent timeframe.The problem is not so much the bulk of perfect tagged stuff , but the ones that need special treatment. It could turn out to be a bigger job than anticipated right now. Glenn ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
You could also make a csv file with the diffs and open that with the OpenData plugin in JOSM. (see my presentation at ESI on import VMM monitoring stations ) But of course that requires people to install this plugin. or you could add all changes in such a way that they are also added to the tool that Ben proposes. m On Wed, Oct 23, 2013 at 10:13 AM, Glenn Plas gl...@byte-consult.be wrote: On 2013-10-22 20:53, Kurt Roeckx wrote: On Mon, Oct 21, 2013 at 10:45:22PM +0200, Kurt Roeckx wrote: On Mon, Oct 21, 2013 at 10:06:03PM +0200, Kurt Roeckx wrote: I really see no good reason not to add those IDs at this point. I don't see the harm in them. I can only see them being useful. I would actually want to propose a different import strategy: - Add the CRAB IDs to all existing addresses in Flanders - Import the rest or large parts of CRAB in one big import So after feedback on this, I want to propose that instead of actually importing this that we provide the data that this import tool would generate in such a way that it's easy for people to take the data and import it themself, potentially after fixing things. This would make it easier to improve the import tool after getting feedback of what it generates wrong. If you could export to OSM format , that would be awesome. Like in the way Overpass does this. In pseudo: - get data from osm (assuming here , the data is partial, so lets say, everything with an 'addr' tag in your field of view.) , the same effect you have when exporting a certain key using overpass. - get data from crab, craft is as such (preparse it) to facilitate merging with osm data set. - Make the diff, but create an OSM compliant xml (with meta data, otherwise you won't be able to create a changeset from it) - open the changeset with JOSM, verify, correct, validate and push. So, truthfully, I think a tool like you envision is still interesting and the more we do, the better and less manual JOSM work to do. But we need to do chunks of it, we should do this for small area's.it's also easier to (later on) fix things that went wrong yet unnoticed, that way you don't have to deal with huge changesets finding that single node on page 450 (ever tried paging through changesets using the site ? ;-) . Even a perfect full import in one go would give us headaches later. It keeps things managable I think it's great you want to do this, I'm just not too positive about the success and it's not that I doubt your skills, it's that I doubt we'll be able to cover all exceptions that you usually run into in a decent timeframe.The problem is not so much the bulk of perfect tagged stuff , but the ones that need special treatment. It could turn out to be a bigger job than anticipated right now. Glenn __**_ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.**org/listinfo/talk-behttps://lists.openstreetmap.org/listinfo/talk-be ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
On 2013-10-23 10:28, Marc Gemis wrote: You could also make a csv file with the diffs and open that with the OpenData plugin in JOSM. (see my presentation at ESI on import VMM monitoring stations ) But of course that requires people to install this plugin. The great thing about having to go dig deep in the data itself is that it will be fairly easy to structure this the way JOSM does it. Since you already have to deal with different formats, why spend time on an extra one ? All you need to do is save 'the result' set locally and start looking at the format. It would be a huge feature. I'm not against csv's but their use is limited, xml's (not my favorite!) however gives a structure to the data and makes the data also more human readable. Next to being an established standard. or you could add all changes in such a way that they are also added to the tool that Ben proposes. I would just not invent a new output format to work with locally... Although using sqlite as a locale storage (whatever tool) would also help a lot speedwise. When using subsets of data instead of 'everthing from bounding box', the plus-side of working with a limited dataset is that the manual editing , selecting things 'in bulk' is a lot easier in a tool like JOSM, as such JOSM becomes your tool to manually process and verify it. A well known tool... So you don't need to reinvent the wheel (validation for example). Just glue JOSM right in. as an example, when Colruyt is taken over by Delhaize and you want to change the operator on all stores with name 'Colruyt', my steps would be: - Export using Overpass, in the query decide if you want to do nodes or ways (or both). - .osm file only containing the nodes I'm interested in (and metadata) - Validate right of the bat to get an idea of the current state - open in JOSM, search/replace using all features in JOSM (regex, case insensitive, key exact and non exact search) - Do this again for all typo's in common keys (and delete the crap) - Validate ( use errors to focus on certain keys ) - search for bad keys, stuff that doesn't belong on those nodes (always something lingering around) - Check addr:* keys , all of them. - Use Address plugins to complete missing information - Validate - Search for notes, comments and read them (they might give clues about problems you missed). Update them as you fix. - Validate - Prepare for merge problems (someone might have touched one of the nodes/ways) - Now Fix remaining validation errors until it's 100% clean (or are false) - Validate - Upload - Merge (if needed) - Upload Any tool that comes our way that supports OSM format kan be injected in any of those phases. That's awesome. I would be using a tool like that. Since it can take input from any source that you are able to bring to common ground, being OSM format. (XML). I would be using all tools that are suited and add value in such a way to meet a certain problem. If the tools that parse CRAB would use plugins to read from a(ny) datasource ... Then you might be creating something that lasts. I would love to try it out. Glenn m On Wed, Oct 23, 2013 at 10:13 AM, Glenn Plas gl...@byte-consult.be mailto:gl...@byte-consult.be wrote: On 2013-10-22 20:53, Kurt Roeckx wrote: On Mon, Oct 21, 2013 at 10:45:22PM +0200, Kurt Roeckx wrote: On Mon, Oct 21, 2013 at 10:06:03PM +0200, Kurt Roeckx wrote: I really see no good reason not to add those IDs at this point. I don't see the harm in them. I can only see them being useful. I would actually want to propose a different import strategy: - Add the CRAB IDs to all existing addresses in Flanders - Import the rest or large parts of CRAB in one big import So after feedback on this, I want to propose that instead of actually importing this that we provide the data that this import tool would generate in such a way that it's easy for people to take the data and import it themself, potentially after fixing things. This would make it easier to improve the import tool after getting feedback of what it generates wrong. If you could export to OSM format , that would be awesome. Like in the way Overpass does this. In pseudo: - get data from osm (assuming here , the data is partial, so lets say, everything with an 'addr' tag in your field of view.) , the same effect you have when exporting a certain key using overpass. - get data from crab, craft is as such (preparse it) to facilitate merging with osm data set. - Make the diff, but create an OSM compliant xml (with meta data, otherwise you won't be able to create a changeset from it) - open the changeset with JOSM, verify, correct, validate and push. So, truthfully, I think a tool like you envision is still
Re: [OSM-talk-be] CRAB Import Tool
as you pointed out, it will only work for point data. But that's what those addresses in Crab are. And indeed it was meant as a suggestion for Kurt. So he can choose from a number of possible workflows. m On Wed, Oct 23, 2013 at 1:43 PM, Glenn Plas gl...@byte-consult.be wrote: On 2013-10-23 12:43, Marc Gemis wrote: On Wed, Oct 23, 2013 at 12:16 PM, Glenn Plas gl...@byte-consult.bewrote: On 2013-10-23 10:28, Marc Gemis wrote: You could also make a csv file with the diffs and open that with the OpenData plugin in JOSM. (see my presentation at ESI on import VMM monitoring stations ) But of course that requires people to install this plugin. The great thing about having to go dig deep in the data itself is that it will be fairly easy to structure this the way JOSM does it. Since you already have to deal with different formats, why spend time on an extra one ? All you need to do is save 'the result' set locally and start looking at the format. It would be a huge feature. I'm not against csv's but their use is limited, xml's (not my favorite!) however gives a structure to the data and makes the data also more human readable. Next to being an established standard. You know that after importing the csv file, you have an osm layer that you can save and work with. So if you happen to have a tool that exports the new/update records in csv format (any decent DB-tool can generate files in that format), you do not have to write a tool that generates OSM-xml. So when you can determine the new/updated records in the Crab DB (I do not know in which format the updates from Crab are provided) with a simple SQL query, you export the result in csv, import with the OpenData plugin and you don't need to write any other program. That's all that I wanted to say. Yeah, I understand the reasoning but I'm thinking at some point you want to inject something halfway , so in this case, you need to update the csv's (record, line based). Then it gets complicated -perhaps-. I was just thinking idealistically out loud. I'm thinking, how would a complex building be represented in a csv format, I think I would get frustrated using it as my local app working storage (that's what I consider it to be in this case) Your shortcut to results seems worth considering, I aggree. Thanks for bringing this feature to my attention too. It doesn't hurt bringing idea's on the table I hope. Glenn ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
okay, I got it now. Thanks for clarifying. And of course, you're right. You need to limit the data somehow to work with it. m On Wed, Oct 23, 2013 at 1:52 PM, Glenn Plas gl...@byte-consult.be wrote: or you could add all changes in such a way that they are also added to the tool that Ben proposes. I would just not invent a new output format to work with locally... Although using sqlite as a locale storage (whatever tool) would also help a lot speedwise. No, not for local use, but to share the updates with other mappers. Local use context = your workmemory, when dealing with possible large datasets I would use a way to not do load it all in memory first before launching logic at it : if you want those kind of tools try whatever is around to provide routing with OSM (*) data. So I think this got misinterpreted here. I'm envisioning any tool to be some sort of man-in-the middle. Glenn (*) Except Routino ( http://http://routino.org/ ) which creates fast local storage to serve. http://routing.bitless.be/www/routino/router.html ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
On Wed, Oct 23, 2013 at 10:13:29AM +0200, Glenn Plas wrote: In pseudo: - get data from osm (assuming here , the data is partial, so lets say, everything with an 'addr' tag in your field of view.) , the same effect you have when exporting a certain key using overpass. - get data from crab, craft is as such (preparse it) to facilitate merging with osm data set. - Make the diff, but create an OSM compliant xml (with meta data, otherwise you won't be able to create a changeset from it) - open the changeset with JOSM, verify, correct, validate and push. So I was looking at changeset formats yesterday. It seems that josm has it's own xml format for that which would contain all the data and then have things like action=delete, and so is a changeset format. But it's not something I really like since it doesn't contain just the difference but the whole objects. And it seems that this is the only one supported by josm. But none of the diff files really seem to do what I'm expecting a diff to do. Kurt ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
I assume you are talking about http://wiki.openstreetmap.org/wiki/API_v0.6 ? This API is not only used by JOSM, but by any editor, including iD I assume. m On Wed, Oct 23, 2013 at 6:54 PM, Kurt Roeckx k...@roeckx.be wrote: On Wed, Oct 23, 2013 at 10:13:29AM +0200, Glenn Plas wrote: In pseudo: - get data from osm (assuming here , the data is partial, so lets say, everything with an 'addr' tag in your field of view.) , the same effect you have when exporting a certain key using overpass. - get data from crab, craft is as such (preparse it) to facilitate merging with osm data set. - Make the diff, but create an OSM compliant xml (with meta data, otherwise you won't be able to create a changeset from it) - open the changeset with JOSM, verify, correct, validate and push. So I was looking at changeset formats yesterday. It seems that josm has it's own xml format for that which would contain all the data and then have things like action=delete, and so is a changeset format. But it's not something I really like since it doesn't contain just the difference but the whole objects. And it seems that this is the only one supported by josm. But none of the diff files really seem to do what I'm expecting a diff to do. Kurt ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
On Wed, Oct 23, 2013 at 07:17:42PM +0200, Marc Gemis wrote: I assume you are talking about http://wiki.openstreetmap.org/wiki/API_v0.6 ? This API is not only used by JOSM, but by any editor, including iD I assume. I was talking about things like: http://wiki.openstreetmap.org/wiki/JOSM_file_format http://wiki.openstreetmap.org/wiki/OsmChange http://wiki.openstreetmap.org/wiki/Overpass_API/Augmented_Diffs Kurt On Wed, Oct 23, 2013 at 6:54 PM, Kurt Roeckx k...@roeckx.be wrote: On Wed, Oct 23, 2013 at 10:13:29AM +0200, Glenn Plas wrote: In pseudo: - get data from osm (assuming here , the data is partial, so lets say, everything with an 'addr' tag in your field of view.) , the same effect you have when exporting a certain key using overpass. - get data from crab, craft is as such (preparse it) to facilitate merging with osm data set. - Make the diff, but create an OSM compliant xml (with meta data, otherwise you won't be able to create a changeset from it) - open the changeset with JOSM, verify, correct, validate and push. So I was looking at changeset formats yesterday. It seems that josm has it's own xml format for that which would contain all the data and then have things like action=delete, and so is a changeset format. But it's not something I really like since it doesn't contain just the difference but the whole objects. And it seems that this is the only one supported by josm. But none of the diff files really seem to do what I'm expecting a diff to do. Kurt ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
On Tue, Oct 22, 2013 at 05:36:28AM +0200, Marc Gemis wrote: So you are going to write an algorithm that matching addresses in OSM with addresses in Crab in order to add an id. Right now there are already addresses in OSM that are not in Crab. The same might happen next year. People might have added POIs with addresses. So you will always need an address based matching algorithm. So there is no reason to add the Crab id in OSM. I don't follow your reasoning here. Addresses in OSM but not CRAB shouldn't be a problem. I also don't understand your comment about POIs with an address. It's not because you can match a lot of the addresses based on al algorithm that you can find all of them. This *will* require people to manually fix things and manually add the relation between the 2. What do you mean by Fix our data ? Is Crab suddenly the holy grail ? I didn't say anything like that. I just say that our data *does* contain errors. I'm also pretty sure theirs contain errors. If we look at the differences we need to find out which one is correct, and then try to get the correct information in both. Their DB contains mistakes as well. I'm against a full automatic import. I'm still in favor of the workflow that Ben proposes. Using a website to download a street. Manually merging with existing data, drawing buildings, merging or splitting buildings were needed. Who wrote a few days back that house nodes without buildings are not so good (I'm not saying it was you) ? An automatic import cannot prevent that. I did say that I would prefer the address information is added to building, but that just having a housenumber and no building is better then nothing. I also don't see myself drawing all the buildings when I'm going to import the address information because it will take a lot more time. But I will at some point draw them. You might have different priorities than I. If I were to write an import tool, I would be careful on when to import something, and when in doubt don't import the address. I already have several rule in my head that could be useful. But it looks to me like nobody wants me to do this, so I'm not going to put any effort in this. It would be nice though to have something like Jo did for the busstops. Have a table for mismatches between the OSM data and the imported data. Such a list could be generated every year to see which data should be added or updated AGIV delivers updated files on daily basis. There should not be a problem to actually also compare them on daily basis, and update the list of nodes that still need to be imported on daily basis. But I don't see myself putting time in this if there is no relation between the 2 databases. Kurt ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
FWIW I also see value in adding a backreference to CRAB in the OSM data. It will make it a lot easier to do automated follow up and comparison in the long run. I also see value in the slow but steady way of having contributors integrate the data. It's slower by an order of magnitude, but indeed community is more important than content in the long run. Jo 2013/10/22 Kurt Roeckx k...@roeckx.be On Tue, Oct 22, 2013 at 05:36:28AM +0200, Marc Gemis wrote: So you are going to write an algorithm that matching addresses in OSM with addresses in Crab in order to add an id. Right now there are already addresses in OSM that are not in Crab. The same might happen next year. People might have added POIs with addresses. So you will always need an address based matching algorithm. So there is no reason to add the Crab id in OSM. I don't follow your reasoning here. Addresses in OSM but not CRAB shouldn't be a problem. I also don't understand your comment about POIs with an address. It's not because you can match a lot of the addresses based on al algorithm that you can find all of them. This *will* require people to manually fix things and manually add the relation between the 2. What do you mean by Fix our data ? Is Crab suddenly the holy grail ? I didn't say anything like that. I just say that our data *does* contain errors. I'm also pretty sure theirs contain errors. If we look at the differences we need to find out which one is correct, and then try to get the correct information in both. Their DB contains mistakes as well. I'm against a full automatic import. I'm still in favor of the workflow that Ben proposes. Using a website to download a street. Manually merging with existing data, drawing buildings, merging or splitting buildings were needed. Who wrote a few days back that house nodes without buildings are not so good (I'm not saying it was you) ? An automatic import cannot prevent that. I did say that I would prefer the address information is added to building, but that just having a housenumber and no building is better then nothing. I also don't see myself drawing all the buildings when I'm going to import the address information because it will take a lot more time. But I will at some point draw them. You might have different priorities than I. If I were to write an import tool, I would be careful on when to import something, and when in doubt don't import the address. I already have several rule in my head that could be useful. But it looks to me like nobody wants me to do this, so I'm not going to put any effort in this. It would be nice though to have something like Jo did for the busstops. Have a table for mismatches between the OSM data and the imported data. Such a list could be generated every year to see which data should be added or updated AGIV delivers updated files on daily basis. There should not be a problem to actually also compare them on daily basis, and update the list of nodes that still need to be imported on daily basis. But I don't see myself putting time in this if there is no relation between the 2 databases. Kurt ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
OK de relatie it is! :-) Met vriendelijke groeten, Best regards, Ben Abelshausen 2013/10/20 Marc Gemis marc.ge...@gmail.com relatie is voor mij ok, en waarschijnlijk voor alle ervaren JOSM mappers ook wel. de anderen zullen we dan ervaren moeten maken :-) m 2013/10/19 Kurt Roeckx k...@roeckx.be On Sat, Oct 19, 2013 at 02:00:12PM +0200, Ben Abelshausen wrote: Hallo, Wat zeg ik nu om de adressen te importeren? Het is het één of het ander: - Alles op de node. - Alles op relatie behalve nummer. Ik dacht dat het al duidelijk was, maar ik ben voorstaander van het op de relatie te doen. Kurt ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
On Sun, Oct 20, 2013 at 5:35 PM, Kurt Roeckx k...@roeckx.be wrote: So one thing I noticed is that they contain dates from when the housenumber was valid until when it stopped being valid. For the cases I've looked at, it seems that you just imported those housenumbers, even though they really don't seem to exist anymore. So can you take a look at how you generate those files? I'm also wandering if we want to add some kind of reference to the ID from CRAB. Hi Kurt, I will investigate the end-date issue. :-) I'm not in favour of keeping external id's in OSM, the address itself should be the id and you cannot rely on it staying on the map because anyone can remove it. Met vriendelijke groeten, Best regards, Ben Abelshausen ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
On Mon, Oct 21, 2013 at 08:05:50PM +0200, Ben Abelshausen wrote: I'm also wondering if we want to add some kind of reference to the ID from CRAB. I'm not in favour of keeping external id's in OSM, the address itself should be the id and you cannot rely on it staying on the map because anyone can remove it. I was just thinking about how to keep in sync with their database, and to try and find out which points are mapped and which are not. I'm also considering trying to do a full import and merging existing nodes / houses with their data and things like that. I would actually like to mostly automate importing updates comming from them. I think that at some point we will at least need a way to see see the difference between the 2, and having a direct relation betweem them could make things easier. Kurt ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
On 2013-10-21 20:57, Kurt Roeckx wrote: On Mon, Oct 21, 2013 at 08:05:50PM +0200, Ben Abelshausen wrote: I'm also wondering if we want to add some kind of reference to the ID from CRAB. I'm not in favour of keeping external id's in OSM, the address itself should be the id and you cannot rely on it staying on the map because anyone can remove it. I was just thinking about how to keep in sync with their database, and to try and find out which points are mapped and which are not. I'm also considering trying to do a full import and merging existing nodes / houses with their data and things like that. I would actually like to mostly automate importing updates comming from them. I think that at some point we will at least need a way to see see the difference between the 2, and having a direct relation betweem them could make things easier. Yup, you'll need a foreign key if you want to manage that automatically. I aggree. If not you could endup merging the same stuff all over again. Or worse, delete old nodes and create new ones, enlarging the changesets. But I would caution for doing an automagical full import and merge unless you're doing only for the referenced stuff, but new data and merged I would stll want to review it. I've hardly ever had this sort of database migration work at the first attempt Now lets hope someone at CRAB isn't planning on doing the same but in the other direction ;-) , I keep wondering what kind of feedback loop that would create Glenn ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
Met vriendelijke groeten, Best regards, Ben Abelshausen On Mon, Oct 21, 2013 at 9:26 PM, Glenn Plas gl...@byte-consult.be wrote: On 2013-10-21 20:57, Kurt Roeckx wrote: On Mon, Oct 21, 2013 at 08:05:50PM +0200, Ben Abelshausen wrote: I'm also wondering if we want to add some kind of reference to the ID from CRAB. I'm not in favour of keeping external id's in OSM, the address itself should be the id and you cannot rely on it staying on the map because anyone can remove it. I was just thinking about how to keep in sync with their database, and to try and find out which points are mapped and which are not. I'm also considering trying to do a full import and merging existing nodes / houses with their data and things like that. I would actually like to mostly automate importing updates comming from them. I think that at some point we will at least need a way to see see the difference between the 2, and having a direct relation betweem them could make things easier. Yup, you'll need a foreign key if you want to manage that automatically. I aggree. If not you could endup merging the same stuff all over again. Or worse, delete old nodes and create new ones, enlarging the changesets. But I would caution for doing an automagical full import and merge unless you're doing only for the referenced stuff, but new data and merged I would stll want to review it. I've hardly ever had this sort of database migration work at the first attempt Now lets hope someone at CRAB isn't planning on doing the same but in the other direction ;-) , I keep wondering what kind of feedback loop that would create Glenn __**_ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.**org/listinfo/talk-behttps://lists.openstreetmap.org/listinfo/talk-be ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
Oeps, sorry for the empty mail! :-) I think we can use CRAB as a source but we will never be able to update automatically but even without an ID we should be able to detect changes and if not then most likely there will be some error in OSM or CRAB (if streetnames do not match or something). I even think there is no 'one-id-per-address' in the CRAB datamodel. I will ask someone on Wednesday when I will be back at AGIV but I think there are only linked buildings/percelen and those tend to change too. Met vriendelijke groeten, Best regards, Ben Abelshausen On Mon, Oct 21, 2013 at 9:34 PM, Ben Abelshausen ben.abelshau...@gmail.comwrote: Met vriendelijke groeten, Best regards, Ben Abelshausen On Mon, Oct 21, 2013 at 9:26 PM, Glenn Plas gl...@byte-consult.be wrote: On 2013-10-21 20:57, Kurt Roeckx wrote: On Mon, Oct 21, 2013 at 08:05:50PM +0200, Ben Abelshausen wrote: I'm also wondering if we want to add some kind of reference to the ID from CRAB. I'm not in favour of keeping external id's in OSM, the address itself should be the id and you cannot rely on it staying on the map because anyone can remove it. I was just thinking about how to keep in sync with their database, and to try and find out which points are mapped and which are not. I'm also considering trying to do a full import and merging existing nodes / houses with their data and things like that. I would actually like to mostly automate importing updates comming from them. I think that at some point we will at least need a way to see see the difference between the 2, and having a direct relation betweem them could make things easier. Yup, you'll need a foreign key if you want to manage that automatically. I aggree. If not you could endup merging the same stuff all over again. Or worse, delete old nodes and create new ones, enlarging the changesets. But I would caution for doing an automagical full import and merge unless you're doing only for the referenced stuff, but new data and merged I would stll want to review it. I've hardly ever had this sort of database migration work at the first attempt Now lets hope someone at CRAB isn't planning on doing the same but in the other direction ;-) , I keep wondering what kind of feedback loop that would create Glenn __**_ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.**org/listinfo/talk-behttps://lists.openstreetmap.org/listinfo/talk-be ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
On Mon, Oct 21, 2013 at 09:40:07PM +0200, Ben Abelshausen wrote: I think we can use CRAB as a source but we will never be able to update automatically but even without an ID we should be able to detect changes and if not then most likely there will be some error in OSM or CRAB (if streetnames do not match or something). I even think there is no 'one-id-per-address' in the CRAB datamodel. I will ask someone on Wednesday when I will be back at AGIV but I think there are only linked buildings/percelen and those tend to change too. They actually have pretty good documentation, and really thought about their data model. They have a total of 20 tables with foreign keys between them. Their is a many-to-many relationship between addresses and points on the map. That is a building can have multiple addresses, and 1 address can have multiple buildings. An address can also have subaddresses. And each table has an ID in it. Having the IDs in osm would make it a lot easier to compare the databases. Having 2 databases you can compare is very handy to check for mistakes. You can of course compare their old database against a newer and try to import that, but I have a feeling that that is going to give you more manual work in the long end. I think it's important to try automate things. I really see no good reason not to add those IDs at this point. I don't see the harm in them. I can only see them being useful. Kurt ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
On Mon, Oct 21, 2013 at 10:06:03PM +0200, Kurt Roeckx wrote: I really see no good reason not to add those IDs at this point. I don't see the harm in them. I can only see them being useful. I would actually want to propose a different import strategy: - Add the CRAB IDs to all existing addresses in Flanders - Import the rest or large parts of CRAB in one big import The first part would actually be the hard part, and would need to be done carefully. It will probably require large parts to be checked manually. We would probably first need to fix our existing data, which I think is useful to do anyway. Therefor it would be useful that if you're going to import data from CRAB that you don't complicate that and already import the IDs. It should then be possible to combine both ways of importing. PS: Do we actually already have clearance to import this data? Kurt ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
So you are going to write an algorithm that matching addresses in OSM with addresses in Crab in order to add an id. Right now there are already addresses in OSM that are not in Crab. The same might happen next year. People might have added POIs with addresses. So you will always need an address based matching algorithm. So there is no reason to add the Crab id in OSM. What do you mean by Fix our data ? Is Crab suddenly the holy grail ? Their DB contains mistakes as well. I'm against a full automatic import. I'm still in favor of the workflow that Ben proposes. Using a website to download a street. Manually merging with existing data, drawing buildings, merging or splitting buildings were needed. Who wrote a few days back that house nodes without buildings are not so good (I'm not saying it was you) ? An automatic import cannot prevent that. It would be nice though to have something like Jo did for the busstops. Have a table for mismatches between the OSM data and the imported data. Such a list could be generated every year to see which data should be added or updated regards m On Mon, Oct 21, 2013 at 10:45 PM, Kurt Roeckx k...@roeckx.be wrote: On Mon, Oct 21, 2013 at 10:06:03PM +0200, Kurt Roeckx wrote: I really see no good reason not to add those IDs at this point. I don't see the harm in them. I can only see them being useful. I would actually want to propose a different import strategy: - Add the CRAB IDs to all existing addresses in Flanders - Import the rest or large parts of CRAB in one big import The first part would actually be the hard part, and would need to be done carefully. It will probably require large parts to be checked manually. We would probably first need to fix our existing data, which I think is useful to do anyway. Therefor it would be useful that if you're going to import data from CRAB that you don't complicate that and already import the IDs. It should then be possible to combine both ways of importing. PS: Do we actually already have clearance to import this data? Kurt ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
On Mon, Oct 14, 2013 at 08:26:43PM +0200, Ben Abelshausen wrote: I also converted the entire dataset to readable csv files and the coordinates to lat/lon: https://github.com/xivk/crab-tools/tree/master/crab_csv So I've been looking at at their database, which contains all kinds of useful data. So one thing I noticed is that they contain dates from when the housenumber was valid until when it stopped being valid. For the cases I've looked at, it seems that you just imported those housenumbers, even though they really don't seem to exist anymore. So can you take a look at how you generate those files? I'm also wandering if we want to add some kind of reference to the ID from CRAB. Kurt ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
Hallo, Wat zeg ik nu om de adressen te importeren? Het is het één of het ander: - Alles op de node. - Alles op relatie behalve nummer. Met vriendelijke groeten, Best regards, Ben Abelshausen ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
Wat mij betreft alles in de relatie. Om valjdatietools tevreden te stellen waren we begonnen met addr:street ook op de node te zetten. Jo Op 19 okt. 2013 14:01 schreef Ben Abelshausen ben.abelshau...@gmail.com het volgende: Hallo, Wat zeg ik nu om de adressen te importeren? Het is het één of het ander: - Alles op de node. - Alles op relatie behalve nummer. Met vriendelijke groeten, Best regards, Ben Abelshausen ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
On Saturday 19 October 2013 14:00:12 Ben Abelshausen wrote: Hallo, Wat zeg ik nu om de adressen te importeren? Het is het één of het ander: - Alles op de node. - Alles op relatie behalve nummer. relatie +1 Ben ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
On Sat, Oct 19, 2013 at 02:00:12PM +0200, Ben Abelshausen wrote: Hallo, Wat zeg ik nu om de adressen te importeren? Het is het één of het ander: - Alles op de node. - Alles op relatie behalve nummer. Ik dacht dat het al duidelijk was, maar ik ben voorstaander van het op de relatie te doen. Kurt ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
relatie is voor mij ok, en waarschijnlijk voor alle ervaren JOSM mappers ook wel. de anderen zullen we dan ervaren moeten maken :-) m 2013/10/19 Kurt Roeckx k...@roeckx.be On Sat, Oct 19, 2013 at 02:00:12PM +0200, Ben Abelshausen wrote: Hallo, Wat zeg ik nu om de adressen te importeren? Het is het één of het ander: - Alles op de node. - Alles op relatie behalve nummer. Ik dacht dat het al duidelijk was, maar ik ben voorstaander van het op de relatie te doen. Kurt ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
2013/10/16 Marc Gemis marc.ge...@gmail.com 1) Zouden we misschien kunnen afstappen van de regel om huisnummers op de gebouwen te plaatsen ? Zeker in steden waar al grote blokken getekend zijn, is het een serieuze job om die allemaal te verbeteren (want op Bing gebaseerd) en dan nog eens te splitsen. En wat is de toegevoegde waarde ? Je zal bovendien nog dikwijls naar de Agiv website moeten gaan kijken om te weten welke bijgebouwen nu bij welk huisnummer hoor. Mensen die toch willen splitsen kunnen dat nog altijd doen. Het is een beetje zoals de kerken die nu soms met 1 enkel punt zijn aangeduid. Later kan iemand nog altijd het gebouw tekenen. Normaal zijn die adressen uit CRAB, als ze op een gebouw staan als, ook in de csv-files even accuraat als op de AGIV-webiste. Misschien zit er nog een fout in mijn script. Ik ben wel voorstander van het plaatsen van adressen op gebouwen omdat dat echt een meerwaarde is en het duidelijk is welk gebouw bij welk adres hoort. Natuurlijk weet ik ook dat dit niet overal kan. Als je de discussie over de import in new york hebt gevolgd zijn daar dezelfde discussies aan de gang. De bottom-line voor mij: Ik wacht liever langer tot alle adressen erin zitten dan voor een andere import-manier te gaan omdat de manier van werken sneller is. De adressen zouden er toch alleen maar voor geocoding in zitten en daarvoor is de AGIV dataset toch al beschikbaar. 2) De associatedStreet relaties: laat die vallen. Dat is voor velen te complex. Zeker in de gevallen waar de straat ook de gemeentegrens is, of als er al een relatie bestaat. Wie wil kan ze gemakkelijk zelf creëren. Met find in JOSM kan je alles dat in 1 relatie hoort in 2 keer vinden. Nog eens kijken of het niet mogelijk is met 1 find (als er een OR bestaat) Hier ben ik het eigenlijk wel grotendeels mee eens. De tool van de fransen is echter het enige dat ik op deze moment heb. Ik weet niet hoeveel werk het zou zijn om die relatie te laten vallen. Dus wat houden we over: nodes met alle adres informatie (dus inclusief addr:street, addr:city en addr:postcode) die misschien een beetje verschoven moeten worden, wat duplicaten hier en daar verwijderen en dan terug opladen naar OSM. Dit is eenvoudig en gaat snel vooruit. Alle adres informatie is er dan ook als dit gebeurd is. Nadeel is dat nominatim alle adres informatie (buiten het huisnummer) niet bekijkt en de node bij de dichtsbij zijnde straat rekent. Dit zou bij hoekhuizen een probleem kunnen zijn, maar de fout ligt hier bij de afweging van nominatim om de index kleiner te houden. Dat is in mijn ogen ook een nominatim probleem en uiteindelijk geen probleem voor 99% van de adressen die we zouden mappen. Met vriendelijke groeten, Best regards, Ben Abelshausen ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
2013/10/16 Ben Abelshausen ben.abelshau...@gmail.com 1) Zouden we misschien kunnen afstappen van de regel om huisnummers op de gebouwen te plaatsen ? Zeker in steden waar al grote blokken getekend zijn, is het een serieuze job om die allemaal te verbeteren (want op Bing gebaseerd) en dan nog eens te splitsen. En wat is de toegevoegde waarde ? Je zal bovendien nog dikwijls naar de Agiv website moeten gaan kijken om te weten welke bijgebouwen nu bij welk huisnummer hoor. Mensen die toch willen splitsen kunnen dat nog altijd doen. Het is een beetje zoals de kerken die nu soms met 1 enkel punt zijn aangeduid. Later kan iemand nog altijd het gebouw tekenen. Normaal zijn die adressen uit CRAB, als ze op een gebouw staan als, ook in de csv-files even accuraat als op de AGIV-webiste. Misschien zit er nog een fout in mijn script. Ik heb geen voorbeeld van waar het misloopt, maar staan de huisnummers echt op een huis, of staan ze op een perceel ? Bij steden, waar de huisnummers dicht op elkaar liggen kan ik me voorstellen dat je wel naar de AGIV website moet gaan omdat de OSM huizen misschien niet mooi gealigneerd staan of teveel opgesplitst zijn. Op de website kan je dan de contour van het huis zien. Ik heb deze methode al herhaaldelijk moeten toepassen voor mijn werk rond beschermde monumenten. Ook al omdat de geimporteerde nodes niet boven het huis vallen. De hoeveelheid werk om een bestaande closed way of erger nog multipoligon te gaan splitsen is niet te onderschatten enkel en alleen om dan een node op een stukje gebouw te kunnen plaatsen. Als er geen huis/huizen staat/staan ben ik er ook wel voorstander van om ze te tekenen. In dat opzicht is Lille misschien geen goed voorbeeld. Een 2de demo zou een stuk van Gent of Antwerpen moeten nemen, omdat daar de problematiek anders is. groeten m. ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
On Wed, Oct 16, 2013 at 09:06:00AM +0200, Marc Gemis wrote: 2013/10/16 Ben Abelshausen ben.abelshau...@gmail.com 1) Zouden we misschien kunnen afstappen van de regel om huisnummers op de gebouwen te plaatsen ? Zeker in steden waar al grote blokken getekend zijn, is het een serieuze job om die allemaal te verbeteren (want op Bing gebaseerd) en dan nog eens te splitsen. En wat is de toegevoegde waarde ? Je zal bovendien nog dikwijls naar de Agiv website moeten gaan kijken om te weten welke bijgebouwen nu bij welk huisnummer hoor. Mensen die toch willen splitsen kunnen dat nog altijd doen. Het is een beetje zoals de kerken die nu soms met 1 enkel punt zijn aangeduid. Later kan iemand nog altijd het gebouw tekenen. Normaal zijn die adressen uit CRAB, als ze op een gebouw staan als, ook in de csv-files even accuraat als op de AGIV-webiste. Misschien zit er nog een fout in mijn script. Ik heb geen voorbeeld van waar het misloopt, maar staan de huisnummers echt op een huis, of staan ze op een perceel ? Ik heb de indruk dat er veel op het huis zelf staan, maar een deel op het perceel. Ik heb het gevoel dat i.v.m. GRB ze daar mee bezig zijn geweest. Kurt ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
Gilbert, Je hebt gelijk over de 'experience' en het aantal clicks. Een visualisatie is wel mogelijk denk ik, ik zal het eens vragen. De tool is ook nog in ontwikkeling. Ik denk dat we dat eventueel nog wel kunnen verbeteren. Met vriendelijke groeten, Best regards, Ben Abelshausen 2013/10/15 Gilbert Hersschens gherssch...@gmail.com Heb er eens wat mee gedold. Naar mijn mening wat betreft het aantal clicks niet zo veel verschil met het manueel overnemen met de JOSM adres preset van adres gegevens uit de AGIV kaart ( http://ogc.beta.agiv.be/gdiviewer/?simple=true). Als er nu nog een terugkoppeling was om vanuit JOSM weer op die gele druppel in http://addr.openstreetmap.fr/vlaanderen/ te komen, dat zou wel handig zijn. Wat ik ook mis is een visuele indicatie hoeveel data al gedaan / nog te doen is (een beetje zoals bij Urbis of bij maproulette, etc.). Tenslotte is de user experience nogal belabberd (sorry als ik zo kritisch ben). Zodra je de status in de druppel hebt aangepast is hij weg. Heb niet eens de kans om commentaar of mijn naam in te geven... Gilbert ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
On Tue, Oct 15, 2013 at 05:54:26AM +0200, Marc Gemis wrote: I thought the consensus from an earlier discussion here was 1) addr:housenumber + addr:street on building (not as node) 2) for those that want to do it: associatedStreet relation with name, If you use a associatedStreet relation, you want to set the street only once there and not for each node or building. On the other hand if you don't use the associatedStreet relation you do want to have the addr:street on the node or building. It would of course be nicer to have a shape for each buiding, but a node with the address is better then nothing. Kurt ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
On Tue, Oct 15, 2013 at 01:08:14AM +0200, Jo wrote: side note: I prefer to put the source tags (AGIV;CRAB) on the changeset instead of on each and every separate object we're adding. I'm sorry about adding Herentalsesteenweg. I should have read the message before trying to click through to test it... Anyway, it's properly attributed and it's mostly manual labour anyway. It's far less automated than the UrbIS integration, as we still have to draw the building outlines ourselves. I had to create the associatedStreet relation myself. Was that intentional? When I tried it it had an associatedStreet with the name added, maybe something changed? Kurt ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
On Tue, Oct 15, 2013 at 10:56 PM, Kurt Roeckx k...@roeckx.be wrote: When I tried it it had an associatedStreet with the name added, maybe something changed? No not yet. I will request an update tomorrow. I was thinking about only adding adresses where there are building outlines. Met vriendelijke groeten, Best regards, Ben Abelshausen ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
On Tue, Oct 15, 2013 at 10:52 PM, Kurt Roeckx k...@roeckx.be wrote: On Tue, Oct 15, 2013 at 05:54:26AM +0200, Marc Gemis wrote: I thought the consensus from an earlier discussion here was 1) addr:housenumber + addr:street on building (not as node) 2) for those that want to do it: associatedStreet relation with name, If you use a associatedStreet relation, you want to set the street only once there and not for each node or building. then some quality assurance tools will start complaining. and since their developers don't like associatedStreet relations, their will never be a fix. Many simple apps based on OSM also ignore the associatedStreet relations, so you will never have a complete address. Even OpenLinkMap requires it I believe As a programmer I'm in favor of the associatedStreet relation (do not repeat the same data over and over), but I also see that the OSM community is not ready to use it. regards m ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
Ik ben het ook eens met het aantal clicks, maar dat wist ik al uit ervaring (zie mail van gisteren). Ik vrees een beetje dat er mensen gaan zijn die vol enthousiasme gaan beginnen, maar het snel gaan opgeven vanwege het vele werk. Daarom dacht ik aan het volgende: 1) Zouden we misschien kunnen afstappen van de regel om huisnummers op de gebouwen te plaatsen ? Zeker in steden waar al grote blokken getekend zijn, is het een serieuze job om die allemaal te verbeteren (want op Bing gebaseerd) en dan nog eens te splitsen. En wat is de toegevoegde waarde ? Je zal bovendien nog dikwijls naar de Agiv website moeten gaan kijken om te weten welke bijgebouwen nu bij welk huisnummer hoor. Mensen die toch willen splitsen kunnen dat nog altijd doen. Het is een beetje zoals de kerken die nu soms met 1 enkel punt zijn aangeduid. Later kan iemand nog altijd het gebouw tekenen. 2) De associatedStreet relaties: laat die vallen. Dat is voor velen te complex. Zeker in de gevallen waar de straat ook de gemeentegrens is, of als er al een relatie bestaat. Wie wil kan ze gemakkelijk zelf creëren. Met find in JOSM kan je alles dat in 1 relatie hoort in 2 keer vinden. Nog eens kijken of het niet mogelijk is met 1 find (als er een OR bestaat) Dus wat houden we over: nodes met alle adres informatie (dus inclusief addr:street, addr:city en addr:postcode) die misschien een beetje verschoven moeten worden, wat duplicaten hier en daar verwijderen en dan terug opladen naar OSM. Dit is eenvoudig en gaat snel vooruit. Alle adres informatie is er dan ook als dit gebeurd is. Nadeel is dat nominatim alle adres informatie (buiten het huisnummer) niet bekijkt en de node bij de dichtsbij zijnde straat rekent. Dit zou bij hoekhuizen een probleem kunnen zijn, maar de fout ligt hier bij de afweging van nominatim om de index kleiner te houden. just my .5 cent m 2013/10/15 Ben Abelshausen ben.abelshau...@gmail.com Gilbert, Je hebt gelijk over de 'experience' en het aantal clicks. Een visualisatie is wel mogelijk denk ik, ik zal het eens vragen. De tool is ook nog in ontwikkeling. Ik denk dat we dat eventueel nog wel kunnen verbeteren. Met vriendelijke groeten, Best regards, Ben Abelshausen 2013/10/15 Gilbert Hersschens gherssch...@gmail.com Heb er eens wat mee gedold. Naar mijn mening wat betreft het aantal clicks niet zo veel verschil met het manueel overnemen met de JOSM adres preset van adres gegevens uit de AGIV kaart ( http://ogc.beta.agiv.be/gdiviewer/?simple=true). Als er nu nog een terugkoppeling was om vanuit JOSM weer op die gele druppel in http://addr.openstreetmap.fr/vlaanderen/ te komen, dat zou wel handig zijn. Wat ik ook mis is een visuele indicatie hoeveel data al gedaan / nog te doen is (een beetje zoals bij Urbis of bij maproulette, etc.). Tenslotte is de user experience nogal belabberd (sorry als ik zo kritisch ben). Zodra je de status in de druppel hebt aangepast is hij weg. Heb niet eens de kans om commentaar of mijn naam in te geven... Gilbert ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
Seems like a good interface. Looking forward to start importing We might have to setup some sessions (google hangouts, face-to-face) for people that are not familiar with JOSM. Also for people that are familiar, we might give a session with some tricks on the fastest way to combine the imported data and the existing OSM data. thanks a lot for your hard work regards m On Mon, Oct 14, 2013 at 8:26 PM, Ben Abelshausen ben.abelshau...@gmail.comwrote: Hi, What does everybody think of this tool to use a reference and to keep track of what has been imported and what not for CRAB: http://addr.openstreetmap.fr/vlaanderen/ It contains only data from one village (Lille) in de kempen but it should be clear how this would work. We can also translate the UI into dutch. Workflow: - Open JOSM. - Make sure remote control is enabled. - Click one of the roads that can be imported/conflated. - Mark as done when done. I also converted the entire dataset to readable csv files and the coordinates to lat/lon: https://github.com/xivk/crab-tools/tree/master/crab_csv Please don't import anything before we have sent the import plan to the imports list. Met vriendelijke groeten, Best regards, Ben Abelshausen ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
Hello all, New to OSM and JOSM. If there are any sessions about it you can count me in. Living in the center of Belgium, Dutch is mother language but French is also OK. P_Verschueren : Twitter and OSM nickname P Op 14-okt.-2013 om 21:21 heeft Marc Gemis marc.ge...@gmail.com het volgende geschreven: Seems like a good interface. Looking forward to start importing We might have to setup some sessions (google hangouts, face-to-face) for people that are not familiar with JOSM. Also for people that are familiar, we might give a session with some tricks on the fastest way to combine the imported data and the existing OSM data. thanks a lot for your hard work regards m On Mon, Oct 14, 2013 at 8:26 PM, Ben Abelshausen ben.abelshau...@gmail.com wrote: Hi, What does everybody think of this tool to use a reference and to keep track of what has been imported and what not for CRAB: http://addr.openstreetmap.fr/vlaanderen/ It contains only data from one village (Lille) in de kempen but it should be clear how this would work. We can also translate the UI into dutch. Workflow: - Open JOSM. - Make sure remote control is enabled. - Click one of the roads that can be imported/conflated. - Mark as done when done. I also converted the entire dataset to readable csv files and the coordinates to lat/lon: https://github.com/xivk/crab-tools/tree/master/crab_csv Please don't import anything before we have sent the import plan to the imports list. Met vriendelijke groeten, Best regards, Ben Abelshausen ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
Since the CRAB data is actually from flanders, it would probably make sense to do this either in English or Dutch. Kurt On Mon, Oct 14, 2013 at 09:28:57PM +0200, Peter Verschueren wrote: Hello all, New to OSM and JOSM. If there are any sessions about it you can count me in. Living in the center of Belgium, Dutch is mother language but French is also OK. P_Verschueren : Twitter and OSM nickname P Op 14-okt.-2013 om 21:21 heeft Marc Gemis marc.ge...@gmail.com het volgende geschreven: Seems like a good interface. Looking forward to start importing We might have to setup some sessions (google hangouts, face-to-face) for people that are not familiar with JOSM. Also for people that are familiar, we might give a session with some tricks on the fastest way to combine the imported data and the existing OSM data. thanks a lot for your hard work regards m On Mon, Oct 14, 2013 at 8:26 PM, Ben Abelshausen ben.abelshau...@gmail.com wrote: Hi, What does everybody think of this tool to use a reference and to keep track of what has been imported and what not for CRAB: http://addr.openstreetmap.fr/vlaanderen/ It contains only data from one village (Lille) in de kempen but it should be clear how this would work. We can also translate the UI into dutch. Workflow: - Open JOSM. - Make sure remote control is enabled. - Click one of the roads that can be imported/conflated. - Mark as done when done. I also converted the entire dataset to readable csv files and the coordinates to lat/lon: https://github.com/xivk/crab-tools/tree/master/crab_csv Please don't import anything before we have sent the import plan to the imports list. Met vriendelijke groeten, Best regards, Ben Abelshausen ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
On Mon, Oct 14, 2013 at 08:26:43PM +0200, Ben Abelshausen wrote: Hi, What does everybody think of this tool to use a reference and to keep track of what has been imported and what not for CRAB: http://addr.openstreetmap.fr/vlaanderen/ So looking at the data, I've already found several places where I think the place of the node isn't really where the building is, and that maybe some numbers are missing. Is AGIV interested in feedback about this? And if so, how are we going to do that? If we move the node to an other place, I guess it make sense to remove the source tag for that node? Kurt ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
On Mon, Oct 14, 2013 at 9:39 PM, Kurt Roeckx k...@roeckx.be wrote: So looking at the data, I've already found several places where I think the place of the node isn't really where the building is, and that maybe some numbers are missing. Just some context: If we can add addresses to building we will have added value because AGIV does not do this in most cases. Usually this is just at the 'perceel' level that's why sometimes it seems like this is inaccurate. The tool is from Frédéric Rodrigo (from OSM-FR), I did not create this I just sent him some of the files after I met him at State of the Map :-) The source is wrong obviously and I guess we should also add postalcode and commune name?? AGIV can always take OSM and have a look at our work if they want to. Next year there will also be a presentation about what we did at their annual conference. Hopefully by then we can show off some of our high-quality work! :-) Met vriendelijke groeten, Best regards, Ben Abelshausen ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
On Mon, Oct 14, 2013 at 09:59:29PM +0200, Ben Abelshausen wrote: The source is wrong obviously and I guess we should also add postalcode and commune name?? Adding postal code and commune name doesn't make sense to me in most cases. At least not for how it works in Belgium. That information belongs on the administrative relations. Adding it would be duplication of information we already have, or should have. And they ussualy end up conflicting after some time. The only case this makes sense to me are in cases where it's unclear. For instance when an administrative relation runs through a building the official address will be on one side of it and you need to add extra data to make clear where it is. Kurt ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
On Monday 14 October 2013 22:13:09 Kurt Roeckx wrote: On Mon, Oct 14, 2013 at 09:59:29PM +0200, Ben Abelshausen wrote: The source is wrong obviously and I guess we should also add postalcode and commune name?? Adding postal code and commune name doesn't make sense to me in most cases. At least not for how it works in Belgium. That information belongs on the administrative relations. Adding it would be duplication of information we already have, or should have. And they ussualy end up conflicting after some time. The only case this makes sense to me are in cases where it's unclear. For instance when an administrative relation runs through a building the official address will be on one side of it and you need to add extra data to make clear where it is. I'd add them, our administrative borders aren't all very accurate and it's not like they always have a single postal code in every boundary, see Antwerp for example. Ben ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
side note: I prefer to put the source tags (AGIV;CRAB) on the changeset instead of on each and every separate object we're adding. I'm sorry about adding Herentalsesteenweg. I should have read the message before trying to click through to test it... Anyway, it's properly attributed and it's mostly manual labour anyway. It's far less automated than the UrbIS integration, as we still have to draw the building outlines ourselves. I had to create the associatedStreet relation myself. Was that intentional? Polyglot 2013/10/15 Jo winfi...@gmail.com This works quite well in combination with the building tools plugin. Switch on: use address nodes under buildings. Then use x to extrude to get the shape of the building right. A hangout to demonstrate this would be a good idea. I think the street name is missing. I prefer to add information about postcode, village and country through associatedStreet relations. Jo 2013/10/14 Kurt Roeckx k...@roeckx.be On Mon, Oct 14, 2013 at 01:31:02PM -0700, Ben Laenen wrote: I'd add them, our administrative borders aren't all very accurate and it's not like they always have a single postal code in every boundary, see Antwerp for example. That just looks like a good reason to properly map those administrative borders to me. But I guess it's currently not easy to see what the CRAB data says which postal code it belongs to if we don't see it anywhere. If we do add it, I suggest only in the assosiatedStreet relation. As far as I know there really shouldn't be a problem adding boundary=postal_code relations for places like Antwerpen. Kurt ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be ___ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
Re: [OSM-talk-be] CRAB Import Tool
I thought the consensus from an earlier discussion here was 1) addr:housenumber + addr:street on building (not as node) 2) for those that want to do it: associatedStreet relation with name, postalcode, city That's the way I'm doing it since then. After the meeting in Lier in February I wrote a script that converts the waypoint files that I make with my GPS into an osm-file. It looks a lot like the one you can download now with AGIV/Crab data My experience with starting from that coverted waypoint file and ending up with 1 2 above: (Using JOSM, + you need building plugin utilstool2 plugin terrace tool) 1. there is no building in osm, it's a single house. You can use the building tool with the right configuration. 2. no building, semi-detached house (tweewoonst). Building tool works for first half. Some more work (I can explain in a session) for the second half 3. no building, terrace (rijhuizen). Could be done with e.g. terrace tool without housenumbers and then merging the data. Using the terrace tool without the imported data makes much more sense here (goes a lot faster) 4. There is an outline, even the individual houses. Merge each outline with the correct housenumber 5. There is an outline (typical not a nice rectangle) for all houses in the terrace. First split, then merge as in 4 6. There is a multipolygon (e.g. for huizenblok in cities). I suggest dropping the multipolygon and to start drawing the individual houses with 1-3 Overall it's a slow job, requiring a lot of steps. In the end I went back to the terracer and address interpolation tools (to generate individual housenumbers), because that seems to go faster. Don't forget that the building outlines are often not very detailed (based on Bing or 3D shapes), so they might require tweaking as well. And what about POIs ? I recommend to add addr:street and addr:housenumber to them as well, but keep them as individual points within the building outline. I see this approach also on other mailing lists. The main reason is that the building and the POI do not share other properties than the address. I repeat the address data, because there is no tool that assigns the address from the building to the POIs inside it. I don't expect to see support, as many countries do not assign the housenumber to the building. @Kurt, during my surveys I found already quite some houses that are not in the AGIV/Crab database. Others have reported similar issues. We were aware of this when we got the data. Someone (a Ben I thought) mentioned that the cities are now responsible for updating the data. The list of cities that committed to this is short. (10 or so). The others are still working on this. The GIS responsible got in touch with Gilbert because she is doing exactly the same thing: tracing buildings form aerial images and assigning tags. @Peter: welcome and thanks for your interest. If you wish we could setup a google hangout this Friday to get you more familiar with JOSM and the tools I mention above. We could do this even without the data from Crab. I still have some survey data that I can use as an example. It could be in Dutch or English. regards m On Tue, Oct 15, 2013 at 1:08 AM, Jo winfi...@gmail.com wrote: side note: I prefer to put the source tags (AGIV;CRAB) on the changeset instead of on each and every separate object we're adding. I'm sorry about adding Herentalsesteenweg. I should have read the message before trying to click through to test it... Anyway, it's properly attributed and it's mostly manual labour anyway. It's far less automated than the UrbIS integration, as we still have to draw the building outlines ourselves. I had to create the associatedStreet relation myself. Was that intentional? Polyglot 2013/10/15 Jo winfi...@gmail.com This works quite well in combination with the building tools plugin. Switch on: use address nodes under buildings. Then use x to extrude to get the shape of the building right. A hangout to demonstrate this would be a good idea. I think the street name is missing. I prefer to add information about postcode, village and country through associatedStreet relations. Jo 2013/10/14 Kurt Roeckx k...@roeckx.be On Mon, Oct 14, 2013 at 01:31:02PM -0700, Ben Laenen wrote: I'd add them, our administrative borders aren't all very accurate and it's not like they always have a single postal code in every boundary, see Antwerp for example. That just looks like a good reason to properly map those administrative borders to me. But I guess it's currently not easy to see what the CRAB data says which postal code it belongs to if we don't see it anywhere. If we do add it, I suggest only in the assosiatedStreet relation. As far as I know there really shouldn't be a problem adding boundary=postal_code relations for places like Antwerpen. Kurt ___ Talk-be mailing list Talk-be@openstreetmap.org
