Re: [HOT] JOSM people validating / mapping in Nepal, please run JOSM's validator before finishing a tile
Hello Dave, This is amazing to see the vast number of invalid tags. This really calls into question the integrity of the database. Do you have much luck getting people to run the validator? I am baffled that the data validation does not take place right at the data entry stage. This is very common in database applications. All the critical fields have validation rules so that the operator can neither skip the critical fields nor enter data that is not applicable to that field. If JOSM, complex as it is, is lacking input data validation, that is a serious failing, in my opinion. For this type of mission, complete and accurate data is critical. You do not have the luxury of time hoping that people will bother with a post entry validation process. I see a discussion about how to label seasonal/intermittent streams but there is obviously no standardized tag for this. How can this be? Who will know that two separate queries will need to be run to discover all intermittent or seasonal streams? Data integrity is fundamental to any GIS/database project, but especially for one supporting a real-time emergency. Thanks, Cheers . . . . . . . . Spring At 06-05-2015 14:06 Wednesday, Dave Corley wrote: This is especially applicable for the second pass tasks that are going on right now but should be considered for anyone doing any Nepal tasks using JOSM. When you are mapping / validating a tile, please run the validator before you mark the tile as complete. I can promise you, you will find things to fix that will improve the data. Here's a OSMI link showing tagging issues only in the Nepal region http://url.ie/yzu8 There are a lot! There will be many other issues (unconnected ways, unclosed ways etc etc) that are quite quick and easy to fix so please take the extra few mins at the end of a tile to improve pre-existing data. Thanks, Dave aka DaCor ___ HOT mailing list HOT@openstreetmap.org https://lists.openstreetmap.org/listinfo/hot ___ HOT mailing list HOT@openstreetmap.org https://lists.openstreetmap.org/listinfo/hot
Re: [HOT] JOSM people validating / mapping in Nepal, please run JOSM's validator before finishing a tile
On Thu, May 07, 2015 at 12:57:06AM -0700, Springfield Harrison wrote: Hello Dave, This is amazing to see the vast number of invalid tags. This really calls into question the integrity of the database. Do you have much luck getting people to run the validator? In OSM there is no such thing as an invalid tag. Everybody is allowed to invent new tags although your expectation should not be that there is a single data consumer who will make use of it. The common set of tags has been selected by a do-ocracy - You see data consumers making use of a certain tag and you start tagging to influence these data consumers. I am baffled that the data validation does not take place right at the data entry stage. This is very common in database applications. All the critical fields have validation rules so that the operator can neither skip the critical fields nor enter data that is not applicable to that field. If JOSM, complex as it is, is lacking input data validation, that is a serious failing, in my opinion. For this type of mission, complete and accurate data is critical. You do not have the luxury of time hoping that people will bother with a post entry validation process. Both ways are perfectly valid - validating on input or on data consumption. OSM has gone the way of validating or using data on the consumer side which i am very happy about. I have invented a tag myself for hinting a routing instance for infrastructure e.g. Telecoms cables. If there would have been an input validation i could not have done so. Flo -- Florian Lohoff f...@zz.de We need to self-defense - GnuPG/PGP enable your email today! signature.asc Description: Digital signature ___ HOT mailing list HOT@openstreetmap.org https://lists.openstreetmap.org/listinfo/hot
Re: [HOT] JOSM people validating / mapping in Nepal, please run JOSM's validator before finishing a tile
There are a number of issues and trade offs. First JOSM actually does do validation but many new mappers use id which apparently does not. If you look at the numbers about half of the mappers are new mappers to OSM on this project so we get a great deal of mapping from them. Some know very little about computers, some have PhDs in computerized mapping. Within OSM there are guidelines but nothing stronger on tags. There is a feature page which has recommended tags but that is all they are. HOT attempts to use more standardized tags. Currently we put no restriction on what mappers can map. It might be sensible to ask new mappers at maperthons to restrict themselves to a subset such as buildings using JOSM building tool. I think there is thought being spent at the moment on how to improve matters, better training videos, what is the best way to make use of the resources available etc. It is worth mentioning that the HOT maps are still the best available in many circumstances. Cheerio John On 7 May 2015 at 03:57, Springfield Harrison stellar...@gmail.com wrote: Hello Dave, This is amazing to see the vast number of invalid tags. This really calls into question the integrity of the database. Do you have much luck getting people to run the validator? I am baffled that the data validation does not take place right at the data entry stage. This is very common in database applications. All the critical fields have validation rules so that the operator can neither skip the critical fields nor enter data that is not applicable to that field. If JOSM, complex as it is, is lacking input data validation, that is a serious failing, in my opinion. For this type of mission, complete and accurate data is critical. You do not have the luxury of time hoping that people will bother with a post entry validation process. I see a discussion about how to label seasonal/intermittent streams but there is obviously no standardized tag for this. How can this be? Who will know that two separate queries will need to be run to discover all intermittent or seasonal streams? Data integrity is fundamental to any GIS/database project, but especially for one supporting a real-time emergency. Thanks, Cheers . . . . . . . . Spring At 06-05-2015 14:06 Wednesday, Dave Corley wrote: This is especially applicable for the second pass tasks that are going on right now but should be considered for anyone doing any Nepal tasks using JOSM. When you are mapping / validating a tile, please run the validator before you mark the tile as complete. I can promise you, you will find things to fix that will improve the data. Here's a OSMI link showing tagging issues only in the Nepal region http://url.ie/yzu8 There are a lot! There will be many other issues (unconnected ways, unclosed ways etc etc) that are quite quick and easy to fix so please take the extra few mins at the end of a tile to improve pre-existing data. Thanks, Dave aka DaCor ___ HOT mailing list HOT@openstreetmap.org https://lists.openstreetmap.org/listinfo/hot ___ HOT mailing list HOT@openstreetmap.org https://lists.openstreetmap.org/listinfo/hot ___ HOT mailing list HOT@openstreetmap.org https://lists.openstreetmap.org/listinfo/hot
Re: [HOT] JOSM people validating / mapping in Nepal, please run JOSM's validator before finishing a tile
Hi all, Just to clarify, josm id both now validate but it's possible newbies don't know how to handle the returned errors and just ignore them. The untagged link in my original email was given as a sample only. There are also many, many issues with routing, boundaries, waterways, etc etc etc. Granted there may be some false positives, but those would be the exception. Dave On 7 May 2015 12:27, Florian Lohoff f...@zz.de wrote: On Thu, May 07, 2015 at 12:57:06AM -0700, Springfield Harrison wrote: Hello Dave, This is amazing to see the vast number of invalid tags. This really calls into question the integrity of the database. Do you have much luck getting people to run the validator? In OSM there is no such thing as an invalid tag. Everybody is allowed to invent new tags although your expectation should not be that there is a single data consumer who will make use of it. The common set of tags has been selected by a do-ocracy - You see data consumers making use of a certain tag and you start tagging to influence these data consumers. I am baffled that the data validation does not take place right at the data entry stage. This is very common in database applications. All the critical fields have validation rules so that the operator can neither skip the critical fields nor enter data that is not applicable to that field. If JOSM, complex as it is, is lacking input data validation, that is a serious failing, in my opinion. For this type of mission, complete and accurate data is critical. You do not have the luxury of time hoping that people will bother with a post entry validation process. Both ways are perfectly valid - validating on input or on data consumption. OSM has gone the way of validating or using data on the consumer side which i am very happy about. I have invented a tag myself for hinting a routing instance for infrastructure e.g. Telecoms cables. If there would have been an input validation i could not have done so. Flo -- Florian Lohoff f...@zz.de We need to self-defense - GnuPG/PGP enable your email today! -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (GNU/Linux) iQIVAwUBVUtMMZDdQSDLCfIvAQrVpQ//aDHY2adxhOkuyNjSKgsagYDFLyBPIrrb Cv345sq8Y1tEwcu0QXqYLQb5dHB+g/g0bv5sSl3Gd3TqJZyNbx9Hao5JVB60DDDo RSBNExNiEtioQvpTRYBGaqWRqZbtgizaPman+nmc0dxSxvOrrzxtxE+UUyGzmIEV D8o14PXFDnjIkW1Bd1kGqGTGNC2i6sUggMi65b/v0g4p+XxzOKrJzTSx/Zou8/ny LABAbWyFtJI+/nJaA1/d5+bcGWlttlms9dtxGw3yS+zP1uyvtbv2WqxPWGuK1vav xJ4MX0GeFddM7CGrCc4wGVMv48w10fbpvmJ8jJauEi0fdfYanHPcvn4jkGQgZ/HN LO75PSLRLFwgN+eBhKw3DYLCtZO3Rh9ABmSTWHwujzzAdI5OllNfco76asbFvu/c W+0/L1QUuJDMTOO5J5Z3oXYDtaut9KoRF3mM+dNo/ZQqTl93SlzWDftEem2nCDiS IHbZ62OA2YBxx1Tn6uU9Kd8rbRl1tYn8zNe+Ewjk0YDJZxlEwlf4t+502ucjc3Fx Kv0xRepzxPInbSZ/DjhqsDZlCM7gk5fT2xHC5KrlPvGGV3MB5tN94v+o3EEvC+jz Or/eVXnYvxojp9O1IeFVrXszyUji0am9WlxwKMUEuHscRU59V/syW+T55rMdGcf0 mmYMadQXlak= =Lsbt -END PGP SIGNATURE- ___ HOT mailing list HOT@openstreetmap.org https://lists.openstreetmap.org/listinfo/hot
Re: [HOT] JOSM people validating / mapping in Nepal, please run JOSM's validator before finishing a tile
An interesting thread - I work for a national mapping agency and we collate vector data from all over the world. My personal view is that a vector database can either be consistent or complete and, when over certain size, is rarely both. We run complex validation post-compilation across millions of datasets and do some corrections automatically. Fixing them at source is impossible - it's preferable to support the community with encoding guides and comprehensive tools (some inconsistencies are just personal, national or cartographic preferences). My observation on OSM is that the same dynamic applies but the benefits far far outweigh the inconsistencies. as the project grows so its organisation and tools must evolve to adapt to the larger population jon. On Thu, May 7, 2015 at 1:10 PM, Dave Corley davecor...@gmail.com wrote: Hi all, Just to clarify, josm id both now validate but it's possible newbies don't know how to handle the returned errors and just ignore them. The untagged link in my original email was given as a sample only. There are also many, many issues with routing, boundaries, waterways, etc etc etc. Granted there may be some false positives, but those would be the exception. Dave On 7 May 2015 12:27, Florian Lohoff f...@zz.de wrote: On Thu, May 07, 2015 at 12:57:06AM -0700, Springfield Harrison wrote: Hello Dave, This is amazing to see the vast number of invalid tags. This really calls into question the integrity of the database. Do you have much luck getting people to run the validator? In OSM there is no such thing as an invalid tag. Everybody is allowed to invent new tags although your expectation should not be that there is a single data consumer who will make use of it. The common set of tags has been selected by a do-ocracy - You see data consumers making use of a certain tag and you start tagging to influence these data consumers. I am baffled that the data validation does not take place right at the data entry stage. This is very common in database applications. All the critical fields have validation rules so that the operator can neither skip the critical fields nor enter data that is not applicable to that field. If JOSM, complex as it is, is lacking input data validation, that is a serious failing, in my opinion. For this type of mission, complete and accurate data is critical. You do not have the luxury of time hoping that people will bother with a post entry validation process. Both ways are perfectly valid - validating on input or on data consumption. OSM has gone the way of validating or using data on the consumer side which i am very happy about. I have invented a tag myself for hinting a routing instance for infrastructure e.g. Telecoms cables. If there would have been an input validation i could not have done so. Flo -- Florian Lohoff f...@zz.de We need to self-defense - GnuPG/PGP enable your email today! -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (GNU/Linux) iQIVAwUBVUtMMZDdQSDLCfIvAQrVpQ//aDHY2adxhOkuyNjSKgsagYDFLyBPIrrb Cv345sq8Y1tEwcu0QXqYLQb5dHB+g/g0bv5sSl3Gd3TqJZyNbx9Hao5JVB60DDDo RSBNExNiEtioQvpTRYBGaqWRqZbtgizaPman+nmc0dxSxvOrrzxtxE+UUyGzmIEV D8o14PXFDnjIkW1Bd1kGqGTGNC2i6sUggMi65b/v0g4p+XxzOKrJzTSx/Zou8/ny LABAbWyFtJI+/nJaA1/d5+bcGWlttlms9dtxGw3yS+zP1uyvtbv2WqxPWGuK1vav xJ4MX0GeFddM7CGrCc4wGVMv48w10fbpvmJ8jJauEi0fdfYanHPcvn4jkGQgZ/HN LO75PSLRLFwgN+eBhKw3DYLCtZO3Rh9ABmSTWHwujzzAdI5OllNfco76asbFvu/c W+0/L1QUuJDMTOO5J5Z3oXYDtaut9KoRF3mM+dNo/ZQqTl93SlzWDftEem2nCDiS IHbZ62OA2YBxx1Tn6uU9Kd8rbRl1tYn8zNe+Ewjk0YDJZxlEwlf4t+502ucjc3Fx Kv0xRepzxPInbSZ/DjhqsDZlCM7gk5fT2xHC5KrlPvGGV3MB5tN94v+o3EEvC+jz Or/eVXnYvxojp9O1IeFVrXszyUji0am9WlxwKMUEuHscRU59V/syW+T55rMdGcf0 mmYMadQXlak= =Lsbt -END PGP SIGNATURE- ___ HOT mailing list HOT@openstreetmap.org https://lists.openstreetmap.org/listinfo/hot ___ HOT mailing list HOT@openstreetmap.org https://lists.openstreetmap.org/listinfo/hot
[HOT] JOSM people validating / mapping in Nepal, please run JOSM's validator before finishing a tile
This is especially applicable for the second pass tasks that are going on right now but should be considered for anyone doing any Nepal tasks using JOSM. When you are mapping / validating a tile, please run the validator before you mark the tile as complete. I can promise you, you will find things to fix that will improve the data. Here's a OSMI link showing tagging issues only in the Nepal region http://url.ie/yzu8 There are a lot! There will be many other issues (unconnected ways, unclosed ways etc etc) that are quite quick and easy to fix so please take the extra few mins at the end of a tile to improve pre-existing data. Thanks, Dave aka DaCor ___ HOT mailing list HOT@openstreetmap.org https://lists.openstreetmap.org/listinfo/hot
