Re: [Talk-ca] Here we go again...
On 2011-Mar-06, at 9:15 AM, Samuel Longiaru wrote: Hi Dan, Your procedure sounds pretty similar to mine, and working around Kamloops likely is equivalent in terms of the kinds of features we see. It's nice to see that the method I've been following isn't totally out in left field :-) You probably do this as well, but before running the validator, I step around the edge of the import and connect streams, powerlines, and anything else that I think needs connecting. The auto-fix on duplicate nodes just seems to merge the nodes but doesn't combine the ways. As you, I very rarely have found the need to import a road as previous GeoBase or other imports have already provided the same information. Earlier on in my editing, I hadn't thought of connecting features around the edge of an import, though I've been trying to do it lately. One of these days, I'm going to have to go back to areas I already worked on, to do exactly that. I simplify some features as well (streams and some lake shorelines mostly) but I try to remember to simplify before merging the selection onto the OSM layer. Simplifying later often gives the warning that you are deleting nodes outside the uploaded data area. If I get a conflict, this is where it happens. I hadn't been paying much attention to where I did the simplification - thanks for the tip! I'll be sure to do it before merging into my working OSM layer, and see if that helps. You do, however, seem to have much better luck than I have had on failed imports. On 4 or 5 different occasions, an upload has hung (sometimes for hours) and a cancel has resulted in nodes only (no way information) being uploaded to the server. This behavior is quite consistent. The result is 6-8,000 isolated nodes blasted across the import block. I've then had to download the area from OSM and manually remove each node. Rather frustrating. I don't know the ins and outs of the OSM backend, but could you be picking up errors at that point? JOSM never seems to sort it out for me. :( I guess I have been luckier than you, in a way. I've only noticed one import so far that's failed spectacularly, which got me to figuring out how to use the reverter plugin. I think it saved my bacon in that situation - the idea of manually going to find and delete thousands of isolated nodes is a process I really wanted to avoid at all costs. But James Ewen has noticed some missing roads in an area I was working on southeast of Edmonton that I'm starting to think may have been the result of a flaky upload (at least one of those missing roads is one I remember working on specifically). What makes me worried is that I didn't even realize there was a problem in this area until he noticed those roads (I had assumed that JOSM had just fixed things when I retried afterwards) - in some ways, thousands of isolated nodes would have been preferable, since at least I'd have noticed there was a problem. I think I'll be reserving any OSM editing days for those when my Internet connection is as stable as possible. Dan -- Syzygy Research Technology Box 83, Legal, AB T0G 1L0 Canada Phone: 780-961-2213 ___ Talk-ca mailing list Talk-ca@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-ca
Re: [Talk-ca] Mapping cut blocks in wooded areas
Bryan, I would have to agree with your argument. I have some knowledge of the forestry GIS that is used here in NB and it would be a daunting task to include cut blocks in the forest. There is more than enough OSM work in Canada just getting the road network built it would be counterproductive to spend a lot of time on forest cut blocks. Bernie. -- Bernie Connors, P.Eng Service New Brunswick (506) 444-2077 45°56'25.21N, 66°38'53.65W www.snb.ca/geonb/http://www.snb.ca/geonb/ From: Bryan Crosby [mailto:azubr...@gmail.com] Sent: Saturday, 2011-03-05 01:58 To: 'talk-ca' Subject: Re: [Talk-ca] Mapping cut blocks in wooded areas I would tag it as natural=wood as I don’t feel that there is any distinction between a 2-year old stand and a 250 year old stand in terms of being wood, or forest. They are merely different ages. Licensees maintain incredibly accurate and up-to-date maps that indicate the different openings and their respective stages of development. They have dedicated GIS guys that maintain these maps as fast as techies bring it in. I suppose, in theory, an OSM tag could be used to indicate the stage of opening development, but one would require the date of harvesting, the date of planting and the dates of the silviculture surveys to accurately assess the phase. Unless you are a forester you won’t have access to that information and would be guessing. I just feel that attempting to seriously map out such temporary features accurately goes way beyond the ability of OSM (at this point, at least). Bryan From: Samuel Longiaru [mailto:longi...@shaw.ca] Sent: March-04-11 9:43 PM To: talk-ca Subject: Re: [Talk-ca] Mapping cut blocks in wooded areas I very much see your point which is why I was asking for some direction. I guess it comes down to whether the map should reflect what we see at some given snapshot in time, or whether it is reflecting the overall landuse scheme. In short, while standing in the middle of a clear-cut, would it be more accurate that my map show that spot as wooded or not wooded? Sam L. -Original Message- From: Bryan Crosby azubr...@gmail.commailto:bryan%20crosby%20%3cazubr...@gmail.com%3e To: 'talk-ca' talk-ca@openstreetmap.orgmailto:'talk-ca'%20%3ctalk...@openstreetmap.org%3e Subject: Re: [Talk-ca] Mapping cut blocks in wooded areas Date: Fri, 04 Mar 2011 21:11:20 -0800 RE: cut-blocks As someone who has spent done time as a forest technician, I strongly advise against mapping forestry activity. Cut block spatial data changes daily and any images used to trace are out of date. There are literally tens of thousands of clear cuts in British Columbia alone and there is absolutely no way OSM mappers would be able to keep up with changes. Keep in mind that most clearcuts on crown land (and in some cases, private land) are temporary openings in various stages forest development. A 2 year old stand is just as much a forest as a 25 year old free-to-grow stand or a 250 year old stand of timber. I believe that mapping a privately held ‘Christmas’ tree farm would be pertinent, but these are radically different from commercial forestry openings. I would also advise extreme caution in using images to map forest development roads unless are working on a high traffic mainline. Many spur roads are in various stages of deactivation. It may look like a road from the outdated image, but it may have been completely deactivated and replanted. A site inspection is the only way to be sure. Bryan British Columbia From: Daniel Begin [mailto:jfd...@hotmail.com] Sent: March-04-11 8:19 PM To: 'Samuel Longiaru'; 'talk-ca' Subject: Re: [Talk-ca] Mapping cut blocks in wooded areas Hi Samuel, About tagging forested areas, I would use landuse=forest only if it is obvious on the field that the area is managed/harvested, as for landuse=orchard or landuse=vineyard. We have a lot of Christmas tree plantations in the area and I map them as landuse=forest because it is obvious on the imagery and on the field. If it is difficult to determine if an area is under timber lease or not, because it looks the same, I would keep it natural=wood... About Cut blocks, I would map the hole they create that wooded area. If the area is replanted, then some OSM contributor will remove the hole you map in 10-20 years from now! Mapping the reality is the best we can do and because the reality changes over time, we can keep mapping !-) Daniel From: Samuel Longiaru [mailto:longi...@shaw.ca] Sent: March-04-11 21:45 To: talk-ca Subject: [Talk-ca] Mapping cut blocks in wooded areas Hi Everybody, I've been importing CanVec mostly south of Kamloops for the past several weeks and am going to take some time now to go back and bring stuff up to date. One question I have though is in regards to how to treat cut blocks in the wooded areas. I see according to the map features
Re: [Talk-ca] Mapping cut blocks in wooded areas
Hi all, just for your information. The Canvec vegetation layer will be replaced by a state of the art image classification. It is based on a subset of the GeoBase Land Cover product. I don't know if cut blocks will be included or excluded of the wooded areas. Daniel From: Connors, Bernie (SNB) [mailto:bernie.conn...@snb.ca] Sent: March 7, 2011 08:29 To: 'Bryan Crosby'; 'talk-ca' Subject: Re: [Talk-ca] Mapping cut blocks in wooded areas Bryan, I would have to agree with your argument. I have some knowledge of the forestry GIS that is used here in NB and it would be a daunting task to include cut blocks in the forest. There is more than enough OSM work in Canada just getting the road network built it would be counterproductive to spend a lot of time on forest cut blocks. Bernie. -- Bernie Connors, P.Eng Service New Brunswick (506) 444-2077 45°56'25.21N, 66°38'53.65W www.snb.ca/geonb/ From: Bryan Crosby [mailto:azubr...@gmail.com] Sent: Saturday, 2011-03-05 01:58 To: 'talk-ca' Subject: Re: [Talk-ca] Mapping cut blocks in wooded areas I would tag it as natural=wood as I don't feel that there is any distinction between a 2-year old stand and a 250 year old stand in terms of being wood, or forest. They are merely different ages. Licensees maintain incredibly accurate and up-to-date maps that indicate the different openings and their respective stages of development. They have dedicated GIS guys that maintain these maps as fast as techies bring it in. I suppose, in theory, an OSM tag could be used to indicate the stage of opening development, but one would require the date of harvesting, the date of planting and the dates of the silviculture surveys to accurately assess the phase. Unless you are a forester you won't have access to that information and would be guessing. I just feel that attempting to seriously map out such temporary features accurately goes way beyond the ability of OSM (at this point, at least). Bryan From: Samuel Longiaru [mailto:longi...@shaw.ca] Sent: March-04-11 9:43 PM To: talk-ca Subject: Re: [Talk-ca] Mapping cut blocks in wooded areas I very much see your point which is why I was asking for some direction. I guess it comes down to whether the map should reflect what we see at some given snapshot in time, or whether it is reflecting the overall landuse scheme. In short, while standing in the middle of a clear-cut, would it be more accurate that my map show that spot as wooded or not wooded? Sam L. -Original Message- From: Bryan Crosby azubr...@gmail.com mailto:bryan%20crosby%20%3cazubr...@gmail.com%3e To: 'talk-ca' talk-ca@openstreetmap.org mailto:'talk-ca'%20%3ctalk...@openstreetmap.org%3e Subject: Re: [Talk-ca] Mapping cut blocks in wooded areas Date: Fri, 04 Mar 2011 21:11:20 -0800 RE: cut-blocks As someone who has spent done time as a forest technician, I strongly advise against mapping forestry activity. Cut block spatial data changes daily and any images used to trace are out of date. There are literally tens of thousands of clear cuts in British Columbia alone and there is absolutely no way OSM mappers would be able to keep up with changes. Keep in mind that most clearcuts on crown land (and in some cases, private land) are temporary openings in various stages forest development. A 2 year old stand is just as much a forest as a 25 year old free-to-grow stand or a 250 year old stand of timber. I believe that mapping a privately held 'Christmas' tree farm would be pertinent, but these are radically different from commercial forestry openings. I would also advise extreme caution in using images to map forest development roads unless are working on a high traffic mainline. Many spur roads are in various stages of deactivation. It may look like a road from the outdated image, but it may have been completely deactivated and replanted. A site inspection is the only way to be sure. Bryan British Columbia From: Daniel Begin [mailto:jfd...@hotmail.com] Sent: March-04-11 8:19 PM To: 'Samuel Longiaru'; 'talk-ca' Subject: Re: [Talk-ca] Mapping cut blocks in wooded areas Hi Samuel, About tagging forested areas, I would use landuse=forest only if it is obvious on the field that the area is managed/harvested, as for landuse=orchard or landuse=vineyard. We have a lot of Christmas tree plantations in the area and I map them as landuse=forest because it is obvious on the imagery and on the field. If it is difficult to determine if an area is under timber lease or not, because it looks the same, I would keep it natural=wood... About Cut blocks, I would map the hole they create that wooded area. If the area is replanted, then some OSM contributor will remove the hole you map in 10-20 years from now! Mapping the reality is the best we can do and because the reality changes
Re: [Talk-ca] Mapping cut blocks in wooded areas
Good Morning Everyone, Thanks for the feedback on my question about mapping cut blocks in wooded areas. I do appreciate it and understand the points that are being made by Daniel, Bryan and Bernie. I've thought a lot about this, however, and would like to address some of the points as it touches a bit on the varied philosophies we all bring to the project. The arguments against mapping the cut blocks as openings in the wood cover seem to fall along the following lines:... and I hope I am not over simplifying or expanding too much: 1) Change is the nature of the beast. When we map wood (or more accurately, forest in OSM terms) it is implied that there will be logging operations taking place within, with the result that the degree of cover is constantly changing as is the maturity of the trees. To try to keep up is fruitless. When one sees a wooded area on a map one should probably just assume that it will be highly variable. 2) There are people who are maintaining this information to a much greater level of detail and accuracy than OSM could ever hope to do. 3) To map them is counterproductive. I have to say that I am not entirely convinced by these well-pointed arguments. 1) Yes, change is what goes on in wooded areas. That's WHY we map them. The value of mapping is often in what we learn by comparing maps from one time period to another. It's important to map so that we can track that change. Any map is simply a snapshot of what exists at a moment in time and the maps themselves are outdated the moment they are made as they are commonly based on outdated data. To use the argument that we shouldn't map a feature simply because it will be out-of-date tomorrow, or next week, or next year I just don't think is convincing. The neighborhood in which I live in Kamloops has changed significantly even in the last two or three years. Houses, businesses and new streets have appeared, trails have disappeared, streams have been diverted. It's a struggle to keep maps up-to-date. But it doesn't mean that it is pointless to map the features as best we can, with the most currently available data we have available to us. 2) The fact that better data exist elsewhere is great. Better data lead to better forest management practices and greater sustainability. But those data are likely unavailable to us or would need to be heavily culled for those relevant to OSM. The City of Kamloops has incredibly detailed maps of the city freely available online. There are overlays for every curb, parking meter and telephone pole. Does that mean that we don't map the city in lesser detail? 3) Would it be counterproductive to map cut-blocks? Well, not counterproductive, but maybe differently productive. I think one of the beauties of this project is that we map things that are not just important to ourselves, but maybe also to someone else. I get a real hoot out of just browsing through the list of official map features - things like... power=cable_distribution_cabinet, amenity=baby_hatch (where one can anonymously drop off a baby for adoption) and barrier=stile vs. turnstile. These must be important and useful to someone. ??? So are cut blocks useful to anyone who might use an OSM map or load OSM data into their GPS? Hard to say. Maybe to an amateur birding group or a geocaching club or to someone who is hurt and is looking for the nearest clearing for evacuation. Don't know. Don't presume to know. So am I going to start mapping all the cut blocks? Following my arguments I probably should... but I probably won't. I will go back and clean up the gross errors and inconsistencies I find in the CanVec data as it relates to natural=wood, some of which are offsets along tile boundaries and inconsistent mapping of cuts along power lines and pipelines. That will probably be enough as it involves breaking and rebuilding relations which just make me scream. But I'll try to clean up what we have first. Anyway, I hope this doesn't sound like a rant because it really isn't. While I lean towards mapping them, the arguments against are well made. We don't maps waves on the ocean... we accept that it will be variable. Maybe natural=wood is similar? Dunno. That's why I asked for guidance. Truly, thanks for your responses. Sam L. -Original Message- From: Connors, Bernie (SNB) bernie.conn...@snb.ca To: 'Bryan Crosby' azubr...@gmail.com, 'talk-ca' talk-ca@openstreetmap.org Subject: Re: [Talk-ca] Mapping cut blocks in wooded areas Date: Mon, 07 Mar 2011 09:28:45 -0400 Bryan, I would have to agree with your argument. I have some knowledge of the forestry GIS that is used here in NB and it would be a daunting task to include cut blocks in the forest. There is more than enough OSM work in Canada just getting the road network built it would be counterproductive to spend a lot of time on forest cut blocks. Bernie. -- Bernie Connors, P.Eng Service New
Re: [Talk-ca] Mapping cut blocks in wooded areas
Some more thoughts, Despite the idealism displayed by mapping cutblocks, I really believe that this should not become standard practice as there are too many variables at work that would interfere with the accuracy, and integrity of the dynamic features in question. Until that can be addressed (possibly with the next CANVEC release) I dont believe we should be aggressively pursuing this. I encourage users to fire up Bing sat tiles for Northern British Columbia to get a feel for the massive numbers of openings. I will adamantly state now that I will not, ever map cutlblocks for any CANVEC I import unless Im provided with solid data to properly tag the blank openings. Bryan From: Daniel Begin [mailto:jfd...@hotmail.com] Sent: March-07-11 10:22 AM To: 'Samuel Longiaru'; 'Connors, Bernie (SNB)' Cc: 'talk-ca' Subject: Re: [Talk-ca] Mapping cut blocks in wooded areas Samuel wrote: Would it be counterproductive to map cut-blocks? Well, not counterproductive, but maybe differently productive Thanks for your thought Samuel, This is exactly what I love in this project, the possibility of being productive differently. Mapping the entire planet through volunteers work could have sound counterproductive five years ago Cheers, Daniel _ From: Samuel Longiaru [mailto:longi...@shaw.ca] Sent: March-07-11 12:17 To: Connors, Bernie (SNB) Cc: 'talk-ca' Subject: Re: [Talk-ca] Mapping cut blocks in wooded areas Good Morning Everyone, Thanks for the feedback on my question about mapping cut blocks in wooded areas. I do appreciate it and understand the points that are being made by Daniel, Bryan and Bernie. I've thought a lot about this, however, and would like to address some of the points as it touches a bit on the varied philosophies we all bring to the project. The arguments against mapping the cut blocks as openings in the wood cover seem to fall along the following lines:... and I hope I am not over simplifying or expanding too much: 1) Change is the nature of the beast. When we map wood (or more accurately, forest in OSM terms) it is implied that there will be logging operations taking place within, with the result that the degree of cover is constantly changing as is the maturity of the trees. To try to keep up is fruitless. When one sees a wooded area on a map one should probably just assume that it will be highly variable. 2) There are people who are maintaining this information to a much greater level of detail and accuracy than OSM could ever hope to do. 3) To map them is counterproductive. I have to say that I am not entirely convinced by these well-pointed arguments. 1) Yes, change is what goes on in wooded areas. That's WHY we map them. The value of mapping is often in what we learn by comparing maps from one time period to another. It's important to map so that we can track that change. Any map is simply a snapshot of what exists at a moment in time and the maps themselves are outdated the moment they are made as they are commonly based on outdated data. To use the argument that we shouldn't map a feature simply because it will be out-of-date tomorrow, or next week, or next year I just don't think is convincing. The neighborhood in which I live in Kamloops has changed significantly even in the last two or three years. Houses, businesses and new streets have appeared, trails have disappeared, streams have been diverted. It's a struggle to keep maps up-to-date. But it doesn't mean that it is pointless to map the features as best we can, with the most currently available data we have available to us. 2) The fact that better data exist elsewhere is great. Better data lead to better forest management practices and greater sustainability. But those data are likely unavailable to us or would need to be heavily culled for those relevant to OSM. The City of Kamloops has incredibly detailed maps of the city freely available online. There are overlays for every curb, parking meter and telephone pole. Does that mean that we don't map the city in lesser detail? 3) Would it be counterproductive to map cut-blocks? Well, not counterproductive, but maybe differently productive. I think one of the beauties of this project is that we map things that are not just important to ourselves, but maybe also to someone else. I get a real hoot out of just browsing through the list of official map features - things like... power=cable_distribution_cabinet, amenity=baby_hatch (where one can anonymously drop off a baby for adoption) and barrier=stile vs. turnstile. These must be important and useful to someone. ??? So are cut blocks useful to anyone who might use an OSM map or load OSM data into their GPS? Hard to say. Maybe to an amateur birding group or a geocaching club or to someone who is hurt and is looking for the nearest clearing for evacuation. Don't know. Don't presume to know. So am I going to start mapping all the cut blocks?
Re: [Talk-ca] Mapping cut blocks in wooded areas
One more My crew and I would sometimes (not often, but sometimes) get lost while tracking down timber stands or cutblocks because our maps were out of date and not accurate enough and we were a local crew. I cant tell how many times we had to direct southern planting crews and the odd newbie forester who got turned around on some spur road. These maps were coming from the licensees and the BC Ministry of Forests (depending on the contract) which offered the most accurate and up-to-date spatial data available. Someone had pulled the bridge out, someone had deactivated the road, the block hadnt been cut yet, there was a block that wasnt mapped yet and the info either didnt make it to the Woodlands office or it was still being processed by the GIS guys. Small things like that would cause great delays for us and could potentially be disastrous for people who really have no business being in the bush. The idea of mapping forestry features in OSM assumes that someone will be around to maintain the data. The probabilities for that are greater in urban environments than they are for middle-of-nowhere (literally) forest development areas. I sincerely apologize if Im coming off too aggressive about forest mapping. It is just that I strongly believe (based on my own education and experience) that this is a chunk of wood way too big for the OSM community to pull off with accuracy and integrity. Bryan From: Daniel Begin [mailto:jfd...@hotmail.com] Sent: March-07-11 10:22 AM To: 'Samuel Longiaru'; 'Connors, Bernie (SNB)' Cc: 'talk-ca' Subject: Re: [Talk-ca] Mapping cut blocks in wooded areas Samuel wrote: Would it be counterproductive to map cut-blocks? Well, not counterproductive, but maybe differently productive Thanks for your thought Samuel, This is exactly what I love in this project, the possibility of being productive differently. Mapping the entire planet through volunteers work could have sound counterproductive five years ago Cheers, Daniel _ From: Samuel Longiaru [mailto:longi...@shaw.ca] Sent: March-07-11 12:17 To: Connors, Bernie (SNB) Cc: 'talk-ca' Subject: Re: [Talk-ca] Mapping cut blocks in wooded areas Good Morning Everyone, Thanks for the feedback on my question about mapping cut blocks in wooded areas. I do appreciate it and understand the points that are being made by Daniel, Bryan and Bernie. I've thought a lot about this, however, and would like to address some of the points as it touches a bit on the varied philosophies we all bring to the project. The arguments against mapping the cut blocks as openings in the wood cover seem to fall along the following lines:... and I hope I am not over simplifying or expanding too much: 1) Change is the nature of the beast. When we map wood (or more accurately, forest in OSM terms) it is implied that there will be logging operations taking place within, with the result that the degree of cover is constantly changing as is the maturity of the trees. To try to keep up is fruitless. When one sees a wooded area on a map one should probably just assume that it will be highly variable. 2) There are people who are maintaining this information to a much greater level of detail and accuracy than OSM could ever hope to do. 3) To map them is counterproductive. I have to say that I am not entirely convinced by these well-pointed arguments. 1) Yes, change is what goes on in wooded areas. That's WHY we map them. The value of mapping is often in what we learn by comparing maps from one time period to another. It's important to map so that we can track that change. Any map is simply a snapshot of what exists at a moment in time and the maps themselves are outdated the moment they are made as they are commonly based on outdated data. To use the argument that we shouldn't map a feature simply because it will be out-of-date tomorrow, or next week, or next year I just don't think is convincing. The neighborhood in which I live in Kamloops has changed significantly even in the last two or three years. Houses, businesses and new streets have appeared, trails have disappeared, streams have been diverted. It's a struggle to keep maps up-to-date. But it doesn't mean that it is pointless to map the features as best we can, with the most currently available data we have available to us. 2) The fact that better data exist elsewhere is great. Better data lead to better forest management practices and greater sustainability. But those data are likely unavailable to us or would need to be heavily culled for those relevant to OSM. The City of Kamloops has incredibly detailed maps of the city freely available online. There are overlays for every curb, parking meter and telephone pole. Does that mean that we don't map the city in lesser detail? 3) Would it be counterproductive to map cut-blocks? Well, not counterproductive, but maybe differently productive. I think one of the beauties of
[Talk-ca] Did I just find a potentially significant bug in JOSM?
I've spent the past several hours trying to find out out why some of my edits resulted in some accidental road removals. I think I've found the problem, which seems to relate to an apparent bug in JOSM, or at least a preference somewhere set to something unintuitive that I haven't been able to find - as such, I thought it would be best to communicate with the group to see if anyone's experienced this problem before, or at least give a heads up as to what has been happening to me in case it's inadvertently happening to others. I downloaded an area around the Beaumont, Alberta region to try and add some roads from Canvec that don't exist in OSM (roads which I could have sworn I've added when I worked on the same area a month or two ago). I then saved this downloaded area, along with the changes I made, to an .osm file in JOSM. I ran the Validator which found a few duplicate nodes and such. Telling it to fix errors, it did its thing. And then, I clicked on warnings and told it to fix those. Lo and behold, some (though not all) of the roads I'd just added suddenly disappeared. Since validation was the last step in my process before I uploaded the data, and I was often zoomed all the way out when I did so, I hadn't noticed this happen before, but it probably had been going on right before I uploaded my changes. In particular, sometimes I would replace lower quality OSM roads with better Canvec data, and if JOSM was deleting some of those Canvec roads I'd added in its fix warnings validation step, those original OSM roads would have disappeared without replacements. I tried to isolate further exactly what it was doing, and at least in this situation it looks like it may be related to unnamed ways. I have 41 unnamed ways in my data. If I click on the unnamed ways folder specifically in the warnings validation dialog, it doesn't give me the option to fix them - it shouldn't, since it doesn't know what to call them anyway. But if I click on the parent (root) icon just labelled Warnings, the fix button is enabled, and I thought it was just supposed to fix all the enclosed warnings in categories that it is able to fix. When I click to fix all warnings like this, in the progress bar, I see that it spends some time fixing unnamed ways. And then those ways are gone when it's done. I was using JOSM 3751, so I checked for updates and found that 3961 was available. I've updated my JOSM to see if it's still an issue. But now, when I do the global fix warnings on the data set, it gets part way through, then consistently gives me an unexpected exception (coding error). When I dismiss that dialog, I find that those roads have again been removed, so it seems as though this is still a problem. The unexpected exception isn't encouraging either. I suppose that I can work around the problem by selectively choosing fix in the validations warning area for each of the separate categories, instead of doing them all at once. Though now I'm unsure as to whether or not there is still an underlying problem with fixing warnings that may give rise to an inadvertent loss of data. At the very least, I wanted to make a posting about it so that others can be warned away from having the same problems I have run into. If anyone is interested in trying to duplicate the problems I'm having, just let me know your email address and I can send you the .osm file I'm working with (compressed, it's about 600 kB). And of course, if anyone has any ideas, or suggestions about something stupid I'm doing, please let me know! Dan -- Syzygy Research Technology Box 83, Legal, AB T0G 1L0 Canada Phone: 780-961-2213 ___ Talk-ca mailing list Talk-ca@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-ca
Re: [Talk-ca] Did I just find a potentially significant bug in JOSM?
Dan... Have to admit that I VERY rarely if ever use the fix button, and then only after I've isolated the issue for a specific error or warning. I do errors first, then warnings. But after each correction, I revalidate to refresh the list. Maybe that's overkill, but it keeps the exceptions from being thrown. I think you're getting the exceptions because you've modified the data, but have not updated the list. Usually, I just highlight the specific error or warning, hit the SELECT button, then the 3 key to go there. I fix it manually if I can, then re-validate. Then move to the next issue. I'm too much of a scaredy-cat to highlight a whole class of errors and let JOSM fix them automatically. Like I said, I'm not sure JOSM likes me. I'll try selecting an unnamed way warning and fixing to see what happens. If it removes the way well, that's certainly ONE way to fix it. Sam L. -Original Message- From: Dan Charrois d...@syz.com To: Talk-CA OpenStreetMap talk-ca@openstreetmap.org Subject: [Talk-ca] Did I just find a potentially significant bug in JOSM? Date: Mon, 07 Mar 2011 17:42:09 -0700 I've spent the past several hours trying to find out out why some of my edits resulted in some accidental road removals. I think I've found the problem, which seems to relate to an apparent bug in JOSM, or at least a preference somewhere set to something unintuitive that I haven't been able to find - as such, I thought it would be best to communicate with the group to see if anyone's experienced this problem before, or at least give a heads up as to what has been happening to me in case it's inadvertently happening to others. I downloaded an area around the Beaumont, Alberta region to try and add some roads from Canvec that don't exist in OSM (roads which I could have sworn I've added when I worked on the same area a month or two ago). I then saved this downloaded area, along with the changes I made, to an .osm file in JOSM. I ran the Validator which found a few duplicate nodes and such. Telling it to fix errors, it did its thing. And then, I clicked on warnings and told it to fix those. Lo and behold, some (though not all) of the roads I'd just added suddenly disappeared. Since validation was the last step in my process before I uploaded the data, and I was often zoomed all the way out when I did so, I hadn't noticed this happen before, but it probably had been going on right before I uploaded my changes. In particular, sometimes I would replace lower quality OSM roads with better Canvec data, and if JOSM was deleting some of those Canvec roads I'd added in its fix warnings validation step, those original OSM roads would have disappeared without replacements. I tried to isolate further exactly what it was doing, and at least in this situation it looks like it may be related to unnamed ways. I have 41 unnamed ways in my data. If I click on the unnamed ways folder specifically in the warnings validation dialog, it doesn't give me the option to fix them - it shouldn't, since it doesn't know what to call them anyway. But if I click on the parent (root) icon just labelled Warnings, the fix button is enabled, and I thought it was just supposed to fix all the enclosed warnings in categories that it is able to fix. When I click to fix all warnings like this, in the progress bar, I see that it spends some time fixing unnamed ways. And then those ways are gone when it's done. I was using JOSM 3751, so I checked for updates and found that 3961 was available. I've updated my JOSM to see if it's still an issue. But now, when I do the global fix warnings on the data set, it gets part way through, then consistently gives me an unexpected exception (coding error). When I dismiss that dialog, I find that those roads have again been removed, so it seems as though this is still a problem. The unexpected exception isn't encouraging either. I suppose that I can work around the problem by selectively choosing fix in the validations warning area for each of the separate categories, instead of doing them all at once. Though now I'm unsure as to whether or not there is still an underlying problem with fixing warnings that may give rise to an inadvertent loss of data. At the very least, I wanted to make a posting about it so that others can be warned away from having the same problems I have run into. If anyone is interested in trying to duplicate the problems I'm having, just let me know your email address and I can send you the .osm file I'm working with (compressed, it's about 600 kB). And of course, if anyone has any ideas, or suggestions about something stupid I'm doing, please let me know! Dan -- Syzygy Research Technology Box 83, Legal, AB T0G 1L0 Canada Phone: 780-961-2213 ___ Talk-ca mailing list Talk-ca@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-ca
[Talk-ca] Reporting Canvec errors
Hi I've racked up a list of errors is Canvec data. We've probably gone over this before, but how do I report errors and out of date information in Canvec. Sam Dyck ___ Talk-ca mailing list Talk-ca@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-ca
Re: [Talk-ca] Did I just find a potentially significant bug in JOSM?
No, if I highlight an unnamed way warning, and select it, the fix key is grayed out. That makes sense. Sam L. -Original Message- From: Samuel Longiaru longi...@shaw.ca To: Dan Charrois d...@syz.com Cc: Talk-CA OpenStreetMap talk-ca@openstreetmap.org Subject: Re: [Talk-ca] Did I just find a potentially significant bug in JOSM? Date: Mon, 07 Mar 2011 18:13:37 -0800 Dan... Have to admit that I VERY rarely if ever use the fix button, and then only after I've isolated the issue for a specific error or warning. I do errors first, then warnings. But after each correction, I revalidate to refresh the list. Maybe that's overkill, but it keeps the exceptions from being thrown. I think you're getting the exceptions because you've modified the data, but have not updated the list. Usually, I just highlight the specific error or warning, hit the SELECT button, then the 3 key to go there. I fix it manually if I can, then re-validate. Then move to the next issue. I'm too much of a scaredy-cat to highlight a whole class of errors and let JOSM fix them automatically. Like I said, I'm not sure JOSM likes me. I'll try selecting an unnamed way warning and fixing to see what happens. If it removes the way well, that's certainly ONE way to fix it. Sam L. -Original Message- From: Dan Charrois d...@syz.com To: Talk-CA OpenStreetMap talk-ca@openstreetmap.org Subject: [Talk-ca] Did I just find a potentially significant bug in JOSM? Date: Mon, 07 Mar 2011 17:42:09 -0700 I've spent the past several hours trying to find out out why some of my edits resulted in some accidental road removals. I think I've found the problem, which seems to relate to an apparent bug in JOSM, or at least a preference somewhere set to something unintuitive that I haven't been able to find - as such, I thought it would be best to communicate with the group to see if anyone's experienced this problem before, or at least give a heads up as to what has been happening to me in case it's inadvertently happening to others. I downloaded an area around the Beaumont, Alberta region to try and add some roads from Canvec that don't exist in OSM (roads which I could have sworn I've added when I worked on the same area a month or two ago). I then saved this downloaded area, along with the changes I made, to an .osm file in JOSM. I ran the Validator which found a few duplicate nodes and such. Telling it to fix errors, it did its thing. And then, I clicked on warnings and told it to fix those. Lo and behold, some (though not all) of the roads I'd just added suddenly disappeared. Since validation was the last step in my process before I uploaded the data, and I was often zoomed all the way out when I did so, I hadn't noticed this happen before, but it probably had been going on right before I uploaded my changes. In particular, sometimes I would replace lower quality OSM roads with better Canvec data, and if JOSM was deleting some of those Canvec roads I'd added in its fix warnings validation step, those original OSM roads would have disappeared without replacements. I tried to isolate further exactly what it was doing, and at least in this situation it looks like it may be related to unnamed ways. I have 41 unnamed ways in my data. If I click on the unnamed ways folder specifically in the warnings validation dialog, it doesn't give me the option to fix them - it shouldn't, since it doesn't know what to call them anyway. But if I click on the parent (root) icon just labelled Warnings, the fix button is enabled, and I thought it was just supposed to fix all the enclosed warnings in categories that it is able to fix. When I click to fix all warnings like this, in the progress bar, I see that it spends some time fixing unnamed ways. And then those ways are gone when it's done. I was using JOSM 3751, so I checked for updates and found that 3961 was available. I've updated my JOSM to see if it's still an issue. But now, when I do the global fix warnings on the data set, it gets part way through, then consistently gives me an unexpected exception (coding error). When I dismiss that dialog, I find that those roads have again been removed, so it seems as though this is still a problem. The unexpected exception isn't encouraging either. I suppose that I can work around the problem by selectively choosing fix in the validations warning area for each of the separate categories, instead of doing them all at once. Though now I'm unsure as to whether or not there is still an underlying problem with fixing warnings that may give rise to an inadvertent loss of data. At the very least, I wanted to make a posting about it so that others can be warned away from having the same problems I have run into. If anyone is interested in trying to duplicate the problems I'm having, just let me know your email address and I can send you the .osm file I'm working with (compressed, it's about
Re: [Talk-ca] Did I just find a potentially significant bug in JOSM?
Thats funny, because I submitted a bug just a few hours ago because the latest (dev) build of Validator + JOSm wouldnt merge duplicate nodes on a canvec boundary.. Anyways, clearly JOSM is a work in progress, Russell On 2011-03-07, at 6:23 PM, Samuel Longiaru longi...@shaw.ca wrote: No, if I highlight an unnamed way warning, and select it, the fix key is grayed out. That makes sense. Sam L. -Original Message- *From*: Samuel Longiaru longi...@shaw.casamuel%20longiaru%20%3clongi...@shaw.ca%3e *To*: Dan Charrois d...@syz.com dan%20charrois%20%3c...@syz.com%3e *Cc*: Talk-CA OpenStreetMap talk-ca@openstreetmap.orgtalk-ca%20openstreetmap%20%3ctalk...@openstreetmap.org%3e *Subject*: Re: [Talk-ca] Did I just find a potentially significant bug in JOSM? *Date*: Mon, 07 Mar 2011 18:13:37 -0800 Dan... Have to admit that I VERY rarely if ever use the fix button, and then only after I've isolated the issue for a specific error or warning. I do errors first, then warnings. But after each correction, I revalidate to refresh the list. Maybe that's overkill, but it keeps the exceptions from being thrown. I think you're getting the exceptions because you've modified the data, but have not updated the list. Usually, I just highlight the specific error or warning, hit the SELECT button, then the 3 key to go there. I fix it manually if I can, then re-validate. Then move to the next issue. I'm too much of a scaredy-cat to highlight a whole class of errors and let JOSM fix them automatically. Like I said, I'm not sure JOSM likes me. I'll try selecting an unnamed way warning and fixing to see what happens. If it removes the way well, that's certainly ONE way to fix it. Sam L. -Original Message- *From*: Dan Charrois d...@syz.com dan%20charrois%20%3c...@syz.com%3e *To*: Talk-CA OpenStreetMap talk-ca@openstreetmap.orgtalk-ca%20openstreetmap%20%3ctalk...@openstreetmap.org%3e *Subject*: [Talk-ca] Did I just find a potentially significant bug in JOSM? *Date*: Mon, 07 Mar 2011 17:42:09 -0700 I've spent the past several hours trying to find out out why some of my edits resulted in some accidental road removals. I think I've found the problem, which seems to relate to an apparent bug in JOSM, or at least a preference somewhere set to something unintuitive that I haven't been able to find - as such, I thought it would be best to communicate with the group to see if anyone's experienced this problem before, or at least give a heads up as to what has been happening to me in case it's inadvertently happening to others. I downloaded an area around the Beaumont, Alberta region to try and add some roads from Canvec that don't exist in OSM (roads which I could have sworn I've added when I worked on the same area a month or two ago). I then saved this downloaded area, along with the changes I made, to an .osm file in JOSM. I ran the Validator which found a few duplicate nodes and such. Telling it to fix errors, it did its thing. And then, I clicked on warnings and told it to fix those. Lo and behold, some (though not all) of the roads I'd just added suddenly disappeared. Since validation was the last step in my process before I uploaded the data, and I was often zoomed all the way out when I did so, I hadn't noticed this happen before, but it probably had been going on right before I uploaded my changes. In particular, sometimes I would replace lower quality OSM roads with better Canvec data, and if JOSM was deleting some of those Canvec roads I'd added in its fix warnings validation step, those original OSM roads would have disappeared without replacements. I tried to isolate further exactly what it was doing, and at least in this situation it looks like it may be related to unnamed ways. I have 41 unnamed ways in my data. If I click on the unnamed ways folder specifically in the warnings validation dialog, it doesn't give me the option to fix them - it shouldn't, since it doesn't know what to call them anyway. But if I click on the parent (root) icon just labelled Warnings, the fix button is enabled, and I thought it was just supposed to fix all the enclosed warnings in categories that it is able to fix. When I click to fix all warnings like this, in the progress bar, I see that it spends some time fixing unnamed ways. And then those ways are gone when it's done. I was using JOSM 3751, so I checked for updates and found that 3961 was available. I've updated my JOSM to see if it's still an issue. But now, when I do the global fix warnings on the data set, it gets part way through, then consistently gives me an unexpected exception (coding error). When I dismiss that dialog, I find that those roads have again been removed, so it seems as though this is still a problem. The unexpected exception isn't encouraging either. I suppose that I can work around the problem by selectively choosing fix in the validations warning area for each of the separate categories, instead of doing them
Re: [Talk-ca] Did I just find a potentially significant bug in JOSM?
On Mon, Mar 7, 2011 at 9:39 PM, Russell P skiinf...@gmail.com wrote: Thats funny, because I submitted a bug just a few hours ago because the latest (dev) build of Validator + JOSm wouldnt merge duplicate nodes on a canvec boundary.. Anyways, clearly JOSM is a work in progress, Yes, it is probably worth making a bug report for both of these. Please note that josm has it's own trac, not on osm.org. http://josm.openstreetmap.de/ ___ Talk-ca mailing list Talk-ca@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-ca
Re: [Talk-ca] Did I just find a potentially significant bug in JOSM?
Sam wrote I think you're getting the exceptions because you've modified the data, but have not updated the list. This is definitely an issue I have encountered with Validator. If you run the validator to check for errors, then delete some nodes or ways yourself, then try to use validator's auto-fix, it will attempt to modify objects that you have since deleted. This causes it great confusion and can mess things up. Validator tends to be good about maintaining internal consistency, but not external. I don't do blanket fixes like Dan, I like to drill down a level or two like Sam, just to make sure that Validator is doing things properly. If you do open a ticket for this, be sure to let us know about it (ticket number). Adam On Mon, Mar 7, 2011 at 7:30 PM, Richard Weait rich...@weait.com wrote: On Mon, Mar 7, 2011 at 9:39 PM, Russell P skiinf...@gmail.com wrote: Thats funny, because I submitted a bug just a few hours ago because the latest (dev) build of Validator + JOSm wouldnt merge duplicate nodes on a canvec boundary.. Anyways, clearly JOSM is a work in progress, Yes, it is probably worth making a bug report for both of these. Please note that josm has it's own trac, not on osm.org. http://josm.openstreetmap.de/ ___ Talk-ca mailing list Talk-ca@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-ca ___ Talk-ca mailing list Talk-ca@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-ca
Re: [Talk-ca] Did I just find a potentially significant bug in JOSM?
Thanks Adam, and everyone else who responded to my message. I'm glad to learn
the bug is repeatable by others - that means I'm not going completely crazy.
Plus, it's nice to have a good understanding now why some of the ways I'd
uploaded had mysteriously disappeared.
I'm glad you were able to distill down the problem so succinctly.
validatorfail.osm shows off the bug well, and is much easier to follow what's
going on.
I'd apparently been lulled into trusting JOSM's ability to fix simple things
like duplicate nodes a little too much. It's definitely a time-saver when it
works, but not if roads are lost in the process. I'm hoping that a slightly
revised procedure of fixing things just one category at a time (rather than at
the top-level Warnings category) should be a good workaround.
In the meantime, I agree that a bug report should definitely be filed about
this. At the very least, disabling top-level fixes would force people to go
through category by category. I'm sure I'm not the only one who has trusted
(or will trust) that the top-level fix is a good way to reduce the manual
workload. Your validatorfail.osm file is a perfect example of how to reproduce
the problem, and should probably be included with the bug report - I can
definitely try to figure out how to submit the bug report to
http://josm.openstreetmap.de/ if you're busy or no one else has yet, and you
don't mind my including the file. But as the person who really narrowed down
the issue, the honor for reporting it (if there is such a thing) should be
yours.. :-)
In any case, those blanket fixes will be a thing of the past for me - I'm
hoping that the category-by-category approach doesn't cause any issues.
Other than this hiccup, I've found JOSM to be quite intuitive and reliable,
though as was mentioned, it is a work in progress and likely always will be.
It's just a matter of finding where its specific weaknesses are, and then
avoiding those.
Dan
On 2011-Mar-07, at 8:31 PM, Adam Dunn wrote:
Just reproduced this issue with JOSM 3751. Most warnings are not
auto-fixable in Validator. I have only been able to find two warn
level issues that validator will auto-fix: Nodes at same position, and
duplicated nodes.
As stated by Dan, clicking on a sublevel within Warn (such as unnamed
highway) will disable auto-fix, but selecting top level Warn will
enable the auto-fix button (if there is a duplicate node sublevel).
Using auto-fix when there are both duplicate nodes and unnamed
highways will cause the dupe node *and* the unnamed highway to be
deleted.
Method to reproduce (in 3751):
1 - Initialize a blank layer in JOSM (eg. download middle of ocean or
open an empty .osm file).
2 - Create a way that is of highway={residential,secondary,etc
(whatever is necessary to raise a unnamed highway warning, not
road)}
3 - Create a node (no tags necessary)
4 - Copy and paste node into same position (zoom in excessively to
position duplicated node on top of original)
5 - Run Validator
6 - Click on top-level Warnings category, then click on the Fix button
7 - Watch road disappear
Alternative method:
1 - Download attached .osm file (this is a minimal file to reproduce
the bug, and is easily human readable in any text editor)
2 - Run Validator
3 - Click on Warnings category, then click Fix button
4 - Watch road disappear
Adam
-Original Message-
From: Samuel Longiaru longi...@shaw.ca
To: Dan Charrois d...@syz.com
Cc: Talk-CA OpenStreetMap talk-ca@openstreetmap.org
Subject: Re: [Talk-ca] Did I just find a potentially significant bug in
JOSM?
Date: Mon, 07 Mar 2011 18:13:37 -0800
Dan...
Have to admit that I VERY rarely if ever use the fix button, and then only
after I've isolated the issue for a specific error or warning. I do errors
first, then warnings. But after each correction, I revalidate to refresh
the list. Maybe that's overkill, but it keeps the exceptions from being
thrown. I think you're getting the exceptions because you've modified the
data, but have not updated the list. Usually, I just highlight the specific
error or warning, hit the SELECT button, then the 3 key to go there. I
fix it manually if I can, then re-validate. Then move to the next issue.
I'm too much of a scaredy-cat to highlight a whole class of errors and let
JOSM fix them automatically. Like I said, I'm not sure JOSM likes me.
I'll try selecting an unnamed way warning and fixing to see what happens.
If it removes the way well, that's certainly ONE way to fix it.
Sam L.
-Original Message-
From: Dan Charrois d...@syz.com
To: Talk-CA OpenStreetMap talk-ca@openstreetmap.org
Subject: [Talk-ca] Did I just find a potentially significant bug in JOSM?
Date: Mon, 07 Mar 2011 17:42:09 -0700
I've spent the past several hours trying to find out out why some of my
edits resulted in some accidental road removals. I think I've found the
problem, which seems
