Re: [Talk-us] Fixing TIGER street name abbreviations
On Mon, Apr 30, 2012 at 11:28 PM, Serge Wroclawski emac...@gmail.com wrote: On Mon, Apr 30, 2012 at 8:14 PM, Paul Johnson ba...@ursamundi.org wrote: There have been some limited automated expansions, though they can be problematic, because abbreviations can mean many possible things. Expanding abbreviations requires a bit of a human touch. Creating abbreviations in the renderer when so desired, not so much. This is true, but if one is talking about the TIGER data, there are a number of hints that can make this problem virtually nil. There's a tag tiger:name_type key that contains the value of the expandable name section, eg. St or Ln or Pky. AFAIK these are always expandable to Street, Lane and Parkway. And of course one must only expand the name_tag value if it's the last component of the name string, eg. Ln Ln should be Ln Lane. This should be fairly easy to construct in a regex, but one should be careful of it. Those two rules should eliminate a vast majority of expansion issues. If we only expand TIGER data, then it should be a fairly straightforward process. Of course such a script should be peer reviewed and tested, but I'm confident that the error rate will be very low. I guess this would be okay, so long as it gets peer reviewed and tested by a group including you. And for those few exceptions where the expansion is wrong, a human review process will turn this up and make it fairly correctable. In fact, I'd argue that the problems won't be subtle, making them easy to spot and fix. How would the human review process work? Isn't it better to do the review *before* editing the database? In return, we'll save hundreds, maybe thousands of man hours doing expansions. Useless expansions, though. ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Fixing TIGER street name abbreviations
On Tue, May 1, 2012 at 1:06 PM, Serge Wroclawski emac...@gmail.com wrote: The other point that's being missed is that we as a community already accept an error rate in our data that's far larger than any potential mistake rate on a well written script. If the script makes one error in 1000 streets, it will be doing a better job than a vast majority of manual mappers, and like manual mappers, they can be corrected. If someone manually expanded 1,000,000 street name abbreviations, and made 1,000 mistakes, it would not be acceptable. If they were doing something more useful than expanding street name abbreviations, fine. But expanding street name abbreviations, according to a very simple heuristic which can easily be done at the preprocessing stage, is not very useful. If this is going to be done, I hope the error rate is much smaller than 1 in 1,000. ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Fixing TIGER street name abbreviations
On 5/1/2012 12:59 PM, Anthony wrote: I'm not sure what you're saying. Automatically expanding abbreviations is a terrible idea. If an abbreviation is unambiguous, then it can be expanded during the preprocessing step. If, on the other hand, it is ambiguous, then you are turning ambiguous data into incorrect data, which certainly diminishes the data. What preprocessing step? TIGER data has already been imported. The types of errors I'm referring to are where you go to upload from JOSM, then decide to slavishly submit to the validator's warnings about abbreviated street names. What person manually types 2 - 3 dozen versions of Street , Avenue, Boulevard, Point, Circle without any typos? ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Fixing TIGER street name abbreviations
On Tue, May 1, 2012 at 1:18 PM, Nathan Edgars II nerou...@gmail.com wrote: On 5/1/2012 12:59 PM, Anthony wrote: Automatically expanding abbreviations is a terrible idea. If an abbreviation is unambiguous, then it can be expanded during the preprocessing step. If, on the other hand, it is ambiguous, then you are turning ambiguous data into incorrect data, which certainly diminishes the data. Not quite. We have various TIGER tags that break the name into pieces, and allow automated expansion where the name field may be ambiguous. (Though occasionally these tags are wrong.) I'm not sure what you're disagreeing with. Either it is unambiguous (due to TIGER tags or whatever), and therefore can be done during the preprocessing step. Or it is ambiguous, and needs human intervention/review. ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Fixing TIGER street name abbreviations
On 5/1/2012 1:23 PM, Anthony wrote: On Tue, May 1, 2012 at 1:18 PM, Nathan Edgars IInerou...@gmail.com wrote: On 5/1/2012 12:59 PM, Anthony wrote: Automatically expanding abbreviations is a terrible idea. If an abbreviation is unambiguous, then it can be expanded during the preprocessing step. If, on the other hand, it is ambiguous, then you are turning ambiguous data into incorrect data, which certainly diminishes the data. Not quite. We have various TIGER tags that break the name into pieces, and allow automated expansion where the name field may be ambiguous. (Though occasionally these tags are wrong.) I'm not sure what you're disagreeing with. Either it is unambiguous (due to TIGER tags or whatever), and therefore can be done during the preprocessing step. Or it is ambiguous, and needs human intervention/review. The TIGER tags are not exactly standard OSM tags that belong in the database. Better that we get rid of them at the same time as we expand abbreviations. ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Fixing TIGER street name abbreviations
On Tue, May 1, 2012 at 1:26 PM, Nathan Edgars II nerou...@gmail.com wrote: On 5/1/2012 1:23 PM, Anthony wrote: On Tue, May 1, 2012 at 1:18 PM, Nathan Edgars IInerou...@gmail.com wrote: On 5/1/2012 12:59 PM, Anthony wrote: Automatically expanding abbreviations is a terrible idea. If an abbreviation is unambiguous, then it can be expanded during the preprocessing step. If, on the other hand, it is ambiguous, then you are turning ambiguous data into incorrect data, which certainly diminishes the data. Not quite. We have various TIGER tags that break the name into pieces, and allow automated expansion where the name field may be ambiguous. (Though occasionally these tags are wrong.) I'm not sure what you're disagreeing with. Either it is unambiguous (due to TIGER tags or whatever), and therefore can be done during the preprocessing step. Or it is ambiguous, and needs human intervention/review. The TIGER tags are not exactly standard OSM tags that belong in the database. Better that we get rid of them at the same time as we expand abbreviations. On that point, I strongly agree. And actually, if the bot is going to be smart enough to look at the history, to find deleted TIGER tags, then maybe there is some advantage to doing this during the preprocessing step (which would often not have access to history data). ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Fixing TIGER street name abbreviations
On Tue, May 1, 2012 at 1:31 PM, Anthony o...@inbox.org wrote: And actually, if the bot is going to be smart enough to look at the history, to find deleted TIGER tags, then maybe there is some advantage to doing this during the preprocessing step (which would often not have access to history data). What I mean is that, if the bot is going to look at the history, then there would be an advantage to letting the bot run. But I am assuming this could be done with much less than a 1/1000 error rate. 1/10,000 would maybe be acceptable. 1/100,000 would be okay. ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Fixing TIGER street name abbreviations
On 5/1/2012 1:21 PM, Anthony wrote: The preprocessing step between downloading the data from OSM and doing something with it. That assumes that the TIGER tags will always be present to assist with proper automatic expansion. And I'd rather have the US data in line with the world-wide OSM data where it makes sense. That way the US can consume OSM US data with tools developed worldwide, without the tool writers needing to implement US-specific rules. After analysis, most of the US opinions fall on the side of no abbreviations. ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Fixing TIGER street name abbreviations
On Tue, May 1, 2012 at 12:26 PM, Nathan Edgars II nerou...@gmail.comwrote: The TIGER tags are not exactly standard OSM tags that belong in the database. Better that we get rid of them at the same time as we expand abbreviations. Although the tiger:* keys aren't standard, the information they store is very useful. There are plenty of people that might want to know the different parts of a road name, so we should simply rename these tags instead of completely blowing the data away. ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
[Talk-us] What happened to the license-change highlighting in Potlatch 2?
Hello all, For the past couple of days in Potlatch 2, the ways and points done by nonagreers to the license have been highlighted in red and orange. I thought, At last, a useful tool for remapping! Considering most of downtown LA has to be redone, it was a blessing to have it. Now, this morning, I had to sign in again, and the highlighting is gone. What happened? It's not like we have an abundance of tools to help us with the remapping. Is this just more of the lack of communication, organization and direction that has characterized the effort so far? Charlotte Charlotte Wolter 927 18th Street Suite A Santa Monica, California 90403 +1-310-597-4040 techl...@techlady.com Skype: thetechlady The Four Internet Freedoms Freedom to visit any site on the Internet Freedom to access any content or service that is not illegal Freedom to attach any device that does not interfere with the network Freedom to know all the terms of a service, particularly any that would affect the first three freedoms. ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] What happened to the license-change highlighting in Potlatch 2?
I think it was added a while back. I mostly use josm. I like the license check plugin. It seems easier to use than potlatch which seems to highlight every possible license issue. With josm you just click on an item on the list, right click to zoom to problem on the node. Then I usually clean up all of the items near by. When your done with all the items the area is clean. The plugin gives three categories of license issues, Data Loss, Possible Data Loss, and Harmless Data Loss. I've been concentrating of the first category. There are so many Data Loss nodes that I'll never get done by the time the bot starts running. (Even if its another couple of months!) josm has more of a learning curve, but better features, cutting and pasting for example. I haven't figured out how to assign actions to a key yet. I have been adding turning circles and pedestrian cross walks as I happen on them. It would be nice to be able to hit a key to add a turning circle to the end of a way. Clifford On Tue, May 1, 2012 at 12:38 PM, Charlotte Wolter techl...@techlady.comwrote: ****Got it! Many thanks, Clifford. ****By the way, has Show license status been on the Options list for a while? I don't remember seeing it a couple of days ago, when I used it to update something. Also, was there ever a formal announcement that this capability was available? Charlotte At 11:23 AM 5/1/2012, you wrote: Check the options. There must have been an update to potlatch which reverts the options back to a default. Mine was the same way just now. Clifford On Tue, May 1, 2012 at 11:11 AM, Charlotte Wolter techl...@techlady.com wrote: Hello all, For the past couple of days in Potlatch 2, the ways and points done by nonagreers to the license have been highlighted in red and orange. I thought, At last, a useful tool for remapping! Considering most of downtown LA has to be redone, it was a blessing to have it. Now, this morning, I had to sign in again, and the highlighting is gone. What happened? It's not like we have an abundance of tools to help us with the remapping. Is this just more of the lack of communication, organization and direction that has characterized the effort so far? Charlotte Charlotte Wolter 927 18th Street Suite A Santa Monica, California 90403 +1-310-597-4040 techl...@techlady.com Skype: thetechlady The Four Internet Freedoms Freedom to visit any site on the Internet Freedom to access any content or service that is not illegal Freedom to attach any device that does not interfere with the network Freedom to know all the terms of a service, particularly any that would affect the first three freedoms. ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us -- Clifford I have promised to cut down on my swearing and drinking, which I have. Unfortunately, this has left me dim-witted and nearly speechless. Adapted from The Lion by Nelson DeMille -or- If you can't explain it simply, you don't understand it well enough. Albert Einstein ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us ** ** Charlotte Wolter 927 18th Street Suite A Santa Monica, California 90403 +1-310-597-4040 techl...@techlady.com Skype: thetechlady *The Four Internet Freedoms* Freedom to visit any site on the Internet Freedom to access any content or service that is not illegal Freedom to attach any device that does not interfere with the network Freedom to know all the terms of a service, particularly any that would affect the first three freedoms. -- Clifford I have promised to cut down on my swearing and drinking, which I have. Unfortunately, this has left me dim-witted and nearly speechless. Adapted from *The Lion* by Nelson DeMille -or- If you can't explain it simply, you don't understand it well enough. Albert Einstein ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Fixing TIGER street name abbreviations
On Tue, May 1, 2012 at 1:36 PM, Mike N nice...@att.net wrote: On 5/1/2012 1:21 PM, Anthony wrote: The preprocessing step between downloading the data from OSM and doing something with it. That assumes that the TIGER tags will always be present to assist with proper automatic expansion. I'm not sure what you mean, because I am not making that assumption at all. And I'd rather have the US data in line with the world-wide OSM data where it makes sense. That way the US can consume OSM US data with tools developed worldwide, without the tool writers needing to implement US-specific rules. After analysis, most of the US opinions fall on the side of no abbreviations. I don't think anyone in this thread is arguing against expanding abbreviations. The question is whether or not it's okay for a bot to expand abbreviations. And to a large extent that depends on how accurate the bot will be. If the bot is sure to be 100% accurate, then hey, no problem. But I don't believe that is the case. ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
Re: [Talk-us] Fixing TIGER street name abbreviations
On Tue, May 1, 2012 at 1:41 PM, Ian Dees ian.d...@gmail.com wrote: On Tue, May 1, 2012 at 12:26 PM, Nathan Edgars II nerou...@gmail.com wrote: The TIGER tags are not exactly standard OSM tags that belong in the database. Better that we get rid of them at the same time as we expand abbreviations. Although the tiger:* keys aren't standard, the information they store is very useful. There are plenty of people that might want to know the different parts of a road name, so we should simply rename these tags instead of completely blowing the data away. I guess that's okay too, though personally I get so annoyed by the redundant data (*) that I couldn't be bothered. Why street relations never caught on is beyond me. (*) I.E. adding base_name=Main to the 100 different ways that Main Street is split up into. ___ Talk-us mailing list Talk-us@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk-us
