Re: [OSM-talk] Changing capitalization (Lima)
Igor added a export-tag command to Maperitive that creates a local CSV file that can be brought into a spreadsheet. You need a local copy of the .OSM file or possibly .pbf version should work as well. It's very useful for spotting tags that are misspelt. Once there you can concat red tape to create xml commands to get JOSM to upload the changes. I've a couple of sample VB programs that run done a local file and make changes, a sort of off line bot if you like but I wouldn't like them to escape into general use there are too many bots out there at the moment. Cheerio John On 1 June 2012 22:05, Jaakko Helleranta.com wrote: > (Added HOT to the list as the issue here is probably of interest In the > areas where HOT is active.) > > Andrzej, > What was your process / what tools did you use? > > In general: > I've dreamed of solutions to fix not only capitalization errors / fuckups > but also various other typing errors / mistakes in names to use especvially > in Haiti where I've spent countless hours trying to combat things that > various helping hands are creating on a steady basis. > > My dream solution would be a simple(?) export-import tool that would make > it possible to: > 1) Export desired tag values (1 at a time) to csv / > your_favorite_spreadsheet_format > 2) Fix the typos/etc errors in that give data > 3) Import/Upload the fixed data back to OSM. > > As Alex originally questioned/thought_aloud fixing of names _could_ happen > simply by editing the xml file manually -- but you'd need to always > remember to add the "edited" tag into the right place for the edit to work > / get registered in the data upload. > .. And such data manipulation is simply _so_ much easier in a spreadsheet > application -- and can be outsourced to people who understand nothing about > geo/tech tools but who know how to fix CapiTAliZaTi0n errors / typos. > > The additional reason I'd _love_ to have such tool is that it could make > it possible to create a simple list of eg all schools, hospitals, > restaurants, what_not within a given osm file. > > .. Especially convenient would be if the developer of such tool would make > it possible to export / list more than just one field into the > csv/spreadsheet file; say, name, address, phone, email, website, > description, etc. > > Adding and updating such attribute data would imo be _so_ much easier / > more efficient in a spreadsheet than one-by-one in JOSM/Potlacth/etc. > > If someone would have a Kickstarter/etc project to do this (or can point > me to an existing such solution) I'd be very much willing to contribute, > say, $100-200 depending on the richness of features of such tool. > > Cheers, > -Jaakko > > Sent from my BlackBerry® device from Digicel > -- > Mobile: +509-37-26 91 54, Skype/GoogleTalk: jhelleranta > > -Original Message- > From: andrzej zaborowski > Date: Fri, 1 Jun 2012 16:34:21 > To: Alex Barth > Cc: openstreetmap; > Subject: Re: [OSM-talk] Changing capitalization (Lima) > > On 31 May 2012 17:39, Alex Barth wrote: > > We're currently working with Ruben (user Rub21) on fixing street name > capitalization in Lima - a lot of the street names are ALL CAPS where they > should be properly capitalized. We're doing this work manually right now > and are well under way. It's quite time intensive though - any examples of > where such a cleanup process has been automated on OSM before? > > > > I ran such a process on the POI names in Girona that were imported > just before the SOTM'10. Accents were correct already and python > dealt with them correctly. The only special cases were some > prepositions that are written in lower case and the Catalan use of > apostrophe. > > http://www.openstreetmap.org/browse/changeset/5073672 is one of the > changesets. > > Cheers > > ___ > talk mailing list > talk@openstreetmap.org > http://lists.openstreetmap.org/listinfo/talk > ___ > talk mailing list > talk@openstreetmap.org > http://lists.openstreetmap.org/listinfo/talk > ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Changing capitalization (Lima)
(Added HOT to the list as the issue here is probably of interest In the areas where HOT is active.) Andrzej, What was your process / what tools did you use? In general: I've dreamed of solutions to fix not only capitalization errors / fuckups but also various other typing errors / mistakes in names to use especvially in Haiti where I've spent countless hours trying to combat things that various helping hands are creating on a steady basis. My dream solution would be a simple(?) export-import tool that would make it possible to: 1) Export desired tag values (1 at a time) to csv / your_favorite_spreadsheet_format 2) Fix the typos/etc errors in that give data 3) Import/Upload the fixed data back to OSM. As Alex originally questioned/thought_aloud fixing of names _could_ happen simply by editing the xml file manually -- but you'd need to always remember to add the "edited" tag into the right place for the edit to work / get registered in the data upload. .. And such data manipulation is simply _so_ much easier in a spreadsheet application -- and can be outsourced to people who understand nothing about geo/tech tools but who know how to fix CapiTAliZaTi0n errors / typos. The additional reason I'd _love_ to have such tool is that it could make it possible to create a simple list of eg all schools, hospitals, restaurants, what_not within a given osm file. .. Especially convenient would be if the developer of such tool would make it possible to export / list more than just one field into the csv/spreadsheet file; say, name, address, phone, email, website, description, etc. Adding and updating such attribute data would imo be _so_ much easier / more efficient in a spreadsheet than one-by-one in JOSM/Potlacth/etc. If someone would have a Kickstarter/etc project to do this (or can point me to an existing such solution) I'd be very much willing to contribute, say, $100-200 depending on the richness of features of such tool. Cheers, -Jaakko Sent from my BlackBerry® device from Digicel -- Mobile: +509-37-26 91 54, Skype/GoogleTalk: jhelleranta -Original Message- From: andrzej zaborowski Date: Fri, 1 Jun 2012 16:34:21 To: Alex Barth Cc: openstreetmap; Subject: Re: [OSM-talk] Changing capitalization (Lima) On 31 May 2012 17:39, Alex Barth wrote: > We're currently working with Ruben (user Rub21) on fixing street name > capitalization in Lima - a lot of the street names are ALL CAPS where they > should be properly capitalized. We're doing this work manually right now and > are well under way. It's quite time intensive though - any examples of where > such a cleanup process has been automated on OSM before? > I ran such a process on the POI names in Girona that were imported just before the SOTM'10. Accents were correct already and python dealt with them correctly. The only special cases were some prepositions that are written in lower case and the Catalan use of apostrophe. http://www.openstreetmap.org/browse/changeset/5073672 is one of the changesets. Cheers ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Changing capitalization (Lima)
Wow, thanks so much - I'm running out of time right now and can't look at the file. How could I prep such a file myself? On May 31, 2012, at 3:13 PM, Worst Fixer wrote: > Hello. > > 2012/5/31 Alex Barth : >> We're currently working with Ruben (user Rub21) on fixing street name >> capitalization in Lima - a lot of the street names are ALL CAPS where they >> should be properly capitalized. We're doing this work manually right now and >> are well under way. It's quite time intensive though - any examples of where >> such a cleanup process has been automated on OSM before? > > I appreciate your fight with bad tags and bad imports. > > I prepared a file with edits for highways in Lima area. > > http://worstfixer.000a.biz/y2-lima-hw/edits-to-upload.osm.gz > > Overview: > http://worstfixer.000a.biz/y2-lima-hw/overview-short.html > http://worstfixer.000a.biz/y2-lima-hw/overview-full.html > > Ich also deleted source= if it is the only tag on nodes, and removed > single addr:street instance when it was equal to name tag. > > If you notice some big flaw in my case change algoritm, mail me > privately. Ich will re-make file for you. > > Please review it and upload. If you want to upload it without complete > review, please follow Mechanical Edits Policy: > > http://wiki.openstreetmap.org/wiki/Mechanical_Edit_Policy > >> tel (202) 250-3633 > > This ist world wide international list. Write phone numbers in > international format. > > -- > WorstFixer, twitter: http://twitter.com/WorstFixer Alex Barth http://twitter.com/lxbarth tel (202) 250-3633 ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Changing capitalization (Lima)
On 31 May 2012 17:39, Alex Barth wrote: > We're currently working with Ruben (user Rub21) on fixing street name > capitalization in Lima - a lot of the street names are ALL CAPS where they > should be properly capitalized. We're doing this work manually right now and > are well under way. It's quite time intensive though - any examples of where > such a cleanup process has been automated on OSM before? > I ran such a process on the POI names in Girona that were imported just before the SOTM'10. Accents were correct already and python dealt with them correctly. The only special cases were some prepositions that are written in lower case and the Catalan use of apostrophe. http://www.openstreetmap.org/browse/changeset/5073672 is one of the changesets. Cheers ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Changing capitalization (Lima)
Steve Doerr wrote: On 31/05/2012 20:13, Worst Fixer wrote: If you notice some big flaw in my case change algoritm, mail me privately. I'd only comment that a really intelligent de-capitalization algorithm would attempt to supply the accents that are missing from the capitalized forms. I had to do this for a (non-OSM) Canadian dataset where the English placenames were in mixed-case but the French (Quebecois) were in ALL CAPS - presumably done deliberately to avoid the issue of accents. The best approach is probably to use a dictionary generated from other placenames. In the case of OSM we already have a great source of such data. I could probably do this for you, or at least provide a mapping table that you could combine with your current conversion script. But not until next week. Let me know if you're interested. Regards, Phil. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Changing capitalization (Lima)
On 31/05/2012 20:13, Worst Fixer wrote: If you notice some big flaw in my case change algoritm, mail me privately. I'd only comment that a really intelligent de-capitalization algorithm would attempt to supply the accents that are missing from the capitalized forms. Some languages that use accents tolerate omitting them from capital letters but not from lower-case letters. Maybe a better algorithm could be developed by a Spanish native-speaker. Admittedly, accents could be supplied at a later date. -- Steve ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Changing capitalization (Lima)
Hello. 2012/5/31 Alex Barth : > We're currently working with Ruben (user Rub21) on fixing street name > capitalization in Lima - a lot of the street names are ALL CAPS where they > should be properly capitalized. We're doing this work manually right now and > are well under way. It's quite time intensive though - any examples of where > such a cleanup process has been automated on OSM before? I appreciate your fight with bad tags and bad imports. I prepared a file with edits for highways in Lima area. http://worstfixer.000a.biz/y2-lima-hw/edits-to-upload.osm.gz Overview: http://worstfixer.000a.biz/y2-lima-hw/overview-short.html http://worstfixer.000a.biz/y2-lima-hw/overview-full.html Ich also deleted source= if it is the only tag on nodes, and removed single addr:street instance when it was equal to name tag. If you notice some big flaw in my case change algoritm, mail me privately. Ich will re-make file for you. Please review it and upload. If you want to upload it without complete review, please follow Mechanical Edits Policy: http://wiki.openstreetmap.org/wiki/Mechanical_Edit_Policy > tel (202) 250-3633 This ist world wide international list. Write phone numbers in international format. -- WorstFixer, twitter: http://twitter.com/WorstFixer ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Changing capitalization (Lima)
Maarten - What editor (plugin?) do you suggest for this procedure? Or would you just edit the OSM file with a text editor? If I do that I'm missing change set entries, no? On May 31, 2012, at 12:03 PM, Maarten Deen wrote: > On 31-5-2012 17:39, Alex Barth wrote: >> We're currently working with Ruben (user Rub21) on fixing street name >> capitalization in Lima - a lot of the street names are ALL CAPS where they >> should be properly capitalized. We're doing this work manually right now and >> are well under way. It's quite time intensive though - any examples of where >> such a cleanup process has been automated on OSM before? > > How difficult is it to download an area to an OSM file, check for name= CAPS>, change these to proper caps and upload the file? > > I'd say no more than an hours work. > > Regards, > Maarten > > > ___ > talk mailing list > talk@openstreetmap.org > http://lists.openstreetmap.org/listinfo/talk Alex Barth http://twitter.com/lxbarth tel (202) 250-3633 ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Changing capitalization (Lima)
Thanks, Andy - We're in the process of fixing some of these issues, too. There's been a pretty botched import in Lima a while ago. On May 31, 2012, at 12:10 PM, SomeoneElse wrote: > Alex Barth wrote: >> We're currently working with Ruben (user Rub21) on fixing street name >> capitalization in Lima - a lot of the street names are ALL CAPS where they >> should be properly capitalized. > > Ye gods. > > Looking at that patch of Lima it appears that name capitalisation is the > least of your worries. Perhaps a starting point might be a few NE-SW and > SE-NW GPS traces (to help align the imagery), and a quick on-the-ground > survey of some of the obviously problematical areas (e.g. way 111912131, > which is unconnected to anything, and on the Bing imagery is a football > field). > > That should give you an idea of the positional validity of what's there - > there's no point in wasting time manually correcting "ROAD A" to "Road A" if > Road A doesn't actually exist any more. > > Obviously the quick survey to get GPS traces available could pinpoint some > key POIs too. > > Cheers, > Andy > > > ___ > talk mailing list > talk@openstreetmap.org > http://lists.openstreetmap.org/listinfo/talk Alex Barth http://twitter.com/lxbarth tel (202) 250-3633 ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Changing capitalization (Lima)
Alex Barth wrote: We're currently working with Ruben (user Rub21) on fixing street name capitalization in Lima - a lot of the street names are ALL CAPS where they should be properly capitalized. Ye gods. Looking at that patch of Lima it appears that name capitalisation is the least of your worries. Perhaps a starting point might be a few NE-SW and SE-NW GPS traces (to help align the imagery), and a quick on-the-ground survey of some of the obviously problematical areas (e.g. way 111912131, which is unconnected to anything, and on the Bing imagery is a football field). That should give you an idea of the positional validity of what's there - there's no point in wasting time manually correcting "ROAD A" to "Road A" if Road A doesn't actually exist any more. Obviously the quick survey to get GPS traces available could pinpoint some key POIs too. Cheers, Andy ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Changing capitalization (Lima)
On 31-5-2012 17:39, Alex Barth wrote: We're currently working with Ruben (user Rub21) on fixing street name capitalization in Lima - a lot of the street names are ALL CAPS where they should be properly capitalized. We're doing this work manually right now and are well under way. It's quite time intensive though - any examples of where such a cleanup process has been automated on OSM before? How difficult is it to download an area to an OSM file, check for name=, change these to proper caps and upload the file? I'd say no more than an hours work. Regards, Maarten ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
[OSM-talk] Changing capitalization (Lima)
We're currently working with Ruben (user Rub21) on fixing street name capitalization in Lima - a lot of the street names are ALL CAPS where they should be properly capitalized. We're doing this work manually right now and are well under way. It's quite time intensive though - any examples of where such a cleanup process has been automated on OSM before? Example all caps in Lima: http://osm.org/go/NNcgaXMQ https://skitch.com/alexbarth/874nu/openstreetmap Alex Barth http://twitter.com/lxbarth tel (202) 250-3633 ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
