Re: [OSM-talk] Removing home point cloud(s) from multiple GPX files
Thanks Ian, What I would really like is not to remove clouds automatically, but to be able to specify several circles of exclusion; around but not quite centred on my home, my workplace, and certain relatives houses from which I have started and ended many tracks. I'd then like to be able to easily process large numbers of files in a batch with this. It's not necessarily true that there are massive clouds in each individual file (though some probably do), but you end up with a cloud from all the files added together that clearly indicate where you live/work etc, and produces an area on the map where a cloud hides useful data. Ideally users would be able to save the exclusion zones they have set, to make processing future files easier. It is rarely useful to save large numbers of points from route outside these locations as the chances are they will already have been thoroughly mapped. Regards, David --- On Sat, 18/4/09, Ian Dees wrote: > From: Ian Dees > Subject: Re: [OSM-talk] Removing home point cloud(s) from multiple GPX files > To: "David Ebling" > Cc: talk@openstreetmap.org > Date: Saturday, 18 April, 2009, 12:21 AM > [off-list] > > David, can you send me a sample file? I'd be happy to > write something that consumes a GPX file and tries to pull > out the points that are clumped together like that. > > Or maybe you've already figured out a solution to your > problem and don't need such a thing. > > > Let me know, > Ian > > > On Fri, Apr 17, 2009 at 2:21 PM, > David Ebling > wrote: > > > > > During the downtime I figured I should really catch up with > cleaning up my huge backlog of GPX files and prepare them > for upload. I have many many hours of tracks on my HDD that > I have not uploaded because many of them are polluted with > point clouds around my house and my relatives' houses. I > am not happy to upload these for privacy reasons and also > for data purity reasons. > > > > > > I am not very command line compatible, and can't find > *any* gui tool for Mac or Windows that will let me do this. > GPX Babel only seems to let you remove points outside a > radius not inside the radius, unless you use the exclude > option which appears to be command line only. It also takes > lat and long in an annoying format (decimal minutes, neither > decimal degrees nor degrees, minutes and seconds.) It also > seems only to be able to process one file at a time on the > Mac version I've played with. > > > > > > Is there any program out there that will do this easily? If > not OSM would really benefit from one, as I think there are > many people like me who aren't uploading GPX because > cleaning them up is simply too much effort. Here's my > idea for someone with more programming skills than me: > > > > > > -A dialogue box that uses an OSM slippy map to draw circles > of exclusion on the map, with a guidance note suggesting > that they are near and covering but not exactly centred on > your home/work/other point cloud locations. > > > > -Ability to batch process that's user friendly > > -Output to a new folder > > -Ideally, upload direct to OSM, to be considerate to other > users, perhaps over a specified time interval. > > > > Any programmers out there want to take up this idea while > we have some down time? :D Please? :) I'll upload lots > of GPX files in return! ;) > > > > Thanks, > > > > Dave > > > > > > > > > > > > ___ > > talk mailing list > > talk@openstreetmap.org > > http://lists.openstreetmap.org/listinfo/talk > > > > ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Removing home point cloud(s) from multiple GPX files
Karl Newman wrote: > I want the same thing, and I looked into doing this with GPSBabel, too, and > while it has the capability to filter points inside or outside a radius (or > polygon, etc.), that function does not work on trackpoints, and the author > had said something to the effect of "it doesn't make sense to have it > enabled for trackpoints because it would make false tracks (jumps, etc.)". Here is what I do for this. This is a Linux shell batch script, but should be adaptable for DOS batch files. This converts the trackpoints to waypoints, filters on them, and then converts back to trackpoints. It deletes any names added to the waypoints (trackpoints) in the process. It also breaks out the tracks to separate output files based on time gaps in the captured data. That way I just capture one big file but get multiple files out. - ExcludeFile=~/gps/gpsbabel-exclude-shapes.dat gpsbabel\ -i nmea -f "$InputFile" \ -x discard,hdop=5.5 \ -x transform,wpt=trk,del\ -x polygon,file=$ExcludeFile,exclude\ -x transform,trk=wpt,del\ -o gpx -F - \ | grep -v -e '' -e '' -e ''\ | gpsbabel \ -i gpx -f - \ -x track,pack,split=5m,title="T%Y%m%d-%H%M" \ -o gpx -F - \ >"$NewFileName" - Your exclude file would contain polygon coordinates something like: - # West Hyland Hills # -122.8252,45.4622,-122.8145,45.4755 45.4622 -122.8252 45.4622 -122.8145 45.4755 -122.8145 45.4755 -122.8252 45.4622 -122.8252 # East Hyland Hills # -122.8145,45.4657,-122.8058,45.4755 45.4657 -122.8145 45.4657 -122.8058 45.4755 -122.8058 45.4755 -122.8145 45.4657 -122.8145 - Like most free solutions, this is a starter example and not a complete solution; you'll need to tweak it to your situation... The primary thing lost in this scenario is probably any waypoints in the original capture. (I don't capture waypoints myself so I didn't accomodate for that.) There are probably some more steps you can do to split those off first. Good luck, - Alan ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Removing home point cloud(s) from multiple GPX files
The software "GPS Utility" www.gpsu.co.uk will do this automatically in two steps for any gpx track. Select track, then from 'tools' menu 'generate track waypoints' then 'compress track'. User can set the 'error' i.e. the 'off-track' distance. It is also possible to view (and optionally tweak individual track waypoints) before committing the operation. There is a free 'lite' version of the software - but the shareware version is well worth the very modest price. Highly recommended to anyone with a lot of track, route and waypoint processing to do - and also a file format converter that is at least as good as GPS Babel. (I'm not getting any commission for this!). Mike Harris -Original Message- From: David Ebling [mailto:dave_ebl...@yahoo.co.uk] Sent: 17 April 2009 20:21 To: talk@openstreetmap.org Subject: [OSM-talk] Removing home point cloud(s) from multiple GPX files During the downtime I figured I should really catch up with cleaning up my huge backlog of GPX files and prepare them for upload. I have many many hours of tracks on my HDD that I have not uploaded because many of them are polluted with point clouds around my house and my relatives' houses. I am not happy to upload these for privacy reasons and also for data purity reasons. I am not very command line compatible, and can't find *any* gui tool for Mac or Windows that will let me do this. GPX Babel only seems to let you remove points outside a radius not inside the radius, unless you use the exclude option which appears to be command line only. It also takes lat and long in an annoying format (decimal minutes, neither decimal degrees nor degrees, minutes and seconds.) It also seems only to be able to process one file at a time on the Mac version I've played with. Is there any program out there that will do this easily? If not OSM would really benefit from one, as I think there are many people like me who aren't uploading GPX because cleaning them up is simply too much effort. Here's my idea for someone with more programming skills than me: -A dialogue box that uses an OSM slippy map to draw circles of exclusion on the map, with a guidance note suggesting that they are near and covering but not exactly centred on your home/work/other point cloud locations. -Ability to batch process that's user friendly -Output to a new folder -Ideally, upload direct to OSM, to be considerate to other users, perhaps over a specified time interval. Any programmers out there want to take up this idea while we have some down time? :D Please? :) I'll upload lots of GPX files in return! ;) Thanks, Dave ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Removing home point cloud(s) from multiple GPX files
On Fri, Apr 17, 2009 at 6:21 PM, Ian Dees wrote: > [off-list] Sorry all, I guess GMail forgot my "reply to sender" preference there. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Removing home point cloud(s) from multiple GPX files
[off-list] David, can you send me a sample file? I'd be happy to write something that consumes a GPX file and tries to pull out the points that are clumped together like that. Or maybe you've already figured out a solution to your problem and don't need such a thing. Let me know, Ian On Fri, Apr 17, 2009 at 2:21 PM, David Ebling wrote: > > During the downtime I figured I should really catch up with cleaning up my > huge backlog of GPX files and prepare them for upload. I have many many > hours of tracks on my HDD that I have not uploaded because many of them are > polluted with point clouds around my house and my relatives' houses. I am > not happy to upload these for privacy reasons and also for data purity > reasons. > > I am not very command line compatible, and can't find *any* gui tool for > Mac or Windows that will let me do this. GPX Babel only seems to let you > remove points outside a radius not inside the radius, unless you use the > exclude option which appears to be command line only. It also takes lat and > long in an annoying format (decimal minutes, neither decimal degrees nor > degrees, minutes and seconds.) It also seems only to be able to process one > file at a time on the Mac version I've played with. > > Is there any program out there that will do this easily? If not OSM would > really benefit from one, as I think there are many people like me who aren't > uploading GPX because cleaning them up is simply too much effort. Here's my > idea for someone with more programming skills than me: > > -A dialogue box that uses an OSM slippy map to draw circles of exclusion on > the map, with a guidance note suggesting that they are near and covering but > not exactly centred on your home/work/other point cloud locations. > -Ability to batch process that's user friendly > -Output to a new folder > -Ideally, upload direct to OSM, to be considerate to other users, perhaps > over a specified time interval. > > Any programmers out there want to take up this idea while we have some down > time? :D Please? :) I'll upload lots of GPX files in return! ;) > > Thanks, > > Dave > > > > > > ___ > talk mailing list > talk@openstreetmap.org > http://lists.openstreetmap.org/listinfo/talk > ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Removing home point cloud(s) from multiple GPX files
David Ebling wrote: > During the downtime I figured I should really catch up with cleaning > up my huge backlog of GPX files and prepare them for upload. I have > many many hours of tracks on my HDD that I have not uploaded because > many of them are polluted with point clouds around my house and my > relatives' houses. I am not happy to upload these for privacy reasons > and also for data purity reasons. > > I am not very command line compatible, and can't find *any* gui tool > for Mac or Windows that will let me do this. GPX Babel only seems to > let you remove points outside a radius not inside the radius, unless > you use the exclude option which appears to be command line only. It > also takes lat and long in an annoying format (decimal minutes, > neither decimal degrees nor degrees, minutes and seconds.) It also > seems only to be able to process one file at a time on the Mac > version I've played with. > > Is there any program out there that will do this easily? If not OSM > would really benefit from one, as I think there are many people like > me who aren't uploading GPX because cleaning them up is simply too > much effort. Here's my idea for someone with more programming skills > than me: > > -A dialogue box that uses an OSM slippy map to draw circles of > exclusion on the map, with a guidance note suggesting that they are > near and covering but not exactly centred on your home/work/other > point cloud locations. -Ability to batch process that's user friendly > -Output to a new folder -Ideally, upload direct to OSM, to be > considerate to other users, perhaps over a specified time interval. > Try this program http://www.gpspassion.com/FORUMSEN/topic.asp?TOPIC_ID=106662 It won't fulfill all your wishes but it is a GUI based editor that allows you to select individual tracks in a gpx file, prune undesired parts of tracks and optionally view them in Google Maps or OSM! You'll have to edit each track individually, unfortunately. No batch mode. But there is a long weekend ahead with no mapping.. Ole / polderrunner ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Removing home point cloud(s) from multiple GPX files
On Fri, Apr 17, 2009 at 1:48 PM, Frederik Ramm wrote: > Hi, > > Karl Newman wrote: > >> I want the same thing, and I looked into doing this with GPSBabel, too, >> and >> while it has the capability to filter points inside or outside a radius >> (or >> polygon, etc.), that function does not work on trackpoints, and the author >> had said something to the effect of "it doesn't make sense to have it >> enabled for trackpoints because it would make false tracks (jumps, etc.)". >> > > Those free software authors are bastards aren't they ;-) > > Someone on talk-de recently posted the following magic spell he's using > with gpsbabel to simplify his tracks, clear them of point clouds and exclude > certain areas: > > -x nuketypes,waypoints,routes > -x transform,wpt=trk > -x nuketypes,tracks > -x polygon,file=europa.txt > -x polygon,file=gruener.txt,exclude > -x sort,time > -x transform,trk=wpt > -x nuketypes,waypoints > -x track,pack,sdistance=0.2k,split=20s > -x position,distance=1m > > The clever bit seems to be the -x transform incantantions with which he > converts trackpoints to waypoints before applying the polygon filter, then > transforms them back! > > Bye > Frederik > Cool. I'll have to check that out. I'm curious to see what information is lost in the translation. Karl ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Removing home point cloud(s) from multiple GPX files
Hi, Karl Newman wrote: > I want the same thing, and I looked into doing this with GPSBabel, too, and > while it has the capability to filter points inside or outside a radius (or > polygon, etc.), that function does not work on trackpoints, and the author > had said something to the effect of "it doesn't make sense to have it > enabled for trackpoints because it would make false tracks (jumps, etc.)". Those free software authors are bastards aren't they ;-) Someone on talk-de recently posted the following magic spell he's using with gpsbabel to simplify his tracks, clear them of point clouds and exclude certain areas: -x nuketypes,waypoints,routes -x transform,wpt=trk -x nuketypes,tracks -x polygon,file=europa.txt -x polygon,file=gruener.txt,exclude -x sort,time -x transform,trk=wpt -x nuketypes,waypoints -x track,pack,sdistance=0.2k,split=20s -x position,distance=1m The clever bit seems to be the -x transform incantantions with which he converts trackpoints to waypoints before applying the polygon filter, then transforms them back! Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Removing home point cloud(s) from multiple GPX files
On Fri, Apr 17, 2009 at 12:21 PM, David Ebling wrote: > > During the downtime I figured I should really catch up with cleaning up my > huge backlog of GPX files and prepare them for upload. I have many many > hours of tracks on my HDD that I have not uploaded because many of them are > polluted with point clouds around my house and my relatives' houses. I am > not happy to upload these for privacy reasons and also for data purity > reasons. > > I am not very command line compatible, and can't find *any* gui tool for > Mac or Windows that will let me do this. GPX Babel only seems to let you > remove points outside a radius not inside the radius, unless you use the > exclude option which appears to be command line only. It also takes lat and > long in an annoying format (decimal minutes, neither decimal degrees nor > degrees, minutes and seconds.) It also seems only to be able to process one > file at a time on the Mac version I've played with. > > Is there any program out there that will do this easily? If not OSM would > really benefit from one, as I think there are many people like me who aren't > uploading GPX because cleaning them up is simply too much effort. Here's my > idea for someone with more programming skills than me: > > -A dialogue box that uses an OSM slippy map to draw circles of exclusion on > the map, with a guidance note suggesting that they are near and covering but > not exactly centred on your home/work/other point cloud locations. > -Ability to batch process that's user friendly > -Output to a new folder > -Ideally, upload direct to OSM, to be considerate to other users, perhaps > over a specified time interval. > > Any programmers out there want to take up this idea while we have some down > time? :D Please? :) I'll upload lots of GPX files in return! ;) > > Thanks, > > Dave > I want the same thing, and I looked into doing this with GPSBabel, too, and while it has the capability to filter points inside or outside a radius (or polygon, etc.), that function does not work on trackpoints, and the author had said something to the effect of "it doesn't make sense to have it enabled for trackpoints because it would make false tracks (jumps, etc.)". At least, that's what I found when I last checked about maybe a year ago. I don't know why it couldn't just split it into however many tracks it needs to show the discontinuity before/after the exclusion zone. Karl ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
[OSM-talk] Removing home point cloud(s) from multiple GPX files
During the downtime I figured I should really catch up with cleaning up my huge backlog of GPX files and prepare them for upload. I have many many hours of tracks on my HDD that I have not uploaded because many of them are polluted with point clouds around my house and my relatives' houses. I am not happy to upload these for privacy reasons and also for data purity reasons. I am not very command line compatible, and can't find *any* gui tool for Mac or Windows that will let me do this. GPX Babel only seems to let you remove points outside a radius not inside the radius, unless you use the exclude option which appears to be command line only. It also takes lat and long in an annoying format (decimal minutes, neither decimal degrees nor degrees, minutes and seconds.) It also seems only to be able to process one file at a time on the Mac version I've played with. Is there any program out there that will do this easily? If not OSM would really benefit from one, as I think there are many people like me who aren't uploading GPX because cleaning them up is simply too much effort. Here's my idea for someone with more programming skills than me: -A dialogue box that uses an OSM slippy map to draw circles of exclusion on the map, with a guidance note suggesting that they are near and covering but not exactly centred on your home/work/other point cloud locations. -Ability to batch process that's user friendly -Output to a new folder -Ideally, upload direct to OSM, to be considerate to other users, perhaps over a specified time interval. Any programmers out there want to take up this idea while we have some down time? :D Please? :) I'll upload lots of GPX files in return! ;) Thanks, Dave ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
