Re: [OSM-dev] OSMand Live can steal your money

[Toby Murray]

Well originally they weren't even using HTTPS for that form
submission. I opened an issue about it and at least HTTPS has been
implemented since then.

Issue: https://github.com/osmandapp/osmandapp.github.io/issues/37

Toby

On Fri, Jan 12, 2018 at 7:15 AM, Darafei "Komяpa" Praliaskouski
 wrote:
> Hi,
>
> https://osmand.net/osm_live requests user's OSM password and e-mail in
> exchange of promise of bitcoin payment.
>
> There is no way to check that the password is not being collected, with or
> without knowledge of service authors. At least 1100 accounts may be
> affected.
>
> Simplest attack vector may be "if password matches on google drive of this
> e-mail and there's a backup of wallet there and password matches there too,
> get all the money from there".
>
> What can be done on osm.org side to mitigate it?
> Can password reset be forced for affected users, and for those who keep
> coming to that form?
>
> ___
> dev mailing list
> dev@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/dev
>

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] retrieve user`s editing log

[Toby Murray]

The editing API is primarily intended for editing the map. While some
non-editing usage is permitted, if your application will be making
many calls on a continuous basis, the editing API will not be the
appropriate thing to use. See here:
https://wiki.openstreetmap.org/wiki/API_usage_policy

If you are looking for a continuous feed of changes, there are data
diffs that are published on a minutely, hourly or daily basis. These
are available from http://planet.osm.org/replication/ and documented
at https://wiki.openstreetmap.org/wiki/Planet.osm/diffs

There are tools to help consume these. Osmosis is the one that comes
to mind first but I think there are some others as well.

If you are just after the changeset metadata (so no map objects, just
what is visible for example here:
http://www.openstreetmap.org/api/0.6/changeset/42768543 ) then you can
use the changeset replication diffs at
http://planet.osm.org/replication/changesets/ for which I have a
parser that will both import the historical dump and keep up with
minutely diffs: https://github.com/ToeBee/ChangesetMD

This is a lot less data to deal with than full map object history
which can be rather overwhelming.

Toby

On Fri, Oct 7, 2016 at 6:27 PM, Liming Zhang  wrote:
> I think I find something to try now. In API v0.6, it state:
>
> History: GET /api/0.6/[node|way|relation]/#id/history
>
> Retrieves all old versions of an element.
>
>
> I believe this maybe what I am looking for.
>
>
> -
> Liming(Lem) Zhang
>
> Presidential Scholar, PhD student, George Mason University
>
>
> On Fri, Oct 7, 2016 at 6:59 PM, Liming Zhang  wrote:
>>
>> Thank you! I am pretty new to OSM, so I might mislead my question. What I
>> want is something like JSON`s  pair data. The raw logging of
>> different users and associated information, something looks like:
>> { [user name: Jack,
>> time: 08/20/2012,
>> editing Node: ...,
>> editing Line: ...,
>> editing Tag: ...] }
>>
>> I am not just looking for a historial snapshot/version of a data set.
>> Basically, it is some API which can fetch data from OSM. Not sure if this is
>> clear. Of course, I am in a quite initial stage of idea. Just try to explore
>> possibility and current development in the community.
>>
>>
>>
>> -----
>> Liming(Lem) Zhang
>>
>> Presidential Scholar, PhD student, George Mason University
>>
>>
>> On Fri, Oct 7, 2016 at 6:34 PM, Toby Murray  wrote:
>>>
>>> What do you mean by "editing log"? Do you just want a list of
>>> changesets? Or do you want all the map objects they have touched as
>>> well?
>>>
>>> Do you want historic data for analysis or real-time updates for
>>> current edits? Or both?
>>>
>>> Depending on these options, this could be a trivial task or an involved
>>> project.
>>>
>>> Toby
>>>
>>> On Fri, Oct 7, 2016 at 5:17 PM, Liming Zhang 
>>> wrote:
>>> > Hello,
>>> >
>>> > I want to develop a simple app to retrieve user`s editing log from
>>> > openstreetmap. Anyone know how to do that? or some correct direction to
>>> > go
>>> > after?
>>> >
>>> > Thank you!
>>> >
>>> > -
>>> > Liming(Lem) Zhang
>>> >
>>> > Presidential Scholar, PhD student, George Mason University
>>> >
>>> >
>>> > ___
>>> > dev mailing list
>>> > dev@openstreetmap.org
>>> > https://lists.openstreetmap.org/listinfo/dev
>>> >
>>
>>
>

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] retrieve user`s editing log

[Toby Murray]

What do you mean by "editing log"? Do you just want a list of
changesets? Or do you want all the map objects they have touched as
well?

Do you want historic data for analysis or real-time updates for
current edits? Or both?

Depending on these options, this could be a trivial task or an involved project.

Toby

On Fri, Oct 7, 2016 at 5:17 PM, Liming Zhang  wrote:
> Hello,
>
> I want to develop a simple app to retrieve user`s editing log from
> openstreetmap. Anyone know how to do that? or some correct direction to go
> after?
>
> Thank you!
> -
> Liming(Lem) Zhang
>
> Presidential Scholar, PhD student, George Mason University
>
>
> ___
> dev mailing list
> dev@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/dev
>

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] [Talk-us] [OSM-talk] Slack

[Toby Murray]

I have set up a Slack bot using some software[1] that relays messages
between a slack channel and an IRC channel. It is listening in #osm on
OFTC and the #irc slack channel in Steve's team.

I love my irssi+screen IRC setup however it kind of breaks down when
it comes to a phone-friendly interface. So I may use this slack
channel to hop on IRC from my phone sometimes.

We'll see how it goes. It hasn't seen too much traffic yet.

Toby


[1] https://github.com/ekmartin/slack-irc

On Sun, Mar 27, 2016 at 2:07 AM, Steve Coast  wrote:
> https://en.wikipedia.org/wiki/Slack_(software)
>
> On Mar 27, 2016, at 12:02 AM, Maarten Deen  wrote:
>
> On 2016-03-26 20:59, Steve Coast wrote:
>
> Ok so look, Slack took over the world. And it turns out it’s pretty
> good and useful. Let’s have an official OSM slack.
>
>
> Maybe I'm living under a rock, but I only know Slack as a short for
> Slackware, a Linux distribution.
>
> What is this and why do I need this? Maybe a little explanation?
>
> Regards,
> Maarten
>
>
>
> ___
> Talk-us mailing list
> talk...@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-us
>

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Changefile Downloads Stalling

[Toby Murray]

I was one of the people on IRC yesterday and can confirm that I am seeing
similar problems when downloading files from planet.osm.org while at work
(university campus) however when downloading to a VM I have with
cloudatcost (I think they are in Canada), I am not having any problems.

I just tried to download the latest note dump file. The first time it
stalled out at 2.6 MB. A second attempt stalled at 1.something then picked
back up to 3 MB. Now it is trickling in at less than 70 KB/sec.

Since this looks like something happening along the way and not at either
end, here is a trace from me to planet.osm.org:

 1?: [LOCALHOST] pmtu 1500
 1:  no reply
 2:  no reply
 3:  kanren-ksu-pplant-border.peer.net.kanren.net  2.479ms
 4:  bb-ksu-hale-e2-1.bb.net.kanren.net3.196ms
 5:  bb-kc-walnut-e2-4.bb.net.kanren.net   6.768ms
 6:  et-10-0-0.1050.rtr.kans.net.internet2.edu 5.911ms
 7:  et-9-0-0.106.rtr.chic.net.internet2.edu  17.584ms
 8:  et-9-0-0.115.rtr.wash.net.internet2.edu  35.037ms
 9:  abilene-wash.mx1.fra.de.geant.net   127.550ms
10:  ae1.mx1.ams.nl.geant.net133.560ms
11:  ae2.mx1.lon.uk.geant.net155.513ms
12:  janet-gw.mx1.lon.uk.geant.net   158.483ms
13:  ae29.londpg-sbr1.ja.net 169.894ms
14:  be24.londic-rbr1.ja.net 157.260ms
15:  imperial-college.ja.net 158.031ms
16:  me-rach.net.ic.ac.uk142.874ms
17:  no reply
18:  no reply


Toby


On Tue, Nov 11, 2014 at 7:31 PM, Humphries, Grant 
wrote:

>  Hi All,
>
>
>
> I have been trying intermittently for a few weeks now to use osmupdate to
> refresh a local OSM extract.  The problem that I am having relates to the
> download of changefiles from planet.openstreetmap.org.  Ultimately I want
> to update my extract once per day, so most of the changefiles that need to
> be downloaded and applied are of the hourly variety, such as this one:
> http://planet.openstreetmap.org/replication/hour/000/018/973.osc.gz
>
>
>
> These files are generally 2 to 5 MB in size and the problem that I’m
> having is that my download will stall, sometimes for 20 to 30 minutes,
> sometimes up to a couple of hours, on one or more of these small
> changefiles virtually every time I attempt to use osmupdate.  The odd thing
> is that changefiles that are downloaded back-to-back, and that are roughly
> the same size can take vastly different amounts of time to complete.  For
> instance today osmupdate started with changefile 18975 and 18974 which took
> seconds, but then 18973 took around 20+ minutes.  I’m running this tool on
> a linux cent os server, and osmupdate uses wget to download the files.
> I’ve tried using wget and curl on the same files outside of osmupdate and
> the problem persists.  I’ve also tried downloading the files through
> Firefox on a windows PC and had similar problems.  I’ve asked others on IRC
> to attempt the same downloads with varying results, some experience a
> similar lag, and some are able to download the same files I’m having
> trouble with in seconds.
>
>
>
> I was just going to say that this file in particular has given me trouble
> https://planet.openstreetmap.org/replication/hour/000/018/823.osc.gz as I
> have tried to download it many times and it has always stalled and taken an
> extremely long time to complete, but I just tried it one more time and it
> took 8.8 seconds.  So is this kind of variance to be expected?
>
>
>
> Thanks in advance for any help
>
>
>
> Best,
>
>
>
> -Grant
>
>
>
> ___
> dev mailing list
> dev@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/dev
>
>
___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] current documentation on using osmosis to populate an empty osm database?

[Toby Murray]

Well the reference for the current version is always at
http://wiki.openstreetmap.org/wiki/Osmosis/Detailed_Usage

Which schema are you trying to populate? A rails database or a pgnapshot?

Toby


On Thu, Aug 7, 2014 at 3:19 PM, Richard Welty 
wrote:

> i'm trying to import an osm extract into an empty osm database using
> osmosis and i'm not at all clear on where current documentation is.
> there are some howtos and examples out there covering what appear
> to be obsolete versions which isn't helping.
>
> can anyone point me at something current?
>
> thanks,
>richard
>
> --
> rwe...@averillpark.net
>  Averill Park Networking - GIS & IT Consulting
>  OpenStreetMap - PostgreSQL - Linux
>  Java - Web Applications - Search
>
>
>
> ___
> dev mailing list
> dev@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/dev
>
>
___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] visual timestamp from ways

[Toby Murray]

What's wrong with the /status URL on tiles? It works for osm.org and
Mapquest Open. Doesn't seem to work for cloudmade but then I don't think
your plan would work any better. From what I've seen, cloudmade does
generally update their data relatively quickly but their tile server is
either overloaded or configured in such a way that it is a pain to actually
ensure that a given tile is freshly rendered. Sometimes I have seen fresh
tiles right next to stale ones. So you changing data somewhere in the
middle of the ocean will have no bearing on the freshness of a tile served
up 5,000 miles away.

Toby



On Fri, Oct 4, 2013 at 9:21 AM, Jaak Laineste wrote:

> Hello,
>
> I have from time-to-time problem that I don't know how old is a particular
> tile service, even for my own ones I'm not sure if update is working
> properly or not. For example, are Cloudmade and MapQuest tiles updated?
> Idea: add visible timestamp ("watermark", "microprint") using fake ways to
> the global map, so it would be rendered with more or less any possible
> layout, but only if you know from where to look for it. Maybe near null
> island. Or my home.
>
> Technically, I'd have a robot which updates timestamp in OSM API every
> hour, and draws GMT date and time using usual ways with highway=residential
> tag, so to see it you just zoom in to specific place.
>
> Questions: has anyone done this, or something even better? Any objections ?
>
> --
> Jaak
>
>
> ___
> dev mailing list
> dev@openstreetmap.org
> https://lists.openstreetmap.org/listinfo/dev
>
>
___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] About the geographic data in a .osm file

[Toby Murray]

First of all, did you notice that the cloudmade extracts are over a year
old? I would suggest looking at geofabrik for up to date extracts:
http://download.geofabrik.de/north-america.html

As for your tiles quesion, I'm not quite sure what you mean. Are you asking
why there is any data outside of Delaware? Or why you are getting blank
tiles where you don't have any data?

The cloudmade extracts never were too accurate on the state borders. When I
first downloaded Kansas they were missing the north border by a couple of
miles and had extra on the south. So it doesn't surprise me that the
cloudmade extract has extra data in it.

As for tile generation, the tile rendering process doesn't care about where
you do/don't have data. It just renders the tiles you tell it to render. If
there happens to be no data in that area then the tile will be blank.

Toby



On Fri, May 24, 2013 at 3:08 PM, Vince Berubey wrote:

> I have downloaded 
> "delaware.osm.bz2
>  "
> http://downloads.cloudmade.com/americas/northern_america/united_states/delaware#downloads_breadcrumbs
>
> The boundary box I used when I generated my tiles from Generated_tyles.py
>  were a little bit bigger than the Delaware region.
>
> What I don't understand is why I was able to generate tiles that were in
> my boundary box, but outside the Delaware area.
>
> So, the .osm file does not exactly contain only the geographic data from
> the region that you have downladed?
>
> Please look at this file, the red marks are the Delaware borders. I have a
> lot of tiles generated outside of these borders:
>
> http://postimg.org/image/ogwbjz5ln/
>
> Thank you for any explanation about it.
>
> Best regards.
>
>
> ___
> dev mailing list
> dev@openstreetmap.org
> http://lists.openstreetmap.org/listinfo/dev
>
>
___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Problem in recent potlatch2?

[Toby Murray]

Well P2 has been creating single node ways for as long as it has existed :)

But I guess this happening at 0,0 is a little more interesting...

Toby



On Thu, May 9, 2013 at 2:30 AM, Martin Koppenhoefer
wrote:

> Not sure if this is already known, I just spotted a recent PL2 edit where
> a way with 1 node on 0,0 was created, probably a bug.
>
> http://www.openstreetmap.org/browse/changeset/16013103
>
> cheers,
> Martin
>
> ___
> dev mailing list
> dev@openstreetmap.org
> http://lists.openstreetmap.org/listinfo/dev
>
>
___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] [Talk-us] Chicago Hack Weekend

[Toby Murray]

I booked my flight last night. You should too!

Toby


On Sat, Mar 23, 2013 at 12:52 PM, Ian Dees  wrote:
> Hi list-goers,
>
> Knight-Mozilla OpenNews and myself are hosting a hack weekend April 27th and
> 28th in the heart of downtown Chicago.
>
> These sorts of hack weekends are a chance for the OSM developer community to
> get together and work on projects to improve and grow the software behind
> OSM. If you don't have anything specific to work on and are well-versed in
> Rails or JavaScript, Python or Java, there's plenty for you to help with.
>
> We'll have food and beverages along with some great socializing after long
> days of coding or discussing features we'd like to see in OSM.
>
> For more detailed information, please visit the wiki page here:
> http://wiki.openstreetmap.org/w/index.php?title=Chicago_Hack_Weekend_April_2013
>
> -Ian
>
> ___
> Talk-us mailing list
> talk...@openstreetmap.org
> http://lists.openstreetmap.org/listinfo/talk-us
>

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Getting lat and long from an address

[Toby Murray]

On Tue, Mar 19, 2013 at 10:05 AM, drugdu  wrote:
> I’m trying develop an desktop application  to get the lat and long base on an
> address, the problem is that I must do this offline, I tried to export the
> map, but I need all my county and the export tab says that I need to choose
> a smaller area, it’s my first time working with maps and I really don’t have
> any idea where I can start to learning about this, I prefer to work on c# or
> iron python or iron ruby, I hope you can help me.

All of the OSM data is available via a giant planet file on
planet.osm.org. This is probably more data than you want to work with
but there are several places that provide extracts on the
country/state/city level. Please read over this page in our wiki:

http://wiki.openstreetmap.org/wiki/Planet

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Seeking advice for OSM based routing application

[Toby Murray]

On Sun, Mar 17, 2013 at 4:52 PM, Matt Hoover  wrote:
> I recently developed a fairly simple open-source application for
> creating/editing/analyzing GPX files.  It relies heavily on OSM tiles and
> JMapViewer (a JOSM component).  If interested, the beta version is hosted at
> http://www.gpxcreator.com.
>
> Currently the only way to create a GPX file is by pointing and clicking on
> the map repeatedly, for each point in the route.  My ultimate goal though is
> to have start-to-end routing, and I'd like it to work for backcountry
> hiking/biking/etc trails.  OSM seems to be the best map system I've found
> yet as far as having a lot of these trails included in its maps (in USA west
> coast at least, where I've looked so far).  I haven't worked much with
> actual OSM data yet (only tiles), but I understand the main way to download
> the data is in an XML format.  In trying to accomplish my goal of adding
> routing to my application, I'm faced with what I see as a few limited
> options, and none of them seem ideal.  I'd appreciate insight/advice from
> people more experienced both with OSM data and routing implementations.
> Here are the options as I see them:
>
> 1) Host OSM data myself and run a routing algorithm on the server.  This
> would be ideal in theory, but in reality I doubt I can afford to do
> something so big.
> 2) Have the routing algorithm run on client machine, and request the data it
> needs from OSM with each routing attempt.  This seems bad for a number of
> reasons.  Even to do a relatively short route ( < 100 miles), it seems like
> a pretty big chunk of data would need to be downloaded to the client (like
> 50MB at least in some quick tests I did).  This is not ideal for the client,
> nor is it nice to the OSM server.  Plus all that XML would need to be parsed
> on the client and re-structured into some format the routing algorithm could
> use, which would be time consuming for the client.
> 3) Use one of the existing servers which has its own data, routing
> implementation, and public API.  This sounds like a very promising idea at
> first.  However, I'm running into some weird issues, and I've tried every
> single option listed in OSM wiki.  I first tried YOURS.  See the linked
> image here, an attempted route near [38.198108, -119.883907].  The routing
> algorithm refuses to jump from the road to the trail, instead choosing to do
> a long roundabout trip.  I tried several other online routers, and they all
> made the same choice, so I naturally assumed maybe there was a problem with
> the OSM data at this point.  But I examined it up close with JOSM and the
> ways seem connected properly (though I could be mistaken since I haven't
> used JOSM much).  To make matters stranger, I finally found a single service
> (out of all the many I tried), which is able to route correctly across that
> junction!  This service is BRouter, which the linked image here shows
> performing as desired.  However, BRouter does not have a public API, so I
> would seem to be out of luck for all options here.
> 4) Client side routing logic using OSM server-side data?  The routing
> algorithm would make many single element requests to the server, as it runs,
> downloading individual nodes/ways needed for the routing, rather than all
> data in vicinity as in option 2 above.  I don't even think this is a
> legitimate idea, since I would guess that there would be way too many
> messages sent back and forth between client and server, and communication
> cost would be far too expensive (though overall amount of data downloaded
> would be far less than option 2 above).  But the idea crossed my mind so I
> figured I'd add it to the list.  Maybe I'm wrong here though and the
> communication cost is affordable?
>
> Are the any other options I've missed?  Are any of my considerations of
> these 4 options wrong?  Which is the best option for my situation, in your
> opinion?
>
> Thanks in advance for any advice anybody can offer.  I really appreciate it.

Well options 2 and 4 are absolutely not feasible, assuming you are
talking about using the osm.org map query API. The OSM API is for
editing the map only, not for clients to query for use in some 3rd
party service. Of course some small non-editing related calls get
through but if you start querying a lot of data you will get blocked
pretty quickly. For non edit related querying there are other services
like jxapi or overpass. But even then, this does not sound like the
best idea.

The specific example you linked to does seem very odd. It does look
like the MapQuest Open routing service behaves correctly thought:
http://mapq.st/1067Po9

Looking on a couple of other routing services, the OSRM demo is set
for cars only so it won't even try to go on the highway=path. No clue
where the cloudmade router's head is. It will obviously use
highway=path but for some reason it is refusing to transition to it
right there. I can't figure out why either.

Toby

___

Re: [OSM-dev] Reminder: Node 32-bit exhaustion

[Toby Murray]

FYI, I have dubbed this the N2B (Node 2 Billion(ish)) problem :)

Just think what would happen if we could get 1/1000th the media
coverage of Y2K...

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Deploying Potlatch2 - local test version

[Toby Murray]

On Sun, Jan 20, 2013 at 11:38 AM, Rob Nickerson
 wrote:
>
> Hi All,
>
> I'm attempting to deploy P2 following the instructions on the wiki. At the
> moment I want to deploy this locally (as in, not on a public server), so
> that I can have a play with the snapshot sever / merging tool. I do not need
> to upload anything.
>
> Is it not possible to do this without registering my "server" with osm?

When I was testing a small patch for P2 I followed this page:

http://wiki.openstreetmap.org/wiki/Potlatch_2/Developer_Documentation

and had a running P2 instance up that talked to the dev API in no time
flat. I had to create a new account in the dev rails port but P2
worked fine against it.

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Planet file date unclear at planet.osm.org

[Toby Murray]

The timestamp you see in the directory listing is the last time the
file was modified, not when it was created. It is also in server local
time. Right now that happens to be the same as UTC but when London
enters daylight savings time it will be an hour off.

But the osm.bz2 file has the timestamp that you need inside of it in
the second line. No need to parse the whole file.

bzcat planet-130102.osm.bz2 | head -n 2



I used the pbf this last time which doesn't have the timestamp in it
but since the pbf is generated from the .osm.bz2 file, the same
timestamp applies to both.

Toby



On Fri, Jan 11, 2013 at 7:02 AM, sly (sylvain letuffe)
 wrote:
> Hi,
>
> For those who use the planet from planet.osm.org and apply diffs afterward,
> pay attention to the generation date of the planet.
>
> Here :
> http://planet.osm.org/planet/
>
> It says "latest" with a file generation date of 05-Jan-2013 05:08 but it isn't
> exactly clear when is the latest object in that file (or if there is, I'm not
> aware of the trick)
>
> To check the date of the latest object (beside parsing the whole file) you can
> go to  http://planet.osm.org/planet/2013/ and check the file's name date, then
> take some margin *before* to take your diff import sequence number.
>
> ps: on a side note, I heard there was ongoing work to provide a file's
> timestamp in the pbf file itselft, is that operationnal ? or planned to be ?
>
> --
> sly (sylvain letuffe)
>
> ___
> dev mailing list
> dev@openstreetmap.org
> http://lists.openstreetmap.org/listinfo/dev

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Screenshots from OpenStreetMap-Carto spot checking

[Toby Murray]

On Wed, Dec 19, 2012 at 11:51 AM, Alex Barth  wrote:
> I did some spot checking on the OSM-Carto style and summarized what I found 
> here in this diary entry. I've opened corresponding issues on the GitHub 
> project where appropriate. From a style perspective the carto port is 
> incredibly close and pretty much good to go. It looks like the performance 
> questions [1,2] that came up are more of a hurdle right now, others can speak 
> more educatedly to this.
>
> http://www.openstreetmap.org/user/lxbarth/diary/18261
>
> [1] https://github.com/gravitystorm/openstreetmap-carto/issues/16
> [2] https://github.com/gravitystorm/openstreetmap-carto/issues/20

I was doing some poking around yesterday and noticed that county
borders (admin_levl=6) aren't being rendered at zoom 9 and 10. But I
do recall some confusion on the existing style between ways and
relations. I don't remember the details but there was some difference
between the zoom level at which ways and relations tagged with
admin_level=6 got rendered. Maybe this caused confusion when porting
to carto? Easy place to see this difference:
http://bl.ocks.org/d/4271706/#9.00/39.4664/-96.9125

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Minutely changeset feed

[Toby Murray]

On Wed, Nov 7, 2012 at 10:15 AM, Matt Amos  wrote:
> On Fri, Nov 2, 2012 at 6:34 AM, Toby Murray  wrote:
>>
>> It looks like changesets don't show up until they are closed. This
>> makes sense since then you don't have to worry about information
>> changing.
>
>
> hmm... this shouldn't be the case - changesets should show up when they're
> opened and again when they're closed. if they're opened and closed in the
> same period, it will only show the closed state. this seems to be what
> happens with most changesets - they open and close within a few seconds.

I looked at 5 consecutive diffs just now and there was not a single
open changeset. Other random ones I've looked at in the past also
don't have any open ones. I find this highly unlikely since it is
actually kind of hard to manually close a changeset in P2. So I
suspect you have a bug. Possibly the same bug that affected the weekly
planet dump until a few weeks ago when I noticed it and Tom fixed it:

http://git.openstreetmap.org/planetdump.git/commitdiff/e406711139b9eb8db9d32fa4094325ec3841082c?hp=beb331bcdf6e5565b02fe61be397c150bc03

The problem was that all changesets have the closed_at field set to
one hour in the future whenever the changeset is touched. This is
stored in UTC. If one hour has passed since the last change, meaning
that closed_at is in the past, the changeset is considered closed. But
the SQL that determines the changeset's openness uses the now()
function which by default returns server local time. During daylight
savings time in London this means that, the "closed_at > now()" check
always returned false so all changesets were marked as being closed in
the weekly dump.

But then again, DST is no longer in effect... So it may be something else.


>>
>> However, what will it do in the edge case where a changeset is closed
>> but then reopened? Not possible you say? I once had an upload take
>> over an hour to process. It did eventually succeed but the changeset
>> was marked as closed in my changeset list for several minutes before
>> the upload finished and then it was suddenly open again. I am assuming
>> in this case the changeset would show up twice in the minutely diffs
>> and cause the INSERT queries of all consumers to explode violently :)
>
>
> yeah... i don't think the code is able to handle this at the moment... maybe
> it would see the re-opening, but probably not. any specific examples of this
> would be great, btw, to help me start building up some test cases.

Ok well if the changeset normally shows up twice in the feed anyway
then tools will need to handle this so it shouldn't be a problem on
the consumer side. But yes, the code generating the diffs does need to
take this into account. I know the osmosis diffs are based on database
transactions so they pick up any row that has changed since the last
diff. Applied to changesets, this would actually produce an entry in
the stream every time a chunk was uploaded (in JOSM chunked upload
mode)

By examples, do you mean a changeset ID? Since this is all time based
that won't do much good since now the changeset will look like any
other changeset that I held open for 2 hours.

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

[OSM-dev] Minutely changeset feed

[Toby Murray]

Since I have been playing with changeset metadata[1] a bit, Paul
Norman recently pointed out to me that there are now minutely diffs of
changesets available from planet.osm [2]. Would anyone care to share
some juicy details like where it came from and how it works?
Apparently it was mentioned in an EWG meeting. I should really check
one of those out...

Anyway, some observations/questions:

I see the state file is in a different format than all the other
replication state files. Probably not a big deal but is there value in
consistency?

Are there any existing tools to use this stream?

It looks like changesets don't show up until they are closed. This
makes sense since then you don't have to worry about information
changing.

However, what will it do in the edge case where a changeset is closed
but then reopened? Not possible you say? I once had an upload take
over an hour to process. It did eventually succeed but the changeset
was marked as closed in my changeset list for several minutes before
the upload finished and then it was suddenly open again. I am assuming
in this case the changeset would show up twice in the minutely diffs
and cause the INSERT queries of all consumers to explode violently :)

[1] https://github.com/ToeBee/ChangesetMD
[2] http://planet.openstreetmap.org/replication/changesets/

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] how to make way and relation history less expensive

[Toby Murray]

On Sat, Oct 20, 2012 at 11:50 AM, Kolossos
 wrote:
> Am 17.10.2012 12:39, schrieb Martin Koppenhoefer:
>
>> OK, but besides of reducing the load on the API the suggested
>> behaviour would also be more clearly laid out, at least for everything
>> that is not quite small. You won't usually look in detail at a list of
>> tens or hundreds of nodes and multiplied by every single version (and
>> maybe hundreds of versions).
>>
>> cheers,
>> Martin
>
>
> I would prefer the way of mediawiki to handling versions. The user get a
> short list of versions and can select diffs.

Are you talking about the API history call or the /browse/ pages history?

I *think* one of the most expensive parts of the /browse/ pages is
looking up all the sub-elements to be able to put tag-based icons by
nodes and put way names and the "(also part of way 
())" text next to things. This is why the history API call is
more likely to return than the /browse/ page in a browser.

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] (no subject)

[Toby Murray]

On Thu, Oct 11, 2012 at 10:03 AM, Toby Murray  wrote:
> On Thu, Oct 11, 2012 at 7:24 AM, Roland Olbricht  
> wrote:
>> Dear all,
>>
>> could somebody please explain why
>> http://www.openstreetmap.org/api/0.6/relation/970776
>> contains
>> 
>> but
>> http://www.openstreetmap.org/api/0.6/way/62318915/history
>> looks deleted?
>>
>> Best regards,
>>
>> Roland
>
> The way was added to the relation in version 51 at
> 2010-06-20T19:29:34Z and then the way was deleted one second later at
> 2010-06-20T19:29:35Z. In theory this should not be possible. However
> this edit was done in Potlatch 1 in live edit mode and according to
> RichardF on IRC, there were some race conditions between Potlatch 1
> and the API that could be to blame. It seems like this is the most
> likely explanation for this data.

After running a query oh my local database I think there are a total
of 208 relations which contain ways that are deleted. I may go through
and clean them up after I get back from SOTM-US. Hopefully since P1 is
declining in usage, there won't be any more created.

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] (no subject)

[Toby Murray]

On Thu, Oct 11, 2012 at 7:24 AM, Roland Olbricht  wrote:
> Dear all,
>
> could somebody please explain why
> http://www.openstreetmap.org/api/0.6/relation/970776
> contains
> 
> but
> http://www.openstreetmap.org/api/0.6/way/62318915/history
> looks deleted?
>
> Best regards,
>
> Roland

The way was added to the relation in version 51 at
2010-06-20T19:29:34Z and then the way was deleted one second later at
2010-06-20T19:29:35Z. In theory this should not be possible. However
this edit was done in Potlatch 1 in live edit mode and according to
RichardF on IRC, there were some race conditions between Potlatch 1
and the API that could be to blame. It seems like this is the most
likely explanation for this data.

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] [Talk-us] Abbreviating names in tools

[Toby Murray]

On Thu, Oct 4, 2012 at 1:26 PM, andrzej zaborowski  wrote:
> Hi,
> there's been a lot of talk at one point about abbreviating names in
> the OSM database vs. doing it when processing the data at consumers
> end.  Since mapnik now supports alternative label placements I gave
> rendering automatically abbreviated names a try.  This resulted in a
> (so far) tiny C library (https://github.com/balrog-kun/shrtnms) that
> abbreviates the names of map features that you give it.  It's very
> rough but it already handles a couple of the main corner cases.  It's
> just a start on collecting the list of all the abbreviations
> applicable in all the map languages.  Currently only has basic lists
> for Polish, Spanish and English, where the rules don't differ so much.
>  As an example German is more tricky.
>
> I see two ways to use it for rendering:
>
> * inside mapnik stylesheets, perhaps by calling the C function from
> the SQL queries.  This would require adding the postgres bindings,
> something I've never done.
> * inside osm2pgsql so that the abbreviated names are stored in table columns.
>
> As a quick hack I went for the latter option.  It gets tricky with
> hstore and multilingual maps but it works as a first attempt.  I have
> a patch at 
> http://osm.trail.pl/osm2pgsql/0002-Generate-short-name-columns.patch
> that makes necessary (small) changes and a snapshot of the library
> code, to osm2pgsql code.
>
> It ignores the actual language of a name and just tries to apply all
> the possible abbreviations from its list.  This will eventually need a
> solution perhaps based on the location of a given map object and a
> fixed list of country polygons with their main languages.  Perhaps it
> can use a common solution with highway shields rendering.
>
> One of the stylesheets at osm.trail.pl currently uses this code (only
> Europe imported at this time, but I CC'd talk-us because there was a
> long thread about this at one time).  You can see that at z16 here
> "Norbroke Street" is spelt in full:
> http://a.osm.trail.pl/osmapa.pl/16/32723/21790.png
>
> and z15 shows Norbroke St when there's not enough space.  (That
> stylesheet would also merge the two segments of Norbroke St and only
> show the name once, had that street not had a gap there.)
> http://b.osm.trail.pl/osmapa.pl/15/16361/10895.png
>
> How to apply
>
> In addition to the osm2pgsql patch you also need to add the
> auto-generated tags short_name and shortest_name to your osm2pgsql
> .style file so that they end up in the mapnik db.  Then in the mapnik
> 2 stylesheet where you use a  foo=bar>[name], you need to change it to:
>
>  placement-type="list">[name][short_name][shortest_name]
>
> If there's a tag short_name or shortest_name in OSM data, they'll
> override the autogenerated versions.  The difference between those two
> columns is in the degree to which they try to shorten the name, for
> instance for name=West Fulton Street, the new tags will be:
>
> short_name=W Fulton St
> shortest_name=Fulton
>
> For Polish if a street is named after a person, the person's
> first/second names will be first shortened to their initials and then,
> if necessary, omitted as is normally seen in cartography.  For Spanish
> all articles and prepositions are also stripped.  Unfortunately if the
> same rules are applied to US city names that often come from Spanish
> (Los Angeles) the same thing will happend (resulting in "Angeles"
> which makes no sense).  I also notice that different spanish
> abbreviations are used in Spain (Avenida -> Avda.) than in Latin
> America (Avenida -> Av.), so it's something to have in mind if running
> a global tile server or other service.
>
> Suggestions and lists of missing phrases are welcome, perhaps through
> github (but I'm away from my main machine this week).  If this is
> something that users want to see in map tiles then that list will be
> needed at some point even though the code doesn't yet handle all the
> nuances correctly.

Very nice work. I'm not sure if this will be of help to you or not but
as part of writing my ogr2osm translation for TIGER 2011 I converted
the TIGER technical documentation appendix E (Feature Name Types) from
PDF to a CSV file for easier processing. TIGER probably uses some
abbreviations that people may not always expect but it is a handy list
to have at least.

https://github.com/ToeBee/ogr2osm-translations/blob/master/tiger2011_abbrev.csv

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] [GSoC] Improvements to Vespucci

[Toby Murray]

Not sure if this has anything to do with your changes or not but I can
no longer use Bing imagery. If I select Bing as the background,
Vespucci crashes when I go back to the map. Then I can't even start it
up until I nuke all data to reset the background option back to mapnik
tiles. If it isn't related to your changes, I'm wondering if it might
have to do with problems people were seeing with Bing in JOSM and P2 a
couple of days ago.

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] [GSoC] Improvements to Vespucci

[Toby Murray]

On Wed, Jul 25, 2012 at 4:46 PM, Jan Schejbal
 wrote:
> Am 2012-07-25 17:30, schrieb Toby Murray:
>> Question: What does the "Save" button do? I couldn't find any files on
>> the file system after I used it. Maybe I was just looking in the wrong
>> place?
>
> Sorry, first reply didnt go to list:
> I have removed the button, as it only does what the auto-save feature
> does automatically (and now reliably).

Aha. All questions answered. I thought maybe you had implemented the
"save to file to edit in JOSM before uploading" feature while I wasn't
looking :)

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] [GSoC] Improvements to Vespucci

[Toby Murray]

I finally decided to give this a try. For some reason I couldn't
install your APK at first. It just failed saying "Software not
installed" or something along those lines. I uninstalled Vespucci from
my phone and then your APK worked fine. So now sure what was going on
there.

But now that I have it working, I think I like your changes!

Question: What does the "Save" button do? I couldn't find any files on
the file system after I used it. Maybe I was just looking in the wrong
place?

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

[OSM-dev] Note to users of overapss/xapi like services (mirrored download and such)

[Toby Murray]

I noticed today that since the license bot is going full throttle, my
server hasn't been able to keep up with minutely diffs any more. Right
now I'm about 4 hours behind. My server isn't exactly optimized for
performance so this might not be affecting all users of minutely diffs
but please beware that the increased activity may have impacted some
3rd party services. If you are downloading data to edit from anywhere
but api.openstreetmap.org, please double check your source and verify
that it is up to date. I know both the jxapi and overpass are capable
of putting a timestamp in their response, if configured correctly.

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Update on redaction bot and minutely diffs

[Toby Murray]

Andy just pointed out that it should probably be mentioned that you
only need to mess with the osmosis state file if your osmosis was up
to date with planet.osm.org when the invalid diffs were generated and
it progressed beyond sequence number 141272.

I believe Grant deleted the invalid diffs pretty quickly so if you
were trailing behind on updates or if you only update periodically,
your osmosis may not have gotten up to the invalid diffs and no action
is required. So check the sequence number before replacing the state
file.

Toby


On Thu, Jul 12, 2012 at 9:32 AM, Toby Murray  wrote:
> As I mentioned yesterday, the bot caused some problems with minutely
> diffs. Andy just sent a more detailed message about the technical
> problems to the rebuild list but here is a quick update for the user
> side of things.
>
> Some invalid diffs were generated yesterday. These have been removed
> from planet.osm.org and replaced this morning with valid files after a
> workaround was put into osmosis. This means that if you have a setup
> that is consuming minutely diffs, you may need to manually intervene.
> According to Grant, the last valid diff was:
> http://planet.openstreetmap.org/redaction-period/minute-replicate/000/141/272.osc.gz
>
> So you will want to stop osmosis and replace its state.txt file with
> the contents of:
> http://planet.openstreetmap.org/redaction-period/minute-replicate/000/141/272.state.txt
> And then restart your minutely update process. The first 20 or so
> diffs are substantially larger than normal so it may take a while to
> catch up.
>
> As for the bot itself, it was stopped after a few hours yesterday
> because some problems were discovered with its handling of relation
> members. Again, see the rebuild list for details. The short version is
> that he fixed it and as of a few minutes ago the bot is running again.
>
> There was also a problem caused in JOSM which might affect you if you
> are working in an area with other people. If someone else deletes a
> node and then you try to update it, JOSM had problems dealing with the
> deleted node because of a slight change in the API. This has already
> been fixed according to this trac ticket:
> http://josm.openstreetmap.de/ticket/7847
>
> So I think we've had our glitches. It'll be smooth sailing from here
> on, right? :)
>
> Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

[OSM-dev] Update on redaction bot and minutely diffs

[Toby Murray]

As I mentioned yesterday, the bot caused some problems with minutely
diffs. Andy just sent a more detailed message about the technical
problems to the rebuild list but here is a quick update for the user
side of things.

Some invalid diffs were generated yesterday. These have been removed
from planet.osm.org and replaced this morning with valid files after a
workaround was put into osmosis. This means that if you have a setup
that is consuming minutely diffs, you may need to manually intervene.
According to Grant, the last valid diff was:
http://planet.openstreetmap.org/redaction-period/minute-replicate/000/141/272.osc.gz

So you will want to stop osmosis and replace its state.txt file with
the contents of:
http://planet.openstreetmap.org/redaction-period/minute-replicate/000/141/272.state.txt
And then restart your minutely update process. The first 20 or so
diffs are substantially larger than normal so it may take a while to
catch up.

As for the bot itself, it was stopped after a few hours yesterday
because some problems were discovered with its handling of relation
members. Again, see the rebuild list for details. The short version is
that he fixed it and as of a few minutes ago the bot is running again.

There was also a problem caused in JOSM which might affect you if you
are working in an area with other people. If someone else deletes a
node and then you try to update it, JOSM had problems dealing with the
deleted node because of a slight change in the API. This has already
been fixed according to this trac ticket:
http://josm.openstreetmap.de/ticket/7847

So I think we've had our glitches. It'll be smooth sailing from here
on, right? :)

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] [OSM-talk] Licence redaction ready to begin

[Toby Murray]

On Tue, Jul 10, 2012 at 2:10 AM, Paul Norman  wrote:
>> From: Richard Fairhurst [mailto:rich...@systemed.net]
>> Sent: Monday, July 09, 2012 1:47 PM
>> Subject: [OSM-talk] Licence redaction ready to begin
>>
>> Test runs have shown that the bot is functioning as we want it to, but
>> we will of course be monitoring its progress. We are currently expecting
>> it to take in the order of one month to complete; given the many
>> variables I'm afraid we can't give a more precise steer yet, but we'll
>> aim to keep everyone updated as it runs (via the announce@ and talk@
>> lists).
>>
>> There will be _no_ API outage and no other interruption to editing. When
>> the bot is running in your area, please do save your edits frequently to
>> minimise the likelihood of conflict.
>
> I know that there was some discussion as to if it would be necessary to
> refrain from conducting any imports while the redaction is in progress.
> Imports are not likely to conflict with the redaction bot but may cause load
> issues on the API. If it is (or becomes) necessary to restrict imports while
> it is running, can we get an official announcement to the blog (or
> announce@)

Also, will the diff location change again once the real redaction
period starts? The original intent of moving the diffs was to protect
consumers from damage to their map and to force them to take some
action because of the license change. However since there was no
license action to take, a lot of them have updated to point at the
"redaction period" diffs. Hopefully all of them are aware of what is
happening now so it may not be critical but still something to
consider.

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] GSoC-Video based speed limit and roadsign detector and Request for videos of videomapping

[Toby Murray]

On Tue, Jun 26, 2012 at 4:17 AM, Nikhil Upadhye
 wrote:
>  Hi,
>
> Thanks everyone for videos.
> Here is update of my project. My repositories can be found
> at http://github.com/nikhil9/
> I have developed code using javacv for surf based sample matching. It
> matches given sample image with provided image. This code is working good
> for similar light condition images. I am working on varying light
> conditions. I am also preparing sample datasets for haartraining. I have
> created sample test plugin 'videoprocessor' which I will further develop to
> process images/videos from josm.

Well if you really want harsh lighting, you might look at the video I
just uploaded the other day from biking across Kansas. It is full
motion video too, not timelapse. It was shot while heading directly
into a sunrise so the lighting is terrible. Good luck :)

I don't think there are any interesting signs past the 65 mph sign at
just after the 2 minute mark.

http://www.youtube.com/watch?v=rT499J-C4wM

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] GSoC-Video based speed limit and roadsign detector and Request for videos of videomapping

[Toby Murray]

On Sun, May 20, 2012 at 4:19 AM, Nikhil Upadhye
 wrote:
> Hello Everyone,
>
> I am Nikhil Upadhye. I will be developing plugin for JOSM for Video based
> speed limit and roadsign detection as a part of Google Summer of Code this
> year. I will be using OpenCV java bindings - JavaCV for video processing
> purpose.
> I have setup wiki page at following link
>
> http://wiki.openstreetmap.org/wiki/Google_Summer_of_Code/2012/Video_Based_Speed_Limit_and_Road_Sign_Detector_plugin_for_JOSM
>
> I would like to request mappers to share videos which they have recorded
> for video-mapping. Variety of such videos will be required to test
> and develop plugin. So it would be great if someone could share such videos.
>
> Any suggestions regarding project are welcome.

I was going to reply to this earlier but didn't get to it immediately
and now I've been gone for a few days. Just had a conversation on IRC
that reminded me of this topic.

I do have some videos that might be of use to you. They are time-lapse
videos that I made using a GoPro camera mounted on my bicycle on some
longer rides. Pictures were taken at 5 second intervals and then
stitched into a 10 fps video. Any given sign is only going to be
visible for maybe 2-5 frames, depending on how fast I was going.

The videos are on youtube in 1080p, under a CC license so I think it
should give you the option to download to a .mp4 file for local use. I
also still have the original 11 megapixel stills if that would be of
help.

The two videos are here:
http://www.youtube.com/watch?v=Q8I7H8qwtqI
http://www.youtube.com/watch?v=h43tsSNqYRA

I also have a shorter ride in the city that I did in full motion video
with the camera mounted on my helmet and then sped up. It is kind of
shaky. I could supply the original speed video if that would help. I
believe it is only 720p.
http://www.youtube.com/watch?v=pJ_OkzGBoek

I'm doing Bike Across Kansas next week so I will probably be posting
more similar videos in the future :)

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Tile acess statistics?

[Toby Murray]

On Tue, May 1, 2012 at 10:56 AM, Nathan Vander Wilt
 wrote:
> I'm trying to estimate bandwidth and storage requirements for a potential 
> project, and was hoping I could find info on OSM tile usage patterns.
>
> What I'm wondering is how the distribution of "all tiles" to "tiles that 
> people actually use" looks like. Questions like e.g. "Are there some tiles 
> that are pretty much never accessed?" "What percent of tiles are very 
> frequently accessed?" "What are the most active cities/countries/etc.?" (I'm 
> sure I'll think of more questions as I get further along, but that's 
> basically the kinds of stats I'd like to be able to play with.)
>
> I found some older User Agent statistics here:
> http://wiki.openstreetmap.org/wiki/Stats#Tile_usage_stats
>
> And an even older script that uses the access log here:
> http://lists.openstreetmap.org/pipermail/dev/2009-November/017651.html
>
> ...but I cannot find any link to the actual access logs used for those 
> statistics. Are there recent "raw"/anonymized logs available for community 
> number crunching experiments?

Don't know about logs but here is another wiki page that is relevant:
http://wiki.openstreetmap.org/wiki/Tile_Disk_Usage

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Planet Updates

[Toby Murray]

Replication was broken on purpose because some data consumers may not
want to process the redaction diffs. In general I might suggest that
tools targeted at mappers probably should process them whereas things
that go to end users may not want to since it will break the map
pretty hard for a while. Unfortunately because the timing of the
license bot has been rather undefined, the diffs got moved to the new
location a bit prematurely. At the time the database came out of
read-only mode, the license bot was going to start "any day now" so
the diffs were moved. But  now we have 3 weeks worth of "redaction"
diffs with no redaction in them.

Toby


On Wed, Apr 18, 2012 at 5:39 AM, Shane Reynolds  wrote:
> Hi Frederik,
>
> Thanks for the info. Realised the scripts would need to change once the new
> DB was available, just did not want to deal with that twice if the new
> planets were likely to appear soon (we always use the latest planet file and
> then just the diffs for the current week). However as it sounds like the new
> DB is a little while away so it may be worth processing the redaction
> directory in the interim period.
>
> Shane
>
>
>
>
> On 17 April 2012 21:14, Frederik Ramm  wrote:
>>
>> Shane,
>>
>>
>> On 04/17/2012 04:22 PM, Shane Reynolds wrote:
>>>
>>> We are basically wondering whether we should try and process the
>>> Redaction directory for the time being until the new stuff is online.
>>> However I don't want to change all the import scripts etc if the new
>>> planet is likely to be online soon. Anyone have any rough ETA on this?
>>
>>
>> My personal and totally unoffical and not-in-touch-with-OSMF-board
>> assumption is now that the license change will be complete some time in May.
>> (This year.)
>>
>> But: No matter when the diffs are back, they will be in a different
>> location than before, so you will have to change your configs anyway. This
>> is because everyone will have to do a full import after the license change;
>> you can't just go on processing diffs and act as if your local database had
>> magically become ODbL.
>>
>> Bye
>> Frederik
>>
>>
>> --
>> Frederik Ramm  ##  eMail frede...@remote.org  ##  N49°00'09" E008°23'33"
>>
>> ___
>> dev mailing list
>> dev@openstreetmap.org
>> http://lists.openstreetmap.org/listinfo/dev
>
>
>
> ___
> dev mailing list
> dev@openstreetmap.org
> http://lists.openstreetmap.org/listinfo/dev
>

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Tile centered on POI... howto?

[Toby Murray]

On Tue, Apr 17, 2012 at 3:15 AM, Morten Kjeldgaard  wrote:
> Hi,
>
> I'd like to present tiles centered on certain POIs for use on a web page.
> Pretty much like it is done in opestreetmap.org's node browser, except I'd
> like a square image.
>
> Is there a smart way to do this, other than downloading the center tile plus
> the 8 surrounding tiles, assembling the 9 tiles into a 768 x 768 image, and
> finally cropping it down to a 256 x 256 tile centered on the POI?

You could use a static map API:
http://wiki.openstreetmap.org/wiki/Static_map_images

If you plan on getting any kind of load I would suggest looking at the
MapQuest Open one since it doesn't depend on limited OSMF servers. It
is also gives you a lot of control over the final product.

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Yevaud SSD Drive

[Toby Murray]

On Thu, Apr 12, 2012 at 4:33 PM, John Perrin  wrote:
> Hi,
>
> I've posted this question on the OSM Q & A site a well, not sure what the
> best forum for the question is, so please forgive the dual post if you also
> follow that site.
>
> Basically, I was just inquiring into the specific need for the SSD drive on
> the yevaud tile server.  I'm looking to run an OSM tile server that can
> handle roughly 200,000 - 400,000 map views a day and have taken this as a
> good benchmark for the server spec.  However the SSD is half the cost of
> reproducing a server with that spec.  I was just wondering exactly what the
> disk was used for, and why is specifically needed the SSD drive. I can see
> the purchase logged in the server upgrade history, but I can't see any
> reason explaining why it was needed.

See the yearly graph here:
http://munin.openstreetmap.org/openstreetmap/yevaud.openstreetmap/renderd_queue.html
and here:
http://munin.openstreetmap.org/openstreetmap/yevaud.openstreetmap/renderd_processed.html

The second graph had a hiccup in August that makes it kind of hard to
read the other values but at the left edge you can see the render
queue was full for a lot of the time and it was dropping render
requests a lot. Then in May of last year the SSD was installed. Now I
think the only time the render queue gets full is short spikes of peak
load (when OSM hits slashot and the like)

The SSD holds the postgis database that mapnik executes queries
against to pull the data for rendering. Before the SSD, I believe
yevaud was running a fairly beefy RAID 10 so the SSD really did make a
big difference.

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Can tag values be longer than 255 chars?

[Toby Murray]

On Wed, Apr 4, 2012 at 10:10 AM, Morten Olsen Lysgaard
 wrote:
> I'm running an instance of the OSM stack. I'd like to have some key values
> longer than 255 chars. Is this a hardcoded limit. Can it be lifted? Does it
> need a migration?
>
> As always, thanks for the help! =)

It is part of the database schema:
https://github.com/openstreetmap/openstreetmap-website/blob/master/db/structure.sql

Look specifically at the create statement for node_tags, way_tags and
relation_tags tables.

I imagine it might also be encoded in rails somewhere too in order to
check uploads and send error messages back to the client. Not sure
about that though.

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] minute dumps

[Toby Murray]

On Wed, Apr 4, 2012 at 12:55 PM, Frederik Ramm  wrote:
> Hi
>
>
> On 04/04/2012 07:21 PM, Eugene Alvin Villar wrote:
>>
>> This might be getting off-topic to this list but to respond: according
>> to this article[1], the author has contacted Bing and that Bing
>> "stated that they use no OSM data whatsoever."
>
>
> This is clearly wrong since Bing have already admitted to use OSM data for
> the blurring of military areas in Germany.
>
> I guess it will be the typical communications cock-up with journalists and
> press officers. The original sentence probably contained a few qualifying
> adjectives which were then omitted in the article.
>
> I think it would, for example, be correct to say that Bing uses no OSM data
> whatsoever *in the US* *in its current product range*" - which is not saying
> that they don't use OSM data elsewhere, as they do for the German blurring,
> and not saying that they don't have any OSM-based products in the pipeline
> or at least in the lab either.
>
> (http://frontdoor.cloudapp.net/ doesn't count - it claims to contribute data
> to OpenStreetMap but it doesn't *use* OSM.)


Is this still running? I believe it requires silverlight so I never
actually saw it first-hand. Either way, Bing concsuming OSM data is
not news...

http://www.bing.com/community/site_blogs/b/maps/archive/2010/08/02/bing-maps-adds-open-street-maps-layer.aspx

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Training during read only database period

[Toby Murray]

On Sat, Mar 31, 2012 at 1:22 PM, Mike  Dupont
 wrote:
> Yes, we have the fosm.org server you can use commonmap.com
>
> and I have today implemented a test server with a fake oauth that allows you
> to edit with potlach, josm and merkaartor
> code is here https://github.com/h4ck3rm1k3/FOSM-Api, you can easily set it
> up, requires no database, the only thing that is hardcoded is the path where
> it writes the output, i will have to make that configurable.
>
> The potlatch is installed here you can try it.
> pine02.fosm.org/random.dev.openstreetmap.org/potlatch2/potlatch2.html
>
> the api root is here : you can configure your client to use it.
> pine02.fosm.org/FOSM-Api/OSM-API-Proxy/public/dispatch.cgi,
>
> the changesets are written here :
> http://pine02.fosm.org/FOSM-Api/OSM-API-Proxy/public/changesets/


For the record, none of these are OSM services. FOSM is a fork of OSM
data performed by a few people who were not happy with the license
change and decided to make their own OSM-like project that will remain
under a creative commons license. Also, last I knew they didn't do
live rendering of changes so that may not be as useful for your
purposes.

If you were planning on using JOSM, you should still be able to
download data and make changes locally. You just won't be able to
upload them. I'm not sure if Potlatch will be available at all or not.

One thing you could do is download data and edit it in JOSM and then
save it out to a file on disk and render it locally using, for
example, Maperitive. I have a blog post about using Maperitive if you
would like to investigate this more:
http://ksmapper.blogspot.com/2011/08/simple-openstreetmap-tile-rendering.html

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] GSoC - Improvements to Vespucci

[Toby Murray]

On Sat, Mar 24, 2012 at 3:53 PM, Graham Jones  wrote:
> Hi Jan,
> My personal order of preference for the features you suggest would be:
>
> Simple Long Click POI Menu - this would be a big step forward for  usability
> - I think the application has most of the fundamental features that I would
> use, but it takes too many clicks to do things.   It would be good to write
> the code so that the contents of the menu can be customised for other uses
> (ie very specialised simple editors etc.).
> Auto Save Function - I think I have lost things before, but don't know how.
> Undo function - Always good in case you make a mistake.
> Custom API support - this sounds potentially useful, but not something I
> have a need for at the moment.
> Offline Background Map - I am not really interested in this because I try to
> use vespucci when I am out and can look at the real world, so don't need a
> background map for reference.
>
> In terms of estimating how much effort is required, it really depends on how
> quickly you manage to get up to speed with the existing code.  This means
> that the first thing you work on will take most of the time, and the others
> are likely to be (comparatively) straightforward.
>
> I would recommend writing your application proposal as a 'base project' with
> maybe a couple of the features you are thinking of, which you can be
> confident in achieving, and identify the others as potential extensions if
> it goes smoothly.
>
> Hope that helps.

As a Vespucci user myself I have a couple of thoughts.

Any kind of save feature should save out to a file that can be read by
JOSM. I always have to go back and clean up my vesupucci edits after I
upload them. Sometimes it is great to be able to upload from the
phone. But sometimes I would rather just take the phone home, load a
file into JOSM and upload correct data on the first attempt. This is
in the issue tracker:
http://code.google.com/p/osmeditor4android/issues/detail?id=83

Also, anyone looking at the code should keep an eye out for this:
http://code.google.com/p/osmeditor4android/issues/detail?id=91

It's a nasty bug that I suspect some people aren't even aware is
affecting their edits. The first time I discovered it, I had been
mapping for over an hour and ended up having to throw most of it away
because my data had been completely corrupted when this bug was
triggered 30 minutes into my session. The only way to spot it is to
double check the tags on every new object you create.

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Web-devs: check your maps, Tiles@home does'nt exists anymore.

[Toby Murray]

On Thu, Mar 15, 2012 at 1:46 PM, yvecai  wrote:
> Just to say that a lot of websites showing a map are broken due to
> openlayers looking for osmarenderer layer.
> So, check yours!

A good first place to start might be our own wiki...
http://wiki.openstreetmap.org/wiki/Idaho
http://wiki.openstreetmap.org/wiki/Arizona
http://wiki.openstreetmap.org/wiki/Arkansas
http://wiki.openstreetmap.org/wiki/North_Dakota

And that's just a few specific states I looked up. Is there any way to
do a search for all of the wiki instances of the  component
that reference the osmarender layer?

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] How extract the comments of each changeset?

[Toby Murray]

On Wed, Mar 14, 2012 at 9:03 AM, Maurizio Napolitano  wrote:
> I need to identify the data imported into OpenStreetMap in a region.
> In particular, activities related to the tracing of aerial photos.
> This information is often contained in the changeset comments.
> How can I rebuild this information?
> I think it is very difficult

You can use the weekly changeset dump from
http://planet.openstreetmap.org - it contains all the metadata for all
changesets including the bounding box that they affected, number of
objects and all tags including comments.

I wrote a python script to parse it into a postgres database so that
it can be queried:
https://github.com/ToeBee/ChangesetMD

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] [Talk-us] Tracking Deleted Streets and Comparison Tools

[Toby Murray]

On Sat, Feb 18, 2012 at 6:03 PM, Humphries, Grant  wrote:
> Hello,
>
> I am attempting to track deleted streets and other changes that have
> significant effects on routing between versions of OSM data that are
> approximately one and two months apart.  My region of interest for this
> process is the Portland metropolitan area which is home to about 2 million
> people and covers around 17,000 square kilometers.
>
> I work on a trip planner that routes on OSM data so the purpose of this
> effort is to ensure the integrity of the data each time we update, which
> happens every several weeks.  Does any one know of tools or processes in
> existence that could be used for this type of task?

This reminds me of the TIGER fixup "routing grid" tools I have heard
rumors of. I didn't join OSM until after these tools had served their
purpose and been discontinued. But as I understand it, the idea was to
do kind of a routing unit test on OSM data to detect where things were
broken. In this case it took the form of 250 cities in the US that you
should always be able to route between with somewhat predictable route
distances. Distances outside of the norm were flagged as troublesome.
Here is a wiki page about it:
http://wiki.openstreetmap.org/wiki/TIGER_fixup/250_cities/routing_grid

Not sure how many of the tools still exist or still work. But it might
be a starting point.

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Using changesets-latest.osm

[Toby Murray]

On Mon, Jan 30, 2012 at 9:47 AM, Josh Doe  wrote:
> On Sat, Jan 28, 2012 at 5:01 PM, Toby Murray  wrote:
>> I present ChangesetMD:
>>
>> https://github.com/ToeBee/ChangesetMD
>
> What kind of stats are you (and others) thinking about generating? I'm
> interested in seeing editor stats such as a time series of editor
> usage, which editors people use (predominantly one or a mix, progress
> from using one to another, etc.), the frequency and "quality" of
> comments by editor, what tags people use besides the standard
> comment/source/etc.
> -Josh

My initial use of this data was to get a specific list of changesets
for dealing with the license change. But I certainly had in mind that
it would be useful for some interesting analysis like what you have in
mind.

Off the top, I can say there are just over 700 unique key values used
in tagging changesets. Some of them are obvious errors, some of them
are actually interesting. There are 630,000 changesets with no changes
in them. Didn't P1 create a changeset as soon as it was fired up, even
if no changes were made? There are 239 changesets with 50,0001
objects. Off-by-one error in the API? :)

And that's just the first few things that came to mind. I haven't had
much time to get into the data yet. Heck, I'm still working on
processing my survey data from Christmas :/

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Using changesets-latest.osm

[Toby Murray]

I present ChangesetMD:

https://github.com/ToeBee/ChangesetMD

As noted in the readme, this is my first real python so I can't
guarantee it won't try to erase your hard drive although this seems
kind  of unlikely :)

Right now I have indexes on tag key/values and user name. Depending on
what you are querying on, others might be called for. I'll see what
happens when I start playing with the data.

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Using changesets-latest.osm

[Toby Murray]

On Sat, Jan 21, 2012 at 5:17 PM, Frederik Ramm  wrote:
> Hi,
>
>
> On 01/21/2012 11:34 PM, Toby Murray wrote:
>>
>> So is there some other utility that has already been written to parse
>> the changeset dump and push it into a database of some kind?
>
>
> Pardon the question but you *are* aware that changesets-latest.osm.bz2 does
> only contain changeset metadata, not the actual information on *which*
> objects have been changed in each changeset? Therefore, while importing that
> into an apidb schema is perhaps not completely useless, it is of limited
> use.

Yes, I am aware of this. I want to query the metadata. Comment tags,
bboxes, users and such. I don't really care if it is in the apidb
schema or not. My first thought was that this would "just work" with
osmosis but that obviously isn't the case. I was just wondering if
someone else already has something. It sounds like the answer is no.
Which kind of makes me wonder why the changeset dump even exists if
there are no standard tools to use it. But whatever. It's not like
parsing XML is hard. I can write something to do what I need pretty
quickly.

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

[OSM-dev] Using changesets-latest.osm

[Toby Murray]

I wanted to do some poking through changesets so I downloaded
changesets-latest.osm. I thought I could import it into an apidb
schema using osmosis. I used the --write-apidb task and osmosis
created a single empty changeset with a replication=true tag so it is
obviously trying to do something very different than I intended.

So is there some other utility that has already been written to parse
the changeset dump and push it into a database of some kind? I can see
how pushing it into an apidb schema would be problematic since you
would either have to disable the foreign key constraint on users or
create dummy user records with bogus values in some of the required
fields.

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] How did Mapquest do the localized rendering schemes?

[Toby Murray]

On Tue, Dec 27, 2011 at 12:38 PM, Stephan Knauss 
> Thanks for the pointer to the styles. Any Idea how they determine which
> style to use for which geometry?
> I did not see anything that looks like a way to select the one style or the
> other based on the geographic location.
>
> Can you do the location check while rendering or would it be a preprocessing
> step during import? I did not see an additional "region" column so no idea
> how the selection was done.

I thought this was the case but after a little searching, I came up
with proof. MapQuest added this feature to Mapnik:
http://lists.openstreetmap.org/pipermail/talk-us/2010-November/004820.html

So it is apparently determined at render time. No clue how though :)

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Nominatim (npi / osm)?

[Toby Murray]

I am not very familiar with the internal workings of Nominatim but I
do know that it takes some very serious hardware to build a database
from the ground up using an OSM planet file. I believe this is why the
pre-indexed feature came to be. It allows you to maintain a nominatim
database without overly expensive equipment. As far as I know,
MapQuest Open is currently the only source of pre-indexed data. Have
you read their pages about it?

http://developer.mapquest.com/web/products/open/nominatim/indexed

>From that page, there is a link to the developer guide which has
additional details. Although looking through there it seems like their
service may be broken. The last full dump is from March and the last
diff is from October...

Toby


On Fri, Dec 16, 2011 at 3:31 PM, Nonmaskable Interrupt
 wrote:
> Hello!
>
> I am doing some development and experimentation with nominatim and am
> confused by what is the best  path forward with creating/searching nominatim
> databases
>
> One seems to be NPI
> (http://wiki.openstreetmap.org/wiki/Nominatim/Structured) and the other is
> via osm2pgsql/gazetteer (http://wiki.openstreetmap.org/wiki/Nominatim)
>
> In some ways the two different software distributions referenced in those
> discussions do not seem to conflict, but in others they do -- e.g.
> conflicting documentation, database/table names, shared objects etc.  I've
> looked for an active discussion to explain the situation, but can't seem to
> find anything.
>
> Thanks.
>
> ___
> dev mailing list
> dev@openstreetmap.org
> http://lists.openstreetmap.org/listinfo/dev
>

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] OSM-3D tagging info

[Toby Murray]

On Fri, Dec 16, 2011 at 7:17 AM, Jaime Crespo  wrote:
> (compared to 1-metre-acurate always-updatable import).

So you've solved the import update problem? Mind sharing? There are
13778200 TIGER ways here in the US that are very interested in this
topic.

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] [OSM-talk] Friends

[Toby Murray]

On Thu, Dec 15, 2011 at 5:19 PM, Martijn van Exel  wrote:
> Cool. It gives me a 500 error when you're not logged in though.

There exist people who aren't logged in to osm.org at all times?!

But seriously, this is great. Thanks!

Now I see how lazy some of my OSM friends have been. Next feature
request: a "poke" button! (no, just kidding)

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] [OSM-talk] Friends

[Toby Murray]

Nice!

On Wed, Dec 14, 2011 at 8:55 PM, Mikel Maron  wrote:
>
> 
> From: Toby Murray 
> To: t...@openstreetmap.org
> Sent: Friday, December 2, 2011 5:03 PM
> Subject: Re: [OSM-talk] Friends
>
> One thing I have thought might not be too hard to code up and provide
> some use would be to have a "Recent edits by my friends" page that
> just accumulates recent edits by your friends onto one page and
> displays it with bboxes like the single user edit history page. Right
> now you can only see the changeset comments from the last edit your
> friends have made. To see more is at least 2 clicks for each friend.
>
>
> Yup, sounded cool and simple. Pull request made!
>
> https://github.com/mikelmaron/openstreetmap-website/commit/db497585b41a68278883ab8dbd3f2c56179b28a8

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] License Change View on OSM Inspector

[Toby Murray]

Suggestion:

Add a link to view the selected object here:
http://osm.mapki.com/history/

It is much easier to view than the history on osm.org plus it actually
works for objects with high version numbers unlike osm.org that just
times out when you try to bring up the history.

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] ER Diagram of OSM Database

[Toby Murray]

Well there aren't any database level foreign key constraints in the
osm2pgsql schema so database visualization tools won't give you
anything useful to look at. Each table is its own island. This schema
is denormalized for rendering performance and the import process is
lossy. Only objects defined in the style are imported. Everything else
is ignored. The wiki page explains things a little although it could
use some cleanup by someone who is more familiar with it. Phrases like
"I noticed these tables being present after an import in slim mode.
They seem to be for the first import stage" don't exactly inspire
confidence...

http://wiki.openstreetmap.org/wiki/Osm2pgsql/schema

Toby


On Mon, Dec 12, 2011 at 10:20 PM, Parveen Arora  wrote:
> On Mon, Dec 12, 2011 at 4:14 PM, Andreas Hubel  wrote:
>> Hello Parveen,
>>
>> Am 12.12.2011 um 19:50 schrieb Parveen Arora:
>>> I was searching for ER Diagram of OpenStreetMap's Database to
>>> understand completely,
>> I think the best way is to understand the different parts step by step and 
>> not to try to understand everything at the first time.
>>
>>> I found one component diagram
>>> http://wiki.openstreetmap.org/wiki/Component_overview
>>>
>>> But I want one only of database?
>> There is not one database. The main services provided by the foundation 
>> actually use two different databases, as shown in the component overview you 
>> already found.
>> Green: PostgreSQL backend
>>        aliases: api db, main db, core db
>> Yellow: PostGIS
>>        aliases: osm2pgsql db, Mapnik db
> Yes, I want to have the diagram Yellow:PostGIS, the diagram of
> database of osm file stored using osm2pgsql.
>
>
>> We should definitely discuss about using uniform names in future.
> Yes, I realized this lately after posting the email, I will take care
> of it from next time.
>
>
>
> Thank You.
>
>
>
>
> --
> Parveen Arora
> www.parveenarora.in
> E-Mail: m...@parveenarora.in
>
> ___
> dev mailing list
> dev@openstreetmap.org
> http://lists.openstreetmap.org/listinfo/dev

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Fw: Re: Merkaator port to Android

[Toby Murray]

On Sun, Dec 4, 2011 at 5:54 AM, Nick Whitelegg
 wrote:
> -Forwarded by Nick Whitelegg/FT/Solent on 04/12/2011 11:53AM -
> To: jaak.laine...@gmail.com
> From: Nick Whitelegg/FT/Solent
> Date: 04/12/2011 11:53AM
> Subject: Re: [OSM-dev] Merkaator port to Android
>
>>- drawing draft lines and notes on top of that, saves it as GPX which
>>needs postprocessing on JOSM
>
> (Sorry, meant to send this to the list. Damn the email client! ;-) )
>
> This is actually quite similar to an idea I had in mind for opening up OSM 
> contribution to casual walkers/hikers. Such an app would record GPX (only). 
> However, a user would be able to select the highway type (footpath, 
> bridleway, cycleway etc) which would then be tagged in the GPX.

This is exactly what OSMTracker for Android does. It records GPX
traces and lets you add any of various preset notes via big graphical
buttons which are recorded in waypoints. When the file is opened in
JOSM it shows the text that was recorded in the waypoints. It also
supports recording audio and taking pictures. As long as the
pictures/audio clips are exported along with the GPX file, JOSM shows
a small icon for these as well and when you click on it, it either
opens the built in photo viewer or your system's default auido
playback application to play back the audio note.

It even listens for the media button event that Android has so it will
automatically start recording an audio clip when you push the
play/pause/answer button on your headset. I recently used this to
record things on a car trip after it got too dark to take pictures.

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Merkaator port to Android

[Toby Murray]

I would suggest looking at helping to improve Vesupcci. It already
does several things mentioned here and I think a few other things are
at least theoretically on the roadmap. It is certainly usable on my
Samsung Galaxy S. Editing geometry is kind of tricky and I ususally
end up going back in JOSM and fixing things after I upload from
Vespucci. But I don't see many options to change that on a small touch
screen. Tablets might work better. Having an "orthogonalize" button
might be neat though. One outstanding feature request is to save to a
file that you can open in JOSM and edit before uploading.

It has tagging presets built in although they are not graphical... it
just offers autocomplete suggestions for tag keys and values that it
knows about. So you have to know which tag you want, it just helps you
fill it in quicker. But it does have a button that will send you to
the wiki for the selected key.

It also guesses the road name when you add an addr:street tag. In my
experience it does fairly well.

It displays Bing imagery by default but has several other options.

It even does some minimal validation - highlighting streets with no name.

Toby



2011/12/2 Matthias Meißer :
> Well I've got Merkaator running on my OpenPandora handhelt (Angstrome Linux)
> and noticed that this kind of editors (let's call them GIS centred) isn't
> what will work on mobile devices in the field.
> I used osm2go as well and it's realy clother to my needs but is unfortunatly
> abandoned and currently not that good for tapping devices. On the other
> sides regular Smartphones are just to small (virtual keypad) so you might
> need a real hardware keyboard as the Pandora offers, to add streetnames etc.
>
> What in my opinion will work esp. on Tablets is:
> -easy to use download data (select area on map, not entering them
> numericaly)
> -ultimate reduced UI (focused on adding more attributes and just POIs, not
> for complex geometry, as this is best done with a mouse)
> -mission schemas that customize the layout/workflow:
> Let's say you want to add housenumbers, so you tap on the house. The editor
> suggests the next road and already predicts the housenumber by what you
> entered to house before).
> Another usecase might be to add 3D featuers, where a wizzard presents you
> different shapes of roofs, color table, ...
> -ability to take georeferenced audio-notes, photos and embedd them
> immediately
>
> Yes a HTML5 might do the job and as Josh noticed, this will simplify the
> deployment for mobile platforms. On the other hand I would really suggest
> offline editing.
>
> But this are just ideas...would be great if anybody would give it a try to
> see if this might work :)
>
> bye
> Matthias
>
> Am 02.12.2011 14:26, schrieb Josh Doe:
>
>> On Fri, Dec 2, 2011 at 8:13 AM, Ian Dees  wrote:
>>>
>>> On Fri, Dec 2, 2011 at 6:56 AM, Jaak Laineste
>>> wrote:


 Hi,
  as you may now, during GSoC QGIS was ported to Android tablets
 (http://hub.qgis.org/projects/quantum-gis/wiki/QGIS_Mobile_GSoC_2011).
 Merkaator has technically same base elements: Qt, GEOS, PROJ.4 etc, so
 based on this experience porting of Merkaator could be also possible
 with much smaller fuss. It should be even smaller work than getting
 JOSM working under Android.

  Question: do you know anyone who really would need and use it?
>>>
>>>
>>>
>>> Jaak,
>>>
>>> Porting JOSM to Android would be an interesting academic task, but
>>> several
>>> problems lead me to believe it would not be useful for general use:
>>>
>>> - java.awt is nowhere to be found on Android. JOSM's rendering engine
>>> would
>>> have to be completely re-written.
>>> - javax.swing is nowhere to be found on Android. JOSM's UI system would
>>> have
>>> to be completely re-written.
>>> - JOSM's UI is based on menus and keyboard shortcuts. These don't have
>>> good
>>> analogs in Android and would have to be re-thought.
>>>
>>> I'd much rather see the time spent solving these problems put towards a
>>> general purpose editor that is specifically designed for a tablet. Maybe
>>> something general enough that an iOS and Android developer could use the
>>> same design with platform-specific tweaks.
>>
>>
>> All valid points, however it may be useful to reuse much of the
>> non-GUI code for handling file loading, validation, search, etc. Or
>> perhaps it may be more worthwhile to write a nice HTML5 app that can
>> work on iOS, Android, and the web.
>>
>> I think the hardest thing is coming up with a design that works well
>> on tablets. I think first and foremost it should offer a simple and
>> robust way to edit POIs. A good preset system is a must (share with
>> Potlatch2 or JOSM, don't create a new one!). Allow ways to be created
>> both by tapping nodes and drawing (with simplification algorithm). And
>> definitely have online and offline modes.
>>
>> I'd encourage you to create a wiki article to try and define what
>> should go in to a tablet app.
>>
>> -Josh

Re: [OSM-dev] speeding up loading an OSM dump into PostGIS?

[Toby Murray]

On Wed, Nov 30, 2011 at 12:25 AM, Ákos Maróy  wrote:
> or, to put it in the other perspective: what hardware would make be
> needed to make this process faster?

I know there has been some work in osm2pgsql to make it multithreaded
to help out a few parts that ARE CPU bound (maybe more during diff
application?) but at the end of the day...

1) solid state drive
2) fast (15K RPM) drives
3) striped RAID

See what an SSD did for yevaud (the osm.org tile rendering server) back in May:
http://munin.openstreetmap.org/openstreetmap/yevaud.openstreetmap/renderd_queue.html

The render queue went from "always full" to "always empty" overnight.

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] OSM-3D discussions

[Toby Murray]

On 11/27/11, Matthias Meißer  wrote:
> You might check this out:
> http://wiki.openstreetmap.org/wiki/3D_Development/Modelling#Modelling_tools

I'm on a slow tethered phone so I don't have links in front of me but
yes, I have seen at least a couple of open 3D modeling languages. Not
very familiar with the tools to use them but it is impossible that
they are harder to use than P2/JOSM with 50 ways, half of which are
overlapping others, to describe a single building.

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] OSM-3D discussions

[Toby Murray]

On 11/27/11, andrzej zaborowski  wrote:
> Finally I'm sure a number of OSMers oppose adding too much detail to
> the OSM database and especially some of the 3d modelling features
> don't seem to fit our data model (which is very 2d + tags).
> Personally I think it's unavoidable that all of these details are
> going to be added at some point, though.

My problem isn't a lot of details, it is that (as you pointed out) the
OSM data model is not the right tool for the job of describing very
complex 3D objects. Trying to mash 3D modeling into OSM results in
impossibly complex data for any but the most advanced users to
maintain. There are existing tools that are "the right tool" for 3D
modeling.

I would rather see more thought put into coming up with a better way
of linking external databases to OSM data and then use an external
database for these 3D models.

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] How to Develop a Plugin for JOSM

[Toby Murray]

On Wed, Oct 26, 2011 at 8:59 AM, Joerg Moldenhauer
 wrote:
> Hi,
>
> I try to develop a little plugin for JOSM by using the developersguide
> (http://josm.openstreetmap.de/wiki/DevelopersGuide/DevelopingPlugins), but
> I am facing some problems.
>
> My first problem concerns setting up the plugin environment. I have
> checked out the plugin environment with Eclipse. The Problem is, that the
> autocorrect of Eclipse doesn't work in the environment. For exempal if i
> change some code in the existing plugins, my errors are not shown. The
> icons of the classes are also a little different then usually (they have a
> hollow J). Did I make some mistake by checking out the environment?
>
> Can somebody please give me a little tutorial how to start developing a
> JOSM Plugin with Ecllipse?

I'm not at home so I don't have the details of my setup in front of me
but I do recall the videos on the developer guide page being a big
help:
http://josm.openstreetmap.de/wiki/DevelopersGuide

I believe I checked out the JOSM source from the JOSM svn repository
and the plugin I wanted to modify came from the OSM repository.

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Osmosis apidb schema - "visible" column and deleted nodes/ways/relations ?

[Toby Murray]

On Sun, Oct 9, 2011 at 9:12 AM, kimaidou  wrote:
> Hi list,
>
> I am trying to set up and osmosis database (apidb schema). Installation and
> data importation are done. I would like to know what exactly does the column
> "visible" represent in the nodes and ways tables ? I thought "visible =
> False" meant "the node/way has been deleted, but I checked, and for example,
> this node (with visible = False) does still exist :
> http://www.openstreetmap.org/browse/node/26694402
>
> What is the best way to find which nodes/ways have been deleted between to
> diff diles imports ?

I believe you are correct. The node in question has never had
visible=false according to its full history:
http://www.openstreetmap.org/api/0.6/node/26694402/history

So I don't know why your database would indicate that it is not visible.

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] OSM XML declaration, JOSM, Osmosis et al.

[Toby Murray]

On Wed, Sep 21, 2011 at 1:44 PM, Paul Norman  wrote:
>> -Original Message-
>> From: Frederik Ramm [mailto:frede...@remote.org]
>>
>> Thinking about this, I like the stripping of the version number.
>>
>> We're having a lot of trouble with people doing mass edits by using XAPI
>> etc. to download e.g. everything with name=McDonalds world-wide and
>> replacing that with name=McDonald's - breaking a lot of correct things
>> in the process. Not serving the version number on XAPI, Overpass and
>> others would send a clear signal that these services are not to be used
>> for editing.
>
> Like NE2's http://www.openstreetmap.org/browse/node/682273592?
>
> In spite of the problems with mass-edits based on XAPI queries, it is still
> essential for some parts of my workflow.
>
> I used my local jxapi instance extensively fixing NHD tagging in California.
> See http://lists.openstreetmap.org/pipermail/talk-us/2011-March/005438.html
> for details.
>
> I have also used it for mapping highways in large areas, adding highway
> relations and occasionally for normal mapping when I didn't want to hit API
> limits.
>
> If jxapi ever stops serving data usable for editing then I will not pull the
> latest code updates for my local copy.

+1

I have my own xapi instance that I use for editing as well. It can
indeed be risky and the type of edits you can perform is limited but
there are some edits that would simply be impossible without the XAPI.
Granted, most of the use cases are for cleaning up imported data which
of course is an entirely different conversation...

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Related terms in the wiki for improved tag search

[Toby Murray]

On Thu, Sep 1, 2011 at 2:21 PM, Stefan Keller  wrote:
> One of the crucial point there is that synonyms and related terms are
> found (term being one ore more words). Example: A search for "church"
> will show "amenity=place_of_worship".
>
> In order to achieve this a controlled word list (a thesaurus) becomes
> necessary. This is a list of terms where some are OpenStreetMap
> specific. This can not be found in a general purpose thesaurus.

This seems very similar to the nominatim "special phrases" page.
Perhaps there can be some reuse or cooperation?

http://wiki.openstreetmap.org/wiki/Nominatim/Special_Phrases

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Custom costs with pgRouting and osm2pgrouting

[Toby Murray]

On Wed, Aug 31, 2011 at 1:04 AM, Josh Doe  wrote:
> I've used osm2pgrouting to import a small area, and have been running
> pgRouting's driving_distance() function simply using length as the
> cost, but now I'd like to compute custom costs that take into account
> things like maxspeed, surface, etc. The simplest example is doing
> least-time rather than least-distance by using length/maxspeed. I
> haven't been able to find any examples that show how to do this, and
> would greatly appreciate working code or pointers to docs I may have
> missed.
> Thanks,
> -Josh

I may be trying to do something similar soon. The local bicyle club
wants to have a routing service that takes the "bikeability" of roads
into account. Some information has already been tagged on all the
roads in the city so now I just need to find a routing engine that
will use the tags. Sorry, don't have any answers yet - I just started
looking at this last night :)

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] OSM on BlackBerry

[Toby Murray]

On Fri, Aug 26, 2011 at 4:23 AM, Christian Anger
 wrote:
> Thanks a lot for your messages! It seems that the best solution for me would
> be to set up an own tile server, download the tiles from OSM and then
> providing them to the users of our app. We have planned to let the app
> display maps offline, so a permanent data service wouldn't be necessary.
>
> @Toby: Styling our own tiles is a great idea. That way, we could provide the
> most appropriate map representation for our users. Is there a documentation
> on tile styling?

There are a couple of ways of going about this.

You could set up your own complete OSM rendering stack. You would
download the OSM database dump available at
http://planet.openstreetmap.org/ and import it into a postgis database
and then use that to render tiles on demand using a customized map
style. Not a walk in the park but doable. One of the better guides
that I am currently aware of is one by Richard Weait:
http://weait.com/content/build-your-own-openstreetmap-server

There is also an Amazon machine image called Tile Drawer that lets you
do your rendering in EC2. They talk about doing a small region. Not
sure if it scales well to world wide coverage.
http://tiledrawer.com/

Lastly, there are services like CloudMade that serve up tiles from
their servers with your style changes (for a fee of course). I think
there is another service besides CloudMade that does this but I don't
remember off the top of my head.

I'm not really an expert at the rendering end of things but that's
what I know from hanging around the OSM community for a while.

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] OSM on BlackBerry

[Toby Murray]

On Wed, Aug 24, 2011 at 3:51 AM, Christian Anger
 wrote:
>
> Our product is a very successful sports assistant app on the BlackBerry
> platform. Since we are currently trying to improve our app's map
> representation and BlackBerry Maps causes some problems, we consider to
> switch to OSM.
>
> However, we couldn't find a lot of supporting material for the BlackBerry
> platform and I therefore would be glad if you could give me some information
> on OSM support for the BlackBerry platform.
> Does there exist a library that can be used for OSM on BlackBerry?
> Where can we find source code samples or general information on how to use
> OSM on BlackBerry?

Unfortunately it seems like most of the BlackBerry apps listed on the
wiki[1] are proprietary so I'm guessing there isn't much in the way of
existing libraries.

Does your app do offline map display or is it assumed that there is
always data service available? Just downloading tiles shouldn't be too
hard to implement although as pointed out, you would need to abide by
our tile usage policy which might mean setting up your own tile
server. The upside of this is that you could style your own tiles to
hilight bicycle/pedestrian features which might let you set yourself
apart from other apps that probably just use default road maps. Plus,
you could include a hint to users that if their favorite bike
path/trail isn't on the map, they can add it themselves! :)

[1] http://wiki.openstreetmap.org/wiki/Blackberry

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Ways

[Toby Murray]

I recently came across a way that doesn't exactly fit your cases but
was still very weird and wrong. I couldn't really even figure out how
to fix it easily in JOSM so I just deleted and retraced it (it was
just a motorway_link):

http://www.openstreetmap.org/browse/way/13194837/history

It looked like a normal way but doubled back on itself. It has node
121671201 listed 3 times and 1290473243 twice.

I guess this probably wouldn't create problems for the cases you are
talking about but it does show that current editors (P2 from May in
this case) do sometimes produce unexpected anomalies.

Toby


On Thu, Aug 11, 2011 at 12:07 PM, ant  wrote:
> I'm currently writing processing software and a couple of questions
> regarding ways have come up. None of the following examples makes practical
> sense, and I believe (hope) that the editors prevent such things from
> happening. But what about ...
>
> ... subsequent duplicate references?
>
> 
>        
>        
>        
>        
> 
>
> ... closed ways trivial enough not to represent an area?
>
> 
>        
>        
>        
> 
>
> 
>        
>        
> 
>
>
> The reason why it bothers me is that I feel my program should deal with
> these cases. So the technical part of my question is: How does the API
> handle it? Which of the above should I expect to be found in the planet
> file, which not?
>
> cheers
> ant
>
> ___
> dev mailing list
> dev@openstreetmap.org
> http://lists.openstreetmap.org/listinfo/dev
>

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Merging Changes with Bounding Boxes

[Toby Murray]

Not 100% sure. The API documentation in the wiki does state that
lat/lon are required parameters for a delete operation. It doesn't say
that they must be the same as they were for the previous version of
the object. So I'm not sure if you can count on the lat/long being
identical.

Regardless of the API, you could certainly get a location change and
then a delete for the same node in one change file. I'm not sure what
happens if a user moves and then deletes a node, all before uploading.
This might depend on the editing software. Some editors might just
drop any changes and only send the delete operation while others could
upload any changes first and then the delete.

Toby


On Fri, Jul 15, 2011 at 10:28 AM, Andrew Ayre  wrote:
> Thanks.
>
> You mentioned that the change file only contains the new version of
> objects. If a node is being deleted is the lat/lon in the change file
> guaranteed to be identical to the lat/lon for that node an up-to-date
> planet file?
>
> Are there any options to getting the old location of a moved node
> without writing my own Osmosis patch (if that's even possible)?
>
> Andy
>
> On 7/15/2011 3:35 PM, Toby Murray wrote:
>> The problems with change files + bounding boxes is that the change
>> file only contains the NEW version of objects. So when you clip it to
>> a bounding box, nodes that started out inside of the bbox and were
>> moved outside will get eliminated from the change and you will keep
>> the old version of the node in your database.
>

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Merging Changes with Bounding Boxes

[Toby Murray]

The problems with change files + bounding boxes is that the change
file only contains the NEW version of objects. So when you clip it to
a bounding box, nodes that started out inside of the bbox and were
moved outside will get eliminated from the change and you will keep
the old version of the node in your database.

Depending on your needs you might be able to get away with this if you
leave a big enough buffer around the data that you really want to keep
up to date. Any data in this buffer zone should not be trusted.

I'm also not sure what happens to changes that don't affect any
geometry. For example if all you do in a changeset is modify a
relation, it kind of breaks the changeset bounding box on osm.org so
I'm not sure what would happen when clipping the change. (
http://trac.openstreetmap.org/ticket/1861 )

Toby

On Fri, Jul 15, 2011 at 6:34 AM, Andrew Ayre  wrote:
> Hi, currently I extract a piece of map data and store it. Later on I
> want to apply a set of changes to it. Here are my commands:
>
> osmosis --read-pbf britishisles.osm.pbf --buffer bufferCapacity=12000
> --bb bottom=51 top=52 left=-1 right=1 --write-pbf london.osm.pbf
>
> osmosis --read-xml-change changes.osc --read-pbf london.osm.pbf
> --apply-change --buffer bufferCapacity=12000 --bb top=52 bottom=51
> left=-1 right=1 --write-pbf londonnew.osm.pbf
>
> The merge command takes 40 seconds to run but I have to do this a lot,
> so I am wondering if it is possible to clip the changes to a bounding
> box before the merge? Would that improve performance? For example
> changes.osc might cover the whole of the planet.
>
> I tried adding a --bb command after --read-xml-change but Osmosis
> complained the input format was the wrong type.
>
> Or is it that applying the change and then the bounding box yields the
> best performance performance?
>
> thanks, Andy
>
> --
> Andy
> PGP Key ID: 0xDC1B5864
>
> ___
> dev mailing list
> dev@openstreetmap.org
> http://lists.openstreetmap.org/listinfo/dev
>

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] [OSM-legal-talk] license change effect on un-tagged nodes

[Toby Murray]

On Wed, Jul 6, 2011 at 6:31 PM, Anthony  wrote:
> Can someone please explain the logic of not recording major changes in the
> database such as shifting an entity?

It follows pretty naturally out of the database schema. Anything that
modifies the ways, way_tags or way_nodes tables creates a new version
of the way. Things that only affect the node tables such as moving the
location of a node or changing tags on the node do not affect any of
the way tables so no new version is created.

The same thing happens with relations and their members. You can add a
maxspeed= tag to a way and it doesn't affect the relation that way is
a part of. That would actually make touching long route relations a
conflict nightmare so I'm pretty glad this isn't the case.

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Is there any Debian package of osmosis available?

[Toby Murray]

On Fri, Jul 1, 2011 at 4:01 PM, John Smith  wrote:
>
> No idea why you have trouble finding the osmosis package:
>
> http://packages.ubuntu.com/natty/osmosis

I saw this but the version in that package is from February 2010.
There have been a few changes since then...

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Is there any Debian package of osmosis available?

[Toby Murray]

Hmm this would be a perfect use for the openstreetmap PPA on launchpad:
https://launchpad.net/~openstreetmap/+archive/ppa

Looks like it is pretty much dead at the moment though.

Toby


On Fri, Jul 1, 2011 at 1:07 PM, Parveen Arora  wrote:
> Hello All,
> I have searched through Synaptic Package Manager in Ubuntu for osmosis
> but found nothing.
> But I need it to include in Tile-Server Automatic setup tool-chain.
> Please let me know If there is any available or how to get that?
>
> Otherwise is there anyway to install it along-with a debian package
> that installs every software and dependency   required to set up Tile
> Server.
> Thank You.
>
> --
> Parveen Arora
> www.parveenarora.in
> E-Mail: o...@parveenarora.in
>
> ___
> dev mailing list
> dev@openstreetmap.org
> http://lists.openstreetmap.org/listinfo/dev
>

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] osm2pgsql for 64-bit IDs

[Toby Murray]

On Sun, Jun 5, 2011 at 6:38 PM, Scott Crosby  wrote:
> Out of curiosity, I did a test on this, because I was planning on
> exploiting this 'conventional wisdom'. I was hoping that 99% of nodes
> occur in one way, but on a 2010-08 planet, 85% of nodes occur in one
> way, 12% in 2, and 1% in more than two..

Now for extra credit, how many of the single way nodes are from
ridiculously over-noded imports like some of the TIGER and CanVec
data? :)

I would not be surprised if that number were as high as 20%

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] XAPI and other solutions

[Toby Murray]

On Sun, Apr 24, 2011 at 5:43 AM, Pierre-Alain Dorange  wrote:
> Yes, but they are huge...
> My original place=* for france is 50 MB (not compressed).
>
> France extract if 2 GB compressed, i don't think my python script was
> able to handle such a huge data.

That's why you use osmosis to trim it down to what you want.

The following osmosis command gives me a 34 MB XML file when run on
the France PBF file from geofabrik.

osmosis --read-pbf france.osm.pbf \
 --tf accept-ways "place=*" \
 --tf accept-nodes "place=*"
 --tf accept-relations "place=*"
 --write-xml france_place.osm

Toby

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] XAPI and other solutions

[Toby Murray]

You can set up your own jXAPI server and disable the bounding box
limit. But this takes about 500 GB of drive space and several days of
processing.

Another option is to use osmosis to filter objects out the planet file:

http://wiki.openstreetmap.org/wiki/Osmosis/Detailed_Usage_0.39#--tag-filter_.28--tf.29

Toby


On Sat, Apr 23, 2011 at 2:57 PM, Pierre-Alain Dorange  wrote:
> Hi, i'm a french contributor and i look for advice for a project.
>
> I'm processing free french city official data (from insee, official stat
> agency) and i have to match those city with osm city (place=*) to
> geolocalize it on a map.
> The project is to present cctv surveillance in the country (see demo at
> [1]).
>
> To get those data and preprocess them i used XAPI to retrive all nodes
> with tag place=* :
>
> The result is fine but XAPI servers are randomly running and data seems
> old. New Java XAPI servers do not accept the bounding box : too large.
>
> What are the best way to retrieve OSM raw data (xml format would be
> preferable) to update regulary the project's data ?
>
> Thanks.
>
> [1] project demo
> http://www.leretourdelautruche.com/map/cctv/cctv_map.php
>
> These map show city global data (user contributions) and individual cctv
> camera localisation (from OSM man_made=surveillance) when you zoom to a
> city with cctv mapped (ie. Strasbourg) :
>  .5831&lon=7.75238&layers=B0T>
>
> --
> Pierre-Alain Dorange
> OSM experiences : 
>
>
> ___
> dev mailing list
> dev@openstreetmap.org
> http://lists.openstreetmap.org/listinfo/dev
>

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev