On Fri, May 6, 2011 at 2:48 PM, James Ewen wrote:
> Is there a way to know how much load is being exerted by each type of
> application out there?
The sysadmins know, but I expect that the logs can't be exposed due to
privacy law.
> We have a newer application that uses OSM maps. The author is aware of
> the usage policy, and is trying to be gentle on the servers, but as
> the number of people running the application grows, the demand on the
> server grows.
>
> It would be good to be able to see which applications are putting
> unreasonable burdens on the server.
We've been reaching out to the worst offenders and getting them to
correct their behaviour. Sadly, apps are popping up faster than we
can sort them out. All users are now rate limited, but the apps make
demands that are so out of perspective that even rate limits don't
eliminate the harm they do.
Here's an example. Of our max-zoom tiles z18, only a few percent are
ever viewed. Ever. By anybody. So we don't render them except on
demand. That's expensive to do in terms of server resources, but
mappers love it. Mappers can see their mapping results right away and
that means we get better data, faster.
Bulk downloaders take 100% of the tiles in their area of interest. We
know only a few percent will actually be viewed. But they ask us to
render them all. That takes tiles away from mappers. More
specifically, that's why Bernie is still waiting for his tiles today.
Have a look at the render queue.
http://munin.openstreetmap.org/openstreetmap/yevaud.openstreetmap/renderd_queue.html
It's been running flat out for weeks. When we added resources last
year, it helped for a bit, but more apps keep popping up and taking
our resources. We cannot win that race.
> The program by default downloads and caches tiles locally, but purges
> the tiles after a day or two. We can over ride the purge, which I do,
> keeping tiles for ever. The program can query for tile updates, which
> means changes get propagated even when the tiles are stored for ever.
Be sure to join #osm-dev on irc.oftc.net and ask for help from the sys
admins. They'll get you sorted out.
Any application that bulk downloads tiles from the OSM servers is
currently at risk of being cut off without further warning. The
answer is longer and more complicated than this, but this is what it
boils down to.
___
Talk-ca mailing list
Talk-ca@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk-ca
