Re: [OSM-talk] Some tiles not rendering?

[Svavar Kjarrval]

Enhancing the queue is not meant neccesserily to make it longer. 
Although I should've referred to the queueing process to be more clear.


As Andrew pointed out, a really long queue can cause latency issues in 
regards to rendering so that's not such a good idea. There are at least 
two ways to counteract the problem:


1. Instead of rejecting rendering requests due to a full queue, 
implement a secondary queue which will feed the primary queue when it's 
not too busy. If someone requests a re-render of a tile in the secondary 
queue (and the primary queue is not full), the tile request is deleted 
from the secondary queue.

2. Enhance the rendering process so the queue is processed quicker.

With regards,
Svavar Kjarrval

On 04/14/2011 03:37 PM, Ian Dees wrote:

How would you enhance the queue? Make it longer? How much longer?

On Thu, Apr 14, 2011 at 10:29 AM, Svavar Kjarrval > wrote:


Why is the queue not enhanced to avoid instances where
tile-rendering queries are rejected due to a full queue?

With regards,
Svavar Kjarrval

On 04/14/2011 03:15 PM, Peter Wendorff wrote:

Hi Nakor.
As far as I know the queue is the whole knowledge about tiles
which have to be rerendered.
If a tile has to be rerendered due to a changeset, that tile
is submitted to the queue exactly once, at the time the queue
management reads that changeset.
If the queue is full at that time, it's in fact not added -
and not added later, too.

An empty queue at night does not help if there is no call to
add your tile to the queue.

regards
Peter

Am 14.04.2011 16:52, schrieb Nakor:



Also from the graph it looks like the queue when
almost empty during the past 24 hours so why would
the tile not be rendered in that case?


Really?  The queue has been full 18 hours a day or
more for the past two weeks.

Bob


Sorry I was not clear "The queue went almost empty (for
some time) during the past 24 hours" would have been more
correct.

That still leaves 6 hours where the queue is close to
empty as seen on the graph for today between 0:00 and 6:00
approximately. If there are tiles waiting to be rendered
why do not they get processed during that time?



___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Some tiles not rendering?

[Andrew Guertin]

On 04/14/2011 11:29 AM, Svavar Kjarrval wrote:
> Why is the queue not enhanced to avoid instances where tile-rendering
> queries are rejected due to a full queue?

I have no knowledge about the specifics of this server, but I can speak
generically:

Requests are coming in faster than the server can process them. This
means that some requests have to be dropped, no matter what*. Once you
know that, the length of the queue doesn't matter much.

The advantage of a shorter queue is that it reduces latency. Since each
request has to wait for everything in front of it to be processed, the
fewer things in front of the one you care about, the faster it'll happen.

The advantage of a longer queue is that it can smooth out any variations
in request speed. If sometimes requests come in faster than the server
can process, and sometimes they come in slower, then requests can build
up in the fast times and the server can catch up when they slow. This is
apparently what happens to OSM between 00:00 and 06:00 on the graph.

By lengthening the queue, it would be possible to make the server be
constantly busy during that period as well. However, it would come at
the expense of increased latency, and depending on how many requests are
being dropped, filling in that period of lesser activity might not make
much of a difference.


* Technically, you could let the queue will grow indefinitely and time
to process any given request will approach infinity. That's not a good
idea though.

___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Some tiles not rendering?

[Lennard]

Nakor wrote:
> Thanks for the explanations. So that means that the particular tiles
> that got rejected because the queue was full could stay "due to be
> rendered" forever supposing there are no more changes made to the data
> they conatin?

Not forever, but until such a time that somebody requests the tile again
(by looking at it), thereby causing it to be added to the queue again. If
the queue isn't full.

A change in data they contain will not cause a tile to be added to the
rendering queue. Only tiles@home proactively renders every tile that got
changed.

Tiles that got dropped from the queue are forgotten. They are not rendered
later on, when the queue isn't full.

-- 
Lennard


___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Some tiles not rendering?

[Joseph Reeves]

You can, I believe, right click on a tile and do "view image" (in
Firefox at least, it may be different in whatever browser you use).
This brings up a URL such as:

http://tile.openstreetmap.org/14/8149/5492.png

add /dirty to the end:

http://tile.openstreetmap.org/14/8149/5492.png/dirty

Request that and you get "Tile submitted for rendering". Presumably
this needs an empty queue, however.

Cheers, Joseph




On 14 April 2011 16:34, Nakor  wrote:
> On 4/14/2011 11:15 AM, Peter Wendorff wrote:
>>
>> Hi Nakor.
>> As far as I know the queue is the whole knowledge about tiles which have
>> to be rerendered.
>> If a tile has to be rerendered due to a changeset, that tile is submitted
>> to the queue exactly once, at the time the queue management reads that
>> changeset.
>> If the queue is full at that time, it's in fact not added - and not added
>> later, too.
>>
>> An empty queue at night does not help if there is no call to add your tile
>> to the queue.
>>
>> regards
>> Peter
>>
>
> Peter,
>
> Thanks for the explanations. So that means that the particular tiles that
> got rejected because the queue was full could stay "due to be rendered"
> forever supposing there are no more changes made to the data they conatin?
>
> Thanks.
>
> ___
> talk mailing list
> talk@openstreetmap.org
> http://lists.openstreetmap.org/listinfo/talk
>

___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Some tiles not rendering?

[Ian Dees]

How would you enhance the queue? Make it longer? How much longer?

On Thu, Apr 14, 2011 at 10:29 AM, Svavar Kjarrval wrote:

> Why is the queue not enhanced to avoid instances where tile-rendering
> queries are rejected due to a full queue?
>
> With regards,
> Svavar Kjarrval
>
> On 04/14/2011 03:15 PM, Peter Wendorff wrote:
>
>> Hi Nakor.
>> As far as I know the queue is the whole knowledge about tiles which have
>> to be rerendered.
>> If a tile has to be rerendered due to a changeset, that tile is submitted
>> to the queue exactly once, at the time the queue management reads that
>> changeset.
>> If the queue is full at that time, it's in fact not added - and not added
>> later, too.
>>
>> An empty queue at night does not help if there is no call to add your tile
>> to the queue.
>>
>> regards
>> Peter
>>
>> Am 14.04.2011 16:52, schrieb Nakor:
>>
>>>
>>>
  Also from the graph it looks like the queue when almost empty during
> the past 24 hours so why would the tile not be rendered in that case?
>

 Really?  The queue has been full 18 hours a day or more for the past two
 weeks.

 Bob

>>>
>>> Sorry I was not clear "The queue went almost empty (for some time) during
>>> the past 24 hours" would have been more correct.
>>>
>>> That still leaves 6 hours where the queue is close to empty as seen on
>>> the graph for today between 0:00 and 6:00 approximately. If there are tiles
>>> waiting to be rendered why do not they get processed during that time?
>>>
>>>
___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Some tiles not rendering?

[Nakor]

On 4/14/2011 11:15 AM, Peter Wendorff wrote:

Hi Nakor.
As far as I know the queue is the whole knowledge about tiles which 
have to be rerendered.
If a tile has to be rerendered due to a changeset, that tile is 
submitted to the queue exactly once, at the time the queue management 
reads that changeset.
If the queue is full at that time, it's in fact not added - and not 
added later, too.


An empty queue at night does not help if there is no call to add your 
tile to the queue.


regards
Peter



Peter,

Thanks for the explanations. So that means that the particular tiles 
that got rejected because the queue was full could stay "due to be 
rendered" forever supposing there are no more changes made to the data 
they conatin?


Thanks.

___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Some tiles not rendering?

[Svavar Kjarrval]

Why is the queue not enhanced to avoid instances where tile-rendering 
queries are rejected due to a full queue?


With regards,
Svavar Kjarrval

On 04/14/2011 03:15 PM, Peter Wendorff wrote:

Hi Nakor.
As far as I know the queue is the whole knowledge about tiles which 
have to be rerendered.
If a tile has to be rerendered due to a changeset, that tile is 
submitted to the queue exactly once, at the time the queue management 
reads that changeset.
If the queue is full at that time, it's in fact not added - and not 
added later, too.


An empty queue at night does not help if there is no call to add your 
tile to the queue.


regards
Peter

Am 14.04.2011 16:52, schrieb Nakor:




Also from the graph it looks like the queue when almost empty 
during the past 24 hours so why would the tile not be rendered in 
that case?


Really?  The queue has been full 18 hours a day or more for the past 
two weeks.


Bob


Sorry I was not clear "The queue went almost empty (for some time) 
during the past 24 hours" would have been more correct.


That still leaves 6 hours where the queue is close to empty as seen 
on the graph for today between 0:00 and 6:00 approximately. If there 
are tiles waiting to be rendered why do not they get processed during 
that time?


___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk




___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk


___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Some tiles not rendering?

[Peter Wendorff]

Hi Nakor.
As far as I know the queue is the whole knowledge about tiles which have 
to be rerendered.
If a tile has to be rerendered due to a changeset, that tile is 
submitted to the queue exactly once, at the time the queue management 
reads that changeset.
If the queue is full at that time, it's in fact not added - and not 
added later, too.


An empty queue at night does not help if there is no call to add your 
tile to the queue.


regards
Peter

Am 14.04.2011 16:52, schrieb Nakor:




Also from the graph it looks like the queue when almost empty during 
the past 24 hours so why would the tile not be rendered in that case?


Really?  The queue has been full 18 hours a day or more for the past 
two weeks.


Bob


Sorry I was not clear "The queue went almost empty (for some time) 
during the past 24 hours" would have been more correct.


That still leaves 6 hours where the queue is close to empty as seen on 
the graph for today between 0:00 and 6:00 approximately. If there are 
tiles waiting to be rendered why do not they get processed during that 
time?


___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk




___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Some tiles not rendering?

[Nakor]

Also from the graph it looks like the queue when almost empty during 
the past 24 hours so why would the tile not be rendered in that case?


Really?  The queue has been full 18 hours a day or more for the past 
two weeks.


Bob


Sorry I was not clear "The queue went almost empty (for some time) 
during the past 24 hours" would have been more correct.


That still leaves 6 hours where the queue is close to empty as seen on 
the graph for today between 0:00 and 6:00 approximately. If there are 
tiles waiting to be rendered why do not they get processed during that time?


___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Some tiles not rendering?

[Thomas Davie]

On 14 Apr 2011, at 14:44, Nakor wrote:

> On 4/14/2011 9:40 AM, Vladimir Vyskocil wrote:
>> Hi,
>> 
>> You can see here : 
>> http://munin.openstreetmap.org/openstreetmap/yevaud.openstreetmap/index.html#renderd
>>  that the render queue is filed, and if I understand it well new render 
>> requests are rejected until the render queue decrease...
>> 
>> Vlad.
>> 
>> On 14 avr. 2011, at 15:16, Nakor wrote:
>> 
> 
> So why would some tiles render and some other not? If it is full all requests 
> should be rejected right?

Suppose the queue has 998 tiles in it.  Two will get added, the next few will 
be rejected... Suppose now that the renderer finishes another meta-tile, 
dropping the queue down again, and allowing some more to be added to the end.  
The result is seemingly random tiles will  get rendered and others won't.

> Also from the graph it looks like the queue when almost empty during the past 
> 24 hours so why would the tile not be rendered in that case?

Really?  The queue has been full 18 hours a day or more for the past two weeks.

Bob___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Some tiles not rendering?

[Nakor]

On 4/14/2011 9:40 AM, Vladimir Vyskocil wrote:

Hi,

You can see here : 
http://munin.openstreetmap.org/openstreetmap/yevaud.openstreetmap/index.html#renderd
 that the render queue is filed, and if I understand it well new render 
requests are rejected until the render queue decrease...

Vlad.

On 14 avr. 2011, at 15:16, Nakor wrote:



So why would some tiles render and some other not? If it is full all 
requests should be rejected right?


Also from the graph it looks like the queue when almost empty during the 
past 24 hours so why would the tile not be rendered in that case?


___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Some tiles not rendering?

[Vladimir Vyskocil]

Hi,

You can see here : 
http://munin.openstreetmap.org/openstreetmap/yevaud.openstreetmap/index.html#renderd
 that the render queue is filed, and if I understand it well new render 
requests are rejected until the render queue decrease...

Vlad.

On 14 avr. 2011, at 15:16, Nakor wrote:

>  Hello,
> 
> It looks like some tiles are not rendering. For instance 
> http://b.tile.openstreetmap.org/17/35226/48373.png/status has been "due to be 
> rendered" for the last 24 hours when 
> http://b.tile.openstreetmap.org/15/8816/12104.png/status has been updated 
> today.
> 
> Why would some tile render and some not?
> 
>  Thanks,
> 
> N.
> 
> ___
> talk mailing list
> talk@openstreetmap.org
> http://lists.openstreetmap.org/listinfo/talk


___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk

[OSM-talk] Some tiles not rendering?

[Nakor]

  Hello,

It looks like some tiles are not rendering. For instance 
http://b.tile.openstreetmap.org/17/35226/48373.png/status has been "due 
to be rendered" for the last 24 hours when 
http://b.tile.openstreetmap.org/15/8816/12104.png/status has been 
updated today.


Why would some tile render and some not?

  Thanks,

N.

___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/listinfo/talk