Re: Using template fragment caching *inside* a sitewide cache: possible?

[Matt Andrews]

I tried using the approach above (template fragment cache everywhere 
instead of sitewide) and the same problem was there: the inner cache didn't 
reflect cachebusting changes.

I ended up rewriting the code to pull the view fragments via AJAX (which 
did respect the cachebusting). You can see the results 
here: http://www.scenepointblank.com/ (it's the "popular right now" tabbed 
widget near the bottom right.

On Thursday, 14 February 2013 17:19:28 UTC, Matt Andrews wrote:
>
> Hi Tom,
>
> Yep, you've got the problem right. Reading it back that way I see the 
> issue much more clearly...
>
> One approach I thought of: what if, instead of the sitewide caching, I 
> used template-fragment caching for *every* view (there aren't that many) 
> and invalidated the relevant ones along with the nested caches when needed 
> (there are probably only five or so views where this would need to happen)?
>
> I'm on shared hosting so I don't think Varnish is going to be a 
> possibility...
>
> thanks!
>
> On Thursday, 14 February 2013 16:02:53 UTC, Tom Evans wrote:
>>
>> On Mon, Feb 11, 2013 at 1:42 PM, Matt Andrews  
>> wrote: 
>> > Hi all, 
>> > 
>> > I've been experimenting with an expensive query I need to call 
>> (essentially 
>> > grabbing data from some Google APIs). I tried this experiment: 
>> > 
>> > A sitewide cache with a long (days) expiry time 
>> > A template fragment with its own separate cache inside a view cached by 
>> the 
>> > sitewide cache -- this fragment simply displays the current time. 
>> > A view function which clears the named template fragment cache. 
>> > 
>> > The behaviour I expected was that on first pageload, the fragment would 
>> > display the time of page render, and all subsequent reloads would 
>> display 
>> > the same time. This was true. 
>> > 
>> > The second part, though, was that I expected to be able to call the 
>> view 
>> > function to clear the fragment cache and then reload the page to see 
>> the 
>> > time update. This didn't happen. 
>> > 
>> > Is it possible to achieve this behaviour? Essentially I want to run a 
>> > background cron task which just hits the Google API and updates my 
>> cached 
>> > fragments - the Google data changes every 15 minutes but the sitewide 
>> cache 
>> > has several hours timeout, normally. 
>> > 
>> > Hope this makes sense. 
>> > 
>> > Matt 
>> > 
>>
>> Hi Matt 
>>
>> Can I restate the problem to see if I am getting it right? 
>>
>> You have a page fragment, 'FRAG', stored in the cache. 
>> You have a page, 'PAGE', using that fragment stored in the cache. 
>> You update 'FRAG' in the cache. 
>> You reload 'PAGE', and it has the old contents of 'FRAG' instead of 
>> the updated one. 
>>
>> If so, this is to be expected. The site cache caches entire responses, 
>> with a key derived from that request. When a new request comes in, a 
>> key is generated from that request. If the key is found in the cache, 
>> the cached response is returned. 
>>
>> At no point does the cache know that one cache entry is built using 
>> another, or that invalidating the fragment should invalidate any page 
>> containing that fragment. 
>>
>> One potential solution is to use something like Varnish, an HTTP 
>> accelerator/cache that has support for ESI - Edge Side Includes - that 
>> allow you to build up responses from multiple components, and 
>> intelligently cache the components. Eg, an example of what you are 
>> trying to do in Varnish using ESI: 
>>
>> https://www.varnish-cache.org/trac/wiki/ESIfeatures#Anesi:includeexample 
>>
>> Alternatively, I may have completely misunderstood this :) 
>>
>> Cheers 
>>
>> Tom 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Using template fragment caching *inside* a sitewide cache: possible?

[Matt Andrews]

Hi Tom,

Yep, you've got the problem right. Reading it back that way I see the issue 
much more clearly...

One approach I thought of: what if, instead of the sitewide caching, I used 
template-fragment caching for *every* view (there aren't that many) and 
invalidated the relevant ones along with the nested caches when needed 
(there are probably only five or so views where this would need to happen)?

I'm on shared hosting so I don't think Varnish is going to be a 
possibility...

thanks!

On Thursday, 14 February 2013 16:02:53 UTC, Tom Evans wrote:
>
> On Mon, Feb 11, 2013 at 1:42 PM, Matt Andrews 
>  
> wrote: 
> > Hi all, 
> > 
> > I've been experimenting with an expensive query I need to call 
> (essentially 
> > grabbing data from some Google APIs). I tried this experiment: 
> > 
> > A sitewide cache with a long (days) expiry time 
> > A template fragment with its own separate cache inside a view cached by 
> the 
> > sitewide cache -- this fragment simply displays the current time. 
> > A view function which clears the named template fragment cache. 
> > 
> > The behaviour I expected was that on first pageload, the fragment would 
> > display the time of page render, and all subsequent reloads would 
> display 
> > the same time. This was true. 
> > 
> > The second part, though, was that I expected to be able to call the view 
> > function to clear the fragment cache and then reload the page to see the 
> > time update. This didn't happen. 
> > 
> > Is it possible to achieve this behaviour? Essentially I want to run a 
> > background cron task which just hits the Google API and updates my 
> cached 
> > fragments - the Google data changes every 15 minutes but the sitewide 
> cache 
> > has several hours timeout, normally. 
> > 
> > Hope this makes sense. 
> > 
> > Matt 
> > 
>
> Hi Matt 
>
> Can I restate the problem to see if I am getting it right? 
>
> You have a page fragment, 'FRAG', stored in the cache. 
> You have a page, 'PAGE', using that fragment stored in the cache. 
> You update 'FRAG' in the cache. 
> You reload 'PAGE', and it has the old contents of 'FRAG' instead of 
> the updated one. 
>
> If so, this is to be expected. The site cache caches entire responses, 
> with a key derived from that request. When a new request comes in, a 
> key is generated from that request. If the key is found in the cache, 
> the cached response is returned. 
>
> At no point does the cache know that one cache entry is built using 
> another, or that invalidating the fragment should invalidate any page 
> containing that fragment. 
>
> One potential solution is to use something like Varnish, an HTTP 
> accelerator/cache that has support for ESI - Edge Side Includes - that 
> allow you to build up responses from multiple components, and 
> intelligently cache the components. Eg, an example of what you are 
> trying to do in Varnish using ESI: 
>
> https://www.varnish-cache.org/trac/wiki/ESIfeatures#Anesi:includeexample 
>
> Alternatively, I may have completely misunderstood this :) 
>
> Cheers 
>
> Tom 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Using template fragment caching *inside* a sitewide cache: possible?

[Tom Evans]

On Mon, Feb 11, 2013 at 1:42 PM, Matt Andrews  wrote:
> Hi all,
>
> I've been experimenting with an expensive query I need to call (essentially
> grabbing data from some Google APIs). I tried this experiment:
>
> A sitewide cache with a long (days) expiry time
> A template fragment with its own separate cache inside a view cached by the
> sitewide cache -- this fragment simply displays the current time.
> A view function which clears the named template fragment cache.
>
> The behaviour I expected was that on first pageload, the fragment would
> display the time of page render, and all subsequent reloads would display
> the same time. This was true.
>
> The second part, though, was that I expected to be able to call the view
> function to clear the fragment cache and then reload the page to see the
> time update. This didn't happen.
>
> Is it possible to achieve this behaviour? Essentially I want to run a
> background cron task which just hits the Google API and updates my cached
> fragments - the Google data changes every 15 minutes but the sitewide cache
> has several hours timeout, normally.
>
> Hope this makes sense.
>
> Matt
>

Hi Matt

Can I restate the problem to see if I am getting it right?

You have a page fragment, 'FRAG', stored in the cache.
You have a page, 'PAGE', using that fragment stored in the cache.
You update 'FRAG' in the cache.
You reload 'PAGE', and it has the old contents of 'FRAG' instead of
the updated one.

If so, this is to be expected. The site cache caches entire responses,
with a key derived from that request. When a new request comes in, a
key is generated from that request. If the key is found in the cache,
the cached response is returned.

At no point does the cache know that one cache entry is built using
another, or that invalidating the fragment should invalidate any page
containing that fragment.

One potential solution is to use something like Varnish, an HTTP
accelerator/cache that has support for ESI - Edge Side Includes - that
allow you to build up responses from multiple components, and
intelligently cache the components. Eg, an example of what you are
trying to do in Varnish using ESI:

https://www.varnish-cache.org/trac/wiki/ESIfeatures#Anesi:includeexample

Alternatively, I may have completely misunderstood this :)

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Using template fragment caching *inside* a sitewide cache: possible?

[Matt Andrews]

Apologies for bumping this thread, but is there anybody with any insight on 
this? Really driving me crazy!

On Monday, 11 February 2013 13:42:05 UTC, Matt Andrews wrote:
>
> Hi all,
>
> I've been experimenting with an expensive query I need to call 
> (essentially grabbing data from some Google APIs). I tried this experiment:
>
>- A sitewide cache with a long (days) expiry time
>- A template fragment with its own separate cache *inside* a view 
>cached by the sitewide cache -- this fragment simply displays the current 
>time.
>- A view function which clears the named template fragment cache.
>
> The behaviour I expected was that on first pageload, the fragment would 
> display the time of page render, and all subsequent reloads would display 
> the same time. This was true.
>
> The second part, though, was that I expected to be able to call the view 
> function to clear the fragment cache and then reload the page to see the 
> time update. This didn't happen.
>
> Is it possible to achieve this behaviour? Essentially I want to run a 
> background cron task which just hits the Google API and updates my cached 
> fragments - the Google data changes every 15 minutes but the sitewide cache 
> has several hours timeout, normally.
>
> Hope this makes sense.
>
> Matt
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Using template fragment caching *inside* a sitewide cache: possible?

[Matt Andrews]

Hi all,

I've been experimenting with an expensive query I need to call (essentially 
grabbing data from some Google APIs). I tried this experiment:

   - A sitewide cache with a long (days) expiry time
   - A template fragment with its own separate cache *inside* a view cached 
   by the sitewide cache -- this fragment simply displays the current time.
   - A view function which clears the named template fragment cache.

The behaviour I expected was that on first pageload, the fragment would 
display the time of page render, and all subsequent reloads would display 
the same time. This was true.

The second part, though, was that I expected to be able to call the view 
function to clear the fragment cache and then reload the page to see the 
time update. This didn't happen.

Is it possible to achieve this behaviour? Essentially I want to run a 
background cron task which just hits the Google API and updates my cached 
fragments - the Google data changes every 15 minutes but the sitewide cache 
has several hours timeout, normally.

Hope this makes sense.

Matt

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.