subject:"\[symfony\-users\] Auto\-filling caches"

Re: [symfony-users] Auto-filling caches

2011-02-11 Thread Robert Gründler

thanks for the tip. This is what i've come up with so far:

// in the template the sf_cache_key is set containing the current page

include_component('article','articles',array('page' = $page,
'sf_cache_key' = 'articles_' . $page))

// in the task the cached version of the first 10 pages should be
pre-filled:

$manager = sfContext::getInstance()-getViewCacheManager();
$browser = new sfBrowser();
$browser-setHttpHeader(X-Requested-With,XMLHttpRequest);

for ($page = 1; $page 11; $page++) {
$content =
$browser-get('/articles/page/'.$page)-getResponse()-getContent();
$manager-setPartialCache('article', '_articles', 'articles_' .
$page, $content);

}

The task runs fine, and i'm able to get the correct page using
sfBrowser-get(), but when i hit
one of the first 10 pages after running the task, i'm still getting an
uncached response.

From what i understand, internally the sfPartialView does the same call
in the render() method:

$retval = $this-viewCache-setPartialCache($this-moduleName,
$this-actionName, $this-cacheKey, $retval);

I've also set the correct application/environment for the task, so this
shouldn't be the problem.

Any hint what im missing here?

thanks!

-robert

On 2/10/11 11:14 PM, Benoit Montuelle wrote:

Hi,

I would make it using a task which uses sfBrowser on the ajax urls,
eventually using DB queries to generate them. If you use html cache it
should do the job. The only thing is to run the task with the good env
argument so it generates cache for 'prod'.

Now if you use sfViewCacheManager, it seems you can do it more
naturally when a new post is created, with setPageCache
goog_277624711http://trac.symfony-project.org/browser/branches/1.4/lib/view/sfViewCacheManager.class.php#937method.

Regards
Benoit

2011/2/10 Robert Gründler r.gruend...@gmail.com
mailto:r.gruend...@gmail.com

Hi,

Imagine a blog where each BlogEntry has one Author, and can be
tagged with one ore more Tags.
The mainpage displays a paginated list of all blogentries. Every
entry displays the title, author and the tags. The pagination
is implemented using AJAX.

To display the paginated list using symfony + doctrine, 3 Database
queries are needed:

1. select count(*) # get the total number of entries
2. select distinct ... limit ... offset # get the actual ids of
blog entries for that page
3. select ... where in (...) # retrieve the data to display for
that page

When the number of blogs and the authors + tags is getting large,
these 3 queries can take up to 800 - 1000 ms, which is
not fast enough, if you want ajax pagination.

If a single page is cached, the ajax request takes about 200 - 300
ms, but the uncached page takes over 1 second to load.

What we're thinking about is to pre-fill the cache of every page
in that list, so that no user ever hits an uncached page. This
pre-fillling
could take place in a cronjob, which is run every 5 minutes or so
- frequent enough for this use case.

What i was thinking about is to set the cache-lifetime of the
partial for a single page to one day, and update the cached results
in the background - which actually is the part i'm not sure how to
implement.

Anyone knows if this kind of logic could be implemented using
symfony + doctrine?

thanks!

-robert

--
If you want to report a vulnerability issue on symfony, please

send it to security at symfony-project.com
http://symfony-project.com

You received this message because you are subscribed to the Google
Groups symfony users group.
To post to this group, send email to
symfony-users@googlegroups.com mailto:symfony-users@googlegroups.com
To unsubscribe from this group, send email to
symfony-users+unsubscr...@googlegroups.com
mailto:symfony-users%2bunsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/symfony-users?hl=en

--
If you want to report a vulnerability issue on symfony, please send it
to security at symfony-project.com

You received this message because you are subscribed to the Google
Groups symfony users group.
To post to this group, send email to symfony-users@googlegroups.com
To unsubscribe from this group, send email to
symfony-users+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/symfony-users?hl=en

--
If you want to report a vulnerability issue on symfony, please send it to
security at symfony-project.com

Re: [symfony-users] Auto-filling caches

2011-02-11 Thread Robert Gründler

should have rtfm:

factories.yml:

view_cache_manager:
param:
cache_key_use_host_name: false

just in case if anyone runs into the same problem...

On 2/11/11 9:49 AM, Robert Gründler wrote:

thanks for the tip. This is what i've come up with so far:

// in the template the sf_cache_key is set containing the current page

include_component('article','articles',array('page' = $page,
'sf_cache_key' = 'articles_' . $page))

// in the task the cached version of the first 10 pages should be
pre-filled:

$manager = sfContext::getInstance()-getViewCacheManager();
$browser = new sfBrowser();
$browser-setHttpHeader(X-Requested-With,XMLHttpRequest);

for ($page = 1; $page 11; $page++) {
$content =
$browser-get('/articles/page/'.$page)-getResponse()-getContent();
$manager-setPartialCache('article', '_articles', 'articles_' .
$page, $content);

}

The task runs fine, and i'm able to get the correct page using
sfBrowser-get(), but when i hit
one of the first 10 pages after running the task, i'm still getting an
uncached response.

From what i understand, internally the sfPartialView does the same
call in the render() method:

$retval = $this-viewCache-setPartialCache($this-moduleName,
$this-actionName, $this-cacheKey, $retval);

I've also set the correct application/environment for the task, so
this shouldn't be the problem.

Any hint what im missing here?

thanks!

-robert

On 2/10/11 11:14 PM, Benoit Montuelle wrote:

Hi,

I would make it using a task which uses sfBrowser on the ajax urls,
eventually using DB queries to generate them. If you use html cache
it should do the job. The only thing is to run the task with the good
env argument so it generates cache for 'prod'.

Regards
Benoit

2011/2/10 Robert Gründler r.gruend...@gmail.com
mailto:r.gruend...@gmail.com

Hi,

To display the paginated list using symfony + doctrine, 3
Database queries are needed:

When the number of blogs and the authors + tags is getting large,
these 3 queries can take up to 800 - 1000 ms, which is
not fast enough, if you want ajax pagination.

If a single page is cached, the ajax request takes about 200 -
300 ms, but the uncached page takes over 1 second to load.

What we're thinking about is to pre-fill the cache of every
page in that list, so that no user ever hits an uncached page.
This pre-fillling
could take place in a cronjob, which is run every 5 minutes or so
- frequent enough for this use case.

Anyone knows if this kind of logic could be implemented using
symfony + doctrine?

thanks!

-robert

--
If you want to report a vulnerability issue on symfony, please

send it to security at symfony-project.com
http://symfony-project.com

You received this message because you are subscribed to the Google
Groups symfony users group.
To post to this group, send email to
symfony-users@googlegroups.com
mailto:symfony-users@googlegroups.com
To unsubscribe from this group, send email to
symfony-users+unsubscr...@googlegroups.com
mailto:symfony-users%2bunsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/symfony-users?hl=en

--
If you want to report a vulnerability issue on symfony, please send
it to security at symfony-project.com

--
If you want to report a vulnerability issue on symfony, please send it to
security at symfony-project.com

You received this message because you are subscribed to the Google
Groups symfony users group.
To post to this group, send email to symfony-users@googlegroups.com
To unsubscribe from this

Re: [symfony-users] Auto-filling caches

2011-02-11 Thread Gareth McCumskey

Why not just chain ajax requests? Have two divs, one visible one invisible.
Load what the person wants to see with one ajax request. When that completes
it calls another ajax request which loads the next page into the invisible
div. When the person goes to the next page, the act of making that page
visible starts loading the next page into the hidden div.

2011/2/10 Robert Gründler r.gruend...@gmail.com

 Hi,

 Imagine a blog where each BlogEntry has one Author, and can be tagged with
 one ore more Tags.
 The mainpage displays a paginated list of all blogentries. Every entry
 displays the title, author and the tags. The pagination
 is implemented using AJAX.

 To display the paginated list using symfony + doctrine, 3 Database queries
 are needed:

 1. select count(*)  # get the total number of entries
 2. select distinct ... limit ... offset # get the actual ids of blog
 entries for that page
 3. select ... where in (...)  # retrieve the data to display for that page

 When the number of blogs and the authors + tags is getting large, these 3
 queries can take up to 800 - 1000 ms, which is
 not fast enough, if you want ajax pagination.

 If a single page is cached, the ajax request takes about 200 - 300 ms, but
 the uncached page takes over 1 second to load.

 What we're thinking about is to pre-fill the cache of every page in that
 list, so that no user ever hits an uncached page. This pre-fillling
 could take place in a cronjob, which is run every 5 minutes or so -
 frequent enough for this use case.

 What i was thinking about is to set the cache-lifetime of the partial for a
 single page to one day, and update the cached results
 in the background - which actually is the part i'm not sure how to
 implement.

 Anyone knows if this kind of logic could be implemented using symfony +
 doctrine?

 thanks!

 -robert

 --
 If you want to report a vulnerability issue on symfony, please send it to
 security at symfony-project.com

 You received this message because you are subscribed to the Google
 Groups symfony users group.
 To post to this group, send email to symfony-users@googlegroups.com
 To unsubscribe from this group, send email to
 symfony-users+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/symfony-users?hl=en




-- 
Gareth McCumskey
http://garethmccumskey.blogspot.com
twitter: @garethmcc
identi.ca: @garethmcc

-- 
If you want to report a vulnerability issue on symfony, please send it to 
security at symfony-project.com

You received this message because you are subscribed to the Google
Groups symfony users group.
To post to this group, send email to symfony-users@googlegroups.com
To unsubscribe from this group, send email to
symfony-users+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/symfony-users?hl=en

Re: [symfony-users] Auto-filling caches

2011-02-11 Thread Robert Gründler

this approach could work too, but you don't know in advance which page 
the user clicks next, so you'd have to
pre-load all currently available pages, which would lead to unnecessary 
server requests.


The hostname thing was solved by this if anyone runs into the same problem:

  view_cache_manager:
class: sfViewCacheManager
param:
  cache_key_use_host_name: false



On 2/11/11 10:27 AM, Gareth McCumskey wrote:
Why not just chain ajax requests? Have two divs, one visible one 
invisible. Load what the person wants to see with one ajax request. 
When that completes it calls another ajax request which loads the next 
page into the invisible div. When the person goes to the next page, 
the act of making that page visible starts loading the next page into 
the hidden div.


2011/2/10 Robert Gründler r.gruend...@gmail.com 
mailto:r.gruend...@gmail.com


Hi,

Imagine a blog where each BlogEntry has one Author, and can be
tagged with one ore more Tags.
The mainpage displays a paginated list of all blogentries. Every
entry displays the title, author and the tags. The pagination
is implemented using AJAX.

To display the paginated list using symfony + doctrine, 3 Database
queries are needed:

1. select count(*)  # get the total number of entries
2. select distinct ... limit ... offset # get the actual ids of
blog entries for that page
3. select ... where in (...)  # retrieve the data to display for
that page

When the number of blogs and the authors + tags is getting large,
these 3 queries can take up to 800 - 1000 ms, which is
not fast enough, if you want ajax pagination.

If a single page is cached, the ajax request takes about 200 - 300
ms, but the uncached page takes over 1 second to load.

What we're thinking about is to pre-fill the cache of every page
in that list, so that no user ever hits an uncached page. This
pre-fillling
could take place in a cronjob, which is run every 5 minutes or so
- frequent enough for this use case.

What i was thinking about is to set the cache-lifetime of the
partial for a single page to one day, and update the cached results
in the background - which actually is the part i'm not sure how to
implement.

Anyone knows if this kind of logic could be implemented using
symfony + doctrine?

thanks!

-robert

-- 
If you want to report a vulnerability issue on symfony, please

send it to security at symfony-project.com
http://symfony-project.com

You received this message because you are subscribed to the Google
Groups symfony users group.
To post to this group, send email to
symfony-users@googlegroups.com mailto:symfony-users@googlegroups.com
To unsubscribe from this group, send email to
symfony-users+unsubscr...@googlegroups.com
mailto:symfony-users%2bunsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/symfony-users?hl=en




--
Gareth McCumskey
http://garethmccumskey.blogspot.com
twitter: @garethmcc
identi.ca http://identi.ca: @garethmcc

--
If you want to report a vulnerability issue on symfony, please send it 
to security at symfony-project.com


You received this message because you are subscribed to the Google
Groups symfony users group.
To post to this group, send email to symfony-users@googlegroups.com
To unsubscribe from this group, send email to
symfony-users+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/symfony-users?hl=en


--
If you want to report a vulnerability issue on symfony, please send it to 
security at symfony-project.com

You received this message because you are subscribed to the Google
Groups symfony users group.
To post to this group, send email to symfony-users@googlegroups.com
To unsubscribe from this group, send email to
symfony-users+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/symfony-users?hl=en

Re: [symfony-users] Auto-filling caches

2011-02-10 Thread Benoit Montuelle

Hi,

Now if you use sfViewCacheManager, it seems you can do it more naturally
when a new post is created, with setPageCache goog_277624711
http://trac.symfony-project.org/browser/branches/1.4/lib/view/sfViewCacheManager.class.php#937
method.

Regards
Benoit

2011/2/10 Robert Gründler r.gruend...@gmail.com

Hi,

Imagine a blog where each BlogEntry has one Author, and can be tagged with
one ore more Tags.
The mainpage displays a paginated list of all blogentries. Every entry
displays the title, author and the tags. The pagination
is implemented using AJAX.

To display the paginated list using symfony + doctrine, 3 Database queries
are needed:

1. select count(*) # get the total number of entries
2. select distinct ... limit ... offset # get the actual ids of blog
entries for that page
3. select ... where in (...) # retrieve the data to display for that page

When the number of blogs and the authors + tags is getting large, these 3
queries can take up to 800 - 1000 ms, which is
not fast enough, if you want ajax pagination.

If a single page is cached, the ajax request takes about 200 - 300 ms, but
the uncached page takes over 1 second to load.

What we're thinking about is to pre-fill the cache of every page in that
list, so that no user ever hits an uncached page. This pre-fillling
could take place in a cronjob, which is run every 5 minutes or so -
frequent enough for this use case.

What i was thinking about is to set the cache-lifetime of the partial for a
single page to one day, and update the cached results
in the background - which actually is the part i'm not sure how to
implement.

Anyone knows if this kind of logic could be implemented using symfony +
doctrine?

thanks!

-robert

--
If you want to report a vulnerability issue on symfony, please send it to
security at symfony-project.com

Re: [symfony-users] Auto-filling caches

Re: [symfony-users] Auto-filling caches

Re: [symfony-users] Auto-filling caches

Re: [symfony-users] Auto-filling caches

Re: [symfony-users] Auto-filling caches

5 matches

Site Navigation

Mail list logo

Footer information