Thanks Ian, Graham and Sergio for your help.
for past couple of days I am trying to figure out how our
mod-cache-requester should spawn thread (or set of threads).
Currently, I am considering following option. please let me know what
you think about this approach.
- mod-cache-requester would be
@httpd.apache.org
Subject: Re: mod-cache-requestor plan
Thanks Ian, Graham and Sergio for your help.
for past couple of days I am trying to figure out how our
mod-cache-requester should spawn thread (or set of threads).
Currently, I am considering following option. please let me know what
you think about
something needs to be regenerated.
Bye
Sergio
-Original Message-
From: Parin Shah [mailto:[EMAIL PROTECTED]
Sent: venerdì 22 luglio 2005 8.02
To: dev@httpd.apache.org
Subject: Re: mod-cache-requestor plan
Thanks Ian, Graham and Sergio for your help.
for past couple of days I
Hi All,
We are now almost at consesus about this new mod-cache-requester
module's mechanism. and now I believe its good time to start
implementing the module.
But before I could do that, I need some help from you guys.
- I am now comfortable with mod-cache, mod-mem-cache, cache_storage.c,
: mercoledì 20 luglio 2005 8.34
To: dev@httpd.apache.org
Subject: Re: mod-cache-requestor plan
Hi All,
We are now almost at consesus about this new mod-cache-requester
module's mechanism. and now I believe its good time to start
implementing the module.
But before I could do that, I need some help from you
Parin Shah wrote:
2. how mod-cache-requester can generate the sub request just to reload
the content in the cache.
Look inside mod_include - it uses subrequests to be able to embed pages
within other pages.
Regards,
Graham
--
Parin Shah wrote:
you should be using a mix of
# requests
last access time
cost of reproducing the request.
Just to double check, we would insert entry into the 'refresh queue'
only if the page is requested and the page is soon-to-be-expired. once
it is in the queue we would use above
you should be using a mix of
# requests
last access time
cost of reproducing the request.
Just to double check, we would insert entry into the 'refresh queue'
only if the page is requested and the page is soon-to-be-expired. once
it is in the queue we would use above parameters to
Parin Shah wrote:
- I would prefer the approach where we maintain priority queue to keep
track of popularity. But again you guys have more insight and
understanding. so whichever approach you guys decide, I am ready to
work on it! ;-)
Beware of scope creep - we can always start with something
On 7/16/05, Graham Leggett [EMAIL PROTECTED] wrote:
Parin Shah wrote:
- I would prefer the approach where we maintain priority queue to keep
track of popularity. But again you guys have more insight and
understanding. so whichever approach you guys decide, I am ready to
work on it! ;-)
Parin Shah wrote:
On 7/15/05, Colm MacCarthaigh [EMAIL PROTECTED] wrote:
On Fri, Jul 15, 2005 at 01:23:29AM -0500, Parin Shah wrote:
- we need to maintain a counter for url in this case which would
decide the priority of the url. But mainting this counter should be a
low overhead operation,
Thanks all for for your thoughts on this issue.
The priority re-fetch would make sure the
popular pages are always in cache, while others are allowed to die at
their expense.
So every request for an object would update a counter for that url?
- we need to maintain a counter for url
On Fri, Jul 15, 2005 at 01:23:29AM -0500, Parin Shah wrote:
- we need to maintain a counter for url in this case which would
decide the priority of the url. But mainting this counter should be a
low overhead operation, I believe.
Is a counter strictly speaking the right approach? Why not a
On 7/15/05, Colm MacCarthaigh [EMAIL PROTECTED] wrote:
On Fri, Jul 15, 2005 at 01:23:29AM -0500, Parin Shah wrote:
- we need to maintain a counter for url in this case which would
decide the priority of the url. But mainting this counter should be a
low overhead operation, I believe.
Is
On 7/13/05 6:36 PM, Ian Holsman [EMAIL PROTECTED] wrote:
Hi There.
just remember that this project is Parin's SoC project, and he is
expected to do the code on it.
sure. I am expected to do what's best for my employer and the httpd
project.
While normally I think it would be great to
On 7/13/05 6:41 PM, Ian Holsman [EMAIL PROTECTED] wrote:
a pool of threads read the queue and start fetching the content, and
re-filling the cache with fresh responses.
How is this better than simply having an external cron job to fetch the
urls? You have total control of throttling there
Akins, Brian wrote:
On 7/13/05 6:41 PM, Ian Holsman [EMAIL PROTECTED] wrote:
a pool of threads read the queue and start fetching the content, and
re-filling the cache with fresh responses.
How is this better than simply having an external cron job to fetch the
urls? You have total
This was a private message. I will continue this one offline.
Akins, Brian wrote:
On 7/13/05 6:36 PM, Ian Holsman [EMAIL PROTECTED] wrote:
Hi There.
just remember that this project is Parin's SoC project, and he is
expected to do the code on it.
sure. I am expected to do what's best for
On 7/14/05 9:59 AM, Ian Holsman [EMAIL PROTECTED] wrote:
that wouldn't keep track of the popularity of the given url, only when
it is stored.
Which would be a useful input to something like htcacheclean so that it does
not have to scan directories.
The priority re-fetch would make
On 7/12/05 10:27 PM, Parin Shah [EMAIL PROTECTED] wrote:
Also, one of the flaws of mod_disk_cache (at least the version I am looking
at) is that it deletes objects before reloading them. It is better for many
reasons to only replace them. That's the best way to accomplish what I
Parin Shah wrote:
- In this case, what would be the criteria to determine which pages
should be refreshed and which should be left out. intitially I thought
that all the pages - those are about to expire and have been requested -
should be refreshed. but, if we consider keeping non-popular
On 7/13/05 2:43 PM, Graham Leggett [EMAIL PROTECTED] wrote:
This was one of the basic design goals of the new cache, but the code
for it was never written.
It was logged as a bug against the original v1.3 proxy cache, which
suffered from thundering herd when cache entries expired.
At
We have been down this road. The way one might solve it is to allow
mod_cache to be able to reload an object while serving the old one.
Example:
cache /A for 600 seconds
after 500 seconds, request /A with special header (or from special client,
etc) and cache does not serve from
[EMAIL PROTECTED]
Data: Sun, 10 Jul 2005 23:24:10 -0500
A: dev@httpd.apache.org
Oggetto: mod-cache-requestor plan
Hi All,
I am a newbie. I am going to work on mod-cache and a new module
mod-cache-requester as a part of Soc program.
Small description of the module is as follows.
When
Parin Shah said:
When the page expires from the cache, it is removed from cache and
thus next request has to wait until that page is reloaded by the
back-end server.
This is not strictly true - when a page expires from the cache, a
conditional request is sent to the backend server, and if a
Hi all
I basically agree with Graham, with just one observation on multi-threaded
subrequests.
I believe the basic idea of forwarding multiple requests on the back end can be
a very good idea, but needs some bounds as Graham suggests.
In my opinion you can define a mod_cache_requester
- Cache freshness of an URL is checked on each hit to the URL. This runs the risk of allowing non-popular (but possibly expensive) URLs to expire
without the chance to be refreshed.
- Cache freshness is checked in an independant thread, which monitors the cached URLs for freshness at
I believe the basic idea of
forwarding multiple requests on the back end can be a very good idea,
but needs some bounds as Graham suggests. ..
its an interesting thought. But after Graham's opinion, I am not too
sure about performance improvement/overload incured by threads ratio.
if we could
Hi All,
I am a newbie. I am going to work on mod-cache and a new module
mod-cache-requester as a part of Soc program.
Small description of the module is as follows.
When the page expires from the cache, it is removed from cache and
thus next request has to wait until that page is reloaded by
29 matches
Mail list logo
