Re: Miscellaneous questions

2008-03-17 Thread Dag-Erling Smørgrav
"Michael S. Fischer" <[EMAIL PROTECTED]> writes: > Dag-Erling Smørgrav <[EMAIL PROTECTED]> writes: > > I think the default timeout on backends connection may be a little > > short, though. > I assume this is the thread_pool_timeout parameter? No, that's how long an idle worker thread is kept alive

Re: Miscellaneous questions

2008-03-17 Thread Augustin Amann
Dag-Erling Smørgrav a écrit : > "Michael S. Fischer" <[EMAIL PROTECTED]> writes: > >> Dag-Erling Smørgrav <[EMAIL PROTECTED]> writes: >> >>> I think the default timeout on backends connection may be a little >>> short, though. >>> >> I assume this is the thread_pool_timeout parameter

Re: Miscellaneous questions

2008-03-17 Thread C. Handel
On Mon, Mar 17, 2008 at 10:04 AM, Augustin Amann <[EMAIL PROTECTED]> wrote: > I'm thinking about the same idea of a reverse-proxy cache cluster for work. > I think that one way of doing that is to use HaProxy (*haproxy*.1wt.eu) > which implement such hash function. You could use it in front of >

Re: Miscellaneous questions

2008-03-17 Thread Cherife Li
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 2008-3-17 17:56, C. Handel wrote: | On Mon, Mar 17, 2008 at 10:04 AM, Augustin Amann <[EMAIL PROTECTED]> wrote: |> I'm thinking about the same idea of a reverse-proxy cache cluster for work. |> I think that one way of doing that is to use HaProxy

Re: Miscellaneous questions

2008-03-17 Thread Michael S. Fischer
On Mon, Mar 17, 2008 at 12:42 AM, Dag-Erling Smørgrav <[EMAIL PROTECTED]> wrote: > "Michael S. Fischer" <[EMAIL PROTECTED]> writes: > > > Dag-Erling Smørgrav <[EMAIL PROTECTED]> writes: > > > I think the default timeout on backends connection may be a little > > > short, though. > > I assume thi

Re: Miscellaneous questions

2008-03-17 Thread Poul-Henning Kamp
In message <[EMAIL PROTECTED]>, "Mich ael S. Fischer" writes: >On Mon, Mar 17, 2008 at 12:42 AM, Dag-Erling Sm=F8rgrav <[EMAIL PROTECTED]> >wro= >te: >> "Michael S. Fischer" <[EMAIL PROTECTED]> writes: >> >> > Dag-Erling Sm=F8rgrav <[EMAIL PROTECTED]> writes: >> > > I think the default timeout on

Re: Miscellaneous questions

2008-03-17 Thread Dag-Erling Smørgrav
"Poul-Henning Kamp" <[EMAIL PROTECTED]> writes: > "Michael S. Fischer" <[EMAIL PROTECTED]> writes: > > "Dag-Erling Smørgrav" <[EMAIL PROTECTED]> writes: > > > "Michael S. Fischer" <[EMAIL PROTECTED]> writes: > > > > "Dag-Erling Smørgrav" <[EMAIL PROTECTED]> writes: > > > > > I think the default tim

Re: Miscellaneous questions

2008-03-17 Thread Poul-Henning Kamp
In message <[EMAIL PROTECTED]>, =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= writes: >> > What does the timeout pertain to? Connect time? Response time? >> Actually, I don't think we have any non-default timeout on the >> backends now: connect timeout is whatever the kernel uses and we don't >> set any

Re: Miscellaneous questions

2008-03-17 Thread Michael S. Fischer
On Mon, Mar 17, 2008 at 8:57 AM, Poul-Henning Kamp <[EMAIL PROTECTED]> wrote: > >No, we were talking about how long an idle backend connection is kept > >open (or at least I was). > > Yes I know :-) > > And we don't do anything to close those before the backend closes on > us, we have no reas

Re: Miscellaneous questions

2008-03-17 Thread Poul-Henning Kamp
In message <[EMAIL PROTECTED]>, "Mich ael S. Fischer" writes: >On Mon, Mar 17, 2008 at 8:57 AM, Poul-Henning Kamp <[EMAIL PROTECTED]> wrote: > >> >No, we were talking about how long an idle backend connection is kept >> >open (or at least I was). >> >> Yes I know :-) >> >> And we don't do anyth

Re: how to...accelarate randon access to millions of images?

2008-03-17 Thread Sascha Ottolski
Am Sonntag 16 März 2008 15:54:42 schrieben Sie: > Sascha Ottolski <[EMAIL PROTECTED]> writes: > > Now my question is: what kind of hardware would I need? Lots of RAM > > seems to be obvious, what ever "a lot" may be...What about the disk > > subsystem? Should I look into something like RAID-0 with

Re: how to...accelarate randon access to millions of images?

2008-03-17 Thread Sascha Ottolski
Michael, thanks a lot for taking the time to give me such a detailed answer. please see my replies below. Am Sonntag 16 März 2008 18:00:42 schrieb Michael S. Fischer: > On Fri, Mar 14, 2008 at 1:37 PM, Sascha Ottolski <[EMAIL PROTECTED]> wrote: > > The challenge is to server 20+ million image

Re: how to...accelarate randon access to millions of images?

2008-03-17 Thread C. Handel
> > On Fri, Mar 14, 2008 at 1:37 PM, Sascha Ottolski <[EMAIL PROTECTED]> > wrote: > > > The challenge is to server 20+ million image files, I guess with > > > up to 1500 req/sec at peak. > well, wo far I have analyzed the webserver logs of one week. this > indicates that indeed there would b

Re: how to...accelarate randon access to millions of images?

2008-03-17 Thread Poul-Henning Kamp
In message <[EMAIL PROTECTED]>, "C. Ha ndel" writes: >Serving static files using a http-accelerator seems odd to me. Uhm, this is a pretty weird statement if you stop to think about it :-) There is no way you could serve a dynamic file from an accellerator without defining it as 'static' for a m

Re: how to...accelarate randon access to millions of images?

2008-03-17 Thread C. Handel
On Mon, Mar 17, 2008 at 8:06 PM, Poul-Henning Kamp <[EMAIL PROTECTED]> wrote: > >Serving static files using a http-accelerator seems odd to me. > > Uhm, this is a pretty weird statement if you stop to think about it :-) > > There is no way you could serve a dynamic file from an accellerator > w

Re: how to...accelarate randon access to millions of images?

2008-03-17 Thread Poul-Henning Kamp
In message <[EMAIL PROTECTED]>, "C. H andel" writes: > Finally, I would advice you guys to seriously look at flash-"disk" >> drives. The virtual elimination of seektime is just what you want >> from a web server or cache. > >Having Flash Drives for 400GB of content could kill some budgets ;)

Re: Miscellaneous questions

2008-03-17 Thread DHF
Dag-Erling Smørgrav wrote: >> One good way to do this is to put a pass-only Varnish instance (i.e., >> a content switch) in front of a set of intermediate backends (Varnish >> caching proxies), each of which is assigned to cache a subset of the >> possible URI namespace. >> >> However, in order to

Re: Miscellaneous questions

2008-03-17 Thread Michael S. Fischer
On Mon, Mar 17, 2008 at 3:32 PM, DHF <[EMAIL PROTECTED]> wrote: > This is called CARP/"Cache Array Routing Protocol" in squid land. > Here's a link to some info on it: > > http://docs.huihoo.com/gnu_linux/squid/html/x2398.html > > It works quite well for reducing the number of globally duplicat