Re: [squid-users] Architecture for scaling delivery of large static files

2009-07-16 Thread Amos Jeffries
Jamie Tufnell wrote: On Thu, Jul 16, 2009 at 12:29 PM, Amos Jeffries wrote: On Thu, 16 Jul 2009 09:22:25 +1000, Jamie Tufnell We are talking files up-to-1GB in size here. Taking that into consideration, would you still recommend this architecture? Yes, the architecture itself is sound. OK

Re: [squid-users] Architecture for scaling delivery of large static files

2009-07-16 Thread Jamie Tufnell
On Thu, Jul 16, 2009 at 12:29 PM, Amos Jeffries wrote: > On Thu, 16 Jul 2009 09:22:25 +1000, Jamie Tufnell >> We are talking files up-to-1GB in size here.  Taking that into >> consideration, would you still recommend this architecture? > > Yes, the architecture itself is sound. OK cool. Thanks a

Re: [squid-users] Architecture for scaling delivery of large static files

2009-07-16 Thread Henrik Nordstrom
tor 2009-07-16 klockan 20:13 +1200 skrev Amos Jeffries: > Thanks Henrik. I was a bit unsure of that split. It's from the sfileno and sdirno being packed in the same 32-bit slot, both signed.. Relatively easy to change in the code if needed. Regards Henrik

Re: [squid-users] Architecture for scaling delivery of large static files

2009-07-16 Thread Amos Jeffries
Henrik Nordstrom wrote: tor 2009-07-16 klockan 14:29 +1200 skrev Amos Jeffries: For you with MB->GB files in Squid-2 that changes to faster Squid due to limiting RAM-cache to small files, with lots of large fast disks. Squid is limited to a few million (2^24) cache _objects_ per cache_dir, an

Re: [squid-users] Architecture for scaling delivery of large static files

2009-07-16 Thread Adrian Chadd
I was going to say; I'm tweaking the performance of a cache with 21 million objects in it now. Thats a bti bigger than 2^24. 2009/7/16 Henrik Nordstrom : > tor 2009-07-16 klockan 14:29 +1200 skrev Amos Jeffries: > >> For you with MB->GB files in Squid-2 that changes to faster Squid due to >> limit

Re: [squid-users] Architecture for scaling delivery of large static files

2009-07-15 Thread Henrik Nordstrom
tor 2009-07-16 klockan 14:29 +1200 skrev Amos Jeffries: > For you with MB->GB files in Squid-2 that changes to faster Squid due to > limiting RAM-cache to small files, with lots of large fast disks. Squid is > limited to a few million (2^24) cache _objects_ per cache_dir, and up to 32 (2^6) cache

Re: [squid-users] Architecture for scaling delivery of large static files

2009-07-15 Thread Amos Jeffries
On Thu, 16 Jul 2009 09:22:25 +1000, Jamie Tufnell wrote: > Thank you both for your responses, good to hear I might be on the right > track! > > Amos wrote: >> Just note that for MB or so scale files in memory Squid-2 is a snail, and >> Squid-3 does not yet provide collapsed forwarding. > > We ar

Re: [squid-users] Architecture for scaling delivery of large static files

2009-07-15 Thread Adrian Chadd
2009/7/16 Jamie Tufnell : > We are talking files up-to-1GB in size here.  Taking that into > consideration, would you still recommend this architecture? On disk? Sure. The disk buffer cache helps quite a bit. In memory ? (as in, the squid hot object cache; not the buffer cache) ? Not without inv

Re: [squid-users] Architecture for scaling delivery of large static files

2009-07-15 Thread Jamie Tufnell
Thank you both for your responses, good to hear I might be on the right track! Amos wrote: > Just note that for MB or so scale files in memory Squid-2 is a snail, and > Squid-3 does not yet provide collapsed forwarding. We are talking files up-to-1GB in size here. Taking that into consideration,

Re: [squid-users] Architecture for scaling delivery of large static files

2009-07-15 Thread Amos Jeffries
Jamie Tufnell wrote: Hi, I am wondering if Squid is the right tool to solve a scaling problem we're having. Our static content is currently served directly from Apache boxes to the end-user: User <=> Apache Originally it was just one Apache box but its Disk IO became saturated and now we have

[squid-users] Architecture for scaling delivery of large static files

2009-07-15 Thread Jamie Tufnell
Hi, I am wondering if Squid is the right tool to solve a scaling problem we're having. Our static content is currently served directly from Apache boxes to the end-user: User <=> Apache Originally it was just one Apache box but its Disk IO became saturated and now we have three Apache boxes, ea