[squid-users] Performance tuning of SMP + Large rock

2014-02-12 Thread Rajiv Desai
Hi, I am using squid cache as a forward caching proxy. CONTEXT: For my use case since: 1. the average object size is ~80KB (moreover > 32KB), 2. the proxy server has multiple cores available 3. the throughput requirement is high (upto 1Gbps) I have configured squid to use SMP + LargeRock. I am

Fwd: [squid-users] Performance tuning of SMP + Large rock

2014-02-13 Thread Rajiv Desai
(resending to squid-users) On Thu, Feb 13, 2014 at 12:19 PM, Alex Rousskov wrote: > On 02/12/2014 08:04 PM, Rajiv Desai wrote: >> Hi, >> >> I am using squid cache as a forward caching proxy. >> >> CONTEXT: >> >> For my use case since: >> 1

Re: Fwd: [squid-users] Performance tuning of SMP + Large rock

2014-02-13 Thread Rajiv Desai
On Thu, Feb 13, 2014 at 3:02 PM, Alex Rousskov wrote: > On 02/13/2014 03:01 PM, Rajiv Desai wrote: > >> When using LargeRock, what doe the io pattern correspond to? 16KB >> random reads if slot size is 16KB? > > I am not 100% sure the reads are always 16KB. Squid may reque

Re: Fwd: [squid-users] Performance tuning of SMP + Large rock

2014-02-16 Thread Rajiv Desai
On Sat, Feb 15, 2014 at 10:12 AM, Dr.x wrote: > @Rajiv Desai > > have u found increasing in bandwidth saving when u used large rock ?? Yes. Large rock works pretty well with multiple SMP workers (in my limited experience for past 5 days). I get 85% hit rate for previously read (an

[squid-users] Seemingly incorrect behavior: squid cache getting filled up on PUT requests

2014-02-16 Thread Rajiv Desai
I am using Squid Cache: Version 3.HEAD-20140127-r13248 My cache dir is configured to use rock (Large rock with SMP): cache_dir rock /mnt/squid-cache 256000 max-size=4194304 My refresh pattern is permissive to cache all objects: refresh_pattern . 129600 100% 129600 ignore-auth I uploaded 30 GB of

Re: [squid-users] Seemingly incorrect behavior: squid cache getting filled up on PUT requests

2014-02-16 Thread Rajiv Desai
On Sun, Feb 16, 2014 at 3:39 PM, Amos Jeffries wrote: > On 17/02/2014 11:41 a.m., Rajiv Desai wrote: >> I am using Squid Cache: >> Version 3.HEAD-20140127-r13248 >> >> My cache dir is configured to use rock (Large rock with SMP): >> cache_dir rock /mnt/squid-cache

Re: [squid-users] Seemingly incorrect behavior: squid cache getting filled up on PUT requests

2014-02-16 Thread Rajiv Desai
that subsequent GETs for the same objects after PUTs do get a cache MISS. On Sun, Feb 16, 2014 at 3:45 PM, Rajiv Desai wrote: > On Sun, Feb 16, 2014 at 3:39 PM, Amos Jeffries wrote: >> On 17/02/2014 11:41 a.m., Rajiv Desai wrote: >>> I am using Squid Cache: >>> Ver

Re: [squid-users] Seemingly incorrect behavior: squid cache getting filled up on PUT requests

2014-02-17 Thread Rajiv Desai
hInsert: Inserting Entry e:=p2DV/0x168f990*3 key 'AC671962CFC5644F4B22DA51C242DA50' 2014/02/17 00:06:55.128 kid1| ctx: exit level 0 2014/02/17 00:06:55.128 kid1| store.cc(858) write: storeWrite: writing 17 bytes for 'AC671962CFC5644F4B22DA51C242DA50' On Sun, Feb 16, 2014 at 11:07 PM, Rajiv

Re: [squid-users] Seemingly incorrect behavior: squid cache getting filled up on PUT requests

2014-02-17 Thread Rajiv Desai
(!method.respMaybeCacheable()) return false; -Rajiv On Mon, Feb 17, 2014 at 12:54 AM, Rajiv Desai wrote: > FWIW, from debug logs in cache.log, it seems like PUT responses are > being cached. > I am fairly new to using squid so I am be completely misreading these. > Just trying t

[squid-users] What are recommended settings for optimal sharing of cache between SMP workers?

2014-02-18 Thread Rajiv Desai
Hello, Need some guidance for optimal sharing of cache amongst SMP workers using Large rock. Context: I am using squid cache with 8 SMP workers and a 200 GB rock cache stored on SMP. (Using squid-3.HEAD-20140127-r13248 which has LargeRock support to cache objects > 32 KB). I have set : maximum_

[squid-users] Re: What are recommended settings for optimal sharing of cache between SMP workers?

2014-02-18 Thread Rajiv Desai
Internal Data Structures: 477 StoreEntries 477 StoreEntries with MemObjects 468 Hot Object Cache Items 39070 on-disk objects On Tue, Feb 18, 2014 at 1:52 AM, Rajiv Desai wrote: > Hello, > > Need some guidance for optimal sharing of cache amongst SMP workers > usin

[squid-users] Re: What are recommended settings for optimal sharing of cache between SMP workers?

2014-02-18 Thread Rajiv Desai
On Tue, Feb 18, 2014 at 2:42 AM, Rajiv Desai wrote: > Some more info: > > Following are mgr:storedir stats after back to back downloads for 4 GB > data (ie same 2 GB twice). > Perhaps the 477 StoreEntries with MemObjects AND 468 Hot Object > Cache Items are not shared? Nah

Re: [squid-users] Re: What are recommended settings for optimal sharing of cache between SMP workers?

2014-02-18 Thread Rajiv Desai
I dug deeper into the original issue I reported where some objects were not gettings cache HITs on subsequent reads. It seems like there is a bug where an object overwrites previously written object: For a similar "multiple downloads on same data via squid" test, I directed both store and cache lo

Re: [squid-users] Re: What are recommended settings for optimal sharing of cache between SMP workers?

2014-02-18 Thread Rajiv Desai
On Tue, Feb 18, 2014 at 9:28 PM, Alex Rousskov wrote: > On 02/18/2014 04:11 PM, Rajiv Desai wrote: > >> It seems like there is a bug where an object overwrites previously >> written object: > > This is not really a bug -- hash collisions do happen, as you have > discove

Re: [squid-users] Re: What are recommended settings for optimal sharing of cache between SMP workers?

2014-02-19 Thread Rajiv Desai
When the cache is nearing capacity, will the collision rate increase dramatically? Is there any special handling to reduce collisions and follow lru eviction? On Tue, Feb 18, 2014 at 10:30 PM, Rajiv Desai wrote: > On Tue, Feb 18, 2014 at 9:28 PM, Alex Rousskov > wrote: >> On 02/18/2

[squid-users] ignore-auth when signature is passed as a query

2014-02-22 Thread Rajiv Desai
Hi, I use ignore-auth for caching urls with signature and that works great. For some endpoints, signature is added as a query (which is otherwise an "Authorization" header). Is there a way to perhaps ignore-query? Currently, I see that replies are cached even when they have queries, however query