Re: How to buffer a POST request

2008-06-11 Thread Mikko Kettunen
Henrik Nordstrom wrote: > On ons, 2008-06-11 at 12:51 +0300, Mikko Kettunen wrote: > >> Yes, I read something about this on squid-users list, there seems to be >> 8kB buffer for this if I understood right. > > The buffer is bigger than that. But not unlimited. Ok. > The big change needed is tha

Re: Squid 3 HEAD - compiler options

2008-06-11 Thread Adrian Chadd
Don't omit the frame pointer in 64 bit mode, it makes walking the stack for things like profiling callgraphs impossible. Adrian On Thu, Jun 12, 2008, Amos Jeffries wrote: > > Just for the sake of consistency and preference for better > > troubleshooting and testing, are there any CFLAGS that I

Re: Squid 3 HEAD - compiler options

2008-06-11 Thread Amos Jeffries
> Just for the sake of consistency and preference for better > troubleshooting and testing, are there any CFLAGS that I should be using > during configure and compile of Squid 3HEAD? I was using the same > options I was using with 2.6, which were: > > CFLAGS="-DNUMTHREADS=65 -march=nocona -03 -pipe

Re: Round-robin bugs

2008-06-11 Thread Mark Nottingham
On 12/06/2008, at 11:45 AM, Henrik Nordstrom wrote: On tor, 2008-06-12 at 10:53 +1000, Mark Nottingham wrote: Any chance of getting the proposed fixes in 2241, 2376 and 2378 into 2.7STABLE3? 2241: yes Thanks (was Mike's patch, not mine). 2376: if fixed See followup. 2378: maybe Do

Re: Bug #1893; why's it incomplete?

2008-06-11 Thread Adrian Chadd
On Thu, Jun 12, 2008, Henrik Nordstrom wrote: > On tor, 2008-06-12 at 09:40 +0800, Adrian Chadd wrote: > > Argh! > > > > Please start putting relevant details into: > > > > http://wiki.squid-cache.org/Features/VaryProcessing > > Why, when there is a bugzilla report for this bug? > > It's not a

Re: Bug #1893; why's it incomplete?

2008-06-11 Thread Henrik Nordstrom
On tor, 2008-06-12 at 09:40 +0800, Adrian Chadd wrote: > Argh! > > Please start putting relevant details into: > > http://wiki.squid-cache.org/Features/VaryProcessing Why, when there is a bugzilla report for this bug? It's not a new Vary requirement, it's just a bug in that patch. Regards Henr

Re: Round-robin bugs

2008-06-11 Thread Henrik Nordstrom
On tor, 2008-06-12 at 10:53 +1000, Mark Nottingham wrote: > Any chance of getting the proposed fixes in 2241, 2376 and 2378 into > 2.7STABLE3? 2241: yes 2376: if fixed 2378: maybe Regards Henrik signature.asc Description: This is a digitally signed message part

Re: Bug #1893; why's it incomplete?

2008-06-11 Thread Adrian Chadd
Argh! Please start putting relevant details into: http://wiki.squid-cache.org/Features/VaryProcessing That way we can have a proper list of whats required. Adrian On Thu, Jun 12, 2008, Mark Nottingham wrote: > These issues may be relevant: > http://tools.ietf.org/wg/httpbis/trac/ticket/37

Re: Bug #1893; why's it incomplete?

2008-06-11 Thread Henrik Nordstrom
On tor, 2008-06-12 at 10:57 +1000, Benno Rice wrote: > On 11/06/2008, at 10:24 PM, Adrian Chadd wrote: > > > On Wed, Jun 11, 2008, Henrik Nordstrom wrote: > >> On ons, 2008-06-11 at 17:50 +0800, Adrian Chadd wrote: > >> > >>> Are there any objections to removing this patch from Squid-2.HEAD > >>

Re: Bug #1893; why's it incomplete?

2008-06-11 Thread Mark Nottingham
These issues may be relevant: http://tools.ietf.org/wg/httpbis/trac/ticket/37 http://tools.ietf.org/wg/httpbis/trac/ticket/38 If I understand correctly, Henrik is talking about a case where these two responses might be seen from an origin: HTTP/1.1 200 OK Content-Type: application/foo Cac

Re: Bug #1893; why's it incomplete?

2008-06-11 Thread Benno Rice
On 11/06/2008, at 10:24 PM, Adrian Chadd wrote: On Wed, Jun 11, 2008, Henrik Nordstrom wrote: On ons, 2008-06-11 at 17:50 +0800, Adrian Chadd wrote: Are there any objections to removing this patch from Squid-2.HEAD and documenting this discussion (and other Vary related stuff) on a wiki p

Round-robin bugs

2008-06-11 Thread Mark Nottingham
Any chance of getting the proposed fixes in 2241, 2376 and 2378 into 2.7STABLE3? Cheers, -- Mark Nottingham [EMAIL PROTECTED]

Squid 3 HEAD - compiler options

2008-06-11 Thread Ritter, Nicholas
Just for the sake of consistency and preference for better troubleshooting and testing, are there any CFLAGS that I should be using during configure and compile of Squid 3HEAD? I was using the same options I was using with 2.6, which were: CFLAGS="-DNUMTHREADS=65 -march=nocona -03 -pipe -fomit-fr

Subscription User Introduction

2008-06-11 Thread Ritter, Nicholas
Greetings to all of the developers, As per the squid website, I am taking a moment to introduce myself to the list. My name is Nicholas Ritter, I am a network/systems engineer working for a small retail chain in the Midwest of the US. I have been using Linux since about 1997 and squid since about

Re: aio #2: why aren't we doing async writes?

2008-06-11 Thread Adrian Chadd
On Wed, Jun 11, 2008, Henrik Nordstrom wrote: > On ons, 2008-06-11 at 20:01 +0800, Adrian Chadd wrote: > > > What I need to do is get this Sun box actually running Solaris and try to > > get dtrace to tell me what time is spent in the main thread blocking on > > disk writes. > > On Solaris you de

Re: aio #2: why aren't we doing async writes?

2008-06-11 Thread Henrik Nordstrom
On ons, 2008-06-11 at 20:01 +0800, Adrian Chadd wrote: > What I need to do is get this Sun box actually running Solaris and try to > get dtrace to tell me what time is spent in the main thread blocking on > disk writes. On Solaris you defenitely want async writes and closes, as Solaris UFS by def

Re: Bug #1893; why's it incomplete?

2008-06-11 Thread Adrian Chadd
On Wed, Jun 11, 2008, Henrik Nordstrom wrote: > On ons, 2008-06-11 at 17:50 +0800, Adrian Chadd wrote: > > > Are there any objections to removing this patch from Squid-2.HEAD and > > documenting this discussion (and other Vary related stuff) on a wiki page? > > No problem. Ok. I'll commit a reve

Re: aio #2: why aren't we doing async writes?

2008-06-11 Thread Adrian Chadd
On Wed, Jun 11, 2008, Henrik Nordstrom wrote: > Oh, forgot to say that there is #define's in store_asyncufs.h tuning > this. But the non-default settings hasn't been tested in a very very > long time.. (2000 timeframe) Yeah, I'm testing async write at the moment. But I'm not exactly testing it wi

Re: aio #2: why aren't we doing async writes?

2008-06-11 Thread Adrian Chadd
On Wed, Jun 11, 2008, Henrik Nordstrom wrote: > Simply because when testing on Linux they very rarely block, with the fs > buffers absorbing them fully as long as you don't run too short on > memory. Ah. > But yes, when you reach the absolute limit of what the drives can > sustain the buffers wi

Re: aio #2: why aren't we doing async writes?

2008-06-11 Thread Henrik Nordstrom
On ons, 2008-06-11 at 13:14 +0200, Henrik Nordstrom wrote: > On ons, 2008-06-11 at 17:48 +0800, Adrian Chadd wrote: > > Subject says it all - open, read and unlink are async, but write and close > > aren't. > > Why is at least write not async? > > Simply because when testing on Linux they very ra

Re: aio completion notification?

2008-06-11 Thread Henrik Nordstrom
On ons, 2008-06-11 at 18:43 +0800, Adrian Chadd wrote: > That wouldn't be terribly difficult. It could then service just > the queue for that storedir. Hm, ok. Exactly. Regards Henrik signature.asc Description: This is a digitally signed message part

Re: aio #2: why aren't we doing async writes?

2008-06-11 Thread Henrik Nordstrom
On ons, 2008-06-11 at 17:48 +0800, Adrian Chadd wrote: > Subject says it all - open, read and unlink are async, but write and close > aren't. > Why is at least write not async? Simply because when testing on Linux they very rarely block, with the fs buffers absorbing them fully as long as you don

Re: aio completion notification?

2008-06-11 Thread Adrian Chadd
On Wed, Jun 11, 2008, Henrik Nordstrom wrote: > On ons, 2008-06-11 at 17:47 +0800, Adrian Chadd wrote: > > How would it work? The diskd queues are all sysv msgq's which can't be > > polled as filedescriptors. What would wake up the main Squid process? > > make diskd output something on stdout afte

Re: Bug #1893; why's it incomplete?

2008-06-11 Thread Henrik Nordstrom
On ons, 2008-06-11 at 17:50 +0800, Adrian Chadd wrote: > Are there any objections to removing this patch from Squid-2.HEAD and > documenting this discussion (and other Vary related stuff) on a wiki page? No problem. > A Cacheboy tester has noted that there's a definite drop in hit rate between >

Re: How to buffer a POST request

2008-06-11 Thread Henrik Nordstrom
On ons, 2008-06-11 at 12:51 +0300, Mikko Kettunen wrote: > Yes, I read something about this on squid-users list, there seems to be > 8kB buffer for this if I understood right. The buffer is bigger than that. But not unlimited. The big change needed is that there currently isn't anything delaying

Re: aio completion notification?

2008-06-11 Thread Henrik Nordstrom
On ons, 2008-06-11 at 17:47 +0800, Adrian Chadd wrote: > How would it work? The diskd queues are all sysv msgq's which can't be > polled as filedescriptors. What would wake up the main Squid process? make diskd output something on stdout after sending the completion message to Squid. Regards Henr

Re: How to buffer a POST request

2008-06-11 Thread Mikko Kettunen
Henrik Nordstrom wrote: > On tis, 2008-06-10 at 12:40 +0300, Mikko Kettunen wrote: >> How could I configure squid to buffer whole POST-request before sending >> it to peer cache or originserver? > Good question. It isn't really designed to do this. POST request may be > almost unbounded in size, an

Re: Bug #1893; why's it incomplete?

2008-06-11 Thread Adrian Chadd
On Wed, Jun 11, 2008, Henrik Nordstrom wrote: [snip] > Probably better to rethink the patch to use the "vary id" as a timestamp > instead, marking objects with an older update time as stale. Are there any objections to removing this patch from Squid-2.HEAD and documenting this discussion (and ot

Re: How to buffer a POST request

2008-06-11 Thread Henrik Nordstrom
On tis, 2008-06-10 at 12:40 +0300, Mikko Kettunen wrote: > How could I configure squid to buffer whole POST-request before sending > it to peer cache or originserver? Good question. It isn't really designed to do this. POST request may be almost unbounded in size, and some streaming over HTTP pro

aio #2: why aren't we doing async writes?

2008-06-11 Thread Adrian Chadd
Subject says it all - open, read and unlink are async, but write and close aren't. Why is at least write not async? Adrian -- - Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid Support - - $25/pm entry-level VPSes w/ capped bandwidth charges available in WA -

Re: aio completion notification?

2008-06-11 Thread Adrian Chadd
On Wed, Jun 11, 2008, Henrik Nordstrom wrote: > > All the callback does is read into a temp buffer, and re-schedules for > > read IO. > > Yes.. > > The point of this wakeup is to break the select/poll/epoll/kqueue sleep. I found that in the CVS history. > It could obviously be optimized to onl

Re: aio completion notification?

2008-06-11 Thread Henrik Nordstrom
On ons, 2008-06-11 at 16:31 +0800, Adrian Chadd wrote: > So perusing the aio code in preparation to disassocate it from the Squid > fs code, I found that there's a pipe being used as a notification/wakeup > mechanism. Yes.. > Basically, thread completion writes a ! into the pipe; this wakes up th

Re: Bug #1893; why's it incomplete?

2008-06-11 Thread Henrik Nordstrom
On ons, 2008-06-11 at 01:18 +0800, Adrian Chadd wrote: > Henrik, > > Whats incomplete about the Variant invalidation support in PURGE/CLR in > Squid-2.HEAD? There are many common corner cases where the existing vary_id isn't picked up properly. The worst being if an object goes from not having Va

aio completion notification?

2008-06-11 Thread Adrian Chadd
So perusing the aio code in preparation to disassocate it from the Squid fs code, I found that there's a pipe being used as a notification/wakeup mechanism. Basically, thread completion writes a ! into the pipe; this wakes up the main squid loop and calls a PF callback. All the callback does is r