On 03/30/2009 01:00 PM, Pieter De Wit wrote: > On Mon, 30 Mar 2009 11:38:31 -0600, Alex Rousskov > <rouss...@measurement-factory.com> wrote: >> On 03/30/2009 03:16 AM, Pieter De Wit wrote: >> >>> I gave this some thought. Why don't we build a system close to >>> sendmail's milter system. An API where by clients can plug in and offer >>> services - one can be a traffic counter, traffic limiter (as what we >>> proposing here) and maybe a URL block, a virus scanner etc.
>> We already have such a plugin API for message- and content-related >> tasks. It is called eCAP, and it has been mentioned on this thread. >> >> Compared to eCAP, traffic shaping and quotas are different in scope as >> they work on multiple messages and often do not care about the messages >> at all (just raw bytes traffic). So, we have a few options: >> >> 1) Use eCAP nearly "as is" for traffic shaping and quotas, even though >> it is not a perfect fit for the task. >> >> 2) Significantly enhance eCAP to offer traffic shaping-specific hooks >> (as a standard addition or as a Squid extension), even though it may >> lead to eCAP API bloat. >> >> 3) Develop a different plugin API that specializes in aggregate traffic >> management and is unrelated to eCAP, even though it may lead to >> duplicating a lot of eCAP-related code. > Client/Bandwidth limiting registers with squid > ... > squid get a request to download object x from site y by user z at ip > a.b.c.d etc > squid sends that request to the client (just "text" not the object) > client replies "yes" or "no" - if yes the client needs to track how much > data is left for that user etc. > > (Client here is the limiting software - couldn't think of a better > name...coffee....) > > I feel this should be a persistent connection that is matched by an acl for > speed etc. I doubt the traffic shaping API can be that simple and still cover most useful cases: the size of the object is not always known a priori; some traffic shaping decisions need more than user ID, client IP, and a URL; there are several points at which traffic shaping may take place; etc. I also suspect it would be better to avoid mandating a connection-based interface for this API. A loadable module can use a connection-based approach to communicate with the actual traffic shaper, but the reverse is not possible: once you standardize on the connection-based API, you cannot do embedded modules (which are usually much more efficient). We probably would not invent eCAP if ICAP (which uses TCP connections) and various helpers were good enough from that point of view. > Now to the workings of this. I am leaning towards option 3. I am not sure > how hard it is to maintain "two" API's. To be honest, I am not even sure > how hard it is to maintain one API. I feel that this really only needs two > or three commands, so is it really going to bloat the API ? You need three commands. The person next to you needs another five. By the end of the day, you end up with a few dozen commands OR several distinct APIs. Just look at how many different "helpers" Squid already has and how easy/difficult it is to extend them. However, when I was talking about API bloat, I was not talking about the number of "commands" in the new API. I was talking about bloating the existing API (eCAP) with traffic shaping "commands". > Broken down - what is the most "this API" would need ? We probably should define the scope before counting the "commands": Inbound, outbound, and peer traffic shaping. Shaping based on group membership, user credentials, IP addresses, TCP connection IDs, MIME types, HTTP headers. Shaping during message download. Reconfiguration of quotas. Etc., etc. Cheers, Alex.