Re: squid-2.HEAD:
have you dumped this into bugzilla? Thanks! 2008/9/3 Alexander V. Lukyanov <[EMAIL PROTECTED]>: > Hello! > > I have noticed lots of 'impossible keep-alive' messages in the log. > It appears that httpReplyBodySize incorrectly returns -1 for "304 Not > Modified" replies. Patch to fix it is attached. > > -- > Alexander. >
Re: squid-2.HEAD: fwdComplete/Fail before comm_close
Hiya,
Could you please verify this is still a problem in the latest 2.HEAD
and if so lodge a bugzilla bug report with the patch?
Thanks!
Adrian
2008/8/5 Alexander V. Lukyanov <[EMAIL PROTECTED]>:
> Hello!
>
> Some time ago I had core dumps just after these messages:
>Short response from ...
>httpReadReply: Excess data from ...
>
> I beleave this patch fixes these problems.
>
> Index: http.c
> ===
> RCS file: /squid/squid/src/http.c,v
> retrieving revision 1.446
> diff -u -p -r1.446 http.c
> --- http.c 25 Jun 2008 22:11:20 - 1.446
> +++ http.c 5 Aug 2008 06:05:29 -
> @@ -755,6 +757,7 @@ httpAppendBody(HttpStateData * httpState
> /* Is it a incomplete reply? */
> if (httpState->chunk_size > 0) {
>debug(11, 2) ("Short response from '%s' on port %d. Expecting %"
> PRINTF_OFF_T " octets more\n", storeUrl(entry), comm_local_port(fd),
> httpState->chunk_size);
> + fwdFail(httpState->fwd, errorCon(ERR_INVALID_RESP, HTTP_BAD_GATEWAY,
> httpState->fwd->request));
>comm_close(fd);
>return;
> }
> @@ -774,6 +777,7 @@ httpAppendBody(HttpStateData * httpState
>("httpReadReply: Excess data from \"%s %s\"\n",
>RequestMethods[orig_request->method].str,
>storeUrl(entry));
> + fwdComplete(httpState->fwd);
>comm_close(fd);
>return;
> }
>
>
Re: squid-2.HEAD: storeCleanup and -F option (foreground rebuild)
I've committed a slightly modified version of this - store_rebuild.c
r1.80 . Take a look and see if it works for you.
Thanks!
Adrian
2008/8/5 Alexander V. Lukyanov <[EMAIL PROTECTED]>:
> Hello!
>
> I use squid in transparent mode, so I don't want degraded performance
> while rebuilding and cleanup. Here is a patch I use to make storeCleanup
> do all the work at once before squid starts processing requests, when
> -F option is specified on command line.
>
> Index: store_rebuild.c
> ===
> RCS file: /squid/squid/src/store_rebuild.c,v
> retrieving revision 1.80
> diff -u -p -r1.80 store_rebuild.c
> --- store_rebuild.c 1 Sep 2007 23:09:32 - 1.80
> +++ store_rebuild.c 5 Aug 2008 05:51:43 -
> @@ -68,7 +68,8 @@ storeCleanup(void *datanotused)
> hash_link *link_ptr = NULL;
> hash_link *link_next = NULL;
> validnum_start = validnum;
> -while (validnum - validnum_start < 500) {
> +int limit = opt_foreground_rebuild ? 1 << 30 : 500;
> +while (validnum - validnum_start < limit) {
>if (++bucketnum >= store_hash_buckets) {
>debug(20, 1) (" Completed Validation Procedure\n");
>debug(20, 1) (" Validated %d Entries\n", validnum);
> @@ -147,8 +148,8 @@ storeRebuildComplete(struct _store_rebui
> debug(20, 1) (" Took %3.1f seconds (%6.1f objects/sec).\n", dt,
>(double) counts.objcount / (dt > 0.0 ? dt : 1.0));
> debug(20, 1) ("Beginning Validation Procedure\n");
> -eventAdd("storeCleanup", storeCleanup, NULL, 0.0, 1);
> safe_free(RebuildProgress);
> +storeCleanup(0);
> }
>
> /*
>
>
Re: [MERGE] WCCPv2 Config Cleanup
Amos, why are you pushing through changes to the WCCP configuration stuff at this point in the game? The WCCPv2 stuff works fine (for what it does); it could do with some better documentation but what it really needs is to be broken out from Squid itself and run as a seperate daemon. Adrian 2008/9/13 Henrik Nordstrom <[EMAIL PROTECTED]>: > With the patch the code uses WCCP2_METHOD_.. in some places (config > parsing/dumping) and the context specific ones in other places. This is > even more confusing. > > Very minor detail in any case. > > > On lör, 2008-09-13 at 09:49 +0800, Adrian Chadd wrote: >> The specification defines them as separate entities and using them in >> this fashion makes it clearer for people working on the code. >> >> >> >> Adrian >> >> 2008/9/13 Henrik Nordstrom <[EMAIL PROTECTED]>: >> > On fre, 2008-09-12 at 20:39 +1200, Amos Jeffries wrote: >> > >> >> +#define WCCP2_FORWARDING_METHOD_GRE WCCP2_METHOD_GRE >> >> +#define WCCP2_FORWARDING_METHOD_L2 WCCP2_METHOD_L2 >> > >> >> +#define WCCP2_PACKET_RETURN_METHOD_GRE WCCP2_METHOD_GRE >> >> +#define WCCP2_PACKET_RETURN_METHOD_L2WCCP2_METHOD_L2 >> > >> > Do we still need these? Why not use WCCP2_METHOD_ everywhere if ther are >> > the same value? >> > >> > Regards >> > Henrik >> > >> > > >
Re: apologies for spamming squid-dev with apologies for spamming squid-dev with commit messages
Henrik Nordstrom wrote: On tor, 2008-09-11 at 23:07 -0600, Alex Rousskov wrote: It looks like I mistook the Reply-To: email header for a To: header. I was not really sending those commit messages to squid-dev, just to squid-cvs. So I was not really spamming. Until now... No problem. Got me a little confused there for a while, only seeing merge requests from you on squid-dev... I'm seeing two mails from you every time though, both through squid-cvs. One sent by you with Subject "Rev 123 ..." and one sent by bzr with "/branch/path r123 ..." Amos -- Please use Squid 2.7.STABLE4 or 3.0.STABLE8
Re: [MERGE] WCCPv2 Config Cleanup
With the patch the code uses WCCP2_METHOD_.. in some places (config parsing/dumping) and the context specific ones in other places. This is even more confusing. Very minor detail in any case. On lör, 2008-09-13 at 09:49 +0800, Adrian Chadd wrote: > The specification defines them as separate entities and using them in > this fashion makes it clearer for people working on the code. > > > > Adrian > > 2008/9/13 Henrik Nordstrom <[EMAIL PROTECTED]>: > > On fre, 2008-09-12 at 20:39 +1200, Amos Jeffries wrote: > > > >> +#define WCCP2_FORWARDING_METHOD_GRE WCCP2_METHOD_GRE > >> +#define WCCP2_FORWARDING_METHOD_L2 WCCP2_METHOD_L2 > > > >> +#define WCCP2_PACKET_RETURN_METHOD_GRE WCCP2_METHOD_GRE > >> +#define WCCP2_PACKET_RETURN_METHOD_L2WCCP2_METHOD_L2 > > > > Do we still need these? Why not use WCCP2_METHOD_ everywhere if ther are > > the same value? > > > > Regards > > Henrik > > > >
Re: [MERGE] WCCPv2 Config Cleanup
The specification defines them as separate entities and using them in this fashion makes it clearer for people working on the code. Adrian 2008/9/13 Henrik Nordstrom <[EMAIL PROTECTED]>: > On fre, 2008-09-12 at 20:39 +1200, Amos Jeffries wrote: > >> +#define WCCP2_FORWARDING_METHOD_GRE WCCP2_METHOD_GRE >> +#define WCCP2_FORWARDING_METHOD_L2 WCCP2_METHOD_L2 > >> +#define WCCP2_PACKET_RETURN_METHOD_GRE WCCP2_METHOD_GRE >> +#define WCCP2_PACKET_RETURN_METHOD_L2WCCP2_METHOD_L2 > > Do we still need these? Why not use WCCP2_METHOD_ everywhere if ther are > the same value? > > Regards > Henrik > >
Re: Where to document APIs?
On Fri, 2008-09-12 at 22:28 +0200, Henrik Nordstrom wrote: > On tor, 2008-09-11 at 09:54 -0600, Alex Rousskov wrote: > > > You mean as a large blob of comments at the end of some .h file? The > > cons are: > > I think Adrian is thinking more of blobs next to the implementations > (.cc) > > > As time has passed a little I have this slightly refined view: > > * Overview type documentation for a class, not tied to a specific > method/member: > > - small, in the .h file > > - larger, in the .cc or a src/class.dox, whatever suits the author > best. Personal preference is the .cc as it's that way more likely to be > remembered to update documentation when updating the implementation. > > - when there is no single .cc file matching the class. A src/class.dox > file, or rework of the implementation to get a cleaned match between .h > and .cc files... > * High level overviews or other not tied to a specific class or > implementation > > .dox files So where would you put Comm and AsyncCall API notes I have posted? I am guessing .dox because they are not tied to a specific class, right? Thanks, Alex.
Re: [MERGE] WCCPv2 Config Cleanup
On fre, 2008-09-12 at 20:39 +1200, Amos Jeffries wrote: > +#define WCCP2_FORWARDING_METHOD_GRE WCCP2_METHOD_GRE > +#define WCCP2_FORWARDING_METHOD_L2 WCCP2_METHOD_L2 > +#define WCCP2_PACKET_RETURN_METHOD_GRE WCCP2_METHOD_GRE > +#define WCCP2_PACKET_RETURN_METHOD_L2WCCP2_METHOD_L2 Do we still need these? Why not use WCCP2_METHOD_ everywhere if ther are the same value? Regards Henrik
Re: pseudo-specs for a String class: tokenization
On tor, 2008-09-11 at 10:03 -0600, Alex Rousskov wrote: > You made the right choice. The "modifying methods modify their object" > approach is more efficient and should be expected by most C++ > developers. Yes, the other approach (return a new object) is used mainly for operators. C = A + B; Methods that operate like operators is mostly confusing and needs to be learnt one by one.. Personally I prefer functions over methods for such use.. C = append(a, b) Note: append is a bad example as the + operator is a very good semantic match.. mostly interesting when there is no good operator to use. Regards Henrik
Re: Where to document APIs?
On tor, 2008-09-11 at 09:54 -0600, Alex Rousskov wrote: > You mean as a large blob of comments at the end of some .h file? The > cons are: I think Adrian is thinking more of blobs next to the implementations (.cc) As time has passed a little I have this slightly refined view: * Overview type documentation for a class, not tied to a specific method/member: - small, in the .h file - larger, in the .cc or a src/class.dox, whatever suits the author best. Personal preference is the .cc as it's that way more likely to be remembered to update documentation when updating the implementation. - when there is no single .cc file matching the class. A src/class.dox file, or rework of the implementation to get a cleaned match between .h and .cc files... * members the .h file * inline methods the .cci file * methods the .cc file, next to the implementation. * High level overviews or other not tied to a specific class or implementation .dox files Regards Henrik
Re: apologies for spamming squid-dev with apologies for spamming squid-dev with commit messages
On tor, 2008-09-11 at 23:07 -0600, Alex Rousskov wrote: > It looks like I mistook the Reply-To: email header for a To: header. I > was not really sending those commit messages to squid-dev, just to > squid-cvs. So I was not really spamming. Until now... No problem. Got me a little confused there for a while, only seeing merge requests from you on squid-dev... Regards Henrik
Re: broken regex, refresh_pattern, or request_/reply_header?
the *_header_access directives can only change the external view of the headers sent by Squid, not how Squid parses received headers. To override "bad" things sent by client or server you need to use refresh_pattern overrides. In your case both the ignore-reload and ignore-private overrides. Be warned that both these violate the HTTP specifications so you better know what you are doing when using these.. if used carelessly you may well end up sending one users private bank account details to another user.. On fre, 2008-09-12 at 14:48 -0500, Ritter, Nicholas wrote: > I might be way off on this, but I am wondering if it is my config, or if > it is the squid code. I am using squid-3.HEAD-20080910 and I an trying > to block Cache-Control headers in addition to control the fresh pattern > of jar files, but nothing seems to work. > > I tried various regex patterns including: > > urlpath_regex -i \.jar$ > > and even (just to see if it was my regex or something else): > > request_header_access Cache-Control deny all > reply_header_access Cache-Control deny all > > > as well as (just to see if it was my regex or something else): > > refresh_pattern -i \.jar$ 480 20% 1440ignore-no-cache > ignore-no-store override-expire override-lastmod > refresh_pattern . 0 20% 4320ignore-no-cache > ignore-no-store override-expire override-lastmod > > Despite even the drastic attempts, the access.log still sees > "TCP_CLIENT_REFRESH_MISS/200" > > A sniffer shows that the client is emmitting "Cache-Control: no-cache" > and "Pragma: no-cache" headers fir the request, and the server is > emitting a "Cache-Control: private" header. > > Nick
Re: header_access directive in Squid3/HEAD
On Fri, 2008-09-12 at 12:44 -0500, Ritter, Nicholas wrote: > Was the "header_access" directive/functionality removed or otherwise > broken in Squid3/HEAD? > > I am using squid-3.HEAD-20080910 with the following config lines: > > acl appsds_ctl url_regex -i ^http://apps.ds.atv/ > request_header_access Cache-Control deny appsds_ctl > header_access Cache-Control deny appsds_ctl > header_replace Cache-Control public, s-maxage=14400, max-age=14400 > > > And a "squid -k parse" gives me the following error: > > 2008/09/12 12:44:21| Processing Configuration File: > /usr/local/squidapp/etc/squid.conf (depth 0) > 2008/09/12 12:44:21| cache_cf.cc(360) parseOneConfigFile: squid.conf:473 > unrecognized: 'header_access' There is no such directive as header_access. There are request_header_access and reply_header_access (provided you have not disabled HTTP violations during ./configure). HTH, Alex.
header_access directive in Squid3/HEAD
Was the "header_access" directive/functionality removed or otherwise broken in Squid3/HEAD? I am using squid-3.HEAD-20080910 with the following config lines: acl appsds_ctl url_regex -i ^http://apps.ds.atv/ request_header_access Cache-Control deny appsds_ctl header_access Cache-Control deny appsds_ctl header_replace Cache-Control public, s-maxage=14400, max-age=14400 And a "squid -k parse" gives me the following error: 2008/09/12 12:44:21| Processing Configuration File: /usr/local/squidapp/etc/squid.conf (depth 0) 2008/09/12 12:44:21| cache_cf.cc(360) parseOneConfigFile: squid.conf:473 unrecognized: 'header_access' -Nick
Re: Cleanup patches to follow
On Wed, 2008-09-10 at 23:42 -0600, Alex Rousskov wrote: > If the code is committed, I will go through affected bug reports (that I > know of) and update or close them. Done. If I missed any affected bugs, please point me to them. > Finally, I am aware of one unreported(?) problem related to the shutdown > sequence. I have added an XXX to main.cc already but will discuss that > problem and a fix separately, tomorrow. The bug may lead to assertions > at the very end of the shutdown sequence, but does not affect the > "runtime" code. The problem and a fix sketch have been filed as http://www.squid-cache.org/bugs/show_bug.cgi?id=2460 Alex.
[MERGE] WCCPv2 Config Cleanup
WCCP config cleanups mk2 - clean of the additional extras slipped in last time. - corrected according to Adrians updated info Amos # Bazaar merge directive format 2 (Bazaar 0.90) # revision_id: [EMAIL PROTECTED] # target_branch: file:///src/squid/bzr/trunk/ # testament_sha1: a498a9a51a706bf7632ed8248d8ccf8f44f5684f # timestamp: 2008-09-12 20:15:11 +1200 # base_revision_id: [EMAIL PROTECTED] # bmvbzbywqmjrun2v # # Begin patch === modified file 'src/cf.data.depend' --- src/cf.data.depend 2008-04-03 05:31:29 + +++ src/cf.data.depend 2008-09-12 08:14:32 + @@ -47,6 +47,8 @@ tristate uri_whitespace ushort +wccp2_method +wccp2_amethod wccp2_service wccp2_service_info wordlist === modified file 'src/cf.data.pre' --- src/cf.data.pre 2008-09-03 13:22:55 + +++ src/cf.data.pre 2008-09-12 01:14:37 + @@ -3957,7 +3957,17 @@ LOC: Config.Wccp.router DEFAULT: 0.0.0.0 IFDEF: USE_WCCP -DOC_NONE +DOC_START + Use this option to define your WCCP ``home'' router for + Squid. + + wccp_router supports a single WCCP(v1) router + + wccp2_router supports multiple WCCPv2 routers + + only one of the two may be used at the same time and defines + which version of WCCP to use. +DOC_END NAME: wccp2_router TYPE: IPAddress_list @@ -4005,33 +4015,33 @@ DOC_END NAME: wccp2_forwarding_method -TYPE: int +TYPE: wccp2_method LOC: Config.Wccp2.forwarding_method -DEFAULT: 1 +DEFAULT: gre IFDEF: USE_WCCPv2 DOC_START WCCP2 allows the setting of forwarding methods between the router/switch and the cache. Valid values are as follows: - 1 - GRE encapsulation (forward the packet in a GRE/WCCP tunnel) - 2 - L2 redirect (forward the packet using Layer 2/MAC rewriting) + gre - GRE encapsulation (forward the packet in a GRE/WCCP tunnel) + l2 - L2 redirect (forward the packet using Layer 2/MAC rewriting) Currently (as of IOS 12.4) cisco routers only support GRE. Cisco switches only support the L2 redirect assignment method. DOC_END NAME: wccp2_return_method -TYPE: int +TYPE: wccp2_method LOC: Config.Wccp2.return_method -DEFAULT: 1 +DEFAULT: gre IFDEF: USE_WCCPv2 DOC_START WCCP2 allows the setting of return methods between the router/switch and the cache for packets that the cache decides not to handle. Valid values are as follows: - 1 - GRE encapsulation (forward the packet in a GRE/WCCP tunnel) - 2 - L2 redirect (forward the packet using Layer 2/MAC rewriting) + gre - GRE encapsulation (forward the packet in a GRE/WCCP tunnel) + l2 - L2 redirect (forward the packet using Layer 2/MAC rewriting) Currently (as of IOS 12.4) cisco routers only support GRE. Cisco switches only support the L2 redirect assignment. @@ -4043,16 +4053,16 @@ DOC_END NAME: wccp2_assignment_method -TYPE: int +TYPE: wccp2_amethod LOC: Config.Wccp2.assignment_method -DEFAULT: 1 +DEFAULT: hash IFDEF: USE_WCCPv2 DOC_START WCCP2 allows the setting of methods to assign the WCCP hash Valid values are as follows: - 1 - Hash assignment - 2 - Mask assignment + hash - Hash assignment + mask - Mask assignment As a general rule, cisco routers support the hash assignment method and cisco switches support the mask assignment method. === modified file 'src/protos.h' --- src/protos.h2008-09-03 04:40:31 + +++ src/protos.h2008-09-12 01:14:37 + @@ -813,10 +813,16 @@ SQUIDCEXTERN char *strtokFile(void); #if USE_WCCPv2 + +SQUIDCEXTERN void parse_wccp2_method(int *v); +SQUIDCEXTERN void free_wccp2_method(int *v); +SQUIDCEXTERN void dump_wccp2_method(StoreEntry * e, const char *label, int v); +SQUIDCEXTERN void parse_wccp2_amethod(int *v); +SQUIDCEXTERN void free_wccp2_amethod(int *v); +SQUIDCEXTERN void dump_wccp2_amethod(StoreEntry * e, const char *label, int v); + SQUIDCEXTERN void parse_wccp2_service(void *v); - SQUIDCEXTERN void free_wccp2_service(void *v); - SQUIDCEXTERN void dump_wccp2_service(StoreEntry * e, const char *label, void *v); SQUIDCEXTERN int check_null_wccp2_service(void *v); === modified file 'src/wccp2.cc' --- src/wccp2.cc2008-08-13 03:24:57 + +++ src/wccp2.cc2008-09-12 08:14:32 + @@ -109,14 +109,17 @@ #define WCCP2_CAPABILITY_ASSIGNMENT_METHOD 0x02 #define WCCP2_CAPABILITY_RETURN_METHOD 0x03 -#define WCCP2_FORWARDING_METHOD_GRE0x0001 -#define WCCP2_FORWARDING_METHOD_L2 0x0002 - -#define WCCP2_ASSIGNMENT_METHOD_HASH 0x0001 -#define WCCP2_ASSIGNMENT_METHOD_MASK 0x0002 - -#define WCCP2_PACKET_RETURN_METHOD_GRE 0x0001 -#define WCCP2_PACKET_RETURN_METHOD_L2 0x0002 +#define WCCP2_METHOD_GRE 0x0001 +#define WCCP2_METHOD_L20x0002 + +#define WCCP2_FORWARDING_METHOD_GREWCCP2_METHOD_GRE +#define WCCP2
