Re: [PATCH] Support client connection annotation by helpers via clt_conn_id=ID
A new patch.
Changes:
- clt_conn_id renamed to clt_conn_tag
- Amos requested fixes.
I hope it is OK.
Regards,
Christos
On 06/22/2014 08:43 PM, Tsantilas Christos wrote:
Hi all,
The attached patch replaces existin annotation values with the new one
received from helpers.
Just one question. We are documenting key-value pairs in cf.data.pre
only for url-rewriter helpers, but annotations works for all squid helpers.
Should we move the related url-rewriter section to a global section? If
yes where?
For example something like the following in a global section should be
enough:
The interface for all helpers has been extended to support arbitrary
lists of key=value pairs, with the syntax key=value . Some keys have
special meaning to Squid, as documented here.
Currently Squid understands the following optional key=value pairs
received from URL rewriters:
clt_conn_id=ID
Associates the received ID with the client TCP connection.
The clt_conn_id=ID pair is treated as a regular annotation but
it persists across all transactions on the client connection
rather than disappearing after the current request. A helper
may update the client connection ID value during subsequent
requests by returning a new ID value. To send the connection
ID to the URL rewriter, use url_rewrite_extras:
url_rewrite_extras clt_conn_id=%{clt_conn_id}note ...
On 06/19/2014 09:07 PM, Tsantilas Christos wrote:
On 06/16/2014 06:36 PM, Alex Rousskov wrote:
On 06/15/2014 12:07 AM, Amos Jeffries wrote:
On 15/06/2014 4:58 a.m., Alex Rousskov wrote:
On 06/11/2014 08:52 AM, Tsantilas Christos wrote:
I must also note that this patch adds an inconsistency. All
annotation
key=values pairs received from helpers, accumulated to the
existing key
notes values. The clt_conn_id=Id pair is always unique and replaces
the
existing clt_conn_id=Id annotation pair.
We may want to make all annotations unique, or maybe implement a
configuration mechanism to define which annotations are overwriting
their previous values and which appending the new values.
I suggest making all annotations unique (i.e., new values overwrite
old
ones) because helpers that want to accumulate annotation values can do
that by returning old values along with new ones:
received by helper: name=v1
returned by helper: name=v1,v2
Please note that the opposite does not work: If annotation values are
always accumulated, a helper cannot overwrite/remove the old value.
Doing that would mean passing all existing annotations to every helper
lookup.
Why would that mean the above?
AFAICT, the helper should get only the annotations it needs. That need
is helper-specific and, hence, is configurable via the various _extras
and equivalent directives. That is already supported and does not need
to change.
Here is the overall sketch for supporting unique annotations:
1. Send the helper the annotations it is configured to get
(no changes here).
2. For each unique annotation key received from the helper,
remove any old annotation(s) with the same key.
3. Store annotations received from the helper
(no changes here).
To support explicit annotation deletion, we can adjust #3 to skip
key-value pairs with the value equal to '-'.
If there is not any objection I will implement this scenario.
Looks that this approach is the best and cover more cases than the
accumulated Notes values.
If someones need to accumulate Note values he can configure squid to
send old note value to helper and helper include it in its response.
This is simple.
If required in the future we can implement a configuration parameter
which configures one or more notes as always accumulated. For example:
AccumulateHelperNotes status message
HTH,
Alex.
Support client connection annotation by helpers via clt_conn_tag=TAG.
TCP client connections tagging is useful for faking various forms of
connection-based authentication when standard HTTP authentication cannot be
used. A URL rewriter or, external ACL helper may mark the authenticated
client connection to avoid going through authentication steps during
subsequent requests on the same connection and to share connection
authentication information with Squid ACLs, other helpers, and logs.
After this change, Squid accepts optional clt_conn_tag=TAG pair from a
helper and associates the received TAG with the client TCP connection.
Squid treats the received clt_conn_tag=TAG pair as a regular annotation, but
also keeps it across all requests on the same client connection. A helper may
update the client connection TAG value during subsequent requests.
Also after this patch the notes comming from helpers replaces any existing
note values.
This is a Measurement Factory project
=== modified file 'src/Notes.cc'
--- src/Notes.cc 2014-04-30 09:41:25 +
+++ src/Notes.cc 2014-06-26 10:55:50 +
@@ -14,40 +14,41 @@
* This program is free software; you can redistribute it and/or
[RFC] changeless reconfigure principles
Hello, In most cases, admins reconfigure Squid to change a small portion of the configuration while the majority of squid.conf and Squid functionality stays the same. Yet, Squid dutifully shuts almost everything down, configures modules/features almost from scratch, and then starts them up. This shutdown-configure-start behavior creates user-visible disruptions, complicates code, increases the chance of memory leaks, leads to memory overuse, may hurt caching, spoil statistics, etc. We cannot solve this problem immediately, but we can improve old features as we go and minimize mistakes in the new code. I would like to propose the following four design principles to guide us long-term as we work on individual modules/features or add new ones: P0: Reconfiguration is not meant and will never be able to replace a full Squid shutdown and start. Some residual effects from the initial configuration are likely to persist. Some Squid features/modules/parts may never support reconfiguration. Some subtle/hidden relationships between features may not be fully tracked during reconfiguration. Our goal is to maximize the number of properly supported reconfiguration changes while minimizing reconfiguration-caused problems/disruptions. It is a complex balancing act. P1: If its configuration has not changed, the feature/module does not have to do anything in reaction to reconfigure. It may keep running and functioning as it was. It does not have to shut down and restart. For example, if configuration affecting all aspects of a listening port has not changed at all, Squid does not have to stop listening and close the corresponding socket (even if there are probably some use cases that benefit from Squid doing so). P2: Each feature/module has the right to determine what constitutes its configuration in P1. Even though nearly every Squid feature may have an indirect effect on other Squid features, we have to draw lines somewhere (per P0). The important thing here is consistent, predictable behavior, not 100% coverage graph of all possible inter-feature dependencies. P3. By default, if a feature/module claims that its configuration consists of N direct and related squid.conf settings (S1 ... SN), then the feature should consider its configuration unchanged if * each Si value has not changed AND * for each Si that the feature knows to be a name of a local file containing additional configuration input (e.g., a file containing an SSL certificate), the last modification timestamp of that file has not changed. P4. Any feature/module decision about its configuration changes can be overwritten by an admin. In other words, the admin has the right to force any feature to assume that its configuration has changed. How this is done and whether this is done for all features at once or selectively are separate questions not discussed here for the sake of focus. Do we agree on the above? Needless to say, the vast majority of current modules/features do not follow the above principles today. That is OK. If we agree on these principles, folks can start adjusting the existing features to meet them and ensure that newly proposed features comply when feasible. Any decision would be specific to the feature, of course, but having these principles helps. Furthermore, if we agree, we can add a simple, general API to implement P3 (and P4) logic for any feature that wants to use it without forcing other features to make changes. This will make it much easier to support these principles. Thank you, Alex. P.S. The proposed principles are mostly orthogonal (but can be added) to the hot reconfiguration problems currently discussed at http://wiki.squid-cache.org/Features/HotConf
Re: FATAL: Ipc::Mem::Segment::create failed to shm_open(/squid-cf__metadata.shm): (13) Permission denied
Appears to be a SELinux issue. -- View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/FATAL-Ipc-Mem-Segment-create-failed-to-shm-open-squid-cf-metadata-shm-13-Permission-denied-tp4665373p4666501.html Sent from the Squid - Development mailing list archive at Nabble.com.
