Re: Question ICAP-client

[Henrik Nordstrom]

fre 2007-03-23 klockan 00:07 +0200 skrev Tsantilas Christos:

> Moreover we must not be absolutely sure that the icap servers will
> always be located at the same site as the proxy. Maybe in the future
> some icap-services will be public available for use by everyone..

Which reminds me why there is a login= option to cache_peer.

> The ICAP rfc something says about hop-by-hop headers. Now squid always
> forwards the Proxy-Authorization header for example to the icap server ...

And very many ICAP servers depend on these headers being forwarded,
lacking support for an official standard method of forwarding the
authenticated credentials.

There is even ICAP servers depending on being the entity processing the
Proxy-Authorization header itself with it's own notion of user database,
which is a bit of a mess...

I completely buy both sides of this discussion. But unlike peers the
boundary is not so clearly defined here so there needs to be some
differentiation in trust.

- Trusted to process user credentials (proxy and www)
- Trusted to rewrite requests, or only filter.


Regards
Henrik


signature.asc
Description: Detta är en digitalt signerad	meddelandedel

Re: Question ICAP-client

[Alex Rousskov]

On Fri, 2007-03-23 at 00:07 +0200, Tsantilas Christos wrote:

> Moreover we must not be absolutely sure that the icap servers will
> always be located at the same site as the proxy. Maybe in the future
> some icap-services will be public available for use by everyone..

That future is pretty much here. I know of at least two companies that
want to deploy ICAP servers that way. IMHO, it is only a matter of time
when many ISPs and others will feel as comfortable with remote ICAP
server as busy web site owners feel comfortable with CDNs now.

> The ICAP rfc something says about hop-by-hop headers. Now squid always
> forwards the Proxy-Authorization header for example to the icap server ...

I will file a bug report.

Thank you,

Alex.

Re: Question ICAP-client

[Tsantilas Christos]

Alex Rousskov wrote:
> On Thu, 2007-03-22 at 16:26 +0100, Kinkie wrote:
> 
>> In this regard I see the ICAP server not to be any different from a
>> proxy server, of which it is simply an extension.
> 
> Whether the trust boundary includes both the proxy and the ICAP server
> depends on the setup. Being an "extension" is not always the same as
> being a "trusted extension". And there may be several trust categories
> involved.
> 

Moreover we must not be absolutely sure that the icap servers will
always be located at the same site as the proxy. Maybe in the future
some icap-services will be public available for use by everyone..

> P.S. Still, "sending all the information that the proxy server has to
> the ICAP server" is similar to sending all that information to another
> proxy server: Sometimes it is appropriate, sometimes it is not. The
> patch, however, does not affect what information is sent to the ICAP
> server.

The ICAP rfc something says about hop-by-hop headers. Now squid always
forwards the Proxy-Authorization header for example to the icap server ...


Regards,
 Christos

Re: Question ICAP-client

[Alex Rousskov]

On Thu, 2007-03-22 at 16:26 +0100, Kinkie wrote:

> In this regard I see the ICAP server not to be any different from a
> proxy server, of which it is simply an extension.

Whether the trust boundary includes both the proxy and the ICAP server
depends on the setup. Being an "extension" is not always the same as
being a "trusted extension". And there may be several trust categories
involved.

> I just fail to see any
> added security in not sending all the information that the proxy server
> has to the ICAP server.

As I have tried to clarify, the problem we are discussing on this thread
(and the problem that the now-committed patch works on) is _not_ about
sending information to the ICAP server, but about treating requests
generated by the ICAP server as if they were authenticated by the
client.

$0.02,

Alex.
P.S. Still, "sending all the information that the proxy server has to
the ICAP server" is similar to sending all that information to another
proxy server: Sometimes it is appropriate, sometimes it is not. The
patch, however, does not affect what information is sent to the ICAP
server.

Re: Question ICAP-client

[Kinkie]

On Wed, 2007-03-21 at 18:04 +0100, Stefan Bischof wrote:
> > I am sure we will eventually see compromised or otherwise unfriendly
> > ICAP servers that do nasty things. Such servers would love to do
> nasty
> > things "on behalf" of a client, using client identity if possible.
> Thus,
> > I have a problem with blindly copying sensitive client information
> into
> > requests generated (originated from) the ICAP server. 

Sorry, but I don't see your point.
An user who doesn't encrypt traffic end-to-end implicitly trusts all
nodes that interconnect her to the origin server not to mess things up
relatively to the sensitivity of the exchanged information. This means
her PC, the firewall, the proxy, network switches, origin servers and
content providers and so on.
Any node along the chain could theoretically hijack traffic and steal
her identity or perform actions on her behalf.
In this regard I see the ICAP server not to be any different from a
proxy server, of which it is simply an extension. I just fail to see any
added security in not sending all the information that the proxy server
has to the ICAP server.

-- 
Kinkie <[EMAIL PROTECTED]>

Re: Question ICAP-client

[Alex Rousskov]

On Wed, 2007-03-21 at 18:04 +0100, Stefan Bischof wrote:
> I don't see your point (probably I don't understood something). The 
> ICAP-server already knows the clients username at this point, because of 
> the REQMOD request. If the evil ICAP-server redirects the request to a 
> evil HTTP-server, it wouldn't be a problem to send the username to the 
> HTTP-server and then back to the ICAP-server via RESPMOD.

This is not about transmitting a username to the ICAP server. It is
about making Squid believe that the request that originated from the
ICAP server is authenticated (with _client_ credentials).

An ICAP server can do pretty much anything. However, whether the ICAP
server can do something _while being authenticated as the client_ is up
to the proxy. The current code grants such authenticated status to
requests generated by the ICAP server.

I agree that a client and an admin already have to trust the proxy. The
question is whether the client is authenticating herself with the proxy
under the assumption that her authenticated requests will not be altered
substantially. For example, there may be a big difference between an
anonymous employee surfing porn and a specific/authenticated employee
surfing porn... The admin may have reasonable trust in Squid or another
well-known proxy, but may not have the same level of trust in some
obscure porn-blocking ICAP server being installed and maintained by a
3rd party.


A complete solution to this problem is probably too big to implement in
the foreseeable future. However, it would be nice to have a special mode
in Squid where ICAP request alternations are not allowed (but request
satisfaction and REQMOD are OK). This should take care of most problems
in this context.

Full support of such a safe no-request-alternation mode is not easy
because many ICAP servers use "ICAP 200 OK" to return unmodified
requests. However, we can limit the check to comparing request headers
and ignoring POST/PUT issues.

Another useful "safe ICAP" mode could prevent any adaptations (but allow
blocking), with a "how to detect a blocking message?" problem.

$0.02,

Alex.

Re: Question ICAP-client

[Stefan Bischof]

Hi folks!

Alex Rousskov wrote:

On Sat, 2007-03-10 at 16:00 +0200, Tsantilas Christos wrote:

  

I think that client address/port and squid address/port must copied.
They can not (and must not) changed by an ICAP server.
The same about authentication information  because referred to users
authenticated on squid and this info must not changed by an ICAP server.




We are looking at this from different angles. You perceive an ICAP
server as an extension of a client: a client should have no problem with
any adaptation of its request by such as friendly server.

I am sure we will eventually see compromised or otherwise unfriendly
ICAP servers that do nasty things. Such servers would love to do nasty
things "on behalf" of a client, using client identity if possible. Thus,
I have a problem with blindly copying sensitive client information into
requests generated (originated from) the ICAP server.
  
I don't see your point (probably I don't understood something). The 
ICAP-server already knows the clients username at this point, because of 
the REQMOD request. If the evil ICAP-server redirects the request to a 
evil HTTP-server, it wouldn't be a problem to send the username to the 
HTTP-server and then back to the ICAP-server via RESPMOD.



As an ICAP server developer and seller, I know that many proxy
installations today use 3rd party ICAP servers, and many proxy admins
have very little knowledge about what that ICAP server is actually
doing. Add remote updates to ICAP server software, and you have a
perfect location for a man-in-the-middle attack point.
  
Yes that's right. But, also without transmitting the username in the 
RESPMOD request, this is the perfect location for such a attack. I don't 
see the username as a piece of sensitive information, since it only 
makes sense in the domain of the proxyserver. I understand that the 
username shouldn't leave the proxy domain. But to avoid that, you would 
have to block the username transmission via REQMOD.



What I want to say is in essence: In my opinion the transmission of the 
username in a RESPMOD request doesn't raise the security problems of 
ICAP significantly. Also without transferring the username to the ICAP 
server at all, a man-in-the-middle attack is absolutely possible (and 
supposably not much harder to achieve). I don't see a big advantage of 
getting the username in both requests from an attackers point of view.
I know this doesn't sound very positive. But at last you have to trust 
the ICAP-server as you have to trust your browser, your OS, your 
provider, ..., even your proxy :-).
And altough this sounds a bit idealistic: it's an administrators job to 
know what's going on on his systems.


Please tell me, what you think about that.


Not coping addresses and auth info, we are breaking the ACL checking in
later steps (e.g in FwdState::fwdStart).



>From your point of view, the solution is to copy. From my point of view,
this breakage is a sign that we should be extra careful with copying
such information. That is, there may be a _good_ reason why ACLs do not
"just work" for requests initiated by the ICAP server.


[...]


Stefan, please test and file a bug report if something does not work
right after this change.
  

When I have the time I will (not until end of march).

best regards and thanks for examining my problem!!

Stefan Bischof

Re: Question ICAP-client

[Alex Rousskov]

On Sat, 2007-03-10 at 16:00 +0200, Tsantilas Christos wrote:

> I think that client address/port and squid address/port must copied.
> They can not (and must not) changed by an ICAP server.
> The same about authentication information  because referred to users
> authenticated on squid and this info must not changed by an ICAP server.


We are looking at this from different angles. You perceive an ICAP
server as an extension of a client: a client should have no problem with
any adaptation of its request by such as friendly server.

I am sure we will eventually see compromised or otherwise unfriendly
ICAP servers that do nasty things. Such servers would love to do nasty
things "on behalf" of a client, using client identity if possible. Thus,
I have a problem with blindly copying sensitive client information into
requests generated (originated from) the ICAP server.

As an ICAP server developer and seller, I know that many proxy
installations today use 3rd party ICAP servers, and many proxy admins
have very little knowledge about what that ICAP server is actually
doing. Add remote updates to ICAP server software, and you have a
perfect location for a man-in-the-middle attack point.

> Not coping addresses and auth info, we are breaking the ACL checking in
> later steps (e.g in FwdState::fwdStart).

>From your point of view, the solution is to copy. From my point of view,
this breakage is a sign that we should be extra careful with copying
such information. That is, there may be a _good_ reason why ACLs do not
"just work" for requests initiated by the ICAP server.


> Also I think, there is not any problem with HttpRequest.flags. A small
> research about the flags show me that only the flags.redirected,
> flags.accelerated, flags.transparent, flags.internal,
> flags.internalclient are computed before the ICAP reqmod called. I do
> not think that any of this flags can modified by an ICAP
> server. The other flags set after the ICAP reqmod. (I kept a list with
> flags and section code they set)

I was concerned about this too. Thank you for doing this research! To
increase chances of preserving the above conclusion, I added a
request_flags::cloneAdaptationImmune() method and a comment. Hopefully,
when new flags are added, folks will read the comment and adjust
cloneAdaptationImmune() if needed.

> > I have no problems with committing a fix that always copies this
> > information as a temporary measure, but I think we need to at least put
> > some comments in the code that we are copying user-specific information
> > to a request that did not originate from the user.
> > 
> I think it not so problem, the new request can appeared as originated by
> the same user and the same IP as the original request...

Yes, the request will appear as if it was originated from the user.
Whether that is a problem for requests that actually originated from the
ICAP server depends on the environment.

Christos, I have applied your patch, with minor modifications, to
squid3-icap. Thanks a lot for writing the patch!

Stefan, please test and file a bug report if something does not work
right after this change.

Alex.

Re: Question ICAP-client

[Tsantilas Christos]

Hi Alex,

Alex Rousskov wrote:
> On Wed, 2007-03-07 at 23:57 +0200, Tsantilas Christos wrote:
> 
>> When an http request adapted using ICAP then the client and server
>> addresses and the authentication information does not copied to adapted
>> request.
>> This is will cause problems in any following access control lists
>> proccessing.
> 
> Should this copying be configurable? In some environments, an adapted
> request is not a request "authorized" by the user. Can we always trust
> the ICAP server to essentially impersonate the user?

I think that client address/port and squid address/port must copied.
They can not (and must not) changed by an ICAP server.
The same about authentication information  because referred to users
authenticated on squid and this info must not changed by an ICAP server.
Not coping addresses and auth info, we are breaking the ACL checking in
later steps (e.g in FwdState::fwdStart).

Also I think, there is not any problem with HttpRequest.flags. A small
research about the flags show me that only the flags.redirected,
flags.accelerated, flags.transparent, flags.internal,
flags.internalclient are computed before the ICAP reqmod called. I do
not think that any of this flags can modified by an ICAP
server. The other flags set after the ICAP reqmod. (I kept a list with
flags and section code they set)

> 
> I have no problems with committing a fix that always copies this
> information as a temporary measure, but I think we need to at least put
> some comments in the code that we are copying user-specific information
> to a request that did not originate from the user.
> 
I think it not so problem, the new request can appeared as originated by
the same user and the same IP as the original request...

> In the ICAP 204 case, we should probably always copy (and eventually we
> will be able simply reuse the same request object). For ICAP 200, we
> should be more careful and at least put a TODO comment. If we want to
> distinguish these two cases, then ICAPModXact should do the copying and
> not client_side.
> 
> If you want to move the copying code to ICAPModXact, you can call it
> from handle204NoContent() unconditionally and perhaps from
> parseHttpHead() with a "TODO: make this copying configurable" comment.
> 
Yep, the copying code must placed in ICAPModXact.
I am including a new patch. If you think it can be applied please add
any comment or do any change you think needed

Index: ICAPModXact.cc
===
RCS file: /cvsroot/squid/squid3/src/ICAP/ICAPModXact.cc,v
retrieving revision 1.1.2.22
diff -u -r1.1.2.22 ICAPModXact.cc
--- ICAPModXact.cc  20 Feb 2007 16:22:30 -  1.1.2.22
+++ ICAPModXact.cc  10 Mar 2007 11:52:08 -
@@ -551,7 +551,6 @@
 {
 if (adapted.header) // already allocated
 return;
-
 if (gotEncapsulated("res-hdr")) {
 adapted.setHeader(new HttpReply);
 } else if (gotEncapsulated("req-hdr")) {
@@ -701,6 +700,14 @@
 if (const HttpRequest *oldR = dynamic_cast(oldHead)) {
 HttpRequest *newR = new HttpRequest;
 newR->client_addr = oldR->client_addr;
+   newR->client_port = oldR->client_port;
+   newR->my_addr = oldR->my_addr;
+   newR->my_port = oldR->my_port;
+   newR->flags = oldR->flags;
+   if (oldR->auth_user_request) {
+newR->auth_user_request = oldR->auth_user_request;
+newR->auth_user_request->lock();
+   }
 newHead = newR;
 } else
 if (dynamic_cast(oldHead))
@@ -758,6 +765,21 @@

 if (!parseHead(adapted.header))
 return; // need more header data
+
+   if (HttpRequest *newHead =
dynamic_cast(adapted.header)) {
+   const HttpRequest *oldR = dynamic_cast(virgin.header);
+   Must(oldR);
+   newHead->client_addr = oldR->client_addr;
+   newHead->client_port = oldR->client_port;
+   newHead->my_addr = oldR->my_addr;
+   newHead->my_port = oldR->my_port;
+   newHead->flags = oldR->flags;
+   if (oldR->auth_user_request) {
+   newHead->auth_user_request = oldR->auth_user_request;
+   newHead->auth_user_request->lock();
+   }
+   }
+
 }

 decideOnParsingBody();

Re: Question ICAP-client

[Alex Rousskov]

On Wed, 2007-03-07 at 23:57 +0200, Tsantilas Christos wrote:

> When an http request adapted using ICAP then the client and server
> addresses and the authentication information does not copied to adapted
> request.
> This is will cause problems in any following access control lists
> proccessing.

Should this copying be configurable? In some environments, an adapted
request is not a request "authorized" by the user. Can we always trust
the ICAP server to essentially impersonate the user?

I have no problems with committing a fix that always copies this
information as a temporary measure, but I think we need to at least put
some comments in the code that we are copying user-specific information
to a request that did not originate from the user.

In the ICAP 204 case, we should probably always copy (and eventually we
will be able simply reuse the same request object). For ICAP 200, we
should be more careful and at least put a TODO comment. If we want to
distinguish these two cases, then ICAPModXact should do the copying and
not client_side.

If you want to move the copying code to ICAPModXact, you can call it
from handle204NoContent() unconditionally and perhaps from
parseHttpHead() with a "TODO: make this copying configurable" comment.

I would be happy to commit this code one you think it is ready.

HTH,

Alex.


> Looks that the following patch solves the problem. (But I am to tired
> now for a good testing, tomorrow .)
> 
> Regards,
> Christos
> 
> diff -u -r1.34.4.24 client_side_request.cc
> --- client_side_request.cc  14 Feb 2007 07:19:43 -  1.34.4.24
> +++ client_side_request.cc  7 Mar 2007 21:45:17 -
> @@ -1112,6 +1112,15 @@
>  assert(msg);
> 
>  if (HttpRequest *new_req = dynamic_cast(msg)) {
> +   new_req->client_addr = request->client_addr;
> +new_req->client_port = request->client_port;
> +new_req->my_addr = request->my_addr;
> +new_req->my_port = request->my_port;
> +new_req->flags = request->flags;
> +   if (request->auth_user_request) {
> +new_req->auth_user_request = request->auth_user_request;
> +new_req->auth_user_request->lock();
> +}
>  /*
>   * Replace the old request with the new request.
>   */

Re: Question ICAP-client

[Tsantilas Christos]

When an http request adapted using ICAP then the client and server
addresses and the authentication information does not copied to adapted
request.
This is will cause problems in any following access control lists
proccessing.

Looks that the following patch solves the problem. (But I am to tired
now for a good testing, tomorrow .)

Regards,
Christos

diff -u -r1.34.4.24 client_side_request.cc
--- client_side_request.cc  14 Feb 2007 07:19:43 -  1.34.4.24
+++ client_side_request.cc  7 Mar 2007 21:45:17 -
@@ -1112,6 +1112,15 @@
 assert(msg);

 if (HttpRequest *new_req = dynamic_cast(msg)) {
+   new_req->client_addr = request->client_addr;
+new_req->client_port = request->client_port;
+new_req->my_addr = request->my_addr;
+new_req->my_port = request->my_port;
+new_req->flags = request->flags;
+   if (request->auth_user_request) {
+new_req->auth_user_request = request->auth_user_request;
+new_req->auth_user_request->lock();
+}
 /*
  * Replace the old request with the new request.
  */

Re: Question ICAP-client

[Tsantilas Christos]

Hi Stephan,

Stefan Bischof wrote:
> 
> First I implemented
> http://www.i-cap.org/spec/draft-stecher-icap-subid-00.txt
>  by sending
> X-Include: X-Authenticated-User
> in my OPTIONS response. (I don't know if this draft is really
> implemented, but after a little googlin, I tought so ... Please tell me
> whether it is implemented in the actual 3.0 or not.) Squid didn't send
> me the desired header field (X-Authenticated-User) in the next requests.
> 

It is normal. squid3 does not parse the X-Include header .

> OK, after that i enabled icap_send_client_username in squid.conf (The
> only thing I need is in fact the username, .
> 
This is the right way (also look the icap_client_username_header
parameter.).


>. But in RESPMOD
> requests the username is still missing (not only my application was
> complaining, I used tcpdump to verify this).
>

Yes you are right. There is a bug here ...

Regards,
 Christos