Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?

[Yann Ylavic]

Hi Jim,

disclaimer: I'm not an NTLM expert...

On Tue, Nov 3, 2015 at 3:04 PM, o haya  wrote:
>
> I think that other info that binds to the authenticated user is at the 
> SharePoint application level (maybe persisted in the app on the client side) 
> because it doesn't look like any cookies or headers that are user-specific 
> get sent to SharePoint after the user is authenticated.

Not even the "Authentication: NTLM ..." header?

>
> So, in other words, once the user is authenticated via NTLM, it looks like it 
> doesn't matter which connection any subsequent requests get sent over, since 
> the session info is probably carried in the requests themselves somewhere 
> (SharePoint 2010+ uses an internal STS, and the client-side app gets a token 
> from the STS at some point and then persists it and sends that token along 
> with requests to SharePoint).

No idea where that authenticator/token may be...

I suspect that once the user is authenticated on a connection, that
connection is assumed to be owned by that same user and hence
everything that comes in is the user session (there may be a check on
the authenticator or token if available for each request, so that a
renegotiation is asked on mismatch, but still multiple users can't
share the same connection AFAICT).

What happens if you use a threaded MPM (and start httpd with -X to
leverage the multiplexing, as suggested in a previous message), or if
you use another httpd (likewise threaded and started with -X) in front
of your current proxy?
The goal here is to have something in the chain that multiplexes
requests on a the same connection before the Sharepoint, and hence see
if the sharepoint behaves correctly in this case (I doubt so,
otherwise you wouldn't need any particular httpd configuration or
patch).
Once the first user is NTLM-authenticated with a first connection to
the frontend/multiplexer, open a second connection with no NTLM
authentication (simple HTTP request), and then open a third one for
another user (asking for his/her own NTLM authentication). Make sure
the frontend reuse the same outgoing connection each time.
Then you will see if the sharepoint:
1. is able to handle multiple users on the same connection, or
2. asks for renegotiation (in both cases), or
3. reuses the first session for successives requests (for one and/or
the other case).
Except 1., the users' sessions will break (and even be compromised in
3.), hence you'll need to make sure there is no multiplexer in the
chain (SSL termination on the frontend/proxy, and direct connection to
the backend -- no load-balancer or things like that).

>
> Anyway, that is what I think, as far as the importance of being on the same 
> connection.  Does that make sense?

This requires a testing IMHO...

Regards,
Yann.

-
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org

Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?

[Eric Covener]

On Tue, Oct 27, 2015 at 7:07 PM, o haya  wrote:
> Sorry, but this time, I'm not quite sure what (which aspect of the 
> discussion) you're referring to?

Sorry, was top-posting from my phone.  It's about why  or
ProxySet without anything else specified enabled persistent
connectons. If you don't use ProxyPass and use e.g. RewriteRUle .. [P]
a generic backend "worker" is used -- unless you give httpd that extra
hint.

-
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org

Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?

[Yann Ylavic]

Hi Jim,

On Tue, Oct 27, 2015 at 1:57 AM, o haya  wrote:
>
> First of all, as a kind of an aside remark (sorry for the "pun" :)), from my 
> testing, it appears that if I have "ProxySet keepalive=On" inside a 
> , then the requests to the backend all have "Connection: 
> Keep-Alive" in the requests headers going to the backend server (a SharePoint 
> server).  Conversely if "ProxySet keepalive=Off" is inside the 
> ..., the HTTP requests to the backend have HTTP request header 
> "Connection: closed".  In other words, the "ProxySet keepalive=On/Off" 
> appears to be able to control whether a "Connection: keep-alive" vs. 
> "Connection: closed" gets sent in a HTP request header to the backend.

That's really weird, I can't see anything in the code that can provoke this.
"ProxySet keepalive=On" really only issues a setsockopt(SO_KEEPALIVE,
on) for the backend socket, whereas HTTP keepalive (Connection:
keep-alive/close") is rather controlled by "ProxySet disablereuse=On"
or SetEnv's like force-proxy-request-1.0 and proxy-nokeepalive.
Will test this because it would be an unexpected behaviour (given that
keepalive=off is the default)...

>
> Next:  I think I kind of understood about the proxy-initial-not-pooled 
> setting ==> a new connection from the client always connected to the backend 
> via a new Apache-to-backend connection, but I didn't realize that NTLM meant 
> that all the requests SUBSEQUENT to the NTLM authentication had to ALSO go to 
> the backend via the SAME connection.
>
> Is my interpretation of what you said correct?

Yes, each request on the same connection should contain the same
"Authorization: NTLM " header finally negotiated for
that connection, otherwise the NTLM server will respond with a status
401 (IIRC) to renegotiate a new authenticator.
They may be NTLM implementations that require the authenticator for
the first request only (actually until the third one due to the
client's three-step handshake), but this is even worse because from
there it becomes quite likely that any multiplexer on the route may
not only break NTLM (make it renegotiate again and again) but possibly
mixup sessions since subsequent requests could "steal" the session
(authenticator) of the first/previous user authenticated...

>
>
> I have only been testing one client test at-a-time so far, so probably that 
> was why my testing so far with proxy-initial-not-pooled and NTLM worked, 
> i.e., if there had been multiple clients all authenticating and going to the 
> same SharePoint server, and if I'm understanding what you were saying about 
> the requests going over the same connection that was used for the NTLM 
> authentication, my testing would probably have failed.
>
> Is that correct?
>
>
> Now, I am really glad I asked about this (and that Eric referred me to your 
> "aside connection" discussion).  I will have to raise this with my 
> colleagues, as it appears that the "proxy-initial-not-pooled" setting will 
> not work for any kind of production type situation?

I'm afraid yes, but with MPM prefork! (see below)

>
> I must be doing a lot of "praying", because so far I am not able to cause a 
> problem, at least trying to run 3 different clients.  I don't think that I 
> can actually get the NTLM authentications to occur simultaneously, but I'm 
> pretty sure the sessions are simulataneous, at least part of the time, but 
> even then, the pages seem for all 3 browsers seem to be appearing correctly 
> :(...

This may be due to the small number of connections reaching different
processes, rather than different threads in the same process, or are
you using the prefork MPM?

I should have think about "prefork" before, sorry for that (you
mentioned 2.4.x which made me sadly forget about prefork), but I
indeed realize now that it is very likely to work for NTLM when
proxy-initial-not-pooled is used: no chance that an established
backend connection gets reused underneath the current client
connection (i.e. the session for NTLM).

But with any other threaded MPM (event, worker, windows, ...), if you
try to forcibly make httpd run with a single process (either with
"StartServers 1"+"ServerLimit 1", or simply by using the -DONE_PROCESS
or -X arguments on the command line), you may reach the concurrency
issue quite rapidly with few client connections.

So if the prefork MPM is an option for you (and it works as I assume
it should), I would definitely recommend using it for NTLM, otherwise
I'm afraid you are stuck with the kind of patch I proposed.

Regards,
Yann.

-
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org

Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?

[o haya]

Hi Yann,

A CORRECTION.re. what I said about "ProxySet keepalive=On/Off".

I tested again, because I couldn't exactly remember if, when I tested 
previously, I had commented out the ProxySet directive completely, OR if I had 
just changed "ProxySet keepalive=On" to "ProxySet keepalive=Off".

So the correction is that:

- If ProxySet is commented out completely, then Apache sends "Connection: 
close" to the backend (Sharepoint) server
- If "ProxySet keepalive=On", then Apache sends "Connection: keep-alive" to the 
backend server
- If "ProxySet keepalive=Off", then Apache sends "Connection: keep-alive" to 
the backend server

In other words regardless of whatever ProxySet keepalive was set to "On" or 
"Off", Apache sent "Connection: keep-alive" to the back end server. 

On the other hand, if the "ProxySet" was commented out completely, then Apache 
sent "Connection: close" to the backend server.


Re. the last part of your message, are you saying if the httpd was compiled 
with MPM: prefork, that then the "proxy-initial-not-pooled" would let the 
Apache proxy work for NTLM and no need for the "aside" connections 
functionality?


FYI, I wanted to let you know that I checked, and our httpd was built with MPM: 
prefork.


Thanks!

Jim



----------------
On Tue, 10/27/15, Yann Ylavic <ylavic@gmail.com> wrote:

 Subject: Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?
 To: users@httpd.apache.org, "o haya" <oh...@yahoo.com>
 Date: Tuesday, October 27, 2015, 5:52 PM
 
 Hi Jim,
 
 On Tue, Oct 27, 2015 at 1:57
 AM, o haya <oh...@yahoo.com.invalid>
 wrote:
 >
 > First of
 all, as a kind of an aside remark (sorry for the
 "pun" :)), from my testing, it appears that if I
 have "ProxySet keepalive=On" inside a
 , then the requests to the
 backend all have "Connection: Keep-Alive" in the
 requests headers going to the backend server (a SharePoint
 server).  Conversely if "ProxySet keepalive=Off"
 is inside the ..., the HTTP
 requests to the backend have HTTP request header
 "Connection: closed".  In other words, the
 "ProxySet keepalive=On/Off" appears to be able to
 control whether a "Connection: keep-alive" vs.
 "Connection: closed" gets sent in a HTP request
 header to the backend.
 
 That's really weird, I can't see
 anything in the code that can provoke this.
 "ProxySet keepalive=On" really only
 issues a setsockopt(SO_KEEPALIVE,
 on) for
 the backend socket, whereas HTTP keepalive (Connection:
 keep-alive/close") is rather controlled by
 "ProxySet disablereuse=On"
 or
 SetEnv's like force-proxy-request-1.0 and
 proxy-nokeepalive.
 Will test this because it
 would be an unexpected behaviour (given that
 keepalive=off is the default)...
 
 >
 >
 Next:  I think I kind of understood about the
 proxy-initial-not-pooled setting ==> a new connection
 from the client always connected to the backend via a new
 Apache-to-backend connection, but I didn't realize that
 NTLM meant that all the requests SUBSEQUENT to the NTLM
 authentication had to ALSO go to the backend via the SAME
 connection.
 >
 > Is my
 interpretation of what you said correct?
 
 Yes, each request on the same connection should
 contain the same
 "Authorization: NTLM
 " header finally negotiated for
 that connection, otherwise the NTLM server will
 respond with a status
 401 (IIRC) to
 renegotiate a new authenticator.
 They may be
 NTLM implementations that require the authenticator for
 the first request only (actually until the
 third one due to the
 client's three-step
 handshake), but this is even worse because from
 there it becomes quite likely that any
 multiplexer on the route may
 not only break
 NTLM (make it renegotiate again and again) but possibly
 mixup sessions since subsequent requests could
 "steal" the session
 (authenticator) of the first/previous user
 authenticated...
 
 >
 >
 > I have only been
 testing one client test at-a-time so far, so probably that
 was why my testing so far with proxy-initial-not-pooled and
 NTLM worked, i.e., if there had been multiple clients all
 authenticating and going to the same SharePoint server, and
 if I'm understanding what you were saying about the
 requests going over the same connection that was used for
 the NTLM authentication, my testing would probably have
 failed.
 >
 > Is that
 correct?
 >
 >
 > Now, I am really glad I asked about this
 (and that Eric referred me to your "aside
 connection" discussion).  I will have to raise this
 with my colleagues, as it appears that the
 "proxy-initial-not-pooled" setting will not work
 for any kind of production type situation?
 
 I'

Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?

[Eric Covener]

Check the manuals discussion of how a "worker" is indirectly configured.

On Tue, Oct 27, 2015, 6:55 PM o haya <oh...@yahoo.com.invalid> wrote:

> Hi Yann,
>
> A CORRECTION.re. what I said about "ProxySet keepalive=On/Off".
>
> I tested again, because I couldn't exactly remember if, when I tested
> previously, I had commented out the ProxySet directive completely, OR if I
> had just changed "ProxySet keepalive=On" to "ProxySet keepalive=Off".
>
> So the correction is that:
>
> - If ProxySet is commented out completely, then Apache sends "Connection:
> close" to the backend (Sharepoint) server
> - If "ProxySet keepalive=On", then Apache sends "Connection: keep-alive"
> to the backend server
> - If "ProxySet keepalive=Off", then Apache sends "Connection: keep-alive"
> to the backend server
>
> In other words regardless of whatever ProxySet keepalive was set to "On"
> or "Off", Apache sent "Connection: keep-alive" to the back end server.
>
> On the other hand, if the "ProxySet" was commented out completely, then
> Apache sent "Connection: close" to the backend server.
>
>
> Re. the last part of your message, are you saying if the httpd was
> compiled with MPM: prefork, that then the "proxy-initial-not-pooled" would
> let the Apache proxy work for NTLM and no need for the "aside" connections
> functionality?
>
>
> FYI, I wanted to let you know that I checked, and our httpd was built with
> MPM: prefork.
>
>
> Thanks!
>
> Jim
>
>
>
> 
> On Tue, 10/27/15, Yann Ylavic <ylavic@gmail.com> wrote:
>
>  Subject: Re: [users@httpd] Persistent proxied connections with Apache
> 2.4.x?
>  To: users@httpd.apache.org, "o haya" <oh...@yahoo.com>
>  Date: Tuesday, October 27, 2015, 5:52 PM
>
>  Hi Jim,
>
>  On Tue, Oct 27, 2015 at 1:57
>  AM, o haya <oh...@yahoo.com.invalid>
>  wrote:
>  >
>  > First of
>  all, as a kind of an aside remark (sorry for the
>  "pun" :)), from my testing, it appears that if I
>  have "ProxySet keepalive=On" inside a
>  , then the requests to the
>  backend all have "Connection: Keep-Alive" in the
>  requests headers going to the backend server (a SharePoint
>  server).  Conversely if "ProxySet keepalive=Off"
>  is inside the ..., the HTTP
>  requests to the backend have HTTP request header
>  "Connection: closed".  In other words, the
>  "ProxySet keepalive=On/Off" appears to be able to
>  control whether a "Connection: keep-alive" vs.
>  "Connection: closed" gets sent in a HTP request
>  header to the backend.
>
>  That's really weird, I can't see
>  anything in the code that can provoke this.
>  "ProxySet keepalive=On" really only
>  issues a setsockopt(SO_KEEPALIVE,
>  on) for
>  the backend socket, whereas HTTP keepalive (Connection:
>  keep-alive/close") is rather controlled by
>  "ProxySet disablereuse=On"
>  or
>  SetEnv's like force-proxy-request-1.0 and
>  proxy-nokeepalive.
>  Will test this because it
>  would be an unexpected behaviour (given that
>  keepalive=off is the default)...
>
>  >
>  >
>  Next:  I think I kind of understood about the
>  proxy-initial-not-pooled setting ==> a new connection
>  from the client always connected to the backend via a new
>  Apache-to-backend connection, but I didn't realize that
>  NTLM meant that all the requests SUBSEQUENT to the NTLM
>  authentication had to ALSO go to the backend via the SAME
>  connection.
>  >
>  > Is my
>  interpretation of what you said correct?
>
>  Yes, each request on the same connection should
>  contain the same
>  "Authorization: NTLM
>  " header finally negotiated for
>  that connection, otherwise the NTLM server will
>  respond with a status
>  401 (IIRC) to
>  renegotiate a new authenticator.
>  They may be
>  NTLM implementations that require the authenticator for
>  the first request only (actually until the
>  third one due to the
>  client's three-step
>  handshake), but this is even worse because from
>  there it becomes quite likely that any
>  multiplexer on the route may
>  not only break
>  NTLM (make it renegotiate again and again) but possibly
>  mixup sessions since subsequent requests could
>  "steal" the session
>  (authenticator) of the first/previous user
>  authenticated...
>
>  >
>  >
>  > I have only been
>  testing one client test at

Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?

[o haya]

Eric,

Sorry, but this time, I'm not quite sure what (which aspect of the discussion) 
you're referring to?

Jim



On Tue, 10/27/15, Eric Covener <cove...@gmail.com> wrote:

 Subject: Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?
 To: users@httpd.apache.org
 Date: Tuesday, October 27, 2015, 7:02 PM
 
 Check the
 manuals discussion of how a "worker" is indirectly
 configured.
 
 
 
 On Tue, Oct
 27, 2015, 6:55 PM o haya <oh...@yahoo.com.invalid>
 wrote:
 Hi
 Yann,
 
 
 
 A CORRECTION.re. what I said about "ProxySet
 keepalive=On/Off".
 
 
 
 I tested again, because I couldn't exactly remember if,
 when I tested previously, I had commented out the ProxySet
 directive completely, OR if I had just changed
 "ProxySet keepalive=On" to "ProxySet
 keepalive=Off".
 
 
 
 So the correction is that:
 
 
 
 - If ProxySet is commented out completely, then Apache sends
 "Connection: close" to the backend (Sharepoint)
 server
 
 - If "ProxySet keepalive=On", then Apache sends
 "Connection: keep-alive" to the backend server
 
 - If "ProxySet keepalive=Off", then Apache sends
 "Connection: keep-alive" to the backend server
 
 
 
 In other words regardless of whatever ProxySet keepalive was
 set to "On" or "Off", Apache sent
 "Connection: keep-alive" to the back end
 server.
 
 
 
 On the other hand, if the "ProxySet" was commented
 out completely, then Apache sent "Connection:
 close" to the backend server.
 
 
 
 
 
 Re. the last part of your message, are you saying if the
 httpd was compiled with MPM: prefork, that then the
 "proxy-initial-not-pooled" would let the Apache
 proxy work for NTLM and no need for the "aside"
 connections functionality?
 
 
 
 
 
 FYI, I wanted to let you know that I checked, and our httpd
 was built with MPM: prefork.
 
 
 
 
 
 Thanks!
 
 
 
 Jim
 
 
 
 
 
 
 
 ----------------
 
 On Tue, 10/27/15, Yann Ylavic <ylavic@gmail.com>
 wrote:
 
 
 
  Subject: Re: [users@httpd] Persistent proxied connections
 with Apache 2.4.x?
 
  To: users@httpd.apache.org,
 "o haya" <oh...@yahoo.com>
 
  Date: Tuesday, October 27, 2015, 5:52 PM
 
 
 
  Hi Jim,
 
 
 
  On Tue, Oct 27, 2015 at 1:57
 
  AM, o haya <oh...@yahoo.com.invalid>
 
  wrote:
 
  >
 
  > First of
 
  all, as a kind of an aside remark (sorry for the
 
  "pun" :)), from my testing, it appears that if
 I
 
  have "ProxySet keepalive=On" inside a
 
  , then the requests to
 the
 
  backend all have "Connection: Keep-Alive" in
 the
 
  requests headers going to the backend server (a
 SharePoint
 
  server).  Conversely if "ProxySet
 keepalive=Off"
 
  is inside the ..., the HTTP
 
  requests to the backend have HTTP request header
 
  "Connection: closed".  In other words, the
 
  "ProxySet keepalive=On/Off" appears to be able
 to
 
  control whether a "Connection: keep-alive"
 vs.
 
  "Connection: closed" gets sent in a HTP
 request
 
  header to the backend.
 
 
 
  That's really weird, I can't see
 
  anything in the code that can provoke this.
 
  "ProxySet keepalive=On" really only
 
  issues a setsockopt(SO_KEEPALIVE,
 
  on) for
 
  the backend socket, whereas HTTP keepalive (Connection:
 
  keep-alive/close") is rather controlled by
 
  "ProxySet disablereuse=On"
 
  or
 
  SetEnv's like force-proxy-request-1.0 and
 
  proxy-nokeepalive.
 
  Will test this because it
 
  would be an unexpected behaviour (given that
 
  keepalive=off is the default)...
 
 
 
  >
 
  >
 
  Next:  I think I kind of understood about the
 
  proxy-initial-not-pooled setting ==> a new
 connection
 
  from the client always connected to the backend via a
 new
 
  Apache-to-backend connection, but I didn't realize
 that
 
  NTLM meant that all the requests SUBSEQUENT to the NTLM
 
  authentication had to ALSO go to the backend via the
 SAME
 
  connection.
 
  >
 
  > Is my
 
  interpretation of what you said correct?
 
 
 
  Yes, each request on the same connection should
 
  contain the same
 
  "Authorization: NTLM
 
  " header finally negotiated
 for
 
  that connection, otherwise the NTLM server will
 
  respond with a status
 
  401 (IIRC) to
 
  renegotiate a new authenticator.
 
  They may be
 
  NTLM implementations that require the authenticator for
 
  the first request only (actually until the
 
  third one due to the
 
  client's three-step
 
  handshake), but this is even worse because from
 
  there it becomes quite likely that any
 
  multiplexer on the route may
 
  not only break
 
  NTLM (make it renegotiate again and again) but possibly
 
  mixup sessions since subsequent requests could
 
  "steal

Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?

[o haya]

Hi Yann,

Thank you for responding (with lots of info!).  

First of all, I have to apologize for top-posting.  I am using Yahoo mail, and 
it doesn't seem to allow quoted responses, which makes things kind of difficult 
(i.e., very difficult to intersperse comments since they don't mark the quoted 
email).


First of all, as a kind of an aside remark (sorry for the "pun" :)), from my 
testing, it appears that if I have "ProxySet keepalive=On" inside a 
, then the requests to the backend all have "Connection: 
Keep-Alive" in the requests headers going to the backend server (a SharePoint 
server).  Conversely if "ProxySet keepalive=Off" is inside the 
..., the HTTP requests to the backend have HTTP request header 
"Connection: closed".  In other words, the "ProxySet keepalive=On/Off" appears 
to be able to control whether a "Connection: keep-alive" vs. "Connection: 
closed" gets sent in a HTP request header to the backend.

I am NOT trying to dispute what you said about the "ProxySet" being only about 
"TCP keepalive", but just letting you know what I was seeing during testing and 
also maybe I'm misunderstanding what you meant when you were saying "TCP 
keepalive" vs. "HTTP keepalive"?


Next:  I think I kind of understood about the proxy-initial-not-pooled setting 
==> a new connection from the client always connected to the backend via a new 
Apache-to-backend connection, but I didn't realize that NTLM meant that all the 
requests SUBSEQUENT to the NTLM authentication had to ALSO go to the backend 
via the SAME connection.  

Is my interpretation of what you said correct?


I have only been testing one client test at-a-time so far, so probably that was 
why my testing so far with proxy-initial-not-pooled and NTLM worked, i.e., if 
there had been multiple clients all authenticating and going to the same 
SharePoint server, and if I'm understanding what you were saying about the 
requests going over the same connection that was used for the NTLM 
authentication, my testing would probably have failed.  

Is that correct?


Now, I am really glad I asked about this (and that Eric referred me to your 
"aside connection" discussion).  I will have to raise this with my colleagues, 
as it appears that the "proxy-initial-not-pooled" setting will not work for any 
kind of production type situation?

Thanks!

Jim






- Original Message -
From: Yann Ylavic <ylavic@gmail.com>
To: users@httpd.apache.org; o haya <oh...@yahoo.com>
Cc: 
Sent: Monday, October 26, 2015 6:48 PM
Subject: Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?

Hi Jim,

sorry for the late, I'm not much online these days.

On Sun, Oct 25, 2015 at 9:54 PM, o haya <oh...@yahoo.com.invalid> wrote:
>
> - With respect to proxying NTLM authentication, does the "aside connections"
> functionality that was mentioned earlier accomplish the same thing as using
> the "Proxy keepalive=On and SetEnv proxy-initial-not-pooled"?

Shortly, no.

"ProxySet keepalive=On" is about TCP keepalive (system probes to
prevent long living TCP connections from being dropped by gateways,
i.e. the socket's SO_KEEPALIVE option), and has nothing to do with
HTTP keepalive (multiple HTTP requests sent on the same connection).
Actually, HTTP keepalive is the default for mod_proxy_http, provided
the backend is "declared" with either a ProxyPass line or a 
block (as opposed to eg. a RewriteRule [P]), so you don't need to
configure anything special to get it (whereas on the contrary
"ProxySet disablereuse=on" can be used to disable HTTP keeplive on the
backend side).

"SetEnv proxy-initial-not-pooled" is unfortunately not fully helpful for NTLM.
It allows to always create a new connection to the backend for any new
connection from the client, or said differently, it prevents an
established backend connection (kept alive) from being reused in this
case (see [*] below for the real goal of proxy-initial-not-pooled).
But this gives no garanty on subsequent requests on the same
connection, or worse, subsequent requests on another connection...
Those may reuse any established connection in the pool, or a new
connection, depending on the first one available at the time of each
incoming request.
In other words, mod_proxy_http handles a pool of connections for each
"declared" backend independently/regardless of client-side connections
or requests (basically it's a n client connections multiplexor over m
backend connections), because the HTTP protocol *is* stateless.
And that breaks NTLM because this protocol really authenticates
connections, not requests, assuming there is one single user per
connection (sigh)...

>
> - If not, what are the differences?

So for a proxy to "work" with NTLM, it must associa

Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?

[Yann Ylavic]

Hi Jim,

sorry for the late, I'm not much online these days.

On Sun, Oct 25, 2015 at 9:54 PM, o haya  wrote:
>
> - With respect to proxying NTLM authentication, does the "aside connections"
> functionality that was mentioned earlier accomplish the same thing as using
> the "Proxy keepalive=On and SetEnv proxy-initial-not-pooled"?

Shortly, no.

"ProxySet keepalive=On" is about TCP keepalive (system probes to
prevent long living TCP connections from being dropped by gateways,
i.e. the socket's SO_KEEPALIVE option), and has nothing to do with
HTTP keepalive (multiple HTTP requests sent on the same connection).
Actually, HTTP keepalive is the default for mod_proxy_http, provided
the backend is "declared" with either a ProxyPass line or a 
block (as opposed to eg. a RewriteRule [P]), so you don't need to
configure anything special to get it (whereas on the contrary
"ProxySet disablereuse=on" can be used to disable HTTP keeplive on the
backend side).

"SetEnv proxy-initial-not-pooled" is unfortunately not fully helpful for NTLM.
It allows to always create a new connection to the backend for any new
connection from the client, or said differently, it prevents an
established backend connection (kept alive) from being reused in this
case (see [*] below for the real goal of proxy-initial-not-pooled).
But this gives no garanty on subsequent requests on the same
connection, or worse, subsequent requests on another connection...
Those may reuse any established connection in the pool, or a new
connection, depending on the first one available at the time of each
incoming request.
In other words, mod_proxy_http handles a pool of connections for each
"declared" backend independently/regardless of client-side connections
or requests (basically it's a n client connections multiplexor over m
backend connections), because the HTTP protocol *is* stateless.
And that breaks NTLM because this protocol really authenticates
connections, not requests, assuming there is one single user per
connection (sigh)...

>
> - If not, what are the differences?

So for a proxy to "work" with NTLM, it must associate a single client
connection with a single one on the/each backend side, and pray for
any gateway before it to do the same (yes, proxying NTLM may be
hazardous, one single multiplexor on the route and things get messed
up)!
That's what the "aside connections" patch does, it can create
connections aside from the backend pool (based on the "proxy-aside-c"
environment variable, settable with SetEnv[If] or a RewriteRule, eg.
when "RewriteCond %{HTTP:Authorization} ^NTLM"), and maintain them by
client connection so that all the requests on this connection (setting
the env var) will be routed to their associated backend connection.

However I don't recall all the details of the patch proposed in 2014,
I think I have a simpler/more-to-the-point one now (the previous one
was meant to be generic enough to be accepted in httpd, which did not
happen), so let me have a look when back home and attach it here.

Regards,
Yann.


[*] The goal of proxy-initial-not-pooled is to help recover from a
race-condition error where the proxy sends its request to the backend
while the latter is in the process of closing the connection
(keepalive timeout or whatever). This results in an error (502) being
returned to the client, but while the client "expects" this error on
kept alive connections (because of the same possible race condition on
its side) and can then resend the same request, it will not do this
for newly established connections, and hence proxy-initial-not-pooled
prevents this error from being fatal for the "user experience" by
avoiding the race on the backend side when the user-agent does not
expect it.

-
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org

Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?

[o haya]

Hi,

I must be doing a lot of "praying", because so far I am not able to cause a 
problem, at least trying to run 3 different clients.  I don't think that I can 
actually get the NTLM authentications to occur simultaneously, but I'm pretty 
sure the sessions are simulataneous, at least part of the time, but even then, 
the pages seem for all 3 browsers seem to be appearing correctly :(...

Jim



On Mon, 10/26/15, o haya <oh...@yahoo.com.INVALID> wrote:

 Subject: Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?
 To: "users@httpd.apache.org" <users@httpd.apache.org>
 Cc: "O. Haya" <oh...@yahoo.com>
 Date: Monday, October 26, 2015, 8:57 PM
 
 Hi Yann,
 
 Thank you for responding (with
 lots of info!).  
 
 First of
 all, I have to apologize for top-posting.  I am using Yahoo
 mail, and it doesn't seem to allow quoted responses,
 which makes things kind of difficult (i.e., very difficult
 to intersperse comments since they don't mark the quoted
 email).
 
 
 First of all, as a kind of an aside remark
 (sorry for the "pun" :)), from my testing, it
 appears that if I have "ProxySet keepalive=On"
 inside a , then the requests
 to the backend all have "Connection: Keep-Alive"
 in the requests headers going to the backend server (a
 SharePoint server).  Conversely if "ProxySet
 keepalive=Off" is inside the
 ..., the HTTP requests to the
 backend have HTTP request header "Connection:
 closed".  In other words, the "ProxySet
 keepalive=On/Off" appears to be able to control whether
 a "Connection: keep-alive" vs. "Connection:
 closed" gets sent in a HTP request header to the
 backend.
 
 I am NOT trying to
 dispute what you said about the "ProxySet" being
 only about "TCP keepalive", but just letting you
 know what I was seeing during testing and also maybe I'm
 misunderstanding what you meant when you were saying
 "TCP keepalive" vs. "HTTP keepalive"?
 
 
 Next:  I
 think I kind of understood about the
 proxy-initial-not-pooled setting ==> a new connection
 from the client always connected to the backend via a new
 Apache-to-backend connection, but I didn't realize that
 NTLM meant that all the requests SUBSEQUENT to the NTLM
 authentication had to ALSO go to the backend via the SAME
 connection.  
 
 Is my
 interpretation of what you said correct?
 
 
 I have only been testing one
 client test at-a-time so far, so probably that was why my
 testing so far with proxy-initial-not-pooled and NTLM
 worked, i.e., if there had been multiple clients all
 authenticating and going to the same SharePoint server, and
 if I'm understanding what you were saying about the
 requests going over the same connection that was used for
 the NTLM authentication, my testing would probably have
 failed.  
 
 Is that
 correct?
 
 
 Now, I am really glad I asked about this (and
 that Eric referred me to your "aside connection"
 discussion).  I will have to raise this with my colleagues,
 as it appears that the "proxy-initial-not-pooled"
 setting will not work for any kind of production type
 situation?
 
 Thanks!
 
 Jim
 
 
 
 
 
 
 -
 Original Message -
 From: Yann Ylavic
 <ylavic....@gmail.com>
 To: users@httpd.apache.org;
 o haya <oh...@yahoo.com>
 Cc: 
 Sent: Monday, October 26,
 2015 6:48 PM
 Subject: Re: [users@httpd] Persistent
 proxied connections with Apache 2.4.x?
 
 Hi Jim,
 
 sorry
 for the late, I'm not much online these days.
 
 On Sun, Oct 25, 2015 at 9:54
 PM, o haya <oh...@yahoo.com.invalid>
 wrote:
 >
 > - With
 respect to proxying NTLM authentication, does the
 "aside connections"
 >
 functionality that was mentioned earlier accomplish the same
 thing as using
 > the "Proxy
 keepalive=On and SetEnv proxy-initial-not-pooled"?
 
 Shortly, no.
 
 "ProxySet
 keepalive=On" is about TCP keepalive (system probes
 to
 prevent long living TCP connections from
 being dropped by gateways,
 i.e. the
 socket's SO_KEEPALIVE option), and has nothing to do
 with
 HTTP keepalive (multiple HTTP requests
 sent on the same connection).
 Actually, HTTP
 keepalive is the default for mod_proxy_http, provided
 the backend is "declared" with either
 a ProxyPass line or a 
 block
 (as opposed to eg. a RewriteRule [P]), so you don't need
 to
 configure anything special to get it
 (whereas on the contrary
 "ProxySet
 disablereuse=on" can be used to disable HTTP keeplive
 on the
 backend side).
 
 "SetEnv proxy-initial-not-pooled" is
 unfortunately not fully helpful for NTLM.
 It
 allows to always create a new connection to the backend for
 any new
 connection from the client, or said
 differently, it prevents an
 established
 backend connection (kept alive) from being reused in this
 case (see

Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?

[o haya]

  From: o haya <oh...@yahoo.com.INVALID>
 To: "users@httpd.apache.org" <users@httpd.apache.org>; o haya 
<oh...@yahoo.com> 
 Sent: Friday, October 23, 2015 8:03 PM
 Subject: Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?
   

   

   From: o haya <oh...@yahoo.com.INVALID>
 To: "users@httpd.apache.org" <users@httpd.apache.org> 
Cc: O. Haya <oh...@yahoo.com> 
 Sent: Friday, October 23, 2015 5:02 PM
 Subject: Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?
   



  From: Eric Covener <cove...@gmail.com>
 To: users@httpd.apache.org; o haya <oh...@yahoo.com> 
 Sent: Friday, October 23, 2015 8:14 AM
 Subject: Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?
   
On Thu, Oct 22, 2015 at 12:28 PM, o haya <oh...@yahoo.com.invalid> wrote:


> So I am wondering if there is a way to do this (make all the backend
> connections persistent with the "Connection: keepalive")?


There's a patch in thread "mod_proxy's aside connections proposal"

-
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org


Hi Eric,
Thanks for that reference!
Googling that, and looking at Yann's thread about his patch, and also the bug 
report that preceded that patch, am I understanding things correctly, that the 
problem with NTLM is that all the requests involved in "an" NTLM authentication 
have to go down the same persistent connection (from Apache to the target 
(e.g., SharePoint) machine?

The thing is that I thought that I've been able to get NTLM working, proxying 
through Apache, using "ProxySet keepalive=On" in a  section, e.g.:
http://sharepoint>ProxySet keepalive=OnSetEnv 
proxy-initial-not-pooled
So does the "same persistent connection" requirement mean that if I was doing a 
bunch of simultaneous NTLM logins through the Apache (e.g., to 
http://sharepoint) that some of the login attempts would kind of randomly not 
work (because requests not going down same connection to the SharePoint)?
Thanks,Jim   


Hi,
I've been told that, because we use that "SetEnv proxy-initial-not-pooled", 
that that prevents the problem that I mentioned above.  However, the only 
explanations of what that parameter does seem to be unrelated to this situation 
(they mostly refer to 502 errors?), so I don't quite understand "why?" having 
that setting solves the (potential) problem that I'm asking about with proxying 
NTLM?  
Can someone here explain that?
Thanks,Jim   



Hi,
No one has responded to the above yet, but maybe to be clearer, what I am 
curious about at this point is:
- With respect to proxying NTLM authentication, does the "aside connections" 
functionality that was mentioned earlier accomplish the same thing as using the 
"Proxy keepalive=On and SetEnv proxy-initial-not-pooled"?
- If not, what are the differences?
Thanks,Jim  

Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?

[Eric Covener]

On Thu, Oct 22, 2015 at 12:28 PM, o haya  wrote:
> So I am wondering if there is a way to do this (make all the backend
> connections persistent with the "Connection: keepalive")?


There's a patch in thread "mod_proxy's aside connections proposal"

-
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org

Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?

[o haya]

  From: Eric Covener <cove...@gmail.com>
 To: users@httpd.apache.org; o haya <oh...@yahoo.com> 
 Sent: Friday, October 23, 2015 8:14 AM
 Subject: Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?
   
On Thu, Oct 22, 2015 at 12:28 PM, o haya <oh...@yahoo.com.invalid> wrote:


> So I am wondering if there is a way to do this (make all the backend
> connections persistent with the "Connection: keepalive")?


There's a patch in thread "mod_proxy's aside connections proposal"

-
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org


Hi Eric,
Thanks for that reference!
Googling that, and looking at Yann's thread about his patch, and also the bug 
report that preceded that patch, am I understanding things correctly, that the 
problem with NTLM is that all the requests involved in "an" NTLM authentication 
have to go down the same persistent connection (from Apache to the target 
(e.g., SharePoint) machine?

The thing is that I thought that I've been able to get NTLM working, proxying 
through Apache, using "ProxySet keepalive=On" in a  section, e.g.:
http://sharepoint>ProxySet keepalive=OnSetEnv 
proxy-initial-not-pooled
So does the "same persistent connection" requirement mean that if I was doing a 
bunch of simultaneous NTLM logins through the Apache (e.g., to 
http://sharepoint) that some of the login attempts would kind of randomly not 
work (because requests not going down same connection to the SharePoint)?
Thanks,Jim  

Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?

[o haya]

  From: o haya <oh...@yahoo.com.INVALID>
 To: "users@httpd.apache.org" <users@httpd.apache.org> 
Cc: O. Haya <oh...@yahoo.com> 
 Sent: Friday, October 23, 2015 5:02 PM
 Subject: Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?
   



  From: Eric Covener <cove...@gmail.com>
 To: users@httpd.apache.org; o haya <oh...@yahoo.com> 
 Sent: Friday, October 23, 2015 8:14 AM
 Subject: Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?
   
On Thu, Oct 22, 2015 at 12:28 PM, o haya <oh...@yahoo.com.invalid> wrote:


> So I am wondering if there is a way to do this (make all the backend
> connections persistent with the "Connection: keepalive")?


There's a patch in thread "mod_proxy's aside connections proposal"

-
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org


Hi Eric,
Thanks for that reference!
Googling that, and looking at Yann's thread about his patch, and also the bug 
report that preceded that patch, am I understanding things correctly, that the 
problem with NTLM is that all the requests involved in "an" NTLM authentication 
have to go down the same persistent connection (from Apache to the target 
(e.g., SharePoint) machine?

The thing is that I thought that I've been able to get NTLM working, proxying 
through Apache, using "ProxySet keepalive=On" in a  section, e.g.:
http://sharepoint>ProxySet keepalive=OnSetEnv 
proxy-initial-not-pooled
So does the "same persistent connection" requirement mean that if I was doing a 
bunch of simultaneous NTLM logins through the Apache (e.g., to 
http://sharepoint) that some of the login attempts would kind of randomly not 
work (because requests not going down same connection to the SharePoint)?
Thanks,Jim   


Hi,
I've been told that, because we use that "SetEnv proxy-initial-not-pooled", 
that that prevents the problem that I mentioned above.  However, the only 
explanations of what that parameter does seem to be unrelated to this situation 
(they mostly refer to 502 errors?), so I don't quite understand "why?" having 
that setting solves the (potential) problem that I'm asking about with proxying 
NTLM?  
Can someone here explain that?
Thanks,Jim  

Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?

[Marat Khalili]

> I am wondering if there is a way to do this (make all the backend connections 
> persistent with the "Connection: keepalive")?

IMO there's no way to do it without enumerating all your backend hosts. Apache 
needs to create a separate worker for each host at startup time.


> "Connection: keepalive" in the request headers

Just in case you are actually looking at the headers, it should be "Connection: 
keep-alive" in the obsolete HTTP 1.0, and they are all persistent by default in 
HTTP 1.1 unless "Connection: close" is specified by any side. If SharePoint 
follows any standards, that is.
-- 

With Best Regards,
Marat Khalili

On October 22, 2015 7:28:20 PM GMT+03:00, o haya  
wrote:
>Hi,
>We want to use Apache 2.4.x to proxy through to some SharePoint
>instances using NTLM logins.  From testing, it looks like the
>Apache-to-SharePoint connections need to be persistent, with
>"Connection: keepalive" in the request headers going from
>Apache-to-SharePoint.
>We can do this using a directive:
>ProxySet keepalive=On
>inside a  section, but we would like to do this (enable
>keepalive on the backend connections) on all of the backend
>connections.  But, if we try to put that ProxySet inside a , we
>get a startup error saying that it needs an absolute URL.
>So I am wondering if there is a way to do this (make all the backend
>connections persistent with the "Connection: keepalive")?
>Thanks,Jim