Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?
Hi Jim, disclaimer: I'm not an NTLM expert... On Tue, Nov 3, 2015 at 3:04 PM, o haya wrote: > > I think that other info that binds to the authenticated user is at the > SharePoint application level (maybe persisted in the app on the client side) > because it doesn't look like any cookies or headers that are user-specific > get sent to SharePoint after the user is authenticated. Not even the "Authentication: NTLM ..." header? > > So, in other words, once the user is authenticated via NTLM, it looks like it > doesn't matter which connection any subsequent requests get sent over, since > the session info is probably carried in the requests themselves somewhere > (SharePoint 2010+ uses an internal STS, and the client-side app gets a token > from the STS at some point and then persists it and sends that token along > with requests to SharePoint). No idea where that authenticator/token may be... I suspect that once the user is authenticated on a connection, that connection is assumed to be owned by that same user and hence everything that comes in is the user session (there may be a check on the authenticator or token if available for each request, so that a renegotiation is asked on mismatch, but still multiple users can't share the same connection AFAICT). What happens if you use a threaded MPM (and start httpd with -X to leverage the multiplexing, as suggested in a previous message), or if you use another httpd (likewise threaded and started with -X) in front of your current proxy? The goal here is to have something in the chain that multiplexes requests on a the same connection before the Sharepoint, and hence see if the sharepoint behaves correctly in this case (I doubt so, otherwise you wouldn't need any particular httpd configuration or patch). Once the first user is NTLM-authenticated with a first connection to the frontend/multiplexer, open a second connection with no NTLM authentication (simple HTTP request), and then open a third one for another user (asking for his/her own NTLM authentication). Make sure the frontend reuse the same outgoing connection each time. Then you will see if the sharepoint: 1. is able to handle multiple users on the same connection, or 2. asks for renegotiation (in both cases), or 3. reuses the first session for successives requests (for one and/or the other case). Except 1., the users' sessions will break (and even be compromised in 3.), hence you'll need to make sure there is no multiplexer in the chain (SSL termination on the frontend/proxy, and direct connection to the backend -- no load-balancer or things like that). > > Anyway, that is what I think, as far as the importance of being on the same > connection. Does that make sense? This requires a testing IMHO... Regards, Yann. - To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org For additional commands, e-mail: users-h...@httpd.apache.org
Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?
On Tue, Oct 27, 2015 at 7:07 PM, o haya wrote: > Sorry, but this time, I'm not quite sure what (which aspect of the > discussion) you're referring to? Sorry, was top-posting from my phone. It's about why or ProxySet without anything else specified enabled persistent connectons. If you don't use ProxyPass and use e.g. RewriteRUle .. [P] a generic backend "worker" is used -- unless you give httpd that extra hint. - To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org For additional commands, e-mail: users-h...@httpd.apache.org
Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?
Eric, Sorry, but this time, I'm not quite sure what (which aspect of the discussion) you're referring to? Jim On Tue, 10/27/15, Eric Covener wrote: Subject: Re: [users@httpd] Persistent proxied connections with Apache 2.4.x? To: users@httpd.apache.org Date: Tuesday, October 27, 2015, 7:02 PM Check the manuals discussion of how a "worker" is indirectly configured. On Tue, Oct 27, 2015, 6:55 PM o haya wrote: Hi Yann, A CORRECTION.re. what I said about "ProxySet keepalive=On/Off". I tested again, because I couldn't exactly remember if, when I tested previously, I had commented out the ProxySet directive completely, OR if I had just changed "ProxySet keepalive=On" to "ProxySet keepalive=Off". So the correction is that: - If ProxySet is commented out completely, then Apache sends "Connection: close" to the backend (Sharepoint) server - If "ProxySet keepalive=On", then Apache sends "Connection: keep-alive" to the backend server - If "ProxySet keepalive=Off", then Apache sends "Connection: keep-alive" to the backend server In other words regardless of whatever ProxySet keepalive was set to "On" or "Off", Apache sent "Connection: keep-alive" to the back end server. On the other hand, if the "ProxySet" was commented out completely, then Apache sent "Connection: close" to the backend server. Re. the last part of your message, are you saying if the httpd was compiled with MPM: prefork, that then the "proxy-initial-not-pooled" would let the Apache proxy work for NTLM and no need for the "aside" connections functionality? FYI, I wanted to let you know that I checked, and our httpd was built with MPM: prefork. Thanks! Jim On Tue, 10/27/15, Yann Ylavic wrote: Subject: Re: [users@httpd] Persistent proxied connections with Apache 2.4.x? To: users@httpd.apache.org, "o haya" Date: Tuesday, October 27, 2015, 5:52 PM Hi Jim, On Tue, Oct 27, 2015 at 1:57 AM, o haya wrote: > > First of all, as a kind of an aside remark (sorry for the "pun" :)), from my testing, it appears that if I have "ProxySet keepalive=On" inside a , then the requests to the backend all have "Connection: Keep-Alive" in the requests headers going to the backend server (a SharePoint server). Conversely if "ProxySet keepalive=Off" is inside the ..., the HTTP requests to the backend have HTTP request header "Connection: closed". In other words, the "ProxySet keepalive=On/Off" appears to be able to control whether a "Connection: keep-alive" vs. "Connection: closed" gets sent in a HTP request header to the backend. That's really weird, I can't see anything in the code that can provoke this. "ProxySet keepalive=On" really only issues a setsockopt(SO_KEEPALIVE, on) for the backend socket, whereas HTTP keepalive (Connection: keep-alive/close") is rather controlled by "ProxySet disablereuse=On" or SetEnv's like force-proxy-request-1.0 and proxy-nokeepalive. Will test this because it would be an unexpected behaviour (given that keepalive=off is the default)... > > Next: I think I kind of understood about the proxy-initial-not-pooled setting ==> a new connection from the client always connected to the backend via a new Apache-to-backend connection, but I didn't realize that NTLM meant that all the requests SUBSEQUENT to the NTLM authentication had to ALSO go to the backend via the SAME connection. > > Is my interpretation of what you said correct? Yes, each request on the same connection should contain the same "Authorization: NTLM " header finally negotiated for that connection, otherwise the NTLM server will respond with a status 401 (IIRC) to renegotiate a new authenticator. They may be NTLM implementations that require the authenticator for the first request only (actually until the third one due to the client's three-step handshake), but this is even worse because from there it becomes quite likely that any multiplexer on the route may not only break NTLM (make it renegotiate again and again) but possibly mixup sessions since subsequent requests could "steal" the session (authenticator) of the first/previous user authenticated... > &g
Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?
Check the manuals discussion of how a "worker" is indirectly configured. On Tue, Oct 27, 2015, 6:55 PM o haya wrote: > Hi Yann, > > A CORRECTION.re. what I said about "ProxySet keepalive=On/Off". > > I tested again, because I couldn't exactly remember if, when I tested > previously, I had commented out the ProxySet directive completely, OR if I > had just changed "ProxySet keepalive=On" to "ProxySet keepalive=Off". > > So the correction is that: > > - If ProxySet is commented out completely, then Apache sends "Connection: > close" to the backend (Sharepoint) server > - If "ProxySet keepalive=On", then Apache sends "Connection: keep-alive" > to the backend server > - If "ProxySet keepalive=Off", then Apache sends "Connection: keep-alive" > to the backend server > > In other words regardless of whatever ProxySet keepalive was set to "On" > or "Off", Apache sent "Connection: keep-alive" to the back end server. > > On the other hand, if the "ProxySet" was commented out completely, then > Apache sent "Connection: close" to the backend server. > > > Re. the last part of your message, are you saying if the httpd was > compiled with MPM: prefork, that then the "proxy-initial-not-pooled" would > let the Apache proxy work for NTLM and no need for the "aside" connections > functionality? > > > FYI, I wanted to let you know that I checked, and our httpd was built with > MPM: prefork. > > > Thanks! > > Jim > > > > > On Tue, 10/27/15, Yann Ylavic wrote: > > Subject: Re: [users@httpd] Persistent proxied connections with Apache > 2.4.x? > To: users@httpd.apache.org, "o haya" > Date: Tuesday, October 27, 2015, 5:52 PM > > Hi Jim, > > On Tue, Oct 27, 2015 at 1:57 > AM, o haya > wrote: > > > > First of > all, as a kind of an aside remark (sorry for the > "pun" :)), from my testing, it appears that if I > have "ProxySet keepalive=On" inside a > , then the requests to the > backend all have "Connection: Keep-Alive" in the > requests headers going to the backend server (a SharePoint > server). Conversely if "ProxySet keepalive=Off" > is inside the ..., the HTTP > requests to the backend have HTTP request header > "Connection: closed". In other words, the > "ProxySet keepalive=On/Off" appears to be able to > control whether a "Connection: keep-alive" vs. > "Connection: closed" gets sent in a HTP request > header to the backend. > > That's really weird, I can't see > anything in the code that can provoke this. > "ProxySet keepalive=On" really only > issues a setsockopt(SO_KEEPALIVE, > on) for > the backend socket, whereas HTTP keepalive (Connection: > keep-alive/close") is rather controlled by > "ProxySet disablereuse=On" > or > SetEnv's like force-proxy-request-1.0 and > proxy-nokeepalive. > Will test this because it > would be an unexpected behaviour (given that > keepalive=off is the default)... > > > > > > Next: I think I kind of understood about the > proxy-initial-not-pooled setting ==> a new connection > from the client always connected to the backend via a new > Apache-to-backend connection, but I didn't realize that > NTLM meant that all the requests SUBSEQUENT to the NTLM > authentication had to ALSO go to the backend via the SAME > connection. > > > > Is my > interpretation of what you said correct? > > Yes, each request on the same connection should > contain the same > "Authorization: NTLM > " header finally negotiated for > that connection, otherwise the NTLM server will > respond with a status > 401 (IIRC) to > renegotiate a new authenticator. > They may be > NTLM implementations that require the authenticator for > the first request only (actually until the > third one due to the > client's three-step > handshake), but this is even worse because from > there it becomes quite likely that any > multiplexer on the route may > not only break > NTLM (make it renegotiate again and again) but possibly > mixup sessions since subsequent requests could > "steal" the session > (authenticator) of the first/previous user > authenticated... > > > > > > > I have only been > testing one client test at-a-time so far, so probably that > was why my testing so far with proxy-initia
Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?
Hi Yann, A CORRECTION.re. what I said about "ProxySet keepalive=On/Off". I tested again, because I couldn't exactly remember if, when I tested previously, I had commented out the ProxySet directive completely, OR if I had just changed "ProxySet keepalive=On" to "ProxySet keepalive=Off". So the correction is that: - If ProxySet is commented out completely, then Apache sends "Connection: close" to the backend (Sharepoint) server - If "ProxySet keepalive=On", then Apache sends "Connection: keep-alive" to the backend server - If "ProxySet keepalive=Off", then Apache sends "Connection: keep-alive" to the backend server In other words regardless of whatever ProxySet keepalive was set to "On" or "Off", Apache sent "Connection: keep-alive" to the back end server. On the other hand, if the "ProxySet" was commented out completely, then Apache sent "Connection: close" to the backend server. Re. the last part of your message, are you saying if the httpd was compiled with MPM: prefork, that then the "proxy-initial-not-pooled" would let the Apache proxy work for NTLM and no need for the "aside" connections functionality? FYI, I wanted to let you know that I checked, and our httpd was built with MPM: prefork. Thanks! Jim -------- On Tue, 10/27/15, Yann Ylavic wrote: Subject: Re: [users@httpd] Persistent proxied connections with Apache 2.4.x? To: users@httpd.apache.org, "o haya" Date: Tuesday, October 27, 2015, 5:52 PM Hi Jim, On Tue, Oct 27, 2015 at 1:57 AM, o haya wrote: > > First of all, as a kind of an aside remark (sorry for the "pun" :)), from my testing, it appears that if I have "ProxySet keepalive=On" inside a , then the requests to the backend all have "Connection: Keep-Alive" in the requests headers going to the backend server (a SharePoint server). Conversely if "ProxySet keepalive=Off" is inside the ..., the HTTP requests to the backend have HTTP request header "Connection: closed". In other words, the "ProxySet keepalive=On/Off" appears to be able to control whether a "Connection: keep-alive" vs. "Connection: closed" gets sent in a HTP request header to the backend. That's really weird, I can't see anything in the code that can provoke this. "ProxySet keepalive=On" really only issues a setsockopt(SO_KEEPALIVE, on) for the backend socket, whereas HTTP keepalive (Connection: keep-alive/close") is rather controlled by "ProxySet disablereuse=On" or SetEnv's like force-proxy-request-1.0 and proxy-nokeepalive. Will test this because it would be an unexpected behaviour (given that keepalive=off is the default)... > > Next: I think I kind of understood about the proxy-initial-not-pooled setting ==> a new connection from the client always connected to the backend via a new Apache-to-backend connection, but I didn't realize that NTLM meant that all the requests SUBSEQUENT to the NTLM authentication had to ALSO go to the backend via the SAME connection. > > Is my interpretation of what you said correct? Yes, each request on the same connection should contain the same "Authorization: NTLM " header finally negotiated for that connection, otherwise the NTLM server will respond with a status 401 (IIRC) to renegotiate a new authenticator. They may be NTLM implementations that require the authenticator for the first request only (actually until the third one due to the client's three-step handshake), but this is even worse because from there it becomes quite likely that any multiplexer on the route may not only break NTLM (make it renegotiate again and again) but possibly mixup sessions since subsequent requests could "steal" the session (authenticator) of the first/previous user authenticated... > > > I have only been testing one client test at-a-time so far, so probably that was why my testing so far with proxy-initial-not-pooled and NTLM worked, i.e., if there had been multiple clients all authenticating and going to the same SharePoint server, and if I'm understanding what you were saying about the requests going over the same connection that was used for the NTLM authentication, my testing would probably have failed. > > Is that correct? > > > Now, I am really glad I asked about this (and that Eric referred me to your "aside connection" discussion). I will have to raise this with my colleagues, as it appears that the "proxy-initial-not-pooled" setting will not work for any kind of production type situation? I'm afraid yes, but with MPM prefork! (se
Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?
Hi Jim, On Tue, Oct 27, 2015 at 1:57 AM, o haya wrote: > > First of all, as a kind of an aside remark (sorry for the "pun" :)), from my > testing, it appears that if I have "ProxySet keepalive=On" inside a > , then the requests to the backend all have "Connection: > Keep-Alive" in the requests headers going to the backend server (a SharePoint > server). Conversely if "ProxySet keepalive=Off" is inside the > ..., the HTTP requests to the backend have HTTP request header > "Connection: closed". In other words, the "ProxySet keepalive=On/Off" > appears to be able to control whether a "Connection: keep-alive" vs. > "Connection: closed" gets sent in a HTP request header to the backend. That's really weird, I can't see anything in the code that can provoke this. "ProxySet keepalive=On" really only issues a setsockopt(SO_KEEPALIVE, on) for the backend socket, whereas HTTP keepalive (Connection: keep-alive/close") is rather controlled by "ProxySet disablereuse=On" or SetEnv's like force-proxy-request-1.0 and proxy-nokeepalive. Will test this because it would be an unexpected behaviour (given that keepalive=off is the default)... > > Next: I think I kind of understood about the proxy-initial-not-pooled > setting ==> a new connection from the client always connected to the backend > via a new Apache-to-backend connection, but I didn't realize that NTLM meant > that all the requests SUBSEQUENT to the NTLM authentication had to ALSO go to > the backend via the SAME connection. > > Is my interpretation of what you said correct? Yes, each request on the same connection should contain the same "Authorization: NTLM " header finally negotiated for that connection, otherwise the NTLM server will respond with a status 401 (IIRC) to renegotiate a new authenticator. They may be NTLM implementations that require the authenticator for the first request only (actually until the third one due to the client's three-step handshake), but this is even worse because from there it becomes quite likely that any multiplexer on the route may not only break NTLM (make it renegotiate again and again) but possibly mixup sessions since subsequent requests could "steal" the session (authenticator) of the first/previous user authenticated... > > > I have only been testing one client test at-a-time so far, so probably that > was why my testing so far with proxy-initial-not-pooled and NTLM worked, > i.e., if there had been multiple clients all authenticating and going to the > same SharePoint server, and if I'm understanding what you were saying about > the requests going over the same connection that was used for the NTLM > authentication, my testing would probably have failed. > > Is that correct? > > > Now, I am really glad I asked about this (and that Eric referred me to your > "aside connection" discussion). I will have to raise this with my > colleagues, as it appears that the "proxy-initial-not-pooled" setting will > not work for any kind of production type situation? I'm afraid yes, but with MPM prefork! (see below) > > I must be doing a lot of "praying", because so far I am not able to cause a > problem, at least trying to run 3 different clients. I don't think that I > can actually get the NTLM authentications to occur simultaneously, but I'm > pretty sure the sessions are simulataneous, at least part of the time, but > even then, the pages seem for all 3 browsers seem to be appearing correctly > :(... This may be due to the small number of connections reaching different processes, rather than different threads in the same process, or are you using the prefork MPM? I should have think about "prefork" before, sorry for that (you mentioned 2.4.x which made me sadly forget about prefork), but I indeed realize now that it is very likely to work for NTLM when proxy-initial-not-pooled is used: no chance that an established backend connection gets reused underneath the current client connection (i.e. the session for NTLM). But with any other threaded MPM (event, worker, windows, ...), if you try to forcibly make httpd run with a single process (either with "StartServers 1"+"ServerLimit 1", or simply by using the -DONE_PROCESS or -X arguments on the command line), you may reach the concurrency issue quite rapidly with few client connections. So if the prefork MPM is an option for you (and it works as I assume it should), I would definitely recommend using it for NTLM, otherwise I'm afraid you are stuck with the kind of patch I proposed. Regards, Yann. - To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org For additional commands, e-mail: users-h...@httpd.apache.org
Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?
Hi, I must be doing a lot of "praying", because so far I am not able to cause a problem, at least trying to run 3 different clients. I don't think that I can actually get the NTLM authentications to occur simultaneously, but I'm pretty sure the sessions are simulataneous, at least part of the time, but even then, the pages seem for all 3 browsers seem to be appearing correctly :(... Jim On Mon, 10/26/15, o haya wrote: Subject: Re: [users@httpd] Persistent proxied connections with Apache 2.4.x? To: "users@httpd.apache.org" Cc: "O. Haya" Date: Monday, October 26, 2015, 8:57 PM Hi Yann, Thank you for responding (with lots of info!). First of all, I have to apologize for top-posting. I am using Yahoo mail, and it doesn't seem to allow quoted responses, which makes things kind of difficult (i.e., very difficult to intersperse comments since they don't mark the quoted email). First of all, as a kind of an aside remark (sorry for the "pun" :)), from my testing, it appears that if I have "ProxySet keepalive=On" inside a , then the requests to the backend all have "Connection: Keep-Alive" in the requests headers going to the backend server (a SharePoint server). Conversely if "ProxySet keepalive=Off" is inside the ..., the HTTP requests to the backend have HTTP request header "Connection: closed". In other words, the "ProxySet keepalive=On/Off" appears to be able to control whether a "Connection: keep-alive" vs. "Connection: closed" gets sent in a HTP request header to the backend. I am NOT trying to dispute what you said about the "ProxySet" being only about "TCP keepalive", but just letting you know what I was seeing during testing and also maybe I'm misunderstanding what you meant when you were saying "TCP keepalive" vs. "HTTP keepalive"? Next: I think I kind of understood about the proxy-initial-not-pooled setting ==> a new connection from the client always connected to the backend via a new Apache-to-backend connection, but I didn't realize that NTLM meant that all the requests SUBSEQUENT to the NTLM authentication had to ALSO go to the backend via the SAME connection. Is my interpretation of what you said correct? I have only been testing one client test at-a-time so far, so probably that was why my testing so far with proxy-initial-not-pooled and NTLM worked, i.e., if there had been multiple clients all authenticating and going to the same SharePoint server, and if I'm understanding what you were saying about the requests going over the same connection that was used for the NTLM authentication, my testing would probably have failed. Is that correct? Now, I am really glad I asked about this (and that Eric referred me to your "aside connection" discussion). I will have to raise this with my colleagues, as it appears that the "proxy-initial-not-pooled" setting will not work for any kind of production type situation? Thanks! Jim ----- Original Message - From: Yann Ylavic To: users@httpd.apache.org; o haya Cc: Sent: Monday, October 26, 2015 6:48 PM Subject: Re: [users@httpd] Persistent proxied connections with Apache 2.4.x? Hi Jim, sorry for the late, I'm not much online these days. On Sun, Oct 25, 2015 at 9:54 PM, o haya wrote: > > - With respect to proxying NTLM authentication, does the "aside connections" > functionality that was mentioned earlier accomplish the same thing as using > the "Proxy keepalive=On and SetEnv proxy-initial-not-pooled"? Shortly, no. "ProxySet keepalive=On" is about TCP keepalive (system probes to prevent long living TCP connections from being dropped by gateways, i.e. the socket's SO_KEEPALIVE option), and has nothing to do with HTTP keepalive (multiple HTTP requests sent on the same connection). Actually, HTTP keepalive is the default for mod_proxy_http, provided the backend is "declared" with either a ProxyPass line or a block (as opposed to eg. a RewriteRule [P]), so you don't need to configure anything special to get it (whereas on the contrary "ProxySet disablereuse=on" can be used to disable HTTP keeplive on the backend side). "SetEnv proxy-initial-not-pooled" is unfortunately not fully helpful for NTLM. It allows to always create a new connection to the backend for any new connection from the client, or said differently, it prevents an established backend connection (kept alive) from being reused in this case (see [*] below for the real goal of proxy-initial-not-pooled). But this gives no garanty on subsequent requests on the sam
Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?
Hi Yann, Thank you for responding (with lots of info!). First of all, I have to apologize for top-posting. I am using Yahoo mail, and it doesn't seem to allow quoted responses, which makes things kind of difficult (i.e., very difficult to intersperse comments since they don't mark the quoted email). First of all, as a kind of an aside remark (sorry for the "pun" :)), from my testing, it appears that if I have "ProxySet keepalive=On" inside a , then the requests to the backend all have "Connection: Keep-Alive" in the requests headers going to the backend server (a SharePoint server). Conversely if "ProxySet keepalive=Off" is inside the ..., the HTTP requests to the backend have HTTP request header "Connection: closed". In other words, the "ProxySet keepalive=On/Off" appears to be able to control whether a "Connection: keep-alive" vs. "Connection: closed" gets sent in a HTP request header to the backend. I am NOT trying to dispute what you said about the "ProxySet" being only about "TCP keepalive", but just letting you know what I was seeing during testing and also maybe I'm misunderstanding what you meant when you were saying "TCP keepalive" vs. "HTTP keepalive"? Next: I think I kind of understood about the proxy-initial-not-pooled setting ==> a new connection from the client always connected to the backend via a new Apache-to-backend connection, but I didn't realize that NTLM meant that all the requests SUBSEQUENT to the NTLM authentication had to ALSO go to the backend via the SAME connection. Is my interpretation of what you said correct? I have only been testing one client test at-a-time so far, so probably that was why my testing so far with proxy-initial-not-pooled and NTLM worked, i.e., if there had been multiple clients all authenticating and going to the same SharePoint server, and if I'm understanding what you were saying about the requests going over the same connection that was used for the NTLM authentication, my testing would probably have failed. Is that correct? Now, I am really glad I asked about this (and that Eric referred me to your "aside connection" discussion). I will have to raise this with my colleagues, as it appears that the "proxy-initial-not-pooled" setting will not work for any kind of production type situation? Thanks! Jim - Original Message - From: Yann Ylavic To: users@httpd.apache.org; o haya Cc: Sent: Monday, October 26, 2015 6:48 PM Subject: Re: [users@httpd] Persistent proxied connections with Apache 2.4.x? Hi Jim, sorry for the late, I'm not much online these days. On Sun, Oct 25, 2015 at 9:54 PM, o haya wrote: > > - With respect to proxying NTLM authentication, does the "aside connections" > functionality that was mentioned earlier accomplish the same thing as using > the "Proxy keepalive=On and SetEnv proxy-initial-not-pooled"? Shortly, no. "ProxySet keepalive=On" is about TCP keepalive (system probes to prevent long living TCP connections from being dropped by gateways, i.e. the socket's SO_KEEPALIVE option), and has nothing to do with HTTP keepalive (multiple HTTP requests sent on the same connection). Actually, HTTP keepalive is the default for mod_proxy_http, provided the backend is "declared" with either a ProxyPass line or a block (as opposed to eg. a RewriteRule [P]), so you don't need to configure anything special to get it (whereas on the contrary "ProxySet disablereuse=on" can be used to disable HTTP keeplive on the backend side). "SetEnv proxy-initial-not-pooled" is unfortunately not fully helpful for NTLM. It allows to always create a new connection to the backend for any new connection from the client, or said differently, it prevents an established backend connection (kept alive) from being reused in this case (see [*] below for the real goal of proxy-initial-not-pooled). But this gives no garanty on subsequent requests on the same connection, or worse, subsequent requests on another connection... Those may reuse any established connection in the pool, or a new connection, depending on the first one available at the time of each incoming request. In other words, mod_proxy_http handles a pool of connections for each "declared" backend independently/regardless of client-side connections or requests (basically it's a n client connections multiplexor over m backend connections), because the HTTP protocol *is* stateless. And that breaks NTLM because this protocol really authenticates connections, not requests, assuming there is one single user per connection (sigh)... > > - If not, what are the differences? So for a proxy to "work" with NTLM, it must associate a single client connection with
Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?
Hi Jim, sorry for the late, I'm not much online these days. On Sun, Oct 25, 2015 at 9:54 PM, o haya wrote: > > - With respect to proxying NTLM authentication, does the "aside connections" > functionality that was mentioned earlier accomplish the same thing as using > the "Proxy keepalive=On and SetEnv proxy-initial-not-pooled"? Shortly, no. "ProxySet keepalive=On" is about TCP keepalive (system probes to prevent long living TCP connections from being dropped by gateways, i.e. the socket's SO_KEEPALIVE option), and has nothing to do with HTTP keepalive (multiple HTTP requests sent on the same connection). Actually, HTTP keepalive is the default for mod_proxy_http, provided the backend is "declared" with either a ProxyPass line or a block (as opposed to eg. a RewriteRule [P]), so you don't need to configure anything special to get it (whereas on the contrary "ProxySet disablereuse=on" can be used to disable HTTP keeplive on the backend side). "SetEnv proxy-initial-not-pooled" is unfortunately not fully helpful for NTLM. It allows to always create a new connection to the backend for any new connection from the client, or said differently, it prevents an established backend connection (kept alive) from being reused in this case (see [*] below for the real goal of proxy-initial-not-pooled). But this gives no garanty on subsequent requests on the same connection, or worse, subsequent requests on another connection... Those may reuse any established connection in the pool, or a new connection, depending on the first one available at the time of each incoming request. In other words, mod_proxy_http handles a pool of connections for each "declared" backend independently/regardless of client-side connections or requests (basically it's a n client connections multiplexor over m backend connections), because the HTTP protocol *is* stateless. And that breaks NTLM because this protocol really authenticates connections, not requests, assuming there is one single user per connection (sigh)... > > - If not, what are the differences? So for a proxy to "work" with NTLM, it must associate a single client connection with a single one on the/each backend side, and pray for any gateway before it to do the same (yes, proxying NTLM may be hazardous, one single multiplexor on the route and things get messed up)! That's what the "aside connections" patch does, it can create connections aside from the backend pool (based on the "proxy-aside-c" environment variable, settable with SetEnv[If] or a RewriteRule, eg. when "RewriteCond %{HTTP:Authorization} ^NTLM"), and maintain them by client connection so that all the requests on this connection (setting the env var) will be routed to their associated backend connection. However I don't recall all the details of the patch proposed in 2014, I think I have a simpler/more-to-the-point one now (the previous one was meant to be generic enough to be accepted in httpd, which did not happen), so let me have a look when back home and attach it here. Regards, Yann. [*] The goal of proxy-initial-not-pooled is to help recover from a race-condition error where the proxy sends its request to the backend while the latter is in the process of closing the connection (keepalive timeout or whatever). This results in an error (502) being returned to the client, but while the client "expects" this error on kept alive connections (because of the same possible race condition on its side) and can then resend the same request, it will not do this for newly established connections, and hence proxy-initial-not-pooled prevents this error from being fatal for the "user experience" by avoiding the race on the backend side when the user-agent does not expect it. - To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org For additional commands, e-mail: users-h...@httpd.apache.org
Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?
From: o haya To: "users@httpd.apache.org" ; o haya Sent: Friday, October 23, 2015 8:03 PM Subject: Re: [users@httpd] Persistent proxied connections with Apache 2.4.x? From: o haya To: "users@httpd.apache.org" Cc: O. Haya Sent: Friday, October 23, 2015 5:02 PM Subject: Re: [users@httpd] Persistent proxied connections with Apache 2.4.x? From: Eric Covener To: users@httpd.apache.org; o haya Sent: Friday, October 23, 2015 8:14 AM Subject: Re: [users@httpd] Persistent proxied connections with Apache 2.4.x? On Thu, Oct 22, 2015 at 12:28 PM, o haya wrote: > So I am wondering if there is a way to do this (make all the backend > connections persistent with the "Connection: keepalive")? There's a patch in thread "mod_proxy's aside connections proposal" - To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org For additional commands, e-mail: users-h...@httpd.apache.org Hi Eric, Thanks for that reference! Googling that, and looking at Yann's thread about his patch, and also the bug report that preceded that patch, am I understanding things correctly, that the problem with NTLM is that all the requests involved in "an" NTLM authentication have to go down the same persistent connection (from Apache to the target (e.g., SharePoint) machine? The thing is that I thought that I've been able to get NTLM working, proxying through Apache, using "ProxySet keepalive=On" in a section, e.g.: http://sharepoint>ProxySet keepalive=OnSetEnv proxy-initial-not-pooled So does the "same persistent connection" requirement mean that if I was doing a bunch of simultaneous NTLM logins through the Apache (e.g., to http://sharepoint) that some of the login attempts would kind of randomly not work (because requests not going down same connection to the SharePoint)? Thanks,Jim Hi, I've been told that, because we use that "SetEnv proxy-initial-not-pooled", that that prevents the problem that I mentioned above. However, the only explanations of what that parameter does seem to be unrelated to this situation (they mostly refer to 502 errors?), so I don't quite understand "why?" having that setting solves the (potential) problem that I'm asking about with proxying NTLM? Can someone here explain that? Thanks,Jim Hi, No one has responded to the above yet, but maybe to be clearer, what I am curious about at this point is: - With respect to proxying NTLM authentication, does the "aside connections" functionality that was mentioned earlier accomplish the same thing as using the "Proxy keepalive=On and SetEnv proxy-initial-not-pooled"? - If not, what are the differences? Thanks,Jim
Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?
From: o haya To: "users@httpd.apache.org" Cc: O. Haya Sent: Friday, October 23, 2015 5:02 PM Subject: Re: [users@httpd] Persistent proxied connections with Apache 2.4.x? From: Eric Covener To: users@httpd.apache.org; o haya Sent: Friday, October 23, 2015 8:14 AM Subject: Re: [users@httpd] Persistent proxied connections with Apache 2.4.x? On Thu, Oct 22, 2015 at 12:28 PM, o haya wrote: > So I am wondering if there is a way to do this (make all the backend > connections persistent with the "Connection: keepalive")? There's a patch in thread "mod_proxy's aside connections proposal" - To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org For additional commands, e-mail: users-h...@httpd.apache.org Hi Eric, Thanks for that reference! Googling that, and looking at Yann's thread about his patch, and also the bug report that preceded that patch, am I understanding things correctly, that the problem with NTLM is that all the requests involved in "an" NTLM authentication have to go down the same persistent connection (from Apache to the target (e.g., SharePoint) machine? The thing is that I thought that I've been able to get NTLM working, proxying through Apache, using "ProxySet keepalive=On" in a section, e.g.: http://sharepoint>ProxySet keepalive=OnSetEnv proxy-initial-not-pooled So does the "same persistent connection" requirement mean that if I was doing a bunch of simultaneous NTLM logins through the Apache (e.g., to http://sharepoint) that some of the login attempts would kind of randomly not work (because requests not going down same connection to the SharePoint)? Thanks,Jim Hi, I've been told that, because we use that "SetEnv proxy-initial-not-pooled", that that prevents the problem that I mentioned above. However, the only explanations of what that parameter does seem to be unrelated to this situation (they mostly refer to 502 errors?), so I don't quite understand "why?" having that setting solves the (potential) problem that I'm asking about with proxying NTLM? Can someone here explain that? Thanks,Jim
Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?
From: Eric Covener To: users@httpd.apache.org; o haya Sent: Friday, October 23, 2015 8:14 AM Subject: Re: [users@httpd] Persistent proxied connections with Apache 2.4.x? On Thu, Oct 22, 2015 at 12:28 PM, o haya wrote: > So I am wondering if there is a way to do this (make all the backend > connections persistent with the "Connection: keepalive")? There's a patch in thread "mod_proxy's aside connections proposal" - To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org For additional commands, e-mail: users-h...@httpd.apache.org Hi Eric, Thanks for that reference! Googling that, and looking at Yann's thread about his patch, and also the bug report that preceded that patch, am I understanding things correctly, that the problem with NTLM is that all the requests involved in "an" NTLM authentication have to go down the same persistent connection (from Apache to the target (e.g., SharePoint) machine? The thing is that I thought that I've been able to get NTLM working, proxying through Apache, using "ProxySet keepalive=On" in a section, e.g.: http://sharepoint>ProxySet keepalive=OnSetEnv proxy-initial-not-pooled So does the "same persistent connection" requirement mean that if I was doing a bunch of simultaneous NTLM logins through the Apache (e.g., to http://sharepoint) that some of the login attempts would kind of randomly not work (because requests not going down same connection to the SharePoint)? Thanks,Jim
Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?
On Thu, Oct 22, 2015 at 12:28 PM, o haya wrote: > So I am wondering if there is a way to do this (make all the backend > connections persistent with the "Connection: keepalive")? There's a patch in thread "mod_proxy's aside connections proposal" - To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org For additional commands, e-mail: users-h...@httpd.apache.org
Re: [users@httpd] Persistent proxied connections with Apache 2.4.x?
> I am wondering if there is a way to do this (make all the backend connections > persistent with the "Connection: keepalive")? IMO there's no way to do it without enumerating all your backend hosts. Apache needs to create a separate worker for each host at startup time. > "Connection: keepalive" in the request headers Just in case you are actually looking at the headers, it should be "Connection: keep-alive" in the obsolete HTTP 1.0, and they are all persistent by default in HTTP 1.1 unless "Connection: close" is specified by any side. If SharePoint follows any standards, that is. -- With Best Regards, Marat Khalili On October 22, 2015 7:28:20 PM GMT+03:00, o haya wrote: >Hi, >We want to use Apache 2.4.x to proxy through to some SharePoint >instances using NTLM logins. From testing, it looks like the >Apache-to-SharePoint connections need to be persistent, with >"Connection: keepalive" in the request headers going from >Apache-to-SharePoint. >We can do this using a directive: >ProxySet keepalive=On >inside a section, but we would like to do this (enable >keepalive on the backend connections) on all of the backend >connections. But, if we try to put that ProxySet inside a , we >get a startup error saying that it needs an absolute URL. >So I am wondering if there is a way to do this (make all the backend >connections persistent with the "Connection: keepalive")? >Thanks,Jim