Re: Tunneling non-HTTP through a web proxy with HttpClient
Here's a thought - provide get/setSocket in HttpConnection but make them protected. Then those who want to use them have to work harder (ie make a subclass) and will probably be less likely to use them incorrectly. Joe average user is unlikely to confused by them or use them this way. /Mike Sample - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tunneling non-HTTP through a web proxy with HttpClient
Agreed. This would be a better choice, as least for the short term. Mike On Mar 28, 2004, at 5:55 PM, [EMAIL PROTECTED] wrote: Here's a thought - provide get/setSocket in HttpConnection but make them protected. Then those who want to use them have to work harder (ie make a subclass) and will probably be less likely to use them incorrectly. Joe average user is unlikely to confused by them or use them this way. /Mike Sample - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tunneling non-HTTP through a web proxy with HttpClient
On Fri, 26 Mar 2004 13:18:02 -0500, Michael Becke wrote: As you have guessed by now, I agree that not the socket itself but only the streams should be made available. Yes, I think this is the big question. Is having access to the I/O streams enough or would someone need access to the actual Socket? If it's just the streams then I think we can probably do this now. If we need access to the socket then we need to rethink things a little. Hi Guys, My 2 cents on this is that it should be the Socket, not i/o streams, although more work is involved and there is a control issue at stake (who owns the Socket, making it null in the HttpConnection to detach it etc. etc). By making the Socket available, you can almost guarantee easy intergration with any Java app that uses Sockets for some protocol, but needs to be augmented to get through a web proxy. How about this approach: for the existing version, a quick get/setSocket on HttpConnection with the advisory to be careful and that things may change for 4.0. In 4.0 more care can go into the design wrt control issues etc. but I would still recommend some way to get at the Socket in 4.0. Augmenting the Protocol class for those where isSecure() returns true to have a SocketFactory for the first hop would be great too. This could be added via a new Protocol constructor (so existing stuff doesn't break) and by making the existing Protocol ctor use the DefaultProtocolSocketFactory for the first hop by default (as is currently embedded in the HttpConnection code, I believe). Add this to the method to get this socket factory I mentioned before. I'd be happy to add these changes and send them to someone to check in (2 files for the Protocol stuff). Of course, I have an ulterior motive - I would like to contribute my changes to JXTA for enabling their TCP protocol to tunnel through web proxies using HttpClient. Without access to a Socket from HttpClient, I would have to offer the JXTA changes as well as a hacked version of HttpClient - not likely to help too many people that way. Sorry for a delayed reply - I just got back from a short ski trip on the Wapta Icefield (Banff National Park, Rockies). Fun! Regards, /Mike Sample - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tunneling non-HTTP through a web proxy with HttpClient
Hi Mike Mike Michael Becke wrote: Given that HttpClient is in the business of communicating using HTTP, it is not terribly well suited for other purposes. Since CONNECT is also an HTTP method, it is not totally out of scope for the HttpClient. From RFC 2616, section 9.9: This specification reserves the method name CONNECT for use with a proxy that can dynamically switch to being a tunnel (e.g. SSL tunneling [44]). Maybe we can consider official support for CONNECT as a feature for the big 4.0 API overhaul. cheers, Roland
RE: Tunneling non-HTTP through a web proxy with HttpClient
All Mikes, Roland, and all I believe it _might_ be possible to implement a generic SSL tunnelling mechanism using stock version of HttpClient. The only API change it may requite is adding protected HttpConnection#getSocket method. I may try to hack something up this weekend. Oleg -Original Message- From: Michael Becke [mailto:[EMAIL PROTECTED] Sent: Friday, March 26, 2004 14:44 To: Commons HttpClient Project Subject: Re: Tunneling non-HTTP through a web proxy with HttpClient Hi Mike Mike :) Since CONNECT is also an HTTP method, it is not totally out of scope for the HttpClient. From RFC 2616, section 9.9: Agreed. CONNECT is definitely an HTTP method. The only question is if we should support it's use in non-HTTP contexts. Maybe we can consider official support for CONNECT as a feature for the big 4.0 API overhaul. It seems like it could be a good addition. My only hesitation is in giving direct access to the Socket. This seems like a pretty major departure from our current system. Mike - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] *** The information in this email is confidential and may be legally privileged. Access to this email by anyone other than the intended addressee is unauthorized. If you are not the intended recipient of this message, any review, disclosure, copying, distribution, retention, or any action taken or omitted to be taken in reliance on it is prohibited and may be unlawful. If you are not the intended recipient, please reply to or forward a copy of this message to the sender and delete the message, any attachments, and any copies thereof from your system. *** - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tunneling non-HTTP through a web proxy with HttpClient
Hello Mike, (for 4.0) we can wrap input and output streams around those of the underlying socket. Connection reuse is not an option for that method anyway, and by help of the wrapper we can cut any references to the underlying socket's input and output streams as soon as the connection manager gets back the connection. Of course, that is only required if the socket doesn't create new input and output streams for each connection anyway. As you have guessed by now, I agree that not the socket itself but only the streams should be made available. cheers, Roland Michael Becke [EMAIL PROTECTED] 26.03.2004 14:44 Please respond to Commons HttpClient Project To: Commons HttpClient Project [EMAIL PROTECTED] cc: Subject:Re: Tunneling non-HTTP through a web proxy with HttpClient Hi Mike Mike :) Since CONNECT is also an HTTP method, it is not totally out of scope for the HttpClient. From RFC 2616, section 9.9: Agreed. CONNECT is definitely an HTTP method. The only question is if we should support it's use in non-HTTP contexts. Maybe we can consider official support for CONNECT as a feature for the big 4.0 API overhaul. It seems like it could be a good addition. My only hesitation is in giving direct access to the Socket. This seems like a pretty major departure from our current system. Mike - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Tunneling non-HTTP through a web proxy with HttpClient
Hi Oleg, I guess it's just a typo, but to be sure: The CONNECT method is for establishing a generic tunnel through an HTTP proxy. SSL tunneling is just one application of that. cheers, Roland Kalnichevski, Oleg [EMAIL PROTECTED] 26.03.2004 14:49 Please respond to Commons HttpClient Project To: Commons HttpClient Project [EMAIL PROTECTED] cc: Subject:RE: Tunneling non-HTTP through a web proxy with HttpClient All Mikes, Roland, and all I believe it _might_ be possible to implement a generic SSL tunnelling mechanism using stock version of HttpClient. The only API change it may requite is adding protected HttpConnection#getSocket method. I may try to hack something up this weekend. Oleg -Original Message- From: Michael Becke [mailto:[EMAIL PROTECTED] Sent: Friday, March 26, 2004 14:44 To: Commons HttpClient Project Subject: Re: Tunneling non-HTTP through a web proxy with HttpClient Hi Mike Mike :) Since CONNECT is also an HTTP method, it is not totally out of scope for the HttpClient. From RFC 2616, section 9.9: Agreed. CONNECT is definitely an HTTP method. The only question is if we should support it's use in non-HTTP contexts. Maybe we can consider official support for CONNECT as a feature for the big 4.0 API overhaul. It seems like it could be a good addition. My only hesitation is in giving direct access to the Socket. This seems like a pretty major departure from our current system. Mike - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] *** The information in this email is confidential and may be legally privileged. Access to this email by anyone other than the intended addressee is unauthorized. If you are not the intended recipient of this message, any review, disclosure, copying, distribution, retention, or any action taken or omitted to be taken in reliance on it is prohibited and may be unlawful. If you are not the intended recipient, please reply to or forward a copy of this message to the sender and delete the message, any attachments, and any copies thereof from your system. *** - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Tunneling non-HTTP through a web proxy with HttpClient
You are right. HttpClient uses tunnelling only when doing HTTPS. Hence the confusion. -Original Message- From: Roland Weber [mailto:[EMAIL PROTECTED] Sent: Friday, March 26, 2004 15:03 To: Commons HttpClient Project Subject: RE: Tunneling non-HTTP through a web proxy with HttpClient Hi Oleg, I guess it's just a typo, but to be sure: The CONNECT method is for establishing a generic tunnel through an HTTP proxy. SSL tunneling is just one application of that. cheers, Roland Kalnichevski, Oleg [EMAIL PROTECTED] 26.03.2004 14:49 Please respond to Commons HttpClient Project To: Commons HttpClient Project [EMAIL PROTECTED] cc: Subject:RE: Tunneling non-HTTP through a web proxy with HttpClient All Mikes, Roland, and all I believe it _might_ be possible to implement a generic SSL tunnelling mechanism using stock version of HttpClient. The only API change it may requite is adding protected HttpConnection#getSocket method. I may try to hack something up this weekend. Oleg -Original Message- From: Michael Becke [mailto:[EMAIL PROTECTED] Sent: Friday, March 26, 2004 14:44 To: Commons HttpClient Project Subject: Re: Tunneling non-HTTP through a web proxy with HttpClient Hi Mike Mike :) Since CONNECT is also an HTTP method, it is not totally out of scope for the HttpClient. From RFC 2616, section 9.9: Agreed. CONNECT is definitely an HTTP method. The only question is if we should support it's use in non-HTTP contexts. Maybe we can consider official support for CONNECT as a feature for the big 4.0 API overhaul. It seems like it could be a good addition. My only hesitation is in giving direct access to the Socket. This seems like a pretty major departure from our current system. Mike - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] *** The information in this email is confidential and may be legally privileged. Access to this email by anyone other than the intended addressee is unauthorized. If you are not the intended recipient of this message, any review, disclosure, copying, distribution, retention, or any action taken or omitted to be taken in reliance on it is prohibited and may be unlawful. If you are not the intended recipient, please reply to or forward a copy of this message to the sender and delete the message, any attachments, and any copies thereof from your system. *** - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] *** The information in this email is confidential and may be legally privileged. Access to this email by anyone other than the intended addressee is unauthorized. If you are not the intended recipient of this message, any review, disclosure, copying, distribution, retention, or any action taken or omitted to be taken in reliance on it is prohibited and may be unlawful. If you are not the intended recipient, please reply to or forward a copy of this message to the sender and delete the message, any attachments, and any copies thereof from your system. *** - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Tunneling non-HTTP through a web proxy with HttpClient
I have looked at HttpClient and it does not seem to offer a way to use it as simply a means of doing the CONNECT and proxy auth stuff and then returning the resulting Socket. I would like to do this to tunnel a non-HTTP protocol, specifically the JXTA TCP-based protocol (not JXTA's HTTP stuff). Is there an way to do this with the existing HttpClient API? The fact that HttpConnection holds the Socket as private instance variable and provides no get/setters suggests that there is not. If not, what do you think of the following suggestions to make HttpClient work for to this type of use. The design below is geared for the minimal distrubance of the existing HttpClient API and is not the cleanest way to do it. Changes to HttpClient: 1. augment the Protocol class to include a getProxySocketFactory() method and make HttpConnection class use this method instead of creating DefaultProtocolSocketFactory for secure and proxied connections. The Protocol constructor that had the SecureProtcolSocketFactory arg also needs an additional arg to specifiy this new factory. 2. add public getSocket() and setSocket() methods to the HttpConnection class. 3. Provide a NullMethod class whose execute() method does nothing but keep the HttpConnection parameter for later retrival via getter on the NullMethod object. 4. Provide a NoOpSecureProtocolSocketFactory whose create(socket,...) method simply returns back the socket argument. To use the above setup to tunnel through a web proxy you'd do the following - create/register a Protocol object that returns NoOpSecureProtocolSocketFactory from its getSocketFactory and returns a DefaultProtocolSocketFactory or other for the new getProxySocketFactory() method described above. In JXTA'S case the proxySocketFactory would not be the default one from HttpClient in order to leverage Jxta's socket creation class for the hop to the web proxy). - create NullMethod object - create HttpClient - create HostConfiguration with appropriate proxy info - create HttpState and add appropriate authentication credentials - create SimpleHttpConnectionManager - call httpClient.executeMethod( hostConfiguration, nullMethod, httpState) - get HttpConnection from NullMethod (NullMethod captures httpConnection on its execute() call) - get Socket from HttpConnection and stash it, then setSocket( null ) on HttpConnection so releasing the connection does not close it. This socket is the goal of the whole exercise. - httpState.getConnectinManger.releaseConnection(); What do you think? Thanks in advance, /Mike Sample - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tunneling non-HTTP through a web proxy with HttpClient
Hi Mike, Given that HttpClient is in the business of communicating using HTTP, it is not terribly well suited for other purposes. Having said that you should be able to hack something that works. To get access to the secure tunneled socket I would recommend trying the following: 1) Create a NULL method as you describe. It should do basically nothing except return a 200 status. 2) Change HttpConnection to add a method for getting the socket. 3) Create a custom http connection factory for managing the custom HttpConnections and their sockets. With these three items you should be able to execute proxied methods as normal. After executing a NULL method the connection manager would have access to the HttpConnection and socket to use as needed. The only core HttpClient class you would need to change is HttpConnection I think. Mike On Mar 25, 2004, at 5:30 PM, [EMAIL PROTECTED] wrote: I have looked at HttpClient and it does not seem to offer a way to use it as simply a means of doing the CONNECT and proxy auth stuff and then returning the resulting Socket. I would like to do this to tunnel a non-HTTP protocol, specifically the JXTA TCP-based protocol (not JXTA's HTTP stuff). Is there an way to do this with the existing HttpClient API? The fact that HttpConnection holds the Socket as private instance variable and provides no get/setters suggests that there is not. If not, what do you think of the following suggestions to make HttpClient work for to this type of use. The design below is geared for the minimal distrubance of the existing HttpClient API and is not the cleanest way to do it. Changes to HttpClient: 1. augment the Protocol class to include a getProxySocketFactory() method and make HttpConnection class use this method instead of creating DefaultProtocolSocketFactory for secure and proxied connections. The Protocol constructor that had the SecureProtcolSocketFactory arg also needs an additional arg to specifiy this new factory. 2. add public getSocket() and setSocket() methods to the HttpConnection class. 3. Provide a NullMethod class whose execute() method does nothing but keep the HttpConnection parameter for later retrival via getter on the NullMethod object. 4. Provide a NoOpSecureProtocolSocketFactory whose create(socket,...) method simply returns back the socket argument. To use the above setup to tunnel through a web proxy you'd do the following - create/register a Protocol object that returns NoOpSecureProtocolSocketFactory from its getSocketFactory and returns a DefaultProtocolSocketFactory or other for the new getProxySocketFactory() method described above. In JXTA'S case the proxySocketFactory would not be the default one from HttpClient in order to leverage Jxta's socket creation class for the hop to the web proxy). - create NullMethod object - create HttpClient - create HostConfiguration with appropriate proxy info - create HttpState and add appropriate authentication credentials - create SimpleHttpConnectionManager - call httpClient.executeMethod( hostConfiguration, nullMethod, httpState) - get HttpConnection from NullMethod (NullMethod captures httpConnection on its execute() call) - get Socket from HttpConnection and stash it, then setSocket( null ) on HttpConnection so releasing the connection does not close it. This socket is the goal of the whole exercise. - httpState.getConnectinManger.releaseConnection(); What do you think? Thanks in advance, /Mike Sample - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tunneling non-HTTP through a web proxy with HttpClient
Thanks for the confirming my plans. I appreciate the feedback. Would the HttpClient project consider including the get/setSocket changes? The Protocol changes would be nice to avoid the hardwiring of the connection to the web proxy but I can live with out that. The Jxta code does some clever use of reflection so they can get the best of JDK 1.4 without needing to be compiled with 1.4. Thanks /Mike Sample - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
