Re: HTTP Post and HTTP/100 (continue)
Simon I'd really appreciate it if you could send us the debug trace for analysis. Please refer to the following url for instructions on how wire log can be activated: http://jakarta.apache.org/commons/httpclient/logging.html Your problem should be easily solvable by disabling 100-continue handshake. PostMethod myhttppost = new PostMethod(); myhttppost.setUseExpectHeader(false); Cheers Oleg On Tue, 2003-02-25 at 00:29, Simon Roberts wrote: Gidday, This is probably a dumb-user question, but if it is, then it might need to be documented for other dumb users :) I just checked out the latest CVS HttpClient and tried it with my application (it's using HEAD from a month or two ago), and am having a problem. Our app does HTTP POST (to a Jetty server, as it happens). Previously, httpclient used to just push the request header and body along in one lump, and when the server posted a http-100 (continue) status, it used to complain about continue received, but body already sent. Anyway, it used to work okay for us... Now, the httpClient.execute(method) returns 100, and there is no response (body) from the server, so my app barfs. The question is, how do I make httpclient send the body of the request? (maybe after I get the 100 back). Cheers, thanks, Simon - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: HTTP Post and HTTP/100 (continue)
Aurelien, Same request. Could you please produce a wire log of the HTTP communication that causes the problem you mentioned? Oleg On Tue, 2003-02-25 at 11:17, Aurelien Pernoud wrote: Hey that may be the trouble I'm meeting too and I assumed it to be related to MultiThreadedHttpConnectionManager... Cause the test case I've made locally uses easy post method, and goes fine, but in my apps there are sometimes large post, and maybe they uses the HTTP/100 as I saw this in log : [21 févr. 2003 15:47:07 WARN] - Received status CONTINUE but the body has already been sent [21 févr. 2003 15:47:08 WARN] - Received status CONTINUE but the body has already been sent (...) [21 févr. 2003 15:47:16 WARN] - Recoverable exception caught when reading response [21 févr. 2003 15:47:16 ERROR] - org.apache.commons.httpclient.HttpRecoverableException: org.apache.commons.httpclient.HttpRecoverableException: Error in parsing the status line from the response: unable to find line starting with HTTP/ at org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodBase.jav a:1733) at org.apache.commons.httpclient.HttpMethodBase.writeRemainingRequestBody(HttpM ethodBase.java:2377) at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:963 ) I'll try disabling the feature as you said, see what happens. Oleg Kalnichevski a écrit : Simon I'd really appreciate it if you could send us the debug trace for analysis. Please refer to the following url for instructions on how wire log can be activated: http://jakarta.apache.org/commons/httpclient/logging.html Your problem should be easily solvable by disabling 100-continue handshake. PostMethod myhttppost = new PostMethod(); myhttppost.setUseExpectHeader(false); Cheers Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: HTTP Post and HTTP/100 (continue)
Sure, I'll try to reproduce it with wire log, as it doesn't happen on every request :(( Oleg Kalnichevski a écrit : Aurelien, Same request. Could you please produce a wire log of the HTTP communication that causes the problem you mentioned? Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: HTTP Post and HTTP/100 (continue)
I'm sorry I have trouble enabling wire log within my webapp. I'm using Turbine and in fact it seems HttpClient took the setting from turbine to log it's info. I can't find an easy way to enable wirelog in a specific file... I'm not used to commons logging. Aurelien Pernoud a écrit : Sure, I'll try to reproduce it with wire log, as it doesn't happen on every request :(( Oleg Kalnichevski a écrit : Aurelien, Same request. Could you please produce a wire log of the HTTP communication that causes the problem you mentioned? Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
DO NOT REPLY [Bug 13463] - Request/Response race condition when doing multiple requests on the same connection.
DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT http://nagoya.apache.org/bugzilla/show_bug.cgi?id=13463. ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND INSERTED IN THE BUG DATABASE. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=13463 Request/Response race condition when doing multiple requests on the same connection. --- Additional Comments From [EMAIL PROTECTED] 2003-02-25 15:00 --- Thanks, Sam. You're exactly correct. If I modify my test case server to simply flush its output rather than close it, everything works perfectly. It begs the question, however, should this client fully support HTTP 1.1 servers which don't necessarily maintain a persistent connection? The HTTP 1.1 RFC says the servers SHOULD meet this requirement, not that they MUST meet it. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: HTTP Post and HTTP/100 (continue)
Here is stuff I got from a Post Method, after removing the log of method : [25 févr. 2003 16:05:31 DEBUG] - User-Agent: Jakarta Commons-HttpClient/2.0alpha2 [\r\n] [25 févr. 2003 16:05:31 DEBUG] - Host: localhost:8080 [\r\n] [25 févr. 2003 16:05:31 DEBUG] - Cookie: $Version=0; JSESSIONID=6ECE869D6914A8D1E9E5B22813CAEC52; $Path=/generatedApp [\r\n] [25 févr. 2003 16:05:31 DEBUG] - Content-Length: 178 [\r\n] [25 févr. 2003 16:05:31 DEBUG] - Expect: 100-continue [\r\n] [25 févr. 2003 16:05:31 DEBUG] - Content-Type: application/x-www-form-urlencoded [\r\n] [25 févr. 2003 16:05:31 DEBUG] - [\r\n] [25 févr. 2003 16:05:31 DEBUG] - Expecting response [25 févr. 2003 16:05:34 DEBUG] - Waiting for response timeout [25 févr. 2003 16:05:34 DEBUG] - Response not available. Send the request body [25 févr. 2003 16:05:34 DEBUG] - Request body sent [25 févr. 2003 16:05:34 DEBUG] - And I think that's were the missing HTTP/ exception came, but I'm not sure BTW, I think you should remove the log enter HttpConnection.responseAvaliable(), cause I have like 50.000 lines of it in my log for only one request... :/ Aurelien Pernoud a écrit : I'm sorry I have trouble enabling wire log within my webapp. I'm using Turbine and in fact it seems HttpClient took the setting from turbine to log it's info. I can't find an easy way to enable wirelog in a specific file... I'm not used to commons logging. Aurelien Pernoud a écrit : Sure, I'll try to reproduce it with wire log, as it doesn't happen on every request :(( Oleg Kalnichevski a écrit : Aurelien, Same request. Could you please produce a wire log of the HTTP communication that causes the problem you mentioned? Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
2.0 alpha3 rc4 ready
Hopefully the last release candidate for alpha3, rc4, is ready: http://jakarta.apache.org/builds/jakarta-commons/release/commons-httpclient/v2.0 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: HTTP Post and HTTP/100 (continue)
I'm using Tomcat 4.1.18, and my log for Turbine is set to debug (not that crazy ;)), but httpclient is either logging everything, either nothing... very weird. I'll have to look further into it sometime. What you say is weird, I have another line where my post went ok : Oleg Kalnichevski a écrit : Aurelien This is the expected behavior with non-compliant HTTP servers. The HTTP server apparently does not respond with 100-continue as expected. HttpClient resumes sending the request body after 3 second timeout. What HTTP server are working with? It appears to have issues with HTTP 1.1 spec compliance. As to excessive noise in the logs, try using debug verbosity instead trace one. Oleg On Tue, 2003-02-25 at 16:18, Aurelien Pernoud wrote: Here is stuff I got from a Post Method, after removing the log of method : [25 févr. 2003 16:05:31 DEBUG] - User-Agent: Jakarta Commons-HttpClient/2.0alpha2 [\r\n] [25 févr. 2003 16:05:31 DEBUG] - Host: localhost:8080 [\r\n] [25 févr. 2003 16:05:31 DEBUG] - Cookie: $Version=0; JSESSIONID=6ECE869D6914A8D1E9E5B22813CAEC52; $Path=/generatedApp [\r\n] [25 févr. 2003 16:05:31 DEBUG] - Content-Length: 178 [\r\n] [25 févr. 2003 16:05:31 DEBUG] - Expect: 100-continue [\r\n] [25 févr. 2003 16:05:31 DEBUG] - Content-Type: application/x-www-form-urlencoded [\r\n] [25 févr. 2003 16:05:31 DEBUG] - [\r\n] [25 févr. 2003 16:05:31 DEBUG] - Expecting response [25 févr. 2003 16:05:34 DEBUG] - Waiting for response timeout [25 févr. 2003 16:05:34 DEBUG] - Response not available. Send the request body [25 févr. 2003 16:05:34 DEBUG] - Request body sent [25 févr. 2003 16:05:34 DEBUG] - And I think that's were the missing HTTP/ exception came, but I'm not sure BTW, I think you should remove the log enter HttpConnection.responseAvaliable(), cause I have like 50.000 lines of it in my log for only one request... :/ - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: HTTP Post and HTTP/100 (continue)
Sorry the fiest mail went out too soon... I'm using Tomcat 4.1.18, and my log for Turbine is set to debug (not that crazy ;)), but httpclient is either logging everything, either nothing... very weird. I'll have to look further into it sometime. What you say is weird (to me), I have another line where my post went ok : POST /generatedApp/servlet HTTP/1.1 [\r\n] Adding Host request header User-Agent: Jakarta Commons-HttpClient/2.0alpha2 [\r\n] Host: localhost:8080 [\r\n] Content-Length: 65 [\r\n] Expect: 100-continue [\r\n] Content-Type: application/x-www-form-urlencoded [\r\n] [\r\n] Expecting response Response available HTTP/1.1 100 Continue [\r\n] In fact the only difference with the other log is that the other timed out waiting for response... : Expecting response Waiting for response timeout Response not available. Send the request body Request body sent And I don't understand what happens exactly in this case, either in HttpClient nore on tomcat. You say HttpClient resumes sending the request after timeout, but here you see I don't have any after timeout, I have a Is it good ? I'll try disabling the 100 continue cause I think it's the reason of my trouble. Oleg Kalnichevski a écrit : Aurelien This is the expected behavior with non-compliant HTTP servers. The HTTP server apparently does not respond with 100-continue as expected. HttpClient resumes sending the request body after 3 second timeout. What HTTP server are working with? It appears to have issues with HTTP 1.1 spec compliance. As to excessive noise in the logs, try using debug verbosity instead trace one. Oleg On Tue, 2003-02-25 at 16:18, Aurelien Pernoud wrote: Here is stuff I got from a Post Method, after removing the log of method : [25 févr. 2003 16:05:31 DEBUG] - User-Agent: Jakarta Commons-HttpClient/2.0alpha2 [\r\n] [25 févr. 2003 16:05:31 DEBUG] - Host: localhost:8080 [\r\n] [25 févr. 2003 16:05:31 DEBUG] - Cookie: $Version=0; JSESSIONID=6ECE869D6914A8D1E9E5B22813CAEC52; $Path=/generatedApp [\r\n] [25 févr. 2003 16:05:31 DEBUG] - Content-Length: 178 [\r\n] [25 févr. 2003 16:05:31 DEBUG] - Expect: 100-continue [\r\n] [25 févr. 2003 16:05:31 DEBUG] - Content-Type: application/x-www-form-urlencoded [\r\n] [25 févr. 2003 16:05:31 DEBUG] - [\r\n] [25 févr. 2003 16:05:31 DEBUG] - Expecting response [25 févr. 2003 16:05:34 DEBUG] - Waiting for response timeout [25 févr. 2003 16:05:34 DEBUG] - Response not available. Send the request body [25 févr. 2003 16:05:34 DEBUG] - Request body sent [25 févr. 2003 16:05:34 DEBUG] - And I think that's were the missing HTTP/ exception came, but I'm not sure BTW, I think you should remove the log enter HttpConnection.responseAvaliable(), cause I have like 50.000 lines of it in my log for only one request... :/ - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: HTTP Post and HTTP/100 (continue)
Aurelien, Something is fishy about your setup. I have developed 100-continue handshake support for HttpClient using Tomcat 4.1.18. It does handle 100-continue correctly. I may need to see the complete log of yours in order to figure out what is going on there. The only theory I can come up with at the moment is that it takes Tomcat more 3 seconds to send 100-continue response out (which would be quite weird). Are you using Log4j by any chance? If you do, that explains why you keep on getting all TRACE entries in your logs. As Log4J does not support TRACE verbosity, trace events are printed when DEBUG priority is on Cheers Oleg On Tue, 2003-02-25 at 16:48, Aurelien Pernoud wrote: I'm using Tomcat 4.1.18, and my log for Turbine is set to debug (not that crazy ;)), but httpclient is either logging everything, either nothing... very weird. I'll have to look further into it sometime. What you say is weird, I have another line where my post went ok : Oleg Kalnichevski a écrit : Aurelien This is the expected behavior with non-compliant HTTP servers. The HTTP server apparently does not respond with 100-continue as expected. HttpClient resumes sending the request body after 3 second timeout. What HTTP server are working with? It appears to have issues with HTTP 1.1 spec compliance. As to excessive noise in the logs, try using debug verbosity instead trace one. Oleg On Tue, 2003-02-25 at 16:18, Aurelien Pernoud wrote: Here is stuff I got from a Post Method, after removing the log of method : [25 févr. 2003 16:05:31 DEBUG] - User-Agent: Jakarta Commons-HttpClient/2.0alpha2 [\r\n] [25 févr. 2003 16:05:31 DEBUG] - Host: localhost:8080 [\r\n] [25 févr. 2003 16:05:31 DEBUG] - Cookie: $Version=0; JSESSIONID=6ECE869D6914A8D1E9E5B22813CAEC52; $Path=/generatedApp [\r\n] [25 févr. 2003 16:05:31 DEBUG] - Content-Length: 178 [\r\n] [25 févr. 2003 16:05:31 DEBUG] - Expect: 100-continue [\r\n] [25 févr. 2003 16:05:31 DEBUG] - Content-Type: application/x-www-form-urlencoded [\r\n] [25 févr. 2003 16:05:31 DEBUG] - [\r\n] [25 févr. 2003 16:05:31 DEBUG] - Expecting response [25 févr. 2003 16:05:34 DEBUG] - Waiting for response timeout [25 févr. 2003 16:05:34 DEBUG] - Response not available. Send the request body [25 févr. 2003 16:05:34 DEBUG] - Request body sent [25 févr. 2003 16:05:34 DEBUG] - And I think that's were the missing HTTP/ exception came, but I'm not sure BTW, I think you should remove the log enter HttpConnection.responseAvaliable(), cause I have like 50.000 lines of it in my log for only one request... :/ - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: HTTP Post and HTTP/100 (continue)
Oleg Kalnichevski a écrit : Aurelien, Something is fishy about your setup. I have developed 100-continue handshake support for HttpClient using Tomcat 4.1.18. It does handle 100-continue correctly. I may need to see the complete log of yours in order to figure out what is going on there. The only theory I can come up with at the moment is that it takes Tomcat more 3 seconds to send 100-continue response out (which would be quite weird). Well I resent my mail including the log cause my first mail (the one you responded) went out too soon, do you need more log ? What is moreover a trouble is that the connection wasn't released after the failed post. Indeed as I call method.releaseConnection() and that response Inputstream is empty, I didn't see anything like HttpConnectionManager.releaseConnection: Release connection for [EMAIL PROTECTED] after that request, whereas every other have it. Should I call myself the releaseconnection(connection used for my request) on connectionmanager, after my request ended or not ? Is it safe ? Are you using Log4j by any chance? If you do, that explains why you keep on getting all TRACE entries in your logs. As Log4J does not support TRACE verbosity, trace events are printed when DEBUG priority is on You've got it, I'm using Log4J... Well I'll see what I can do then. Aurelien - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: HTTP Post and HTTP/100 (continue)
Ok I'll try it without the Expect: header tomorrow. But what I can surely say is that I didn't stress Tomcat, I was alone making single requests on it, not even simultaneously. Of course the logging did stress a little, as it logged like 4MB in 30 seconds... I've attached a log more precise on where httpclient went wrong, including trace (thanks to log4j in fact). I have more log of what happened before if you want. I've commented it a little, if you have comments/responses on what I've written please mail me back, as I'm not so familiar with HTTP protocol :) Thanx again for your help, Aurelien Oleg Kalnichevski a écrit : Aurelien, it looks like you have stressed Tomcat to a point that it failed to respond to post requests within 3 sec. Just for a heck of it, try disabling Expect: 100-continue handshake and see if that makes any difference. The problem with connection not being released is troubling. You should file a new bug report for this problem. Cheers Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[PATCH]Re: HTTP Post and HTTP/100 (continue)
What is moreover a trouble is that the connection wasn't released after the failed post. Indeed as I call method.releaseConnection() and that response Inputstream is empty, I didn't see anything like HttpConnectionManager.releaseConnection: Release connection for [EMAIL PROTECTED] after that request, whereas every other have it. You are correct. It seems if an error occurs in HttpMethodBase.execute() before the response stream is set the connection will not be released. The attached patch should fix this. I will go ahead and commit this one after alpha 3 if all agree. Mike Index: java/org/apache/commons/httpclient/HttpMethodBase.java === RCS file: /home/cvspublic/jakarta-commons/httpclient/src/java/org/apache/commons/httpclient/HttpMethodBase.java,v retrieving revision 1.116 diff -u -r1.116 HttpMethodBase.java --- java/org/apache/commons/httpclient/HttpMethodBase.java 25 Feb 2003 02:10:16 - 1.116 +++ java/org/apache/commons/httpclient/HttpMethodBase.java 25 Feb 2003 19:11:54 - @@ -1188,7 +1188,9 @@ // attempting cleanup, don't care about exception. } } - +// Make sure the connection has been released. If the response stream +// has not been set, this is the only way to release the connection. +ensureConnectionRelease(); } /** - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [VOTE] promote 2.0 alpha3rc4 to alpha3 release
+1 Jeffrey Dever wrote: Propose to promote the current 2.0 Alpha 3 release candidate 4 to the offical release of 2.0 Alpha 3. The next release will be 2.0 Beta 1 which will commence after all Beta 1 targeted bugs are closed. +1 yes +/- 0 abstain -1 no http://jakarta.apache.org/site/decisions.html - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Significant HttpClient HttpMethodBase overhaul. Need earlyfeedback
Folks
I am currently working on a patch enabling HttpClient to handle
cross-site redirects.
http://nagoya.apache.org/bugzilla/show_bug.cgi?id=16729
In order to lay foundation for this capability I needed to make quite a
few changes to HttpClient HttpMethodBase. I have opted for a more
substantial overhaul of these classes, than was strictly necessary. I
realize not all of you may agree with my decision. So, I decided to seek
an early feedback from you to make sure I do not go completely astray.
This is what I have done:
I moved complete redirect authenticate logic from HttpMethodBase to
HttpClient. HttpMethodBase
Impact:
- Even though binary interface is unchanged, HttpClient's modus operandi
with regard to redirect authentication changed substantially. People
like Laura Werner,who do not use standard HttpClient and have developed
their own logic around lower level classes will be affected most.
- Cleaner design. Redirect authentication in my opinion logically do
not belong to domain of the HTTP method, rather, they belong to that of
the HTTP agent.
- Over-convoluted HttpMethodBase class got simpler. Under-used
HttpClient class is leveraged more. This is an important architectural
improvement in my humble opinion. If you disagree, please let me know
- I am seriously concerned that this redesign may have adversely
affected connection pooling stuff. Mike, Eric, you are the connection
pooling experts, could you please give me your opinion on that?
- About a dozen of test cases have become obsolete. They will need to be
redesigned. They are all commented out for the time being
As always, any feedback, including that in a form of bad tomatoes thrown
at me will be appreciated
Please note, that cross-site redirect has not been implemented yet.
Cheers
Oleg
Index: java/org/apache/commons/httpclient/HttpClient.java
===
RCS file: /home/cvspublic/jakarta-commons/httpclient/src/java/org/apache/commons/httpclient/HttpClient.java,v
retrieving revision 1.69
diff -u -r1.69 HttpClient.java
--- java/org/apache/commons/httpclient/HttpClient.java 21 Feb 2003 13:48:09 - 1.69
+++ java/org/apache/commons/httpclient/HttpClient.java 25 Feb 2003 19:49:47 -
-63,10 +63,14
package org.apache.commons.httpclient;
+import java.util.Set;
+import java.util.HashSet;
import java.io.IOException;
import java.net.URL;
-
+import java.net.MalformedURLException;
+import java.net.URL;
import org.apache.commons.httpclient.protocol.Protocol;
+import org.apache.commons.httpclient.util.URIUtil;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
-85,6 +89,7
* author a href=mailto:[EMAIL PROTECTED]Michael Becke/a
* author a href=mailto:[EMAIL PROTECTED]Mike Bowler/a
* author Sam Maloney
+ * author a href=mailto:[EMAIL PROTECTED]Oleg Kalnichevski/a
*
* version $Revision: 1.69 $ $Date: 2003/02/21 13:48:09 $
*/
-97,6 +102,9
/** Log object for this class. */
private static final Log LOG = LogFactory.getLog(HttpClient.class);
+/** Maximum number of redirects and authentications that will be followed */
+private static final int MAX_FORWARDS = 100;
+
// --- Constructors
/**
-149,6 +157,18
/** True if strict mode is enabled. */
private boolean strictMode = false;
+/** Whether or not I should automatically follow redirects. */
+private boolean followRedirects = true;
+
+/** Whether or not I should automatically processs authentication. */
+private boolean doAuthentication = true;
+
+/** Realms that we tried to authenticate to */
+private Set realms = null;
+
+/** Proxy Realms that we tried to authenticate to */
+private Set proxyRealms = null;
+
// - Properties
/**
-546,36 +566,73
}
}
-HttpConnection connection = state.getHttpConnectionManager().getConnection(
-methodConfiguration,
-httpConnectionTimeout
-);
-
-try {
-// Catch all possible exceptions to make sure to release the
-// connection, as although the user may call
-// Method-releaseConnection(), the method doesn't know about the
-// connection until HttpMethod.execute() is called.
-
-method.setStrictMode(strictMode);
+method.setStrictMode(strictMode);
-if (!connection.isOpen()) {
-connection.setSoTimeout(soTimeout);
-connection.setConnectionTimeout(connectionTimeout);
-connection.open();
-if (connection.isProxied() connection.isSecure()) {
-method = new ConnectMethod(method);
+this.realms = new HashSet();
+this.proxyRealms = new HashSet();
+//pre-emptively add
Re: HTTP Post and HTTP/100 (continue)
Simon Please try these settings. This should prompt commons-logging to use SimpleLog instead of Log4J, which does not support TRACE verbosity. -Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.SimpleLog -Dorg.apache.commons.logging.simplelog.log.httpclient.wire=debug -Dorg.apache.commons.logging.simplelog.log.org.apache.commons.httpclient=debug Cheers Oleg On Tue, 2003-02-25 at 21:19, Simon Roberts wrote: BTW, I think you should remove the log enter HttpConnection.responseAvaliable(), cause I have like 50.000 lines of it in my log for only one request... :/ As to excessive noise in the logs, try using debug verbosity instead trace one. I know logging is really useful sometimes, but *50,000* lines? That's certainly over-the-top. I get a few hundred every time just talking to a machine on the LAN. Even if the actual log is supressed, there's still the expense of the nested method calls (until it gets somewhere where it can determine that the log is not used). It doesn't seem most of the trace lines that show method entry are useful anyway, especially with log4j which shows them at debug level. Cheers, Simon - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
2.0 beta 1 development
We are now back in the Beta 1 development phase. There are currently 14 outstanding issues to tackle before we can drop another release. Work is going extremely well. Great thanks to everyone that was involved! We should be addressing the beta 1 bugs before other issues. I think that we can reach the beta 1 target within a month. After that there is not too much left before the final release of 2.0. The stream is open. Please test, code, patch and commit away! Jeff (Jandalf) Dever HttpClient 2.0 Release Prime. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Significant HttpClient HttpMethodBase overhaul. Need earlyfeedback
Completely agree. A few months ago we have been discussing possibility of splitting HttpMethod into HttpRequest/HttpResponse pair once we get 2.0 is released. I would wait with more radical changes till then Cheers Oleg On Tue, 2003-02-25 at 21:38, Sam Maloney wrote: Makes sense to me, I would definatly agree on your point that 'Client' logic should be in HttpClient and not in HttpMethodBase. (I would say redirect, auth and even auto-retry would count as 'Client' logic). Sam On Tuesday 25 February 2003 15:27, Oleg Kalnichevski wrote: Folks I am currently working on a patch enabling HttpClient to handle cross-site redirects. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=16729 In order to lay foundation for this capability I needed to make quite a few changes to HttpClient HttpMethodBase. I have opted for a more substantial overhaul of these classes, than was strictly necessary. I realize not all of you may agree with my decision. So, I decided to seek an early feedback from you to make sure I do not go completely astray. This is what I have done: I moved complete redirect authenticate logic from HttpMethodBase to HttpClient. HttpMethodBase Impact: - Even though binary interface is unchanged, HttpClient's modus operandi with regard to redirect authentication changed substantially. People like Laura Werner,who do not use standard HttpClient and have developed their own logic around lower level classes will be affected most. - Cleaner design. Redirect authentication in my opinion logically do not belong to domain of the HTTP method, rather, they belong to that of the HTTP agent. - Over-convoluted HttpMethodBase class got simpler. Under-used HttpClient class is leveraged more. This is an important architectural improvement in my humble opinion. If you disagree, please let me know - I am seriously concerned that this redesign may have adversely affected connection pooling stuff. Mike, Eric, you are the connection pooling experts, could you please give me your opinion on that? - About a dozen of test cases have become obsolete. They will need to be redesigned. They are all commented out for the time being As always, any feedback, including that in a form of bad tomatoes thrown at me will be appreciated Please note, that cross-site redirect has not been implemented yet. Cheers Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Significant HttpClient HttpMethodBase overhaul. Need earlyfeedback
I definitely think this is a good idea. HttpMethodBase is too big. I'm wondering if this is too much of a change for 2.0 though. It will require quite a few changes for users. On a related note, when you implement the redirection handling I would suggest removing the use of the URL class. I think URI is better suited for this purpose. Also, using URL could be a problem with custom protocol types. Mike Oleg Kalnichevski wrote: Folks I am currently working on a patch enabling HttpClient to handle cross-site redirects. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=16729 In order to lay foundation for this capability I needed to make quite a few changes to HttpClient HttpMethodBase. I have opted for a more substantial overhaul of these classes, than was strictly necessary. I realize not all of you may agree with my decision. So, I decided to seek an early feedback from you to make sure I do not go completely astray. This is what I have done: I moved complete redirect authenticate logic from HttpMethodBase to HttpClient. HttpMethodBase Impact: - Even though binary interface is unchanged, HttpClient's modus operandi with regard to redirect authentication changed substantially. People like Laura Werner,who do not use standard HttpClient and have developed their own logic around lower level classes will be affected most. - Cleaner design. Redirect authentication in my opinion logically do not belong to domain of the HTTP method, rather, they belong to that of the HTTP agent. - Over-convoluted HttpMethodBase class got simpler. Under-used HttpClient class is leveraged more. This is an important architectural improvement in my humble opinion. If you disagree, please let me know - I am seriously concerned that this redesign may have adversely affected connection pooling stuff. Mike, Eric, you are the connection pooling experts, could you please give me your opinion on that? - About a dozen of test cases have become obsolete. They will need to be redesigned. They are all commented out for the time being As always, any feedback, including that in a form of bad tomatoes thrown at me will be appreciated Please note, that cross-site redirect has not been implemented yet. Cheers Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Significant HttpClient HttpMethodBase overhaul. Need earlyfeedback
Mike I share your concerns, but the work on it has been blessed by Jandalf. Besides, I see no way of providing cross-site redirects while maintaining full compatibility with the existing HttpClient. Important question. Do you think that the patch will play well with your connection pooling stuff? Oleg On Tue, 2003-02-25 at 22:29, Michael Becke wrote: I definitely think this is a good idea. HttpMethodBase is too big. I'm wondering if this is too much of a change for 2.0 though. It will require quite a few changes for users. On a related note, when you implement the redirection handling I would suggest removing the use of the URL class. I think URI is better suited for this purpose. Also, using URL could be a problem with custom protocol types. Mike Oleg Kalnichevski wrote: Folks I am currently working on a patch enabling HttpClient to handle cross-site redirects. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=16729 In order to lay foundation for this capability I needed to make quite a few changes to HttpClient HttpMethodBase. I have opted for a more substantial overhaul of these classes, than was strictly necessary. I realize not all of you may agree with my decision. So, I decided to seek an early feedback from you to make sure I do not go completely astray. This is what I have done: I moved complete redirect authenticate logic from HttpMethodBase to HttpClient. HttpMethodBase Impact: - Even though binary interface is unchanged, HttpClient's modus operandi with regard to redirect authentication changed substantially. People like Laura Werner,who do not use standard HttpClient and have developed their own logic around lower level classes will be affected most. - Cleaner design. Redirect authentication in my opinion logically do not belong to domain of the HTTP method, rather, they belong to that of the HTTP agent. - Over-convoluted HttpMethodBase class got simpler. Under-used HttpClient class is leveraged more. This is an important architectural improvement in my humble opinion. If you disagree, please let me know - I am seriously concerned that this redesign may have adversely affected connection pooling stuff. Mike, Eric, you are the connection pooling experts, could you please give me your opinion on that? - About a dozen of test cases have become obsolete. They will need to be redesigned. They are all commented out for the time being As always, any feedback, including that in a form of bad tomatoes thrown at me will be appreciated Please note, that cross-site redirect has not been implemented yet. Cheers Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: HTTP Post and HTTP/100 (continue)
Attached is a log of my application (log4j, with most of the HttpConnection.isResponseAvaliable messages removed) BTW: typo in method name The interesting bit is that it times out (3 seconds) rather than getting the 100-continue response. Then, after it has send the body, the 100-continue response is received and returned (which is what is actually causing my problem). The server is a current release version of Jetty http://jetty.mortbay.org/jetty/ which appears to be working perfectly in all other regards. I get the same problem with HTTP PUT. Adding method.setUseExpectHeader(false); seems to fix it. Cheers, Simon - Original Message - From: Oleg Kalnichevski [EMAIL PROTECTED] To: Commons HttpClient Project [EMAIL PROTECTED] Cc: Simon Roberts [EMAIL PROTECTED] Sent: Tuesday, February 25, 2003 10:18 PM Subject: Re: HTTP Post and HTTP/100 (continue) Simon I'd really appreciate it if you could send us the debug trace for analysis. Please refer to the following url for instructions on how wire log can be activated: http://jakarta.apache.org/commons/httpclient/logging.html Your problem should be easily solvable by disabling 100-continue handshake. PostMethod myhttppost = new PostMethod(); myhttppost.setUseExpectHeader(false); Cheers Oleg On Tue, 2003-02-25 at 00:29, Simon Roberts wrote: Gidday, This is probably a dumb-user question, but if it is, then it might need to be documented for other dumb users :) I just checked out the latest CVS HttpClient and tried it with my application (it's using HEAD from a month or two ago), and am having a problem. Our app does HTTP POST (to a Jetty server, as it happens). Previously, httpclient used to just push the request header and body along in one lump, and when the server posted a http-100 (continue) status, it used to complain about continue received, but body already sent. Anyway, it used to work okay for us... Now, the httpClient.execute(method) returns 100, and there is no response (body) from the server, so my app barfs. The question is, how do I make httpclient send the body of the request? (maybe after I get the 100 back). Cheers, thanks, Simon - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Redirect to a different domain not supported?
Did Oleg's work on the redirect made it to alpha3 rc4? Oleg Kalnichevski wrote: Folks I was going to start working on it today. If there are any other takers, please let me know Cheers Oleg On Mon, 2003-02-24 at 07:27, Jeffrey Dever wrote: This is currently on the todo list. Refer to the following bug for more information: http://nagoya.apache.org/bugzilla/show_bug.cgi?id=16729 Adrian Sutton wrote: - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Significant HttpClient HttpMethodBase overhaul. Need earlyfeedback
Oleg, Great that you are taking on this work. I have both general and specific comments. General: * I share other's concerns with restructuring too much. At the risk of causing you much pain (since I'm not the one implementing!), how about the following: Create a new interface HttpMethodWithRedirect, defining getDoAuthentication(), setDoAuthentication(), execute(), getFollowRedirects(), and setFollowRedirects(), all of which are deprecated, and remove(!) the functions from HttpMethod. Have HttpMethodBase implement this new interface. Add a new function to HttpMethod, something like performRequestAndResponse(), which corresponds to your new model for the execute function. Old clients, using the existing method implementations that extend HttpMethodBase, could still get away with calling execute() on the method, although this would now be a deprecated use. HttpClient.executeMethod() would always invoke performRequestAndResponse(). My take is that the above changes, while theoretically breaking anything that implements HttpMethod without extending HttpMethodBase, I don't see as causing much of a problem, because likely nobody is successfully implementing HttpMethod outside of HttpMethodBase, seeing as we've had a lot of trouble getting it right. * Move the functionality that you've moved from HttpMethodBase to HttpClient to yet another new, package level class, perhaps with all static functions. Your new HttpClient and HttpMethodBase could call the shared implementation. * Seems like you need a function on HttpMethod called something like canFollowRedirects() that corresponds to what the unchanged EntityEnclosingMethod does with setFollowRedirects() - namely says you must not follow them. * HttpMethod.executeMethod() now suffers from the same problem that HttpMethodBase.execute() use to have. Some of the miscellaneous activity in this function ought to be extracted into separate routines. I suppose you can pass the buck on that one though. * I worry that your new changes are not going to handle proxies correctly, in weird cases with redirects. The execute method of ConnectMethod does an execute on its wrapped method. Not sure how your changes would match existing functionality, and off the top of my head, I cannot think of any obvious ways to restructure it. Specific: * HttpClient.getDefaultPort() - Protocol.getDefaultPort() would be appropriate to invoke here instead, I think. * HttpMethod.getFollowRedirects() - deprecate as well? This falls under my other comments. As for your question about connection usage, my perspective is that your changes are for the better. Right now, on a redirect, HttpMethodBase keeps the same connection (if it is persistent), and does the redirect or authentication. It seems to me that it would be a better model to return the connection to the pool and get a new one, if only for the architectural clarity of processing a single request/response pair (with a possible 100 Continue thrown in) with a connection, and then releasing the connection. Hope my comments help. -Elric. Oleg Kalnichevski wrote: Folks I am currently working on a patch enabling HttpClient to handle cross-site redirects. http://nagoya.apache.org/bugzilla/show_bug.cgi?id=16729 In order to lay foundation for this capability I needed to make quite a few changes to HttpClient HttpMethodBase. I have opted for a more substantial overhaul of these classes, than was strictly necessary. I realize not all of you may agree with my decision. So, I decided to seek an early feedback from you to make sure I do not go completely astray. This is what I have done: I moved complete redirect authenticate logic from HttpMethodBase to HttpClient. HttpMethodBase Impact: - Even though binary interface is unchanged, HttpClient's modus operandi with regard to redirect authentication changed substantially. People like Laura Werner,who do not use standard HttpClient and have developed their own logic around lower level classes will be affected most. - Cleaner design. Redirect authentication in my opinion logically do not belong to domain of the HTTP method, rather, they belong to that of the HTTP agent. - Over-convoluted HttpMethodBase class got simpler. Under-used HttpClient class is leveraged more. This is an important architectural improvement in my humble opinion. If you disagree, please let me know - I am seriously concerned that this redesign may have adversely affected connection pooling stuff. Mike, Eric, you are the connection pooling experts, could you please give me your opinion on that? - About a dozen of test cases have become obsolete. They will need to be redesigned. They are all commented out for the time being As always, any feedback, including that in a form of bad tomatoes thrown at me will be
Re: Redirect to a different domain not supported?
No. Its still preliminary. Jesus M. Salvo Jr. wrote: Did Oleg's work on the redirect made it to alpha3 rc4? Oleg Kalnichevski wrote: Folks I was going to start working on it today. If there are any other takers, please let me know Cheers Oleg On Mon, 2003-02-24 at 07:27, Jeffrey Dever wrote: This is currently on the todo list. Refer to the following bug for more information: http://nagoya.apache.org/bugzilla/show_bug.cgi?id=16729 Adrian Sutton wrote: - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Significant HttpClient HttpMethodBase overhaul. Need earlyfeedback
Michael Becke wrote: I think it would be possible to add cross site redirects at the HttpClient level without removing the other functionality from the HttpMethod. HttpClient would just need to check the status code and re-try. But, just because it is possible doesn't mean we should do it. If we have the go-ahead to implement this I'm all for it. I implemented it this way in my HttpCache code that sits on top of httpclient but doesn't use the HttpClient class. There's an outer loop in my cache's executeMethod, and it calls the inner loop in HttpMethodBase.execute. It got messy and was a bit hard to get right, but it worked. In general, I think it's cleaner to have all of the redirection and authentication handled in one loop, rather than having two separate ones. It just makes the architecture simpler. The only good reason I can see for going with a two-loop solution is the fact that we're fairly close to a final release right now and might want to keep the code stable. On the other hand, if we're going to make a semantic API change like this, it's probably better to do it now, before 2.0 final. -- Laura - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
MultipartPostMethod Holding File Stream Open?
I'm using a MultipartPostMethod to upload a file to a servlet: File file = new File(strUrl); HttpClient client = new HttpClient(); HostConfiguration hostConfig = new HostConfiguration(); MultipartPostMethod mpPost = new MultipartPostMethod(); hostConfig.setHost(someURL.getHost(), someURL.getPort(), someURL.getProtocol()); client.setConnectionTimeout(3); client.setHostConfiguration(hostConfig); mpPost.addParameter(someName, someValue); mpPost.addParameter(file.getName(), file); mpPost.setPath(strPath); client.executeMethod(mpPost); String confirmUpload = tpPost.getResponseBodyAsString(); mpPost.releaseConnection(); file.delete(); // this is being blocked. After the upload, I would like to delete the file off of my disk. Using other methods of uploading the file (in particular a PutMethod), I was able to then delete the file after the upload. Now that I am using the MultipartPostMethod obj for the upload, I am unable to delete the file (the return value is false, and there is no SecurityException being thrown - no SecurityManager even set as of this point either). So, I guess my question is whether there is a call to the MultipartPostMethod obj that I'm overlooking that would release it's connection (I'm sure that it is opening an InputStream of some sort to read the file contents, in order to form the HTTP message) to the file - so that I can then have unimpeded access to it for other operations?
SocketException While Uploading Large File With MultipartPostMethod
As I mentioned in a previous posting (Subject: MultipartPostMethod Holding File Stream Open?), I'm using the MultipartPostMethod to upload a file to a servlet. Here is the example code that I included in the other posting: File file = new File(strUrl); HttpClient client = new HttpClient(); HostConfiguration hostConfig = new HostConfiguration(); MultipartPostMethod mpPost = new MultipartPostMethod(); hostConfig.setHost(someURL.getHost(), someURL.getPort(), someURL.getProtocol()); client.setConnectionTimeout(3); client.setHostConfiguration(hostConfig); mpPost.addParameter(someName, someValue); mpPost.addParameter(file.getName(), file); mpPost.setPath(strPath); client.executeMethod(mpPost); String confirmUpload = tpPost.getResponseBodyAsString(); mpPost.releaseConnection(); I've been uploading some small text files (about 14KB each) and it seems to work properly in this situation. However, when I try to do the same with a 20MB file (I realize that this is a very large file, however, I want to test it's performance) a SocketException is thrown. java.net.SocketException: Software caused connection abort: socket write error at java.net.SocketOutputStream.socketWrite0(Native Method) at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92) at java.net.SocketOutputStream.write(SocketOutputStream.java:126) at org.apache.commons.httpclient.methods.multipart.FilePart.sendData(FilePart.java:198) at org.apache.commons.httpclient.methods.multipart.Part.send(Part.java:197) at org.apache.commons.httpclient.methods.MultipartPostMethod.writeRequestBody(MultipartPostMethod.java:203) at org.apache.commons.httpclient.HttpMethodBase.writeRequest(HttpMethodBase.java:1974) at org.apache.commons.httpclient.HttpMethodBase.processRequest(HttpMethodBase.java:2298) at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:915) at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:557) at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:474) at test.FileUploader.upload(FileUploader.java:179) at test.FileUploader.main(FileUploader.java:341) Is there some kind of cap on the file size that I can send? If so, at what size is the cap set?
Re: Significant HttpClient HttpMethodBase overhaul. Need early feedback
To implement this feature (full redirects), there are really only three general choices that make sense: 1) augment HttpMethodBase.execute() Pros: current functionality is maintained Cons: makes complex code more complex, 2) have two stage redirection done partially in HttpMethodBase and part in HttpClien Pros: migration path for redirect logic to eventually be all in HttpClient, Cons: would be a hack prone to error 3) move and augment redirect logic into HttpClient.execute( Pros: most logical seperation of concerns, does simplify HttpMethodBase, Oleg has an initial patch Cons: breaks user code From the work that Oleg has done, and the comments here, it seems clear that redirect logic should be in HttpClient. Clearly Oleg is on the right track with his work, and I agree this is the best approach. The problem is that con is a pretty big drawback. I'd say it is to late in the development cycle to make such a change, no matter how rewarding. I did not realize how big of a deal true redirects was going to be to implement. Oleg's patch gives us a good base and evaluation platform, but perhaps we should consider putting this feature on the back burner and do it in 2.1. Jandalf. Adrian Sutton wrote: The only good reason I can see for going with a two-loop solution is the fact that we're fairly close to a final release right now and might want to keep the code stable. On the other hand, if we're going to make a semantic API change like this, it's probably better to do it now, before 2.0 final. Personally, I feel it's too late in the release process to be moving all the redirect code around, however that does depend on how inelegant the other options are. If we don't move it until after the 2.0 release, we should add a note on any method that is likely to be affected by the change that the change will happen and so you shouldn't depend on this code being here. (Much like Sun's warning about serializing Swing objects). I'd also suggest that we make HttpClient more flexible so that Laura can use it directly since that would then discourage people from using the methods directly and changes like this will be much less of a concern in the future. Course, I don't have time to actually do any of this, so feel free to just ignore me. :) Adrian Sutton, Software Engineer Ephox Corporation www.ephox.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[ANNOUNCE] Release of Commons HttpClient 2.0 Alpha 3
This is an intermediate alpha release. The build process used in the previous Alpha 2 changed from generating 4 build artifacts to a single distribution. This one zip contains everything: all the source, the binary jar, the logging dependancy, generated javadoc and required build files for Ant builds and JUnit tests. One zip to rule them all, one zip to find them, one zip to bring them all and in the darkness bind them http://jakarta.apache.org/commons/httpclient/ http://www.apache.org/dist/jakarta/commons/httpclient/ Commons HttpClient Development Team - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
