Re: Save cookies to disk
On Tue, 2007-12-18 at 21:26 +, jdu jdass wrote: Hi, I'm working with httpclient 4 and i need save cookies to my hard disk. I've readed something about ModuleCookie, but I think it's for httpclient 3. does exists any way to write cookies to hard disk with httpclient 4?? You can provide a custom CookieStore implementation with any persistence mechanism you deem appropriate for your application: https://svn.apache.org/repos/asf/httpcomponents/httpclient/trunk/module-client/src/main/java/org/apache/http/client/CookieStore.java Hope this helps Oleg Thanks!! _ Tecnología, moda, motor, viajes,…suscríbete a nuestros boletines para estar siempre a la última http://newsletters.msn.com/hm/maintenanceeses.asp?L=ESC=ESP=WCMaintenanceBrand=WLRU=http%3a%2f%2fmail.live.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Trying to Authenticate to MS Exchange 2003
On Thu, 2008-01-10 at 16:06 -0500, Michael Prichard wrote: Okfigured this one out. Add this: // Sets whether authentication should be attempted preemptively. client.getParams().setAuthenticationPreemptive( true ); And it seems to work perfectly. No idea why! Anyone? This is because BASIC authentication (read: username/password sent in clear text) is used to authenticate preemptively, not NTLM. Apparently the credentials are OK, since you are able to authenticate using BASIC. So, most likely the server has been configured to use NTLMv2 only, whereas HttpClient supports NTLMv1 only. For details see http://wiki.apache.org/jakarta-httpclient/FrequentlyAskedNTLMQuestions Oleg Thx. On Jan 10, 2008, at 3:22 PM, Michael Prichard wrote: Hello All, I am trying to authenticate to MS Exchange as follows: HttpClient client = new HttpClient(); NTCredentials creds = new NTCredentials(_username, _password, localhost, _domain); // set credentials for connection client.getState().setCredentials(new AuthScope(_exchangeServer, AuthScope.ANY_PORT, AuthScope.ANY_REALM), creds); GetMethod get = new GetMethod(prootPath); get.setDoAuthentication(true); status = client.executeMethod(get); When I hit the server I get this back: Jan 10, 2008 3:10:54 PM org.apache.commons.httpclient.auth.AuthChallengeProcessor selectAuthScheme INFO: ntlm authentication scheme selected Jan 10, 2008 3:10:54 PM org.apache.commons.httpclient.HttpMethodDirector processWWWAuthChallenge INFO: Failure authenticating with NTLM any realm@mail.exchangeserver.com:443 I had it working with my internal test server but now when trying to connect to other servers I am getting Auth issues. Ideas? Thanks! Michael - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Trying to Authenticate to MS Exchange 2003
On Fri, 2008-01-11 at 09:36 -0500, Michael Prichard wrote: Ooops...I did not fully read the first message. Sorry. The interesting thing is that when using the browser to hit that server it does not give me a form based login screen like other exchange servers. It pops up a window..very much like you see when you set up a .htaccess with BASIC auth. Hmmmany libs out there that support NTLM? http://jcifs.samba.org/ Oleg On Jan 11, 2008, at 9:27 AM, Oleg Kalnichevski wrote: On Fri, 2008-01-11 at 09:24 -0500, Michael Prichard wrote: What does this mean? Does it mean the username and password are being sent on an open wire? That is exactly what I was trying to tell you in my previous post. Oleg On Jan 11, 2008, at 9:12 AM, Oleg Kalnichevski wrote: On Fri, 2008-01-11 at 09:06 -0500, Michael Prichard wrote: Yes, I got this to work by adding this: // Sets whether authentication should be attempted preemptively. client.getParams().setAuthenticationPreemptive( true ); When authenticating preemptively you ARE NOT using NTLM. (I guess it's fine as long as you are aware of this fact and its security implications) Oleg Thanks! On Jan 11, 2008, at 7:08 AM, Oleg Kalnichevski wrote: On Thu, 2008-01-10 at 16:06 -0500, Michael Prichard wrote: Okfigured this one out. Add this: // Sets whether authentication should be attempted preemptively. client.getParams().setAuthenticationPreemptive( true ); And it seems to work perfectly. No idea why! Anyone? This is because BASIC authentication (read: username/password sent in clear text) is used to authenticate preemptively, not NTLM. Apparently the credentials are OK, since you are able to authenticate using BASIC. So, most likely the server has been configured to use NTLMv2 only, whereas HttpClient supports NTLMv1 only. For details see http://wiki.apache.org/jakarta-httpclient/ FrequentlyAskedNTLMQuestions Oleg Thx. On Jan 10, 2008, at 3:22 PM, Michael Prichard wrote: Hello All, I am trying to authenticate to MS Exchange as follows: HttpClient client = new HttpClient(); NTCredentials creds = new NTCredentials(_username, _password, localhost, _domain); // set credentials for connection client.getState().setCredentials(new AuthScope (_exchangeServer, AuthScope.ANY_PORT, AuthScope.ANY_REALM), creds); GetMethod get = new GetMethod(prootPath); get.setDoAuthentication(true); status = client.executeMethod(get); When I hit the server I get this back: Jan 10, 2008 3:10:54 PM org.apache.commons.httpclient.auth.AuthChallengeProcessor selectAuthScheme INFO: ntlm authentication scheme selected Jan 10, 2008 3:10:54 PM org.apache.commons.httpclient.HttpMethodDirector processWWWAuthChallenge INFO: Failure authenticating with NTLM any realm@mail.exchangeserver.com:443 I had it working with my internal test server but now when trying to connect to other servers I am getting Auth issues. Ideas? Thanks! Michael -- -- - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: httpclient-users- [EMAIL PROTECTED] --- -- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: httpclient-users- [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: HttpClient4 alpha3 release?
On Tue, 2008-02-19 at 14:57 -0800, Aaron wrote: I'm just getting into using HttpClient, and since v4 seems to be coming along well I figured I would start my development with the alphas and move to the beta and production when they are ready, however I am curious as to when you will release alpha3 since according to the tracker all issues are done, and also I am trying to use HttpCore beta1. Which, I've encountered that there are some API usage differences between HttpClient and HttpCore. Its fixed in the source repository, but I was curious when it'll be packaged up and released? Very likely by the end of the week Oleg Thanks for all the hard work! --Aaron - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Can HttpCore NIO extensions support client side of Server push using multipart/x-mixed-replace
On Fri, 2008-02-22 at 16:41 -0500, Stephen More wrote: I see in version 4 there will be HttpCore NIO extensions - event driven, non-blocking I/O. When this is released can it be used to create the client side of a multipart/x-mixed-replace message ? There have already been a number of public releases of HttpCore http://hc.apache.org/news.html An example might be a HTTP Chat application. Individual POSTs could handle sending In new messages, while 1 GET connection would stay open to get all Outgoing messages for the entire session. Will something like this be possible ? Yes, something like that could be possible, but you would basically end up abusing HTTP protocol for something it was never intended for. HTTP is essentially a stateless, request/response oriented protocol. It was never designed for something like chat applications. Oleg -Thanks Steve More - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: DefaultHttpClient - getting contents of an iframe
On Sun, 2008-02-24 at 19:00 -0400, Jeff Kerr wrote: Hi there, I've had some luck following the example at: http://svn.apache.org/repos/asf/httpcomponents/httpclient/trunk/module-client/src/examples/org/apache/http/examples/client/ClientFormLogin.java to login to a web site then get a page I'm looking for. The problem is that the page that comes back has an iframe reference who's source is what I actually want. When I try to get that page with a POST or GET it detects that I'm not making the request from the iframe and redirects me to the enclosing page. Is there a way on my DefaultHttpClient to pretend I'm a browser making a request for an iframe within a page? Hi Jeff, One can program HttpClient to generate virtually identical HTTP messages. Generally you should follow a very simple approach: capture HTTP packets generated by browser using a traffic analyzer or a proxy, examine their composition, tweak HttpClient to generate similar / identical packets. Hope this helps. Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Non-ascii filenames in Multipart POST
On Mon, 2008-02-25 at 16:48 +, Andy Wu wrote: Hi, When making a multipart post uploading a file with a filename containing non-ascii characters, the filename appears to be incorrect written to the http request. If you look at the code, sendDispositionHeader() essentially sends the filename.getBytes( US-ASCII ): http://hc.apache.org/httpclient-3.x/xref/org/apache/commons/httpclient/methods/multipart/FilePart.html However, the browsers I've tested with will send the filename.getBytes( ISO-8859-1 ) version of the filename and thus the software that's I'm working with that receives the upload is decoding the filename correctly. Would anyone else concur that this is a bug? Andy, This is definitely not a bug. Non-ASCII characters are not allowed to be present in the MIME elements such disposition headers. Common browser applications violate the MIME spec. Plain and simple. For those who do not give a flying damn about standards compliance, HttpClient as of release 4.0-alpha3 provides a browser compatibility mode for multipart coded entities. Hope this helps Oleg Thanks Andy Wu - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[ANNOUNCEMENT] HttpComponents HttpClient 4.0-alpha3 released
The HttpComponents project is pleased to announce the release of HttpComponents HttpClient 4.0-alpha3. This version brings another round of API refinements and improvements in functionality. As of this release HttpClient requires Java 5 compatible runtime environment and takes full advantage of generics and new concurrency primitives. This release also introduces new default cookie policy that selects a cookie specification depending on the format of cookies sent by the target host. It is no longer necessary to know beforehand what kind of HTTP cookie support the target host provides. HttpClient is now able to pick up either a lenient or a strict cookie policy depending on the compliance level of the target host. Another notable improvement is a completely reworked support for multipart entities based on Apache mime4j library. --- Download - http://hc.apache.org/downloads.cgi Release notes - http://www.apache.org/dist/httpcomponents/httpclient/RELEASE_NOTES.txt HttpComponents site - http://hc.apache.org/ - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Multi-Threading using different host configurations
On Thu, 2008-02-28 at 09:24 +, Eric Cantonna wrote: Hello everyone, This is my first message on the mailing list (but searches in the archives have already helped me alot). I'm coding some kind of data miner which works rather well now so i'm entering the optimisation phase. I find the example and documentation on the multi-threading a bit sketchy. What is the best way to proceed if I want some threads to be associated with requests for different sites (let's say one thread per site to fix the mind) and I don't want to use absolute URIs for each Method instance. I gather I have to use different HostConfiguration instances for this? Yes. You should also be using a separate HttpState instance per thread of execution. Should/Can I also use different HttpClient instances? No/Yes Should/Can I use different MultiThreadedHttpConnectionManager instances? No/No Oleg I gather I could test the different possibilities and see for myself what works and what doesn't but I thought maybe I could avoid that trouble and ask away... Thanks, _ Microsoft vous recommande de mettre à jour Internet Explorer. http://specials.fr.msn.com/IE7P25 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [Help] NTLM Auth fails after disabling LM hashes of passwords
On Thu, 2008-02-28 at 15:57 +0530, Shailesh Joshi wrote: Hi. We are using HttpClient for NTLM Authentication. We are facing issues when we - disable the storage of LM hashes of user's passwords. (http://support.microsoft.com/kb/299656) Following steps are followed in Windows Server 2003. Control Panel - Administrative Tools - Domain Security Policy - Local Policies - Security Options - Network security: Do not store LAN Manager Hash value on next password change. - Click Enabled and then click OK. After setting this property, we are facing problems in NTLM authentication. Has anyone faced similar issue? Is there any dependency of HttpClient with LM Hashes? I believe this effectively disables NTLMv1 support, the only version of NTLM protocol HttpClient 3.x can interoperate with. For details see http://wiki.apache.org/HttpComponents/FrequentlyAskedNTLMQuestions Oleg Thanks and Regards, Shailesh DISCLAIMER == This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: cookie or session-id is different from the same HttpClient connection?!
On Thu, 2008-02-28 at 17:19 +0800, micky wrote: Dear, I use the HttpClient to login some page, and it will have a cookie according to my login. But when I try to access another page (needed for logon) by the same HttpClient, the response is access denied. Then I print out the cookies of the two connection, I found them are different. But the document said I don't need to care the cookie issue when I use HttpClient to login and continue browsing other pages. Here is some sample pseudo code to demo. - DefaultHttpClient httpclient = new DefaultHttpClient(); HttpPost httpPost = new HttpPost(http://127.0.0.1/login.html;); // response 200, and login ok! HttpResponse response = httpclient.execute(httpPost); HttpEntity entity = response.getEntity(); // cookie with a session id 1 for example // continue to another page httpPost = new HttpPost(http://127.0.0.1/info.html;); // response 200, and access denied HttpResponse response2 = httpclient.execute(httpPost); HttpEntity entity2 = response2.getEntity(); // cookie with a session id 2 for example // the session is different - I think the session id in the cookie should be the same, so that I can browse all pages after the first logon action. But the truth is not?! How it can be that?? Thanks for any reply or suggestion. Sincerely, Micky Micky, Please execute your application with the 'org.apache.http' logger set to DEBUG. -Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.SimpleLog -Dorg.apache.commons.logging.simplelog.log.org.apache.http=DEBUG Feel free to post the log to this list if you need help figuring out the cause of the problem. Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: HttpClient v 4 Alpha 3: WARNING: Invalid use of SingleClientConnManager: connection still allocated.
On Thu, 2008-03-06 at 19:42 +0100, Tov Are Jacobsen wrote:
Hi,
I'm getting a warning and an Exception stack-trace for every single
connection I do after the first one.
I consumeContent after each request, so I think my code should be ok
... and it works as well, but I wonder
if it's possible to find a way to not get those warnings (because it
makes it harder to focus on the real issues with my code)
06.mar.2008 18:50:43 org.apache.http.impl.conn.SingleClientConnManager
revokeConnection
WARNING: Invalid use of SingleClientConnManager: connection still allocated.
Make sure to release the connection before allocating another one.
java.lang.IllegalStateException: Revoking connection to
HttpRoute[{s}-https://xxx.xx.xx]
at
org.apache.http.impl.conn.SingleClientConnManager.revokeConnection(SingleClientConnManager.java:345)
at
org.apache.http.impl.conn.SingleClientConnManager.getConnection(SingleClientConnManager.java:220)
at
org.apache.http.impl.conn.SingleClientConnManager.getConnection(SingleClientConnManager.java:195)
at
org.apache.http.impl.client.DefaultClientRequestDirector.allocateConnection(DefaultClientRequestDirector.java:508)
at
org.apache.http.impl.client.DefaultClientRequestDirector.execute(DefaultClientRequestDirector.java:293)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:500)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:455)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:421)
at no.telerespons.statistikk.GetStat.downloadFile(GetStat.java:120)
at no.telerespons.statistikk.GetStat.main(GetStat.java:43)
Btw. the idea behind the release strategy is great ... but I'm so used
to unallocate that it feels a bit like black magic :-)
Btw. when is the next alpha or betal comming out?
Thanks !
Tov Are,
This is weird. I am unable to reproduce the problem locally. What
release of HttpClient 4.0 are you using? Would it be a deal for you to
try to reproduce the problem using a concise, self-contained test case?
Oleg
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
RE: cookie or session-id is different from the same HttpClient connection?!
On Fri, 2008-03-07 at 13:24 +0800, micky wrote: Dear, I finally figure out how to log!! Please help me find out why these two connections' cookie are different. Because HttpClient does not find the first cookie when executing the second request. You are not creating a new instance of HttpClient for each request, are you? Oleg Thanks for your kindly help. Sincerely, Micky -Original Message- From: Roland Weber [mailto:[EMAIL PROTECTED] Sent: Thursday, March 06, 2008 1:02 AM To: HttpClient User Discussion Subject: Re: cookie or session-id is different from the same HttpClient connection?! Hello Micky, I just use HttpsURLConnection.getHeaderField(Set-Cookie) to get a cookie String, then I use HttpURLConnection.setRequestProperty(Cookie, cookie String) for the next connection. And it worked well. That surprises me. It seems Sun HttpURLConnection didn't handle cookie internally. So it didn't work before you copied the cookie in your application. Interesting. HttpClient maintains a shared default session state for all connections. If you don't do something to prevent it, it will pick up the cookies that were previously set. I think HttpClient should do this as you said too. But I don't know why it doesn't work for my code now. Logging should help to find that out. You can also modify the code and put some System.out.println()s into interesting places, when the cookies for the request being sent are selected. Should I handle the cookie management in the server? The server is Struts on Tomcat, and I didn't do any additional action for handling cookie because I think the server should handle it itself. If it's about a session cookie and your server-side application is a servlet or some other standard J2EE thing, you don't have to worry about session management. When I print the cookie content from HttpClient.getCookieStore().getCookies(), it shows as followings. [version: 0][name: JSESSIONID][name: 8C20EDB1D409B3AFCE1A1D25CCDE59CE][domain: 127.0.0.1][path: /agent][expiry: null] This looks OK. However, it specifies both a domain 127.0.0.1 and a path /agent. If your next request is NOT to 127.0.0.1, then the cookie will not be sent. The comparison is on the string level, so if the next request goes to localhost instead of 127.0.0.1, then it will NOT match. Also, if your next request does not go to a path starting with /agent, this cookie will not match. cheers, Roland - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: cookie or session-id is different from the same HttpClient connection?!
On Tue, 2008-03-11 at 21:23 +0800, micky wrote: Dear, I solved the cookie issues according to the article mentioned by Roland at http://www.nabble.com/forum/ViewPost.jtp?post=15912252framed=y After I GET a request to the server, the session will be established. The following sequences like POST a data or request any other pages by GET or POST will be in the right behavior. But if I POST a request with data to the server first instead of GET, the following sequences just have different sessions and I will get a Accessed Denied warning. I think it's because the HttpClient library will store cookie after the FIRST GET request automatically, and will use it through the following browsing. If the FIRST request is a POST, the HttpClient won't store cookie for the following requests. Am I right ? or any hint? I do not think so. There is absolutely no difference between POST, GET or any other method as far as cookie management is concerned. Please try to put together a self-contained test case and I'll happily take a look at it. Oleg Sincerely, Micky -Original Message- From: micky [mailto:[EMAIL PROTECTED] Sent: Sunday, March 09, 2008 5:24 PM To: 'HttpClient User Discussion' Subject: RE: cookie or session-id is different from the same HttpClient connection?! Dear Oleg, Because HttpClient does not find the first cookie when executing the second request. You are not creating a new instance of HttpClient for each request, are you? No, I am using the same instance of HttpClient for each request. Any suggestion for debugging? Sincerely, Micky - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: properly aborting a GET method?
On Tue, 2008-03-11 at 11:25 -0700, George Ludwig wrote: I am doing some web harvesting, and I'm trying to better handle the situation where a hostile website has my connection and then sends a few packets of garbage data every couple of minutes. In the loop where I read data off the input stream, I check the duration of the entire read process, and if it has taken longer than a preset number of milliseconds, I want to terminate the get method. Initially, I simply threw an exception, then called method.releaseConnection() in the finally block. However, I have since realized that releaseConnection() tries to read the entire response before releasing the connection. This does me no good, since I want to stop consuming the response. So now I call method.abort(), then throw the exception, and then method.releaseConnection() is called in the finally block. Is this the best way to terminate a hostile connection? George, HttpMethod#abort is the way to go Oleg TIA, George Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: HttpClient GET request for a file 2G doesn't work
On Tue, 2008-03-11 at 11:38 -0700, sjlucas wrote: Eric Cantonna wrote: Maybe I'm saying something stupid but 2G is awfully close to maxint. Maybe there is a count of bytes read/available somewhere that use an int and should use a long instead. Just an idea that crossed my mind. Right. But since it's HTTPClient's executeMethod() that's not working right, Are you that sure? I'm trying to find out if there is a limitation in HTTPClient to handle content 2G Please let me know as soon as you succeed in doing so. Oleg (such as due to the HTTPClient using an int instead of a long). - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Sending serialized object
On Tue, 2008-03-11 at 01:49 -0400, [EMAIL PROTECTED] wrote: Hi All, I'm new to HttpClient package. I've noticed that PostMethod class does not have a method that would return OutputStream that I could use to write the serialized object in to. This can be done using java.net.HttpURLConnection class. Any suggestion on how to write same thing using httpclient? Which version? Oleg Thanks Valli - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: PostMethod now known as HttpPost
On Thu, 2008-03-13 at 13:10 +0100, Thorsten Scherler wrote:
Hi all,
I am ATM trying to integrate latest httpclient into Droids. I have some
old code that is based on a prior version of the client.
Some method/class name changes I found myself but some I am not sure how
to implement them.
When my old code was:
PostMethod filePost = new PostMethod(solrBase);
filePost.addRequestHeader(Content-type, text/xml; charset=utf-8);
filePost.addRequestHeader(User-Agent, AGENT);
filePost.setRequestBody(new URL(srcUrl).openStream());
Now I do:
HttpPost filePost = null;
filePost = new HttpPost(solrBase);
filePost.addHeader(Content-type, text/xml; charset=utf-8);
filePost.addHeader(User-Agent, AGENT);
InputStreamEntity reqEntity = new InputStreamEntity(src, -1);
filePost.setEntity(reqEntity);
But for the following I could not found the corresponding method:
public String getResponseCharSet() {
return filePost.getResponseCharSet();
}
public String getResponseBodyAsString() {
return filePost.getResponseBodyAsString();
}
Can you point me to the new methods?
Hi Thorsten,
These methods are now provided by the EntityUtils class
http://hc.apache.org/httpcomponents-core/httpcore/apidocs/org/apache/http/util/EntityUtils.html
Generally I would recommend avoiding consuming response entities as
strings in the real life application unless you are absolutely sure they
are of reasonable / limited length.
Oleg
TIA
salu2
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: Using HTTPClient to call Creative Commons Web Service
On Sat, 2008-03-15 at 10:00 -0700, Wayne Richards wrote: This is probably a newbie question, as I have just started working with the HTTPClient. The question is that I know that the following line placed in a browser address line will return what I want. http://api.creativecommons.org/rest/license/standard/issue?answers=%3Canswers%3E%0A%3Clicense-standard%3E%3Cderivatives%3Esa%3C%2Fderivatives%3E%0A%3Ccommercial%3Ey%3C%2Fcommercial%3E%0A%3Cjurisdiction%3Epl%3C%2Fjurisdiction%3E%0A%3C%2Flicense-standard%3E%0A%3C%2Fanswers%3E%0A Wayne, Use a traffic analyzer to capture packets generated by the browser and then program HttpClient to generate identical / compatible packets. Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: how 2 supress contentType when using multipartentity
On Mon, 2008-03-17 at 11:30 -0700, ghanchakkar wrote:
Please help me.
I building a bot and trying to upload file along with form fields. when
compare the request generated by httpclient with that of microsoft browser,
I notice that httpclient is having following format after the header.
---boundaryid
Content-Disposition: form-data; name=fieldname1
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
value1
---boundaryid
Content-Disposition: form-data; name=fieldname2
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
value2
---boundaryid
:
:
But internet explorer generates request in following format
---boundaryid
Content-Disposition: form-data; name=fieldname1
value1
---boundaryid
Content-Disposition: form-data; name=fieldname2
value2
---boundaryid
:
:
Notice the Content-Type and Content-Transfer-Encoding fields are populated
by HttpClient.
The webserver is not expecting these two fields
Then the webserver is buggy.
and therefore throwing me an
error as it is not able to trace the value for the required formfield (I
guess it is poorly written server)
How do I get rid of these extra two values while using httpclient
multipartentity.
Set them to null.
Hope this helps.
Oleg
please see snippet of my code below
Part[] parts = new Part[files.length + nvPairs.length];
//setup form fields
for(int i = 0 ; i nvPairs.length ; i++){
parts[i] = new StringPart(nvPairs[i].getName(),
nvPairs[i].getValue(),CHAR_SET);
}
//setup files
for(int i = nvPairs.length, j=0 ; j files.length ; j++,i++){
File targetFile = new File(files[j].getValue());
parts[i] = new FilePart(files[j].getValue(), targetFile);
}
post.setRequestEntity(new MultipartRequestEntity(parts, post.getParams()));
httpClient.getHttpConnectionManager().getParams().setConnectionTimeout(5000);
statusCode = httpClient.executeMethod(post);
Please help ASAP.
Thanks in advance.
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: How could cause connection reset
On Fri, 2008-03-14 at 11:24 +0800, hjw wrote: Hi,dear all: I trying to use httpclient to get data from a website once in 1 second ,but when the count of getting data is appropriate to above 300 times,the programme will show a connection reset exception,I must keep the programme to work no breaking,could you tell me what will cause the exception and how to prevent it. I am sorry for my pool English. regards, Raymond Raymond, What is it exactly you are doing? Are you sending a new HTTP request every second or do you send just _one_ HTTP request and then poll the data from the response output stream every second? Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: How could cause connection reset
On Tue, 2008-03-18 at 08:56 +0800, hjw wrote: Oleg Kalnichevski , I am sending a new HTTP request every second and then get the response.butusually connection reset after between 400 and 500 times,and some can over 1000 times. Raymond, Connection resets can happen due to various reasons (firewalls for instance). You just need to retry failed requests. Oleg Raymond 2008/3/18, Oleg Kalnichevski [EMAIL PROTECTED]: On Fri, 2008-03-14 at 11:24 +0800, hjw wrote: Hi,dear all: I trying to use httpclient to get data from a website once in 1 second ,but when the count of getting data is appropriate to above 300 times,the programme will show a connection reset exception,I must keep the programme to work no breaking,could you tell me what will cause the exception and how to prevent it. I am sorry for my pool English. regards, Raymond Raymond, What is it exactly you are doing? Are you sending a new HTTP request every second or do you send just _one_ HTTP request and then poll the data from the response output stream every second? Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: SSL Site
On Mon, 2008-03-17 at 22:19 -0700, Ravichan wrote: Hi .. I followed your steps . I have downloaded the certificate and stored as DER in local. Then I tried to execute the following command. keytool -keystore C:\Program Files\Java\jdk1.6.0\jre\lib\security\cacerts -import -alias mysecurestore -file C:\temp\certfile.cer -trustcacerts Its asking for a password. I am not sure , What password to supply. changeit Sometime Google really helps Oleg Can you please suggest me. olegk wrote: On Wed, 2007-05-02 at 22:22 -0700, RossW wrote: ok cool...i fixed the problem. So first of all i had to connect through proxy first and then secondly i had to add the certificate to the keystore and then add the keystore as a property to code..now working fine. so here is the code which made all the difference. first i had to export the cert from the site...once logged in i just double clicked on the lock icon in IE (on the status bar down the bottom of IE when logged into the secure site)and then found and copy to file button. I saved it as a DER encrypted file to say c:\temp\certfile.cer and then using keytool as follows (keytool can be found in the JDK bin folder) keytool -keystore C:\Program Files\Java\jdk1.6.0\jre\lib\security\cacerts -import -alias mysecurestore -file C:\temp\certfile.cer -trustcacerts System.setProperty(javax.net.ssl.trustStore, C:\\Program Files\\Java\\jdk1.6.0\\jre\\lib\\security\\cacerts); and now is working like a charm. I hope this comes in handy for someone else in future cuz this one really sucked. Ross You may consider using AuthSSLProtocolSocketFactory if you want to avoid having to modify the cacerts file http://svn.apache.org/viewvc/jakarta/commons/proper/httpclient/trunk/src/contrib/org/apache/commons/httpclient/contrib/ssl/ http://svn.apache.org/viewvc/jakarta/commons/proper/httpclient/trunk/src/contrib/org/apache/commons/httpclient/contrib/ssl/AuthSSLProtocolSocketFactory.java?view=markup For details see http://jakarta.apache.org/commons/httpclient/sslguide.html Oleg RossW wrote: ok now i am getting this...the change i made which was causing the prev error was to connect via proxy first. Funny thing was that i was told without any doubt that it was not proxied. Anyways now i am getting SSL cert related errors javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Unknown Source) at com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(Unknown Source) at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Unknown Source) at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Unknown Source) at com.sun.net.ssl.internal.ssl.ClientHandshaker.serverCertificate(Unknown Source) at com.sun.net.ssl.internal.ssl.ClientHandshaker.processMessage(Unknown Source) at com.sun.net.ssl.internal.ssl.Handshaker.processLoop(Unknown Source) at com.sun.net.ssl.internal.ssl.Handshaker.process_record(Unknown Source) at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(Unknown Source) at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(Unknown Source) at com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(Unknown Source) at com.sun.net.ssl.internal.ssl.AppOutputStream.write(Unknown Source) at java.io.BufferedOutputStream.flushBuffer(Unknown Source) at java.io.BufferedOutputStream.flush(Unknown Source) at org.apache.commons.httpclient.HttpConnection.flushRequestOutputStream(Unknown Source) at org.apache.commons.httpclient.HttpMethodBase.writeRequest(Unknown Source) at org.apache.commons.httpclient.HttpMethodBase.execute(Unknown Source) at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(Unknown Source) at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(Unknown Source) at org.apache.commons.httpclient.HttpClient.executeMethod(Unknown Source) at org.apache.commons.httpclient.HttpClient.executeMethod(Unknown Source) at chester_japp.Chester_queue.record_proc(Chester_queue.java:129) at chester_japp.Chester_queue.run(Chester_queue.java:382) at java.lang.Thread.run(Unknown Source) Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.validator.PKIXValidator.doBuild(Unknown Source) at sun.security.validator.PKIXValidator.engineValidate(Unknown Source) at sun.security.validator.Validator.validate(Unknown Source) at
Re: Use variation on StrictSSLProtocolSocketFactory to check for server certs that are about to expire?
On Tue, 2008-03-18 at 08:29 -0700, Karr, David wrote: Has anyone ever implemented a variation on StrictSSLProtocolSocketFactory that could log an event if a server cert has an expiration date within a certain margin from the current date? If someone were to implement this, would this be the right place to put it? David, Certainly feel free to post the source code to the list. Your post will get picked up and indexed by various mail archives. We might also accept your code to the contrib package if it is more than a few lines of changes compared to StrictSSLProtocolSocketFactory Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Trouble shooting informational status
On Wed, 2008-03-19 at 14:31 +0100, Thierry Boileau wrote:
Hello all,
After some tests, I noticed that the HttpClient hangs in the case the
server sends a response with informational status and at the same time
does not support persistent connections. I recognize that this case is
quite unusual or unpractical however it leads to block the client.
I wonder if the behaviour of the
org.apache.commons.httpclient.HttpMethodBase#readResponse method could
not be improved by just adding a call to the shouldCloseConnection
method after checking sthe status code.
if ((status = 100) (status 200)) {
if(shouldCloseConnection(conn)){
break;
}
best regards,
Thierry Boileau
Thierry,
We no longer fix all but most serious bugs in the 3.x codeline. Please
test against the latest 4.0 release or better the SVN snapshot.
Oleg
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: The abort operation is close the socket
On Thu, 2008-03-20 at 10:49 +0800, hjw wrote: Hi,all: I find HttpMethod#abort() will close socket, but I only want to interrupt the IO operation which is using the network. My case is that some important message must post at once when data I get from the same site is match some condition.So , I need to interrupt the IO operation and let the post action has the highest priority. Could you tell me whether abort the is the only way to do for the case,or has a better one? Raymond Raymond, I do not think classic I/O model is applicable for this scenario. Consider using NIO (non-blocking I/O) instead. Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Handling proxies and modifying request header fields
On Thu, 2008-03-20 at 10:31 +0530, Biju P wrote: Hi, I am writing an Applet. The applet must be accessed by users from different domains and networks which are controlled by proxies and firewalls. My concerns are, i) The applet should be able to send requests to external websites thru the proxy (if any proxy exists). ii) The applet should be able to modify the request header fields, such as User-agent field. When I try using normal URLConnection from java.io package, the applet is able to inherit the proxy settings from the browser and send the requests thru that proxy. But, I am getting response code 403 for some of the sites. Can anybody tell me, is this possible by using Apache HttpClient?. No, it is not. You have configure HttpClient manually either using system properties or ProxySelector class if running on JRE 5.0 or newer. Oleg And if so, please specify the version. It would be a great help, if somebody could advise on this. Biju P M Mailto: [EMAIL PROTECTED] =-=-= Notice: The information contained in this e-mail message and/or attachments to it may contain confidential or privileged information. If you are not the intended recipient, any dissemination, use, review, distribution, printing or copying of the information contained in this e-mail message and/or attachments to it are strictly prohibited. If you have received this communication in error, please notify us by reply e-mail or telephone and immediately and permanently delete the message and any attachments. Thank you - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Getting the response time for multithreaded client
On Thu, 2008-03-20 at 18:19 +0100, Uwe Kubosch wrote: Hi all! We are using the Commons HttpClient in a multithreaded application, and frequently there are more threads trying to use the client than the limit set by setMaxConnectionsPerHost. In this situation we still need to log the response time from the server. How can we do that? Measuring the Method.executeMethod call includes the time the thread sleeps before actually submitting the request. We are currently using Commons HttpClient 3.1 which I believe is the latest stable release. Uwe, There is no way around either upgrading to HttpClient 4.0, which has a massively better, modular architecture or hacking the source code of HttpClient 3.x Oleg Any help is greatly appreciated, even quick fixes and hacks :) Uwe - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: httpclient proxy problem
On Thu, 2008-03-20 at 20:38 +0200, Mehmet Kuzu wrote:
Hİ ,
I have a problem about getting response behind proxy which requires
authentication using apache httpclient . Proxy server is squid and it
uses ntlm protocol. I am new to httpclient
API and i may miss something.
Thanks very much,
Best regards,
Mehmet,
The code seems okay. The wirelog seems to indicate an authentication
failure due to invalid credentials. Please double-check values of the
username, password, domain and host.
Oleg
My code is below:
*
public static void main(String args[]) {
Provider prov = new BouncyCastleProvider();
Security.addProvider(prov);
String url = http://www.google.com;;
HttpClient client = new HttpClient();
client.getHostConfiguration().setProxy(proxyHost, proxyPort);
Credentials defaultcreds = new
NTCredentials(userName,password,proxyhost,domain);
client.getState().setProxyCredentials(AuthScope.ANY,defaultcreds);
HttpMethod method = new GetMethod(url);
method.setDoAuthentication( true );
int statusCode;
try {
statusCode = client.executeMethod( client.getHostConfiguration(),
method,client.getState());
System.out.println(Stat= +HttpStatus.getStatusText(statusCode));
} catch (HttpException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
***
log is here
2008/03/20 20:34:52:921 EET [DEBUG] HttpClient - Java version:
1.5.0_03
2008/03/20 20:34:52:921 EET [DEBUG] HttpClient - Java vendor: Sun
Microsystems Inc.
2008/03/20 20:34:52:921 EET [DEBUG] HttpClient - Java class path: C:
\Documents and Settings\Owner\Desktop\setup\eclipse\workspace
\Deneme;C:\Documents and Settings\Owner\Desktop\setup\eclipse
\workspace\Deneme\xmlrpc-client-3.0.jar;C:\Documents and Settings
\Owner\Desktop\setup\eclipse\workspace\Deneme\xmlrpc-common-3.0.jar;C:
\Documents and Settings\Owner\Desktop\setup\eclipse\workspace\Deneme
\mail.jar;C:\Documents and Settings\Owner\Desktop\setup\eclipse
\workspace\Deneme\hibernate3.jar;C:\Documents and Settings\Owner
\Desktop\setup\eclipse\workspace\Deneme
\mysql-connector-java-5.0.8-bin.jar;C:\Documents and Settings\Owner
\Desktop\setup\eclipse\workspace\Deneme\xerces-2.6.2.jar;C:\Documents
and Settings\Owner\Desktop\setup\eclipse\workspace\Deneme
\dom4j-1.6.1.jar;C:\Documents and Settings\Owner\Desktop\setup\eclipse
\workspace\Deneme\commons-logging-1.0.4.jar;C:\Documents and Settings
\Owner\Desktop\setup\eclipse\workspace\Deneme\lib
\commons-collections-2.1.1.jar;C:\Documents and Settings\Owner\Desktop
\setup\eclipse\workspace\Deneme\lib\asm.jar;C:\Documents and Settings
\Owner\Desktop\setup\eclipse\workspace\Deneme\lib\asm-attrs.jar;C:
\Documents and Settings\Owner\Desktop\setup\eclipse\workspace\Deneme
\lib\c3p0-0.9.1.jar;C:\Documents and Settings\Owner\Desktop\setup
\eclipse\workspace\Deneme\lib\cglib-2.1.3.jar;C:\Documents and
Settings\Owner\Desktop\setup\eclipse\workspace\Deneme\lib
\checkstyle-all.jar;C:\Documents and Settings\Owner\Desktop\setup
\eclipse\workspace\Deneme\lib\cleanimports.jar;C:\Documents and
Settings\Owner\Desktop\setup\eclipse\workspace\Deneme\lib
\concurrent-1.3.2.jar;C:\Documents and Settings\Owner\Desktop\setup
\eclipse\workspace\Deneme\lib\connector.jar;C:\Documents and Settings
\Owner\Desktop\setup\eclipse\workspace\Deneme\lib\ehcache-1.2.3.jar;C:
\Documents and Settings\Owner\Desktop\setup\eclipse\workspace\Deneme
\lib\jaas.jar;C:\Documents and Settings\Owner\Desktop\setup\eclipse
\workspace\Deneme\lib\jacc-1_0-fr.jar;C:\Documents and Settings\Owner
\Desktop\setup\eclipse\workspace\Deneme\lib\javassist.jar;C:\Documents
and Settings\Owner\Desktop\setup\eclipse\workspace\Deneme\lib
\jaxen-1.1-beta-7.jar;C:\Documents and Settings\Owner\Desktop\setup
\eclipse\workspace\Deneme\lib\jdbc2_0-stdext.jar;C:\Documents and
Settings\Owner\Desktop\setup\eclipse\workspace\Deneme\lib
\jgroups-2.2.8.jar;C:\Documents and Settings\Owner\Desktop\setup
\eclipse\workspace\Deneme\lib\jta.jar;C:\Documents and Settings\Owner
\Desktop\setup\eclipse\workspace\Deneme\lib\oscache-2.1.jar;C:
\Documents and Settings\Owner\Desktop\setup\eclipse\workspace\Deneme
\lib\proxool-0.8.3.jar;C:\Documents and Settings\Owner\Desktop\setup
\eclipse\workspace\Deneme\lib\swarmcache-1.0rc2.jar;C:\Documents and
Settings\Owner\Desktop\setup\eclipse\workspace\Deneme\lib
\syndiag2.jar;C:\Documents and Settings\Owner\Desktop\setup\eclipse
\workspace\Deneme\lib\versioncheck.jar;C:\Documents and Settings\Owner
Re: Apache HttpClient - Using custom socket factory for failover support
On Thu, 2008-03-20 at 22:05 -0700, Ankur Shah wrote: I am using httpclient to connect to a remote server. Now for each primary server there's also a backup failover server that i must talk to if connection attempts with the primary server fails. To achieve this what i'd ideally like to do is have HttpClient use a custom factory that tries to create a socket connection with the primary server and if that fails (or times out), it'll go about talking to backup server (and retry a few times before giving up altogether). Now, searching on the web, i did find references to a MultiHomeProtocolSocketFactory (that extends DefaultSocketFactory) that is presumably in apache contrib package. But what that does is that in the overriden createSocket method, it takes host as a param (among other thing) and then simply does a InetAddress.getAllByName which is not what i want. From what i understand this caters to a DNS based failover which is not the case here. What i'd ideally like to do is create a factory and have a createSocket method that takes primary and secondary host and port as args and then tries both. I am not sure though how can i make HttpConnection class invoke my method. Ankur, What's the point of trying to implement the failover support on the ProtocolSocketFactory level? What's wrong with just retrying failed requests against a secondary server in case the primary one is unavailable? Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Getting the response time for multithreaded client
On Mon, 2008-03-24 at 22:19 +0100, Uwe Kubosch wrote: Hi Oleg! Thank you for your answer. How would I do it in HttpClient 4.0? The simplest way would be to register a request interceptor to log the start time of the request execution and a response interceptor to log the time a response head (but not the response body) has been received Could you write a minimal example? Yes I could, but this example should be good enough to get you started http://svn.apache.org/repos/asf/httpcomponents/httpclient/trunk/module-client/src/examples/org/apache/http/examples/client/ClientGZipContentCompression.java How reliable is HttpClient 4.0 compared to HttpClient 3.1? Depends on your definition of reliability. 4.0 is certainly not API stable yet. Oleg Uwe On Mon, 2008-03-24 at 20:37 +0100, Oleg Kalnichevski wrote: On Thu, 2008-03-20 at 18:19 +0100, Uwe Kubosch wrote: Hi all! We are using the Commons HttpClient in a multithreaded application, and frequently there are more threads trying to use the client than the limit set by setMaxConnectionsPerHost. In this situation we still need to log the response time from the server. How can we do that? Measuring the Method.executeMethod call includes the time the thread sleeps before actually submitting the request. We are currently using Commons HttpClient 3.1 which I believe is the latest stable release. Uwe, There is no way around either upgrading to HttpClient 4.0, which has a massively better, modular architecture or hacking the source code of HttpClient 3.x Oleg Any help is greatly appreciated, even quick fixes and hacks :) Uwe - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Apache HttpClient - Using custom socket factory for failover support
On Mon, 2008-03-24 at 15:46 -0700, Ankur Shah wrote: Hi Oleg, Thanks for the response. The server that i am running (acting as a client in this particular case) is working with multiple servers (hosting different webservices) with each having a primary and a failover pair. From this perspective the thought was that rather than having different classes (all having their own business logic around invoking the service) handle the connection failed attempt, somehow have some central code that attempts to connect to the primary and fails over to the backup. I also wanted to make sure that in case of a connection failure with primary, a new pool is established against the backup server. From this perspective, an after thought that I had was this. I was going to extend the MultiThreadedHttpConnectionManager and have a failover capable connection manager. In there, i was thinking about exposing a method (or a new constructor) that allows backup host and ports to be added. This manager will cache a map of primary and backup server. In the overriden getConnectionWithTimeout method, i'd have super return me a connection that i'll check and make sure that is valid and if not, i'll go to my cache, get the backup host config and open up a connection to that. Let me know if you think this would be a good idea or not. Ankur Ankur My recommendation would be to implement the fail-over support as a service on top of HttpClient (1) execute a request against the primary host (2) if an exception is thrown, see whether it can be treated as recoverable (3) if so, mark the primary host as unavailable for a certain period time and retry the request against the secondary host (4) when the quarantine time elapses, mark the primary host as potentially available (5) repeat Hope this helps Oleg On Mon, Mar 24, 2008 at 12:47 PM, Oleg Kalnichevski [EMAIL PROTECTED] wrote: On Thu, 2008-03-20 at 22:05 -0700, Ankur Shah wrote: I am using httpclient to connect to a remote server. Now for each primary server there's also a backup failover server that i must talk to if connection attempts with the primary server fails. To achieve this what i'd ideally like to do is have HttpClient use a custom factory that tries to create a socket connection with the primary server and if that fails (or times out), it'll go about talking to backup server (and retry a few times before giving up altogether). Now, searching on the web, i did find references to a MultiHomeProtocolSocketFactory (that extends DefaultSocketFactory) that is presumably in apache contrib package. But what that does is that in the overriden createSocket method, it takes host as a param (among other thing) and then simply does a InetAddress.getAllByName which is not what i want. From what i understand this caters to a DNS based failover which is not the case here. What i'd ideally like to do is create a factory and have a createSocket method that takes primary and secondary host and port as args and then tries both. I am not sure though how can i make HttpConnection class invoke my method. Ankur, What's the point of trying to implement the failover support on the ProtocolSocketFactory level? What's wrong with just retrying failed requests against a secondary server in case the primary one is unavailable? Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Client Cert Authentication
On Thu, 2008-03-27 at 16:03 -0500, Larry Owen wrote: Hello, Can anyone help me out with an example using client certificate authentication? I'm currently trying to modify EasySSLProtocolSocketFactory.java from the contribs in the 3.1 source, but It's completely hopeless. Larry, There is no need to modify EasySSLProtocolSocketFactory. Use AuthSSLProtocolSocketFactory from the same contrib package instead. Oleg thanks, Larry -- The information contained in this communication may be confidential and is intended only for the use of the recipient(s) named above. If the reader of this communication is not the intended recipient(s), you are hereby notified that any dissemination, distribution, or copying of this communication, or any of its contents, is strictly prohibited. If you are not a named recipient or received this communication by mistake, please notify the sender and delete the communication and all copies of it. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: html form code name value pair example
On Sat, 2008-03-29 at 22:27 -0700, Corey Lundon wrote: Is my problem that my Login is based on javascript and HttpClient doesn't execute javascript like a browser does. So is what I am doing impossible. Corey No, it is not impossible. You just need to manually simulate the effect of a form submission performed through a javascript invocation. I still don't see that developer sun java login that the tutorial talks about. I need to see how others handle logging in and then I need to make a log in page like that i guess. An example of an html form login page and corresponding NVPairs would be highly helpfull still. Have you looked at the HttpClient primer? If not, please do. It explains in details how to go about form-based authentication using HttpClient http://wiki.apache.org/HttpComponents/ForAbsoluteBeginners Oleg Corey Lundon [EMAIL PROTECTED] wrote: Hi I need some help using HttpClient I would love an example of a form and the corresponding name value pairs needed. Your tutorial talks about some sun site with a form that i can't even get to. I have a form that uses a button type that executes a javascript and then in turn redirects you to another site there is only one field ... a password field so i was thinking i only need one name value pair for it but i can't seem to get past the login page do i need a name value pair for the button press and if so what would it consist of your tutorial has lots of name value pairs an action, url, username, and password also you talk about only way to see if log on was correct is to check cookie my html form doesn't use cookies as far as i know it is just a form with a field for password you hit button which runs a javascript function and if password is correct you go to new site i was just trying to check status code for a redirection status code or maybe i could just getResponseAsString and see if i am on the right site - You rock. That's why Blockbuster's offering you one month of Blockbuster Total Access, No Cost. - You rock. That's why Blockbuster's offering you one month of Blockbuster Total Access, No Cost. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Problem sending request using HttpClient when the server receiving request goes down and is started again.
2008/04/02 10:51:45:884 IST [DEBUG] header - Host: uat.abc.def.ghi[\r][\n] [4/2/08 10:51:45:884 IST] 237a6051 SystemErr R 2008/04/02 10:51:45:884 IST [DEBUG] header - Host: uat.abc.def.ghi[\r][\n] [4/2/08 10:51:45:884 IST] 2ff8605c SystemErr R 2008/04/02 10:51:45:884 IST [DEBUG] header - Cookie: $Version=0; JSESSIONID=htQCWkn5-9X9ZXuqlf6W9AW:-1; $Path=/[\r][\n] [4/2/08 10:51:45:884 IST] 237a6051 SystemErr R 2008/04/02 10:51:45:884 IST [DEBUG] header - Cookie: $Version=0; JSESSIONID=htQCWkn5-9X9ZXuqlf6W9AW:-1; $Path=/[\r][\n] [4/2/08 10:51:45:884 IST] 2ff8605c SystemErr R 2008/04/02 10:51:45:884 IST [DEBUG] header - [\r][\n] [4/2/08 10:51:45:884 IST] 237a6051 SystemErr R 2008/04/02 10:51:45:884 IST [DEBUG] header - [\r][\n] [4/2/08 10:51:46:009 IST] 2ff8605c SystemErr R 2008/04/02 10:51:46:009 IST [DEBUG] header - HTTP/1.1 200 OK[\r][\n] [4/2/08 10:51:46:009 IST] 2ff8605c SystemErr R 2008/04/02 10:51:46:009 IST [DEBUG] header - Date: Wed, 02 Apr 2008 05:24:39 GMT[\r][\n] [4/2/08 10:51:46:009 IST] 2ff8605c SystemErr R 2008/04/02 10:51:46:009 IST [DEBUG] header - Server: IBM_HTTP_Server[\r][\n] [4/2/08 10:51:46:009 IST] 2ff8605c SystemErr R 2008/04/02 10:51:46:009 IST [DEBUG] header - Transfer-Encoding: chunked[\r][\n] [4/2/08 10:51:46:009 IST] 2ff8605c SystemErr R 2008/04/02 10:51:46:009 IST [DEBUG] header - Content-Type: text/html;charset=ISO-8859-1[\r][\n] [4/2/08 10:51:46:009 IST] 2ff8605c SystemErr R 2008/04/02 10:51:46:009 IST [DEBUG] header - Content-Language: en-US[\r][\n] [4/2/08 10:51:46:056 IST] 237a6051 SystemErr R 2008/04/02 10:51:46:056 IST [DEBUG] HttpMethodBase - Resorting to protocol version default close connection policy [4/2/08 10:51:46:056 IST] 2ff8605c SystemErr R 2008/04/02 10:51:46:056 IST [DEBUG] HttpMethodBase - Resorting to protocol version default close connection policy [4/2/08 10:51:46:056 IST] 237a6051 SystemErr R 2008/04/02 10:51:46:056 IST [DEBUG] HttpMethodBase - Should NOT close connection, using HTTP/1.1 [4/2/08 10:51:46:056 IST] 2ff8605c SystemErr R 2008/04/02 10:51:46:056 IST [DEBUG] HttpMethodBase - Should NOT close connection, using HTTP/1.1 [4/2/08 10:51:46:056 IST] 237a6051 SystemErr R 2008/04/02 10:51:46:056 IST [DEBUG] HttpConnection - Releasing connection back to connection manager. [4/2/08 10:51:46:056 IST] 2ff8605c SystemErr R 2008/04/02 10:51:46:056 IST [DEBUG] HttpConnection - Releasing connection back to connection manager. [4/2/08 10:51:46:056 IST] 2ff8605c SystemErr R 2008/04/02 10:51:46:056 IST [DEBUG] MultiThreadedHttpConnectionManager - Freeing connection, hostConfig=HostConfiguration[host=https://uat.abc.def.ghi] [4/2/08 10:51:46:056 IST] 237a6051 SystemErr R 2008/04/02 10:51:46:056 IST [DEBUG] MultiThreadedHttpConnectionManager - Freeing connection, hostConfig=HostConfiguration[host=https://uat.abc.def.ghi] [4/2/08 10:51:46:056 IST] 2ff8605c SystemErr R 2008/04/02 10:51:46:056 IST [DEBUG] IdleConnectionHandler - Adding connection at: 1207113706056 [4/2/08 10:51:46:056 IST] 2ff8605c SystemErr R 2008/04/02 10:51:46:056 IST [DEBUG] MultiThreadedHttpConnectionManager - Notifying no-one, there are no waiting threads [4/2/08 10:51:46:056 IST] 237a6051 SystemErr R 2008/04/02 10:51:46:056 IST [DEBUG] IdleConnectionHandler - Adding connection at: 1207113706056 [4/2/08 10:51:46:056 IST] 237a6051 SystemErr R 2008/04/02 10:51:46:056 IST [DEBUG] MultiThreadedHttpConnectionManager - Notifying no-one, there are no waiting threads What can be the possible reason for this? What can be the way(s) to avoid this? Regards, Gagneet Oleg Kalnichevski [EMAIL PROTECTED] wrote: On Thu, 2008-03-27 at 11:53 +, gagneet wrote: Hi Oleg, As I was using Singleton HttpClient object the JSESSIONID persisted for every request. I have cleared the persistence by calling clearCookies() method on the HttpState object corresponding to the HttpClient object. Should I call this method after every call or should I call it only if the connection is timed out? Please advice. Regards, Gagneet Gagneet, It really depends on how your application manages conversation state on the client side. Usually applications are expected to maintains an instance of HttpState per distinct user / thread of execution. This way, you would not have to mix cookies that belong to different users, and if an individual connection times out, the HttpState instance associated with it could simply be discarded along with the JSESSIONID session cookie. Hope this helps Oleg - Why delete messages? Unlimited storage is just a click away. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Why delete messages? Unlimited storage is just a click away
Re:
vijay shanker wrote: Hello friends. I am very new to this API and I got to make some decisions. - what should I use in production in my team.(HttpClient4, HttpClient3 or HttpCore). How they are different with each other. * HttpClient 3.x is very stable and relatively well documented * HttpClient 4.0 has a much better, cleaner architecture, is 15-30% faster, but it is still not API stable and does not implement certain features such as NTLM authentication. * HttpCore is a set of low level _transport_ components that can be used to implement HTTP client, server and proxy services. HttpCore does not provide connection management, state management, authentication and other higher level components. HttpClient 4.0 is built on top of HttpCore. Oleg - I need some document regarding connection management and authentication. As I am not aware of these thing I need a very basic example of things that happens to execute at both client desktop and server side in case of authentication. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: httptunnel client 4.0 alpha 3 and preemtive authentication
Sebastiaan van Erk wrote: Hi, I'm trying to use preemptive authentication because I have a non-repeatable (dyanimc) request entity. Currently when posting, this goes wrong because I get a 401 Unauthorized but it has already sent some of the data, and when it tries again with the basic authentication credentials the post fails... (I've implemented the writeTo(OutputStream) method on the Entity). I'm wondering by the way how it can even do the request again because isRepeatable on the entity returns false... I construct my http client as follows: final HttpParams httpParams = new BasicHttpParams(); HttpClientParams.setAuthenticationPreemptive(httpParams, true); final DefaultHttpClient httpClient = new DefaultHttpClient(httpParams); I searched the source for references to the ClientPNames.PREEMPTIVE_AUTHENTICATION constant, but I couldn't find any, so I'm wondering, is preemptive authentication implemented in 4.0 alpha 3 yet? Sebastiaan, There is no support for preemptive authentication at the moment. Anyways, you'll be _much_ better off using the 'expect-continue' handshake or making sure your entity is repeatable. Hope this helps Oleg Or am I doing something wrong? Regards, Sebastiaan - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Duplicate requests
Aaron Shettleroe wrote:
It appears that HttpClient is sending the same request more than once.
Unlikely, unless the RetryHandler is configured to do so.
I tried to code against this.
Here is some code snippets:
private static HttpClient httpClient;
...
final HttpConnectionManagerParams params =
new HttpConnectionManagerParams();
// set some params: timeouts, max conns
...
final MultiThreadedHttpConnectionManager connectionManager =
new MultiThreadedHttpConnectionManager();
connectionManager.setParams(params);
httpClient = new HttpClient(connectionManager);
...
// I define a RetryHandler
getMethod.getParams().setParameter(
HttpMethodParams.RETRY_HANDLER,
updateRetryHandler);
...
// My retry logic. Only retry if a connection exception:
// Don't retry if the maximum retry count has been reached
if(executionCount = MAX_RETRY_COUNT)
{
return false;
}
else if(!method.isRequestSent()
(exception instanceof ConnectException
|| exception instanceof ConnectTimeoutException))
{
return true;
}
return false;
However, the request is being sent more than once.
Please post a context/wire log of the session.
http://hc.apache.org/httpclient-3.x/logging.html
Would the stale checking be the cause of this? Does the stale checking
actually send the request?
No, it does not
Oleg
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: HttpClient Sockettimeout does not work
[EMAIL PROTECTED] wrote:
Hello,
I am using httpclient 3.1. I am trying to make a http request that should
timeout when a response is not revceived within 2 minutes.
See definition of the socket timeout
Oleg
The requirement
is also to retry 3 times before throwing back an application exception.
When I set the socket timeout to 90,000 milliseconds, the timeout never
works. Sometimes, I see the connection waiting for more than 2 mins for a
response instead of throwing a SocketTimeoutException. Below is the code
sample that I am using to make the HttpRequest.
String proxyPortNumber=
(String)PropertiesUtil.getIonProps().getProperty(IonConstants.PROXYPORT);
String proxyServer =
(String)PropertiesUtil.getIonProps().getProperty(IonConstants.PROXYSERVER);
HttpClient client = new HttpClient();
client.getHostConfiguration().setProxy(proxyServer,
Integer.parseInt(proxyPortNumber));
Integer connWaitTime=new Integer(12);
PostMethod postMethod = new PostMethod(url);
HttpMethodRetryHandler retryhandler = new
HttpMethodRetryHandler() {
public boolean retryMethod(
final HttpMethod method,
final IOException exception,
int executionCount) {
if (executionCount = 3) {
// Do not retry if over max retry
count
return false;
}
System.out.println(Retrying Request:: +
executionCount);
if (exception instanceof
java.net.SocketTimeoutException) {
// Retry if the server dropped
connection on us
return true;
}
if (!method.isRequestSent()) {
// Retry if the request has not
been sent fully or
// if it's OK to retry methods
that have been sent
return true;
}
// otherwise do not retry
return false;
}
};
client.getParams().setParameter(HttpMethodParams.RETRY_HANDLER,
retryhandler);
//client.getParams().setParameter(HttpMethodParams.SO_TIMEOUT,connWaitTime);
postMethod.getParams().setParameter(HttpMethodParams.SO_TIMEOUT,connWaitTime);
System.out.println(ConnWaitTime :: +
postMethod.getParams().getParameter(HttpMethodParams.SO_TIMEOUT));
postMethod.setRequestEntity(new
StringRequestEntity(xmlData, null, null));
postMethod.setRequestHeader(Content-type,
text/xml);
int responseCode =
client.executeMethod(postMethod);
BufferedReader rd = new BufferedReader(new
InputStreamReader(postMethod.getResponseBodyAsStream()));
String line;
while ((line = rd.readLine()) != null)
{
response += line + \n;
}
//System.out.println(responseBody +
responseBody);
postMethod.releaseConnection();
I hope you can help me.
Thanks
-
This transmission may contain information that is privileged,
confidential, legally privileged, and/or exempt from disclosure
under applicable law. If you are not the intended recipient, you
are hereby notified that any disclosure, copying, distribution, or
use of the information contained herein (including any reliance
thereon) is STRICTLY PROHIBITED. Although this transmission and
any attachments are believed to be free of any virus or other
defect that might affect any computer system into which it is
received and opened, it is the responsibility of the recipient to
ensure that it is virus free and no responsibility is accepted by
JPMorgan Chase Co., its subsidiaries and affiliates, as
applicable, for any loss or damage arising in any way from its use.
If you received this transmission in error, please immediately
contact the sender and destroy the material in its entirety,
whether in electronic or hard copy format. Thank you.
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: httptunnel client 4.0 alpha 3 and preemtive authentication
Sebastiaan van Erk wrote: Hi Oleg, Thanks for the reply, I figured as much. I prefer to use expect/continue, which I actually do when the proxy supports it, but in some cases unfortunately it doesn't and I must fall back to preemptive. What does seem strange to me is that HttpClient *does* retry the request with the authentication header, as it seems to me that it is impossible if the entity is not repeatable and some data is already sent. An exception might be better? Can you post a wire/context log of the session? Oleg Regards, Sebastiaan Oleg Kalnichevski wrote: Sebastiaan van Erk wrote: Hi, I'm trying to use preemptive authentication because I have a non-repeatable (dyanimc) request entity. Currently when posting, this goes wrong because I get a 401 Unauthorized but it has already sent some of the data, and when it tries again with the basic authentication credentials the post fails... (I've implemented the writeTo(OutputStream) method on the Entity). I'm wondering by the way how it can even do the request again because isRepeatable on the entity returns false... I construct my http client as follows: final HttpParams httpParams = new BasicHttpParams(); HttpClientParams.setAuthenticationPreemptive(httpParams, true); final DefaultHttpClient httpClient = new DefaultHttpClient(httpParams); I searched the source for references to the ClientPNames.PREEMPTIVE_AUTHENTICATION constant, but I couldn't find any, so I'm wondering, is preemptive authentication implemented in 4.0 alpha 3 yet? Sebastiaan, There is no support for preemptive authentication at the moment. Anyways, you'll be _much_ better off using the 'expect-continue' handshake or making sure your entity is repeatable. Hope this helps Oleg Or am I doing something wrong? Regards, Sebastiaan - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Problem sending request using HttpClient when the server receiving request goes down and is started again.
On Thu, 2008-04-03 at 00:57 +0530, Gagneet Chadha wrote: Hi Oleg, No I was setting HttpState object into HttpClient object. Now I see where I went wrong. But the only API method which will allow me to do that is executeMethod(HostConfiguration hostconfig, HttpMethod method, HttpState state). Till now I was using executeMethod(HttpMethod method).Now that means I will have to create a HostConfiguration object which should be singleton. Am I right? Not really. It should not be a singleton. Anyways, you can pass null as HostConfiguration, if you are executing a request with an absolute request URI. HttpClient will create an instance of HostConfiguration for you. Hope this helps Oleg Thanks once again for your help. Regards, Gagneet - Original Message - From: Oleg Kalnichevski [EMAIL PROTECTED] To: HttpClient User Discussion httpclient-users@hc.apache.org Sent: Thursday, April 03, 2008 12:14 AM Subject: Re: Problem sending request using HttpClient when the server receiving request goes down and is started again. gagneet wrote: Hi Oleg, Thanks for your inputs. Now I am making a new HttpState object for each request made. But have found a new problem. This problem occurs when the requests are concurrent. What is evident is that after redirection happens the JSESSIONID of both the requests become same. Could not figure out what makes the JSESSIONID for one request to be overwritten by JSESSIONID of second request. I am using singleton HttpClient object which is created using MultiThreadedHttpConnectionManager. You are passing HttpState as a parameter to the HttpClient#executeMethod, aren't you? Oleg What can be the possible reason for this? What can be the way(s) to avoid this? Regards, Gagneet Oleg Kalnichevski [EMAIL PROTECTED] wrote: On Thu, 2008-03-27 at 11:53 +, gagneet wrote: Hi Oleg, As I was using Singleton HttpClient object the JSESSIONID persisted for every request. I have cleared the persistence by calling clearCookies() method on the HttpState object corresponding to the HttpClient object. Should I call this method after every call or should I call it only if the connection is timed out? Please advice. Regards, Gagneet Gagneet, It really depends on how your application manages conversation state on the client side. Usually applications are expected to maintains an instance of HttpState per distinct user / thread of execution. This way, you would not have to mix cookies that belong to different users, and if an individual connection times out, the HttpState instance associated with it could simply be discarded along with the JSESSIONID session cookie. Hope this helps Oleg - Why delete messages? Unlimited storage is just a click away. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Why delete messages? Unlimited storage is just a click away. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: ChunkedOutputStream and flushing
On Wed, 2008-04-02 at 21:55 +0200, Sebastiaan van Erk wrote:
Hi,
I have a question on the chunked output stream: one of the nice things
about chunked encoding is that you can send your data in little parts.
However, flush() on ChunkedOutputStream does not flush the buffer:
/**
* Flushes the underlying stream, but leaves the internal buffer alone.
* @throws IOException
*/
public void flush() throws IOException {
this.out.flush();
}
This is kind of against the spec of flush on OutputStream:
/**
* Flushes this output stream and forces any buffered output bytes
* to be written out. The general contract of codeflush/code is
* that calling it is an indication that, if any bytes previously
* written have been buffered by the implementation of the output
* stream, such bytes should immediately be written to their
* intended destination.
* ...
I was wondering what the motivation was behind disabling the flush()
option?
To avoid tiny chunks if for some reason the output stream gets flushed
too often.
Feel free to open a bug in JIRA for this issue.
Oleg
Generally if a client does not want to flush the data they won't
call flush, which will cause all the chunks to attain the full chunk
size anyway...
Regards,
Sebastiaan
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: HttpClient with JSF pages
On Wed, 2008-04-02 at 18:46 -0500, Ranthydev Rereddy wrote: Hello. I am attempting to use HttpClient to access a jsf page for uploading files. The jsf page has a submit button that invokes an action on a bean. When I first access the jsf page it returns the file upload page with a jsessionid. When I submit my request with the post method for file upload, the HttpClient instance automatically sends the session id but it does not invoke the action on the bean. Has this HttpClient 3.1 been tested with jsf pages? From the HTTP protocol standpoint JSF is no different than any other web frameworks. They all generate some kind of HTML markup at the end of the day. Have a look at the HttpClient programming primer and double-check your code. http://wiki.apache.org/HttpComponents/ForAbsoluteBeginners Oleg We do not receive any errors and the bean is being instantiated but the bean method is not being invoked. There is no login involved. IE works for the same url. I will happily consider any responses. Thank you, Ranthydev Rereddy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Http connection management/throttling for various applications sending GET requests to the same web server using a single web service
On Thu, 2008-04-03 at 22:22 -0700, Sujith Joseph wrote: Hi, I am trying to implement a way of throttling http connections to a web server from a web service, using httpclient, so that the web server wouldn't be overloaded with more than 'x' concurrent connections, from my web service. (I also need to ensure that the http connections available in the pool are utilized efficiently). Within the web service, the http connections are created for various types of (client) applications and each such application has a preferred limit on the number of concurrent connections, it can create to the server. However if the connections requested by a particular application gets higher than its preferred limit, it could consume free http connections allocated to other applications. The thus eagerly acquired connections must be released, if the rest of the applications requests back for their allotted connections. To implement this, I am thinking of using a custom object pool of SimpleHttpConnectionManager objects for each of the applications and have a logic to maintain and regulate the number of these objects allocated/used, based on the load, during new connection requests. For each web server request, the plan is to create a connection from the SimpleHttpConnectionManager using a HttpClient and release it, after getting a response. Sujith, SimpleHttpConnectionManager holds only one connection object. There is no benefit in pooling SCM instances. You can as well maintain a pool of HTTP connections. Can someone help me with a better solution for this? I know that the MultiTHreadedConnectionManager keeps a pool of connections, but I am not sure, whether we can dynamically manipulate the max connections to a host, after an MTCM object creation (My understanding on MTCM is that it creates as many threads as the max connection limit on its creation). Is there something in Httpclient4.x that might help me in this? HttpClient 4.x API is much, much more modular and flexible. You'll need to write a lot less custom code. I am also thinking of a custom implementation of the HttpConnectionManager for this. Please advise. I do not see a way around writing a custom connection manager, which is not an easy task. Oleg Thanks, Sujith - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Dealing with a server that doesn't always respond with an HTTP response header
On Fri, 2008-04-04 at 14:14 -0400, Jeremy Mayhew wrote: Hi, I am using htmlunit+httpclient to test an embedded web application. Unfortunately there is a large set of requests for which this embedded webserver does not respond with an HTTP header, but instead splats the body without a header. The httpclient rightly throws a protocol exception out on the malformed response. (Firefox and IE will still render the pages without the HTTP response header, so it works for the user case. I can't change the embedded web server code.) I was wondering if there is an easy way to configure httpclient to do best effort on the response and assume that the response is OK so that the response body can be processed as a valid HTML page. Not easily. HttpClient assumes at least a certain degree of sanity in HTTP responses. If your application really needs to handle a complete garbage, the only way I see is to implement a custom HTTP connection that can test whether an incoming stream contains a valid HTTP response or an HTML content, and generate a dummy response in the latter case. One note though, this is possible with HttpClient 4.0 only. You are out of luck with HttpClient 3.x Hope this helps Oleg Thanks! Jeremy Mayhew - Notice: This e-mail is intended solely for use of the individual or entity to which it is addressed and may contain information that is proprietary, privileged and exempt from disclosure under applicable law. If the reader is not the intended recipient or agent responsible for delivering the message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. This communication may also contain data subject to U.S. export laws. If so, that data subject to the International Traffic in Arms Regulation cannot be disseminated, distributed or copied to foreign nationals, residing in the U.S. or abroad, absent the express prior approval of the U.S. Department of State. If you have received this communication in error, please notify the sender by reply e-mail and destroy the e-mail message and any physical copies made of the communication. Thank you. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Unexpectedly long request time-outs
On Mon, 2008-04-07 at 23:31 +0100, Christiaan Lamprecht wrote: Thanks Oleg, Socket timeout defines a maximum of period inactivity between two _consecutive_ I/O operations. I've been looking through the connection manager code trying to find out what effect a ConnectionTimeout and SocketTimeout would have on the subsequent httpclient.executeMethod(hostconfig, method, null) call if the same connection was reused. Christiaan, This has nothing to do with HTTP requests. Socket timeout defines a maximum of period inactivity between two consecutive I/O operations on _that_ socket. For example, if the socket timeout is set to 1000 ms and the server always sends a new IP packet within 999 ms, the socket will never time out. i.e. does either timeout change something significant, so that the connection manager can't merely reuse the connection in the pool. I do not think so Oleg Any thoughts? Thanks again Christiaan - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Using NTLM auth with expect continue
On Thu, 2008-04-10 at 18:57 -0400, Tony Thompson wrote: The request URL is perfectly valid. The response from the server seems to suggest otherwise As I stated in an earlier email, I can do this same exact thing WITHOUT forcing a connection closed before the POST and it works fine. If I force the connection closed before the POST so that the HTTP client has to renegotiate the NTLM auth, it fails. In my test, I am forcing the connection closed so that HTTPClient has to open a new connection (and authenticate it) for the POST. I need to get the expect continue handshake to work before I post the data because I don't want to buffer the content since it has to be posted several times during the handshake. I understand what you are trying to do, but I see nothing wrong with the requests generated by HttpClient Oleg Make sense? Tony -Original Message- From: Oleg Kalnichevski [mailto:[EMAIL PROTECTED] Sent: Thursday, April 10, 2008 5:46 PM To: HttpClient User Discussion Subject: RE: Using NTLM auth with expect continue On Thu, 2008-04-10 at 17:19 -0400, Tony Thompson wrote: Hopefully this has what you need: ... DEBUG (04/10) 17:10:58 [httpclient.wire.content]: h1Bad Request (Invalid URL)/h1 ... This seems to have nothing to do with NTLM authentication. The request appears to have been rejected due to the invalid request URL. Hope this helps Oleg -Original Message- From: Oleg Kalnichevski [mailto:[EMAIL PROTECTED] Sent: Thursday, April 10, 2008 2:52 PM To: HttpClient User Discussion Subject: RE: Using NTLM auth with expect continue On Thu, 2008-04-10 at 09:59 -0400, Tony Thompson wrote: I have debug enabled for org.apache.commons.httpclient so, I am not sure why nothing comes out. Is there a specific class I should turn on debug for that would give you the info you need? Tony I will not be able to be of any meaningful help unless you manage to produce a _complete_ session log that also includes context information (especially that about connections being open and closed, and credentials management) http://hc.apache.org/httpclient-3.x/logging.html Oleg Maybe my last message wasn't clear enough on the POST without NTLM credentials. I tell HTTPClient to execute a method without providing the credentials, I detect the 401 and then I start over and POST again with the credentials. I will probably change it at some point so that I preemptively provide the credentials but at this point it is irrelevant. The POST fails if the credentials are there or not. Here is the same limited trace of the request with the NTLM header. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] This message (and any associated files) is intended only for the use of the individual or entity to which it is addressed and may contain information that is confidential, subject to copyright or constitutes a trade secret. If you are not the intended recipient you are hereby notified that any dissemination, copying or distribution of this message, or files associated with this message, is strictly prohibited. If you have received this message in error, please notify us immediately by replying to the message and deleting it from your computer. Messages sent to and from Stoneware, Inc. may be monitored. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Using NTLM auth with expect continue
: https://bb.s 0340 74 6f 6e 65 2d 77 61 72 65 2e 63 6f 6d 2f 6e 74 tone-ware.com/nt 0350 6c 6d 74 65 73 74 2f 66 6f 72 6d 2e 68 74 6d 6c lmtest/form.html 0360 0d 0a 43 6f 6e 74 65 6e 74 2d 54 79 70 65 3a 20 ..Content-Type: 0370 61 70 70 6c 69 63 61 74 69 6f 6e 2f 78 2d 77 77 application/x-ww 0380 77 2d 66 6f 72 6d 2d 75 72 6c 65 6e 63 6f 64 65 w-form-urlencode 0390 64 0d 0a 43 6f 6e 74 65 6e 74 2d 4c 65 6e 67 74 d..Content-Lengt 03a0 68 3a 20 32 31 0d 0a 48 6f 73 74 3a 20 6d 73 73 h: 21..Host: mss 03b0 62 30 31 2e 73 74 6f 6e 65 2d 77 61 72 65 2e 63 b01.stone-ware.c 03c0 6f 6d 0d 0a 45 78 70 65 63 74 3a 20 31 30 30 2d om..Expect: 100- 03d0 63 6f 6e 74 69 6e 75 65 0d 0a 0d 0a 50 4f 53 54 continuePOST 03e0 20 2f 6e 74 6c 6d 74 65 73 74 2f 64 69 73 70 6c /ntlmtest/displ 03f0 61 a -Original Message- From: Oleg Kalnichevski [mailto:[EMAIL PROTECTED] Sent: Friday, April 11, 2008 3:08 AM To: HttpClient User Discussion Subject: RE: Using NTLM auth with expect continue On Thu, 2008-04-10 at 18:57 -0400, Tony Thompson wrote: The request URL is perfectly valid. The response from the server seems to suggest otherwise As I stated in an earlier email, I can do this same exact thing WITHOUT forcing a connection closed before the POST and it works fine. If I force the connection closed before the POST so that the HTTP client has to renegotiate the NTLM auth, it fails. In my test, I am forcing the connection closed so that HTTPClient has to open a new connection (and authenticate it) for the POST. I need to get the expect continue handshake to work before I post the data because I don't want to buffer the content since it has to be posted several times during the handshake. I understand what you are trying to do, but I see nothing wrong with the requests generated by HttpClient Oleg Make sense? Tony -Original Message- From: Oleg Kalnichevski [mailto:[EMAIL PROTECTED] Sent: Thursday, April 10, 2008 5:46 PM To: HttpClient User Discussion Subject: RE: Using NTLM auth with expect continue On Thu, 2008-04-10 at 17:19 -0400, Tony Thompson wrote: Hopefully this has what you need: ... DEBUG (04/10) 17:10:58 [httpclient.wire.content]: h1Bad Request (Invalid URL)/h1 ... This seems to have nothing to do with NTLM authentication. The request appears to have been rejected due to the invalid request URL. Hope this helps Oleg -Original Message- From: Oleg Kalnichevski [mailto:[EMAIL PROTECTED] Sent: Thursday, April 10, 2008 2:52 PM To: HttpClient User Discussion Subject: RE: Using NTLM auth with expect continue On Thu, 2008-04-10 at 09:59 -0400, Tony Thompson wrote: I have debug enabled for org.apache.commons.httpclient so, I am not sure why nothing comes out. Is there a specific class I should turn on debug for that would give you the info you need? Tony I will not be able to be of any meaningful help unless you manage to produce a _complete_ session log that also includes context information (especially that about connections being open and closed, and credentials management) http://hc.apache.org/httpclient-3.x/logging.html Oleg Maybe my last message wasn't clear enough on the POST without NTLM credentials. I tell HTTPClient to execute a method without providing the credentials, I detect the 401 and then I start over and POST again with the credentials. I will probably change it at some point so that I preemptively provide the credentials but at this point it is irrelevant. The POST fails if the credentials are there or not. Here is the same limited trace of the request with the NTLM header. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] This message (and any associated files) is intended only for the use of the individual or entity to which it is addressed and may contain information that is confidential, subject to copyright or constitutes a trade secret. If you are not the intended recipient you are hereby notified that any dissemination, copying or distribution of this message, or files associated with this message, is strictly prohibited. If you have received this message in error, please notify us immediately by replying to the message and deleting it from your computer. Messages sent to and from Stoneware, Inc. may be monitored. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED
Re: calculate bandwidth of HttpClient
On Tue, 2008-04-15 at 21:41 +0300, Eugeny N Dzhurinsky wrote: On Tue, Apr 15, 2008 at 07:49:12PM +0200, Oleg Kalnichevski wrote: On Tue, 2008-04-15 at 19:43 +0300, Eugeny N Dzhurinsky wrote: Hello there! Is there any way to get the bandwidth, used by HttpClient to download pages? Currently we are using version 3.0.1 of HttpClient, and we had implemented the ProtocolSocketFactory which returns an instance of the custom Socket class, which returns the InputStream which is able to return amount of bytes read from the content stream. Knowing the amount of bytes and time of download, it is trivial to calculate the bandwidth - however the real bandwidth on the network interface is about 2 times more than reported by our application. And we are sure there are no other network activity on the interface, and our calculations seems to be right - the amount of downloaded data equals to the amount of data saved to the persisted storage. It is possible that the remote server is just not able to saturate all available bandwidth. Well, it does. For example, our application displays 1 mbit/seconds, but pktshow/trafshow displays 2 mbit/seconds of bandwidth usage on the physical interface. And we are absolutely sure there are no another application which are accessing network, also we are 99.9% sure there are no errors / miscalculations in our application - the things are pretty straightforward. So the question for now is - are there any other ways to get the bandwidth usage information from HttpClient? None I know of And is the usage of ProtocolSocketFactory the recommended way to get such information? Yes, it is Oleg Thank you in advance! - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: HttpEntity.consumeContent() needs to much time. (HC 4.0 beta)
On Thu, 2008-04-17 at 11:40 +0200, Jan Bracker wrote: Hello, i'm using the DefaultHttpClient for downloading bigger files. When i want to abort a download i call response.getEntity().consumeContent() Why? The purpose of HttpEntity#consumeContent() method is to ensure the underlying connection is re-usable for subsequent requests, not to abort the actual request (I assume that is what i should do if i want to close connections of HttpClient). But the longer content downloaded on that connections get the more time it takes to close the given connection. Please use #abort method instead. I allready looked at source and found the problem within ChunkedInputStream. consumeContent() calls close() to the InputStream and that close results in a call of exhaustInputStream() and that reads the complete rest of content sent by the host. Why is this implemented like that? Why not just close the connection/stream? To be able to use persistent connection reliably. This behaviour is very annoying when wanting to stop a request that leads to a response entity of severel megabytes, especially on connections that are volume payed or have a slow bandwidth. Is there a way to close the connection (consumeContent) of my current response without having to read the complete content? How do i solve my problem of closing connections quick and abrupt? See above. Oleg Greetings, Jan Bracker - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: request GET line
On Sun, 2008-05-04 at 02:55 -0400, Bashiro wrote: I sent this message hoping to get a response but all invain. I hereby send again in case it escaped others. I really need help Thanks. Hello, I will be grateful for any kind of help... The authentication site is: https://www.ChangeIP.com/nic/update But httpclient always searches for authentication at: https://www.ChangeIP.com/ I do not understand what you mean by 'searching' for authentication. This results in authentication failure. Even though I declare connectTo =https://www.ChangeIP.com/nic/update new AuthScope(connectTo, 443, realm) httpclient stops at https://www.ChangeIP.com/ and wait for authentication. The first parameter of AuthScope is expected to be a host name, not a URI. Hope this helps Oleg Now my question is; how do I let httpclient to use hhttps://www.ChangeIP.com/nic/update instead of https://www.ChangeIP.com/ to search for authentication before the GET actually comes ? Thanks in advance Bashiro Drammen-Norway ___ No banners. No pop-ups. No kidding. Make My Way your home on the Web - http://www.myway.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[ANNOUNCEMENT] HttpComponents HttpClient 4.0-alpha4 released
The HttpComponents project is pleased to announce the release of HttpComponents HttpClient 4.0-alpha4. This release marks the completion of the overhaul of the connection management code in HttpClient. All known shortcomings of the old HttpClient 3.x connection management API have been addressed. NTLM authentication remains the only missing major feature in the new codeline that prevents us from moving toward the API freeze. There has been a number of important bug fixes since ALPHA3. All upstream projects are encouraged to upgrade to the latest release. --- Download - http://hc.apache.org/downloads.cgi Release notes - http://www.apache.org/dist/httpcomponents/httpclient/RELEASE_NOTES.txt HttpComponents site - http://hc.apache.org/ - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: generics issue in httpclient 4.0-alpha4
On Sat, 2008-05-10 at 11:03 +0200, Sebastiaan van Erk wrote:
Hi,
First of all, thanks for the new httpclient release. :-)
When I was updating my project I ran into one small generics issue; the
following code will not compile:
new UrlEncodedFormEntity(Collections.singletonList(new
BasicNameValuePair(a, b)));
because UrlEncodedFormEntity wants a ListNameValuePair and the above
code results in a ListBasicNameValuePair. Considering that the list is
probably read only in the httpclient code, the declaration of the
UrlEncodedFormEntity constructor could be changed to:
public UrlEncodedFormEntity (
final List ? extends NameValuePair parameters,
final String encoding) throws UnsupportedEncodingException {
super(URLEncodedUtils.format(parameters, encoding),
encoding);
setContentType(URLEncodedUtils.CONTENT_TYPE);
}
(note the List? extends NameValuePair).
Sebastiaan,
It is obviously a bug. Please open a JIRA issue for it. We also happily
accept patches ;-)
Oleg
Regards,
Sebastiaan
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: Regarding closing of idle connections for non-reusable / non-threaded httpClient instances to ensure complete connection closure
On Thu, 2008-05-15 at 20:27 -0700, Sujith Joseph wrote: Hi Oleg, Thanks for your reply. I really appreciate your help. I have couple of more questions on this. I am currently using httpclient 3.0 library. SimpleConnectionManager.shutdown () method is not available in 3.0 libs. if I am continuing to use these libraries in my application, can I use httpClient.getConnectionManager(). closeIdleConnections(1) to close of connections?. Yes, you can. #closeIdleConnections(0) should be even better Would setting small values like 1 millisecond for idle connection timeout be an overhead to the JVM and impact performance? I do not think so. Would this act as a replacement for SimpleConnectionManager.shutdown()? Partially. At the very least it would leave no idle connection open, which should result in closure of all open sockets. Based on your reply, I assume that you recommend re-using the same httpClient instance for all http requests, instead of creating httpClient instances for each http requests, even if we are not using MultiThreadedConnectionManager. Is it because, that the latter approach has an overhead of creating/closing more connections and no re-use of connections and thereby giving less throughput?. Yes, it is. Just make sure to not execute requests from multiple threads without synchronization. Are you aware of any issues in using HttpClient 3.1 libraries in Websphere Application Server (WAS) 5.1. ? I have not tried this yet. But if you already know this, I think, I could save some testing time . No, I am not. However, I have been lucky enough to have not touched WAS for 4 years. Oleg Thanks, Sujith On Thu, May 15, 2008 at 12:10 PM, Oleg Kalnichevski [EMAIL PROTECTED] wrote: On Tue, 2008-05-13 at 17:36 -0700, Sujith Joseph wrote: Hi, I would like to know whether it is necessary to invoke httpClient.getConnectionManager(). closeIdleConnections(1000), after creating an httpClient (httpClient = new HttpClient()), creating an httpmethod, executing the httpmethod and performing method.releaseConnection(), to ensure that the http connections are properly closed, since I am creating an httpClient instance for each and every http request (since I am not using MultiThreadedConnectionManager). That basically means you are telling HttpClient to close connections that have been idle for 1000 ms. I am not sure that is what you want. If I am creating a httpClient instance (without using MutliThreadedConnectionManager), Does a SimpleHttpConnectionManager gets created for each httpClient creation behind the scenes. Yes, it does. Can I perform a type cast like this ((SimpleHttpConnectionManager)httpClient.getConnectionManager).shutdown() to ensure complete release of connections. Yes, you can. Moreover, you really should. If I am not using the httpClient instance anymore after executing a http method request, do you recommend closing off idle connections / shut down ConnectionManager, to avoid creation of too many open sockets in CLOSE_WAIT state. You _should_ shut down the connection manager if you no longer need it and want to ensure it releases all resources it currently holds Can I re-use the same httpClient instance to perform http method executions, if I am not using a multi-threaded Connection Manager? Absolutely. Oleg Thanks, Sujith - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Pattern for batch file transfers
On Fri, 2008-05-16 at 09:17 -0700, stanlick wrote: Greetings -- I am searching for a generic solution for transferring files between peers using HttpClient and FileUpload. I see a myriad of techniques, but would really appreciate a solution that: 1) streams in/out (don't want to worry about OutOfMemoryErrors) 2) imposes no size limits 3) supports any payload Please see http://hc.apache.org/httpclient-3.x/performance.html#Request_Response_entity_streaming Can someone point me in the right direction? Is HttpClient 4.x the preferred version? Is it still beta? HttpClient 4.0 is still ALPHA. So, if HttpClient 3.x works for you, stick to it. Oleg Thanks, Scott - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Sending serialized object and stream
On Fri, 2008-05-16 at 16:05 -0400, [EMAIL PROTECTED] wrote:
Hi All,
I've requirement to send both serialized java object and stream object.
Here is the code that sends serialized object
ByteArrayOutputStream requestStream = new
ByteArrayOutputStream();
ObjectOutputStream objectOutput = new
ObjectOutputStream(requestStream);
objectOutput.writeObject(request);
objectOutput.close();
InputStream requestContent = new
ByteArrayInputStream(requestStream.toByteArray());
if (requestContent != null) {
post.setRequestEntity(new
InputStreamRequestEntity(requestContent));
}
We need to optionally send stream object as well. I believe we need to
use multipart request entity. The HC currently supports FilePart and
StringPart. Do we need to write another type of Part like StreamPart to
do so?
Yes, you do
Oleg
Help is appreciated.
Thanks in advance
Valli
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: Newbie question port 8080
- SunJGSS 1.0: Sun (Kerberos v5) 2008/05/20 14:01:23:333 NZST [DEBUG] HttpClient - SunSASL 1.5: Sun SASL provider(implements client mechanisms for: DIGEST-MD5, GSSAPI, EXTERNAL, PLAIN, CRAM-MD5; server mechanisms for: DIGEST-MD5, GSSAPI, CRAM-MD5) 2008/05/20 14:01:23:349 NZST [DEBUG] DefaultHttpParams - Set parameter http.useragent = Jakarta Commons-HttpClient/3.1 2008/05/20 14:01:23:364 NZST [DEBUG] DefaultHttpParams - Set parameter http.protocol.version = HTTP/1.1 2008/05/20 14:01:23:380 NZST [DEBUG] DefaultHttpParams - Set parameter http.connection-manager.class = class org.apache.commons.httpclient.SimpleHttpConnectionManager 2008/05/20 14:01:23:380 NZST [DEBUG] DefaultHttpParams - Set parameter http.protocol.cookie-policy = default 2008/05/20 14:01:23:380 NZST [DEBUG] DefaultHttpParams - Set parameter http.protocol.element-charset = US-ASCII 2008/05/20 14:01:23:380 NZST [DEBUG] DefaultHttpParams - Set parameter http.protocol.content-charset = ISO-8859-1 2008/05/20 14:01:23:396 NZST [DEBUG] DefaultHttpParams - Set parameter http.method.retry-handler = [EMAIL PROTECTED] 2008/05/20 14:01:23:411 NZST [DEBUG] DefaultHttpParams - Set parameter http.dateparser.patterns = [EEE, dd MMM HH:mm:ss zzz, , dd-MMM-yy HH:mm:ss zzz, EEE MMM d HH:mm:ss , EEE, dd-MMM- HH:mm:ss z, EEE, dd-MMM- HH-mm-ss z, EEE, dd MMM yy HH:mm:ss z, EEE dd-MMM- HH:mm:ss z, EEE dd MMM HH:mm:ss z, EEE dd-MMM- HH-mm-ss z, EEE dd-MMM-yy HH:mm:ss z, EEE dd MMM yy HH:mm:ss z, EEE,dd-MMM-yy HH:mm:ss z, EEE,dd-MMM- HH:mm:ss z, EEE, dd-MM- HH:mm:ss z] 2008/05/20 14:01:32:114 NZST [DEBUG] HttpConnection - Open connection to iceb_test:80 2008/05/20 14:01:32:130 NZST [DEBUG] header - GET /ICECream/GUI/packageFabricate?customerName=junkpackageName=junktransportType=EXPORTsubmit=Fabricate+Package HTTP/1.1[\r][\n] 2008/05/20 14:01:32:130 NZST [DEBUG] HttpMethodBase - Adding Host request header 2008/05/20 14:01:32:193 NZST [DEBUG] header - User-Agent: Jakarta Commons-HttpClient/3.1[\r][\n] 2008/05/20 14:01:32:193 NZST [DEBUG] header - Host: iceb_test[\r][\n] 2008/05/20 14:01:32:193 NZST [DEBUG] header - [\r][\n] 2008/05/20 14:01:32:193 NZST [DEBUG] header - HTTP/1.1 404 Not Found[\r][\n] On 5/20/08, Oleg Kalnichevski [EMAIL PROTECTED] wrote: richard schmidt wrote: When I try and run this code, I always get a responce from a server at port 80. What makes you think so? Are you sure the request did not get redirected? Please turn on wire / context logging to see exactly what kind of data gets transferred across the wire and on what port http://hc.apache.org/httpclient-3.x/logging.html Hope this helps Oleg Which is rather odd, as the url clearly specifies port 8080! HttpClient client = *new* HttpClient(); HttpMethod get = *new* GetMethod( http://iceb_test:8080/ICECream/GUI/packageFabricate?customerName=junkpackageName=junktransportType=EXPORTsubmit=Fabricate+Package ); I can open the url in a browser as expected The server is a Jboss / tomcat On port 80 is apachie thanks Richard - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: The http core tutorial
On Tue, 2008-05-20 at 20:48 +0800, jeffty wrote: Hi, I'm looking for the tutorial of HttpCore in http://hc.apache.org. And only some examples are available. And is there any detailed articles about it? Thanks in advance. This is all we have at the moment http://wiki.apache.org/HttpComponents/GuidedTourOfHttpCore I am planning to take a long pause right after HttpCore 4.0a2 and HttpClient 4.0a5 releases in order to give people enough time to review them and to decide whether the core can go FINAL and the client can go BETA. This would give me an opportunity to spend some time on improving documentation and, maybe, writing a tutorial on HttpCore programming. Having said that, I am known to be bad at writing documentation. And I have a day-job and a wife to take care of. So, do not take this as a promise. Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Basic Authentication Issue
On Tue, 2008-05-20 at 11:12 -0400, Ashish Kulkarni wrote:
Hi
This is the error i am getting
2008-05-20 11:08:33,437 DEBUG [main] params.DefaultHttpParams
(DefaultHttpParams.java:151) - Set parameter http.auth.scheme-priority =
[Basic]
The client has been configured to accept basic authentication only
2008-05-20 11:08:33,437 DEBUG [main] params.DefaultHttpParams
(DefaultHttpParams.java:151) - Set parameter http.method.retry-handler =
[EMAIL PROTECTED]
2008-05-20 11:08:33,437 DEBUG [main] httpclient.HttpConnection
(HttpConnection.java:692) - Open connection to uskzoas163:80
2008-05-20 11:08:33,487 DEBUG [main] httpclient.HttpMethodBase
(HttpMethodBase.java:1352) - Adding Host request header
2008-05-20 11:08:33,557 DEBUG [main] httpclient.HttpMethodDirector
(HttpMethodDirector.java:843) - Authorization required
2008-05-20 11:08:33,557 DEBUG [main] auth.AuthChallengeProcessor
(AuthChallengeProcessor.java:90) - Supported authentication schemes in the
order of preference: [Basic]
2008-05-20 11:08:33,557 DEBUG [main] auth.AuthChallengeProcessor
(AuthChallengeProcessor.java:111) - Challenge for Basic authentication
scheme not available
2008-05-20 11:08:33,557 WARN [main] httpclient.HttpMethodDirector
(HttpMethodDirector.java:697) - Unable to respond to any of these
challenges: {ntlm=NTLM}
whereas the server supports NTLM authentication only. So, no party.
Besides, this is wrong
new AuthScope(http://uskzoas163/MpssDocDisplay/;, 80),
new UsernamePasswordCredentials(userId, password)
);
AuthScope expects a hostname, not a URI, as the first parameter.
Oleg
GetMethod method = new GetMethod(
http://uskzoas163/MpssDocDisplay/ALTERNATE/PMS-PAD/V_1/00-625080-000_PMS.pdf
);
method.setDoAuthentication( true );
method.getParams().setParameter(HttpMethodParams.RETRY_HANDLER,
new DefaultHttpMethodRetryHandler(3, false));
int statusCode = client.executeMethod(method);
status i get is 401, not authorised,
Ashish
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: Newbie question port 8080
On Wed, 2008-05-21 at 09:29 +1200, richard schmidt wrote: Thanks I will try 4.0beta1 The latest official release is 4.0-alpha4. HttpClient 4.0 is not yet in the BETA phase. BTW do you have a maven POM for it? Yes. Oleg On 5/21/08, Oleg Kalnichevski [EMAIL PROTECTED] wrote: On Tue, 2008-05-20 at 14:04 +1200, richard schmidt wrote: Here is the log. message number 114 says it all! 2008/05/20 14:01:32:114 NZST [DEBUG] HttpConnection - Open connection to iceb_test:80 Richard, The problem turned out to be the underscore in the host name, which is basically illegal for host names. This caused the URI class to misinterpret the request URI. The URI class in HttpClient 3.1 is known to be quite buggy. It has been removed in HttpClient 4.0 This is all I can do for you at the moment. Oleg http://iceb_test:8080/ICECream/GUI/packageFabricate?customerName=junkpackageName=junktransportType=EXPORTsubmit=Fabricate+Package 2008/05/20 14:01:22:880 NZST [DEBUG] HttpClient - Java version: 1.5.0_06 2008/05/20 14:01:22:880 NZST [DEBUG] HttpClient - Java vendor: Sun Microsystems Inc. 2008/05/20 14:01:22:880 NZST [DEBUG] HttpClient - Java class path: C:\Workspace\Grizzly\src\test\resources;C:\Workspace\Grizzly\target\classes;C:\Workspace\Grizzly\target\test-classes;C:\Workspace\Grizzly\src\main\resources;C:\Documents and Settings\schmidt\.m2\repository\com\metservice\ice\icecore\6.5.7\icecore-6.5.7.jar;C:\Documents and Settings\schmidt\.m2\repository\commons-httpclient\commons-httpclient\3.1\commons-httpclient-3.1.jar;C:\Documents and Settings\schmidt\.m2\repository\commons-logging\commons-logging\1.0.4\commons-logging-1.0.4.jar;C:\Documents and Settings\schmidt\.m2\repository\commons-codec\commons-codec\1.2\commons-codec-1.2.jar;C:\Documents and Settings\schmidt\.m2\repository\dom4j\dom4j\1.6.1\dom4j-1.6.1.jar;C:\Documents and Settings\schmidt\.m2\repository\xml-apis\xml-apis\1.0.b2\xml-apis-1.0.b2.jar;C:\Documents and Settings\schmidt\.m2\repository\javax\servlet\servlet-api\2.5\servlet-api-2.5.jar;C:\Documents and Settings\schmidt\.m2\repository\jaxen\jaxen\1.1-beta-6\jaxen-1.1-beta-6.jar;C:\Documents and Settings\schmidt\.m2\repository\jdom\jdom\1.0\jdom-1.0.jar;C:\Documents and Settings\schmidt\.m2\repository\xerces\xmlParserAPIs\2.6.2\xmlParserAPIs-2.6.2.jar;C:\Documents and Settings\schmidt\.m2\repository\xerces\xercesImpl\2.6.2\xercesImpl-2.6.2.jar;C:\Documents and Settings\schmidt\.m2\repository\xom\xom\1.0b3\xom-1.0b3.jar;C:\Documents and Settings\schmidt\.m2\repository\com\ibm\icu\icu4j\2.6.1\icu4j-2.6.1.jar;C:\Documents and Settings\schmidt\.m2\repository\xalan\xalan\2.6.0\xalan-2.6.0.jar;C:\Documents and Settings\schmidt\.m2\repository\org\ccil\cowan\tagsoup\tagsoup\0.9.7\tagsoup-0.9.7.jar;C:\Documents and Settings\schmidt\.m2\repository\junit\junit\3.8.2\junit-3.8.2.jar;C:\Documents and Settings\schmidt\.m2\repository\org\apache\xmlrpc\xmlrpc-client\3.1\xmlrpc-client-3.1.jar;C:\Documents and Settings\schmidt\.m2\repository\org\apache\xmlrpc\xmlrpc-common\3.1\xmlrpc-common-3.1.jar;C:\Documents and Settings\schmidt\.m2\repository\org\apache\ws\commons\util\ws-commons-util\1.0.2\ws-commons-util-1.0.2.jar;C:\Documents and Settings\schmidt\.m2\repository\org\springframework\spring-webmvc\2.5.4\spring-webmvc-2.5.4.jar;C:\Documents and Settings\schmidt\.m2\repository\org\springframework\spring-beans\2.5.4\spring-beans-2.5.4.jar;C:\Documents and Settings\schmidt\.m2\repository\org\springframework\spring-core\2.5.4\spring-core-2.5.4.jar;C:\Documents and Settings\schmidt\.m2\repository\org\springframework\spring-context\2.5.4\spring-context-2.5.4.jar;C:\Documents and Settings\schmidt\.m2\repository\aopalliance\aopalliance\1.0\aopalliance-1.0.jar;C:\Documents and Settings\schmidt\.m2\repository\org\springframework\spring-context-support\2.5.4\spring-context-support-2.5.4.jar;C:\Documents and Settings\schmidt\.m2\repository\org\springframework\spring-web\2.5.4\spring-web-2.5.4.jar;C:\Documents and Settings\schmidt\.m2\repository\commons-logging\commons-logging\1.1.1\commons-logging-1.1.1.jar;/C:/eclipse 3.3 wtp/eclipse/configuration/org.eclipse.osgi/bundles/192/1/.cp/;/C:/eclipse 3.3 wtp/eclipse/configuration/org.eclipse.osgi/bundles/190/1/.cp/ 2008/05/20 14:01:22:911 NZST [DEBUG] HttpClient - Operating system name: Windows XP 2008/05/20 14:01:22:911 NZST [DEBUG] HttpClient - Operating system architecture: x86 2008/05/20 14:01:22:911 NZST [DEBUG] HttpClient - Operating system version: 5.1 2008/05/20 14:01:23:317 NZST [DEBUG] HttpClient - SUN 1.5: SUN (DSA key/parameter generation; DSA signing; SHA-1, MD5 digests; SecureRandom; X.509 certificates; JKS keystore
Re: HttpClient / Core 4.0 question
Vibhav Sharma wrote: Hi, I have used HttpClient 3.x previously. It did not have the ability to do the following. * Client makes a chunked multipart request to server. * Server responds with a chunked response. * Client sends more data as part of the same request. * Server responds with more data in the same response. * till Server or Client closes connection. Basically, treat it like a connection rather than just a request/response pair. This is typically not how http works, but as far as I understand, the reference allows it. Also, in my particular case, there is no need for any cookie management etc. So I am supposing I'll be using HttpCore. I have as yet not taken a look at it and will be doing as soon as possible. But may I ask whether HttpCore, now, covers such a scenario. (Had to implement/hack this completely last time :) ) Hi Vibhav (I hope I got your first name right) HTTP is a request / response oriented protocol, so what you are trying to do can certainly be considered as an abuse of the protocol. Having said that, HTTP gets abused in so many different ways these days, so I guess it is just another way. HttpCore NIO is capable of writing and reading data from the same connection in a full duplex mode. Reading and writing streams are completely independent. One can easily pipeline several requests before getting the first response. HttpCore NIO can also be used for the scenario you described, but you will have to implement your own protocol handler, which is not an easy thing to do. Hope this helps Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Redirected - but where? HttpClient 4
DrewC wrote:
Ok, I guess I just needed to keep banging on this for another hour. I am now
using a non-default context with the execute and then reconstructing the
final require URL with:
HttpUriRequest finalRequest =
(HttpUriRequest)
context.getAttribute(ExecutionContext.HTTP_REQUEST);
HttpHost host = (HttpHost)
context.getAttribute(ExecutionContext.HTTP_TARGET_HOST);
StringBuilder uriString = new StringBuilder();
uriString.append(host.getSchemeName() + ://);
uriString.append(host.getHostName());
if (host.getPort() != -1) {
uriString.append(: + host.getPort());
}
uriString.append(finalRequest.getURI().normalize().toString());
If anyone could confirm this is the right method, that would be appreciated.
Yes, it is. HttpClient does not mutate the content of the original
request. Rather it creates a wrapper around it which does get changed in
the course of the request execution. The only way to get hold of that
object is by retrieving it from the execution context.
Another way of keeping track of all redirects is by providing a custom
redirect handler.
Hope this helps
Oleg
Regards
Drew
DrewC wrote:
Hi, I am trying to do basically exactly what Magnus was attempting, to
access the final full URL to the resource after following
redirects. But I have had no success attempting the techniques seen here,
though I admit to being a bit confused by them. I have not had to deal
with the HttpContext previously and I am not exactly sure which
HttpGetMethod Olek suggests to cast. But with my simplistic approach,
after the request is executed (with httpClient.execute(httpGet) and I have
confirmed several redirects already occured and were followed, the
HttpGet.getURI
(HttpUriRequest)httpGet).getURI()
(HttpUriRequest)httpGet).getRequestLine().getUri()
all still retains the intial URL, not the final one. I am using a shared
HttpClient (created via ThreadSafeClientConnManager), not sure if that
makes any difference.
Any help appreciated.
Drew
olegk wrote:
On Sun, 2008-01-06 at 04:50 -0800, MaGGE wrote:
Hello,
I've just started using HttpClient for a little util I'm making. I'm
letting
it handle_redirects=true, but I need to process the URL it's been
redirected
to. How can I find the current URL after I've executed a request that
was
redirected?
I've tried a couple of things - which I feel is wrong. First thing was
to
disable handle_redirects, and look for the Location header in a loop
of my
own. Challenge with this was that I could not see any way to replace the
URI
for the request object, so I'd have to make a new one - which is a
problem
because my method should be able to process any HttpUriRequest.
Second I tried to add a HttpRequestInterceptor to the client. But all I
can
get from the HttpRequest object given to the process method is the
RequestLine. That one only has the info found behind GET/POST etc, that
is
only the path and method. I'm still lacking the host, protocol and so
on.
I'm sure there's an easy way to do this that I've just missed ... Please
advice. :)
Thanks for your time,
Magnus
Hi Magnus
In addition to getting HttpHost out of the execution context, you may
also
(1) cast HttpRequest to HttpUriRequest in order to get the full request
URI
(2) extend or replace the default RedirectHandler with a custom one and
implement whatever redirect handling logic you deem appropriate
Hope this helps
Oleg
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: How to download .jar file? (I think that I read most of threads and googled alot..)
On Thu, 2008-05-22 at 21:46 -0700, JBRyu wrote:
Hi,
I have a problem with downloading a .jar file from a HTTP server. I'm now
using httpclient-4.0-alpha4 and httpcore-4.0-beta1 libraries. I can download
any .jar file using a http input stream, but the download .jar file is
corrupted even if the size of the downloaded file is the same as that of the
original file on the HTTP server so I can't decompress it because it is
broken. Please see Case #1 for details. So, I use ZipInputStream class for
reading the compressed file from the HTTP server. However, when I read bytes
from the ZipInputStream, it always returns -1 at the first time and is
hanging there. It seems like that the stream is not broken but dosen't do
anything. Please seee Case #2 for details.
I spent lots of time but couldn't figure it out. Is there any one who faced
this problem and resolved it?
Thank you for all your helps.
JB
I do not see any obvious problems with your code aside from that fact
that you should use HttpEntity#getContentLength() method to find out the
length of the incoming content instead of parsing 'content-length'
header manually.
I wrote this small app to test similar scenario and it worked for me. I
got perfectly valid jar.
==
DefaultHttpClient httpclient = new DefaultHttpClient();
HttpGet httpget = new HttpGet(
http://repo1.maven.org/maven2/org/apache/httpcomponents/httpcore/4.0-beta1/httpcore-4.0-beta1.jar;);
System.out.println(executing request: + httpget.getRequestLine());
HttpResponse response = httpclient.execute(httpget);
HttpEntity entity = response.getEntity();
System.out.println();
System.out.println(response.getStatusLine());
if (response.getStatusLine().getStatusCode() == HttpStatus.SC_OK) {
if (entity != null) {
System.out.println(File size: + entity.getContentLength());
File file = new File(local.jar);
InputStream instream = null;
OutputStream outstream = null;
try {
instream = entity.getContent();
outstream = new FileOutputStream(file);
byte[] tmp = new byte[4096];
int l;
while ((l = instream.read(tmp)) != -1) {
outstream.write(tmp, 0, l);
}
outstream.flush();
} finally {
if (instream != null) {
instream.close();
}
if (outstream != null) {
outstream.close();
}
}
}
} else {
if (entity != null) {
entity.consumeContent();
}
}
==
Oleg
Case #1
HttpGet httpGet = new HttpGet(httpFilePath);
HttpResponse response = httpClient.execute(httpGet);
int responseCode = response.getStatusLine().getStatusCode();
// Checks the HTTP server response code.
if ( responseCode ==
Client_HTTPDownloadConstants.HTTP_RESPONSE_CODE_SUCCESSFUL ) {
// Retrieves the target file size from the response header.
Header[] headers = response.getHeaders(Content-Length);
// Retrieves the target file size.
int fileSize = Integer.valueOf(headers[0].getValue());
// Prepares an input stream to read the source file bytes from the HTTP
server.
InputStream fis = response.getEntity().getContent();
// Prepares a file output stream to write the downloaded bytes to the
local file.
FileOutputStream fos = new FileOutputStream(tmpFilePath)
// Initializes the written bytes of the current file.
int bytesWritten = 0;
// Reads the bytes from the HTTP server.
int bytesRead = fis.read(buffer);
while (bytesRead != -1) {
// Writes bytes to the local file.
fos.write(buffer, 0, bytesRead);
// Reads bytes from the local file.
bytesRead = fis.read(buffer);
}
// Closes the input and output streams.
if (fos != null) {
fos.flush();
fos.close();
}
if (fis != null) {
fis.close();
}
}
Case #2
HttpGet httpGet = new HttpGet(httpFilePath);
HttpResponse response = httpClient.execute(httpGet);
int responseCode = response.getStatusLine().getStatusCode();
// Checks the HTTP server response code.
if ( responseCode ==
Client_HTTPDownloadConstants.HTTP_RESPONSE_CODE_SUCCESSFUL ) {
// Retrieves the target file size from the response header.
Header[] headers = response.getHeaders(Content-Length);
// Retrieves the target file size.
int fileSize = Integer.valueOf(headers[0].getValue());
// Prepares an input stream to read the source file bytes from the HTTP
server.
InputStream fis = response.getEntity().getContent();
Re: About releaseconnection method of version 3.1
On Fri, 2008-05-23 at 14:30 +0530, Laxmilal Menaria wrote: Hello, We are using httpclient 3.1 library in our application and sending multiple request using Multiple threads. Releasing connection when the thread completes using httpmethod.releaseconnection() method. But we encountered that the releaseconnection() method doesn't release the connection. Laxmilal, #releaseConnection() method is intended to return the connection back to the connection manager. The connection manager per default will keep persistent connections alive, which implies leaving the socket in the established state. All the connection are in Established state. So now we are tried httpmethod.abort() in place of httpmethod.releaseconnection(), We need your comments on it. Rather call #closeIdleConnections(0) method of the connection manager to close persistent if you do not want them. #abort method is intended to force-terminate connections. You should use it sparingly. Oleg Thanks, Laxmilal - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: difference between request headers and request params
On Fri, 2008-05-23 at 16:03 -0500, Dimitrios Christodoulakis wrote:
Hello, I am a new user of HTTPClient and trying to learn how to use
it. I have read the relevant primer and user guide but I am not sure
what is the difference between headers and parameters.
HTTP headers are elements of the HTTP request sent to the target server.
Parameters are merely attributes that affect the way HttpClient
functions. Parameters can apply to different aspects of HttpClient, not
just request messages.
Do they have
the same functionality for GET and POST requests?
No, they do not
I mean does the
server processes and responds differently to headers or parameters?
No, it does not. The server is completely unaware of the parameters.
They are not a part of the HTTP protocol.
Hope this helps
Oleg
I am trying to set request headers with
addRequestHeader(MyHeaderName, MyHeaderValue); and then after
executing the method I print the headers like:
Header[] reqheaders = authget.getRequestHeaders();
for (int i=1; ireqheaders.length; i++) {
System.out.println(reqheaders[i].getName().toString() + = +
reqheaders[i].getValue().toString());
}
But no headers are displayed.
Thanks for helping
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
[ANNOUNCEMENT] HttpClient 4.0 is nearing API freeze / API review request
Folks The redesign of HttpClient is now complete. All known problems with the old (3.x) API and architecture limitations have been resolved. All features planned for the 4.0 release are now in place. Due to the release schedule of one of our upstream projects we will have to freeze the 4.0 API rather sooner than later. The next release is likely to be BETA1 not ALPHA5, as initially planned. If your project depends on HttpClient, this is the time to check the new API out and let us know what you think. If we do not get more feedback / change requests, the current SVN snapshot is very likely to be the API HttpClient 4.0 will ship with. If you ever wanted to have something fixed or improved in HttpClient API, this is pretty much the last chance to ask for it. The current SVN snapshot can be found here: http://people.apache.org/~olegk/4.0-API-review/ Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Changing order of headers in a multipart post
On Mon, 2008-06-02 at 16:21 -0700, Darrell Esau wrote: Hi all, I'm posting multipart form data to a web service which expects the headers in a certain order. Particularly, each part contains a Content-Type and Content-Disposition header. If I send the data with the Content-Disposition header before the Content-Type header, it fails. Swapping the two makes the request complete successfully. so -- my question -- Is it possible in HttpClient (3.1) to specify order of the part headers? Thanks, -darrell Darrell, The only option you have is to extend the Part class and override its #send(OutputStream) method. Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Out of memory using ThrottlingHttpClientHandler
On Tue, 2008-06-03 at 20:48 +0800, maomaode wrote: Hi, I'm hitting a Out Of Memory error when using ThrottlingHttpClientHandler http://hc.apache.org/httpcomponents-core/httpcore-nio/apidocs/org/apache/http/nio/protocol/ThrottlingHttpClientHandler.html I gave a Executors.|*newCachedThreadPool cid:part1.01010308.04040401@gmail.com*(), should I give a fixed thread pool? James, What is it exactly you are trying to do? How many simultaneous connections are you trying to open? What is the size of the content buffer? it is 20480 (default)? And BTW, what's the workerCount means in the |DefaultConnectingIOReactor? The number of threads I/O reactor uses to dispatch I/O events to the protocol handlers. This number should be equal to that of CPU cores of the system. Hope this helps Oleg Thanks in advance, James - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Log4j with Httpclient 4.x
On Thu, 2008-06-05 at 08:51 +1000, Steven Adams wrote:
But org.apache.http.wire is right?
Yes, it is.
Oleg
-Original Message-
From: Sam Berlin [mailto:[EMAIL PROTECTED]
Sent: Thursday, 5 June 2008 12:03 AM
To: HttpClient User Discussion
Subject: Re: Log4j with Httpclient 4.x
s/patch/path
On Wed, Jun 4, 2008 at 10:02 AM, Sam Berlin [EMAIL PROTECTED] wrote:
Not sure if anyone has pointed out that:
logj4.logger.org.apache.commons.httpclient=DEBUG, httc
...is not the correct patch for httpclient 4.0.
Sam
On Wed, Jun 4, 2008 at 8:42 AM, Steven Adams [EMAIL PROTECTED] wrote:
It's picking up the log4j properties files. As I make changes to the
properties file (change log files etc) the log names do actually change,
I
can also turn the logging down which is also reflected.
I know its straying away.. I guess I just wanted to make sure that my
properties file looks right for using httpclient with log4j..
Thanks
Steve
-Original Message-
From: sebb [mailto:[EMAIL PROTECTED]
Sent: Wednesday, 4 June 2008 9:37 PM
To: HttpClient User Discussion
Subject: Re: Log4j with Httpclient 4.x
On 04/06/2008, Steven Adams [EMAIL PROTECTED] wrote:
Hi,
I know it's using log4j as its outputting everything via the root
category.
But when I try put to it in its own category (log file) it's just not
working.
Any ideas?
Is the log4j properties file having any effect?
If not, then presumably log4j is not finding it.
This is straying off-topic for this mailing list.
-Original Message-
From: sebb [mailto:[EMAIL PROTECTED]
Sent: Monday, 2 June 2008 9:24 PM
To: HttpClient User Discussion
Subject: Re: Log4j with Httpclient 4.x
On 02/06/2008, Steven Adams [EMAIL PROTECTED] wrote:
Hi,
I've been trying to get log4j working with httpclient 4.x.
My properties file looks like the following..
SNIP
logj4.logger.org.apache.commons.httpclient=DEBUG, httc
logj4.logger.org.apache.http.wire=DEBUG, httc
logj4.logger.org.apache.http.headers=DEBUG, httc
log4j.appender.httc =org.apache.log4j.FileAppender
log4j.appender.httc.File=logs/httc.log
log4j.appender.httc.layout=org.apache.log4j.PatternLayout
log4j.appender.httc.layout.ConversionPattern=%p %d{dd MMM
HH:mm:ss}
[%t] %c %x - %m%n
SNIP
Any ideas?
Is it using log4j logging? Or has it picked some other logger?
If so, the following advice just posted to Commons User list might
help:
You could try putting a file named commons-logging.properties in your
app's classpath with the following contents:
org.apache.commons.logging.Log=org.apache.commons.logging.impl.Log4JLogger
This will force commons logging to use log4j.
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: Out of memory using ThrottlingHttpClientHandler
On Fri, 2008-06-06 at 13:35 +0800, maomaode wrote: Hi, I've filed the issue https://issues.apache.org/jira/browse/HTTPCORE-162 Testcase patch attached, also paste the stacktrace Some other questions, In the NIO client sample, ioReactor.connect( new InetSocketAddress(url.getHost(), 80), null, new HttpHost(url.getHost()), null); What's the HttpHost used for? we have InetSocketAddress, isn't it enough? It represents a virtual host. Consider the following: you want to send a request to www.google.cn or www.google.ru, but you need to connect to the physical host www.google.com The HttpHost actually can not pass in the path/port/query information into the handler, So I think we can just pass in URL instead of HttpHost, right? You can pass any object as an attachment to the connection. The same object will passed to the HttpRequestExecutionHandler#initalizeContext upon after the connection has been established. http://hc.apache.org/httpcomponents-core/httpcore-nio/xref/org/apache/http/nio/protocol/HttpRequestExecutionHandler.html#66 And another question is, is it possible to use the IP addresss in the InetSocketAddress？ Yes, it is. The reason is that I want to cache the IP-Host mapping, so there's no DNS resolve time in the crawler. Thanks, James Hope this helps Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Add headers to Redirect request
On Thu, 2008-06-05 at 23:14 -0700, Thuannd wrote: Hi! When after a redirect response a retry is done by the DefaultClientRequestDirector, i think, redirect request cann't add all headers from original request. Some headers, example: Host header , there will be incorrect value if request redirect to another host. Some server will check value of that header then return invalid request. Bug of alpha 4 of HttpClient 4.0. It has already been fixed in SVN trunk. Please upgrade to the latest snapshot of HttpClient Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: HttpClient not sending cookie?
On Mon, 2008-06-09 at 11:25 -0500, Eduardo J. Ortega U. wrote: Hi, all: I am trying to use HttpClient to authenticate to a website, and do a couple of GET and POST requests. I can submit the authentication form, get the cookie, and after getting the cookie, I can make a GET request and I see that it sends the approrpiate Cookie header. Then I attempt to do a POST request, but this one does not send the Cookie, thus the server think I haven't authenticated and nothing works after that :-( 2008/06/09 11:04:19:182 GMT-05:00 [DEBUG] header - POST http://www.fosyga.gov.co/CGI/buda_logon.asp HTTP/1.1[\r][\n] ... 2008/06/09 11:04:19:245 GMT-05:00 [DEBUG] header - Set-Cookie: ASPSESSIONIDSSSDBQST=HPNJBJKCJOJJDBLKGMOLFCNC; path=/[\r][\n] ... 2008/06/09 11:04:19:260 GMT-05:00 [DEBUG] HttpMethodBase - Cookie accepted: $Version=0; ASPSESSIONIDSSSDBQST=HPNJBJKCJOJJDBLKGMOLFCNC; $Path=/ ... 2008/06/09 11:04:19:260 GMT-05:00 [DEBUG] DefaultHttpParams - Set parameter http.protocol.cookie-policy = rfc2109 ... 2008/06/09 11:04:19:260 GMT-05:00 [DEBUG] header - GET http://www.fosyga.gov.co/CGI/buda_consulta_afil.asp?sstat=0 HTTP/1.1[\r][\n] ... 2008/06/09 11:04:19:260 GMT-05:00 [DEBUG] header - Cookie: $Version=0; ASPSESSIONIDSSSDBQST=HPNJBJKCJOJJDBLKGMOLFCNC; $Path=/[\r][\n] ... 2008/06/09 11:04:19:417 GMT-05:00 [DEBUG] header - HTTP/1.1 200 OK[\r][\n] ... 2008/06/09 11:04:19:511 GMT-05:00 [DEBUG] DefaultHttpParams - Set parameter http.protocol.cookie-policy = rfc2109 ... 2008/06/09 11:04:19:557 GMT-05:00 [DEBUG] header - POST http://fosyga.gov.co/CGI/buda_consulta_afil.asp HTTP/1.1[\r][\n] ... NO Cookie: ... header sent here. server replies assigning new cookie: 2008/06/09 11:04:19:792 GMT-05:00 [DEBUG] header - Set-Cookie: ASPSESSIONIDSSSDBQST=IPNJBJKCOILAONHGOMJLIDCB; path=/[\r][\n] See attached full log file. Please ignore the exceptions at the end of the file, as they are a result of not getting the right web page from the server because of the cookie issue. Any help is greatly appreciated. Eduardo, According to the RFC2109, cookies originated from the host www.fosyga.gov.co with the default domain scope may not be sent to the host fosyga.gov.co. Try using the browser compatibility mode. Oleg Thanks, - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: How to encode GET parameter to iso-8859-1?
On Mon, 2008-06-09 at 18:51 +0200, Bas Schulte wrote:
Hi,
I need to encode my parameters in a GET request to a given host in
iso-8859-1. How would I go about that with httpclient (3.1)?
There's EncodingUtil.getBytes that gives me a byte[]:
String messageData = blah;
byte[] bytes = EncodingUtil.getBytes(messageData, iso-8859-1);
How do I feed that byte[] properly to create a NameValuePair that I
can feed to setQueryString, like so:
get.setQueryString(new NameValuePair[]{new NameValuePair(data, ???),
get.setQueryString(
EncodingUtil.formUrlEncode(
new NameValuePair[] {
new NameValuePair(data, blah)}, iso-8859-1));
Hope this helps
Oleg
Any pointers how to solve this (may not even be specific to
httpclient, just never stumbled upon this in java)?
Thanks,
bas.
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: HTTPClient Logon form
On Mon, 2008-06-09 at 13:47 -0700, Benouille wrote: Hi all, I'm new to HTTPClient. I'm trying to logon to a form developped under WEBSPHERE Studio. This form needs a username/password and need to accept cookies. So i try the sample FormLoginDemo in the HTTPClient samples. Unfortunately, when i run it, it does not work ? Console display : Login form get: HTTP/1.1 200 OK Initial set of cookies:None Login form post: HTTP/1.1 200 OK Logon cookies:None But i do not have the JSessionID use to save session, and the response Body is the original login form ? I do not know why, cause i am sending the good parameters. Can someone help me ? Make sure you have read this http://hc.apache.org/httpcomponents-client/primer.html Oleg Tks for advance. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: http pipelining
On Tue, 2008-06-10 at 14:35 +0200, Martin Zdila wrote: hello is there support for http pipelining in the httpclient/httpcore 4? if not, are thare any plans to support it? Martin, The trouble is that pipelining and 'expect:continue' handshaking are (pretty much) mutually exclusive. HttpClient 4.0 provides support for the latter because it is mandatory for HTTP/1.1 compliant HTTP agents. HttpCore NIO extensions, however, are fully pipelining capable. There are plans to develop an asynchronous version of HttpClient at _some_ point of time with the full support for pipelining. There is nothing that prevents you from hacking up a custom version of HttpClient based on HttpCore NIO. Cheers Oleg thanks in advance - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Using httpclient object: local vs object scope
On Wed, 2008-06-11 at 11:23 +0300, Vasilij Savin wrote: Greetings, I have been playing with HttpClient 4 lately. It is pretty nice piece of software, but recently I run into problem I was not able to resolve. Currently I am using HttpClient to provide communication between sistem I am developing and some external one, which uses webservices with XML responses. After studying examples of HC - I decided to use DefaultHttpClient httpclient = new DefaultHttpClient(); as field and reuse for consequent requests to external system. 1st request works fine, but all the following requests fail with exception: org.apache.http.ProtocolException: Content-Length header already present at org.apache.http.protocol.RequestContent.process( RequestContent.java:70) at org.apache.http.protocol.BasicHttpProcessor.process( BasicHttpProcessor.java:290) at org.apache.http.protocol.HttpRequestExecutor.preProcess( HttpRequestExecutor.java:160) at org.apache.http.impl.client.DefaultClientRequestDirector.execute( DefaultClientRequestDirector.java:356) at org.apache.http.impl.client.AbstractHttpClient.execute( AbstractHttpClient.java:501) at org.apache.http.impl.client.AbstractHttpClient.execute( AbstractHttpClient.java:456) at org.apache.http.impl.client.AbstractHttpClient.execute( AbstractHttpClient.java:422) at lt.sintagma.ss5.core.documentImporter.AvilysAccessor.sendRequest( AvilysAccessor.java:80) at lt.sintagma.ss5.core.documentImporter.AvilysAccessor.getDocumentList( AvilysAccessor.java:151) at lt.sintagma.ss5.core.AvilysIntegrationTest.testGetDocument( AvilysIntegrationTest.java:109) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke( NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke( DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.junit.internal.runners.TestMethodRunner.executeMethodBody( TestMethodRunner.java:99) at org.junit.internal.runners.TestMethodRunner.runUnprotected( TestMethodRunner.java:81) at org.junit.internal.runners.BeforeAndAfterRunner.runProtected( BeforeAndAfterRunner.java:34) at org.junit.internal.runners.TestMethodRunner.runMethod( TestMethodRunner.java:75) at org.junit.internal.runners.TestMethodRunner.run( TestMethodRunner.java:45) at org.junit.internal.runners.TestClassMethodsRunner.invokeTestMethod( TestClassMethodsRunner.java:66) at org.junit.internal.runners.TestClassMethodsRunner.run( TestClassMethodsRunner.java:35) at org.junit.internal.runners.TestClassRunner$1.runUnprotected( TestClassRunner.java:42) at org.junit.internal.runners.BeforeAndAfterRunner.runProtected( BeforeAndAfterRunner.java:34) at org.junit.internal.runners.TestClassRunner.run( TestClassRunner.java:52) at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run( JUnit4TestReference.java:38) at org.eclipse.jdt.internal.junit.runner.TestExecution.run( TestExecution.java:38) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests( RemoteTestRunner.java:460) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests( RemoteTestRunner.java:673) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run( RemoteTestRunner.java:386) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main( RemoteTestRunner.java:196) However, if I make httpclient as local variable - it works totally fine. Can someone explain, what am I doing wrong in the first place. I have a feeling that something is not cleared after the first request. But I can not figure out what specifically. Vasilij, Which version of HttpClient are you using? Please consider upgrading the the latest SVN snapshot as there has been a number of important bug fixes since the latest official release (4.0-alpha4). Oleg Vasilij Savin Programuotojas / Programmer Dokumentų ir procesų valdymo projektų skyrius UAB Sintagma Kalvarijų 125, LT-08221 Vilnius [EMAIL PROTECTED] www.sintagma.lt - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [POLL] Logging toolkit for HttpClient 4.x
[x] migrate to java.util.logging With solutions like JULI [1] and jul-Log4j-bridge [2] I see no reason of what so ever for both commons logging and slf4j to exist Oleg [1] http://tomcat.apache.org/tomcat-6.0-doc/logging.html [2] http://people.apache.org/~psmith/logging.apache.org/sandbox/jul-log4j-bridge/ PS: I'll tally the votes on Friday On Mon, 2008-06-02 at 22:36 +0200, Oleg Kalnichevski wrote: Folks, We are currently considering whether we should continue using commons-logging or migrate to SLF4J or java.util.logging. Please let us what your preference is. We are going to make the final decision based on the results of the poll. A simple majority will _not_ automatically impose a particular choice but will strongly influence the decision making process. All users of HttpClient are encouraged to participate. Oleg --- [ ] keep Commons logging [ ] migrate to SLF4J [ ] migrate to java.util.logging --- - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Higher Throughputs
Quintin Beukes wrote: Hey, I made some basic servlets to test throughputs for HttpClient, HttpCore and sun's sun.net.www.protocol.http.HttpURLConnection (or something along those lines, I don't have an API at hand now). I basically just put up an apache session listening on port 8580, and then Tomcat on port 80. In tomcat I dumped 3 servlets, each using one of the 3 methods to retrieve a 51kb page from the apache session. The benchmarks was done with ab with 10 second timeouts. The results are as follows (in byte readings, not bit readings). Doing a direct request to apache I get a throughput of about 33mb/s. Using HttpURLConnection, 4.7MB/s. Using HttpCore 3.6MB/s Using HttpClient 6.03mb/s The request times were in general best with apache/HttpClient, maxing at 38ms for the longest request. This was done at a concurrency of 1 connection. Pushing this up to 5, the HttpCore's connection times stayed the same, but throughput dropped to 980kb/s. HttpClient's connection times now came to the same as HttpCore (230ms for longest request), but it's throughput increased to 7.1mb/s. This is all good and well, but my question is this. Are there any ways I can increase these throughputs? Before just jumping into optimizations suggestions and referals to documentation, could someone perhaps just enlighten me into why these speeds are so different. And especially so low? I do not want to jump into conclusions here, but I seriously suspect something is wrong with your test cases. In particular I do not understand why you are using Tomcat to host the code your are testing. What is the point of that? I made an extensive comparison of HttpCore vs HttpClient (both versions 3.1 and 4.0). HttpCore can easily achieve a throughput of 10,000 (small) requests per second on a fairly average hardware. http://wiki.apache.org/HttpComponents/HttpClient3vsHttpClient4vsHttpCore I am obviously using HttpCore wrong for it to be so slow (I mean, HttpClient is built on core, so it's impossible to be faster). It's just that the documentation for these are sparse, and I'm having a very hard time finding my way around. And beyond this, Tomcat itself gives me fantastic speeds reading static data. Where are the bottlenecks? The HttpCore setup I used was the Basic GET Requests example available on the site. I modified it a bit to allow URL input and target server specified through parameters passed to the servlet. Beyond that I didn't change anything. My assumption is that connections aren't being reused. In fact, I know this is the case, so it logically and obviously removes the validity of the above benchmarks for HttpCore. I'm not going to remove them though. I would really like to get throughputs similar to direct Apache connections (at least 33+ mbyte/s, which is round about the general limit of hard drive transfer rates). Full ethernet throughput is preferable and I'm sure possible. Downloading large files I get speeds of up to 54mbyte/s with Apache. I'm sure this is limited only to the hard drive I tested with, as it's a gigabit connection, and this is just less than half it's full potential. I haven't done tests using memory transfer. Will try it when I can diagnose the bottlenecks I currently have, and then use them as measurements to gain insights into the speed of the program itself by taking as much of the other bottlenecks in the system away as possible (in this case the limitation of data transfer rates from the hard drives). The three benchmark servlets I used are here: HttpClient: http://quintin.dev.junkmail.co.za/httpclient/Bench.java HttpCore: http://quintin.dev.junkmail.co.za/httpclient/BenchCore.java HttpURLConn: http://quintin.dev.junkmail.co.za/httpclient/BenchURL.java And a utility class I use in the above: http://quintin.dev.junkmail.co.za/httpclient/StreamBinder.java Any insights would be greatly appreciated. I'll try to look at the benchmark code over the weekend. But I would _strongly_ suggest measuring the performance of each respective framework by using direct connections to the target server. Once you have established the baseline you can work on optimizing the code when running inside a servlet container. Oleg regards, - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Higher Throughputs
On Fri, 2008-06-13 at 08:35 +0200, Quintin Beukes wrote: Hey, That's an idea (to first get the code optimized before squeezing it into tomcat). I guess I just continued this way, as our current system is using HttpURLConnection. So I wanted to get an idea of how it will compare. I'll take your advice on running it in separate java apps? I assume that is what you meant direct connections, as in no proxied connections via tomcat. Yep. If someone had to take a wild guess, which of the examples of the site can I use as a base if I'm targeting performance (a good balance between throughput and high connection/second rates would be the best)? Most likely you want to run benchmarks against a site with a predictable routing overhead. If there are too many hops in between you can never be sure exactly what you are measuring: the performance of the HTTP stack or that of your Internet service provider's routing infrastructure. I guess 'localhost' is your best friend. Oleg Q On Thu, Jun 12, 2008 at 11:02 PM, Oleg Kalnichevski [EMAIL PROTECTED] wrote: Quintin Beukes wrote: Hey, I made some basic servlets to test throughputs for HttpClient, HttpCore and sun's sun.net.www.protocol.http.HttpURLConnection (or something along those lines, I don't have an API at hand now). I basically just put up an apache session listening on port 8580, and then Tomcat on port 80. In tomcat I dumped 3 servlets, each using one of the 3 methods to retrieve a 51kb page from the apache session. The benchmarks was done with ab with 10 second timeouts. The results are as follows (in byte readings, not bit readings). Doing a direct request to apache I get a throughput of about 33mb/s. Using HttpURLConnection, 4.7MB/s. Using HttpCore 3.6MB/s Using HttpClient 6.03mb/s The request times were in general best with apache/HttpClient, maxing at 38ms for the longest request. This was done at a concurrency of 1 connection. Pushing this up to 5, the HttpCore's connection times stayed the same, but throughput dropped to 980kb/s. HttpClient's connection times now came to the same as HttpCore (230ms for longest request), but it's throughput increased to 7.1mb/s. This is all good and well, but my question is this. Are there any ways I can increase these throughputs? Before just jumping into optimizations suggestions and referals to documentation, could someone perhaps just enlighten me into why these speeds are so different. And especially so low? I do not want to jump into conclusions here, but I seriously suspect something is wrong with your test cases. In particular I do not understand why you are using Tomcat to host the code your are testing. What is the point of that? I made an extensive comparison of HttpCore vs HttpClient (both versions 3.1 and 4.0). HttpCore can easily achieve a throughput of 10,000 (small) requests per second on a fairly average hardware. http://wiki.apache.org/HttpComponents/HttpClient3vsHttpClient4vsHttpCore I am obviously using HttpCore wrong for it to be so slow (I mean, HttpClient is built on core, so it's impossible to be faster). It's just that the documentation for these are sparse, and I'm having a very hard time finding my way around. And beyond this, Tomcat itself gives me fantastic speeds reading static data. Where are the bottlenecks? The HttpCore setup I used was the Basic GET Requests example available on the site. I modified it a bit to allow URL input and target server specified through parameters passed to the servlet. Beyond that I didn't change anything. My assumption is that connections aren't being reused. In fact, I know this is the case, so it logically and obviously removes the validity of the above benchmarks for HttpCore. I'm not going to remove them though. I would really like to get throughputs similar to direct Apache connections (at least 33+ mbyte/s, which is round about the general limit of hard drive transfer rates). Full ethernet throughput is preferable and I'm sure possible. Downloading large files I get speeds of up to 54mbyte/s with Apache. I'm sure this is limited only to the hard drive I tested with, as it's a gigabit connection, and this is just less than half it's full potential. I haven't done tests using memory transfer. Will try it when I can diagnose the bottlenecks I currently have, and then use them as measurements to gain insights into the speed of the program itself by taking as much of the other bottlenecks in the system away as possible (in this case the limitation of data transfer rates from the hard drives). The three benchmark servlets I used are here: HttpClient: http://quintin.dev.junkmail.co.za/httpclient/Bench.java HttpCore: http://quintin.dev.junkmail.co.za/httpclient/BenchCore.java HttpURLConn: http://quintin.dev.junkmail.co.za/httpclient/BenchURL.java And a utility class I use
[POLL][RESULT] Logging toolkit for HttpClient 4.x
Commons Logging: 8 slf4j: 10 java.util.logging:2 20 votes (out of ~330 subscribers) is hardly representative but there is not much I can do about it It looks like there is enough support to keep Commons Logging. Given the fact one can use slf4j as a drop-in replacement for Commons Logging, I think we should continue compiling HttpClient against Commons Logging in order to satisfy the majority of users. Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Higher Throughputs
On Fri, 2008-06-13 at 17:05 +0200, Quintin Beukes wrote: I made some modifications to running it inside a servlet, and 51mb/s HttpCore, gets 30mb/s inside the servlet (while streaming it out). The servlet is also changed a bit, not wrapping the output stream in a BufferedOutputStream, and others. Either way, when the concurrency is 1 I get this 30mb/s, and the following summary: Requests per second:5.97 [#/sec] (mean) Time per request: 1676.132 [ms] (mean) Time per request: 167.613 [ms] (mean, across all concurrent requests) Transfer rate: 1059.17 [Kbytes/sec] received When the concurrency is 10, I get: Requests per second:29.80 [#/sec] (mean) Time per request: 33.557 [ms] (mean) Time per request: 33.557 [ms] (mean, across all concurrent requests) Transfer rate: 30572.62 [Kbytes/sec] received I assume this is the same problem, of connection pooling. But I'm using HttpCore. I googled as well as grepped and scan through the source code, but I could find connection limit preferences for HttpCore. Is there such a thing? Is the connection limit in the way I am using it? Do I need to manage the connections myself? HttpCore does _not_ provide connection management components. One can still use org.apache.http.conn classes from the HttpClient package to put together a lightweight HTTP client in top of HttpCore. Take a look at the ManagerConnectDirect sample shipped with HttpClient. Cheers Oleg Thanks Q On Fri, Jun 13, 2008 at 2:53 PM, Oleg Kalnichevski [EMAIL PROTECTED] wrote: On Fri, 2008-06-13 at 11:56 +0200, Quintin Beukes wrote: Not getting expected rate when running inside tomcat. Here are my benchmark results for running as an application (direct connections): +++ Benchmark 'Benchmark http clients' results. --- Total time taken in milliseconds: HttpClient 4.x: 234337 HttpCore 4.x: 199655 HttpURLConnection: 209033 Total bytes transfered: 1048859 Benchmark: HttpURLConnection Average transfer rate (bytes/second): 50176720 Benchmark: HttpCore 4.x Average transfer rate (bytes/second): 52533572 Benchmark: HttpClient 4.x Average transfer rate (bytes/second): 44758576 +++ Up to 52.5 MB/s using HttpCore. HttpClient being the slowest in this case. This was doing 1 connections each downloading a 1 MB static file from Apache over a Gbit ethernet connection. Through Tomcat I was topping 8mb/s using the current HttpCore layout. But I'll let you know where the exact problem lies. I'm assuming it has something to do with the way I am writing output to tomcat's output stream. Tomcat can definitely handle super speeds as well (with static files), so I'm sure there is a way to get this all be faster. One important thing to watch out for is the proper sizing of the HttpClient connection pool. Per default HttpClient allows only two concurrent connections to the same host. So if you are hitting Tomcat with 100 concurrent connections, you are very likely to end with 100 Tomcat worker threads contending over 2 HttpClient connections. This setup will not scale well for obvious reasons. Make sure HttpClient allocates enough outbound client side connections for all inbound server side connections. Hope this helps Oleg If you want the above benchmark code, you can get it .class and .java in the following to jars: http://quintin.dev.junkmail.co.za/httpclient/new/benchlib.jar http://quintin.dev.junkmail.co.za/httpclient/new/benchhttp.jar I use the following command to run it (also shows the versions of httpclient/commons I'm using): java -cp benchhttp.jar:benchlib.jar:libs/commons-codec-1.3.jar:libs/commons-logging-1.0.4.jar:libs/httpclient-4.0-alpha4.jar:libs/httpcore-4.0-beta1.jar:libs/httpcore-nio-4.0-beta1.jar:libs/httpmime-4.0-alpha4.jar test.AppBench Q On Fri, Jun 13, 2008 at 11:23 AM, Oleg Kalnichevski [EMAIL PROTECTED] wrote: On Fri, 2008-06-13 at 10:18 +0200, Quintin Beukes wrote: I'm doing ethernet. Localhost is also quite unpredictable, as it can go too fast. Doing ethernet I know what the limits are, ie. 100mbit or 1gbit. Eitherway, I setup some benchmarks and got very high troughputs. I also spent some time figuring out HttpCore, so it's much faster than all the others now. My question is then more of the sort, what could the reason for this be? It's probably not HttpClient's fault though. Any ideas how I can find the bottleneck? I am not sure I understand the problem. Is it that HttpClient tends to be slower than HttpCore or that you are not getting the expected throughput rate when running HttpClient inside Tomcat? Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL
Re: Wiki Addition
Quintin Beukes wrote: I'm only guessing, but it might help if I supply the URL to the guide: http://wiki.apache.org/HttpComponents/GeneralHttpCoreIntroduction Quintin Great stuff! Many thanks for this contribution. I'll put a link to it on the project's web site after the 4.0-beta2 release. Oleg On 6/15/08, Quintin Beukes [EMAIL PROTECTED] wrote: Hey, Like I promised, I started creating documentation for HttpComponents. I am creating them as I learn it while implementing it in my own software. Since my software is closed source, I have to make parallel classes which simply use HttpCore in a similar way. I also removed certain features, to allow splitting it into different wiki pages, each focussing on it's own specific topic, for example thread safety, or error handling, and so forth. Here it the first page, which discusses making a simple class that accepts a URL, and supplies interface to read the headers and content. If you can proofread it, tell me of any mistakes or errors I made. As I get the time I will continue making more pages. Improving the connection manager and error handling is what I'm currently busy with, so please let me know of obvious bad practise I'm doing in this guide. It's possible that the improvements focus on those, but it's also possible that I missed them. And those I've missed I would prefer to fix in this one already, since the ones I left in there on purpose are there for a reason (they don't really make the class dangerous, but they do open a door to create a page that focusses on why they are bad and how to do it properly, learning by example). Also, the wiki page: http://wiki.apache.org/HttpComponents/HelpOnParsers has an error on it. Where it discusses line numbering for source code parsers, it gives the example: (#FORMAT python start=10 step=10 numbering=on or #!python numbering=off). This should read (the option is numbers, not numbering): (#FORMAT python start=10 step=10 numbers=on or #!python numbers=off). Being an immutable page, I can't correct this myself. One more, I put the source code files, as separate sections on this page. Is there any way I can attach them, or is this not possible? -- Quintin Beukes - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Wiki Addition
On Sun, 2008-06-15 at 22:03 +0200, Quintin Beukes wrote: Hey, Like I promised, I started creating documentation for HttpComponents. I am creating them as I learn it while implementing it in my own software. Since my software is closed source, I have to make parallel classes which simply use HttpCore in a similar way. I also removed certain features, to allow splitting it into different wiki pages, each focussing on it's own specific topic, for example thread safety, or error handling, and so forth. Here it the first page, which discusses making a simple class that accepts a URL, and supplies interface to read the headers and content. If you can proofread it, tell me of any mistakes or errors I made. Quintin, I am in the process of reviewing the content, but it'll take a few days As I get the time I will continue making more pages. Improving the connection manager and error handling is what I'm currently busy with, so please let me know of obvious bad practise I'm doing in this guide. It's possible that the improvements focus on those, but it's also possible that I missed them. And those I've missed I would prefer to fix in this one already, since the ones I left in there on purpose are there for a reason (they don't really make the class dangerous, but they do open a door to create a page that focusses on why they are bad and how to do it properly, learning by example). Also, the wiki page: http://wiki.apache.org/HttpComponents/HelpOnParsers has an error on it. Where it discusses line numbering for source code parsers, it gives the example: (#FORMAT python start=10 step=10 numbering=on or #!python numbering=off). This should read (the option is numbers, not numbering): (#FORMAT python start=10 step=10 numbers=on or #!python numbers=off). Being an immutable page, I can't correct this myself. I also do not have permissions to change the content of that page. It seems to require admin privileges. One more, I put the source code files, as separate sections on this page. Is there any way I can attach them, or is this not possible? Not that I know of. Once again, many thanks for this contribution. Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Message body in GET method?
On Thu, 2008-06-19 at 12:13 -0700, David Gallardo wrote: I've read section 9.1 and don't see how this would be in violation of the spec. I intend to use the body merely to specify the query, the server doesn't preserve it, so it's an entirely safe, idempotent retrieval operation--like any other GET. I don't mean to sound contentious, Oleg, but what part of the spec, specifically, would this be violating? I'm open to suggestion, if this is wrong, but since this is a query, it doesn't seem appropriate to use a POST or UPDATE. 9.1.1 ... In particular, the convention has been established that the GET and HEAD methods SHOULD NOT have the significance of taking an action other than retrieval. ... 9.3 ... The GET method means retrieve whatever information (in the form of an entity) is identified by the Request-URI. ... Hope this helps Oleg @D David, What you are trying to do / propose is a severe violation of the HTTP spec. See section 9.1 Oleg @D - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: DefaultHttpClientConnection.isOpen();
Quintin Beukes wrote: I'm not sure why accessing it from different threads would cause this to happen? Could you explain this please. Quintin I am only guessing here. All I am saying HTTP connection objects are not threading safe. The only method one can safely can from a different thread is #shutdown. I would certainly help if you managed to reproduce the problem with a test case. Cheers Oleg It is definitely a concurrency issue. I am still narrowing them all down. thanks, Q On Thu, Jun 19, 2008 at 12:02 AM, Oleg Kalnichevski [EMAIL PROTECTED] wrote: On Wed, 2008-06-18 at 22:58 +0200, Quintin Beukes wrote: Hey, Is it possible for this method (DefaultHttpClientConnection.isOpen()) to return true when the socket is closed. If you are not accessing the same connection instance from multiple threads, I do not see how this could happen. Please note only HttpConnection#shutdown method is expected to be threading safe. This sounds like a concurrency issue to me Oleg I have a case where netstat doesn't return ANY open connections to the target server, but isOpen returns true. It's not very easy to reproduce my problem, but I've been able to do it whenever needed by using high concurrency. I'm not sure if isOpen() ever returns true when there is no socket when it seems like my problem isn't there. I've only checked when my problem is occurring, and this seems very odd. Thanks - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: isStale()
Quintin Beukes wrote: Will NIO connections work? Yep. NIO is event driven, so the I/O selector will trigger an event if a channel gets closed by the other side Oleg PS: I'm off to bed. Wife is angry. On 6/21/08, Quintin Beukes [EMAIL PROTECTED] wrote: Hey, So.. keepalive is sort of useless then? Because I realised what my problem is. It wasn't synchronization. It was because connections closed by the server, and then it fails when I try to re-use the connection. Basically what I have is this: I try once, if receiving a NoResponse exception, I remove it from the pool and try again If second try fails as well, it gets removed and I try again but with an isStale() check first. My problem is that these keep failing until I filtered through all connections in the pool. In which case I start making new ones. How does HttpClient handle this? Or what can I do to improve this into being more reliable (in the sense of reducing failures to a minimum). Q On 6/21/08, Oleg Kalnichevski [EMAIL PROTECTED] wrote: Quintin Beukes wrote: Hey, Isn't isStale() supposed to be able to tell me whether the server is accepting responses? No, it is intended to test if the connection is still valid on the client side. Blocking I/O provides no good means of telling if the socket has been closed by the peer. #isStale is a work-around for the problem. Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Quintin Beukes - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[ANNOUNCEMENT] HttpComponents HttpCore 4.0-beta2 released
The Apache HttpComponents project is pleased to announce the release of HttpComponents HttpCore 4.0-beta2. This release adds a number of improvements to the NIO components, most notable being improved asynchronous client side and server side protocol handlers. There has been a number of important bug fixes in HttpCore NIO module, whereas HttpCore base module has had very few changes. All upstream projects dependent on HttpCore NIO are strongly advised to upgrade. The focus of the development efforts is now gradually shifting towards providing better test coverage, documentation and performance optimizations. Download - http://hc.apache.org/downloads.cgi Release notes - http://www.apache.org/dist/httpcomponents/httpcore/RELEASE_NOTES.txt HttpComponents site - http://hc.apache.org/ About HttpComponents Core - The HttpCore components implement the most fundamental aspects of the HTTP protocol. They are nonetheless sufficient to develop basic client side and server side HTTP services with a minimal footprint and no external dependencies. HttpCore NIO extensions can be used to build asynchronous HTTP services based on non-blocking I/O model capable of handling a great number of simultaneous connections with just a few I/O threads. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Looking for commons-logging?
On Fri, 2008-06-27 at 00:19 -0400, RM1X wrote: Hi everyone, Got all of the HttpClient+core downloads, linked them up in the Eclipse project properties, and unfortunately when trying to run the HttpClient I get: Exception in thread main java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory I intuitively thereafter connected the commons-logging jar, but it got mad about missing init files; so I suspect that this isn't the fix... Any help you can offer would be most appreciated. Thanks. Dave Dave, What do you mean by got mad about missing init files? Commons-logging supports, but does not require a config file Apparently there is something wrong with the way you set up your Eclipse project. Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Too Many Connections
On Fri, 2008-06-27 at 11:25 -0700, Ken DeLong wrote:
All of our connections are to the exact same host - in fact, the sending and
receiving processes are on the same actual server. The same dns name is used
for each request (we use the dns name of the machine, not the string
localhost).
Ken,
Try disabling the stale connection check. It can happen, especially with
TLS/SSL, that perfectly good connections are reported as stale. You can
get a better insight as to why connections get dropped by turning on the
context logging. For details see
http://hc.apache.org/httpclient-3.x/logging.html
Oleg
We'll try the multithreaded connection manager next. (I was pretty sure the
calling app was single threaded but I'm not 100% sure).
Thanks!
Ken
-Original Message-
From: Oleg Kalnichevski [mailto:[EMAIL PROTECTED]
Sent: Friday, June 27, 2008 11:20 AM
To: HttpClient User Discussion
Subject: RE: Too Many Connections
On Fri, 2008-06-27 at 09:08 -0700, Ken DeLong wrote:
That didn't help. We changed to
private HttpClient client = new HttpClient(new
SimpleHttpConnectionManager());
Ken,
SimpleHttpConnectionManager re-uses the underlying connection object but it
cannot reuse the network socket when connecting to a different host. Ii needs
to close the old socket and open a new one every time it connects to a
different host. If you would like to keep alive connections to multiple
hosts, consider using MultiThreadedHttpConnectionManager
Regarding connections in TIME_WAIT state please see
http://wiki.apache.org/HttpComponents/FrequentlyAskedConnectionManagementQuestions
Oleg
But we still see the same behavior.
Turning on the debug logs shows messages like
$ fgrep 'Connection is locked. Call to releaseConnection() ignored.'
server.log | wc -l
176
$ fgrep 'Releasing connection back to connection manager.' server.log
| wc -l
264
What would cause the connection to be locked when releaseConnection()
is called?
-Original Message-
From: Pete Keyes [mailto:[EMAIL PROTECTED]
Sent: Thursday, June 26, 2008 3:27 PM
To: HttpClient User Discussion
Cc: Stephen Hiley
Subject: RE: Too Many Connections
You need to use either of these:
- org.apache.commons.httpclient.MultiThreadedHttpConnectionManager
- org.apache.commons.httpclient.SimpleHttpConnectionManager
There are examples of both in the HttpClient site.
In addition to that you may need to tune the O/S to formally release
the CLOSE_WAIT files sooner. There is a delay between when an
application closes a socket and when the O/S can formally release that
socket to avoid orphan packets.
See the documentation for your O/S on setting that value.
...Pete
Starbucks Coffee Co.
2601 Utah Ave S.
Seattle, WA. 98134
(w) 206-318-5933
(c) 206-914-4134
-Original Message-
From: Ken DeLong [mailto:[EMAIL PROTECTED]
Sent: Thursday, June 26, 2008 3:17 PM
To: httpclient-users@hc.apache.org
Cc: Stephen Hiley
Subject: Too Many Connections
We have a method in our app that calls httpclient like this:
private HttpClient client = new HttpClient();
public String send(String urlString, String xml)
{
PostMethod postMethod = new PostMethod(urlString);
// Put the XML in the request
try
{
RequestEntity entity = new
StringRequestEntity(xml, text/xml, UTF-8);
postMethod.setRequestEntity(entity);
}
catch(UnsupportedEncodingException e1)
{
logger.warn(Couldn't encode the xml?, e1);
return +
IContentPublishingService.ACTION_ERROR;
}
// Send the data and get the response
String response = null;
try
{
client.executeMethod(postMethod);
byte[] responseBody =
postMethod.getResponseBody();
response = new String(responseBody);
}
catch(HttpException e)
{
logger.warn(Fatal protocol violation: +
e.getMessage());
return +
IContentPublishingService.ACTION_ERROR;
}
catch(IOException e)
{
logger.warn(Fatal transport error: + e.getMessage());
return +
IContentPublishingService.ACTION_ERROR;
}
finally
{
// Release the connection.
postMethod.releaseConnection();
}
return response;
}
This method is called rapidly by a single thread in our application.
After 1500 calls or so, we start to see java.net.SocketException
Re: Is it possible to get a HttpConnection from HttpClient?
On Fri, 2008-06-27 at 10:43 -0700, Yan aka. Yansheng wrote:
Hi There,
This is my first post here so I hope this is the right audience.
My story: I want to replace a chunk of my code for doing I/O streaming to a
webservice. I use java.net.HttpURLConnection to get the InputStream and
OutputStream, it works fine most of the time but it can get a bit flaky and
I had to write my own retry logic. So I decided to switch to
jakartacommons-httpclient to see if things get better. The problem is I
don't seem to be able to come up with a good replacement from the httpclient
package.
Here is the method for obtaining a HttpConnection I wrote, the problem is
the HostConfiguration returned is not populated.
Yansheng
HttpClient 3.x API was never meant to be used this way. You should
either let HttpClient manage connections, or use connection management
components from HttpClient 4.0
Hope this helps
Oleg
private HttpConnection openConnection(URL url) throws IOException {
HttpMethod method = new PostMethod(url.toString());
HttpConnection conn = null;
// Execute the method.
int statusCode = getHttpClient().executeMethod(method);
if (statusCode != HttpStatus.SC_OK) {
method.releaseConnection();
throw new IOException(Unable to connect to Service: + url);
}
else {
HttpConnectionManager connManager =
client.getHttpConnectionManager();
conn = connManager.getConnection(client.getHostConfiguration());
}
return conn;
}
---
The exception I'm getting:
Exception in thread main java.lang.IllegalArgumentException: host
parameter is null
at
org.apache.commons.httpclient.HttpConnection.setHost(HttpConnection.java:248)
at
org.apache.commons.httpclient.SimpleHttpConnectionManager.getConnectionWithTimeout(SimpleHttpConnectionManager.java:163)
at
org.apache.commons.httpclient.SimpleHttpConnectionManager.getConnection(SimpleHttpConnectionManager.java:191)
at
org.apache.commons.httpclient.SimpleHttpConnectionManager.getConnection(SimpleHttpConnectionManager.java:108)
at
amazon.wma.storage.timber.TimberServiceImpl.openConnection(MyServiceImpl.java:180)
...
Thanks for any help!
Yansheng
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: The Cookie jar's lid seems stuck
On Wed, 2008-07-02 at 11:59 +0200, Quintin Beukes wrote:
Hey,
Due to our search engine for one of our sites being hosted on a
separate domain, we can't do AJAX requests to the original domain.
This isn't much of a problem, as I figured I'd just use HttpClient to
make a servlet which is a proxy to the Apache server.
So... to ensure session state is still maintained, which is a
requirement, I will just proxy the cookies the client sends me as
well.
I made the following method (the connection manager and parameters are
final static members of the class):
private HttpEntity proxyRequest(String baseUrl, String
getParameters, Cookie[] cookies) throws Exception
{
DefaultHttpClient httpClient = new DefaultHttpClient(connManager,
httpParams);
HttpContext context = new
BasicHttpContext(httpClient.getDefaultContext());
HttpGet request = new HttpGet(baseUrl + basketUri);
CookieStore cookieStore = httpClient.getCookieStore();
for (Cookie c : cookies)
{
BasicClientCookie newCookie = new BasicClientCookie(c.getName(),
c.getValue());
newCookie.setDomain(c.getDomain());
newCookie.setPath(c.getPath());
newCookie.setComment(c.getComment());
if (c.getMaxAge() 0)
{
long expiryTime = new Date().getTime() + c.getMaxAge();
newCookie.setExpiryDate(new Date(expiryTime));
}
cookieStore.addCookie(newCookie);
}
HttpResponse response = httpClient.execute(request, context);
HttpEntity entity = response.getEntity();
if (entity == null)
{
throw new NoHttpResponseException(Basket request failed to
return a response.);
}
return entity;
}
The Cookie[] I pass in, is an array of cookies are received by
HttpServletRequest.getCookies().
I stepped the code, and the addCookie() method is definitely being
called. When recieving the response on the other side, I make a dump
of all received cookies to the output. This output is received by
HttpClient and returned to the servlet's output stream.
Then what I'm getting is a dump of 0 cookies. So the request is
successful, but the cookies added to the store is not being sent to
the client.
Can anyone see what I'm doing wrong?
Quintin,
Adding cookies to the cookie store is just a part of the story. A cookie
need to match the origin server in order to be included in subsequent
HTTP requests. Apparently there is something wrong with the domain or
the expiry date.
Take a look at the RequestAddCookies protocol interceptor
Hope this helps
Oleg
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: ConnectMethod usage
On Thu, 2008-07-03 at 15:12 -0700, Mike Cumings wrote:
Hello all,
I've got a question as to the usage of the ConnectMethod. From the API,
it appears that the only non-deprecated way to create a HTTP CONNECT
connection is to construct a ConnectMethod instance, passing in a
HostConfiguration object configured for the target/destination host and port.
I've modelled this in the test case below:
public void testConnect() throws IOException {
this.server.setHttpService(new EchoService());
// Set default host
this.client.getHostConfiguration().setHost(
this.server.getLocalAddress(),
this.server.getLocalPort(),
Protocol.getProtocol(http));
HostConfiguration hConf = new HostConfiguration();
hConf.setHost(
this.server.getLocalAddress(),
this.server.getLocalPort());
ConnectMethod conn = new ConnectMethod(hConf);
try {
this.client.executeMethod(conn);
assertEquals(HttpStatus.SC_OK, conn.getStatusCode());
} finally {
conn.releaseConnection();
}
}
...
So my question is whether this is a bug or operator error? Thanks in advance,
Mike,
What is it exactly you are trying to achieve? CONNECT method is
supposed to be used for one thing and one thing only: to establish a
tunnel through an HTTP proxy server. Why are you trying to execute this
method as an ordinary HTTP request?
Oleg
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: ConnectMethod usage
On Sat, 2008-07-05 at 12:56 -0700, Mike Cumings wrote: Hi Oleg, I'm well ware of what HTTP CONNECT is used for and how it works. When I added the explicit protocol selection in the snippet of code previously sent out, I was just checking to see if the scheme would e inherited from the host configuration. I had meant to cut that code out prior to sending out my email but it looks like I had forgotten do do that.. I'm just trying to get an HTTP CONNECT to work through an upstream proxy to a site passed to my code in the form of a hostname and port authority specification. It seems like it's getting tripped up in httpclient when the authority specification is being parsed/interpreted as a URI. Thanks in advance, Mike Mike, Why don't you just use ProxyClient for that? http://svn.apache.org/repos/asf/httpcomponents/oac.hc3x/trunk/src/examples/ProxyTunnelDemo.java Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: ConnectMethod usage
On Sun, 2008-07-06 at 14:00 -0700, Mike Cumings wrote: Oleg, Perhaps my last response was a bit misleading. I'm not dealing exclusively with HTTP CONNECTs. This is a special purpose proxy which must handle multiple HTTP methods. I could use ProxyClient as a special case within the codebase but would prefer to be able to leverage HttpClient.executeMethod(...) to eliminate special handling in the code. The API suggests that this is possible since executeMethod(...) takes an HttpMethod argument, of which ConnectMethod is an extension. Is this a bad assumption? If not, what in my code needs to change to be able to successfully establish an HTTP CONNECT using ConnectMethod via HttpClient? Thanks again for the assistance, Mike, I do not think this can be done with HttpClient 3.1. HTTP CONNECT method has a different semantic than all other methods. You should consider building a custom HTTP client using components of HttpClient 4.0. The 4.0 API is significantly more modular and flexible than its predecessor. Oleg Mike On Sat, Jul 5, 2008 at 4:14 PM, Oleg Kalnichevski [EMAIL PROTECTED] wrote: On Sat, 2008-07-05 at 12:56 -0700, Mike Cumings wrote: Hi Oleg, I'm well ware of what HTTP CONNECT is used for and how it works. When I added the explicit protocol selection in the snippet of code previously sent out, I was just checking to see if the scheme would e inherited from the host configuration. I had meant to cut that code out prior to sending out my email but it looks like I had forgotten do do that.. I'm just trying to get an HTTP CONNECT to work through an upstream proxy to a site passed to my code in the form of a hostname and port authority specification. It seems like it's getting tripped up in httpclient when the authority specification is being parsed/interpreted as a URI. Thanks in advance, Mike Mike, Why don't you just use ProxyClient for that? http://svn.apache.org/repos/asf/httpcomponents/oac.hc3x/trunk/src/examples/ProxyTunnelDemo.java Oleg - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Content-Encoding management
On Wed, 2008-07-09 at 11:51 -0700, Alexis Midon wrote: Hi all, I'd like to know how Httpclient 3.* deals with compressed content in a way or another. No, it does not for instance may I expect compressed response bodies to be uncompressed transparently and exposed in their Content-Type? No, you can't. Oleg Thanks in adavance, Alexis - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: About retry handler...
On Wed, 2008-07-16 at 10:32 +0530, Jawahar Nayak wrote: Hi, I am using HttpClient3.0. To set retry handler I am using clientParams.setParameter(HttpMethodParams.RETRY_HANDLER, 3); This is wrong. See http://hc.apache.org/httpclient-3.x/exception-handling.html#Custom_exception_handler I want to know, if I got an 404 response then does Httpclient retry automatically for a particular url. This makes no sense of what so ever. What's the point of retrying a request for a non-existing resource? Oleg Thanks - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Ignore response in POST Method
On Wed, 2008-07-16 at 16:24 +0200, Michael Mangeng wrote: Hi Oleg! Thanks for your response. What can i do to force the socket to be closed? I don't want it to remain open for a uncertain time - this could lead to a problem... You can (1) abort the request causing it to force close the underlying connection (2) Extend the default connection manager, override its releaseConnection method and extend it to close all connections returned to the pool. (3) Remove connection from the pool that have been idle for too long. 3 option is recommended Oleg greetings, Michael Oleg Kalnichevski schrieb: On Tue, 2008-07-15 at 11:09 +0200, Michael Mangeng wrote: Hi all! On the page: http://hc.apache.org/httpclient-3.x/methods/post.html ...you write under Common Problems: The most common problem when using the post method is not reading the entire response body and calling releaseConnection regardless of the response received from the server or whether or not the response body is useful to your application. What is _exactly_ the problem if i do this? (don't care about the response and call method.releaseConnection(); right after client.executeMethod(m) returned?) I really don't care about the response data - i only need to do the request. The problem is that lots of people out there do not correctly understand the purpose of the HttpMethod#releaseConnection() method. This method does not close the underlying connection if it can be re-used. The upshot of it is that this method will attempt to read the entire response body before returning the connection to the pool. This, of course, has performance implications. I admin the statement in the Common Problems section is indeed somewhat misleading, though Oleg greetings, Michael - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: HttpClient invalidates session
On Wed, 2008-07-16 at 16:18 -0700, Leena Borle wrote:
Hi,
I have a struts-2 application running on Tomcat-5 with Single Sign on
enabled. What I am trying to do is , use HttpClient(struts2 app) to access
data other WAR(say WAR2) which also has Form Based login system. To avoid
sending user's credentials(Which I don't have in my application), I am
trying to send cookie-JSESSIOIDSSO in the request header.
Everything works fine, I get the data back. But after that, my application
throws error that, session is invalidated.
How do I stop HttpClient from invalidating the session ?
Thanks
Leena.
Leena,
The HTTP session can only be invalidated by the server. HttpClient
merely uses the session cookie to inform the server that requests are
logically related.
Oleg
Here is the code in my application ,
(Struts-2) HttpServletRequest request = ServletActionContext.getRequest();
//try out httpclient
javax.servlet.http.Cookie[] cookies = request.getCookies();
HttpState newState = new HttpState();
javax.servlet.http.Cookie ssoID = null;
for(int c = 0; c cookies.length; c ++) {
javax.servlet.http.Cookie k = cookies[c];
if(k.getName().equalsIgnoreCase(JSESSIONIDSSO))
ssoID=k;
}
HttpClient client = new HttpClient();
client.setState(newState);
GetMethod getMethod = new GetMethod(
http://localhost:8080/war2/exceuteServlet;);
getMethod.getParams().setCookiePolicy(org.apache.commons.httpclient.cookie.CookiePolicy.IGNORE_COOKIES);
getMethod.setRequestHeader(Cookie, ssoID.getName() + = +
ssoID.getValue());
try {
int responseCode =
client.executeMethod(null,getMethod,newState);
String body = getMethod.getResponseBodyAsString();
AppLogger.getLogger().debug(Response code = + responseCode+
\nResponse is + body);
} catch (HttpException e) {
// TODO Auto-generated catch block
} catch (IOException e) {
// TODO Auto-generated catch block
} finally {
// release any connection resources used by the method
getMethod.releaseConnection();
}
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: Long wait times in MultiThreadedHttpConnectionManager.doGetConnection(...)
On Wed, 2008-07-16 at 16:19 -0700, Algirdas Veitas wrote:
Hi,
Our application is using HttpClient 3.0.1 and is experiencing long wait
times (started happening recently it seems) in the method
MultiThreadedHttpConnectionManager.doGetConnection(...). To give a summary
of the application, we have a bunch of threads (message driven beans, JBoss
4.0.2, JDK 1.4) that are processing a message, creating xml out of it and
then sending the xml over an HTTP Post. The HTTP Post is always to one
static URL and the target web server is IIS. Threads are running
concurrently and have a cap of 30 in the pool, all threads are pretty much
constantly processing a request.We release the connection in our finally
block (if (postMethod != null) { postMethod.releaseConnection(); } )
As an example of the long wait times, we profiled one of our worker threads
and it took 3781 ms to process a message and do the HTTP Post. Out of the
3781 ms, 3628 ms are within
MultiThreadedHttpConnectionManager.doGetConnection(...) versus 67 ms to
perform the actual HTTP Post (via PostMethod.execute()).
Our profiler shows that methods called within
MultiThreadedHttpConnectionManager.doGetConnectiont(...) take no more than 1
ms.
Not sure what to make of this at this point, but it seems strange that there
is such a long wait time. We wire up the HttpClient configuration via
Spring, but essentially it looks like this:
httpConnectionManagerParams.setMaxConnections(40);
httpConnectionManagerParams.setSoTimeOut(1500);
multiThreadedHttpConnectionManager.setParams(httpConnectionManagerParams);
httpClient.setHttpConnectionManager(multiThreadedHttpConnectionManager);
Has anyone experienced such behavior before? Perhaps need to fine tune the
configuration?
Please let me know if more information is needed, will certainly provide it.
Thanks,
Al
Per default HttpClient uses only two simultaneous connections to the
same host regardless of how many total connections are allowed. Try
increasing the number of maximum connections per host.
Hope this helps
Oleg
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: PUT Expect:100-continue
On Wed, 2008-07-16 at 17:34 -0700, Uday Subbarayan wrote:
Hi All,
I am having trouble in using HTTPClient 3.1 for PUT method w/
Expect:100-Continue header.
My requirement is that,
[1] client sends a PUT request to the server with header only using this
Expect header and waits for the 100 response back from the server.
[2] Then client sends the body, after it received the 100 status back from
the server.
The problem is that if my server sends back 100-continue, HttpClient
complaints about,
INFO: Discarding unexpected response: HTTP/1.1 100 Continue.
Here is sample code:
String testURL=http://localhost:8080/testci/index;;
HttpClient client = new HttpClient();
PutMethod put = new PutMethod(url);
put.setUseExpectHeader(true);
int statusCode = client.executeMethod(put);
if(statusCode==100){
System.out.println(server response is 100!);
put.setRequestBody(new FileInputStream(test.txt));
}
It looks like i am making some mistake Can some one shed some light here?
Uday,
HTTP agents are not supposed to return 1xx status codes to the caller.
These are special purpose codes that are meant to be used internally
Just let HttpClient handle the expect-continue handshaking.
http://hc.apache.org/httpclient-3.x/performance.html#Expect-continue_handshake
You can see exactly what gets transferred across the wire by turning on
the wire logging on the client side.
http://hc.apache.org/httpclient-3.x/logging.html
Hope this helps
Oleg
Thanks,
-Uday.
-
I do not blog but e-write:
http://uds-web.blogspot.com
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
