from:"Oleg Kalnichevski"

Oh, man. No thunder, just emotional devastation. It has been quite an
ugly squabble and have made a few non-friends among the Infrastructure
folks. 

Anyways, we need to make an announcement on the site in order to preempt
(some of) the confusion among the users.

Finally this issue is done with and we can move on with the migration
process

Evil Comrade Oleg


On Mon, 2004-10-11 at 04:46, Adrian Sutton wrote:
 Hi all,
 Oleg has done a fantastic job in finally getting HttpClient converted 
 from a component to a real project in bugzilla.  There are a couple of 
 things remaining that some help could be used on:
 
 1. Check that everything looks right after the migration (the standard 
 bugzilla installation on nagoya should now show the migrated version).
 
 2. Decide if we now want to convert over to JIRA or stay with Bugzilla.
 
 Most of our issues with bugzilla should be solved now that we're a full 
 project, however JIRA is much better supported than Bugzilla so we may 
 want to move anyway.
 
 Let me know your thoughts and we'll unleash evil comrade Oleg on the 
 infrastructure team again. :)
 
 Three big cheers to Oleg for following through on this.  I don't mean 
 to steal the thunder of the announcement but figured I'd keep the 
 messages flowing during the night shift (aka: Australian day time).
 
 Regards,
 
 Adrian Sutton.
 --
 Intencha tomorrow's technology today
 Ph: 3420 4584 0422236329
 35 Prenzler St
 Upper Mount Gravatt 4122
 Australia QLD
 www.intencha.com


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [ANNOUNCE] Release of Commons HttpClient 2.0.2

Javen

HttpClient 2.0.x represents the stable branch whereas HttpClient 3.0 is
still in ALPHA development stage and may be subject to API change. We
are planning to freeze the 3.0 API quite soon and start working toward
the code freeze and a first stable release 

For detailed information on new features in HttpClient 3.0 please refer
to:

http://jakarta.apache.org/commons/httpclient/3.0/news.html

Oleg

On Mon, 2004-10-11 at 08:18, Javen Fang wrote:
 http://jakarta.apache.org/commons/httpclient/3.0/index.html
 
 But what I see the new is  3.0. 
 
 what is different is 2.0 and 3.0
 
 
 On Sun, 10 Oct 2004 23:12:15 -0700 (PDT), Srinivas Velidanda
 [EMAIL PROTECTED] wrote:
  Hi,
  
  I have been using HttpClient version 2.0.1. But have some problems while uploading 
  multiple files. Please let me know
  
  1. Can I upload multiple files using httpclient api from client to server (client 
  and server running on different systems).
  
  2. I worked with sample code at
  
  http://www.theserverside.com/articles/article.tss?l=HttpClient_FileUpload. 
  Everything works fine if I run the sample on the same systems but I cannot do the 
  same by making the client and server on different systems.
  
  I used the following sample at the above url
  
  Listing 9-6. HttpMultiPartFileUpload.java
  package com.commonsbook.chap9;
  import java.io.File;
  import java.io.IOException;
  
  import org.apache.commons.httpclient.HttpClient;
  import org.apache.commons.httpclient.methods.MultipartPostMethod;
  
  public class HttpMultiPartFileUpload {
 private static String url =
   http://localhost:8080/HttpServerSideApp/ProcessFileUpload.jsp;;
  
 public static void main(String[] args) throws IOException {
 HttpClient client = new HttpClient();
 MultipartPostMethod mPost = new MultipartPostMethod(url);
 client.setConnectionTimeout(8000);
  
 // Send any XML file as the body of the POST request
 File f1 = new File(students.xml);
 File f2 = new File(academy.xml);
 File f3 = new File(academyRules.xml);
  
 System.out.println(File1 Length =  + f1.length());
 System.out.println(File2 Length =  + f2.length());
 System.out.println(File3 Length =  + f3.length());
  
 mPost.addParameter(f1.getName(), f1);
 mPost.addParameter(f2.getName(), f2);
 mPost.addParameter(f3.getName(), f3);
  
 int statusCode1 = client.executeMethod(mPost);
  
 System.out.println(statusLine + mPost.getStatusLine());
 mPost.releaseConnection();
 }
  }
  
  and for processing file upload i used the following JSP
  
  %@ page contentType=text/html;charset=windows-1252%
  %@ page import=org.apache.commons.fileupload.DiskFileUpload%
  %@ page import=org.apache.commons.fileupload.FileItem%
  %@ page import=java.util.List%
  %@ page import=java.util.Iterator%
  %@ page import=java.io.File%
  html
  head
  meta http-equiv=Content-Type content=text/html; charset=windows-1252
  titleProcess File Upload/title
  /head
  %
 System.out.println(Content Type =+request.getContentType());
  
 DiskFileUpload fu = new DiskFileUpload();
 // If file size exceeds, a FileUploadException will be thrown
 fu.setSizeMax(100);
  
 List fileItems = fu.parseRequest(request);
 Iterator itr = fileItems.iterator();
  
 while(itr.hasNext()) {
   FileItem fi = (FileItem)itr.next();
  
   //Check if not form field so as to only handle the file inputs
   //else condition handles the submit button input
   if(!fi.isFormField()) {
 System.out.println(\nNAME: +fi.getName());
 System.out.println(SIZE: +fi.getSize());
 //System.out.println(fi.getOutputStream().toString());
 File fNew= new File(application.getRealPath(/), fi.getName());
  
 System.out.println(fNew.getAbsolutePath());
 fi.write(fNew);
   }
   else {
 System.out.println(Field =+fi.getFieldName());
   }
 }
  %
  body
  Upload Successful!!
  /body
  /html
  
  3. But I could not do the same by making the server and client on different 
  systems.
  
  4. Pl. let me know for the same purpose is there any patch available with current 
  version released.  i.e.2.0.2
  
  5. Is there a way to build Multipart request without putting the Http File Item in 
  the form.
 i.e. dynamically getting the file Items created programmatically at client side 
  and
 passing the Multipart request to the server
  
  thanks,
  Srinivas.
  
  
  Michael Becke [EMAIL PROTECTED] wrote:
  The Jakarta Commons team is pleased to announce the release of
  HttpClient 2.0.2. This release greatly improves the performance of
  executing methods where the response contains little or no content.
  Please visit the HttpClient 2.0 web site
  for more information on
  HttpClient and this release.
  
  Thank you,

Re: [ANNOUNCE] Release of Commons HttpClient 2.0.2

Hi Sudhakar,

Why should a personal opinion of one guy hurt? We are all entitled to
have our own opinion. As imperfect as HttpClient may be, it is still
being used by numerous state of the art projects such as Spring
framework, and to me that counts more than an opinion of one guy.

1. API is not stable

This is basically correct. See point 2.

2. API is flaky

This is basically correct. The API is far from being perfect. This said,
in my opinion is still wy better than Java Net API. The
reason why we can't keep the API stable is because early on (before any
of the current committers joined the project) a few unfortunate design
have been made. Now we are facing an unpleasant task of fixing them
while supporting the existing user base. This is not an easy task

3. No proper documentation and easy to understand explanation

Allow me to disagree. Some folks expressed opinion that HttpClient had
better documentation than other projects.

http://www.onjava.com/pub/a/onjava/2003/06/25/commons.html?page=last

This is not to say that documentation could not be better, but we rarely
have people complaining about the quality of support for HttpClient

4. No proper examples provided by team to understand API.

Allow me to disagree. See for yourself

http://jakarta.apache.org/commons/httpclient/tutorial.html
http://cvs.apache.org/viewcvs.cgi/jakarta-commons/httpclient/src/examples/?only_with_tag=HTTPCLIENT_2_0_BRANCH

4. Security problems may araise

Whenever you venture out of your home you are at risk of being hit by a
bus. Whenever you have your computer connected to a public network,
security problems may arise. What specific security problems that expert
was referring to?

5. Takes much memory when compared with Java Net API

Try posting 200MB file with Java Net API and tell me about memory
consumption.

6. My code becomes complicated, if I use HttpCLient

Whenever you write code, it can get complicated unless you keep it
simple

7. As name it says limited to Http Protocol. - Which is not suitable for multi
protocol
implementations.

This is correct. We have no plans changing that.

So I need some clarifications from Development team of HttpClient API.
I hope I get the Positive response from the team regarding all matters what I have
mentioned
above.

I hope that clarifies things a little

Cheers

Oleg

***
The information in this email is confidential and may be legally privileged. Access
to this email by anyone other than the intended addressee is unauthorized. If you are
not the intended recipient of this message, any review, disclosure, copying,
distribution, retention, or any action taken or omitted to be taken in reliance on it
is prohibited and may be unlawful. If you are not the intended recipient, please
reply to or forward a copy of this message to the sender and delete the message, any
attachments, and any copies thereof from your system.
***

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

[ANNOUNCE] HttpClient is now a separate project in [Bugzilla] issue tracking system


HttpClient project has taken a very important step toward becoming a
full-fledged Jakarta level project. From today, HttpClient is a separate
project in Apache Bugzilla issue tracking system. It is no longer a
component of the Commons project. Please use the following details when
filing bug reports for 2.0 and 3.0 branches of HttpClient: 

Product: HttpClient
Component: Commons HttpClient

Use the following URL for convenience:
http://issues.apache.org/bugzilla/enter_bug.cgi?product=HttpClient

All the existing issues, closed and open, are still available under
their usual URLs.

With HttpClient being a separate project in Bugzilla we will be able to
define release versions and target milestones independently from Jakarta
Commons 

Oleg

***
The information in this email is confidential and may be legally privileged.  Access 
to this email by anyone other than the intended addressee is unauthorized.  If you are 
not the intended recipient of this message, any review, disclosure, copying, 
distribution, retention, or any action taken or omitted to be taken in reliance on it 
is prohibited and may be unlawful.  If you are not the intended recipient, please 
reply to or forward a copy of this message to the sender and delete the message, any 
attachments, and any copies thereof from your system.
***

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

[Bugzilla] vs [JIRA] revisited


Folks

Due to the recent upgrade of HttpClient to a full project in Bugzilla
JIRA no longer has a definitive edge over Bugzilla. Nonetheless, JIRA
still a newer and more flexible system which can potentially make our
life and that of our users simpler. 

http://nagoya.apache.org/jira/browse/INFRA-74

The only fundamental gripe with JIRA that some folks have is that it is
not open-source. I do not think we should try to be holier than the Pope
himself, since too many projects have already defected to JIRA 

Religious reasons aside, I also foresee technical difficulties too.
Currently (at least with Bugzilla 2.14.2) there appears no way to make a
project completely read-only. Submission of new reports can be disabled,
but one can still modify the existing bug reports.

There is a few options:

(1) Ask folks to resubmit their comments in JIRA when important / ignore
modifications made in Bugzilla when unimportant.

(2) Tweak Bugzilla a little to prevent mutation of existing reports.
Note, supposedly Apache Bugzilla is already a fork. So, the real trouble
here is to convince the Infrastructure folks to apply a patch, and more
importantly reapply it every time Bugzilla is upgraded. Now, the REAL
REAL trouble is that it almost took a full scale nuclear conflict
between the Evil Russian Empire (me) and the Freedom Alliance (the
Infrastructure team) to implement what turned out to be a fairly minor
change to get HttpClient promoted to a project status. From now on, I am
deeply skeptical about everything that has to do with Bugzilla
maintenance. Besides, I may not survive yet another squabble with Noel.

(3) Wait until other Jakarta Commons project migrate first. Long may be
the wait, though 

So, what's the popular opinion on that? Non-committers are highly
encouraged to voice their opinion too

Evil Comrade Oleg   


***
The information in this email is confidential and may be legally privileged.  Access 
to this email by anyone other than the intended addressee is unauthorized.  If you are 
not the intended recipient of this message, any review, disclosure, copying, 
distribution, retention, or any action taken or omitted to be taken in reliance on it 
is prohibited and may be unlawful.  If you are not the intended recipient, please 
reply to or forward a copy of this message to the sender and delete the message, any 
attachments, and any copies thereof from your system.
***

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [Bugzilla] vs [JIRA] revisited


Hi Roland
I agree this makes sense. I am not sure, though, whether we can
reasonably expect things to happen in one 'Big Bang': mailing lists, new
CVS repository, web site, JIRA migration and so on. Most likely not. 

Besides, unless we start getting MASSIVELY more feedback on 3.0, I am
not sure what else we can do but start hacking on 4.0 branch while 3.0
still goes through its natural development cycle: alpha - beta - rc -
release. I sense the work on 4.0 may well commence as early as next
month. Basically that will buy us some time, but not much. I certainly
want to start the discussion on the 4.0 architecture (at least in broad
strokes) very, very soon.

This, again, makes the issue of issue tracking system highly important,
as we better have a sane road map and reasonably well articulated
strategy as soon as people start asking what the heck Jakarta HttpClient
4.0 is all about and how on earth we ended up with 3 API incompatible
branches.

Oleg


On Mon, 2004-10-11 at 16:43, Roland Weber wrote:
 Hi Oleg,
 
 could we synchronize the switch with the 4.0 implementation?
 In other words, continue with Bugzilla for 2.0 and 3.0, but once
 work gets started on 4.0, where the API changes and the package
 names probably change, then the bug tracking system changes
 as well?
 
 cheers,
   Roland

***
The information in this email is confidential and may be legally privileged.  Access 
to this email by anyone other than the intended addressee is unauthorized.  If you are 
not the intended recipient of this message, any review, disclosure, copying, 
distribution, retention, or any action taken or omitted to be taken in reliance on it 
is prohibited and may be unlawful.  If you are not the intended recipient, please 
reply to or forward a copy of this message to the sender and delete the message, any 
attachments, and any copies thereof from your system.
***

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: redirect problem in HttpClient

2004-10-09 Thread Oleg Kalnichevski

Zulfi,
According to the log the request body does get written to the socket
output stream:

[DEBUG] wire - - SOAP-ENV:Envelope
...
ello/SOAP-ENV:Body/SOAP-ENV:Envelope

So, really I do not think this has anything to do with HttpClient given
the data.

Do try out different settings: CONTENT_LENGTH_AUTO and
CONTENT_LENGTH_CHUNKED and see if that makes any difference.

I suggest you try to reproduce the problem using a simple (or as simple
as possible) test case, something which I could also run locally on my
machine. I am afraid we will be unable to help you unless you manage to
narrow the problem down, at least somewhat

Cheers,

Oleg

On Sat, 2004-10-09 at 00:10, Zulfi Umrani wrote:
Hi Oleg,
I am setting the request body as InputStream, but I do set the
Content-Length header as well. I do print out the bytes for body as
string before I call executeMethod and I do see the body being printed
even for the transaction in question. The InputStream for body is
created out of those bytes. I have enabled full wire(header and content)
+ context logging by setting the following properties. Log is once again
attahced for the transaction in question. Let me know if I have to set
some other property as well.

System.setProperty(org.apache.commons.logging.Log,
org.apache.commons.logging.impl.SimpleLog);
System.setProperty(org.apache.commons.logging.simplelog.showdatetime,
true);
System.setProperty(org.apache.commons.logging.simplelog.log.httpclient.wire,
debug);
System.setProperty(org.apache.commons.logging.simplelog.log.org.apache.commons.httpclient,
debug);

Thanks,
Zulfi
[EMAIL PROTECTED] 10/8/2004 1:25:07 PM
Zulfi,

My only guess is that this is a thread synchronization related
problem,
as the problem appears to be irregular.

How do you set the request body? As an InputStream? How do you set the
content length? Explicitly or as CONTENT_LENGTH_AUTO? Can it be that
the
input stream is exhausted before HttpClient has a chance to send its
content across the wire?

Can you send me the context/wire log for the HTTP transaction in
question?

Oleg

On Fri, 2004-10-08 at 22:10, Zulfi Umrani wrote:
Hi Oleg,
Yes, the connection is closed by the server. But it is closed
because
the HttpClient did not send it the request BODY. The problem is that
occasionally HttpClient does not send the request as I described. For
a
few times, it does send the request and then it just doesnt send it,
but
only once. And then it is back to sending request body. The server
tries
to read the body but can not, hence it closes the connection for the
reuqest which had no body and had Content-Length header value 1. I
have sent you the reuqest in error from the tunnel. Looks like it
sent
the Content-Length header, but not the body of the request.

Hope this helps,
Zulfi
[EMAIL PROTECTED] 10/8/2004 12:46:04 PM
Zulfi

HttpRecoverableException: Software caused connection abort: recv
failed usually means that the connection was closed on the server
side
while HttpClient was still reading the response. The is more likely
to
be the server side problem.

What exactly do you mean by occasionally HttpClient does not send
the
request body. Does it throw an exception, or blocks, or something
else?

Oleg

On Fri, 2004-10-08 at 19:53, Zulfi Umrani wrote:
I am getting the following exception for redirect responses from
HttpClient. Attached is the log from HttpClient. I know that
HttpClient
does not support redirect directly, hence we have written some
code
to
handle that. The problem seems to be that, occasionally HttpClient
does
not send the request body though it writes out the Content-Length
header. If you send the same request one after another by starting
a
new
JVM for each request, such as running a client from commandline,
you
will see that, it sends the request body for a few times and then
it
does not send it once and then it starts to send it again. This
behavior
keeps repeating. Below is the request sent out by HttpClient and
response received by it, captured through a tunnel. I will
appreciate
if
someone can explain the behavior.

Exception in thread main java.rmi.RemoteException:
java.net.SocketException: S
oftware caused connection abort: recv failed; nested exception is:
org.apache.commons.httpclient.HttpRecoverableException:
java.net.SocketE
xception: Software caused connection abort: recv failed
at

com.sssw.jbroker.web.ServiceException.mapToRemote(ServiceException.ja
va:92)
at
hello.HelloBinding_Stub.sayHello(HelloBinding_Stub.java:85)
at hello.Client.main(Client.java:25)
Caused by: org.apache.commons.httpclient.HttpRecoverableException:
java.net.Sock
etException: Software caused connection abort: recv failed
at

[PATCH] ChunkedInputStream no longer requires an HttpMethod parameter


Folks,
I would like to make ChunkedInputStream a little more reusable by making
HttpMethod parameter optional. Please let me know if you agree/disagree

Oleg   


***
The information in this email is confidential and may be legally privileged.  Access 
to this email by anyone other than the intended addressee is unauthorized.  If you are 
not the intended recipient of this message, any review, disclosure, copying, 
distribution, retention, or any action taken or omitted to be taken in reliance on it 
is prohibited and may be unlawful.  If you are not the intended recipient, please 
reply to or forward a copy of this message to the sender and delete the message, any 
attachments, and any copies thereof from your system.
***Index: ChunkedInputStream.java
===
RCS file: /home/cvspublic/jakarta-commons/httpclient/src/java/org/apache/commons/httpclient/ChunkedInputStream.java,v
retrieving revision 1.22
diff -u -r1.22 ChunkedInputStream.java
--- ChunkedInputStream.java	18 Apr 2004 23:51:34 -	1.22
+++ ChunkedInputStream.java	8 Oct 2004 14:15:54 -
@@ -49,7 +49,7 @@
  * not requiring the client to remember to read the entire contents of the
  * response./p
  *
- * @author Ortwin Gl
+ * @author Ortwin Glueck
  * @author Sean C. Sullivan
  * @author Martin Elwin
  * @author Eric Johnson
@@ -80,33 +80,41 @@
 private boolean closed = false;
 
 /** The method that this stream came from */
-private HttpMethod method;
+private HttpMethod method = null;
 
 /** Log object for this class. */
 private static final Log LOG = LogFactory.getLog(ChunkedInputStream.class);
+
 /**
+ * ChunkedInputStream constructor
  *
- *
- * @param in must be non-null
- * @param method must be non-null
+ * @param in the raw input stream
+ * @param method the originating HTTP method. Can be ttnull/tt.
  *
  * @throws IOException If an IO error occurs
  */
 public ChunkedInputStream(
 final InputStream in, final HttpMethod method) throws IOException {
 
-  if (in == null) {
-throw new IllegalArgumentException(InputStream parameter may not be null);
-  }
-  if (method == null) {
-throw new IllegalArgumentException(HttpMethod parameter may not be null);
-  }
+	if (in == null) {
+		throw new IllegalArgumentException(InputStream parameter may not be null);
+	}
 this.in = in;
 this.method = method;
 this.pos = 0;
 }
 
 /**
+ * ChunkedInputStream constructor
+ *
+ * @param in the raw input stream
+ *
+ * @throws IOException If an IO error occurs
+ */
+public ChunkedInputStream(final InputStream in) throws IOException {
+	this(in, null);
+}
+/**
  * p Returns all the data in a chunked stream in coalesced form. A chunk
  * is followed by a CRLF. The method returns -1 as soon as a chunksize of 0
  * is detected./p
@@ -301,17 +309,21 @@
 private void parseTrailerHeaders() throws IOException {
 Header[] footers = null;
 try {
-footers = HttpParser.parseHeaders(in, 
-method.getParams().getHttpElementCharset());
+String charset = US-ASCII;
+if (this.method != null) {
+charset = this.method.getParams().getHttpElementCharset();
+}
+footers = HttpParser.parseHeaders(in, charset);
 } catch(HttpException e) {
 LOG.error(Error parsing trailer headers, e);
 IOException ioe = new IOException(e.getMessage());
 ExceptionUtil.initCause(ioe, e); 
 throw ioe;
 }
-
-for (int i = 0; i  footers.length; i++) {
-method.addResponseFooter(footers[i]);
+if (this.method != null) {
+for (int i = 0; i  footers.length; i++) {
+this.method.addResponseFooter(footers[i]);
+}
 }
 }
 
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: getting through a proxy server


Madeleine,

If you activate wire/context logging in your application you'll get more
details on what exactly goes wrong

http://jakarta.apache.org/commons/httpclient/logging.html

If you need help reading the log, feel free to post it to this list. Do
obfuscate security sensitive bits such as logon credentials prior to
posting.

Oleg  


On Fri, 2004-10-08 at 16:45, Madeleine Wright wrote:
 Please can someone suggest the simplest way to access a URL 
 programatically through a proxy server?  I'm using HttpClient and the 
 proxy bits of my code look like this (everything else works fine - I can 
 access all sites inside the firewall):
 
 HttpClient client = new HttpClient();
 .
 client.getHostConfiguration().setProxy(proxyHost, proxyPort);
 Credentials creds = new UsernamePasswordCredentials(userName, password);
 client.getState().setProxyCredentials(realm, proxyHost, defaultcreds);
 ...
 GetMethod method = new GetMethod(url);
 
 I realize I seem to be supplying proxy host details twice!  But I can't 
 otherwise see how to set the proxy port?  I keep getting an access 
 denied message from the proxy server, indicating that I have not 
 authenticated myself.  Does anyone know how I do that other than the 
 method above (I am sure the actual proxy details given are correct).
 
 Thanks.
 
 Mad
 
 -
 To unsubscribe, e-mail: [EMAIL PROTECTED]
 For additional commands, e-mail: [EMAIL PROTECTED]

***
The information in this email is confidential and may be legally privileged.  Access 
to this email by anyone other than the intended addressee is unauthorized.  If you are 
not the intended recipient of this message, any review, disclosure, copying, 
distribution, retention, or any action taken or omitted to be taken in reliance on it 
is prohibited and may be unlawful.  If you are not the intended recipient, please 
reply to or forward a copy of this message to the sender and delete the message, any 
attachments, and any copies thereof from your system.
***

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

RE: getting through a proxy server


Bob,
You are absolutely right about method.getHostConfiguration() taking
precedence over client.getHostConfiguration(). However, HttpClient does
copy the proxy information from the agent host config to the method host
config. So, that should not be the reason for proxy problems Madeleine
has been having.
 
We realize this can be horribly confusing. As of release 3.0 the use of
HttpMethod#getHostConfiguration() will be deprecated and discouraged

Oleg


On Fri, 2004-10-08 at 17:01, St Jacques, Robert wrote:
 I believe that your problem is the fact that by calling the GetMethod(url)
 constructor, you are creating a method with its own host configuration; in
 this case, the method's host configuration is used when the method is
 executed (as opposed to the default host configuration that you have created
 on the connection).
 
 In other words, the host, port, and path information for the specific method
 invocation are extracted from the URL that you used to construct the
 GetMethod.  If you replace this code:
 
 client.getHostConfiguration()...
 
 with this code:
 
 method.getHostConfiguration()...
 
 You should be all set.  Either that or call the parameter-less GetMethod
 constructor, which will cause it to default to the host configuration for
 your client.
 
 OR you could call:
 
 client.executeMethod( client.getHostConfiguration(), method,
 client.getState())
 
 That should work, too.
 
 Hope that helps,
 Bob
 
 -Original Message-
 From: Yuzwa, Erik [mailto:[EMAIL PROTECTED] 
 Sent: Friday, October 08, 2004 10:53 AM
 To: 'Commons HttpClient Project'
 Subject: RE: getting through a proxy server
 
 
 Madeleine,
 
 Stupid question but is your proxy server using NTLM authentication? 
 
 I had to do some hoop jumping to get NTLM to work properly, but it's working
 now if you need some code.
 
 Erik
 
 -Original Message-
 From: Madeleine Wright [mailto:[EMAIL PROTECTED] 
 Sent: Friday, October 08, 2004 8:46 AM
 To: Commons HttpClient Project
 Subject: getting through a proxy server
 
 
 Please can someone suggest the simplest way to access a URL 
 programatically through a proxy server?  I'm using HttpClient and the 
 proxy bits of my code look like this (everything else works fine - I can 
 access all sites inside the firewall):
 
 HttpClient client = new HttpClient();
 . client.getHostConfiguration().setProxy(proxyHost,
 proxyPort); Credentials creds = new UsernamePasswordCredentials(userName,
 password); client.getState().setProxyCredentials(realm, proxyHost,
 defaultcreds); ... GetMethod method = new
 GetMethod(url);
 
 I realize I seem to be supplying proxy host details twice!  But I can't 
 otherwise see how to set the proxy port?  I keep getting an access 
 denied message from the proxy server, indicating that I have not 
 authenticated myself.  Does anyone know how I do that other than the 
 method above (I am sure the actual proxy details given are correct).
 
 Thanks.
 
 Mad
 
 -
 To unsubscribe, e-mail:
 [EMAIL PROTECTED]
 For additional commands, e-mail:
 [EMAIL PROTECTED]
 
 -
 To unsubscribe, e-mail:
 [EMAIL PROTECTED]
 For additional commands, e-mail:
 [EMAIL PROTECTED]
 
 -
 To unsubscribe, e-mail: [EMAIL PROTECTED]
 For additional commands, e-mail: [EMAIL PROTECTED]

***
The information in this email is confidential and may be legally privileged.  Access 
to this email by anyone other than the intended addressee is unauthorized.  If you are 
not the intended recipient of this message, any review, disclosure, copying, 
distribution, retention, or any action taken or omitted to be taken in reliance on it 
is prohibited and may be unlawful.  If you are not the intended recipient, please 
reply to or forward a copy of this message to the sender and delete the message, any 
attachments, and any copies thereof from your system.
***

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: redirect problem in HttpClient

Zulfi

HttpRecoverableException: Software caused connection abort: recv
failed usually means that the connection was closed on the server side
while HttpClient was still reading the response. The is more likely to
be the server side problem. 

What exactly do you mean by occasionally HttpClient does not send the
request body. Does it throw an exception, or blocks, or something else?

Oleg
 

On Fri, 2004-10-08 at 19:53, Zulfi Umrani wrote:
 I am getting the following exception for redirect responses from
 HttpClient. Attached is the log from HttpClient. I know that HttpClient
 does not support redirect directly, hence we have written some code to
 handle that. The problem seems to be that, occasionally HttpClient does
 not send the request body though it writes out the Content-Length
 header. If you send the same request one after another by starting a new
 JVM for each request, such as running a client from commandline, you
 will see that, it sends the request body for a few times and then it
 does not send it once and then it starts to send it again. This behavior
 keeps repeating. Below is the request sent out by HttpClient and
 response received by it, captured through a tunnel. I will appreciate if
 someone can explain the behavior.
 
 Exception in thread main java.rmi.RemoteException:
 java.net.SocketException: S
 oftware caused connection abort: recv failed; nested exception is:
 org.apache.commons.httpclient.HttpRecoverableException:
 java.net.SocketE
 xception: Software caused connection abort: recv failed
 at
 com.sssw.jbroker.web.ServiceException.mapToRemote(ServiceException.ja
 va:92)
 at hello.HelloBinding_Stub.sayHello(HelloBinding_Stub.java:85)
 at hello.Client.main(Client.java:25)
 Caused by: org.apache.commons.httpclient.HttpRecoverableException:
 java.net.Sock
 etException: Software caused connection abort: recv failed
 at
 org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodB
 ase.java:1965)
 at
 org.apache.commons.httpclient.HttpMethodBase.processRequest(HttpMetho
 dBase.java:2659)
 at
 org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.j
 ava:1093)
 at
 org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.jav
 a:675)
 at
 org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.jav
 a:558)
 
 Request going out from HttpClient:
 POST /services/hello HTTP/1.1 
 SOAPAction: http://www.hello/sayHello;
 Content-Type: text/xml; charset=utf-8 
 Connection: keep-alive 
 User-Agent: Jakarta Commons-HttpClient/2.0final 
 Host: 840serv7:54892
 Content-Length: 421 
 
 Response received by HttpClient:
 HTTP/1.1 307 redirected 
 Set-Cookie: dispatch-http-rt=840serv7:80; path=/; 
 Content-Length: 0 
 Location: http://840serv7:80/services/hello
 
 Thanks,
 Zulfi
 
 __
 -
 To unsubscribe, e-mail: [EMAIL PROTECTED]
 For additional commands, e-mail: [EMAIL PROTECTED]


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: redirect problem in HttpClient

Zulfi,

My only guess is that this is a thread synchronization related problem,
as the problem appears to be irregular. 

How do you set the request body? As an InputStream? How do you set the
content length? Explicitly or as CONTENT_LENGTH_AUTO? Can it be that the
input stream is exhausted before HttpClient has a chance to send its
content across the wire?

Can you send me the context/wire log for the HTTP transaction in
question?

Oleg

On Fri, 2004-10-08 at 22:10, Zulfi Umrani wrote:
 Hi Oleg,
 Yes, the connection is closed by the server. But it is closed because
 the HttpClient did not send it the request BODY. The problem is that
 occasionally HttpClient does not send the request as I described. For a
 few times, it does send the request and then it just doesnt send it, but
 only once. And then it is back to sending request body. The server tries
 to read the body but can not, hence it closes the connection for the
 reuqest which had no body and had Content-Length header value  1. I
 have sent you the reuqest in error from the tunnel. Looks like it sent
 the Content-Length header, but not the body of the request. 
 
 Hope this helps,
 Zulfi
  [EMAIL PROTECTED] 10/8/2004 12:46:04 PM 
 Zulfi
 
 HttpRecoverableException: Software caused connection abort: recv
 failed usually means that the connection was closed on the server
 side
 while HttpClient was still reading the response. The is more likely to
 be the server side problem. 
 
 What exactly do you mean by occasionally HttpClient does not send the
 request body. Does it throw an exception, or blocks, or something
 else?
 
 Oleg
  
 
 On Fri, 2004-10-08 at 19:53, Zulfi Umrani wrote:
  I am getting the following exception for redirect responses from
  HttpClient. Attached is the log from HttpClient. I know that
 HttpClient
  does not support redirect directly, hence we have written some code
 to
  handle that. The problem seems to be that, occasionally HttpClient
 does
  not send the request body though it writes out the Content-Length
  header. If you send the same request one after another by starting a
 new
  JVM for each request, such as running a client from commandline, you
  will see that, it sends the request body for a few times and then it
  does not send it once and then it starts to send it again. This
 behavior
  keeps repeating. Below is the request sent out by HttpClient and
  response received by it, captured through a tunnel. I will appreciate
 if
  someone can explain the behavior.
  
  Exception in thread main java.rmi.RemoteException:
  java.net.SocketException: S
  oftware caused connection abort: recv failed; nested exception is:
  org.apache.commons.httpclient.HttpRecoverableException:
  java.net.SocketE
  xception: Software caused connection abort: recv failed
  at
 
 com.sssw.jbroker.web.ServiceException.mapToRemote(ServiceException.ja
  va:92)
  at
 hello.HelloBinding_Stub.sayHello(HelloBinding_Stub.java:85)
  at hello.Client.main(Client.java:25)
  Caused by: org.apache.commons.httpclient.HttpRecoverableException:
  java.net.Sock
  etException: Software caused connection abort: recv failed
  at
 
 org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodB
  ase.java:1965)
  at
 
 org.apache.commons.httpclient.HttpMethodBase.processRequest(HttpMetho
  dBase.java:2659)
  at
 
 org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.j
  ava:1093)
  at
 
 org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.jav
  a:675)
  at
 
 org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.jav
  a:558)
  
  Request going out from HttpClient:
  POST /services/hello HTTP/1.1 
  SOAPAction: http://www.hello/sayHello;
  Content-Type: text/xml; charset=utf-8 
  Connection: keep-alive 
  User-Agent: Jakarta Commons-HttpClient/2.0final 
  Host: 840serv7:54892
  Content-Length: 421 
  
  Response received by HttpClient:
  HTTP/1.1 307 redirected 
  Set-Cookie: dispatch-http-rt=840serv7:80; path=/; 
  Content-Length: 0 
  Location: http://840serv7:80/services/hello 
  
  Thanks,
  Zulfi
  
 
 __
 
 -
  To unsubscribe, e-mail:
 [EMAIL PROTECTED] 
  For additional commands, e-mail:
 [EMAIL PROTECTED] 
 
 
 -
 To unsubscribe, e-mail:
 [EMAIL PROTECTED] 
 For additional commands, e-mail:
 [EMAIL PROTECTED] 
 
 
 -
 To unsubscribe, e-mail: [EMAIL PROTECTED]
 For additional commands, e-mail: [EMAIL PROTECTED]
 


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [VOTE] 2.0.2 release

2004-10-06 Thread Oleg Kalnichevski

+1

On Wed, 2004-10-06 at 03:48, Michael Becke wrote:
 After a little delay...  I propose that we mark the latest code in CVS  
 HTTPCLIENT_2_0 as 2.0.2  and proceed with a release.  Please vote as  
 follows:
  
 --
   Vote:  HttpClient 2.0.2 release
   [x] +1 I am in favor of the release, and will help support it.
   [ ] +0 I am in favor of the release, but am unable to help support it.
   [ ] -0 I am not in favor of the release.
   [ ] -1 I am against this proposal (must include a reason).
 
  
 --
 
 
 -
 To unsubscribe, e-mail: [EMAIL PROTECTED]
 For additional commands, e-mail: [EMAIL PROTECTED]
 


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: Problem with Preferences Architecture

2004-10-06 Thread Oleg Kalnichevski


 Actually the new Preferences Architecture is quite neat and great if it all
 works. It is only the HostConfiguration that does not work and I would love
 to figure out why.
 

Vikram,

I believe the problem with HostConfiguration has been fixed in CVS HEAD.
We'd love to hear from you if now you find the new preferences
architecture working as expected.

Cheers,

Oleg


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: Unlocatable exception withinhttpclient.HttpRecoverableException

2004-10-05 Thread Oleg Kalnichevski


Jeff,

Sometimes, usually when under heavy load, the web server may be able to
receive requests but unable to process them due to low resources (most
commonly worker threads). In such a case the web server may simply have
no other choice but to drop the connection without sending back any
status code. This in turn causes HttpClient to throw a
HttpRecoverableException unable to find line starting with HTTP

Can it be that you have been stressing the server a bit too much?

Anyways, usually unable to find line starting with HTTP exception is
not a HttpClient fault, but that of the target server. The regrettable
thing about it it is a poor choice of exception class name and a
misleading error message. (This problem has already been fixed in
HttpClient 3.0)

Hope this helps a little

Oleg

On Tue, 2004-10-05 at 08:27, Jeff Fern wrote:
 Hello all,
 
 I am running Tomcat 5.0.27 with Java 1.4.2 on my local machine (Debian
 sarge). I have a Java webapp running on Tomcat, which outputs XML and I am
 using Cocoon 2.1 to style the XML with a static XSLT stylesheet to generate
 HTML.
 
 I have managed to generate an bug which occurs about 80% of the time.
 
 When I submit a form, cocoon generates one of a couple of errors:
 
 org.apache.cocoon.ProcessingException: There is a problem with the URI:
 
  
 http://127.0.0.1:8080/Conference/GetConference?conferenceName=UbiComp%20200
 5
 http://127.0.0.1:8080/Conference/GetConference?conferenceName=UbiComp%202005
 :
 
 org.apache.commons.httpclient.HttpRecoverableException:
 
 org.apache.commons.httpclient.HttpRecoverableException: Error in parsing the
 status line from the response: unable to find line starting with HTTP
 
 or org.apache.cocoon.ProcessingException: There is a problem with the
 
 URI:  http://127.0.0.1:8080/Conference/AddDetails
 http://127.0.0.1:8080/Conference/AddDetails:
 
 org.apache.commons.httpclient.HttpRecoverableException:
 
 java.net.SocketException: Socket closed
 
 Trying to re-submit the same information sometimes displays the correct
 results, sometimes generates an error.
 
 I have enabled logging with log4j using the examples at
 http://jakarta.apache.org/commons/httpclient/logging.html
 http://jakarta.apache.org/commons/httpclient/logging.html The only ouput
 generated in the log file is:
 
 - Recoverable exception caught when processing request
 
 - Recoverable exception caught but MethodRetryHandler.retryMethod()
 
 returned false, rethrowing exception
 
 I have also tried my code on Tomcat 5.5 with Java 1.5 and get the exact same
 results.
 
 I am unable to locate any piece of my code that causes this error.
 
 Any help or suggestions would be much appreciated,
 
 Regards,
 
  
 
 
 
 
 
 
 
 -Jeff Fern

***
The information in this email is confidential and may be legally privileged.  Access 
to this email by anyone other than the intended addressee is unauthorized.  If you are 
not the intended recipient of this message, any review, disclosure, copying, 
distribution, retention, or any action taken or omitted to be taken in reliance on it 
is prohibited and may be unlawful.  If you are not the intended recipient, please 
reply to or forward a copy of this message to the sender and delete the message, any 
attachments, and any copies thereof from your system.
***

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: java.net.SocketException: Connection reset

2004-10-03 Thread Oleg Kalnichevski

Pavel,
It does seem a little unusual and does appear likely to be a problem on
the server side. 'Connection reset' error usually occurs in the
following two cases:

(1) the server drops the connection on the unsuspecting HttpClient while
it is still busy transmitting the request. The most common cause of this
problem is an authorization error. Can it be that your application uses
invalid credentials?

(2) the server fails to send any response due to an internal error and
simply drops the connection without sending any status code back to the
client

I believe that the latter case is more likely than the former, but let's
not rule out any possibility. 

Things to try:

(1) Activate so called 'expect-continue' handshake and see if that makes
any difference.

http://jakarta.apache.org/commons/httpclient/apidocs/org/apache/commons/httpclient/methods/ExpectContinueMethod.html#setUseExpectHeader(boolean)

(2) Consider executing a GET against the url containing the form , get
the client authenticated, and only then execute the POST (very much like
the browser does)

Hope this helps

Vsego,

Oleg

On Sat, 2004-10-02 at 22:30, Skandiska OY wrote:
 Greeting from Estonia
 
 I'm trying to pull data from an online price quotation system
 using HTTPClient's POST method. SOMETIMES (rather rarely) the thing
 works, mostly not. Is something really wrong with the server or
 undesigned goofed in the RTFM stage ?
 
 The URL is accessible and the form works fine under my Mozilla browser.
 
 Many thanks for your effort
 Pavel
 +++ Java prog +++
 /*
  * Created on 26.09.2004
  *
  */
 package ee.skandiska.webBotTest;
 
 import java.io.*;
 import org.apache.commons.httpclient.*;
 import org.apache.commons.httpclient.methods.*;
 
 import com.quiotix.html.parser.*;
 /**
  * @author Pavel
  *
  */
 public class rapBot {
 
   public void run() {
System.setProperty(org.apache.commons.logging.Log,
 org.apache.commons.logging.impl.SimpleLog);
System.setProperty(org.apache.commons.logging.simplelog.showdatetime,
 true);
System.setProperty(org.apache.commons.logging.simplelog.log.httpclient
 .wire, debug);
System.setProperty(org.apache.commons.logging.simplelog.log.org.apache
 .commons.httpclient, debug);
 
Credentials creds = null;
creds = new UsernamePasswordCredentials(sorry,
 cant_disclose_this);
//create a singular HttpClient object
HttpClient client = new HttpClient();
//set all timeouts to 20 seconds
client.setConnectionTimeout(2);
client.setTimeout(2);
client.setHttpConnectionFactoryTimeout(2);
//set the default credentials
if (creds != null) {
client.getState().
   setCredentials(INDEX Member Area, www.diamonds.net, creds);
}
String url = http://www.diamonds.net/rapnet/prices/prcwks.asp;;
//create a method object
PostMethod postMethod = new PostMethod(url);
NameValuePair[] data = {
  new NameValuePair(submit2, Calculate),
  new NameValuePair(shape, Round),
  new NameValuePair(size, 1.0),
 new NameValuePair(color, F),
 new NameValuePair(clar, VVS1)
};
postMethod.setRequestBody(data);
postMethod.setFollowRedirects(true);
DefaultMethodRetryHandler retryHandler = new
 DefaultMethodRetryHandler();
retryHandler.setRetryCount(5);
retryHandler.setRequestSentRetryEnabled(true);
postMethod.setMethodRetryHandler(retryHandler);
//execute the method
String responseBody = null;
try{
int statusCode = client.executeMethod(postMethod);
Reader reader = new
 InputStreamReader(postMethod.getResponseBodyAsStream());
HtmlDocument document = new HtmlParser(reader).HtmlDocument();
document.accept(new HtmlScrubber(HtmlScrubber.DEFAULT_OPTIONS
 | HtmlScrubber.TRIM_SPACES));
System.out.println(+++ parsed HTML body +++);
document.accept(new HtmlDumper(System.out));
reader.close();
 //   responseBody = postMethod.getResponseBodyAsString();
} catch (HttpException he) {
System.err.println(Http error connecting to ' + url + ');
System.err.println(he.getMessage());
System.exit(-4);
} catch (IOException ioe){
System.err.println(Unable to connect to ' + url + ');
System.exit(-3);
} catch (ParseException e) {
   e.printStackTrace();
   System.exit(-5);
   }
//write out the request headers
System.out.println(*** Request ***);
System.out.println(Request Path:  + postMethod.getPath());
System.out.println(Request Query:  + postMethod.getQueryString());
Header[] requestHeaders = postMethod.getRequestHeaders();
for (int i=0; irequestHeaders.length;

Re: convert usage if URLConnection to HttpClient

2004-10-02 Thread Oleg Kalnichevski

Jason,

Take a look at the HttpClient tutorial and the authentication guide.
That should get you started:

http://jakarta.apache.org/commons/httpclient/tutorial.html
http://jakarta.apache.org/commons/httpclient/authentication.html

Same code can be found here:

http://cvs.apache.org/viewcvs.cgi/jakarta-commons/httpclient/src/examples/BasicAuthenticationExample.java?rev=1.1.2.3only_with_tag=HTTPCLIENT_2_0_BRANCHview=markup

If all this will not help, let me know

Oleg

On Sat, 2004-10-02 at 20:16, Jason Novotny wrote:
 Hi,
 
 I had been using some crufty code using URLConnection to perform 
 basic auth to retrieve the list of applications using the Tomcat manager 
 webapp. I'd like to convert this to use commons-httpclient-2.0.1.
 
 It should be pretty simple I think-- what is required is to invoke 
 http://127.0.0.1/manager?list performing basic authentication using name 
 and password and then I get back a response which has a specific format.
 My current code is shown below and I'd like to know what the 3 or 5 
 magic lines are to do the same thing using HttpClient.
 
 Thanks very much, Jason
 
 try {
 String serverName = req.getServerName();
 int serverPort = req.getServerPort();
 URL u = new URL(http://; + serverName + : + serverPort + 
 /manager + command);
 URLConnection con = u.openConnection();
 
 String up = USERNAME + : + PASSWORD;
 String encoding = null;
 // check to see if sun's Base64 encoder is available.
 try {
 sun.misc.BASE64Encoder encoder =
 (sun.misc.BASE64Encoder)
 
 Class.forName(sun.misc.BASE64Encoder).newInstance();
 encoding = encoder.encode(up.getBytes());
 } catch (Exception ex) { // sun's base64 encoder isn't available
 throw new TomcatManagerException(No 
 sun.misc.BASE64Encoder availoable in JDK!);
 }
 
 con.setRequestProperty(Authorization, Basic  + encoding);
 con.setDoInput(true);
 con.setUseCaches(false);
 con.connect();
 
 if (con instanceof HttpURLConnection) {
 HttpURLConnection httpConnection = (HttpURLConnection) con;
 // test for 401 result (HTTP only)
 if (httpConnection.getResponseCode() == 
 HttpURLConnection.HTTP_UNAUTHORIZED) {
 throw new TomcatManagerException(HTTP Authorization 
 failure!);
 }
 }
 
 BufferedReader reader = new BufferedReader(new 
 InputStreamReader(con.getInputStream()));
 
 // get first line
 // should be something like:
 // OK - some information text
 String line = null;
 
 line = reader.readLine();
 StringTokenizer tokenizer = new StringTokenizer(line, -);
 if (tokenizer.countTokens() == 2) {
 String rc = tokenizer.nextToken();
 String description = tokenizer.nextToken();
 result = new TomcatWebAppResult(rc, description);
 }
 
 while ((line = reader.readLine()) != null) {
 result.addWebAppDescriptor(line);
 }
 reader.close();
 
 } catch (IOException e) {
 throw new TomcatManagerException(Unable to perform command: 
 , e);
 }
 
 
 -
 To unsubscribe, e-mail: [EMAIL PROTECTED]
 For additional commands, e-mail: [EMAIL PROTECTED]
 


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: threads problem with many connections

2004-10-01 Thread Oleg Kalnichevski

On Fri, 2004-10-01 at 12:57, Guillaume Cottenceau wrote:
Hello,

We use HttpClient for performing several HTTP post in parallel in
our applications. We have a problem when the server(s) receiving
our HTTP post either answers very slowly, or goes mad and sends
garbage data over and over: the connection stays open forever,
but more important, the Java threads as well. We have situations
where we reach the maximum of Java threads our (tomcat)
application is configured to handle, and our whole application is
then unusable.

It seems that java.nio is capable of using only one thread for
several lowlevel (OS) socket connections, and is actually also
quite efficient.

Guillaume,

Please correct me if my understanding of the problem is incorrect, it is
Tomcat that runs out of connections, not HttpClient. In fact, there's
only one case when HttpClient spawns a new thread, and if so is desired,
this can be disabled. The thread management is entirely responsibility
of your application. One does not have to use one thread per socket
model even without NIO

I have seen that Oleg Kalnichevski has already expressed his
views several times on the subject, and I have seen that you want
to keep 1.2 compatibility, so java.nio out.

http://www.mail-archive.com/[EMAIL PROTECTED]/msg05551.html
http://www.mail-archive.com/[EMAIL PROTECTED]/msg06998.html

(btw, I don't agree with in my opinion there's absolutely
nothing that NIO can bring in in terms of performance to
client-side applications - well I agree that pure performance
is not the problem but threads and memory consumption surely is -
so in my opinion there is a lot to win with java.nio in
httpclient)

I have done quite a bit of NIO programming lately and respectfully stick
to my original opinion.

My question is, since you don't want to lose 1.2 compatibility
before 4.0, is there then a way to solve a typical too many
threads problem such as the one we have? Do you people never had
the same problem? Or have found a way to solve it?

Feasible approach is to have one monitor thread checking on the status
of active connections or/and processing incoming connections, and a
number of worker threads in a shared pool to do the actual work.

It seems the HTTP protocol doesn't have anything resembling a
global timeout for a given connection (e.g. after x seconds,
close the receiving channel even if server hasn't finished
sending), and thus normally httpclient doesn't provide such a
thing. Do you think this should be investigated/implemented in
some way?

I do not recall HTTP protocol as such dealing with timeouts in any way.
I may be wrong, though. This is certainly not a protocol concern.
HttpClient does provide so called socket timeout. When set to a positive
number, the connection will terminate with an IOException after the
specified period of inactivity:

http://jakarta.apache.org/commons/httpclient/apidocs/org/apache/commons/httpclient/HttpClient.html#setTimeout(int)

Hope this helps

Oleg

Thanks, and best regards.

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: threads problem with many connections

2004-10-01 Thread Oleg Kalnichevski


  Feasible approach is to have one monitor thread checking on the status
  of active connections or/and processing incoming connections, and a
  number of worker threads in a shared pool to do the actual work.
 
 Actually since probably most people using httpclient with many
 connections would need that, would it make sense to have it
 shared somewhere, for example in httpclient?
 

I assume you are developing some kind of HTTP proxy, since you have
HttpClient running inside a servlet container. While not being entirely
impossible (I have one serving a population of a large Swiss canton),
HttpClient 2.x and 3.x have not been specifically designed to be used in
such a manner. My guts tell me that it might involve quite a few nasty
hacks if to be made truly generic and reusable. In my option it is just
not worth the trouble and will not bear results generic enough to be
useful in most of scenarios.

We are planning to embark on a fundamental redesign of HttpClient in a
not so distant future (past 3.0b1 I'd say). Our intent is to turn
HttpClient into a generic HTTP toolkit which can be used to rapidly
assemble HTTP agents, proxies or embedded lightweight servers. 

At that point we will probably seriously consider using NIO and provide
some sort of thread management/connection management classes or use an
external library for that end.   

But first things first, and one thing at a time

 commons/httpclient/apidocs/org/apache/commons/httpclient/HttpClient.html#setTimeout(int)
  
 
 Yes, we're already using that (and connection timeouts as well).
 We're just in trouble when the post never finishes, but actually
 the server is very slow (if it sends one byte per second for
 example, but on a regular basis) or goes mad.

Basically what you are talking about is ability to abort execution of
HTTP methods. Consider using HttpClient 3.0a2 which provides such
capability. You'll have to add a little code to handle timing of method
execution, which should not be that difficult

Oleg

***
The information in this email is confidential and may be legally privileged.  Access 
to this email by anyone other than the intended addressee is unauthorized.  If you are 
not the intended recipient of this message, any review, disclosure, copying, 
distribution, retention, or any action taken or omitted to be taken in reliance on it 
is prohibited and may be unlawful.  If you are not the intended recipient, please 
reply to or forward a copy of this message to the sender and delete the message, any 
attachments, and any copies thereof from your system.
***

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: threads problem with many connections

2004-10-01 Thread Oleg Kalnichevski



 Is it considered safe to interrupt the execute task that way? Is
 method.releaseConnection() the way to go for full cleanup of
 underlying resources, or the interruption might leave things in a
 bad state?

There's no definitive answer to this question. TimeoutController does
not actually interrupt the method execution. All is does is leaving the
client object at the mercy of garbage collector. The client may linger
in the memory until the second coming or get GCed in a reasonable amount
of time. The exact behaviour is unfortunately JRE specific.

Oleg 

***
The information in this email is confidential and may be legally privileged.  Access 
to this email by anyone other than the intended addressee is unauthorized.  If you are 
not the intended recipient of this message, any review, disclosure, copying, 
distribution, retention, or any action taken or omitted to be taken in reliance on it 
is prohibited and may be unlawful.  If you are not the intended recipient, please 
reply to or forward a copy of this message to the sender and delete the message, any 
attachments, and any copies thereof from your system.
***

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: ATTN Open-source projects using HttpClient

2004-09-30 Thread Oleg Kalnichevski


Hi St.Ack,
Your feedback is really appreciated. I am quite happy that we now have
one less development list to spam ;)

See my comments inline

 The upgrade took way longer than I anticipated, a couple of days rather 
 than a couple of hours.  While some of the time was spent on refactoring 
 only slightly related to the httpcilent upgrade and testing to see all 
 httpclient used features still work post upgrade, the bulk of the time 
 was spent on redoing our auth system to fit the redesigned httpclient 
 auth system. I had trouble figuring out how things work now in the 
 absence of example. Our usage is a little out-of-the-ordinary in that we 
 manage own store of credentials and manage when to load them onto a 
 httpmethod.  Previous, HttpAuthenticator would select the scheme for 
 me.  Now it seems like I have to do it myself using AuthChallengeParser 
 and then iterate over the returns.  In general the new auth system 
 changes look to be for the best. 

I am not sure if that's going to be of help in your particular case, I
just want to note that one may replace the standard authentication
schemes with custom ones and provide additional custom schemes, if so is
desired:

http://jakarta.apache.org/commons/httpclient/3.0/authentication.html#Custom%20authentication%20scheme

 The IBM SSL socket timeout issues I'm seeing when I get an SSLSocket 
 with a timeout (I set the timeout by getting a socket with the null arg 
 constructor then doing an SSLSocket$connect with a timeout).  The 
 exceptions do not happen when I use SUN JVM 1.4.2.  These are probably 
 IBM JVM issues but I'll list them here anyways:
 
 1. The IBM JVM 141 (cxia321411-20030930) NPEs setting the NoTcpDelay.  
 Is anyone else seeing this?
  java.lang.NullPointerException
 at com.ibm.jsse.bf.setTcpNoDelay(Unknown Source)
 at 
 org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:683)
 at 
 org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1328)
 
 2. Using the IBM JVM 142, its saying SSL connection not open when we go 
 to use inputstreams.
  java.net.SocketException: Socket is not connected
 at java.net.Socket.getInputStream(Socket.java:726) at 
 com.ibm.jsse.bs.getInputStream(Unknown Source)
 at 
 org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:715)
 at 
 org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1328)
 

We have already had a few reports regarding IBM JSSE semantical
incompatibilities with Sun JSSE. It appears IBM JSSE implementation
unlike Sun's does not like attempts to set socket parameters when the
socket is closed. I believe it is clearly a bug in IBM JSSE but we can
think of working it around in HttpClient. 


 By way of feedback on the 3.0 API, I'll describe the two places where 
 the API is lacking regards our requirements forcing us to do yucky 
 overlays.  First some context.  The crawler must record the response 
 headers and response content exactly as it comes back over the wire and 
 its supposed to be tenacious.
 
 Regards recording exactly what the server sent us, we overlay 
 HttpConnection with a version that wraps the socket input and output 
 streams.  Here's the diff:
 
 +// HERITRIX import.
 +import org.archive.util.HttpRecorder;
 +
  /**
   * An abstraction of an HTTP [EMAIL PROTECTED] InputStream} and [EMAIL PROTECTED] 
 OutputStream}
   * pair, together with the relevant attributes.
 @@ -676,7 +679,6 @@
  highly interactive environments, such as some client/server
  situations. In such cases, nagling may be turned off through
  use of the TCP_NODELAY sockets option. */
 -
  socket.setTcpNoDelay(this.params.getTcpNoDelay());
  socket.setSoTimeout(this.params.getSoTimeout());
 
 @@ -701,8 +703,23 @@
  if (inbuffersize  2048) {
  inbuffersize = 2048;  }
 -inputStream = new 
 BufferedInputStream(socket.getInputStream(), inbuffersize);
 -outputStream = new 
 BufferedOutputStream(socket.getOutputStream(), outbuffersize);
 +// START HERITRIX Change
 +HttpRecorder httpRecorder = HttpRecorder.getHttpRecorder();
 +if (httpRecorder == null) {
 +inputStream = new BufferedInputStream(
 +socket.getInputStream(), inbuffersize);
 +outputStream = new BufferedOutputStream(
 +socket.getOutputStream(), outbuffersize);
 +} else {
 +inputStream = httpRecorder.inputWrap((InputStream)
 +(new BufferedInputStream(socket.getInputStream(),
 +inbuffersize)));
 +outputStream = httpRecorder.outputWrap((OutputStream)
 +(new BufferedOutputStream(socket.getOutputStream(),
 +

RE: HttpClient + HTTPS + NTLM Authentication = HTTP/1.1 401AccessDenied

2004-09-30 Thread Oleg Kalnichevski

:
 keystone.ibanksystems.com
 [\r][\n]
 2004/09/30 10:05:34:042 CDT [DEBUG] header -  Expect:
 100-continue[\r][\n]
 2004/09/30 10:05:34:042 CDT [DEBUG] header -  Content-Type:
 multipart/form-da
 ta; boundary=314159265358979323846[\r][\n]
 2004/09/30 10:05:34:042 CDT [DEBUG] header -  [\r][\n]
 2004/09/30 10:05:34:072 CDT [DEBUG] header -  HTTP/1.1 100
 Continue[\r][\n]
 2004/09/30 10:05:34:072 CDT [DEBUG] header -  Server:
 Microsoft-IIS/5.0[\r][\
 n]
 2004/09/30 10:05:34:072 CDT [DEBUG] header -  Date: Thu, 30 Sep 2004
 15:05:30
  GMT[\r][\n]
 2004/09/30 10:05:34:072 CDT [DEBUG] header -  IISExport: This web
 site was ex
 ported using IIS Export v3.0[\r][\n]
 2004/09/30 10:05:34:072 CDT [DEBUG] HttpMethodBase - OK to continue
 received
 2004/09/30 10:05:34:122 CDT [DEBUG] header -  HTTP/1.1 200
 OK[\r][\n]
 2004/09/30 10:05:34:122 CDT [DEBUG] header -  Server:
 Microsoft-IIS/5.0[\r][\
 n]
 2004/09/30 10:05:34:122 CDT [DEBUG] header -  Date: Thu, 30 Sep 2004
 15:05:30
  GMT[\r][\n]
 2004/09/30 10:05:34:122 CDT [DEBUG] header -  IISExport: This web
 site was ex
 ported using IIS Export v3.0[\r][\n]
 2004/09/30 10:05:34:122 CDT [DEBUG] header -  Content-Length:
 2878[\r][\n]
 2004/09/30 10:05:34:122 CDT [DEBUG] header -  Content-Type:
 text/html[\r][\n]
 
 2004/09/30 10:05:34:122 CDT [DEBUG] header -  Set-Cookie:
 ASPSESSIONIDACQABDQ
 S=OPDPAAPBNOGEOCJAHGNLNBKC; path=/[\r][\n]
 2004/09/30 10:05:34:122 CDT [DEBUG] header -  Cache-control:
 private[\r][\n]
 
 2004/09/30 10:05:34:142 CDT [DEBUG] HttpMethodBase - Cookie accepted:
 $Version=
 0; ASPSESSIONIDACQABDQS=OPDPAAPBNOGEOCJAHGNLNBKC; $Path=/
 Status Line:  HTTP/1.1 200 OK
 Status Code:  200
 2004/09/30 10:05:34:142 CDT [DEBUG] HttpMethodBase - Resorting to
 protocol versi
 on default close connection policy
 2004/09/30 10:05:34:142 CDT [DEBUG] HttpMethodBase - Should NOT close
 connection
 , using HTTP/1.1
 2004/09/30 10:05:34:142 CDT [DEBUG] HttpConnection - Releasing
 connection back t
 o connection manager.
 Press any key to continue . . .
 
 Please reply at your earliest convenience.
 
 Chris
 
 -Original Message-
 From: Oleg Kalnichevski [mailto:[EMAIL PROTECTED] 
 Sent: Wednesday, September 29, 2004 5:10 PM
 To: Commons HttpClient Project
 Subject: RE: HttpClient + HTTPS + NTLM Authentication = HTTP/1.1
 401AccessDenied
 
 Christopher,
 Ok, I see. This is weird. I can't explain it. Maybe I am just too tired
 right now and should go to bed.
 
 Actually it is preferred to not do a POST against a protected URL. One
 should do a GET or a HEAD first, get authenticated, get a session
 cookie, and than do a POST.
 
 Another thing to try is turning on 'expect: continue' handshake
 
 http://jakarta.apache.org/commons/httpclient/apidocs/org/apache/commons/
 httpclient/methods/ExpectContinueMethod.html#setUseExpectHeader(boolean)
 
 Oleg
 
 On Wed, 2004-09-29 at 23:59, Burke, Christopher wrote:
  Oleg,
  Thanks for your prompt response.  The main problem is that the file
 has
  not been uploaded, but the return code is 200.  I am trying to post
 the
  File object 'f' to the 'F1' textbox in the following form (File f =
 new
  File(C:/secureHttp/anotherLog.log);).  I believe my code is correct.
 
  
  I am at a loss.  What could be the problem?  
  
  FORM ENCTYPE=multipart/form-data METHOD=POST
  ACTION=siteman.asp?u=Dd=c:\im\
  FONT SIZE=1 FACE=Arial, Helvetica, sans-serifNAME OF DESTINATION
  FOLDER ON WEB SITE/FONTBR
  FONT SIZE=4 FACE=Arial, Helvetica,
 sans-serifBc:\im\/B/FONTP
  FONT SIZE=1 FACE=Arial, Helvetica, sans-serifPATHNAME OF LOCAL
  DOCUMENTBR(SEND THIS FILE TO THE WEB SERVER)/FONTBRINPUT
 SIZE=30
  TYPE=FILE NAME=F1P
  INPUT TYPE=SUBMIT VALUE=UPLOAD nbsp;
  INPUT TYPE=SUBMIT NAME=POSTACTION VALUE=CANCEL
  PFONT SIZE=2 FACE=Arial, Helvetica, sans-serifIf the
  B[BROWSE...]/B button is not displayed,
  BRyou must upgrade your A
 HREF=http://www.netscape.com;Netscape/A
  or A HREF=http://www.microsoft.com;Microsoft/A browser.
  /FORM/
  
  Thanks again for your help, Oleg.
  
  Christopher
  
  -Original Message-
  From: Oleg Kalnichevski [mailto:[EMAIL PROTECTED] 
  Sent: Wednesday, September 29, 2004 4:29 PM
  To: Commons HttpClient Project
  Subject: Re: HttpClient + HTTPS + NTLM Authentication = HTTP/1.1
  401Access Denied
  
  Christopher,
  What is exactly the problem? 
  
  The authentication succeeded:
  HTTP/1.1 200 OK
  
  Session cookie has been sent:
  ASPSESSIONIDAQQBDABR=LMNNMHNALPPKIBENMNNANHGP
  
  NTLM authentication scheme is a stateful one and requires multiple
  challenges/responses. The first 401 Access Denied response is
 perfectly
  OK. 
  
  For details see:
  http://davenport.sourceforge.net/ntlm.html
  WARNING: contains utter insanity ;-)
  
  Oleg
  
  On Wed, 2004-09-29 at 23:10, Burke, Christopher wrote:
   All,
   I need help implementing a Commons HttpClient solution to post files
  to
   a web server via an ASP page.  This seems somewhat straightforward,
  but
   I am having trouble with the NTLM

Re: ATTN Open-source projects using HttpClient

2004-09-30 Thread Oleg Kalnichevski

 That'd be grand if its possible (We like the IBM JVMs' speed and more 
 detailed thread dumps).  We used to subclass httpclient so we could do 
 the below, moving the setting of the timeout till after the open.  
 HttpClient 3.0 now sets timeout, etc., after the open seemingly so our 
 subclass is no longer necessary (Hurray!).

St.Ack,

Could you retest HttpClient 3.0a3 with IBM JRE and let us know if the
problem still persists?


 HttpRecorder duplicates all sent and received to files on disk.  It wraps the 
 (buffered) socket streams with input/output streams that do the duplication.  
 Subsequently, the file is fed to a set of processors to with as they wilt.  Link 
 extraction is main task performed by processors.
 
 We need to record what was sent over the wire preserving order and all bytes sent 
 back and forth (We're trying to archive the web).  If there's a less intrusive way 
 of getting what we need, we'd love to hear of it.
 


One possibility would be some thing like that:

public class HeritixMethod extends GetMethod {

  class HeritixConnectionWrapper extends HttpConnection {
 ...
  }

  public int execute(HttpState state, HttpConnection conn) 
throws HttpException, IOException {
return super.execute(state, new HeritixConnectionWrapper(conn));
  }
}

It's ugly but should prevent you from having to patch HttpClient. A
better solution would be a pluggable HttpConnection implementation. This
will have to wait until 4.0, though

Oleg


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: Problem with Preferences Architecture


Probably that's not the way it is supposed to be. I'll see what can be
done about it.

Oleg

On Wed, 2004-09-29 at 11:53, Ortwin Glck wrote:
 Vikram Goyal wrote:

  Hello,
 
  I am not sure if this is a bug or I am not running this right.
 
  I am trying to get the preferences architecture to modify the http.useragent
  property at the client level, and then retrieve the value at the method and
  host level. The value gets percolated down to the Method level but the Host
  level value does not change and gives the global value. See code below.
 
  As per the 3.0 B2 documentation, the Method and Host params should try and
  retrieve the value of the http.useragent from their local cache, and if it
  they do not find it, should go up the heirarchy, till they reach the global
  params. Since in the code below, the value is set at the Client level, one
  lower than the Global level, it should be retrieved from the Client level
  for both Method and Host params.

 Vikram, please ignore my first post.

 The problem is, that HttpClient creates a new HostConfiguration object
 internally and sets default on that. So your original object is not
 modified. So within HttpClient the host configuration will use the right
 user agent string. But you can not see this from the outside.

 This is line 389 in HttpClient.java (HEAD):
   HostConfiguration methodConfiguration = new
 HostConfiguration(hostConfiguration);

 Ortwin Glck

***
The information in this email is confidential and may be legally privileged.  Access 
to this email by anyone other than the intended addressee is unauthorized.  If you are 
not the intended recipient of this message, any review, disclosure, copying, 
distribution, retention, or any action taken or omitted to be taken in reliance on it 
is prohibited and may be unlawful.  If you are not the intended recipient, please 
reply to or forward a copy of this message to the sender and delete the message, any 
attachments, and any copies thereof from your system.
***

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: Problem with Preferences Architecture


Vikram,

I hope what is to follow will clarify things a little

(1)
===
HostConfiguration host = new HostConfiguration();
host.setHost(www.google.com);
===

These are the defaults that apply only when some host parameters have
not been explicitly defined by the method itself. Thus, they are not
supposed to mutate as a result of method execution

Consider the following:
===
HttpClient client = new HttpClient();
 
HostConfiguration hostconfig = new HostConfiguration();
hostconfig.setHost(www.mycompnany.com);

HttpMethod httpget = new GetMethod(http://www.anothercompnany.com/;);

client.executeMethod(hostconfig, httpget);
===

I assume you'd still want this method to be executed against
http://www.anothercompnany.com/ not http://www.mycompnany.com/, right?

So, if you want to know the exact host parameters applied during the
method execution, you should call

===

method.getHostConfguration().getParams().getParameter(http.useragent));
===

Please let me know if that sounds reasonable/unreasonable

(2)
All this said, I have to admit that current host configuration related
code is a horrible mess. I will provide a refactoring patch shortly

Hope this helps

Oleg


On Wed, 2004-09-29 at 11:14, Vikram Goyal wrote:
 Hello,
 
 I am not sure if this is a bug or I am not running this right.
 
 I am trying to get the preferences architecture to modify the http.useragent
 property at the client level, and then retrieve the value at the method and
 host level. The value gets percolated down to the Method level but the Host
 level value does not change and gives the global value. See code below.
 
 As per the 3.0 B2 documentation, the Method and Host params should try and
 retrieve the value of the http.useragent from their local cache, and if it
 they do not find it, should go up the heirarchy, till they reach the global
 params. Since in the code below, the value is set at the Client level, one
 lower than the Global level, it should be retrieved from the Client level
 for both Method and Host params.
 
 Regards,
 Vikram
 
 import org.apache.commons.httpclient.HttpClient;
 import org.apache.commons.httpclient.methods.GetMethod;
 import org.apache.commons.httpclient.HostConfiguration;
 
 public class HttpClientTest {
 
  public static void main(String args[]) throws Exception {
 
   HttpClient client = new HttpClient();
   client.getParams().setParameter(http.useragent, My Browser);  // set
 the value here
 
   HostConfiguration host = new HostConfiguration();
   host.setHost(www.google.com);
 
   GetMethod method = new GetMethod(/);
 
   int returnCode = client.executeMethod(host, method);
 
   System.err.println(User-Agent:  +
 host.getParams().getParameter(http.useragent));  // does not print My
 Browser
 
   System.err.println(User-Agent:  +
 method.getParams().getParameter(http.useragent)); // prints My Browser
 
   method.releaseConnection();
  }
 }
 
 
 
 -
 To unsubscribe, e-mail: [EMAIL PROTECTED]
 For additional commands, e-mail: [EMAIL PROTECTED]

***
The information in this email is confidential and may be legally privileged.  Access 
to this email by anyone other than the intended addressee is unauthorized.  If you are 
not the intended recipient of this message, any review, disclosure, copying, 
distribution, retention, or any action taken or omitted to be taken in reliance on it 
is prohibited and may be unlawful.  If you are not the intended recipient, please 
reply to or forward a copy of this message to the sender and delete the message, any 
attachments, and any copies thereof from your system.
***

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: Problem with Preferences Architecture



 I am testing the new Preferences Architecture before writing about it in a
 Book that I am working on. I have spent the whole of today looking at the
 source code but could not locate the problem, so it is bugging me now. It
 makes sense, the architecture I mean, but it is just not working right for
 the HostConfiguration.

Vikram,
That is why HttpClient 3.0 is still ALPHA. We are grateful to you for
experimenting with the preferences architecture and giving us the
feedback. I agree HostConfiguration stuff is ugly and am working on
fixing it right now

Oleg  

***
The information in this email is confidential and may be legally privileged.  Access 
to this email by anyone other than the intended addressee is unauthorized.  If you are 
not the intended recipient of this message, any review, disclosure, copying, 
distribution, retention, or any action taken or omitted to be taken in reliance on it 
is prohibited and may be unlawful.  If you are not the intended recipient, please 
reply to or forward a copy of this message to the sender and delete the message, any 
attachments, and any copies thereof from your system.
***

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: [VOTE] 2.0.2 release

+1

On Wed, 2004-09-29 at 14:19, Michael Becke wrote:
 I propose that we mark the latest code in CVS HTTPCLIENT_2_0 as 2.0.2  
 and proceed with a release.  Please vote as follows:
 
  
 --
   Vote:  HttpClient 2.0.2 release
   [x] +1 I am in favor of the release, and will help support it.
   [ ] +0 I am in favor of the release, but am unable to help support it.
   [ ] -0 I am not in favor of the release.
   [ ] -1 I am against this proposal (must include a reason).

  
 --
 
 
 -
 To unsubscribe, e-mail: [EMAIL PROTECTED]
 For additional commands, e-mail: [EMAIL PROTECTED]

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: Problem with Preferences Architecture


 Actually the new Preferences Architecture is quite neat and great if it all
 works. It is only the HostConfiguration that does not work and I would love
 to figure out why.

Vikram,
Basically it appears that (1) we assume it should work one way, (2)
whereas you assume it should work quite the other way around, (3) and on
top of that the Hostconfiguration code appears broken. ;-)

As soon as we all agree to how exactly things are supposed to work, I'll
fix the code and everything should go back to normal.

So, please bear with us for a while, and keep on giving us feedback.
I'll do my best to have the first take on the fix done by the end of the
day.

Cheers,

Oleg


 
 
 
 -
 To unsubscribe, e-mail: [EMAIL PROTECTED]
 For additional commands, e-mail: [EMAIL PROTECTED]
 


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: streaming responses

Bob,

There's no special magic involved. Make sure you use
HttpMethod#getResponseBodyAsStream, which will return the raw input
stream, and not its buffering counterparts
HttpMethod#getResponseBodyAsString and HttpMethod#getResponseBody.

You should not worry about chunking. HttpClient will decode chunked
input on fly, if necessary. Just grab the raw input stream and do
whatever response retrieval suits your application best.

Hope this helps

Oleg


On Wed, 2004-09-29 at 20:43, St Jacques, Robert wrote:
 Howdy,
 
 I've just started using HttpClient for testing.  The product that I am
 developing includes a software download feature that downloads (sometimes
 very large) files over the internet using HTTP.  The reason we use HTTP is
 that many of our customers are unwilling to open their proxies or firewalls
 to other protocols.
 
 In order to accommodate these downloads, we must either be able to stream
 the response to a GET, or we need to be able to insure that the response we
 get from the server will be chunked in such a way that the chunks are small
 enough to buffer.  I know that HttpClient supports chunking, but I am unable
 to 'force' our web server to chunk up the data.  Furthermore, the HTTP 1.1
 RFC indicates that the server does not need to guarantee that a response,
 even if chunked, will be small enough to conveniently buffer on the client.
 
 What is the best way to use HttpClient to retrieve large amounts of data (as
 large as 200 MB or more)?
 
 Thanks,
 Bob St. Jacques
 Member Research Technical Staff
 Xerox Corp.
 (585)231-8306
 


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

RE: streaming responses

Bob,

Buffering response body message is written ONLY inside
getResponseBody, which in its turn uses getResponseBodyAsStream. See for
yourself:

http://jakarta.apache.org/commons/httpclient/xref/org/apache/commons/httpclient/HttpMethodBase.html#681

It means what there's a bit of code in your application that calls
getResponseBody or getResponseBodyAsString methods, which causes the
buffering of the response body.

getResponseBodyAsStream does reconstruct the input stream in case the
response has been already buffered, but per default it returns the raw
input stream, when available. Feel free to examine the source:

http://jakarta.apache.org/commons/httpclient/xref/org/apache/commons/httpclient/HttpMethodBase.html#709

What would HttpClient be worth if it always buffered the response body?

I have no idea what led you to believe that processResponseHeaders
method buffers the response. The method is even called
processResponseHeaders not processResponseBody or something

Here's what processResponseBody method does:

(1) Gets the raw input stream from the connection
http://jakarta.apache.org/commons/httpclient/xref/org/apache/commons/httpclient/HttpMethodBase.html#2014

(2) If the chunk-encoding is used, wraps it with ChunkedInputStream. No
buffering
http://jakarta.apache.org/commons/httpclient/xref/org/apache/commons/httpclient/HttpMethodBase.html#2038

(3) If Content-Length header is used, wraps it with
ContentLengthInputStream, which basically ensures that content will not
be read past its length. No buffering
http://jakarta.apache.org/commons/httpclient/xref/org/apache/commons/httpclient/HttpMethodBase.html#2072

(4) If there's anything wrong with either Transfer-Encoding or
Content-Length header, it just leaves the raw input stream alone. No
buffering

(5) Finally it attaches a AutoCloseInputStream to the resultant input
stream, which is intended to close the underlying connection once the
entire response is consumed. No buffering.

(6) Keeps the resultant input stream.
http://jakarta.apache.org/commons/httpclient/xref/org/apache/commons/httpclient/HttpMethodBase.html#1979

(7) Now, if getResponseBodyAsStream is called, it will simply return the
input stream without any further manipulations:
http://jakarta.apache.org/commons/httpclient/xref/org/apache/commons/httpclient/HttpMethodBase.html#710

(8) End of story

Hope this clarifies things a bit

Oleg

On Wed, 2004-09-29 at 21:17, St Jacques, Robert wrote:
 Oleg,
 
 This is exactly what I tried to do, but the HttpClient seems to buffer the
 response anyway.  I poked through the code a bit, and did some searching on
 the mailing list, and this seems to be the case.  Also, when I run tests,
 the log output from the HttpClient seems to indicate that the request is
 being buffered before I even get an opportunity to call
 getResponseBodyAsStream.  I tried this on a request that is about 17 MB
 and just sat back and watched as the response was buffered; I ended up
 terminating the program after a minute or two without ever actually getting
 past the execute method call.  Here is the output from the logs:
 
   enter HttpMethodBase.processResponseHeaders(HttpState, HttpConnection)
   enter GetMethod.readResponseBody(HttpState, HttpConnection)
   enter HttpMethodBase.readResponseBody(HttpState, HttpConnection)
   enter HttpMethodBase.readResponseBody(HttpState, HttpConnection)
   enter HttpConnection.getResponseInputStream()
   Buffering response body
 lots of logs detailing the bytes read
 
 From here it looks like the processResponseHeaders method (which is called
 by the HttpMethodBase during an execute) automatically buffers the response.
 Is this true, or am I totally misreading the signs?
 
 Thanks,
 Bob
 
 -Original Message-
 From: Oleg Kalnichevski [mailto:[EMAIL PROTECTED] 
 Sent: Wednesday, September 29, 2004 3:04 PM
 To: Commons HttpClient Project
 Subject: Re: streaming responses
 
 
 Bob,
 
 There's no special magic involved. Make sure you use
 HttpMethod#getResponseBodyAsStream, which will return the raw input stream,
 and not its buffering counterparts HttpMethod#getResponseBodyAsString and
 HttpMethod#getResponseBody.
 
 You should not worry about chunking. HttpClient will decode chunked input on
 fly, if necessary. Just grab the raw input stream and do whatever response
 retrieval suits your application best.
 
 Hope this helps
 
 Oleg
 
 
 On Wed, 2004-09-29 at 20:43, St Jacques, Robert wrote:
  Howdy,
  
  I've just started using HttpClient for testing.  The product that I am 
  developing includes a software download feature that downloads 
  (sometimes very large) files over the internet using HTTP.  The reason 
  we use HTTP is that many of our customers are unwilling to open their 
  proxies or firewalls to other protocols.
  
  In order to accommodate these downloads, we must either be able to 
  stream the response to a GET, or we need to be able to insure that the 
  response we get from the server will be chunked

RE: streaming responses

Damn, I have almost beaten Leo Tolstoy's War and Peace and it turned
out just that ;-)

On Wed, 2004-09-29 at 22:01, St Jacques, Robert wrote:
 Sure enough, this is the case; I was calling getResponseBody deep in my
 code.  Thanks for helping me find it ;)
 
 -Original Message-
 From: Oleg Kalnichevski [mailto:[EMAIL PROTECTED] 
 Sent: Wednesday, September 29, 2004 3:56 PM
 To: Commons HttpClient Project
 Subject: RE: streaming responses
 
 
 Bob,
 
 Buffering response body message is written ONLY inside getResponseBody,
 which in its turn uses getResponseBodyAsStream. See for
 yourself:
 
 http://jakarta.apache.org/commons/httpclient/xref/org/apache/commons/httpcli
 ent/HttpMethodBase.html#681
 
 It means what there's a bit of code in your application that calls
 getResponseBody or getResponseBodyAsString methods, which causes the
 buffering of the response body.
 
 getResponseBodyAsStream does reconstruct the input stream in case the
 response has been already buffered, but per default it returns the raw input
 stream, when available. Feel free to examine the source:
 
 http://jakarta.apache.org/commons/httpclient/xref/org/apache/commons/httpcli
 ent/HttpMethodBase.html#709
 
 What would HttpClient be worth if it always buffered the response body?
 
 I have no idea what led you to believe that processResponseHeaders method
 buffers the response. The method is even called processResponseHeaders not
 processResponseBody or something
 
 Here's what processResponseBody method does:
 
 (1) Gets the raw input stream from the connection
 http://jakarta.apache.org/commons/httpclient/xref/org/apache/commons/httpcli
 ent/HttpMethodBase.html#2014
 
 (2) If the chunk-encoding is used, wraps it with ChunkedInputStream. No
 buffering
 http://jakarta.apache.org/commons/httpclient/xref/org/apache/commons/httpcli
 ent/HttpMethodBase.html#2038
 
 (3) If Content-Length header is used, wraps it with
 ContentLengthInputStream, which basically ensures that content will not be
 read past its length. No buffering
 http://jakarta.apache.org/commons/httpclient/xref/org/apache/commons/httpcli
 ent/HttpMethodBase.html#2072
 
 (4) If there's anything wrong with either Transfer-Encoding or
 Content-Length header, it just leaves the raw input stream alone. No
 buffering
 
 (5) Finally it attaches a AutoCloseInputStream to the resultant input
 stream, which is intended to close the underlying connection once the entire
 response is consumed. No buffering.
 
 (6) Keeps the resultant input stream.
 http://jakarta.apache.org/commons/httpclient/xref/org/apache/commons/httpcli
 ent/HttpMethodBase.html#1979
 
 (7) Now, if getResponseBodyAsStream is called, it will simply return the
 input stream without any further manipulations:
 http://jakarta.apache.org/commons/httpclient/xref/org/apache/commons/httpcli
 ent/HttpMethodBase.html#710
 
 (8) End of story
 
 Hope this clarifies things a bit
 
 Oleg
 
 On Wed, 2004-09-29 at 21:17, St Jacques, Robert wrote:
  Oleg,
  
  This is exactly what I tried to do, but the HttpClient seems to buffer 
  the response anyway.  I poked through the code a bit, and did some 
  searching on the mailing list, and this seems to be the case.  Also, 
  when I run tests, the log output from the HttpClient seems to indicate 
  that the request is being buffered before I even get an opportunity to 
  call getResponseBodyAsStream.  I tried this on a request that is 
  about 17 MB and just sat back and watched as the response was 
  buffered; I ended up terminating the program after a minute or two 
  without ever actually getting past the execute method call.  Here is 
  the output from the logs:
  
enter HttpMethodBase.processResponseHeaders(HttpState, HttpConnection)
enter GetMethod.readResponseBody(HttpState, HttpConnection)
enter HttpMethodBase.readResponseBody(HttpState, HttpConnection)
enter HttpMethodBase.readResponseBody(HttpState, HttpConnection)
enter HttpConnection.getResponseInputStream()
Buffering response body
  lots of logs detailing the bytes read
  
  From here it looks like the processResponseHeaders method (which is 
  called
  by the HttpMethodBase during an execute) automatically buffers the 
  response. Is this true, or am I totally misreading the signs?
  
  Thanks,
  Bob
  
  -Original Message-
  From: Oleg Kalnichevski [mailto:[EMAIL PROTECTED]
  Sent: Wednesday, September 29, 2004 3:04 PM
  To: Commons HttpClient Project
  Subject: Re: streaming responses
  
  
  Bob,
  
  There's no special magic involved. Make sure you use 
  HttpMethod#getResponseBodyAsStream, which will return the raw input 
  stream, and not its buffering counterparts 
  HttpMethod#getResponseBodyAsString and HttpMethod#getResponseBody.
  
  You should not worry about chunking. HttpClient will decode chunked 
  input on fly, if necessary. Just grab the raw input stream and do 
  whatever response retrieval suits your application best.
  
  Hope this helps
  
  Oleg
  
  
  On Wed

Re: ATTN Open-source projects using HttpClient

Eric,
The intention is to keep HttpClient 3.0 compile-compatible with
HttpClient 2.x (except for all that ugly stuff that has been deprecated
in 2.0). Feel free to submit a patch for the HttpException class.

Keep us posted on the progress

Cheers,

Oleg

On Wed, 2004-09-29 at 21:10, Eric Johnson wrote:
 Some feedback, finally.
 
 I attempted to compile the client tools of the slide project with the 
 HttpClient 3.0 alpha release.
 
 You'll probably not be surprised to hear that it didn't compile.  It 
 turns out that a few missing functions in the HttpException class cause 
 the compile failures.  I added those few functions in to my local copy, 
 and everything compiled.
 
 The code that I have that runs on top of those libraries also compiled 
 without any further changes.
 
 I didn't actually run any of the code, though.  I suspect that the 
 changes in exception handling would make any such endeavors more work 
 than I want to tackle right now.
 
 I think my next step is to ask that the client libraries of the Slide 
 project start using different version numbers from the server side 
 tools, and then suggest a 3.0 release of the slide client libraries that 
 will use the 3.0 HttpClient.  Since getting it to compile was easy, I 
 suspect that the rest of it will be fairly straightforward too.
 
 -Eric.
 
 Oleg Kalnichevski wrote:
 
 As far as I know the following projects rely on HttpClient 2.0 as a
 required or optional dependency
 
 * Apache Jakarta Slide (http://jakarta.apache.org/slide/)
 * Apache Jakarta Cactus (http://jakarta.apache.org/cactus/)
 * Apache Axis (http://ws.apache.org/axis/)
 * Apache XML-RPC (http://ws.apache.org/xmlrpc/)
 * Spring Framework (http://www.springframework.org/)
 * HtmlUntit (http://htmlunit.sourceforge.net/)
 * XINS (http://xins.sourceforge.net/)
 
 I just wonder if any of the projects' committers are currently
 subscribed to or monitor this mail list?
 
 We would love to know if there are any plans to evaluate HttpClient 3.0,
 or any migration plans already. What can we do to make the transition,
 if planned, easier?
 
 Oleg
 
 
 -
 To unsubscribe, e-mail: [EMAIL PROTECTED]
 For additional commands, e-mail: [EMAIL PROTECTED]
 
 
   
 
 
 -
 To unsubscribe, e-mail: [EMAIL PROTECTED]
 For additional commands, e-mail: [EMAIL PROTECTED]
 


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: Concurrent requests to same host


 Please clarify the following.
 
 1) Do I need to instantiate HttpClient only once? (using Singleton).
 

You should. There's nothing wrong with having multiple HttpClient
instances, but we generally recommend having only a single one


 2) Can I use MultiThreadedHttpConnectionManager in this scenario?
 Since I'm hitting only same host all the time, I do not know how to
 use this. Could you please give an example. (I want concurrent user
 requests executed in multi threading manner. )
 

You must. SimpleHttpConnectionManager used per default is NOT
thread-safe. Regardless how many hosts you intent to hit, you have to
use MultiThreadedHttpConnectionManager when accessing HttpClient from
multiple threads.

For details refer to the HttpClient multithreading guide:

http://jakarta.apache.org/commons/httpclient/threading.html

Hope this helps

Oleg


 Thank you, 
 Vijay
 
 -
 To unsubscribe, e-mail: [EMAIL PROTECTED]
 For additional commands, e-mail: [EMAIL PROTECTED]
 


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: HttpClient + HTTPS + NTLM Authentication = HTTP/1.1 401 Access Denied

Christopher,
What is exactly the problem? 

The authentication succeeded:
HTTP/1.1 200 OK

Session cookie has been sent:
ASPSESSIONIDAQQBDABR=LMNNMHNALPPKIBENMNNANHGP

NTLM authentication scheme is a stateful one and requires multiple
challenges/responses. The first 401 Access Denied response is perfectly
OK. 

For details see:
http://davenport.sourceforge.net/ntlm.html
WARNING: contains utter insanity ;-)

Oleg

On Wed, 2004-09-29 at 23:10, Burke, Christopher wrote:
 All,
 I need help implementing a Commons HttpClient solution to post files to
 a web server via an ASP page.  This seems somewhat straightforward, but
 I am having trouble with the NTLM authentication.
 
 Code Snippet:
 String url =
 https://keystone.ibanksystems.com/carlsontest/siteman.asp?u=Yd=c:\\im\
 \;
 
 NTCredentials creds = new
 NTCredentials(user,password,keystone.ibanksystems.com,domain);
 HttpClient client = new HttpClient();
 MultipartPostMethod mpPostMethod = new MultipartPostMethod(url);
 
 client.getState().setCredentials(null, null, creds);
 
 File f = new File(C:/secureHttp/anotherLog.log);
 
 //mpPostMethod.addParameter(F1,f.getName(),f);
 mpPostMethod.addParameter(F1,f);
 
 int statusCode = client.executeMethod(mpPostMethod);
 
 System.out.println(Status Line:   + mpPostMethod.getStatusLine());
 System.out.println(Status Code:   + statusCode);
 
 mpPostMethod.releaseConnection();
 
 Debug Output:
 2004/09/29 15:53:44:425 CDT [DEBUG] HttpClient - Java version: 1.4.2_05
 2004/09/29 15:53:44:425 CDT [DEBUG] HttpClient - Java vendor: Sun
 Microsystems I
 nc.
 2004/09/29 15:53:44:425 CDT [DEBUG] HttpClient - Java class path:
 .;..;C:\j2sdk1
 .4.2_05\bin;C:\apacheCommons\commons-httpclient.jar;C:\apacheCommons\com
 mons-log
 ging-api.jar;C:\apacheCommons\commons-logging.jar;C:\apacheCommons\commo
 ns-codec
 -1.3.jar
 2004/09/29 15:53:44:425 CDT [DEBUG] HttpClient - Operating system name:
 Windows
 XP
 2004/09/29 15:53:44:425 CDT [DEBUG] HttpClient - Operating system
 architecture:
 x86
 2004/09/29 15:53:44:425 CDT [DEBUG] HttpClient - Operating system
 version: 5.1
 2004/09/29 15:53:44:565 CDT [DEBUG] HttpClient - SUN 1.42: SUN (DSA
 key/paramete
 r generation; DSA signing; SHA-1, MD5 digests; SecureRandom; X.509
 certificates;
  JKS keystore; PKIX CertPathValidator; PKIX CertPathBuilder; LDAP,
 Collection Ce
 rtStores)
 2004/09/29 15:53:44:565 CDT [DEBUG] HttpClient - SunJSSE 1.42: Sun JSSE
 provider
 (implements RSA Signatures, PKCS12, SunX509 key/trust factories, SSLv3,
 TLSv1)
 2004/09/29 15:53:44:565 CDT [DEBUG] HttpClient - SunRsaSign 1.42: SUN's
 provider
  for RSA signatures
 2004/09/29 15:53:44:565 CDT [DEBUG] HttpClient - SunJCE 1.42: SunJCE
 Provider (i
 mplements DES, Triple DES, AES, Blowfish, PBE, Diffie-Hellman, HMAC-MD5,
 HMAC-SH
 A1)
 2004/09/29 15:53:44:565 CDT [DEBUG] HttpClient - SunJGSS 1.0: Sun
 (Kerberos v5)
 2004/09/29 15:53:45:857 CDT [DEBUG] HttpConnection -
 HttpConnection.setSoTimeout
 (0)
 2004/09/29 15:53:45:857 CDT [DEBUG] HttpMethodBase - Execute loop try 1
 2004/09/29 15:53:45:857 CDT [DEBUG] header -  POST
 /carlsontest/siteman.asp?u
 =Yd=c:\im\ HTTP/1.1[\r][\n]
 2004/09/29 15:53:45:857 CDT [DEBUG] HttpMethodBase - Adding Host request
 header
 2004/09/29 15:53:45:867 CDT [DEBUG] header -  User-Agent: Jakarta
 Commons-Htt
 pClient/2.0.1[\r][\n]
 2004/09/29 15:53:45:867 CDT [DEBUG] header -  Host:
 keystone.ibanksystems.com
 [\r][\n]
 2004/09/29 15:53:45:867 CDT [DEBUG] header -  Content-Length:
 965[\r][\n]
 2004/09/29 15:53:45:867 CDT [DEBUG] header -  Content-Type:
 multipart/form-da
 ta; boundary=314159265358979323846[\r][\n]
 2004/09/29 15:53:46:037 CDT [DEBUG] header -  [\r][\n]
 2004/09/29 15:53:46:107 CDT [DEBUG] header -  HTTP/1.1 401 Access
 Denied[\r][
 \n]
 2004/09/29 15:53:46:107 CDT [DEBUG] header -  Server:
 Microsoft-IIS/5.0[\r][\
 n]
 2004/09/29 15:53:46:107 CDT [DEBUG] header -  Date: Wed, 29 Sep 2004
 20:53:50
  GMT[\r][\n]
 2004/09/29 15:53:46:107 CDT [DEBUG] header -  WWW-Authenticate:
 Negotiate[\r]
 [\n]
 2004/09/29 15:53:46:107 CDT [DEBUG] header -  WWW-Authenticate:
 NTLM[\r][\n]
 
 2004/09/29 15:53:46:107 CDT [DEBUG] header -  Connection:
 close[\r][\n]
 2004/09/29 15:53:46:107 CDT [DEBUG] header -  Content-Length:
 4431[\r][\n]
 2004/09/29 15:53:46:107 CDT [DEBUG] header -  Content-Type:
 text/html[\r][\n]
 
 2004/09/29 15:53:46:107 CDT [DEBUG] HttpMethodBase - Authorization
 required
 2004/09/29 15:53:46:117 CDT [DEBUG] HttpAuthenticator - Authenticating
 with the
 default authentication realm at keystone.ibanksystems.com
 2004/09/29 15:53:46:117 CDT [DEBUG] HttpMethodBase -
 HttpMethodBase.execute(): S
 erver demanded authentication credentials, will try again.
 2004/09/29 15:53:46:127 CDT [DEBUG] HttpMethodBase - Should close
 connection in
 response to Connection: close
 
 2004/09/29 15:53:46:127 CDT [DEBUG] HttpMethodBase - Execute loop try 2
 2004/09/29 15:53:46:127 CDT [DEBUG] HttpMethodBase - Opening the
 connection.
 2004/09/29 15:53:46:167 CDT

RE: HttpClient + HTTPS + NTLM Authentication = HTTP/1.1 401Access Denied