Porting httpclient 3 API to 4.3.3
Hello, I'm porting some old code that's using httpclient 3 and I'm having some trouble porting the makeLenient() [0] method call. I've managed to identify the properties being set and I ported the ones I was able to but i'm stuck on setting these properties: http.protocol.unambiguous-statusline = false http.protocol.strict-transfer-encoding = false http.protocol.reject-head-body = false http.protocol.warn-extra-input = false http.protocol.status-line-garbage-limit = 0 I'll appreciate it if anyone can point me to the right direction on which elements of the fluent syntax API I should inspect in order to set the above properties. [0] https://hc.apache.org/httpclient-3.x/apidocs/org/apache/commons/httpclient/params/HttpMethodParams.html#makeLenient%28%29
Re: How to set single-cookie-header with the 4.3 API?
Thanks for the help! On Sun, May 11, 2014 at 5:20 PM, Oleg Kalnichevski wrote: > On Sat, 2014-05-10 at 10:49 +0300, d_k wrote: > > Thank you very much for the help! > > > > So to my understanding after reading the code for BrowserCompatSpec [0] > is > > that when using the "CookieSpecs.BROWSER_COMPATIBILITY" when building the > > cookie spec the single cookie header policy is already in place? > > > > > > Yes, this is correct. > > Oleg > > > > - > To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org > For additional commands, e-mail: httpclient-users-h...@hc.apache.org > >
Re: How to set single-cookie-header with the 4.3 API?
Thank you very much for the help!
So to my understanding after reading the code for BrowserCompatSpec [0] is
that when using the "CookieSpecs.BROWSER_COMPATIBILITY" when building the
cookie spec the single cookie header policy is already in place?
[0]
https://svn.apache.org/repos/asf/httpcomponents/httpclient/tags/4.3.3/httpclient/src/main/java/org/apache/http/impl/cookie/BrowserCompatSpec.java
On Tue, May 6, 2014 at 3:43 PM, wrote:
> Hi.
>
> From the Chapter 3 of "HttpClient Tutorial" [1] it seems that that can be
> achieved by registering a CookieSpecProvider [2], for example, and using
> BestMatchSpecFactory [3]:
>
>> Registry r = RegistryBuilder.
>> create()
>> // override default behaviour of BEST_MATCH (2nd arg of
>> BestMatchSpecFactory's constructor specifies "single cookie header")
>> .register(CookieSpecs.BEST_MATCH, new BestMatchSpecFactory(null,
>> true))
>> // or use "custom" cookie spec, register other cookie specs as
>> needed
>> .register("oneheader-best-match", new BestMatchSpecFactory(null,
>> true))
>> .build();
>>
>> RequestConfig requestConfig = RequestConfig.custom()
>> .setCookieSpec("oneheader-best-match")
>> .build();
>>
>> CloseableHttpClient httpclient = HttpClients.custom()
>> .setDefaultCookieSpecRegistry(r)
>> .setDefaultRequestConfig(requestConfig)
>> .build();
>>
> (Note: code _not_ tested)
>
> For reference, the default CookieSpecProvider registry created by
> HttpClientBuilder, as of version 4.3.3 [4], is:
>
>> cookieSpecRegistry = RegistryBuilder.create()
>> .register(CookieSpecs.BEST_MATCH, new BestMatchSpecFactory())
>> .register(CookieSpecs.STANDARD, new RFC2965SpecFactory())
>> .register(CookieSpecs.BROWSER_COMPATIBILITY, new
>> BrowserCompatSpecFactory())
>> .register(CookieSpecs.NETSCAPE, new NetscapeDraftSpecFactory())
>> .register(CookieSpecs.IGNORE_COOKIES, new IgnoreSpecFactory())
>> .register("rfc2109", new RFC2109SpecFactory())
>> .register("rfc2965", new RFC2965SpecFactory())
>> .build();
>>
>
> HTH.
> Best regards.
>
> [1] https://hc.apache.org/httpcomponents-client-4.3.x/
> tutorial/html/statemgmt.html#d5e557
> [2] https://hc.apache.org/httpcomponents-client-4.3.x/
> httpclient/apidocs/org/apache/http/cookie/CookieSpecProvider.html
> [3] https://hc.apache.org/httpcomponents-client-4.3.x/
> httpclient/apidocs/org/apache/http/impl/cookie/BestMatchSpec.html
> [4] https://svn.apache.org/repos/asf/httpcomponents/httpclient/
> tags/4.3.3/httpclient/src/main/java/org/apache/http/
> impl/client/HttpClientBuilder.java
>
>
> On 06/05/14 08:11, d_k wrote:
>
>> Hello,
>>
>> I'm porting httpclient 3 code to the 4.3 API and I was unable to find a
>> way
>> to set the single-cookie-header setting using the 4.3 API
>>
>> The original line states:
>> params.setBooleanParameter(HttpMethodParams.SINGLE_COOKIE_HEADER, true);
>>
>> And I was wondering how to write it with the 4.3 API.
>>
>> If it helps I think the 4.1.1 API is something like this:
>>
>> params.setBooleanParameter(CookieSpecPNames.SINGLE_COOKIE_HEADER, true);
>>
>>
>
> -
> To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
> For additional commands, e-mail: httpclient-users-h...@hc.apache.org
>
>
How to set single-cookie-header with the 4.3 API?
Hello, I'm porting httpclient 3 code to the 4.3 API and I was unable to find a way to set the single-cookie-header setting using the 4.3 API The original line states: params.setBooleanParameter(HttpMethodParams.SINGLE_COOKIE_HEADER, true); And I was wondering how to write it with the 4.3 API. If it helps I think the 4.1.1 API is something like this: params.setBooleanParameter(CookieSpecPNames.SINGLE_COOKIE_HEADER, true);
Re: Authentication with multiple realms.
Alright. Thank you very much for the help! Currently JIRA (https://issues.apache.org/jira/browse/HTTPCLIENT) is down for maintenance but i'll open an issue when its back online. On Tue, Mar 25, 2014 at 12:02 PM, Oleg Kalnichevski wrote: > On March 25, 2014 10:55:20 AM CET, d_k wrote: > >Thank you very much for the support. :-) > > > >So it appears that upgrading httpclient won't solve this issue? > > > > This problem can be solved in 4.4 only, so you will have to upgrade no > matter what. > > Oleg > > - > To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org > For additional commands, e-mail: httpclient-users-h...@hc.apache.org > >
Re: Authentication with multiple realms.
Thank you very much for the support. :-) So it appears that upgrading httpclient won't solve this issue? On Tue, Mar 25, 2014 at 11:27 AM, Oleg Kalnichevski wrote: > On Tue, 2014-03-25 at 09:29 +0200, d_k wrote: > > Its looks like the first GET is challenged and the credentials are > provided > > but when executing the second GET the authentication is requested for a > > different realm but because the AuthState already had credentials they > were > > used. > > > > The member targetAuthState in DefaultRequestDirector holds the old > > credentials and although the realm changed they were not invalidated. > > > > The problem is that auth caches presently do not take auth realm into > account, which is wrong. > > Please raise a JIRA for this defect. > > Oleg > > > > > On Mon, Mar 24, 2014 at 10:11 PM, Oleg Kalnichevski >wrote: > > > > > On Mon, 2014-03-24 at 18:38 +0200, d_k wrote: > > > > I'm upgrading a Nutch plugin that used httpclient 3.x and because > Nutch > > > was > > > > already dependent on httpclient 4.1.1 I decided to avoid adding a new > > > > dependency on httpclient 4.3 and use the existing dependency on 4.1.1 > > > > > > > > > > > > > > I can take a look at the wire log tomorrow but I would strongly > > > recommend upgrading regardless. HttpClient authentication code has > > > undergone a fairly major rewrite in the 4.2 release. Upgrading at least > > > to the latest release in the 4.2 series would probably be necessary > > > anyway. > > > > > > Oleg > > > > > > > > > > On Mon, Mar 24, 2014 at 6:00 PM, Oleg Kalnichevski > > > > wrote: > > > > > > > > > On Mon, 2014-03-24 at 17:54 +0200, d_k wrote: > > > > > > Hello. > > > > > > > > > > > > I'm using httpclient-4.1.1 and i'm trying to authenticate with > > > different > > > > > > users on different realms but it seems to fail. > > > > > > > > > > Before I even start looking into this, is there a reason you are > not > > > > > using a newer version of HttpClient (preferably 4.3.x)? > > > > > > > > > > Oleg > > > > > > > > > > > > > > > > > > > > > - > > > > > To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org > > > > > For additional commands, e-mail: > httpclient-users-h...@hc.apache.org > > > > > > > > > > > > > > > > > > > > > > - > > > To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org > > > For additional commands, e-mail: httpclient-users-h...@hc.apache.org > > > > > > > > > > - > To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org > For additional commands, e-mail: httpclient-users-h...@hc.apache.org > >
Re: Authentication with multiple realms.
Its looks like the first GET is challenged and the credentials are provided but when executing the second GET the authentication is requested for a different realm but because the AuthState already had credentials they were used. The member targetAuthState in DefaultRequestDirector holds the old credentials and although the realm changed they were not invalidated. On Mon, Mar 24, 2014 at 10:11 PM, Oleg Kalnichevski wrote: > On Mon, 2014-03-24 at 18:38 +0200, d_k wrote: > > I'm upgrading a Nutch plugin that used httpclient 3.x and because Nutch > was > > already dependent on httpclient 4.1.1 I decided to avoid adding a new > > dependency on httpclient 4.3 and use the existing dependency on 4.1.1 > > > > > > I can take a look at the wire log tomorrow but I would strongly > recommend upgrading regardless. HttpClient authentication code has > undergone a fairly major rewrite in the 4.2 release. Upgrading at least > to the latest release in the 4.2 series would probably be necessary > anyway. > > Oleg > > > > On Mon, Mar 24, 2014 at 6:00 PM, Oleg Kalnichevski > wrote: > > > > > On Mon, 2014-03-24 at 17:54 +0200, d_k wrote: > > > > Hello. > > > > > > > > I'm using httpclient-4.1.1 and i'm trying to authenticate with > different > > > > users on different realms but it seems to fail. > > > > > > Before I even start looking into this, is there a reason you are not > > > using a newer version of HttpClient (preferably 4.3.x)? > > > > > > Oleg > > > > > > > > > > > > - > > > To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org > > > For additional commands, e-mail: httpclient-users-h...@hc.apache.org > > > > > > > > > > - > To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org > For additional commands, e-mail: httpclient-users-h...@hc.apache.org > >
Re: Authentication with multiple realms.
I'm upgrading a Nutch plugin that used httpclient 3.x and because Nutch was already dependent on httpclient 4.1.1 I decided to avoid adding a new dependency on httpclient 4.3 and use the existing dependency on 4.1.1 On Mon, Mar 24, 2014 at 6:00 PM, Oleg Kalnichevski wrote: > On Mon, 2014-03-24 at 17:54 +0200, d_k wrote: > > Hello. > > > > I'm using httpclient-4.1.1 and i'm trying to authenticate with different > > users on different realms but it seems to fail. > > Before I even start looking into this, is there a reason you are not > using a newer version of HttpClient (preferably 4.3.x)? > > Oleg > > > > - > To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org > For additional commands, e-mail: httpclient-users-h...@hc.apache.org > >
Authentication with multiple realms.
Hello.
I'm using httpclient-4.1.1 and i'm trying to authenticate with different
users on different realms but it seems to fail.
i create several AuthScope objects with the same host and port and a
different realm for each one.
If I pass schema as null then httpclient will attempt to send the username
and password for every WWW-Authenticate header?
When the first request is asked to authenticate it is authenticated for
'realmx' with the user 'userx' and a password and the authentication is
successful and when I perform another request to the same host and port but
for a different realm the method getCredentials() of the authState in the
updateAuthState method in
org.apache.http.impl.client.DefaultRequestDirector returns the credentials
for 'userx' (line 1215) and because it is not null the authentication
fails, the credentials are set to null and the method handleResponse
returns null causing the method 'execute' to exit without authenticating.
Either the AuthState should be invalidated or am I missing something else?
What is the correct execution path for multiple request to the same host
for different realms?
This is the log, you can see the first request succeeds but the second one
fails.
2014-03-24 17:32:31,604 INFO main mortbay.log (Slf4jLog.java:info(67)) -
Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via
org.mortbay.log.Slf4jLog
2014-03-24 17:32:31,612 DEBUG main mortbay.log (Slf4jLog.java:debug(40)) -
Container Server@4685d9a4 + org.mortbay.jetty.servlet.Context@7d93c227{/,null}
as handler
2014-03-24 17:32:31,612 DEBUG main mortbay.log (Slf4jLog.java:debug(40)) -
Container SessionHandler@6fafced9 + ServletHandler@7f3541d7 as handler
2014-03-24 17:32:31,613 DEBUG main mortbay.log (Slf4jLog.java:debug(40)) -
Container SessionHandler@6fafced9 +
org.mortbay.jetty.servlet.HashSessionManager@175cb80 as sessionManager
2014-03-24 17:32:31,613 DEBUG main mortbay.log (Slf4jLog.java:debug(40)) -
Container org.mortbay.jetty.servlet.Context@7d93c227{/,null} +
SessionHandler@6fafced9 as handler
2014-03-24 17:32:31,622 DEBUG main mortbay.log (Slf4jLog.java:debug(40)) -
Checking Resource aliases
2014-03-24 17:32:31,637 DEBUG main mortbay.log (Slf4jLog.java:debug(40)) -
Container ServletHandler@7f3541d7 +
org.apache.jasper.servlet.JspServlet-2007433549 as servlet
2014-03-24 17:32:31,638 DEBUG main mortbay.log (Slf4jLog.java:debug(40)) -
Container ServletHandler@7f3541d7 +
(S=org.apache.jasper.servlet.JspServlet-2007433549,[*.jsp]) as
servletMapping
2014-03-24 17:32:31,647 DEBUG main mortbay.log (Slf4jLog.java:debug(40)) -
filterNameMap=null
2014-03-24 17:32:31,647 DEBUG main mortbay.log (Slf4jLog.java:debug(40)) -
pathFilters=null
2014-03-24 17:32:31,648 DEBUG main mortbay.log (Slf4jLog.java:debug(40)) -
servletFilterMap=null
2014-03-24 17:32:31,648 DEBUG main mortbay.log (Slf4jLog.java:debug(40)) -
servletPathMap={*.jsp=org.apache.jasper.servlet.JspServlet-2007433549}
2014-03-24 17:32:31,648 DEBUG main mortbay.log (Slf4jLog.java:debug(40)) -
servletNameMap={org.apache.jasper.servlet.JspServlet-2007433549=org.apache.jasper.servlet.JspServlet-2007433549}
2014-03-24 17:32:32,407 INFO main httpclient.Http
(HttpBase.java:logConf(289)) - http.proxy.host = null
2014-03-24 17:32:32,407 INFO main httpclient.Http
(HttpBase.java:logConf(290)) - http.proxy.port = 8080
2014-03-24 17:32:32,412 INFO main httpclient.Http
(HttpBase.java:logConf(291)) - http.timeout = 6
2014-03-24 17:32:32,412 INFO main httpclient.Http
(HttpBase.java:logConf(292)) - http.content.limit = 65536
2014-03-24 17:32:32,412 INFO main httpclient.Http
(HttpBase.java:logConf(293)) - http.agent = Nutch-Test/Nutch-2.2.1 (Nutch
protocol-httpclient test)
2014-03-24 17:32:32,412 INFO main httpclient.Http
(HttpBase.java:logConf(294)) - http.accept.language =
en-us,en-gb,en;q=0.7,*;q=0.3
2014-03-24 17:32:32,413 INFO main httpclient.Http
(HttpBase.java:logConf(295)) - http.accept =
text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
2014-03-24 17:32:32,431 TRACE main httpclient.Http
(Http.java:setCredentials(383)) - Credentials - username: userx; set as
default for realm: ; scheme:
2014-03-24 17:32:32,432 ERROR main httpclient.Http (Http.java:setConf(174))
- Could not read httpclient-auth-test.xml : null
2014-03-24 17:32:32,486 DEBUG main mortbay.log (Slf4jLog.java:debug(40)) -
Container Server@4685d9a4 + SelectChannelConnector@127.0.0.1:47500 as
connector
2014-03-24 17:32:32,490 INFO main mortbay.log (Slf4jLog.java:info(67)) -
jetty-6.1.26
2014-03-24 17:32:32,514 DEBUG main mortbay.log (Slf4jLog.java:debug(40)) -
Container Server@4685d9a4 + org.mortbay.thread.QueuedThreadPool@50f2fd20 as
threadpool
2014-03-24 17:32:32,518 DEBUG main mortbay.log (Slf4jLog.java:debug(40)) -
started org.mortbay.thread.QueuedThreadPool@50f2fd20
2014-03-24 17:32:32,547 DEBUG main mortbay.log (Slf4jLog.java:debug(40)) -
Container Server@4685d9a4 +
org.mortbay.jetty.servlet.HashSessionIdManager@7b7597b6 as sessionIdManager
2014-03-24 17:32:32
maintaining authentication state between multiple threads/hosts
I'm using HttpClient 4.1.1 and i'm performing multiple requests to several web servers (using nutch 2.x) that requires authentication. I've read that I can save the authentication state between calls by passing the same HttpContext but that I shouldn't reuse the same HttpContext between different threads. So in effect i'll have to authenticate from every thread. i can use the SyncBasicHttpContext but without knowing exactly how nutch is using it I won't be able to know if its indeed safe to use it or not. Is there a chance that the HttpClient itself is using the SyncBasicHttpContext in an unsafe manner? To my understanding Nutch reuses the HttpClient between all threads but to maintain the authentication state for multiple hosts I will have to maintain a map between a host and its HttpContext. Is there a similar capability built into HttpClient that can do that or do I need to use my own Map? A simple Map between a host and an HttpContext is enough?
