I've committed the retry patch to the java adapter (rev 15983). Thx for
the patch!
Bart
On 12-1-2009 14:21, Wouter Zelle wrote:
I've attached two patches. The wdbp one does an instance-of. The
webdav-client patch doesn't, because the method is new there.
Regards,
Wouter
On Mon, Jan 12, 2009 at 1:06 PM, Bart van der Schans
<[email protected]>wrote:
> On 12-1-2009 12:42, Wouter Zelle wrote:
>
>> I've had similar problems. I've never been able to copy a considerable
>> part
>> of a repository using the repositorycopy or repository-datacopy tools. I
>> always get HttpRecoverableExceptions. They do not consistently happen to
>> the
>> same files, so it seems that it is a non-deterministic issue (probably
>> timing-related). Instead of trying to fix the actual issue, I decided to
>> try
>> and fix the symptoms by adding retries to hippo-tools/hippo-wdbp. In
>> WebdavBatchProcessor there is an executeMethod(HttpMethod method) that
>> performs the actual httpclient call. I added 3 retries like so:
>>
>> /**
>> * Retries up to three times
>> *
>> * @return httpclient response code
>> */
>> public int executeMethod(HttpMethodBase method) throws IOException {
>> DefaultMethodRetryHandler retryHandler = new
>> DefaultMethodRetryHandler();
>> retryHandler.setRetryCount(3);
>> method.setMethodRetryHandler(retryHandler);
>>
>> return m_httpClient.executeMethod(method);
>> }
>>
>> This change works very well for me. In one single repositorycopy run I
>> copied 6358 files between different repositories, where I had 22
>> recoverable
>> exceptions, which all succeeded on a retry. I also did many other runs
>> that
>> went well. So I strongly suggest that this change gets adopted by Hippo.
>> Even if the underlying bug is fixed, there may still be network hiccups
>> that
>> preferably should not stop a multi-hour copy. Please note that I did
>> change
>> the method API to HttpMethodBase instead of HttpMethod. This allows a
>> guarantee that the supplied method will be retried. An alternative is to
>> keep the same API and perform an instanceof check & cast inside the
>> method.
>>
> I agree. This makes the java adapter more fault resilient. Changing
the api
> is probably going to be a problem, so I think we should go for a
instanceof
> check.
>
>
>> The same fix can be applied to the different WebdavMethods in
>> hippo-repository-java-adapter to fix Nick's issue. The easiest fix is to
>> put
>> the method below in WebdavMethod:
>>
>> /**
>> * Retries up to three times
>> *
>> * @return httpclient response code
>> */
>> protected int executeMethod(HttpClient httpclient, HttpMethodBase
method)
>> throws IOException {
>> DefaultMethodRetryHandler retryHandler = new
>> DefaultMethodRetryHandler();
>> retryHandler.setRetryCount(3);
>> method.setMethodRetryHandler(retryHandler);
>>
>> return httpclient.executeMethod(method);
>> }
>>
>> Then the subclasses of WebdavMethod should call this method instead of
>> client.executeMethod(method).
>>
> I've to look into this on how to do this exactly. It would be great
if you
> can create a patch.
>
> Regards,
> Bart
>
>
>
>
>> Regards,
>>
>> Wouter
>>
>> On Wed, Nov 5, 2008 at 6:15 PM, Nick Stolwijk
>> <[email protected]>wrote:
>>
>> > Hello,
>> >
>> > When we put a little load on our application, which writes XML files
>> > into the repository, now and then a ClientException occurs:
>> >
>> > Caused by: nl.hippo.client.api.ClientException: Exception while
>> > executing PUT method.:
>> > org.apache.commons.httpclient.HttpRecoverableException: Error in
>> > parsing the status line from the response: unable to find line
>> > starting with "HTTP"
>> > at nl.hippo.client.webdav.method.Put$1.doExec(Put.java:60)
>> > at
>> >
>> nl.hippo.client.webdav.method.WebdavMethod.execute(WebdavMethod.java:62)
>> > at nl.hippo.client.webdav.method.Put.execute(Put.java:45)
>> > at
>> >
>>
>>
nl.hippo.client.webdav.service.WebdavServiceImpl.executePut(WebdavServiceImpl.java:151)
>> > at
>> >
>>
>>
com.example.service.Store2RepoServiceImpl.saveFile(Store2RepoServiceImpl.java:107)
>> > ... 43 more
>> > Caused by: org.apache.commons.httpclient.HttpRecoverableException:
>> > org.apache.commons.httpclient.HttpRecoverableException: Error in
>> > parsing the status line from the response: unable to find line
>> > starting with "HTTP"
>> > at
>> >
>>
>>
org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodBase.java:1962)
>> > at
>> >
>>
>>
org.apache.commons.httpclient.HttpMethodBase.processRequest(HttpMethodBase.java:2653)
>> > at
>> >
>>
>>
org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1087)
>> > at
>> >
>>
>>
org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:643)
>> > at
>> >
>>
>>
org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:497)
>> > at nl.hippo.client.webdav.method.Put$1.doExec(Put.java:56)
>> > ... 47 more
>> >
>> > Is this a known exception? I am not able to reproduce it clearly with
>> > a subset of requests.
>> >
>> > When we're doing a bulk update, we do two different updates on one
>> > file. Could it be an issue with putting the same file multiple times
>> > very soon after each other?
>> >
>> > With regards,
>> >
>> > Nick Stolwijk
>> > ~Java Developer~
>> >
>> > Iprofs BV.
>> > Claus Sluterweg 125
>> > 2012 WS Haarlem
>> > www.iprofs.nl
>> > ********************************************
>> > Hippocms-dev: Hippo CMS development public mailinglist
>> >
>> > Searchable archives can be found at:
>> > MarkMail: http://hippocms-dev.markmail.org
>> > Nabble: http://www.nabble.com/Hippo-CMS-f26633.html
>> >
>> >
>>
>>
>> --
>> Met vriendelijke groet,
>>
>> Wouter Zelle
>> ********************************************
>> Hippocms-dev: Hippo CMS development public mailinglist
>>
>> Searchable archives can be found at:
>> MarkMail: http://hippocms-dev.markmail.org
>> Nabble: http://www.nabble.com/Hippo-CMS-f26633.html
>>
>>
>
> --
> Hippo B.V. - Amsterdam
> Oosteinde 11, 1017 WT, Amsterdam, +31(0)20-5224466
>
> Hippo USA Inc. - San Francisco
> 101 H Street, Suite Q, Petaluma CA, 94952-3329, +1 (707) 773-4646
> -----------------------------------------------------------------
> http://www.onehippo.com - [email protected]
> -----------------------------------------------------------------
>
> ********************************************
> Hippocms-dev: Hippo CMS development public mailinglist
>
> Searchable archives can be found at:
> MarkMail: http://hippocms-dev.markmail.org
> Nabble: http://www.nabble.com/Hippo-CMS-f26633.html
>
>
--
Met vriendelijke groet,
Wouter Zelle
--
Hippo B.V. - Amsterdam
Oosteinde 11, 1017 WT, Amsterdam, +31(0)20-5224466
Hippo USA Inc. - San Francisco
101 H Street, Suite Q, Petaluma CA, 94952-3329, +1 (707) 773-4646
-----------------------------------------------------------------
http://www.onehippo.com - [email protected]
-----------------------------------------------------------------
********************************************
Hippocms-dev: Hippo CMS development public mailinglist
Searchable archives can be found at:
MarkMail: http://hippocms-dev.markmail.org
Nabble: http://www.nabble.com/Hippo-CMS-f26633.html
