Thanks,
I've done some extended testing and unfortunately it seems that my previous
fix doesn't fix the "error in parsing the status line from the response:
unable to find line starting with "HTTP" issue (that Nick encountered).
Httpclient doesn't actually retry after that supposedly recoverable
exception. So I came up with this more elaborate fix:
public int executeMethod(HttpMethod method) throws IOException
{
final int maxRetries = 3;
int retryCount = maxRetries;
while (true) {
if (retryCount != maxRetries) {
m_errorLogWriter.println("Retrying");
}
try {
return m_httpClient.executeMethod(method); //Exit point for
a correct call
} catch (HttpRecoverableException e) {
m_errorLogWriter.println("Recoverable error while executing
webdav call: " +
e.getMessage() + " - " + e.getReasonCode() + ": " +
e.getReason());
retryCount--;
if (retryCount == 0) {
m_errorLogWriter.println("3 retries failed, aborting.");
throw e; //Exit point for
failure after 3 retries
}
}
}
}
I've tested this one and it works better.
Regards,
Wouter
On Fri, Jan 16, 2009 at 2:55 PM, Bart van der Schans <[email protected]>wrote:
> I've committed the retry patch to the java adapter (rev 15983). Thx for the
> patch!
>
> Bart
>
>
>
> On 12-1-2009 14:21, Wouter Zelle wrote:
>
>> I've attached two patches. The wdbp one does an instance-of. The
>> webdav-client patch doesn't, because the method is new there.
>>
>> Regards,
>>
>> Wouter
>>
>> On Mon, Jan 12, 2009 at 1:06 PM, Bart van der Schans
>> <[email protected]>wrote:
>>
>> > On 12-1-2009 12:42, Wouter Zelle wrote:
>> >
>> >> I've had similar problems. I've never been able to copy a considerable
>> >> part
>> >> of a repository using the repositorycopy or repository-datacopy tools.
>> I
>> >> always get HttpRecoverableExceptions. They do not consistently happen
>> to
>> >> the
>> >> same files, so it seems that it is a non-deterministic issue (probably
>> >> timing-related). Instead of trying to fix the actual issue, I decided
>> to
>> >> try
>> >> and fix the symptoms by adding retries to hippo-tools/hippo-wdbp. In
>> >> WebdavBatchProcessor there is an executeMethod(HttpMethod method) that
>> >> performs the actual httpclient call. I added 3 retries like so:
>> >>
>> >> /**
>> >> * Retries up to three times
>> >> *
>> >> * @return httpclient response code
>> >> */
>> >> public int executeMethod(HttpMethodBase method) throws IOException {
>> >> DefaultMethodRetryHandler retryHandler = new
>> >> DefaultMethodRetryHandler();
>> >> retryHandler.setRetryCount(3);
>> >> method.setMethodRetryHandler(retryHandler);
>> >>
>> >> return m_httpClient.executeMethod(method);
>> >> }
>> >>
>> >> This change works very well for me. In one single repositorycopy run I
>> >> copied 6358 files between different repositories, where I had 22
>> >> recoverable
>> >> exceptions, which all succeeded on a retry. I also did many other runs
>> >> that
>> >> went well. So I strongly suggest that this change gets adopted by
>> Hippo.
>> >> Even if the underlying bug is fixed, there may still be network
>> hiccups
>> >> that
>> >> preferably should not stop a multi-hour copy. Please note that I did
>> >> change
>> >> the method API to HttpMethodBase instead of HttpMethod. This allows a
>> >> guarantee that the supplied method will be retried. An alternative is
>> to
>> >> keep the same API and perform an instanceof check & cast inside the
>> >> method.
>> >>
>> > I agree. This makes the java adapter more fault resilient. Changing
>> the api
>> > is probably going to be a problem, so I think we should go for a
>> instanceof
>> > check.
>> >
>> >
>> >> The same fix can be applied to the different WebdavMethods in
>> >> hippo-repository-java-adapter to fix Nick's issue. The easiest fix is
>> to
>> >> put
>> >> the method below in WebdavMethod:
>> >>
>> >> /**
>> >> * Retries up to three times
>> >> *
>> >> * @return httpclient response code
>> >> */
>> >> protected int executeMethod(HttpClient httpclient, HttpMethodBase
>> method)
>> >> throws IOException {
>> >> DefaultMethodRetryHandler retryHandler = new
>> >> DefaultMethodRetryHandler();
>> >> retryHandler.setRetryCount(3);
>> >> method.setMethodRetryHandler(retryHandler);
>> >>
>> >> return httpclient.executeMethod(method);
>> >> }
>> >>
>> >> Then the subclasses of WebdavMethod should call this method instead of
>> >> client.executeMethod(method).
>> >>
>> > I've to look into this on how to do this exactly. It would be great
>> if you
>> > can create a patch.
>> >
>> > Regards,
>> > Bart
>> >
>> >
>> >
>> >
>> >> Regards,
>> >>
>> >> Wouter
>> >>
>> >> On Wed, Nov 5, 2008 at 6:15 PM, Nick Stolwijk
>> >> <[email protected]>wrote:
>> >>
>> >> > Hello,
>> >> >
>> >> > When we put a little load on our application, which writes XML files
>> >> > into the repository, now and then a ClientException occurs:
>> >> >
>> >> > Caused by: nl.hippo.client.api.ClientException: Exception while
>> >> > executing PUT method.:
>> >> > org.apache.commons.httpclient.HttpRecoverableException: Error in
>> >> > parsing the status line from the response: unable to find line
>> >> > starting with "HTTP"
>> >> > at nl.hippo.client.webdav.method.Put$1.doExec(Put.java:60)
>> >> > at
>> >> >
>> >>
>> nl.hippo.client.webdav.method.WebdavMethod.execute(WebdavMethod.java:62)
>> >> > at nl.hippo.client.webdav.method.Put.execute(Put.java:45)
>> >> > at
>> >> >
>> >>
>> >>
>>
>> nl.hippo.client.webdav.service.WebdavServiceImpl.executePut(WebdavServiceImpl.java:151)
>> >> > at
>> >> >
>> >>
>> >>
>>
>> com.example.service.Store2RepoServiceImpl.saveFile(Store2RepoServiceImpl.java:107)
>> >> > ... 43 more
>> >> > Caused by: org.apache.commons.httpclient.HttpRecoverableException:
>> >> > org.apache.commons.httpclient.HttpRecoverableException: Error in
>> >> > parsing the status line from the response: unable to find line
>> >> > starting with "HTTP"
>> >> > at
>> >> >
>> >>
>> >>
>>
>> org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodBase.java:1962)
>> >> > at
>> >> >
>> >>
>> >>
>>
>> org.apache.commons.httpclient.HttpMethodBase.processRequest(HttpMethodBase.java:2653)
>> >> > at
>> >> >
>> >>
>> >>
>>
>> org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1087)
>> >> > at
>> >> >
>> >>
>> >>
>>
>> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:643)
>> >> > at
>> >> >
>> >>
>> >>
>>
>> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:497)
>> >> > at nl.hippo.client.webdav.method.Put$1.doExec(Put.java:56)
>> >> > ... 47 more
>> >> >
>> >> > Is this a known exception? I am not able to reproduce it clearly
>> with
>> >> > a subset of requests.
>> >> >
>> >> > When we're doing a bulk update, we do two different updates on one
>> >> > file. Could it be an issue with putting the same file multiple times
>> >> > very soon after each other?
>> >> >
>> >> > With regards,
>> >> >
>> >> > Nick Stolwijk
>> >> > ~Java Developer~
>> >> >
>> >> > Iprofs BV.
>> >> > Claus Sluterweg 125
>> >> > 2012 WS Haarlem
>> >> > www.iprofs.nl
>> >> > ********************************************
>> >> > Hippocms-dev: Hippo CMS development public mailinglist
>> >> >
>> >> > Searchable archives can be found at:
>> >> > MarkMail: http://hippocms-dev.markmail.org
>> >> > Nabble: http://www.nabble.com/Hippo-CMS-f26633.html
>> >> >
>> >> >
>> >>
>> >>
>> >> --
>> >> Met vriendelijke groet,
>> >>
>> >> Wouter Zelle
>> >> ********************************************
>> >> Hippocms-dev: Hippo CMS development public mailinglist
>> >>
>> >> Searchable archives can be found at:
>> >> MarkMail: http://hippocms-dev.markmail.org
>> >> Nabble: http://www.nabble.com/Hippo-CMS-f26633.html
>> >>
>> >>
>> >
>> > --
>> > Hippo B.V. - Amsterdam
>> > Oosteinde 11, 1017 WT, Amsterdam, +31(0)20-5224466
>> >
>> > Hippo USA Inc. - San Francisco
>> > 101 H Street, Suite Q, Petaluma CA, 94952-3329, +1 (707) 773-4646
>> > -----------------------------------------------------------------
>> > http://www.onehippo.com - [email protected]
>> > -----------------------------------------------------------------
>> >
>> > ********************************************
>> > Hippocms-dev: Hippo CMS development public mailinglist
>> >
>> > Searchable archives can be found at:
>> > MarkMail: http://hippocms-dev.markmail.org
>> > Nabble: http://www.nabble.com/Hippo-CMS-f26633.html
>> >
>> >
>>
>>
>> --
>> Met vriendelijke groet,
>>
>> Wouter Zelle
>>
>>
>
> --
> Hippo B.V. - Amsterdam
> Oosteinde 11, 1017 WT, Amsterdam, +31(0)20-5224466
>
> Hippo USA Inc. - San Francisco
> 101 H Street, Suite Q, Petaluma CA, 94952-3329, +1 (707) 773-4646
> -----------------------------------------------------------------
> http://www.onehippo.com - [email protected]
> -----------------------------------------------------------------
> ********************************************
> Hippocms-dev: Hippo CMS development public mailinglist
>
> Searchable archives can be found at:
> MarkMail: http://hippocms-dev.markmail.org
> Nabble: http://www.nabble.com/Hippo-CMS-f26633.html
>
>
--
Met vriendelijke groet,
Wouter Zelle
********************************************
Hippocms-dev: Hippo CMS development public mailinglist
Searchable archives can be found at:
MarkMail: http://hippocms-dev.markmail.org
Nabble: http://www.nabble.com/Hippo-CMS-f26633.html
