RE: svn 1.8.9 checkout crashes (debugging details provided)
On 2014-05-28 05:47, Bert Huijben wrote:
I'm guessing these patches (r2361 fixes a stupid compilation error
introduced in r2360) fix the problem. They fix the segfault I could
reproduce while trying here.
I'm having a cold right now, which is still affecting my coding
quality, so
I would like some confirmation from others before calling this problem
completely fixed.
For testing my case, I did this in outgoing.c using your macro and have
done a big bunch of test runs:
/*/
if (SERF_BUCKET_READ_ERROR(status)) {
if (APR_STATUS_IMPLIES_HANGUP(status)) {
fprintf(stderr, "crash ahead (status: %ld)\n", (long) status);
getchar();
}
/* Report the request as 'died'/'cancelled' to the application */
(void)(*request->handler)(request,
/*/
Whenever the macro triggers, I get a segfault (or, to put it another
way,
whenever I don't get a segfault, the macro doesn't trigger). So it's
stable for me in at least one direction -- I don't get any false
positives.
The only thing that may be a problem is a segfault without the macro
triggering -- a false negative. I say this because I (stupidly?)
believed
that FreeBSD's support for the POLLHUP poll() event meant I didn't
have to check APR_STATUS_IS_ECONNRESET(status) or
APR_STATUS_IS_ECONNABORTED(status), so my first instance of debug
code had only
if (status == SERF_ERROR_REQUEST_LOST) { ... }
With this, there was one and only one case in which I got a segfault
without the test triggering. This segfault occurred at the same code
point as the other segfaults, but I didn't get to check 'status'. It's
possible it was ECONNRESET or ECONNABORTED (maybe related to some
non-poll event), but I can't be sure.
Since adding the new debug code, every segfault has been associated with
SERF_ERROR_REQUEST_LOST (120102 or whatever it is), so at least svn
is usable now in my case.
I'll get the FreeBSD users in this forum thread to test your fix:
http://forums.freebsd.org/viewtopic.php?f=43&t=46620
If it seems to do the trick, I can send it to the freebsd-ports mailing
list and the port maintainer for wider testing. I've noticed the trunk
version of outgoing.c diverges a fair bit from the 1.3.5 version, but
from what I can tell, a 1.3.5 patch based on your fix ought to keep
1.3.5 afloat until 1.3.6 comes along.
You've been awesome! Hope you get well soon.
RE: svn 1.8.9 checkout crashes (debugging details provided)
> -Original Message- > From: dereck...@openmailbox.org [mailto:dereck...@openmailbox.org] > Sent: woensdag 28 mei 2014 13:10 > To: Bert Huijben > Cc: users@subversion.apache.org > Subject: RE: svn 1.8.9 checkout crashes (debugging details provided) > > On 2014-05-26 05:39, Bert Huijben wrote: > > Your testscase works more stable than my original testcase. > > > > It looks like the problem is somehow triggered by upgrading from serf > > 1.3.4 > > to 1.3.5. I'm trying to reach out to the serf developers to see how we > > should fix this problem. > > > > I think you can work around this problem by using serf 1.3.4, or by > > increasing the maximum number of requests per connection on the server. > > > > Bert > > Hey, Bert. Just caught this follow-up message of yours. > > A few users on the FreeBSD forums are getting bus errors / segfaults as > well, > and I was trying to gather more testcases, but it looks as if you've > been > able to reproduce the crashes and have identified the problem. > > Are your latest r2360/r2361 commits on serf related to this? The r2360 > commit message mentions a requests-per-connection limit. I'm guessing these patches (r2361 fixes a stupid compilation error introduced in r2360) fix the problem. They fix the segfault I could reproduce while trying here. I'm having a cold right now, which is still affecting my coding quality, so I would like some confirmation from others before calling this problem completely fixed. Bert
RE: svn 1.8.9 checkout crashes (debugging details provided)
On 2014-05-26 05:39, Bert Huijben wrote: Your testscase works more stable than my original testcase. It looks like the problem is somehow triggered by upgrading from serf 1.3.4 to 1.3.5. I'm trying to reach out to the serf developers to see how we should fix this problem. I think you can work around this problem by using serf 1.3.4, or by increasing the maximum number of requests per connection on the server. Bert Hey, Bert. Just caught this follow-up message of yours. A few users on the FreeBSD forums are getting bus errors / segfaults as well, and I was trying to gather more testcases, but it looks as if you've been able to reproduce the crashes and have identified the problem. Are your latest r2360/r2361 commits on serf related to this? The r2360 commit message mentions a requests-per-connection limit. Cheers.
RE: svn 1.8.9 checkout crashes (debugging details provided)
> -Original Message-
> From: Bert Huijben [mailto:b...@qqmail.nl]
> Sent: maandag 26 mei 2014 13:12
> To: dereck...@openmailbox.org; users@subversion.apache.org
> Subject: RE: svn 1.8.9 checkout crashes (debugging details provided)
>
>
>
> > -Original Message-
> > From: dereck...@openmailbox.org [mailto:dereck...@openmailbox.org]
> > Sent: maandag 26 mei 2014 12:54
> > To: users@subversion.apache.org
> > Subject: svn 1.8.9 checkout crashes (debugging details provided)
> >
> > BASIC DETAILS
> > =
> >
> > I'm experiencing intermittent checkout crashes with svn 1.8.9 on FreeBSD
> > that
> > aren't easily reproduced. I've included the latest three crash logs (the
> > earlier crashes were similar to the first two I've included -- seen in
> > gdb to
> > occur at the same point -- but I didn't save the core files for those).
> >
> > The checkout will usually run for minutes at a time before the crashes
> > occur, displaying a lengthy list of successfully added files ("A ...")
> > before
> > segfaulting somewhere through. On some runs, I've managed to checkout
> > everything without incident. There appears to be no rhyme or reason to
> > which
> > particular file fetch causes the crash.
> >
> > If there's anything I can do to help a fix along, please let me know.
>
> I had a similar crash on that same line on Windows (using 1.8.9 and latest
> serf) and at least one other user reported a problem that might have the
> same root cause... as his checkout triggered that error for me.
>
> If you find a way to reproduce it, please let us know as the current trace
> doesn't really explain what causes this problem in 1.8.9, but not in older
> versions.
Your testscase works more stable than my original testcase.
It looks like the problem is somehow triggered by upgrading from serf 1.3.4
to 1.3.5. I'm trying to reach out to the serf developers to see how we
should fix this problem.
I think you can work around this problem by using serf 1.3.4, or by
increasing the maximum number of requests per connection on the server.
Bert
RE: svn 1.8.9 checkout crashes (debugging details provided)
> -Original Message-
> From: dereck...@openmailbox.org [mailto:dereck...@openmailbox.org]
> Sent: maandag 26 mei 2014 12:54
> To: users@subversion.apache.org
> Subject: svn 1.8.9 checkout crashes (debugging details provided)
>
> BASIC DETAILS
> =
>
> I'm experiencing intermittent checkout crashes with svn 1.8.9 on FreeBSD
> that
> aren't easily reproduced. I've included the latest three crash logs (the
> earlier crashes were similar to the first two I've included -- seen in
> gdb to
> occur at the same point -- but I didn't save the core files for those).
>
> The checkout will usually run for minutes at a time before the crashes
> occur, displaying a lengthy list of successfully added files ("A ...")
> before
> segfaulting somewhere through. On some runs, I've managed to checkout
> everything without incident. There appears to be no rhyme or reason to
> which
> particular file fetch causes the crash.
>
> If there's anything I can do to help a fix along, please let me know.
I had a similar crash on that same line on Windows (using 1.8.9 and latest
serf) and at least one other user reported a problem that might have the
same root cause... as his checkout triggered that error for me.
If you find a way to reproduce it, please let us know as the current trace
doesn't really explain what causes this problem in 1.8.9, but not in older
versions.
Bert
