On Fri, Mar 2, 2012 at 8:12 AM, Daniel Shahaf <danie...@elego.de> wrote:
> Jason Wong wrote on Fri, Mar 02, 2012 at 07:32:38 -0800:
>> On Fri, Mar 2, 2012 at 2:58 AM, Daniel Shahaf <danie...@elego.de> wrote:
>> > Jason Wong wrote on Thu, Mar 01, 2012 at 10:01:26 -0800:
>> >> I have had a developer here create a build of the latest SVN code
>> >> with your changes you mentioned in r1294470 for the svnadmin verify
>> >
>> > Okay, that's great news, for two reasons:
>> >
>> > 1. It means building svn on windows isn't as painful as it used to be :)
>>
>> Actually, it did take some work to get it going as we did not have
>> another system available to us and also did not have VC++ 6. We had
>> to use VS 2010 in order to do this. Also, for the other components
>> required (python,perl etc), the files after the install were copied
>> to the workstation to see if it would work as we did not want to
>> change the current workstation configuration by running the
>> installers. All in all, it did seem to work.
>>
>
> Okay. The normal build requires just the *.exe and *.dll files to be
> placed appropriately (such that the *.exe's and httpd's find their
> libsvn_* DLL's at runtime) --- it doesn't require Administrator access,
> for example.
>
> To clarify, Perl is only required to build OpenSSL; it is not required
> to build APR, Neon, or Subversion.
>
>> >
>> > 2. It means I can ask you to build a custom server with the 'inprocess'
>> > cache disabled, or (if all else fails) to bisect, per my previous email.
>> >
>> > One of the things you could try is to disable caching: simply modify
>> > the function create_cache() in libsvn_fs_fs/caching.c to always return
>> > NULL in *CACHE_P. See below for another suggestion.
>> >
>> >> command. We have run 'svnadmin verify' against every revision of our
>> >> hotcopy of our repository taken when we first brought this issue to
>> >> the forums and are now tracking down each of the revisions to see
>> >> what actions were being done at those times.
>> >>
>> >
>> > Thanks! I do hope this work enables us to pinpoint and fix the bug.
>>
>> I will be going through the list to see what else was happening at the
>> same time on the apache server since it was alluded to that there may
>> be concurrency issues. I know the last two times that this error has
>> popped up, we had two svn operations starting at around the same time
>> according to the Apache logs. I will go through the previous apache
>> history to see if this was always the case or not.
>>
>
> Thanks, looking forward to hear what you come up with.
>
> FWIW, Justin's reply suggests that the error was seen on three different
> platforms --- Windows, Solaris, and FreeBSD --- so that should narrow
> down the range of possible explanations.
>
> (I'll also note that at ASF's installation we are not running into new
> instances of the bug.)
Hi Daniel.
I haven't gone through all the cases yet, but I have made progress
through quite a number of them and a pattern seems to be coming up.
I have attached 2 txt files. One shows the modified svnadmin verify
output from the binaries we built. The other shows the revisions and
what appears to have been occuring at the time of the bug. I figure
better to provide this now rather than delay any longer for the rest
of the results.
I will continue to go through the rest of the events and see if
there are other differences seen when the issue occurs. I hope
this information helps.
Thanks.
Jason
SVN log history for predecessor node error: from svnadmin verify
svnadmin: E160004: predecessor count for the root node-revision is wrong:
r45558 has 45557, but r45557 has 45557
svnadmin: E160004: predecessor count for the root node-revision is wrong:
r46947 has 46945, but r46946 has 46945
svnadmin: E160004: predecessor count for the root node-revision is wrong:
r46997 has 46994, but r46996 has 46994
svnadmin: E160004: predecessor count for the root node-revision is wrong:
r47004 has 47000, but r47003 has 47000
svnadmin: E160004: predecessor count for the root node-revision is wrong:
r47006 has 47001, but r47005 has 47001
svnadmin: E160004: predecessor count for the root node-revision is wrong:
r47193 has 47187, but r47192 has 47187
svnadmin: E160004: predecessor count for the root node-revision is wrong:
r47715 has 47708, but r47714 has 47708
svnadmin: E160004: predecessor count for the root node-revision is wrong:
r47718 has 47710, but r47717 has 47710
svnadmin: E160004: predecessor count for the root node-revision is wrong:
r50049 has 50040, but r50048 has 50040
svnadmin: E160004: predecessor count for the root node-revision is wrong:
r50963 has 50953, but r50962 has 50953
svnadmin: E160004: predecessor count for the root node-revision is wrong:
r51481 has 51470, but r51480 has 51470
svnadmin: E160004: predecessor count for the root node-revision is wrong:
r51684 has 51672, but r51683 has 51672
svnadmin: E160004: predecessor count for the root node-revision is wrong:
r52082 has 52069, but r52081 has 52069
svnadmin: E160004: predecessor count for the root node-revision is wrong:
r53220 has 53205, but r53219 has 53206
svnadmin: E160004: predecessor count for the root node-revision is wrong:
r53298 has 53282, but r53297 has 53282
svnadmin: E160004: predecessor count for the root node-revision is wrong:
r54248 has 54231, but r54247 has 54231
svnadmin: E160004: predecessor count for the root node-revision is wrong:
r54277 has 54259, but r54276 has 54259
svnadmin: E160004: predecessor count for the root node-revision is wrong:
r55183 has 55163, but r55182 has 55164
svnadmin: E160004: predecessor count for the root node-revision is wrong:
r55467 has 55446, but r55466 has 55446
svnadmin: E160004: predecessor count for the root node-revision is wrong:
r57871 has 57849, but r57870 has 57849
svnadmin: E160004: predecessor count for the root node-revision is wrong:
r58073 has 58050, but r58072 has 58050
svnadmin: E160004: predecessor count for the root node-revision is wrong:
r58075 has 58048, but r58074 has 58051
svnadmin: E160004: predecessor count for the root node-revision is wrong:
r58354 has 58325, but r58353 has 58326
svnadmin: E160004: predecessor count for the root node-revision is wrong:
r58696 has 58664, but r58695 has 58666
svnadmin: E160004: predecessor count for the root node-revision is wrong:
r58787 has 58754, but r58786 has 58754
SVN actions at time of revisions affected:
failed r45558 - 2011-01-26 11:39:27 - merge 5 files
- r45557 - 2011-01-26 11:39:22 - copy to operation (from revision 45556)
: - Apache logs - r45557 starts before and finishs after the r45558 begins
: - Apache logs, both revisions have have "MERGE" and "DELETE" of the
transactions about 5 seconds apart.
: - Apache logs - location worked in by both are different.
: - Apache logs - a few entries recorded from a few seconds earlier (Apache
log has entries for this event not in chonological order)
failed r46947 - 2011-03-09 15:44:13 - add folder
- r46946 - 2011-03-09 15:44:11 - copy to operation (from revision 46945)
: - Apache logs - r46946 starts before and finishs after the r46947 begins
: - Apache logs, both revisions have have "MERGE" and "DELETE" of the
transactions about 2 seconds apart
: - Apache logs - location worked in by both are different.
: - Apache logs - a few entries recorded from a few seconds earlier (Apache
log has entries for this event not in chonological order)
failed r46997 - 2011-03-09 15:47:34 - add folder
- r46996 - 2011-03-09 15:47:34 - item deleted
: - Apache logs - both revisions have merge/delete statements overlapping at
same time.
: - Apache logs - location worked in by both are different.
: - Apache logs - a few entries recorded from a few seconds earlier (Apache
log has entries for this event not in chonological order)
failed r47004 - 2011-03-09 15:47:57 - add folder (from revision 47702)
- r47003 - 2011-03-09 15:47:56 - copy to operation (from revision 47702)
: - Apache logs - both revisions have merge/delete statements overlapping at
same time.
: - Apache logs - location worked in by both are different.
: - Apache logs - a few entries recorded from a few seconds earlier (Apache
log has entries for this event not in chonological order)
failed r47006 - 2011-03-09 15:48:02 - add folder
- r47005 - 2011-03-09 15:48:00 - copy to operation (from revision 47004)
: - Apache logs - both revisions have merge/delete statements overlapping at
same time.
: - Apache logs - location worked in by both are different.
: - Apache logs - a few entries recorded from a few seconds earlier (Apache
log has entries for this event not in chonological order)
failed r47193 - 2011-03-11 17:19:37 - merge files (2)
- r47192 - 2011-03-11 17:19:35 - add operation, marking a build (from revision
47187)
: - Apache logs - both revisions have merge/delete statements overlapping at
same time.
: - Apache logs - location worked in by both are different.
: - Apache logs - a few entries recorded from a few seconds earlier (Apache
log has entries for this event not in chonological order)
: - Apache logs - earlier revision still occurring when second revision
starts and issues "PUT" statement.
The "PUT" statements happen at the same time as the
"Delete" from the earlier revision
failed r47715 - 2011-03-28 13:39:47 - merge files (6)
- r47714 - 2011-03-28 13:39:23 - add operation - mark build (from revision
47709)
: - Apache logs - both revisions have merge/delete statements overlapping at
same time.
: - Apache logs - location worked in by both are different.
: - Apache logs - a few entries recorded from a few seconds earlier (Apache
log has entries for this event not in chonological order)
: - Apache logs - earlier revision still occurring when second revision
starts and issues "PUT" statement.
failed r47718 - 2011-03-28 13:40:09 - add operation - mark build (from revision
47708)
- r47717 - 2011-03-28 13:40:07 - add operation - mark merge (from revision
47709)
: - Apache logs - both revisions have merge/delete statements overlapping at
same time.
: - Apache logs - location worked in by both are different.
: - Apache logs - a few entries recorded from a few seconds earlier (Apache
log has entries for this event not in chonological order)
failed r50049 - 2011-05-31 11:10:28 - add operation - mark build
- r50048 - 2011-05-31 11:10:27 - copy to operation - (from revision 50047)
: - Apache logs - both revisions have merge/delete statements overlapping at
same time.
: - Apache logs - location worked in by both are different.
: - Apache logs - a few entries recorded from a few seconds earlier (Apache
log has entries for this event not in chonological order)
failed r50963 - 2011-06-22 10:32:48 - copy to operation - (from revision 50961)
- r50962 - 2011-06-22 10:32:47 - Delete operation - remove a merged story
: - Apache logs - both revisions have merge/delete statements overlapping at
same time.
: - Apache logs - location worked in by both are different.
: - Apache logs - a few entries recorded from a few seconds earlier (Apache
log has entries for this event not in chonological order)
failed r51481 - 2011-06-29 11:44:12 - add folder
- r51480 - 2011-06-29 11:44:10 - Add operation - (from revision 51462)
: - Apache logs - both revisions have merge/delete statements overlapping at
same time.
: - Apache logs - location worked in by both are different.
: - Apache logs - a few entries recorded from a few seconds earlier (Apache
log has entries for this event not in chonological order)
failed r51684 - 2011-07-05 14:37:29 - copy to operation - (from revision 51682)
- r51683 - 2011-07-05 14:37:27 - mark merge - (from revision 51673)
: - Apache logs - both revisions have merge/delete statements where one
"delete" ends and the other "merge" begins.
: - Apache logs - location worked in by both are different.
: - Apache logs - a few entries recorded from a few seconds earlier (Apache
log has entries for this event not in chonological order)
failed r52082 - 2011-07-13 09:01:14 - merge operation - (2)
- r52081 - 2011-07-13 09:01:14 - copy to operation - (from revision 52080)
: - Apache logs - both revisions have merge/delete statements overlapping at
same time.
: - Apache logs - location worked in by both are different.
: - Apache logs - a few entries recorded from a few seconds earlier (Apache
log has entries for this event not in chonological order)
failed r53220 - 2011-08-12 11:05:51 - merge operation - (14)
- r53219 | txsmith | 2011-08-12 11:05:47 copy to operation - (from revision
53218)
: - Apache logs - both revisions have merge/delete statements about 2
seconds apart.
: - Apache logs - location worked in by both are different.
: - Apache logs - a few entries recorded from a few seconds earlier (Apache
log has entries for this event not in chonological order)
: - Apache logs - r53219 has merge/delete operations occurring in between
"PUT" commands for other revision.
failed r53298 - 2011-08-15 15:12:22 - copy to operation - (from revision 53296)
- r53297 - 2011-08-15 15:12:22 - add operation -(from revision 53291)
: - Apache logs - both revisions have merge/delete statements overlapping at
same time.
: - Apache logs - location worked in by both are different.
: - Apache logs - a few entries recorded from a few seconds earlier (Apache
log has entries for this event not in chonological order)
