RE: svn commit: r990537 - /subversion/branches/performance/subversion/libsvn_subr/svn_temp_ser ializer.c

2010-08-30 Thread Bert Huijben


 -Original Message-
 From: Daniel Shahaf [mailto:d...@daniel.shahaf.name]
 Sent: zondag 29 augustus 2010 16:36
 To: Johan Corveleyn
 Cc: dev@subversion.apache.org; stef...@apache.org
 Subject: Re: svn commit: r990537 -
 /subversion/branches/performance/subversion/libsvn_subr/svn_temp_ser
 ializer.c
 
 Johan Corveleyn wrote on Sun, Aug 29, 2010 at 16:24:24 +0200:
  Hi Stefan,
 
  On Sun, Aug 29, 2010 at 12:32 PM,  stef...@apache.org wrote:
   Author: stefan2
   Date: Sun Aug 29 10:32:08 2010
   New Revision: 990537
  
   URL: http://svn.apache.org/viewvc?rev=990537view=rev
   Log:
   Looking for the cause of Johan Corveleyn's crash (see
   http://svn.haxx.se/dev/archive-2010-08/0652.shtml), it
   seems that wrong / corrupted data contains backward
   pointers, i.e. negative offsets. That cannot happen if
   everything works as intended.
 
  I've just retried my test after this change (actually with
  performance-bra...@990579, so updated just 10 minutes ago). Now I get
  the assertion error, after running log or blame on that particular
  file:
 
  [[[
  $ svnserve -d -r c:/research/svn/experiment/repos
  Assertion failed: *ptr  buffer, file
  ..\..\..\subversion\libsvn_subr\svn_temp_serializer.c, line 282
 
  This application has requested the Runtime to terminate it in an unusual
 way.
  Please contact the application's support team for more information.
  ]]]
 
  Is there any way I can find more information about this failure, so I
  can help you diagnose the problem?
 
 
 s/assert/SVN_ERR_MALFUNCTION_NO_RETURN/

assert and SVN_ERR_MALFUNCTION() have their own use.

assert() is only evaluated in Maintainer/DEBUG mode and isn't evaluated at
runtime for normal users. So this assertion is good for tests that are too
slow for general use or for things that shouldn't happen but are not really
an error. 
E.g. it is used through svn_dirent_*(), svn_uri_*(), etc. 
[A url is really just a valid unix path which can be canonicalized by
removing double slashes, but using it is in almost every case a user error.
So this triggers a maintainer/debug only assertion]

This allows easier debugging of these errors.

SVN_ERR_MALFUNCTION() and SVN_ERR_ASSERT() are for tests that also need
evaluation in release mode. The fatal error is avoidable by installing a
error callback.


The SVN_ERR_MALFUNCTION_NO_RETURN and SVN_ERR_ASSERT_NO_RETURN() function
should be avoided (real fix: Make the function return svn_error_t*) as these
will trigger an error and then an abort at runtime. So it will CRASH third
party application when this error occurs there.

Bert



Re: svn commit: r990537 - /subversion/branches/performance/subversion/libsvn_subr/svn_temp_ser ializer.c

2010-08-30 Thread Stefan Fuhrmann

Bert Huijben wrote:
  

-Original Message-
From: Daniel Shahaf [mailto:d...@daniel.shahaf.name]
Sent: zondag 29 augustus 2010 16:36
To: Johan Corveleyn
Cc: dev@subversion.apache.org; stef...@apache.org
Subject: Re: svn commit: r990537 -
/subversion/branches/performance/subversion/libsvn_subr/svn_temp_ser
ializer.c

Johan Corveleyn wrote on Sun, Aug 29, 2010 at 16:24:24 +0200:


Hi Stefan,

On Sun, Aug 29, 2010 at 12:32 PM,  stef...@apache.org wrote:
  

Author: stefan2
Date: Sun Aug 29 10:32:08 2010
New Revision: 990537

URL: http://svn.apache.org/viewvc?rev=990537view=rev
Log:
Looking for the cause of Johan Corveleyn's crash (see
http://svn.haxx.se/dev/archive-2010-08/0652.shtml), it
seems that wrong / corrupted data contains backward
pointers, i.e. negative offsets. That cannot happen if
everything works as intended.


I've just retried my test after this change (actually with
performance-bra...@990579, so updated just 10 minutes ago). Now I get
the assertion error, after running log or blame on that particular
file:

[[[
$ svnserve -d -r c:/research/svn/experiment/repos
Assertion failed: *ptr  buffer, file
..\..\..\subversion\libsvn_subr\svn_temp_serializer.c, line 282

This application has requested the Runtime to terminate it in an unusual
  

way.


Please contact the application's support team for more information.
]]]

Is there any way I can find more information about this failure, so I
can help you diagnose the problem?

  

s/assert/SVN_ERR_MALFUNCTION_NO_RETURN/



assert and SVN_ERR_MALFUNCTION() have their own use.

assert() is only evaluated in Maintainer/DEBUG mode and isn't evaluated at
runtime for normal users. So this assertion is good for tests that are too
slow for general use or for things that shouldn't happen but are not really
an error. 
E.g. it is used through svn_dirent_*(), svn_uri_*(), etc. 
[A url is really just a valid unix path which can be canonicalized by

removing double slashes, but using it is in almost every case a user error.
So this triggers a maintainer/debug only assertion]

This allows easier debugging of these errors.

SVN_ERR_MALFUNCTION() and SVN_ERR_ASSERT() are for tests that also need
evaluation in release mode. The fatal error is avoidable by installing a
error callback.


The SVN_ERR_MALFUNCTION_NO_RETURN and SVN_ERR_ASSERT_NO_RETURN() function
should be avoided (real fix: Make the function return svn_error_t*) as these
will trigger an error and then an abort at runtime. So it will CRASH third
party application when this error occurs there.

  
I'm a bit torn what the ultimate solution would be here. For the time 
being,

I only need that as debug code. Depending on the root cause of the problem,
I might promote it to general threat and then use one of the NO_RETURN
errors because it is still better than a PF.

But it would never be a good choice to just continue operations (e.g. 
disable

the cache and retry) because the server may already operate on or even have
handed out on corrupted data.

-- Stefan^2.



Re: [PATCH] Add a '--fsfs-no-repsharing' flag to svnadmin

2010-08-30 Thread Simon Atanasyan
On Tue, Aug 3, 2010 at 18:20, Daniel Shahaf d...@daniel.shahaf.name wrote:
 Perhaps the patch would face more friendly winds if you retained the API
 bit, but replaced --fsfs-no-rep-sharing by some more generic --config-option
 flag?  (compare 'svn --config-option')

I think moving --fsfs-no-rep-sharing under --config-option will be
inconsistent with existing options --bdb-txn-nosync and
--bdb-log-keep. Besides that --config-option usually sets up
environment for command execution rather than directly affects command
execution result.

Folks do you have any more objections to the patch prevent its applying?

-- 
Simon Atanasyan
VisualSVN Limited


Re: svn commit: r979193 - in /subversion/branches/performance/subversion: include/private/svn_cache.h libsvn_subr/cache-membuffer.c

2010-08-30 Thread C. Michael Pilato
On 08/30/2010 06:31 AM, Stefan Fuhrmann wrote:
 However, I simply fail to identify the typos you are referring
 to below. Could you give me the searchreplacement spec
 (s/typpo/typo/)?
 
 +   * to 2 times the average hit count.
 +   */
 +  average_hits = cache-hit_count / cache-used_entries;

 Typo.

I can't spot this one, either.

 +  c-group_count = directory_size / sizeof (entry_group_t);

 And here.

No space between sizeof and (, perhaps?

-- 
C. Michael Pilato cmpil...@collab.net
CollabNet  www.collab.net  Distributed Development On Demand


Re: [PATCH] was: [WIP] Enable passing copyfrom information for the diff code when dealing with repository diffs.

2010-08-30 Thread Daniel Näslund
Daniel,

Sorry for the delayed response. I've committed the patch with your
suggestions in r990790. Below is just ACK's for those changes.

On Mon, Aug 16, 2010 at 10:59:03PM +0300, Daniel Shahaf wrote:
 Daniel Näslund wrote on Mon, Aug 16, 2010 at 15:16:33 +0200:
  And this time with a patch attached.
  
  On Mon, Aug 16, 2010 at 03:09:49PM +0200, Daniel Näslund wrote:
   I'm touching a lot of code here. Reviews would be much apprecieated.
  
   
   [[[
   Make the diff editor able to receive copyfrom information. Involves
   passing down a 'send_copyfrom_args' to all RA implemtations.
   
   Note that this commit merely allows the copyfrom args to be passed to
   the client. They copyfrom information is not yet stored and used.
 
 This commit changes the ra_svn protocols, but not the other protocols.
 Okay.
 
   ]]]
   
   Thanks,
   Daniel
 
  Index: subversion/libsvn_ra/deprecated.c
  ===
  --- subversion/libsvn_ra/deprecated.c   (revision 985813)
  +++ subversion/libsvn_ra/deprecated.c   (arbetskopia)
  @@ -239,6 +239,30 @@ svn_error_t *svn_ra_get_commit_editor(svn_ra_sessi
  keep_locks, pool);
   }
   
  +svn_error_t * svn_ra_do_diff3(svn_ra_session_t *session,
 
 Style nit: no space after '*'.

Fixed.

  +  apr_pool_t *pool)
  Index: subversion/libsvn_ra_svn/protocol
  ===
  --- subversion/libsvn_ra_svn/protocol   (revision 985813)
  +++ subversion/libsvn_ra_svn/protocol   (arbetskopia)
  @@ -345,7 +345,8 @@ second place for auth-request point as noted below
   
 diff
   params:   ( [ rev:number ] target:string recurse:bool 
  ignore-ancestry:bool
  -url:string ? text-deltas:bool ? depth:word )
  +url:string ? text-deltas:bool ? depth:word 
  +send_copyfrom_param:bool )
 
 Err, shouldn't this be
 
   +url:string ? text-deltas:bool ? depth:word 
   +? send_copyfrom_param:bool )
 
 since clients before-your-change don't send the send_copyfrom_args param?

Yes it should. Fixed.

   Client switches to report command set.
   Upon finish-report, server sends auth-request.
   After auth exchange completes, server switches to editor command set.
  Index: subversion/include/svn_ra.h
  ===
  --- subversion/include/svn_ra.h (revision 985813)
  +++ subversion/include/svn_ra.h (arbetskopia)
  @@ -1291,9 +1291,30 @@ svn_ra_do_status(svn_ra_session_t *session,
* needed, and sending too much data back, a pre-1.5 'recurse'
* directive may be sent to the server, based on @a depth.
*
  - * @since New in 1.5.
  + * @since New in 1.7.
*/
   svn_error_t *
  +svn_ra_do_diff4(svn_ra_session_t *session,
  +const svn_ra_reporter3_t **reporter,
  +void **report_baton,
  +svn_revnum_t revision,
  +const char *diff_target,
  +svn_depth_t depth,
  +svn_boolean_t send_copyfrom_args,
  +svn_boolean_t ignore_ancestry,
  +svn_boolean_t text_deltas,
  +const char *versus_url,
  +const svn_delta_editor_t *diff_editor,
  +void *diff_baton,
  +apr_pool_t *pool);
 
 Need an empty line right here (as I said in a previous mail).

Fixed.

  +/**
  + * Similar to svn_ra_do_diff4(), but with @c send_copyfrom_args set to
  + * FALSE.
  + *
  + * @deprecated Provided for compatibility with the 1.5 API.
  + */
  +SVN_DEPRECATED
  +svn_error_t *
   svn_ra_do_diff3(svn_ra_session_t *session,
   const svn_ra_reporter3_t **reporter,
   void **report_baton,
  @@ -1306,7 +1327,6 @@ svn_ra_do_diff3(svn_ra_session_t *session,
   const svn_delta_editor_t *diff_editor,
   void *diff_baton,
   apr_pool_t *pool);
  -
 
 And here you removed an empty line?  Can we have it back please?
 
 (that way one can use paragraph-motion commands (e.g., '}') when reading
 the source.)

Fixed.

   /**
* Similar to svn_ra_do_diff3(), but taking @c svn_ra_reporter2_t
* instead of @c svn_ra_reporter3_t, and therefore only able to report
  Index: subversion/libsvn_client/repos_diff.c
  ===
  --- subversion/libsvn_client/repos_diff.c   (revision 985813)
  +++ subversion/libsvn_client/repos_diff.c   (arbetskopia)
  @@ -552,6 +552,8 @@ delete_entry(const char *path,
 svn_wc_notify_action_t action = svn_wc_notify_skip;
 svn_boolean_t tree_conflicted = FALSE;
   
  +  SVN_DBG((delete_entry: %s\n, path));
  +
 
 Shouldn't be committed.  (There are a few others in the diff.)

The debug statements are now removed.

  Index: 

Re: svn commit: r990537 - /subversion/branches/performance/subversion/libsvn_subr/svn_temp_serializer.c

2010-08-30 Thread Johan Corveleyn
On Mon, Aug 30, 2010 at 12:05 PM, Stefan Fuhrmann 
stefanfuhrm...@alice-dsl.de wrote:

 Johan Corveleyn wrote:

 On Sun, Aug 29, 2010 at 12:32 PM, stefan2_at_apache.org wrote:
 / Author: stefan2 /
 / Date: Sun Aug 29 10:32:08 2010 /
 / New Revision: 990537 /
 / /
 / URL: http://svn.apache.org/viewvc?rev=990537view=rev 
 http://svn.apache.org/viewvc?rev=990537view=rev /

 / Log: /
 / Looking for the cause of Johan Corveleyn's crash (see /
 / http://svn.haxx.se/dev/archive-2010-08/0652.shtml), it /
 / seems that wrong / corrupted data contains backward /
 / pointers, i.e. negative offsets. That cannot happen if /
 / everything works as intended. /

 I've just retried my test after this change (actually with
 performance-branch_at_990579, so updated just 10 minutes ago). Now I get

 the assertion error, after running log or blame on that particular
 file:

 [[[
 $ svnserve -d -r c:/research/svn/experiment/repos
 Assertion failed: *ptr  buffer, file
 ..\..\..\subversion\libsvn_subr\svn_temp_serializer.c, line 282

 This application has requested the Runtime to terminate it in an unusual
 way.
 Please contact the application's support team for more information.
 ]]]

  That is what I expected looking at the call stacks you posted.
 My preliminary analysis goes as follows:

 * The error seems to be limited to relatively rare occasions.
  That sufficiently excludes alignment issues and plainly wrong
  parameters / function calls.

 * It might be a (still rare) 32-bit-only issue.

 * There seems to be no miscast of types, i.e. the DAG node
  being read and causing the PF is actually a DAG node. Even
  if conflicting keys were used, the structure could still be read
  from the cache and would lead to some logic failure elsewhere.

 What else could it be? Most of the following are rather

 * concurrency issue
 * data corruption within the cache itself
 * some strange serialization issue that needs very specific data
  and / or 32 bit pointers to show up


  Is there any way I can find more information about this failure, so I
 can help you diagnose the problem?

  In fact there is. Just some questions:

 * You are the only one accessing the server and you use
  a single client process?


Yes. All on the same machine actually (my laptop). Accessing the server with
svn://localhost.


 * Does it happen if you log / blame the file for the first time
  and no other requests have been made to the server before?


Yes


 * Does a command line svn log produce some output
  before the crash? If so, is there something unusual happening
  around these revisions (branch replacement or so)?


Yes. Running svn log svn://localhost/trunk/some/path/bigfile.xml yields
969
of the 2279 log entries. From r95849 (last change to this file) down to
r42100. Then it suddenly stops.

I've checked r42100 with log -v, and it only mentions text modification of
bigfile.xml. Same goes for the previous and next revisions in which
bigfile.xml was affected (r42104 and r42042).




 Also, please verify that the crash gets triggered if the server is started
 with the following extra parameters:

 * -c0 -M0 -F0


No crash


 * -c0 -M0


No crash


 * -c0 -M1500 -F0


Crash (actually I did it with -M1000, because M1500 would give me an Out of
memory immediately).


 * -c0 -M1500


Crash (with -M1000 that is)





 Just to be clear: the very same repos does not have this problem when
 accessed by a trunk svnserve.

 I thought so ;) To narrow down the nature of the problem,
 I added some checks that  should be able to discern plain
 data corruption from (de-)serialization issues. Please apply
 either the patch or replace the original files with the versions
 in the .zip file.

 A debug build should then, hopefully, trigger a different
 and more specific assertion.


Ok, will try that now.

-- 
Johan


Re: svn commit: r990916 - in /subversion/trunk/subversion/bindings/javahl/native: ClientContext.cpp ClientContext.h

2010-08-30 Thread Hyrum Wright
On Mon, Aug 30, 2010 at 2:37 PM,  hwri...@apache.org wrote:
 Author: hwright
 Date: Mon Aug 30 19:37:05 2010
 New Revision: 990916

 URL: http://svn.apache.org/viewvc?rev=990916view=rev
 Log:
 JavaHL: Create a persistent C client context, in order to reuse the working
 copy context between client API invocations.

 Note: This introduces a test failure which I believe to be a manifestation of
 a bug in wc-ng.

For the interested, this is the test failure:

[[[
hwri...@orac:~/dev/svn-trunk2$ make check-javahl
/usr/lib/jvm/java-6-openjdk//bin/java
-Dtest.rootdir=/home/hwright/dev/svn-trunk2/subversion/bindings/javahl/test-work
-Dtest.srcdir=/home/hwright/dev/svn-trunk2/subversion/bindings/javahl
-Dtest.rooturl= -Dtest.fstype=
-Djava.library.path=subversion/bindings/javahl/native/.libs:/usr/local/lib
-classpath 
subversion/bindings/javahl/classes:/home/hwright/dev/svn-trunk2/subversion/bindings/javahl/src:/usr/share/java/junit.jar
-Dtest.tests= org.apache.subversion.javahl.RunTests
..E...
...
Time: 26.083
There was 1 error:
1) 
testBasicRevert(org.apache.subversion.javahl.BasicTests)org.apache.subversion.javahl.ClientException:
Attempted to lock an already-locked dir
svn: Working copy
'/home/hwright/dev/svn-trunk2/subversion/bindings/javahl/test-work/working_copies/basic_test19/X'
locked
SQLite error
svn: disk I/O error
svn: disk I/O error
svn: cannot rollback - no transaction is active
Working copy not locked; this is probably a bug, please report
svn: Working copy not locked at
'/home/hwright/dev/svn-trunk2/subversion/bindings/javahl/test-work/working_copies/basic_test19/X'.

at native.subversion.libsvn_wc(wc_db.c:8481)
at native.subversion.libsvn_subr(sqlite.c:118)
at native.subversion.libsvn_subr(sqlite.c:513)
at native.subversion.libsvn_subr(sqlite.c:211)
at native.subversion.libsvn_wc(wc_db.c:8276)
at native.subversion.libsvn_subr(sqlite.c:118)
at native.subversion.libsvn_wc(lock.c:1655)
at native.subversion.libsvn_wc(lock.c:1639)
at native.subversion.libsvn_wc(lock.c:1744)
at native.subversion.libsvn_wc(lock.c:1844)
at native.subversion.libsvn_client(revert.c:176)
at org.apache.subversion.javahl.SVNClient.revert(Native Method)
at 
org.apache.subversion.javahl.BasicTests.testBasicRevert(BasicTests.java:1332)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at org.apache.subversion.javahl.RunTests.main(RunTests.java:116)

FAILURES!!!
Tests run: 52,  Failures: 0,  Errors: 1

make: *** [check-javahl] Error 1
hwri...@orac:~/dev/svn-trunk2$
]]]

I believe it to be a legitimate bug, not only because the client APIs
are being used in a completely reasonable way, but also because the
output told me so. :)

-Hyrum


Re: svn commit: r990537 - /subversion/branches/performance/subversion/libsvn_subr/svn_temp_serializer.c

2010-08-30 Thread Johan Corveleyn
On Mon, Aug 30, 2010 at 9:32 PM, Johan Corveleyn jcor...@gmail.com wrote:

 On Mon, Aug 30, 2010 at 12:05 PM, Stefan Fuhrmann 
 stefanfuhrm...@alice-dsl.de wrote:

 I thought so ;) To narrow down the nature of the problem,
 I added some checks that  should be able to discern plain
 data corruption from (de-)serialization issues. Please apply
 either the patch or replace the original files with the versions
 in the .zip file.

 A debug build should then, hopefully, trigger a different
 and more specific assertion.


 Ok, will try that now.


:-(, I updated to r990759 and applied your attached debug.patch, but I just
get the same assertion (offset by 4 lines), with no additional information:

[[[
$ svnserve -d -r c:/research/svn/experiment/repos
Assertion failed: *ptr  buffer, file
..\..\..\subversion\libsvn_subr\svn_temp_serializer.c, line 286
]]]

Anything else I can try?

FWIW, I do see a lot of warnings during compilation of cache-membuffer.c
(this is after applying your patch, so line-numbers should be interpreted
accordingly):

[[[
..\..\..\subversion\libsvn_subr\cache-membuffer.c(353): warning C4245: '=' :
conversion from 'int' to 'apr_uint64_t', signed/unsigned mismatch
..\..\..\subversion\libsvn_subr\cache-membuffer.c(384): warning C4245: '=' :
conversion from 'int' to 'apr_uint32_t', signed/unsigned mismatch
..\..\..\subversion\libsvn_subr\cache-membuffer.c(433): warning C4245:
'return' : conversion from 'int' to 'apr_uint32_t', signed/unsigned mismatch
..\..\..\subversion\libsvn_subr\cache-membuffer.c(546): warning C4244:
'initializing' : conversion from 'apr_uint64_t' to 'apr_size_t', possible
loss of data
..\..\..\subversion\libsvn_subr\cache-membuffer.c(656): warning C4244:
'initializing' : conversion from 'apr_uint64_t' to 'int', possible loss of
data
..\..\..\subversion\libsvn_subr\cache-membuffer.c(663): warning C4018: '='
: signed/unsigned mismatch
..\..\..\subversion\libsvn_subr\cache-membuffer.c(731): warning C4245: '=' :
conversion from 'int' to 'apr_uint32_t', signed/unsigned mismatch
..\..\..\subversion\libsvn_subr\cache-membuffer.c(732): warning C4245: '=' :
conversion from 'int' to 'apr_uint32_t', signed/unsigned mismatch
..\..\..\subversion\libsvn_subr\cache-membuffer.c(733): warning C4245: '=' :
conversion from 'int' to 'apr_uint32_t', signed/unsigned mismatch
..\..\..\subversion\libsvn_subr\cache-membuffer.c(736): warning C4244:
'function' : conversion from 'apr_uint64_t' to 'apr_size_t', possible loss
of data
..\..\..\subversion\libsvn_subr\cache-membuffer.c(737): warning C4305: 'type
cast' : truncation from 'apr_uint64_t' to 'unsigned char *'
..\..\..\subversion\libsvn_subr\cache-membuffer.c(771): warning C4018: '' :
signed/unsigned mismatch
..\..\..\subversion\libsvn_subr\cache-membuffer.c(773): warning C4245: '=' :
conversion from 'int' to 'apr_uint64_t', signed/unsigned mismatch
..\..\..\subversion\libsvn_subr\cache-membuffer.c(944): warning C4244: '=' :
conversion from 'apr_uint64_t' to 'apr_size_t', possible loss of data
..\..\..\subversion\libsvn_subr\cache-membuffer.c(946): warning C4305: 'type
cast' : truncation from 'apr_uint64_t' to 'char *'
]]]

Cheers,
-- 
Johan


Re: svn commit: r990537 - /subversion/branches/performance/subversion/libsvn_subr/svn_temp_serializer.c

2010-08-30 Thread Stefan Fuhrmann

Johan Corveleyn wrote:
On Mon, Aug 30, 2010 at 12:05 PM, Stefan Fuhrmann 
stefanfuhrm...@alice-dsl.de mailto:stefanfuhrm...@alice-dsl.de wrote:



Is there any way I can find more information about this
failure, so I
can help you diagnose the problem?

In fact there is. Just some questions:

* You are the only one accessing the server and you use
 a single client process?


Yes. All on the same machine actually (my laptop). Accessing the 
server with svn://localhost.
 


* Does it happen if you log / blame the file for the first time
 and no other requests have been made to the server before?


Yes
 


* Does a command line svn log produce some output
 before the crash? If so, is there something unusual happening
 around these revisions (branch replacement or so)?


Yes. Running svn log svn://localhost/trunk/some/path/bigfile.xml 
yields 969
of the 2279 log entries. From r95849 (last change to this file) down 
to r42100. Then it suddenly stops.


I've checked r42100 with log -v, and it only mentions text 
modification of bigfile.xml. Same goes for the previous and next 
revisions in which bigfile.xml was affected (r42104 and r42042).



Good, we should then be able to find the root cause
since this is a simple setup and the crash is deterministic.

Have you tried svn log with a range like -r42104:42042
(or larger)? Perhaps, we can narrow it down to a single
revision (probably svn log -r42042).
 



Also, please verify that the crash gets triggered if the server is
started
with the following extra parameters:

* -c0 -M0 -F0


No crash

That disabled all membuffer as well as the file handle caches
but the serialization code is still active.
 


* -c0 -M0


No crash

So, its not an issue with file handles kept open, positions
not being tracked properly or something.
 


* -c0 -M1500 -F0


Crash (actually I did it with -M1000, because M1500 would give me an 
Out of memory immediately).

Good. -c0 is just the standard setup I use and it does
not prevent the crash. -M1000 is certainly large enough
to keep all data in cache, i.e. the entry eviction / cache
compaction code won't get triggered.
 


* -c0 -M1500


Crash (with -M1000 that is)
 

Again, the file handle cache has no impact.

-- Stefan^2.



Re: svn commit: r990537 - /subversion/branches/performance/subversion/libsvn_subr/svn_temp_serializer.c

2010-08-30 Thread Stefan Fuhrmann

Johan Corveleyn wrote:
On Mon, Aug 30, 2010 at 9:32 PM, Johan Corveleyn jcor...@gmail.com 
mailto:jcor...@gmail.com wrote:


On Mon, Aug 30, 2010 at 12:05 PM, Stefan Fuhrmann
stefanfuhrm...@alice-dsl.de mailto:stefanfuhrm...@alice-dsl.de
wrote:

I thought so ;) To narrow down the nature of the problem,
I added some checks that  should be able to discern plain
data corruption from (de-)serialization issues. Please apply
either the patch or replace the original files with the versions
in the .zip file.

A debug build should then, hopefully, trigger a different
and more specific assertion.


Ok, will try that now.


:-(, I updated to r990759 and applied your attached debug.patch, but I 
just get the same assertion (offset by 4 lines), with no additional 
information:


[[[
$ svnserve -d -r c:/research/svn/experiment/repos
Assertion failed: *ptr  buffer, file 
..\..\..\subversion\libsvn_subr\svn_temp_serializer.c, line 286

]]]

Anything else I can try?


The failure was expected. With the patched code, I just
hope to get more context information.

Could you send me the call stack dump? My suspicion is
that the DAG node is already (partially) garbage before it
actually gets added to the cache.
FWIW, I do see a lot of warnings during compilation of 
cache-membuffer.c (this is after applying your patch, so line-numbers 
should be interpreted accordingly):


[[[
..\..\..\subversion\libsvn_subr\cache-membuffer.c(353): warning C4245: 
'=' : conversion from 'int' to 'apr_uint64_t', signed/unsigned mismatch
..\..\..\subversion\libsvn_subr\cache-membuffer.c(384): warning C4245: 
'=' : conversion from 'int' to 'apr_uint32_t', signed/unsigned mismatch
..\..\..\subversion\libsvn_subr\cache-membuffer.c(433): warning C4245: 
'return' : conversion from 'int' to 'apr_uint32_t', signed/unsigned 
mismatch
..\..\..\subversion\libsvn_subr\cache-membuffer.c(546): warning C4244: 
'initializing' : conversion from 'apr_uint64_t' to 'apr_size_t', 
possible loss of data
..\..\..\subversion\libsvn_subr\cache-membuffer.c(656): warning C4244: 
'initializing' : conversion from 'apr_uint64_t' to 'int', possible 
loss of data
..\..\..\subversion\libsvn_subr\cache-membuffer.c(663): warning C4018: 
'=' : signed/unsigned mismatch
..\..\..\subversion\libsvn_subr\cache-membuffer.c(731): warning C4245: 
'=' : conversion from 'int' to 'apr_uint32_t', signed/unsigned mismatch
..\..\..\subversion\libsvn_subr\cache-membuffer.c(732): warning C4245: 
'=' : conversion from 'int' to 'apr_uint32_t', signed/unsigned mismatch
..\..\..\subversion\libsvn_subr\cache-membuffer.c(733): warning C4245: 
'=' : conversion from 'int' to 'apr_uint32_t', signed/unsigned mismatch
..\..\..\subversion\libsvn_subr\cache-membuffer.c(736): warning C4244: 
'function' : conversion from 'apr_uint64_t' to 'apr_size_t', possible 
loss of data
..\..\..\subversion\libsvn_subr\cache-membuffer.c(737): warning C4305: 
'type cast' : truncation from 'apr_uint64_t' to 'unsigned char *'

None of those should affect the behavior.
..\..\..\subversion\libsvn_subr\cache-membuffer.c(771): warning C4018: 
'' : signed/unsigned mismatch

That one is a theoretical issue on LLP64.
..\..\..\subversion\libsvn_subr\cache-membuffer.c(773): warning C4245: 
'=' : conversion from 'int' to 'apr_uint64_t', signed/unsigned mismatch
..\..\..\subversion\libsvn_subr\cache-membuffer.c(944): warning C4244: 
'=' : conversion from 'apr_uint64_t' to 'apr_size_t', possible loss of 
data
..\..\..\subversion\libsvn_subr\cache-membuffer.c(946): warning C4305: 
'type cast' : truncation from 'apr_uint64_t' to 'char *'

]]]

Uncritical, again. I will fix them all later this week.

Thank you for your time helping me to debug that code!

-- Stefan^2.