Re: Problems with transaction file "next-ids" on Windows

[Mathias Weinert]

Philip Martin <philip.mar...@wandisco.com> wrote:

Mathias Weinert <w...@mccw.de> writes:

Daniel Shahaf <d...@daniel.shahaf.name> wrote:

Philip Martin wrote on Wed, Jul 20, 2011 at 15:12:51 +0100:

Daniel Shahaf <d...@daniel.shahaf.name> writes:

> Mathias Weinert wrote on Wed, Jul 20, 2011 at 14:59:23 +0200:
>>
>> each time when I am loading a certain dump file on Windows which
>> contains one revision with over 100K changed paths I get the error
>> "Can't open file
>> 'c:\Repositories\test\db\transactions\5445-479.txn\next-ids': The
>> requested operation cannot be performed on a file with a user-mapped
>> section open.".

The current implementation writes the file inplace:

static svn_error_t *
write_next_ids(svn_fs_t *fs,
               const char *txn_id,
               const char *node_id,
               const char *copy_id,
               apr_pool_t *pool)
{
  apr_file_t *file;
  svn_stream_t *out_stream;

  SVN_ERR(svn_io_file_open(&file, path_txn_next_ids(fs, txn_id, pool),
                           APR_WRITE | APR_TRUNCATE,
                           APR_OS_DEFAULT, pool));

  out_stream = svn_stream_from_aprfile2(file, TRUE, pool);



  SVN_ERR(svn_stream_close(out_stream));
  return svn_io_file_close(file, pool);
}

Is there any reason we don't switch to our standard pattern: write to a
temp file and rename?  That would give us Subversion's standard retry
loop -- would that fix "requested operation cannot be performed"?

SVN_ERR(svn_stream_printf(out_stream, pool, "%s %s\n", node_id, copy_id));

I replaced svn_io_file_open, svn_stream_printf and svn_stream_close
with svn_io_write_unique and move_into_place (see attached patch).
Although this works correctly it shows bad performance. Loading a dump
from a little repo which mainly has a 2000 files commit now takes
about 400s while with the original version it only took about 100s.

svn_io_write_unique flushes to disk (except 1.7 on Windows) so that
probably explains the slowdown.

Looking at this again, the call to svn_io_file_open already has a retry
loop, so the original error seems to imply that either a) the file is in
use for more than the retry delay or that b) there is some error code
missing from the retry logic.

Maybe we should add ERROR_USER_MAPPED_FILE to the retry logic (see attached patch). BTW the problem only occurs when using Windows svnadmin but not when using cygwin svnadmin (both executed in the same environment) (on cygwin another retry_loop is used which uses different error codes). Unfortunately I can't test the proposed change as I don't have an environment to build Subversion on Windows (without using cygwin).

io.c.diff
Description: io.c.diff