Issue triage: issues 3429 and 3474
Hi, I recently verified that the following two issues are fixed on trunk: - #3429: svn mv A B; svn mv B A generates replace without history - #3474: making a new subdir, moving files into it and then renaming the subdir, breaks history of the moved files Should I mark these as fixed in the issue tracker, or will someone else do that, after double-checking my findings? Or, maybe the best approach: I could add a regression test for these issues, so we can all be sure that they are fixed (and remain fixed), after which they can be marked as fixed. I have not written a test for Subversion before, but I guess I should be able to manage after reading the relevant section from the community guide, and the subversion/tests/README and subversion/tests/cmdline/README it references. Any other pointers or tips are welcome of course :-). Cheers, -- Johan
Re: Issue triage: issues 3429 and 3474
Johan Corveleyn jcor...@gmail.com writes: Or, maybe the best approach: I could add a regression test for these issues, so we can all be sure that they are fixed (and remain fixed), after which they can be marked as fixed. Yes, please. Are there any existing XFAIL tests that apply? They sometimes don't XPASS automatically when the bug is fixed because the test expectation is wrong. I have not written a test for Subversion before, but I guess I should be able to manage after reading the relevant section from the community guide, and the subversion/tests/README and subversion/tests/cmdline/README it references. Any other pointers or tips are welcome of course :-). It's probably easiest to look at some of the recently added tests. Julian added some in r1031411, I added some in r1027031. -- Philip
[PATCH] svn-dev.el - Update paths and links
Log [[[ Update paths and links * tools/dev/svn-dev.el (svn-site-source-tree-top): Rename svn-source-tree-top as svn-site-source-tree-top as now the web site code resides in a separate location. Update doc string. (svn-faq-file, svn-faq-file, svn-url-base, svn-hacking-url): Update location ]]] Thanks and Regards Noorul Index: tools/dev/svn-dev.el === --- tools/dev/svn-dev.el(revision 1032461) +++ tools/dev/svn-dev.el(working copy) @@ -152,15 +152,18 @@ ;;; Net if you don't have a local copy, but it requires a very recent ;;; version of Emacs, so I didn't bother with it here. -kfogel) -(defvar svn-source-tree-top (expand-file-name ~/projects/svn/) - *Top directory of your Subversion source tree. You almost -certainly want to set this in your .emacs, to override the default; -use `(setq svn-source-tree-top \/path/to/the/tree\)'.) +(defvar svn-site-source-tree-top (expand-file-name ~/projects/svn/site/) + *Top directory of your Subversion site source tree of +repository \http://svn.apache.org/repos/asf/subversion/site\;. +You almost certainly want to set this in your .emacs, to override +the default; use `(setq svn-site-source-tree-top +\/path/to/the/site/tree\)'.) -(defvar svn-faq-file (concat svn-source-tree-top /www/faq.html) +(defvar svn-faq-file (concat svn-site-source-tree-top /publish/faq.html) *A local copy of the Subversion FAQ.) -(defvar svn-hacking-file (concat svn-source-tree-top /www/hacking.html) +(defvar svn-hacking-file (concat svn-site-source-tree-top + /docs/community-guide/community-guide.html) *A local copy of the Subversion hacking.html file.) ;; Helper for referring to issue numbers in a user-friendly way. @@ -188,11 +191,13 @@ (start (car bounds)) (end (cdr bounds))) (delete-region start end))) - (insert (format http://svn.collab.net/viewcvs/svn?rev=%sview=rev; rev))) + (insert (format http://svn.apache.org/viewcvs?view=revisionrevision=%s; + rev))) -(defconst svn-url-base http://subversion.tigris.org/;) +(defconst svn-url-base http://subversion.apache.org/;) (defconst svn-faq-url (concat svn-url-base faq.html)) -(defconst svn-hacking-url (concat svn-url-base hacking.html)) +(defconst svn-hacking-url (concat svn-url-base + docs/community-guide/community-guide.html)) (defun svn-html-get-targets (file) Build a list of targets for the Subversion web file FILE.
Re: [PATCH] error leak on performance branch
On Sun, Nov 7, 2010 at 11:58 AM, Stefan Sperling s...@elego.de wrote:
[[[
* subversion/libsvn_fs_util/caching.c
(svn_fs__get_global_membuffer_cache): Fix a possible error leak.
]]]
Index: subversion/libsvn_fs_util/caching.c
===
--- subversion/libsvn_fs_util/caching.c (revision 1032308)
+++ subversion/libsvn_fs_util/caching.c (working copy)
@@ -62,6 +62,7 @@ svn_fs__get_global_membuffer_cache(void)
/* auto-allocate cache*/
apr_allocator_t *allocator = NULL;
apr_pool_t *pool = NULL;
+ svn_error_t *err;
if (apr_allocator_create(allocator))
return NULL;
@@ -75,12 +76,17 @@ svn_fs__get_global_membuffer_cache(void)
apr_allocator_max_free_set(allocator, 1);
pool = svn_pool_create_ex(NULL, allocator);
- svn_cache__membuffer_cache_create
- (cache,
- (apr_size_t)cache_size,
- (apr_size_t)(cache_size / 16),
- ! svn_fs_get_cache_config()-single_threaded,
- pool);
If we're choosing to ignore the error above, you can just wrap the
entire call in an svn_error_clear(). No need to introduce and check a
temp variable.
+ err = svn_cache__membuffer_cache_create
+ (cache,
+ (apr_size_t)cache_size,
+ (apr_size_t)(cache_size / 16),
+ ! svn_fs_get_cache_config()-single_threaded,
+ pool);
+ if (err)
+ {
+ svn_error_clear(err);
+ return NULL;
+ }
Does this early return actually change functionality? If so, this
patch is about more than just fixing an error leak...
}
return cache;
Re: [PATCH] mailer.py : Use python difflib.unified_diff instead of *nix diff command
On 11/05/2010 09:45 PM, Noorul Islam K M wrote:
[[[
If diff command does not exist on the system then use
difflib.unified_diff() to generate diff contents.
* tools/hook-scripts/mailer/mailer.py: Fall back to
difflib.unified_diff() if the script fails to execute native 'diff'
command.
Patch by: Noorul Islam K Mnoorul{_AT_}collab.net
Suggested by: Kamesh Jayachandrankamesh{_AT_}collab.net
]]]
Thanks for the patch Noorul.
I could not *see* the difference in the output. I tried with few file
mod commit mails both via external diff and difflib.
I committed this patch with small refactoring at r1032568.
With regards
Kamesh Jayachandran
Re: svn blame -g causing svnserve to hang mem usage to hit 2GB
On Thu, Nov 4, 2010 at 8:59 PM, Chris Tashjian ct...@thepond.com wrote: I posted this on the users list, but I'm confident that this is a bug. Background: For a while now (off and on for over a year, but more frequently in the last 6+ months) we've been having problems with svn crashing, yet there's no error in the log file. In talking to someone the users list it sounds like svn is actually just hanging. Clients get the following response: svn: Can't connect to host 'svn': No connection could be made because the target machine actively refused it. Our repository has 129K revisions. The format is 4 layout linear, it was created with svnadmin 1.4.x and has since had svnadmin upgrade run both in 1.5 and 1.6. We're currently running SlikSVN 1.6.13 (Win32), but I have previously had this problem dating back to versions of 1.5, both stock and from CollabNet. The issue now happens numerous times per day and it looks like I've discovered why As a test I ran svn blame -g on a file with a bunch of revisions and watched memory usage on the server spin up to 2GB. Chris, By a bunch of revisions what exactly do you mean? Many revisions which were the result of a merge? Or simply many changes made directly to the file (not the result of a merge)? Paul - I'll see if I can get a test repo up with the error. In the meantime, would a copy of the svn:mergeinfo help? Any luck? I should have asked this out of the gate (though I'm sure I know the answer): Is your repos public? Obviously the most direct route to replicating this would be with a copy of your actual data :-) Paul
[PATCH] To improve the documentation of svn_path.h
Hi All, I am a new-comer to the Subversion code-base. I was looking at some of the header files (svn_path.h) and wanted to use some of the functions (svn_path_join()) and noticed that they were deprecated. I thought it would be nice to have a link to the deprecating functions as a part of the documentation. Hence this patch. The log message and patch are as follows. [[[ Improved the documentation of deprecated functions by providing links to the deprecating functions. * subversion/include/svn_path.h (svn_path_internal_style, svn_path_local_style, svn_path_join, svn_path_join_many, svn_path_basename, svn_path_dirname, svn_path_split, svn_path_canonicalize, svn_path_is_canonical, svn_path_get_longest_ancestor, svn_path_get_absolute, svn_path_split, svn_path_condense_targets, svn_path_is_child, svn_path_is_ancestor, svn_path_split_if_file, svn_path_url_add_component) Patch by: Arwin Arni arwin_at_collab.net ]]] Index: subversion/include/svn_path.h === --- subversion/include/svn_path.h(revision 1032465) +++ subversion/include/svn_path.h(working copy) @@ -64,6 +64,10 @@ /** Convert @a path from the local style to the canonical internal style. + * + * New code should use either svn_dirent_internal_style() (for local paths) or + * svn_relpath_internal_style() (for relative paths). + * * @deprecated Provided for backward compatibility with the 1.6 API. */ SVN_DEPRECATED @@ -71,6 +75,10 @@ svn_path_internal_style(const char *path, apr_pool_t *pool); /** Convert @a path from the canonical internal style to the local style. + * + * New code should use either svn_dirent_local_style() (for local paths) or + * svn_relpath_local_style() (for relative paths). + * * @deprecated Provided for backward compatibility with the 1.6 API. */ SVN_DEPRECATED @@ -101,6 +109,9 @@ * @a component won't be detected. An absolute URI can only be used * for the base. * + * New code should use either svn_dirent_join() (for local paths) or + * svn_uri_join() (for urls) or svn_relpath_join() (for relative paths). + * * @deprecated Provided for backward compatibility with the 1.6 API. */ SVN_DEPRECATED @@ -118,7 +129,9 @@ * This function does not support URLs. * * See svn_path_join() for further notes about joining paths. - * + * + * New code should use svn_dirent_join_many() instead. + * * @deprecated Provided for backward compatibility with the 1.6 API. */ SVN_DEPRECATED @@ -139,7 +152,7 @@ * @note If an empty string is passed, then an empty string will be returned. * * New code should use either svn_dirent_basename() (for local paths) or - * svn_uri_basename() (for urls and repository paths). + * svn_uri_basename() (for urls) or svn_relpath_basename (for relative paths). * * @deprecated Provided for backward compatibility with the 1.6 API. */ @@ -154,7 +167,7 @@ * The returned dirname will be allocated in @a pool. * * New code should use either svn_dirent_dirname() (for local paths) or - * svn_uri_dirname() (for urls and repository paths). + * svn_uri_dirname() (for urls) or svn_relpath_dirname() (for relative paths). * * @deprecated Provided for backward compatibility with the 1.6 API. */ @@ -227,7 +240,7 @@ * - pre == and /pre * * New code should use either svn_dirent_split() (for local paths) or - * svn_uri_split() (for urls and repository paths). + * svn_uri_split() (for urls) or svn_relpath_split() (for relative paths). * * @deprecated Provided for backward compatibility with the 1.6 API. */ @@ -266,6 +279,10 @@ * The returned path may be statically allocated, equal to @a path, or * allocated from @a pool. * + * New code should use either svn_dirent_canonicalize() (for local paths) or + * svn_uri_canonicalize() (for urls) or svn_relpath_canonicalize() (for + * relative paths). + * * @deprecated Provided for backward compatibility with the 1.6 API. */ SVN_DEPRECATED @@ -275,6 +292,10 @@ /** Return @c TRUE iff path is canonical. Use @a pool for temporary * allocations. * + * New code should use either svn_dirent_is_canonical() (for local paths) or + * svn_uri_is_canonical() (for urls) or svn_relpath_is_canonical() (for + * relative paths). + * * @since New in 1.5. * @deprecated Provided for backward compatibility with the 1.6 API. */ @@ -300,6 +321,10 @@ * different resources), and (b) share a common ancestor in their path * component, i.e. 'protocol://' is not a sufficient ancestor. * + * New code should use either svn_dirent_get_longest_ancestor() + * (for local paths) or svn_uri_get_longest_ancestor() (for urls) + * or svn_relpath_get_longest_ancestor() (for relative paths). + * * @deprecated Provided for backward compatibility with the 1.6 API. */ SVN_DEPRECATED @@ -314,6 +339,8 @@ * @a relative may be a URL, in which case no attempt is made to convert it, * and a copy of the
Re: [PATCH] To improve the documentation of svn_path.h
On Monday 08 November 2010 07:57 PM, Arwin Arni wrote: Hi All, I am a new-comer to the Subversion code-base. I was looking at some of the header files (svn_path.h) and wanted to use some of the functions (svn_path_join()) and noticed that they were deprecated. I thought it would be nice to have a link to the deprecating functions as a part of the documentation. Hence this patch. The log message and patch are as follows. [[[ Improved the documentation of deprecated functions by providing links to the deprecating functions. * subversion/include/svn_path.h (svn_path_internal_style, svn_path_local_style, svn_path_join, svn_path_join_many, svn_path_basename, svn_path_dirname, svn_path_split, svn_path_canonicalize, svn_path_is_canonical, svn_path_get_longest_ancestor, svn_path_get_absolute, svn_path_split, svn_path_condense_targets, svn_path_is_child, svn_path_is_ancestor, svn_path_split_if_file, svn_path_url_add_component) Patch by: Arwin Arni arwin_at_collab.net ]]] Index: subversion/include/svn_path.h === --- subversion/include/svn_path.h(revision 1032465) +++ subversion/include/svn_path.h(working copy) @@ -64,6 +64,10 @@ /** Convert @a path from the local style to the canonical internal style. + * + * New code should use either svn_dirent_internal_style() (for local paths) or + * svn_relpath_internal_style() (for relative paths). + * * @deprecated Provided for backward compatibility with the 1.6 API. */ SVN_DEPRECATED @@ -71,6 +75,10 @@ svn_path_internal_style(const char *path, apr_pool_t *pool); /** Convert @a path from the canonical internal style to the local style. + * + * New code should use either svn_dirent_local_style() (for local paths) or + * svn_relpath_local_style() (for relative paths). + * * @deprecated Provided for backward compatibility with the 1.6 API. */ SVN_DEPRECATED @@ -101,6 +109,9 @@ * @a component won't be detected. An absolute URI can only be used * for the base. * + * New code should use either svn_dirent_join() (for local paths) or + * svn_uri_join() (for urls) or svn_relpath_join() (for relative paths). + * * @deprecated Provided for backward compatibility with the 1.6 API. */ SVN_DEPRECATED @@ -118,7 +129,9 @@ * This function does not support URLs. * * See svn_path_join() for further notes about joining paths. - * + * + * New code should use svn_dirent_join_many() instead. + * * @deprecated Provided for backward compatibility with the 1.6 API. */ SVN_DEPRECATED @@ -139,7 +152,7 @@ * @note If an empty string is passed, then an empty string will be returned. * * New code should use either svn_dirent_basename() (for local paths) or - * svn_uri_basename() (for urls and repository paths). + * svn_uri_basename() (for urls) or svn_relpath_basename (for relative paths). * * @deprecated Provided for backward compatibility with the 1.6 API. */ @@ -154,7 +167,7 @@ * The returned dirname will be allocated in @a pool. * * New code should use either svn_dirent_dirname() (for local paths) or - * svn_uri_dirname() (for urls and repository paths). + * svn_uri_dirname() (for urls) or svn_relpath_dirname() (for relative paths). * * @deprecated Provided for backward compatibility with the 1.6 API. */ @@ -227,7 +240,7 @@ * - pre == and /pre * * New code should use either svn_dirent_split() (for local paths) or - * svn_uri_split() (for urls and repository paths). + * svn_uri_split() (for urls) or svn_relpath_split() (for relative paths). * * @deprecated Provided for backward compatibility with the 1.6 API. */ @@ -266,6 +279,10 @@ * The returned path may be statically allocated, equal to @a path, or * allocated from @a pool. * + * New code should use either svn_dirent_canonicalize() (for local paths) or + * svn_uri_canonicalize() (for urls) or svn_relpath_canonicalize() (for + * relative paths). + * * @deprecated Provided for backward compatibility with the 1.6 API. */ SVN_DEPRECATED @@ -275,6 +292,10 @@ /** Return @c TRUE iff path is canonical. Use @a pool for temporary * allocations. * + * New code should use either svn_dirent_is_canonical() (for local paths) or + * svn_uri_is_canonical() (for urls) or svn_relpath_is_canonical() (for + * relative paths). + * * @since New in 1.5. * @deprecated Provided for backward compatibility with the 1.6 API. */ @@ -300,6 +321,10 @@ * different resources), and (b) share a common ancestor in their path * component, i.e. 'protocol://' is not a sufficient ancestor. * + * New code should use either svn_dirent_get_longest_ancestor() + * (for local paths) or svn_uri_get_longest_ancestor() (for urls) + * or svn_relpath_get_longest_ancestor() (for relative paths). + * * @deprecated Provided for backward compatibility with the 1.6 API. */ SVN_DEPRECATED @@ -314,6 +339,8 @@ * @a relative may be a URL, in which case
Re: svn blame -g causing svnserve to hang mem usage to hit 2GB
On Mon, Nov 8, 2010 at 9:13 AM, Paul Burba ptbu...@gmail.com wrote: On Thu, Nov 4, 2010 at 8:59 PM, Chris Tashjian ct...@thepond.com wrote: I posted this on the users list, but I'm confident that this is a bug. Background: For a while now (off and on for over a year, but more frequently in the last 6+ months) we've been having problems with svn crashing, yet there's no error in the log file. In talking to someone the users list it sounds like svn is actually just hanging. Clients get the following response: svn: Can't connect to host 'svn': No connection could be made because the target machine actively refused it. Our repository has 129K revisions. The format is 4 layout linear, it was created with svnadmin 1.4.x and has since had svnadmin upgrade run both in 1.5 and 1.6. We're currently running SlikSVN 1.6.13 (Win32), but I have previously had this problem dating back to versions of 1.5, both stock and from CollabNet. The issue now happens numerous times per day and it looks like I've discovered why As a test I ran svn blame -g on a file with a bunch of revisions and watched memory usage on the server spin up to 2GB. Chris, By a bunch of revisions what exactly do you mean? Many revisions which were the result of a merge? Or simply many changes made directly to the file (not the result of a merge)? Paul - I'll see if I can get a test repo up with the error. In the meantime, would a copy of the svn:mergeinfo help? Any luck? Chris, Don't sweat the replication effort too much, I think I'm on the trail of this problem. Using a copy of the old (pre-ASF) Subversion repository I'm seeing unexpectedly high memory use when using log -g on a file with a lot of merge history: 1.6.13.clientsvn blame ^/trunk/subversion/tests/cmdline/merge_tests.py 25 MB Peak Working Set Memory svnserve.exe 1.6.13.clientsvn blame ^/trunk/subversion/tests/cmdline/merge_tests.py -g 291 MB Peak Working Set Memory svnserve.exe More soon... Paul
RE: How to get the message out (or: why there were only 6 people at the ApacheCon meetup)
-Original Message- From: hy...@hyrumwright.org [mailto:hy...@hyrumwright.org] On Behalf Of Hyrum K. Wright Sent: 07 November 2010 02:34 To: Subversion Development Subject: How to get the message out (or: why there were only 6 people at the ApacheCon meetup) [snip] My question: do we need to do this? Is there a niche that isn't being filled by the various corporate sponsors, which we (as a Subversion development community) are better served to do? Do people already know enough about Subversion, such that we don't need to evangelize anymore? I'm sure evangelical missions are still important - mainly to counter the SVN is crap at merging, Mercurial/Git/a.n.otherDVCS is the ultimate SCM that fixes all your problems type arguments that seems to be increasingly popular. Even our Serena dimensions admin (terrible 'enterprisey' ALM SCM) knows about git but doesn't know what SVN'd give him. So, I'd say there are still loads of poor souls needing the word bringing to them, and the FUD dispelling.
Re: [PATCH] To improve the documentation of svn_path.h
[[[
Improved the documentation of deprecated functions by providing links
to the deprecating functions.
* subversion/include/svn_path.h
(svn_path_internal_style, svn_path_local_style,
svn_path_join, svn_path_join_many,
svn_path_basename, svn_path_dirname,
svn_path_split, svn_path_canonicalize,
svn_path_is_canonical, svn_path_get_longest_ancestor,
svn_path_get_absolute, svn_path_split,
svn_path_condense_targets, svn_path_is_child,
svn_path_is_ancestor, svn_path_split_if_file,
Thanks Arwin for the patch 'svn_path_split_if_file' is deprectated but
svn_{uri|dirent|relpath}_split do not seem like its deprecating
function. So removed that portion from your patch.
Committed the patch at r1032610.
With regards
Kamesh Jayachandran
svn_path_url_add_component)
[PATCH] Fix Windows test failures - Python test suite bug in quoting a trailing backslash
Here's a patch to fix the Windows buildbot failures that started at my r1031610. The problem is our _quote_arg() function doesn't do the right thing with a trailing backslash. For a solution it seems better to simply pass the args separately to subprocess.Popen() and let it do the quoting, than to try to fix and maintain our own quoting function. And we can use the undocumented subprocess.list2cmdline() function for generating a command-line string for display purposes. (We could also use that for generating a single-string argument to Popen, but we don't need to because it does it for us if we pass a list of arguments.) I tested this in a WinXP VM, and it seemed to work properly with Python 2.4.1 and 2.4.3 and 2.7. (I ran 1.7-trunk's prop_tests.py 7 against an installed svn 1.6.13, and it passed.) [[[ Fix Windows command-line argument quoting in the Python test harness. Arguments ending with a backslash were not correctly quoted. This reverts r875257. * subversion/tests/cmdline/svntest/main.py (_safe_arg_re, _quote_arg): Delete, as this didn't quote properly. (open_pipe): Pass the list of arguments directly to subprocess.Popen() instead of trying to quote it ourselves. Use subprocess.list2cmdline() to generate a command line for display. (spawn_process): Use subprocess.list2cmdline() instead of _quote_arg() to generate a command line for display. ]]] It reverts Paul's r875257, which was: [[[ Fix test suite's use of subprocess on earlier versions of Python and thus fix the djh-xp-vse2005 buildbot. * subversion/tests/cmdline/svntest/main.py (open_pipe2): Quote arguments to subprocess.Popen(). Later versions of subprocess (2.5.2+ that I know of for sure) don't require this quoting, but versions 2.4.3 do. ]]] I was unable to see exactly what the problem was that r875257 fixed. I would like to be able to say why it's OK to revert it, but I don't yet know. Can Paul or anyone comment or test or review? Or shall I just commit this current patch and see if anyone has problems? - Julian
Re: [PATCH] Fix Windows test failures - Python test suite bug in quoting a trailing backslash
Now with the patch attached.
And here's the email thread about the original change r875257 (aka
r35183): http://svn.haxx.se/dev/archive-2009-01/0316.shtml.
On Mon, 2010-11-08 at 16:20 +, Julian Foad wrote:
Here's a patch to fix the Windows buildbot failures that started at my
r1031610. The problem is our _quote_arg() function doesn't do the right
thing with a trailing backslash. For a solution it seems better to
simply pass the args separately to subprocess.Popen() and let it do the
quoting, than to try to fix and maintain our own quoting function. And
we can use the undocumented subprocess.list2cmdline() function for
generating a command-line string for display purposes. (We could also
use that for generating a single-string argument to Popen, but we don't
need to because it does it for us if we pass a list of arguments.)
I tested this in a WinXP VM, and it seemed to work properly with Python
2.4.1 and 2.4.3 and 2.7. (I ran 1.7-trunk's prop_tests.py 7 against an
installed svn 1.6.13, and it passed.)
[[[
Fix Windows command-line argument quoting in the Python test harness.
Arguments ending with a backslash were not correctly quoted.
This reverts r875257.
* subversion/tests/cmdline/svntest/main.py
(_safe_arg_re, _quote_arg): Delete, as this didn't quote properly.
(open_pipe): Pass the list of arguments directly to subprocess.Popen()
instead of trying to quote it ourselves. Use subprocess.list2cmdline()
to generate a command line for display.
(spawn_process): Use subprocess.list2cmdline() instead of _quote_arg()
to generate a command line for display.
]]]
It reverts Paul's r875257, which was:
[[[
Fix test suite's use of subprocess on earlier versions of Python and thus
fix the djh-xp-vse2005 buildbot.
* subversion/tests/cmdline/svntest/main.py
(open_pipe2): Quote arguments to subprocess.Popen(). Later versions of
subprocess (2.5.2+ that I know of for sure) don't require this quoting, but
versions 2.4.3 do.
]]]
I was unable to see exactly what the problem was that r875257 fixed. I
would like to be able to say why it's OK to revert it, but I don't yet
know.
Can Paul or anyone comment or test or review?
Or shall I just commit this current patch and see if anyone has
problems?
- Julian
Fix Windows command-line argument quoting in the Python test harness.
Arguments ending with a backslash were not correctly quoted.
This reverts r875257.
* subversion/tests/cmdline/svntest/main.py
(_safe_arg_re, _quote_arg): Delete, as this didn't quote properly.
(open_pipe): Pass the list of arguments directly to subprocess.Popen()
instead of trying to quote it ourselves. Use subprocess.list2cmdline()
to generate a command line for display.
(spawn_process): Use subprocess.list2cmdline() instead of _quote_arg()
to generate a command line for display.
--This line, and those below, will be ignored--
Index: subversion/tests/cmdline/svntest/main.py
===
--- subversion/tests/cmdline/svntest/main.py (revision 1032502)
+++ subversion/tests/cmdline/svntest/main.py (working copy)
@@ -332,38 +332,12 @@ def run_command(command, error_expected,
as lists of lines (including line terminators). See run_command_stdin()
for details. If ERROR_EXPECTED is None, any stderr also will be printed.
return run_command_stdin(command, error_expected, 0, binary_mode,
None, *varargs)
-# A regular expression that matches arguments that are trivially safe
-# to pass on a command line without quoting on any supported operating
-# system:
-_safe_arg_re = re.compile(r'^[A-Za-z\d\.\_\/\-\:\...@]+$')
-
-def _quote_arg(arg):
- Quote ARG for a command line.
-
- Simply surround every argument in double-quotes unless it contains
- only universally harmless characters.
-
- WARNING: This function cannot handle arbitrary command-line
- arguments. It can easily be confused by shell metacharacters. A
- perfect job would be difficult and OS-dependent (see, for example,
- http://msdn.microsoft.com/library/en-us/vccelng/htm/progs_12.asp).
- In other words, this function is just good enough for what we need
- here.
-
- arg = str(arg)
- if _safe_arg_re.match(arg):
-return arg
- else:
-if os.name != 'nt':
- arg = arg.replace('$', '\$')
-return '%s' % (arg,)
-
def open_pipe(command, bufsize=0, stdin=None, stdout=None, stderr=None):
Opens a subprocess.Popen pipe to COMMAND using STDIN,
STDOUT, and STDERR. BUFSIZE is passed to subprocess.Popen's
argument of the same name.
Returns (infile, outfile, errfile, waiter); waiter
@@ -372,21 +346,13 @@ def open_pipe(command, bufsize=0, stdin=
# On Windows subprocess.Popen() won't accept a Python script as
# a valid program to execute, rather it wants the Python executable.
if (sys.platform == 'win32') and (command[0].endswith('.py')):
command.insert(0,
Re: [PATCH] error leak on performance branch
On Mon, Nov 08, 2010 at 06:56:16AM -0600, Hyrum K. Wright wrote:
On Sun, Nov 7, 2010 at 11:58 AM, Stefan Sperling s...@elego.de wrote:
[[[
* subversion/libsvn_fs_util/caching.c
(svn_fs__get_global_membuffer_cache): Fix a possible error leak.
]]]
Index: subversion/libsvn_fs_util/caching.c
===
--- subversion/libsvn_fs_util/caching.c (revision 1032308)
+++ subversion/libsvn_fs_util/caching.c (working copy)
@@ -62,6 +62,7 @@ svn_fs__get_global_membuffer_cache(void)
/* auto-allocate cache*/
apr_allocator_t *allocator = NULL;
apr_pool_t *pool = NULL;
+ svn_error_t *err;
if (apr_allocator_create(allocator))
return NULL;
@@ -75,12 +76,17 @@ svn_fs__get_global_membuffer_cache(void)
apr_allocator_max_free_set(allocator, 1);
pool = svn_pool_create_ex(NULL, allocator);
- svn_cache__membuffer_cache_create
- (cache,
- (apr_size_t)cache_size,
- (apr_size_t)(cache_size / 16),
- ! svn_fs_get_cache_config()-single_threaded,
- pool);
If we're choosing to ignore the error above, you can just wrap the
entire call in an svn_error_clear(). No need to introduce and check a
temp variable.
Ah, nice trick. Thanks.
+ err = svn_cache__membuffer_cache_create
+ (cache,
+ (apr_size_t)cache_size,
+ (apr_size_t)(cache_size / 16),
+ ! svn_fs_get_cache_config()-single_threaded,
+ pool);
+ if (err)
+ {
+ svn_error_clear(err);
+ return NULL;
+ }
Does this early return actually change functionality? If so, this
patch is about more than just fixing an error leak...
It doesn't change functionality.
Cache is NULL in this case, and we return it next:
}
return cache;
I'm not sure though if svn_cache__membuffer_cache_create() guarantees
that cache remains NULL in case of error. Maybe this should be
documented as such to allow callers to ignore errors this way.
New patch below, also fixing a space-before-paren and another
similar error leak.
[[[
* subversion/libsvn_fs_util/caching.c
(svn_fs__get_global_membuffer_cache,
svn_fs__get_global_file_handle_cache): Fix a possible error leak.
]]]
Index: subversion/libsvn_fs_util/caching.c
===
--- subversion/libsvn_fs_util/caching.c (revision 1032333)
+++ subversion/libsvn_fs_util/caching.c (working copy)
@@ -75,12 +75,12 @@ svn_fs__get_global_membuffer_cache(void)
apr_allocator_max_free_set(allocator, 1);
pool = svn_pool_create_ex(NULL, allocator);
- svn_cache__membuffer_cache_create
- (cache,
- (apr_size_t)cache_size,
- (apr_size_t)(cache_size / 16),
- ! svn_fs_get_cache_config()-single_threaded,
- pool);
+ svn_error_clear(svn_cache__membuffer_cache_create(
+cache,
+(apr_size_t)cache_size,
+(apr_size_t)(cache_size / 16),
+! svn_fs_get_cache_config()-single_threaded,
+pool));
}
return cache;
@@ -96,10 +96,10 @@ svn_fs__get_global_file_handle_cache(void)
static svn_file_handle_cache_t *cache = NULL;
if (!cache)
-svn_file_handle_cache__create_cache(cache,
-cache_settings.file_handle_count,
-!cache_settings.single_threaded,
-svn_pool_create(NULL));
+svn_error_clear(svn_file_handle_cache__create_cache(
+ cache, cache_settings.file_handle_count,
+ !cache_settings.single_threaded,
+ svn_pool_create(NULL)));
return cache;
}
Re: [PATCH] Fix Windows test failures - Python test suite bug in quoting a trailing backslash
On Mon, Nov 8, 2010 at 11:20 AM, Julian Foad julian.f...@wandisco.com wrote: Here's a patch to fix the Windows buildbot failures that started at my r1031610. The problem is our _quote_arg() function doesn't do the right thing with a trailing backslash. For a solution it seems better to simply pass the args separately to subprocess.Popen() and let it do the quoting, than to try to fix and maintain our own quoting function. And we can use the undocumented subprocess.list2cmdline() function for generating a command-line string for display purposes. (We could also use that for generating a single-string argument to Popen, but we don't need to because it does it for us if we pass a list of arguments.) I tested this in a WinXP VM, and it seemed to work properly with Python 2.4.1 and 2.4.3 and 2.7. (I ran 1.7-trunk's prop_tests.py 7 against an installed svn 1.6.13, and it passed.) [[[ Fix Windows command-line argument quoting in the Python test harness. Arguments ending with a backslash were not correctly quoted. This reverts r875257. * subversion/tests/cmdline/svntest/main.py (_safe_arg_re, _quote_arg): Delete, as this didn't quote properly. (open_pipe): Pass the list of arguments directly to subprocess.Popen() instead of trying to quote it ourselves. Use subprocess.list2cmdline() to generate a command line for display. (spawn_process): Use subprocess.list2cmdline() instead of _quote_arg() to generate a command line for display. ]]] It reverts Paul's r875257, which was: [[[ Fix test suite's use of subprocess on earlier versions of Python and thus fix the djh-xp-vse2005 buildbot. * subversion/tests/cmdline/svntest/main.py (open_pipe2): Quote arguments to subprocess.Popen(). Later versions of subprocess (2.5.2+ that I know of for sure) don't require this quoting, but versions 2.4.3 do. ]]] I was unable to see exactly what the problem was that r875257 fixed. I would like to be able to say why it's OK to revert it, but I don't yet know. Can Paul or anyone comment or test or review? Or shall I just commit this current patch and see if anyone has problems? Hi Julian, +1 on commit this current patch and see if anyone has problems. You've confirmed it works on Windows with 2.4.1, 2.4.3 and 2.7 (and 2.4 is the minimum we claim to support). I can confirm it works with 2.6.5. Paul
Re: svn blame -g causing svnserve to hang mem usage to hit 2GB
On Mon, Nov 8, 2010 at 10:35 AM, Paul Burba ptbu...@gmail.com wrote: On Mon, Nov 8, 2010 at 9:13 AM, Paul Burba ptbu...@gmail.com wrote: On Thu, Nov 4, 2010 at 8:59 PM, Chris Tashjian ct...@thepond.com wrote: I posted this on the users list, but I'm confident that this is a bug. Background: For a while now (off and on for over a year, but more frequently in the last 6+ months) we've been having problems with svn crashing, yet there's no error in the log file. In talking to someone the users list it sounds like svn is actually just hanging. Clients get the following response: svn: Can't connect to host 'svn': No connection could be made because the target machine actively refused it. Our repository has 129K revisions. The format is 4 layout linear, it was created with svnadmin 1.4.x and has since had svnadmin upgrade run both in 1.5 and 1.6. We're currently running SlikSVN 1.6.13 (Win32), but I have previously had this problem dating back to versions of 1.5, both stock and from CollabNet. The issue now happens numerous times per day and it looks like I've discovered why As a test I ran svn blame -g on a file with a bunch of revisions and watched memory usage on the server spin up to 2GB. Chris, By a bunch of revisions what exactly do you mean? Many revisions which were the result of a merge? Or simply many changes made directly to the file (not the result of a merge)? Paul - I'll see if I can get a test repo up with the error. In the meantime, would a copy of the svn:mergeinfo help? Any luck? Chris, Don't sweat the replication effort too much, I think I'm on the trail of this problem. Using a copy of the old (pre-ASF) Subversion repository I'm seeing unexpectedly high memory use when using log -g on a file with a lot of merge history: 1.6.13.clientsvn blame ^/trunk/subversion/tests/cmdline/merge_tests.py 25 MB Peak Working Set Memory svnserve.exe 1.6.13.clientsvn blame ^/trunk/subversion/tests/cmdline/merge_tests.py -g 291 MB Peak Working Set Memory svnserve.exe ^^^ That should have been 691 MB! Sorry! More soon... Paul
Re: [PATCH] Fix Windows test failures - Python test suite bug in quoting a trailing backslash
On Mon, Nov 8, 2010 at 12:05 PM, Paul Burba ptbu...@gmail.com wrote: On Mon, Nov 8, 2010 at 11:20 AM, Julian Foad julian.f...@wandisco.com wrote: Here's a patch to fix the Windows buildbot failures that started at my r1031610. The problem is our _quote_arg() function doesn't do the right thing with a trailing backslash. For a solution it seems better to simply pass the args separately to subprocess.Popen() and let it do the quoting, than to try to fix and maintain our own quoting function. And we can use the undocumented subprocess.list2cmdline() function for generating a command-line string for display purposes. (We could also use that for generating a single-string argument to Popen, but we don't need to because it does it for us if we pass a list of arguments.) I tested this in a WinXP VM, and it seemed to work properly with Python 2.4.1 and 2.4.3 and 2.7. (I ran 1.7-trunk's prop_tests.py 7 against an installed svn 1.6.13, and it passed.) [[[ Fix Windows command-line argument quoting in the Python test harness. Arguments ending with a backslash were not correctly quoted. This reverts r875257. * subversion/tests/cmdline/svntest/main.py (_safe_arg_re, _quote_arg): Delete, as this didn't quote properly. (open_pipe): Pass the list of arguments directly to subprocess.Popen() instead of trying to quote it ourselves. Use subprocess.list2cmdline() to generate a command line for display. (spawn_process): Use subprocess.list2cmdline() instead of _quote_arg() to generate a command line for display. ]]] It reverts Paul's r875257, which was: [[[ Fix test suite's use of subprocess on earlier versions of Python and thus fix the djh-xp-vse2005 buildbot. * subversion/tests/cmdline/svntest/main.py (open_pipe2): Quote arguments to subprocess.Popen(). Later versions of subprocess (2.5.2+ that I know of for sure) don't require this quoting, but versions 2.4.3 do. ]]] I was unable to see exactly what the problem was that r875257 fixed. I would like to be able to say why it's OK to revert it, but I don't yet know. Can Paul or anyone comment or test or review? Or shall I just commit this current patch and see if anyone has problems? Hi Julian, +1 on commit this current patch and see if anyone has problems. You've confirmed it works on Windows with 2.4.1, 2.4.3 and 2.7 (and 2.4 is the minimum we claim to support). I can confirm it works with 2.6.5. Sorry Julian, scratch that last bit. I'm not sure what I did, but I obviously ran the wrong thing (too many trunk WCs apparently!). Running the tests for another change with your patch still in place I noticed a lot of failures. Reverting my changes and running with just your patch I still saw lots of problems, most of which are like this failure of commit_tests.py 41: [[[ CMD: svnadmin.exe create svn-test-work\repositories\commit_tests-41 --bdb-txn-nosync --fs-type=fsfs TIME = 0.094000 CMD: svnadmin.exe dump svn-test-work\local_tmp\repos | svnadmin.exe load svn-test-work\repositories\commit_tests-41 --ignore-uuid TIME = 0.002000 CMD: svn.exe co file:///C:/SVN/src-trunk-2/Release/subversion/tests/cmdline/svn-test-work/repositories/commit_tests-41 svn-test-work\working_copies\commit_tests-41 --config-dir C:\SVN\src-trunk-2\Release\subversion\tests\cmdline\svn-test-work\local_tmp\config --password rayjandom --no-auth-cache --username jrandom TIME = 0.197000 Asvn-test-work\working_copies\commit_tests-41\A Asvn-test-work\working_copies\commit_tests-41\A\B Asvn-test-work\working_copies\commit_tests-41\A\B\lambda Asvn-test-work\working_copies\commit_tests-41\A\B\E Asvn-test-work\working_copies\commit_tests-41\A\B\E\alpha Asvn-test-work\working_copies\commit_tests-41\A\B\E\beta Asvn-test-work\working_copies\commit_tests-41\A\B\F Asvn-test-work\working_copies\commit_tests-41\A\mu Asvn-test-work\working_copies\commit_tests-41\A\C Asvn-test-work\working_copies\commit_tests-41\A\D Asvn-test-work\working_copies\commit_tests-41\A\D\gamma Asvn-test-work\working_copies\commit_tests-41\A\D\G Asvn-test-work\working_copies\commit_tests-41\A\D\G\pi Asvn-test-work\working_copies\commit_tests-41\A\D\G\rho Asvn-test-work\working_copies\commit_tests-41\A\D\G\tau Asvn-test-work\working_copies\commit_tests-41\A\D\H Asvn-test-work\working_copies\commit_tests-41\A\D\H\chi Asvn-test-work\working_copies\commit_tests-41\A\D\H\omega Asvn-test-work\working_copies\commit_tests-41\A\D\H\psi Asvn-test-work\working_copies\commit_tests-41\iota Checked out revision 1. CMD: svn.exe mkdir -m msg --with-revprop bug=42 file:///C:/SVN/src-trunk-2/Release/subversion/tests/cmdline/svn-test-work/repositories/commit_tests-41/dir --config-dir C:\SVN\src-trunk-2\Release\subversion\tests\cmdline\svn-test-work\local_tmp\config --password rayjandom --no-auth-cache --username jrandom TIME = 0.201000 Committed revision 2. UNEXPECTED EXCEPTION: Traceback (most recent call
Re: svn blame -g causing svnserve to hang mem usage to hit 2GB
On Mon, Nov 8, 2010 at 1:23 PM, Paul Burba ptbu...@gmail.com wrote:
On Mon, Nov 8, 2010 at 10:35 AM, Paul Burba ptbu...@gmail.com wrote:
On Mon, Nov 8, 2010 at 9:13 AM, Paul Burba ptbu...@gmail.com wrote:
On Thu, Nov 4, 2010 at 8:59 PM, Chris Tashjian ct...@thepond.com wrote:
I posted this on the users list, but I'm confident that this is a bug.
Background:
For a while now (off and on for over a year, but more frequently in the
last
6+ months) we've been having problems with svn crashing, yet there's no
error in the log file. In talking to someone the users list it sounds like
svn is actually just hanging. Clients get the following response:
svn: Can't connect to host 'svn': No connection could be made
because the target machine actively refused it.
Our repository has 129K revisions. The format is 4 layout linear, it was
created with svnadmin 1.4.x and has since had svnadmin upgrade run both
in
1.5 and 1.6. We're currently running SlikSVN 1.6.13 (Win32), but I have
previously had this problem dating back to versions of 1.5, both stock and
from CollabNet. The issue now happens numerous times per day and it looks
like I've discovered why
As a test I ran svn blame -g on a file with a bunch of revisions and
watched memory usage on the server spin up to 2GB.
Chris,
By a bunch of revisions what exactly do you mean? Many revisions
which were the result of a merge? Or simply many changes made
directly to the file (not the result of a merge)?
Paul - I'll see if I can get a test repo up with the error. In the
meantime, would a copy of the svn:mergeinfo help?
Any luck?
Chris,
Don't sweat the replication effort too much, I think I'm on the trail
of this problem. Using a copy of the old (pre-ASF) Subversion
repository I'm seeing unexpectedly high memory use when using log -g
on a file with a lot of merge history:
1.6.13.clientsvn blame ^/trunk/subversion/tests/cmdline/merge_tests.py
25 MB Peak Working Set Memory svnserve.exe
1.6.13.clientsvn blame ^/trunk/subversion/tests/cmdline/merge_tests.py -g
291 MB Peak Working Set Memory svnserve.exe
^^^
That should have been 691 MB! Sorry!
More soon...
Hi Chris,
Ok, I think I got it. Switching to a Subversion tr...@1032651 (debug)
build and using this repository as a test*:
http://svn.collab.net/tmp/subversion-data-backup/subversion-history-20091115.tar.bz2
The peak working set memory of
'svn blame ^/trunk/subversion/tests/cmdline/merge_tests.py' was 21 MB
'svn blame ^/trunk/subversion/tests/cmdline/merge_tests.py -g' was 574 MB
That's comparable to what you saw with 1.6.13.
Applying some standard pool-fu to
libsvn_repos/rev_hunt.c:(find_merged_revisions|find_interesting_revisions),
see attached patch, and things look a *lot* better:
The peak working set memory of
'svn blame ^/trunk/subversion/tests/cmdline/merge_tests.py' stays at 21 MB
'svn blame ^/trunk/subversion/tests/cmdline/merge_tests.py -g'
drops to 71 MB
A tidy 88% reduction in peak memory usage!
Running the test suite on this patch. Assuming no problems I will be
committing and nominating for backport to 1.6.x.
log:
[[[
Fix blame -g server-side memory leaks.
See http://svn.haxx.se/dev/archive-2010-11/0102.shtml
* subversion/libsvn_repos/rev_hunt.c
(find_interesting_revisions): Implement result/scratch two-pool paradigm.
Don't needlessly allocate path_revision structures until we are sure
we are going to keep it. Rename local variable iter_pool to the
de facto standard iterpool.
(find_merged_revisions): Use a separate iterpool for each nested loop.
Update call to find_interesting_revisions, passing, you guessed it, an
iterpool. Rename local variable iter_pool to the de facto standard
iterpool.
]]]
Paul
* This is the old Tigris Subversion repository, see
http://svn.apache.org/repos/asf/subversion/README.
Index: subversion/libsvn_repos/rev_hunt.c
===
--- subversion/libsvn_repos/rev_hunt.c (revision 1032800)
+++ subversion/libsvn_repos/rev_hunt.c (working copy)
@@ -1085,47 +1085,49 @@
apr_hash_t *duplicate_path_revs,
svn_repos_authz_func_t authz_read_func,
void *authz_read_baton,
- apr_pool_t *pool)
+ apr_pool_t *result_pool,
+ apr_pool_t *scratch_pool)
{
- apr_pool_t *iter_pool, *last_pool;
+ apr_pool_t *iterpool, *last_pool;
svn_fs_history_t *history;
svn_fs_root_t *root;
svn_node_kind_t kind;
/* We switch between two pools while looping, since we need information from
the last iteration to be available. */
- iter_pool = svn_pool_create(pool);
- last_pool = svn_pool_create(pool);
+ iterpool = svn_pool_create(result_pool);
+ last_pool = svn_pool_create(result_pool);
/* The path had better be a file in this revision. */
-
Re: svn blame -g causing svnserve to hang mem usage to hit 2GB
On 11/8/2010 8:54 PM, Paul Burba wrote: On Mon, Nov 8, 2010 at 8:28 PM, Chris Tashjianct...@thepond.com wrote: Paul - 1 more question. What happens when you run additional blame -g commands? Does the memory usage keep growing until it runs out of memory or will it cap itself at some point? In my tests it seems like memory wasn't getting released very often. Granted, memory usage spun up so fast, I'm not positive that it really had much of a chance for that to happen. Chris, I was describing *peak* memory use. The actual working set memory drops from 71 MB to 7 MB once the blame -g is processed in my example. Not sure how familiar you are with APR pools, but all the pools that get allocated to process the request should be destroyed (freed) when the server is done. If memory use is growing slowly after each blame -g, then that isn't happening and we have a separate bug. I just did a little ad hoc testing and there doesn't appear to be a problem. After several blame -g hits, the peak working set for my svnserve.exe process always returns to around 7 MB: 7464 K 7652 K 7500 K 7596 K If you discover anything contrary to this please let us know on the dev list. I just restarted our svn server and ran blame -g against a file. No one else is hitting the server right now and after 5 minutes mem usage is at 681,208K and the peak was 681,316K. (taskmgr screenshot: http://img215.imageshack.us/img215/2060/svnserve.jpg)
Re: svn blame -g causing svnserve to hang mem usage to hit 2GB
On Mon, Nov 8, 2010 at 8:17 PM, Paul Burba ptbu...@gmail.com wrote: On Mon, Nov 8, 2010 at 1:23 PM, Paul Burba ptbu...@gmail.com wrote: On Mon, Nov 8, 2010 at 10:35 AM, Paul Burba ptbu...@gmail.com wrote: On Mon, Nov 8, 2010 at 9:13 AM, Paul Burba ptbu...@gmail.com wrote: On Thu, Nov 4, 2010 at 8:59 PM, Chris Tashjian ct...@thepond.com wrote: I posted this on the users list, but I'm confident that this is a bug. Background: For a while now (off and on for over a year, but more frequently in the last 6+ months) we've been having problems with svn crashing, yet there's no error in the log file. In talking to someone the users list it sounds like svn is actually just hanging. Clients get the following response: svn: Can't connect to host 'svn': No connection could be made because the target machine actively refused it. Our repository has 129K revisions. The format is 4 layout linear, it was created with svnadmin 1.4.x and has since had svnadmin upgrade run both in 1.5 and 1.6. We're currently running SlikSVN 1.6.13 (Win32), but I have previously had this problem dating back to versions of 1.5, both stock and from CollabNet. The issue now happens numerous times per day and it looks like I've discovered why As a test I ran svn blame -g on a file with a bunch of revisions and watched memory usage on the server spin up to 2GB. Chris, By a bunch of revisions what exactly do you mean? Many revisions which were the result of a merge? Or simply many changes made directly to the file (not the result of a merge)? Paul - I'll see if I can get a test repo up with the error. In the meantime, would a copy of the svn:mergeinfo help? Any luck? Chris, Don't sweat the replication effort too much, I think I'm on the trail of this problem. Using a copy of the old (pre-ASF) Subversion repository I'm seeing unexpectedly high memory use when using log -g on a file with a lot of merge history: 1.6.13.clientsvn blame ^/trunk/subversion/tests/cmdline/merge_tests.py 25 MB Peak Working Set Memory svnserve.exe 1.6.13.clientsvn blame ^/trunk/subversion/tests/cmdline/merge_tests.py -g 291 MB Peak Working Set Memory svnserve.exe ^^^ That should have been 691 MB! Sorry! More soon... Hi Chris, Ok, I think I got it. Switching to a Subversion tr...@1032651 (debug) build and using this repository as a test*: http://svn.collab.net/tmp/subversion-data-backup/subversion-history-20091115.tar.bz2 The peak working set memory of 'svn blame ^/trunk/subversion/tests/cmdline/merge_tests.py' was 21 MB 'svn blame ^/trunk/subversion/tests/cmdline/merge_tests.py -g' was 574 MB That's comparable to what you saw with 1.6.13. Applying some standard pool-fu to libsvn_repos/rev_hunt.c:(find_merged_revisions|find_interesting_revisions), see attached patch, and things look a *lot* better: The peak working set memory of 'svn blame ^/trunk/subversion/tests/cmdline/merge_tests.py' stays at 21 MB 'svn blame ^/trunk/subversion/tests/cmdline/merge_tests.py -g' drops to 71 MB A tidy 88% reduction in peak memory usage! Running the test suite on this patch. Assuming no problems I will be committing and nominating for backport to 1.6.x. Nominated r1032808 for backport to 1.6.14.
Re: svn blame -g causing svnserve to hang mem usage to hit 2GB
On Mon, Nov 8, 2010 at 9:09 PM, Chris Tashjian ct...@thepond.com wrote: On 11/8/2010 8:54 PM, Paul Burba wrote: On Mon, Nov 8, 2010 at 8:28 PM, Chris Tashjianct...@thepond.com wrote: Paul - 1 more question. What happens when you run additional blame -g commands? Does the memory usage keep growing until it runs out of memory or will it cap itself at some point? In my tests it seems like memory wasn't getting released very often. Granted, memory usage spun up so fast, I'm not positive that it really had much of a chance for that to happen. Chris, I was describing *peak* memory use. The actual working set memory drops from 71 MB to 7 MB once the blame -g is processed in my example. Not sure how familiar you are with APR pools, but all the pools that get allocated to process the request should be destroyed (freed) when the server is done. If memory use is growing slowly after each blame -g, then that isn't happening and we have a separate bug. I just did a little ad hoc testing and there doesn't appear to be a problem. After several blame -g hits, the peak working set for my svnserve.exe process always returns to around 7 MB: 7464 K 7652 K 7500 K 7596 K If you discover anything contrary to this please let us know on the dev list. I just restarted our svn server and ran blame -g against a file. No one else is hitting the server right now and after 5 minutes mem usage is at 681,208K and the peak was 681,316K. (taskmgr screenshot: http://img215.imageshack.us/img215/2060/svnserve.jpg) I just checked with a trunk build prior to r1032808 it while the memory leak manifests itself while processing the blame -g request (peak memory at 571 MB), the peak working set returns to 9 MB once the blame is processed. So it's not a problem on trunk. I do see what you are describing on 1.6.x though. Working memory peaks at 591 MB and stays there! With r1032808 memory only peaks at 91 MB, but stay there...so yup there is another bug. But it will have to wait till tomorrow for me :-) Paul
Re: [Patch] Make svn_tristate_t compatible with svn_boolean_t
On Thu, Nov 4, 2010 at 08:45, Hyrum K. Wright hyrum_wri...@mail.utexas.edu wrote: On Thu, Nov 4, 2010 at 7:43 AM, Julian Foad julian.f...@wandisco.com wrote: ... Having said all that, +1 on removing the gratuitous inconsistency by applying this patch. Committed r1030909. Gah. Can we please wait a little bit longer on this kind of stuff to allow people in other timezones a chance to weigh in? I have raised this before, Julian. For making changes with some impact, where feedback from the community is desired, then the standard Apache rule is 72 hours. And even if we don't worry about rules, it is simply *respectful* to give others a chance to speak up. Last time, you bumped the format with something like FOUR hours notice. That was really bad. I'm not sure about the extent of the badness here (tho changing something as central as svn_types.h seems pretty obvious as an input-required change), but given that another member of the community has said woah. too quick, then you REALLY need to slow down. This isn't an attempt to slow down your work. This is an attempt (bordering on requirement) for you to work with the rest of the community on changes to our codebase. -g
Re: Trouble building trunk: Undefined references to svn_repos_load_fs3() and svn_repos_get_fs_build_parser3()
[Daniel Trebbien] I tried your suggestion of removing the subversion and libsvn1 packages (I'm running Debian Sid) FWIW, the only package you actually had to remove in order to prevent linker interference is libsvn-dev. That has the *.la files, *.so symlinks, etc. that are used at link time. Peter
