[PATCH] speed up svn_repos_authz_check_access

2013-08-07 Thread Roderich Schupp
Hi, this patch attempts to speed up svn_repos_authz_check_access, esp. when it is called repeatedly during the same HTTP request (or on the same connection). Subversion issues many HTTP requests that result only in a single call to svn_repos_authz_check_access (i.e. just for the path given in the

Re: svn commit: r1506576 - in /subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs: fs.h fs_fs.c low_level.c temp_serializer.c transaction.c

2013-08-07 Thread Daniel Shahaf
Stefan Fuhrmann wrote on Sun, Aug 04, 2013 at 14:32:14 +0200: On Sun, Aug 4, 2013 at 9:12 AM, Daniel Shahaf danie...@elego.de wrote: stef...@apache.org wrote on Wed, Jul 24, 2013 at 14:32:48 -: - const char *uniquifier; + intra-node uniqification content. */ + struct +

Re: svn commit: r1502964 - in /subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs: fs_fs.c fs_fs.h lock.c pack.c util.c util.h

2013-08-07 Thread Daniel Shahaf
Stefan Fuhrmann wrote on Sun, Aug 04, 2013 at 23:17:54 +0200: But keep in mind that it took me 2 full weeks with a diff viewer copying difference after difference and grouping them into somewhat manageable chunks. I would expect that the review (incl. documenting the findings) will certainly

Re: [PATCH] speed up svn_repos_authz_check_access

2013-08-07 Thread Daniel Shahaf
Please use text/* MIME type. The .c file was so but the .patch wasn't, and that gets in the way of reviewing it. More below... Roderich Schupp wrote on Wed, Aug 07, 2013 at 08:27:43 +0200: This patch reduces the cumulative time for svn_repos_authz_check_access (when called repeteadly in the

Re: [PATCH] speed up svn_repos_authz_check_access

2013-08-07 Thread Philip Martin
Roderich Schupp roderich.sch...@gmail.com writes: The patch needs to #include libsvn_subr/config_impl.h in order to gain access to svn_config_t.pool: the cache (apr_hash_t itself, keys and values) must be allocated from the same pool as svn_config_t so that they have the same lifespan. It

Re: [PATCH] speed up svn_repos_authz_check_access

2013-08-07 Thread Ivan Zhakov
On Wed, Aug 7, 2013 at 1:02 PM, Daniel Shahaf danie...@elego.de wrote: Please use text/* MIME type. The .c file was so but the .patch wasn't, and that gets in the way of reviewing it. More below... Roderich Schupp wrote on Wed, Aug 07, 2013 at 08:27:43 +0200: This patch reduces the

Re: [PATCH] speed up svn_repos_authz_check_access

2013-08-07 Thread Daniel Shahaf
Ivan Zhakov wrote on Wed, Aug 07, 2013 at 13:07:27 +0400: On Wed, Aug 7, 2013 at 1:02 PM, Daniel Shahaf danie...@elego.de wrote: Please use text/* MIME type. The .c file was so but the .patch wasn't, and that gets in the way of reviewing it. More below... Roderich Schupp wrote on Wed,

Re: man pages for Subversion

2013-08-07 Thread Stefan Sperling
On Wed, Aug 07, 2013 at 12:07:08AM -0400, James K. Lowden wrote: http://www.schemamania.org/svn/man1/ e.g. http://www.schemamania.org/svn/man1/svn-propset.pdf I offer the above mdoc man pages to the Subversion project as a token of my appreciation, and in the hope they'll be included in

Re: man pages for Subversion

2013-08-07 Thread Daniel Shahaf
Stefan Sperling wrote on Wed, Aug 07, 2013 at 11:27:13 +0200: On Wed, Aug 07, 2013 at 12:07:08AM -0400, James K. Lowden wrote: The pages represent perhaps 30 hours of work. The --help text was processed through the bin/manualize Perl script to create a rough mdoc draft. Then individual

Re: man pages for Subversion

2013-08-07 Thread Daniel Shahaf
James K. Lowden wrote on Wed, Aug 07, 2013 at 00:07:08 -0400: The final result is surely a derived work, but it's not clear to me if the --help text is copyrighted. I am prepare to assign it as befits the project, if need be. The ASF never requires copyright assignment. However, license

Re: [PATCH] speed up svn_repos_authz_check_access

2013-08-07 Thread Roderich Schupp
On Wed, Aug 7, 2013 at 11:05 AM, Philip Martin philip.mar...@wandisco.comwrote: +svn_boolean_t authz_use_cache = TRUE; /* FIXME devel only */ I assume that is temporary, what is the long-term plan? Remove it completely? Yes, this will go away. It's just there for benchmarking, so

Re: [PATCH] speed up svn_repos_authz_check_access

2013-08-07 Thread Roderich Schupp
On Wed, Aug 7, 2013 at 11:07 AM, Ivan Zhakov i...@visualsvn.com wrote: For example, when a client uses a long-lived keep-alive connection and the authz file changes during the life of that connection, will your change cause the changes to be picked up later than they would before your

Re: [PATCH] speed up svn_repos_authz_check_access

2013-08-07 Thread Roderich Schupp
On Wed, Aug 7, 2013 at 11:02 AM, Daniel Shahaf danie...@elego.de wrote: The patch needs to #include libsvn_subr/config_impl.h in order to gain access to svn_config_t.pool: the cache (apr_hash_t itself, keys and values) must be allocated from the same pool as svn_config_t so that they have

Re: man pages for Subversion

2013-08-07 Thread Daniel Shahaf
James K. Lowden wrote on Wed, Aug 07, 2013 at 00:07:08 -0400: I offer the above mdoc man pages to the Subversion project as a token of my appreciation, and in the hope they'll be included in future versions. Thanks! The pages represent perhaps 30 hours of work. The --help text was

Security patches release process

2013-08-07 Thread Daniel Shahaf
(See https://www.apache.org/security/committers for background) For CVE-2013-4131 our process was: (v1) - Receive report - Come up with a fix - Gather 3 +1 votes via shadow status file - Commit fix with innocent log message - Backport without going via STATUS - Tag and roll 1.8.(x+1) - In

Re: Security patches release process

2013-08-07 Thread Justin Erenkrantz
On Wed, Aug 7, 2013 at 5:48 AM, Daniel Shahaf danie...@elego.de wrote: (v3) - Receive report - Come up with a fix - Gather 3 +1 votes via shadow status file - Send pre-notifications - Release a signed .diff file (instead of a tarball) as 1.8.(x+1) - Commit fix with a log message clearly

Re: svn commit: r1507589 - in /subversion/trunk/subversion: libsvn_ra_serf/commit.c tests/cmdline/commit_tests.py

2013-08-07 Thread Philip Martin
bre...@apache.org writes: Author: breser Date: Sat Jul 27 07:08:03 2013 New Revision: 1507589 URL: http://svn.apache.org/r1507589 Log: Fix Issue #4400: commits fail with copy and deep delete under copy. * subversion/libsvn_ra_serf/commit.c (checkout_dir): Generate the working_url when

Re: [PATCH] speed up svn_repos_authz_check_access

2013-08-07 Thread Roderich Schupp
Hi, revised patch attached, hopefully addressing all concerns. No more breaking encapsulation. We simply allocate svn_authz_t from the same pool that svn_config_t is allocated from and remember this pool in a new svn_authz_t member pool.l Create a function authz_create() for that; it uses

Re: [PATCH] speed up svn_repos_authz_check_access

2013-08-07 Thread Ivan Zhakov
On Wed, Aug 7, 2013 at 5:43 PM, Roderich Schupp roderich.sch...@gmail.com wrote: Hi, revised patch attached, hopefully addressing all concerns. No more breaking encapsulation. We simply allocate svn_authz_t from the same pool that svn_config_t is allocated from and remember this pool in a

Re: [PATCH] add support for svnrdump to svn-backup-dumps.py

2013-08-07 Thread Masaru Tsuchiyama
Hello I guess you don't need --non-interactive because standard input won't be a terminal. But you need a way to set the --username/--password args and maybe --trust-server-cert options to svnrdump. Otherwise looks ok to me. Thank you for the comment. I attach a fixed patch. [[[ add

Re: [PATCH] add support for svnrdump to svn-backup-dumps.py

2013-08-07 Thread Daniel Shahaf
Thanks for sending text/plain patches! +++ tools/server-side/svn-backup-dumps.py (working copy) +def get_head_rev_for_url(self): +extra_param = self.get_extra_param() + +# use 'svn yougest' to get the HEAD revision of URL +# 'svn yougest' is supported on

Re: [PATCH] add support for svnrdump to svn-backup-dumps.py

2013-08-07 Thread Daniel Shahaf
+revison_regex = re.compile(^r(\d+)) Typo, revision_regex ... and while at it, use r literals to avoid clashes with a potential future r\d backslash escape sequence.

Will fix for issue 4396 be included in the next release (1.8.2)?

2013-08-07 Thread Alfred von Campe
I am new to the Subversion lists (I'm not subscribed but monitor the lists' feed) and have not seen any chatter about issue 4396. This issue is really affecting our development as we cannot easily view diffs and our automated code review mechanism is broken as well. Most of our major repos

Re: [PATCH] add 'svn:use-commit-times' property

2013-08-07 Thread Daniel Shahaf
A review mostly of the code formatting (whitespace etc). In particular I'm not reviewing the feature being added: Index: subversion/include/svn_props.h === --- subversion/include/svn_props.h(revision 1509957) +++

Re: Will fix for issue 4396 be included in the next release (1.8.2)?

2013-08-07 Thread Daniel Shahaf
Alfred von Campe wrote on Wed, Aug 07, 2013 at 10:20:52 -0400: I don't know how many votes are required for inclusion, but I strongly urge you to vote for this change as it is really affecting our development and in my view is a major regression. Three votes are required. (See

Re: [PATCH] svnmucc assertion failure

2013-08-07 Thread Daniel Shahaf
Alexey Neyman wrote on Sat, Aug 03, 2013 at 14:57:12 -0700: [[[ * subversion/svnmucc/svnmucc.c (main): Check for URLs not having a common ancestor and emit an error message instead of failing an assertion later. ]]] Applied in r1511353 with minor tweaks (added a NULL check, even though

Re: [PATCH] fix exception running gen-make.py

2013-08-07 Thread Daniel Shahaf
masaru tsuchiyama wrote on Thu, Aug 01, 2013 at 00:11:30 +0900: * gen-make.py () : use default value as conf when args[0] is line ending. I don't use windows, but I think the right fix in this case is to prevent (\n in sys.argv) from becoming true in the first place.

Re: [PATCH] create LC_MESSAGES directories and copy subversion.mo to the directories at compile time

2013-08-07 Thread Daniel Shahaf
Masaru Tsuchiyama wrote on Tue, Aug 06, 2013 at 22:35:47 +0900: Hi. Does anybody comment my patch? Probably everyone who knows the windows build system is on holiday. PLease ping again next week and file an issue (and set the 'patch' keyword on it) if you still don't get a response. Daniel

Re: xml output changed - relative paths now appearing?

2013-08-07 Thread Philip Martin
Alexander Haley aha...@meditech.com writes: On Tue, Jul 16, 2013 at 6:26 AM, Bert Huijben b...@qqmail.nl wrote: -Original Message- With 1.7 I get target path=/home/pm/sw/subversion/obj/wc entry path=/home/pm/sw/subversion/obj/wc/foo while with 1.8 I get

Re: Will fix for issue 4396 be included in the next release (1.8.2)?

2013-08-07 Thread Alfred von Campe
On Aug 7, 2013, at 10:58, Daniel Shahaf danie...@elego.de wrote: This already has two votes and will probably get a third and be included in 1.8.2. Great, thanks for the quick response. P.S. Is there a target date for the 1.8.2 release? No. Probably 6-10 weeks after 1.8.1? I think

Re: 1.9.0-alpha1?

2013-08-07 Thread Ben Reser
On Wed, Aug 7, 2013 at 9:29 AM, Daniel Shahaf danie...@elego.de wrote: IIRC our plan was to release 1.9.0 in February/March, branching 3 months before that (Nov/Dec), and releasing an alpha 3 months before _that_. That schedule would have us release an alpha in about 3 weeks. (i.e., a

Re: Will fix for issue 4396 be included in the next release (1.8.2)?

2013-08-07 Thread Ben Reser
On Wed, Aug 7, 2013 at 9:11 AM, Alfred von Campe alf...@von-campe.com wrote: Well, there were 5 weeks between 1.8.0 and 1.8.1 so here is hoping for 5-6 weeks for 1.8.2 instead of 6-10 weeks. That would put the release at the end of August/beginning of September. As I mentioned in my original

Re: [PATCH] speed up svn_repos_authz_check_access

2013-08-07 Thread Roderich Schupp
On Wed, Aug 7, 2013 at 3:59 PM, Ivan Zhakov i...@visualsvn.com wrote: Could you please provide log message for your patch submission [1]. It makes reviewing patch much easier. See below. Did you consider make separate cache_pool and clear if we invoked with different user? I revised the

Re: [PATCH] speed up svn_repos_authz_check_access

2013-08-07 Thread Ben Reser
First of all I'd like to say thanks for doing this. I'd thought briefly about doing something like this yesterday. This is a good start. The biggest issue I see with this is that you're adding an unbounded cache. An attacker can open a connection and starting making requests to fill up a

Re: [PATCH] speed up svn_repos_authz_check_access

2013-08-07 Thread Roderich Schupp
On Wed, Aug 7, 2013 at 9:04 PM, Ben Reser b...@reser.org wrote: An attacker can open a connection and starting making requests to fill up a servers memory. As long as they keep making authz requests as the same user on the same connection they will be able to increase the server usage

Re: [PATCH] speed up svn_repos_authz_check_access

2013-08-07 Thread Ben Reser
On Wed, Aug 7, 2013 at 1:03 PM, Roderich Schupp roderich.sch...@gmail.com wrote: Err... the cache apr_hash_t by construction cannot contain keys (i.e.paths) that are not in the authz file also, so is bounded by the size of the corresponding svn_config_t. In fact, one could precompute the

Re: localisation: status, migration to pootle

2013-08-07 Thread Mattias EngdegÄrd
2 aug 2013 kl. 21.27 skrev Andreas Stieger: I would like to hear from active translation maintainers if they are happy to move their respective translations into Pootle and work on that system. If so I am willing to coordinate the import, even if just for trying it out. With sufficient

Re: [PATCH] speed up svn_repos_authz_check_access

2013-08-07 Thread Roderich Schupp
On Wed, Aug 7, 2013 at 11:08 PM, Ben Reser b...@reser.org wrote: Yes that's true. I know there are people out there with very large authz files though. Your cache isn't going to use much extra memory for most connections. We could shave off some more if we integrated the cache into

Re: man pages for Subversion

2013-08-07 Thread James K. Lowden
On Wed, 7 Aug 2013 12:45:36 +0300 Daniel Shahaf danie...@elego.de wrote: Then individual sections were edited by hand, sometimes to put finishing touches on the formatting, sometimes to clarify the text. How can we incorporate that into the dev cycle? if manual tweaks are required, we

RE: man pages for Subversion

2013-08-07 Thread Bert Huijben
Subversion is used on far more platforms than that have a default 'man' support. Breaking support for Windows, OS/2 and Netware is not an option. So we need a canonical format that is different than one that isn't platform dependent or requires more optional dependencies. Bert From: James K.

Re: man pages for Subversion

2013-08-07 Thread Daniel Shahaf
Bert Huijben wrote on Wed, Aug 07, 2013 at 19:13:53 -0700: Subversion is used on far more platforms than that have a default 'man' support. Breaking support for Windows, OS/2 and Netware is not an option. Why are you even worried about that? No one is going to break support for Windows.

Re: man pages for Subversion

2013-08-07 Thread Daniel Shahaf
On Thu, Aug 08, 2013 at 08:35:14AM +0300, Daniel Shahaf wrote: I have no idea what you're talking about. What is the problem on windows and how does a canonical format fix it? Never mind, I see what you mean now (JAmes' suggestion to generate help output from nroff source)

Re: man pages for Subversion

2013-08-07 Thread Daniel Shahaf
James K. Lowden wrote on Wed, Aug 07, 2013 at 20:13:16 -0400: On Wed, 7 Aug 2013 12:45:36 +0300 Daniel Shahaf danie...@elego.de wrote: Then individual sections were edited by hand, sometimes to put finishing touches on the formatting, sometimes to clarify the text. How can we