date:20100823

Re: Performance branch ready for review

2010-08-23 Thread Johan Corveleyn

On Sun, Aug 22, 2010 at 2:56 PM, Stefan Fuhrmann
stefanfuhrm...@alice-dsl.de wrote:
 Johan Corveleyn wrote:

 On Wed, Aug 18, 2010 at 9:14 PM, Stefan Fuhrmann
 stefanfuhrm...@alice-dsl.de wrote:


 Hi @all,

 I just finished my porting work; the performance branch
 is now fully synchronized with my prototype code.
 From my point of view, review can start now.

 According to my measurements, the code is now faster
 than the original prototype. Large caches provided, a
 single multi-threaded svnserve instance on a modern
 quad-core machine should be able to saturate a 10Gb
 server connection.

 Open issues / things still to do

 * there is an issue with log triggering an assertion()
  - I will investigate that next
 * test mod_web_dav and add FSFS cache configuration
  parameters to it
 * tune membuffer cache eviction strategy such that even
  small caches can have a large impact
 * add tests for the new APIs
 * provide APR patches.

 There are many things I would like to do but they may
 better be deferred to 1.8.


 I tried compiling your branch on Windows (XP) with Visual C++ Express
 2008 (which I also use successfully to build trunk). I had a couple of
 issues. FWIW I'm listing them here. I'm not an expert, just
 pragmatically trying to get the thing to build, so some of these
 things may be user error. Eventually, I was able to build svn.exe,
 but svnadmin.exe and svnserve.exe still fail for me.

 1) In build.conf, I added private\svn_temp_serializer.h and
 private\svn_file_handle_cache.h to the list of msvc-export of
 libsvn_subr. Otherwise, the linker gave problems:

 libsvn_fs_fs-1.lib(fs_fs.obj) : error LNK2019: unresolved external
 symbol _svn_file_handle_cache__has_file referenced in function
 _svn_fs_fs__path_rev_absolute
 libsvn_fs_fs-1.lib(fs_fs.obj) : error LNK2019: unresolved external
 symbol _svn_stream__from_cached_file_handle referenced in function
 _get_node_revision_body
 libsvn_fs_fs-1.lib(fs_fs.obj) : error LNK2019: unresolved external
 symbol _svn_file_handle_cache__open referenced in function
 _get_node_revision_body
 libsvn_fs_fs-1.lib(fs_fs.obj) : error LNK2019: unresolved external
 symbol _svn_file_handle_cache__get_apr_handle referenced in function
 _open_pack_or_rev_file
 libsvn_fs_fs-1.lib(fs_fs.obj) : error LNK2019: unresolved external
 symbol _svn_file_handle_cache__flush referenced in function
 _sync_file_handle_cache
 libsvn_fs_fs-1.lib(fs_fs.obj) : error LNK2019: unresolved external
 symbol _svn_file_handle_cache__close referenced in function
 _svn_fs_fs__rev_get_root
 libsvn_fs_fs-1.lib(caching.obj) : error LNK2019: unresolved external
 symbol _svn_file_handle_cache__create_cache referenced in function
 _get_global_file_handle_cache
 libsvn_fs_fs-1.lib(id.obj) : error LNK2019: unresolved external symbol
 _svn_temp_serializer__pop referenced in function
 _svn_fs_fs__id_serialize
 libsvn_fs_fs-1.lib(temp_serializer.obj) : error LNK2001: unresolved
 external symbol _svn_temp_serializer__pop
 libsvn_fs_fs-1.lib(id.obj) : error LNK2019: unresolved external symbol
 _svn_temp_serializer__push referenced in function
 _svn_fs_fs__id_serialize
 libsvn_fs_fs-1.lib(temp_serializer.obj) : error LNK2001: unresolved
 external symbol _svn_temp_serializer__push
 libsvn_fs_fs-1.lib(id.obj) : error LNK2019: unresolved external symbol
 _svn_temp_serializer__add_string referenced in function
 _serialize_id_private
 libsvn_fs_fs-1.lib(temp_serializer.obj) : error LNK2001: unresolved
 external symbol _svn_temp_serializer__add_string
 libsvn_fs_fs-1.lib(dag.obj) : error LNK2001: unresolved external
 symbol _svn_temp_serializer__add_string
 libsvn_fs_fs-1.lib(id.obj) : error LNK2019: unresolved external symbol
 _svn_temp_deserializer__resolve referenced in function
 _svn_fs_fs__id_deserialize
 libsvn_fs_fs-1.lib(temp_serializer.obj) : error LNK2001: unresolved
 external symbol _svn_temp_deserializer__resolve
 libsvn_fs_fs-1.lib(dag.obj) : error LNK2001: unresolved external
 symbol _svn_temp_deserializer__resolve
 libsvn_fs_fs-1.lib(temp_serializer.obj) : error LNK2019: unresolved
 external symbol _svn_temp_serializer__get referenced in function
 _svn_fs_fs__serialize_txdelta_window
 libsvn_fs_fs-1.lib(dag.obj) : error LNK2001: unresolved external
 symbol _svn_temp_serializer__get
 libsvn_fs_fs-1.lib(temp_serializer.obj) : error LNK2019: unresolved
 external symbol _svn_temp_serializer__init referenced in function
 _svn_fs_fs__serialize_txdelta_window
 libsvn_fs_fs-1.lib(dag.obj) : error LNK2001: unresolved external
 symbol _svn_temp_serializer__init
 libsvn_fs_fs-1.lib(temp_serializer.obj) : error LNK2019: unresolved
 external symbol _svn_temp_deserializer__ptr referenced in function
 _svn_fs_fs__extract_dir_entry
 libsvn_fs_fs-1.lib(dag.obj) : error LNK2019: unresolved external
 symbol _svn_temp_serializer__set_null referenced in function
 _svn_fs_fs__dag_serialize
 ..\..\..\Debug\subversion\libsvn_fs\libsvn_fs-1.dll : fatal error
 LNK1120: 15 unresolved externals

 Here's a patch that solves

Performance branch - svnserve crash in fs_history_prev

2010-08-23 Thread Johan Corveleyn

Hi,

I've taken the performance branch for a spin. Some of performance
increases are awesome (svn log is ~4 times faster on my machine
(tested with a file with 300 revisions)).

However, I also experienced a crash of svnserve, for both svn log
and svn blame of a big file with 2000 revisions (so this is quite
an extreme case). See both .log files in attachment. Something goes
wrong in svn_fs_fs__id_deserialize, during processing of
fs_history_prev.

I also have crash dump files of the crashes. If you want those, let me
know and I'll send them via private email.

This was on Windows XP, a release build with VCE 2008.

Cheers,
-- 
Johan

Re: Performance branch - svnserve crash in fs_history_prev

2010-08-23 Thread Lieven Govaerts

On Mon, Aug 23, 2010 at 11:10 AM, Johan Corveleyn jcor...@gmail.com wrote:
 Hi,

 I've taken the performance branch for a spin. Some of performance
 increases are awesome (svn log is ~4 times faster on my machine
 (tested with a file with 300 revisions)).

 However, I also experienced a crash of svnserve, for both svn log
 and svn blame of a big file with 2000 revisions (so this is quite
 an extreme case). See both .log files in attachment. Something goes
 wrong in svn_fs_fs__id_deserialize, during processing of
 fs_history_prev.

Either you forgot the attachments, or they were dropped by our mailing
sw. Try adding a .txt extension.

Lieven

Re: svn commit: r987592 - /subversion/trunk/subversion/libsvn_ra_neon/util.c

2010-08-23 Thread Julian Foad

On Fri, 2010-08-20, Greg Stein wrote:
 On Fri, Aug 20, 2010 at 13:58,  julianf...@apache.org wrote:
 ...
  +++ subversion/trunk/subversion/libsvn_ra_neon/util.c Fri Aug 20 17:58:10 
  2010
 ...
  -static ne_xml_parser *
  +/* Create a status parser attached to the request REQ.  Detected errors
  +   will be returned there. */
  +static void
   multistatus_parser_create(svn_ra_neon__request_t *req)
   {
multistatus_baton_t *b = apr_pcalloc(req-pool, sizeof(*b));
  -  ne_xml_parser *multistatus_parser =
  -svn_ra_neon__xml_parser_create(req, ne_accept_207,
  -   start_207_element,
  -   svn_ra_neon__xml_collect_cdata,
  -   end_207_element, b);
  +
  +  svn_ra_neon__xml_parser_create(req, ne_accept_207,
 
 Woah. You should have a (void) cast in there and (or?) a comment
 noting that the return value is not used. Otherwise, it looks like a
 programming error.

If you know that the called function attaches the parser to REQ, it
doesn't look so odd.  Its doc string didn't say so, so I've tweaked it.
I also added a comment here at the point of call for those who still
think it looks odd.  (I didn't add a cast to void, as ignoring return
values is a standard idiom in C yet that would make it stand out like a
sore thumb.)

I also wrote a doc string for static function attach_ne_body_reader(),
since it's involved here too.

r988068.

I did notice that all the other callers seem to keep a note of the
returned parser, and use it later in a call to
svn_ra_neon__check_parse_error().  I suppose there's something special
about this particular caller, and it doesn't need to do that.  I'm just
mentioning this in case anyone thinks it's wrong; otherwise I'm assuming
it's as intended.

- Julian

Re: Performance branch - svnserve crash in fs_history_prev

2010-08-23 Thread Johan Corveleyn

Trying again with .txt extension added.

Johan

On Mon, Aug 23, 2010 at 11:40 AM, Julian Foad julian.f...@wandisco.com wrote:
 On Mon, 2010-08-23, Lieven Govaerts wrote:
 Either you forgot the attachments, or they were dropped by our mailing
 sw. Try adding a .txt extension.

 AFAIK, the mailing list strips attachments of MIME type
 'application/octet-stream' and a bunch of other types - see
 https://issues.apache.org/jira/browse/INFRA-1194.  Regrettably, it
 doesn't inform either the sender or the recipients that it's done so.

 - Julian




Process info:
Cmd line: 
C:\research\svn\client_build\svn_branch_performance\dist\bin\svnserve.exe -d -r 
c:/research/svn/experiment/repos
Working Dir: C:\research\svn\experiment\repos
Version:  1.7.0 (dev build), compiled Aug 22 2010, 21:57:27
Platform: Windows OS version 5.1 build 2600 Service Pack 3

Exception: ACCESS_VIOLATION

Registers:
eax=75262f72 ebx=00c2ecc8 ecx=00c2ee6c edx=00c2ee68 esi=00c2ee6c edi=019f
eip=003a4dbf esp=00d7fb5c ebp=0001 efl=00010206
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=

Stacktrace:
#1  0x003a4dbf in svn_fs_fs__id_deserialize()
buffer = 0x003a7031
id = 
#2  0x003a7031 in svn_fs_fs__dag_deserialize()
out = 
data = 
data_len = c2ecc8
pool = 
#3  0x00426d83 in membuffer_cache_get()
#4  0x00426f70 in svn_membuffer_cache_get()
value_p = 
found = 0x00d7fc30
cache_void = 0x00d7fc34
key = 
pool = 
#5  0x0042737a in svn_cache__get()
value_p = 
found = 0x00d7fc30
cache = 
key = 
pool = 
#6  0x0039fde5 in dag_node_cache_get()
node_p = 
#7  0x003a20c6 in open_path()
parent_path_p = 
root = 
path = 
flags = 12991688
txn_id = 
pool = 
fs = 
parent_path = 
path_so_far = 
here = 
entry = 0x00bb2148 �...@¹
next = 
child = 
inherit = 
copy_path = 
cached_node = 
#8  0x003a3d05 in history_prev()
#9  0x003a3ff5 in fs_history_prev()
prev_history_p = 
history = 
cross_copies = 12991760
pool = 
prev_history = 
args = 
#10  0x003d54fd in get_history()
#11  0x003d6708 in do_logs()
fs = 
paths = 
hist_start = c1e110
hist_end = 0
limit = 96305
discover_changed_paths = 0
strict_node_history = 0
include_merged_revisions = 0
revprops = 
descending_order = 12706024
receiver = 
receiver_baton = 0x00405140
authz_read_func = 
authz_read_baton = 0x00402480
pool = 
iterpool = 
any_histories_left = 0
histories = 
rev_mergeinfo = 
revs = 
i = 12934224
current = 17831
send_count = 12706696
mergeinfo = 
has_children = 0
mergeinfo = 
#12  0x003d6cc9 in svn_repos_get_logs4()
repos = 
paths = 
start = c1e110
end = 0
limit = 96305
discover_changed_paths = 0
strict_node_history = 0
include_merged_revisions = 0
revprops = 
authz_read_func = 
authz_read_baton = 0x00402480
receiver = 
receiver_baton = 0x00405140
pool = 
head = b9a0b0
fs = 
descending_order = 96305
send_count = C1E27000BB2148
#13  0x0040559c in log_cmd()
conn = 
pool = 
params = 
baton = 0x00c1dc80
lb = 
full_paths = 
include_merged_revisions = 12704984
limit = C1DEF8
paths = 
revprop_items = 
changed_paths = 0
include_merged_revs_param = 0
start_rev = 0
end_rev = 0
strict_node = 12706064
#14  0x0040962b in svn_ra_svn_handle_commands2()
conn = 
pool = 
commands = 
baton = 0x0040e320
error_on_disconnect = 14155532
subpool = 
cmdname = 
params = 
cmd_hash = 
#15  0x0040781f in serve()
conn = 
params = 
pool = 
warn_baton = 
uuid = 
client_string = 
cap_log = 
client_url = 
ra_client_string = 
b = 
ver = 200C15508
cap_words = 
caplist = 
supports_mergeinfo = 12173800
#16  0x00401325 in serve_thread()
tid = 
data = 0x00b9c1c0
#17  0x6eed47b0 in dummy_worker()
opaque = 0x78543433
#18  0x78543433 in endthreadex()
#19  0x785434c7 in endthreadex()
#20  0x7c80b729 in GetModuleFileNameA()


Loaded modules:
0x0040  
C:\research\svn\client_build\svn_branch_performance\dist\bin\svnserve.exe 
(1.7.0.0, 94208 bytes)
0x7c90  C:\Windows\system32\ntdll.dll (5.1.2600.5755, 729088 bytes)
0x7c80  C:\Windows\system32\kernel32.dll (5.1.2600.5781, 1007616 bytes)
0x77dd  C:\Windows\system32\advapi32.dll (5.1.2600.5755, 634880 bytes)

Re: svn commit: r987956 - /subversion/trunk/build/ac-macros/serf.m4

2010-08-23 Thread Daniel Shahaf

jerenkra...@apache.org wrote on Sun, Aug 22, 2010 at 22:57:26 -:
 Author: jerenkrantz
 Date: Sun Aug 22 22:57:26 2010
 New Revision: 987956
 
 URL: http://svn.apache.org/viewvc?rev=987956view=rev
 Log:
 Add a compile-time version check for serf so we can reject old versions.
 
 * build/ac-macros/serf.m4
   (SVN_LIB_SERF): Set minimum version to 0.3.1 and ensure we see at least 
 that.
 
 Modified:
 subversion/trunk/build/ac-macros/serf.m4
 
 Modified: subversion/trunk/build/ac-macros/serf.m4
 URL: 
 http://svn.apache.org/viewvc/subversion/trunk/build/ac-macros/serf.m4?rev=987956r1=987955r2=987956view=diff
 ==
 --- subversion/trunk/build/ac-macros/serf.m4 (original)
 +++ subversion/trunk/build/ac-macros/serf.m4 Sun Aug 22 22:57:26 2010
 @@ -27,6 +27,10 @@ AC_DEFUN(SVN_LIB_SERF,
  [
serf_found=no
  
 +  serf_check_major=0
 +  serf_check_minor=3
 +  serf_check_patch=1
 +
AC_ARG_WITH(serf,AS_HELP_STRING([--with-serf=PREFIX],
[Serf WebDAV client library]),
[
 @@ -40,7 +44,16 @@ AC_DEFUN(SVN_LIB_SERF,
AC_CHECK_HEADERS(serf.h,[
  save_ldflags=$LDFLAGS
  LDFLAGS=$LDFLAGS -L$serf_prefix/lib
 -AC_CHECK_LIB(serf-0, serf_context_create,[serf_found=yes], ,
 +AC_CHECK_LIB(serf-0, serf_context_create,[
 +  AC_TRY_COMPILE([
 +#include stdlib.h
 +#include serf.h
 +],[
 +#if ! SERF_VERSION_AT_LEAST($serf_check_major, $serf_check_minor, 
 $serf_check_patch)
 +#error Serf version too old: want 
 $serf_check_major.$serf_check_minor.$serf_check_patch, got SERF_VERSION_STRING

Does this actually expand SERF_VERSION_STRING?  A quick independent test
indicates it wouldn't...

 +#endif
 +], [serf_found=yes], [AC_MSG_WARN([Serf version too old: need 
 $serf_check_major.$serf_check_minor.$serf_check_patch])
 +serf_found=no])], ,
$SVN_APRUTIL_LIBS $SVN_APR_LIBS -lz)
  LDFLAGS=$save_ldflags])
CPPFLAGS=$save_cppflags

Re: [RFC] 'External' and 'Switched': common ground

2010-08-23 Thread Julian Foad

On Fri, 2010-08-20 at 20:01 +0200, Stefan Küng wrote:
 On 20.08.2010 19:54, Julian Foad wrote:
 * An 'external' or 'switched' WC node is an immediate child of a
   versioned WC directory. [3]
 
 Don't forget that an external does not necessarily have a versioned WC 
 dir as its immediate parent:
   svn:externals http://server.com/repo/trunk subdir/extdir
 in that case, 'subdir' would be the immediate parent of the external, 
 but it's not versioned.

Ah yes.  r988086.

For file externals, a quick test shows that it just throws errors.  But
it seems to be an intentional and presumably useful part of the
(directory externals) design, so I suppose we should aim to make that
option available for file externals too.

- Julian

Re: [RFC/PATCH] Create a fresh svn_repos_parse_fns3_t

2010-08-23 Thread Julian Foad

On Thu, 2010-08-19 at 10:25 +0300, Daniel Shahaf wrote:
 Ramkumar Ramachandra wrote on Thu, Aug 19, 2010 at 11:12:14 +0530:
  Hi,
  
  I sent a patch a while ago for svn_repos_parse_dumpstream3. While I
  wait for approval, this is an RFC patch describing my future plan once
  that patch gets approved. It can be described tersely as While at it
  (adding the new callback), fix everything that's wrong with the
  struct.

I have just read through Daniel's diff of this patch, and the change
appears to be:

  Add a dumpstream_version_record() method.
  Add scratch pool arguments to all methods.

Is that what you intended?  If so, it sounds sane to me.  (I would name
the scratch pool arguments 'scratch_pool'.)

- Julian

Re: svn commit: r987956 - /subversion/trunk/build/ac-macros/serf.m4

2010-08-23 Thread Justin Erenkrantz

On Mon, Aug 23, 2010 at 4:28 AM, Daniel Shahaf d...@daniel.shahaf.name wrote:
 Does this actually expand SERF_VERSION_STRING?  A quick independent test
 indicates it wouldn't...

Ah, you're right.  Hmm.  Any ideas?  -- justin

Re: svn commit: r987956 - /subversion/trunk/build/ac-macros/serf.m4

2010-08-23 Thread Daniel Shahaf

Justin Erenkrantz wrote on Mon, Aug 23, 2010 at 06:18:14 -0700:
 On Mon, Aug 23, 2010 at 4:28 AM, Daniel Shahaf d...@daniel.shahaf.name 
 wrote:
  Does this actually expand SERF_VERSION_STRING?  A quick independent test
  indicates it wouldn't...
 
 Ah, you're right.  Hmm.  Any ideas?  -- justin

Just drop the macro from the #error and, instead, include the path to
the used serf.h in the error message?

grep the version-number-tuple macros from serf.h manually?

Re: [RFC/PATCH] Create a fresh svn_repos_parse_fns3_t

2010-08-23 Thread Ramkumar Ramachandra

Hi Julian,

Julian Foad writes:
 On Thu, 2010-08-19 at 10:25 +0300, Daniel Shahaf wrote:
  Ramkumar Ramachandra wrote on Thu, Aug 19, 2010 at 11:12:14 +0530:
   Hi,
   
   I sent a patch a while ago for svn_repos_parse_dumpstream3. While I
   wait for approval, this is an RFC patch describing my future plan once
   that patch gets approved. It can be described tersely as While at it
   (adding the new callback), fix everything that's wrong with the
   struct.
 
 I have just read through Daniel's diff of this patch, and the change
 appears to be:
 
   Add a dumpstream_version_record() method.
   Add scratch pool arguments to all methods.
 
 Is that what you intended?  If so, it sounds sane to me.  (I would name
 the scratch pool arguments 'scratch_pool'.)

Yep, and I'm changing the API to pass the scratch pool around. I'm
using Git to stage all my changes and produce a coherent series with
relevant changes, but it's taking some time- expect to see a series
soon though.

-- Ram

Re: merging into locally added files

2010-08-23 Thread Stefan Sperling

On Wed, Aug 04, 2010 at 11:23:00AM -0400, Paul Burba wrote:
 On Wed, Aug 4, 2010 at 10:52 AM, Julian Foad julian.f...@wandisco.com wrote:
  Stefan Sperling wrote:
  It does not seem possible right now to merge into locally added
  files, because the Subversion assumes that the merge target will
  always have a corresponding URL in the repository, and errors out.
 
  With a bit of special-casing during error handling in a few places,
  I succeeded in making this use case work:
  [...]
 
  Hi Stefan.  In my opinion, Subversion should allow such a merge to be
  attempted, and the result should be a textual merge like you've done
  here
 
 Stefan,
 
 This change in behavior seems sound to me and I don't see it causing
 any problems in the merge-tracking realm.

I've committed this change, and two regression tests, in r988188.
Stephen Butler help me figure out some related issues in our test suite,
which we fixed in r988144 and r988175.

Note that while my original post was only about merging into locally added
files, I discovered that merging into locally added directories also works.

One more question: In case a file/directory has been copied, does that
affect its implicit mergeinfo in any way? If so, we should probably make
the code I added to get_full_mergeinfo() handle copies differently
than locally added items.

Stefan

Re: merging into locally added files

2010-08-23 Thread Stefan Sperling

On Wed, Aug 04, 2010 at 12:32:06PM -0400, Paul Burba wrote:
 On Wed, Aug 4, 2010 at 12:14 PM, Julian Foad julian.f...@wandisco.com wrote:
  On Wed, 2010-08-04 at 11:23 -0400, Paul Burba wrote:
  On Wed, Aug 4, 2010 at 10:52 AM, Julian Foad julian.f...@wandisco.com 
  wrote:
   if the tree conflict detection policy is relaxed, and should be a
   tree conflict if the policy is strict.  (Yes, the tree conflict
   detection policy switch only exists in my head.)
 
  I don't follow you there, how would a merge to a file ever be a tree
  conflict?  Or do you mean if our merge target is an added directory?
  Stefan's patch supports that as well.
 
  It's a tree conflict because the incoming change is modification of
  file FOO and the local target is a file *named* FOO but ancestrally
  unrelated to the source file FOO.  It's similar to the situation that
  we'd get if the target branch at one time did have the ancestrally
  related FOO line on it but that FOO was then deleted and replaced by a
  new file named FOO.
 
  - Julian
 
 Got it, thanks Julian.  (Obviously) I'm for the relaxed policy then,
 in this use case.

My take on this is that there is no tree conflict on the target itself.

The user has asked Subversion to merge a set of changes into a target
in the working copy. The target happens to be locally added.
The changes being merged might not be ancestrally related to the target,
but that in itself is no reason for a tree conflict to occur.

Subversion should try its best to apply the changes to the merge target
as instructed. There can occur tree conflicts *inside* of the merge target
if it is a locally added directory. But the target itself is just the root
of the tree the merge operation is applying changes to.

Stefan

RE: [RFC] 'External' and 'Switched': common ground

2010-08-23 Thread Julian Foad

Thanks for this feedback, Bert.  Several important points are emerging.

The first thing to get straight is what future we want for file
externals.  File externals should behave like directory externals.  The
directory externals system is far from a perfect solution to the
high-level challenges that users try to solve, but it has a lot of
precedent behind it now and is at least fairly logical.  Despite us
having released file externals in 1.6 with several differences from
directory externals, I think absolutely all those differences should be
regarded as bugs and we should be bringing them back into line.

(If we do know of any way in which file externals behave better than
directory externals, that should be considered as a potential future
improvement for externals in general.)

In order to know what we're dealing with, we need to check and write
down the current behavioural differences, but in re-implementing file
externals handling, it's unquestionably the directory externals
behaviour that we need to be heading for.


On Fri, 2010-08-20, Bert Huijben wrote: 
  -Original Message-
  From: Julian Foad [mailto:julian.f...@wandisco.com]
* An 'external' or 'switched' WC node is an immediate child of a
  versioned WC directory. [3]

Changed to parent or grandparent etc..

* An 'external' or 'switched' WC node is attached to a repository
  location other than the one implied by its WC parent directory.
 
 An external directory is not connected to it's parent. (You don't get
 it in the list of children using entries and/or nodes). To the parent
 it is just an unversioned directory, but if you look inside you find a
 working copy root that might (accidentally) be from the same
 repository. And it can be a few levels deeper than just an immediate
 child.

OK: an external directory is a WC root.  I hadn't contemplated the
importance of that until now, but I think we can almost *define* the
required behaviour of externals in terms of that statement.

  An external directory is a nested WC that libsvn_client knows
   something about: namely it knows how and when to create it and
   to use it.

I'm suggesting libsvn_client here as an arbitrary but important level
of abstraction where a complex WC is implemented, and below which
there is a concept of single WC (single wc_id) that has no notion of
externals.

We might choose to put this complex WC concept into libsvn_wc instead,
but it's useful for the conceptual distinction to exist somewhere in the
APIs, and this is probably a very good candidate for a chunk of
complexity that we would do well to keep out of libsvn_wc.

Let's see where that approach will take us.


[...]

* Client operations treat the node as part of its parent WC in some
  ways, and treat it as special in other ways, depending on the
  command and on user-specified options.
 
 Yes,
 
 Well, there are no real operations on external directories as they are
 just unversioned to the parent working copy.

At least two client operations, svn update and svn status, recognize
externals and descend into them.

 External files... specialized code... specialized code... specialized
 code.
 
 External files are part of the parent working copy, so things like
 (recursive) locks and things apply on them. But these locks don't
 apply on directory externals as they are in a different working copy.

[...]

  How should we represent external files and external dirs in the WC in a
  unified manner?  Do we need to explicitly mark them as 'external'?  What
  differences in behaviour should 'external' nodes and 'switched' nodes
  exhibit within the WC?

(Heh, those were just meant to be examples of questions that this whole
topic might help to answer.)

 One option would be to pull file externals out of their parent working
 copy and make them their own working copy root. We can handle this in
 WC-DB. (Create a different wcroot id, defining the file to be its own
 root. Fix the update handling and we are done).

Yes!

If we are committed to making file externals like directory externals,
we need to re-specify a file external as a separate WC consisting of a
single file.  And when we try to do that, we have to re-think our
notion of a WC root.  A WC root will no longer always be a directory
and will not always be able to hold its own .svn admin area inside it.

Recall that the ability to check out and work on a single file from a
repository has been requested quite a few times in the life of
Subversion.  That concept of a single-file checkout is alien to WC-1 and
to some assumptions built on our old implementation of WCs, but is not
in itself at all alien to the version control concepts of Subversion.
So a single-file WC is not specifically an externals concept, but
rather a general concept that will be useful in implementing externals.


  Differences in WC State
  ===
  
  This table documents the differences in WC state at a high level.

Re: merging into locally added files

2010-08-23 Thread Julian Foad

On Mon, 2010-08-23 at 18:43 +0200, Stefan Sperling wrote:
 On Wed, Aug 04, 2010 at 12:32:06PM -0400, Paul Burba wrote:
  On Wed, Aug 4, 2010 at 12:14 PM, Julian Foad julian.f...@wandisco.com 
  wrote:
   On Wed, 2010-08-04 at 11:23 -0400, Paul Burba wrote:
   On Wed, Aug 4, 2010 at 10:52 AM, Julian Foad julian.f...@wandisco.com 
   wrote:
if the tree conflict detection policy is relaxed, and should be a
tree conflict if the policy is strict.  (Yes, the tree conflict
detection policy switch only exists in my head.)
  
   I don't follow you there, how would a merge to a file ever be a tree
   conflict?  Or do you mean if our merge target is an added directory?
   Stefan's patch supports that as well.
  
   It's a tree conflict because the incoming change is modification of
   file FOO and the local target is a file *named* FOO but ancestrally
   unrelated to the source file FOO.  It's similar to the situation that
   we'd get if the target branch at one time did have the ancestrally
   related FOO line on it but that FOO was then deleted and replaced by a
   new file named FOO.
  
   - Julian
  
  Got it, thanks Julian.  (Obviously) I'm for the relaxed policy then,
  in this use case.
 
 My take on this is that there is no tree conflict on the target itself.
 
 The user has asked Subversion to merge a set of changes into a target
 in the working copy. The target happens to be locally added.
 The changes being merged might not be ancestrally related to the target,
 but that in itself is no reason for a tree conflict to occur.
 
 Subversion should try its best to apply the changes to the merge target
 as instructed. There can occur tree conflicts *inside* of the merge target
 if it is a locally added directory. But the target itself is just the root
 of the tree the merge operation is applying changes to.

You seem to be talking only about the case where the (locally added)
target is the root of the whole merge, and saying that lack of ancestral
relationship between the source node and this target node doesn't
matter.  Maybe the user performing the merge is fully aware of what
he/she is doing, which is fine in this simple case.  But what about the
general case, where the modification to a locally added node may be
somewhere deep down in a merge that's supposed to be a simple automatic
merge?  If such a modification encounters a new local node (that perhaps
replaces an older node that was once related but has since been moved
away or deleted), some people may want that to be raised as a conflict.
Others, especially those who have set up the situation knowingly, may
not.  Hence my talking about this choice of behaviour as the relaxed
policy option.

- Julian

Re: merging into locally added files

2010-08-23 Thread Paul Burba

On Mon, Aug 23, 2010 at 12:35 PM, Stefan Sperling s...@elego.de wrote:

 One more question: In case a file/directory has been copied, does that
 affect its implicit mergeinfo in any way?

Definitely, a copy *has* implicit mergeinfo, whereas a local addition has none.

Think of it like this:

Do a URL-to-WC copy of Subersion's own tr...@988208 to some-branch-wc.

If we tried this...

  svn merge -c988175 ^/subversion/tr...@head some-branch-wc

...we wouldn't expect r988175 to be merged to the working copy.
Because '/subversion/trunk:988175' is not contained in the copied
target's explicit/inherited mergeinfo, it is the target's implicit
mergeinfo that prevents this.

 If so, we should probably make
 the code I added to get_full_mergeinfo() handle copies differently
 than locally added items.

It doesn't appear that anything needs to be done,
svn_client__derive_location() handles the case where ABSPATH_OR_URL is
a copy, and returns the copy-from URL, not an error.  So your change
in r988188 has no effect in this case.

Paul

 Stefan

Re: merging into locally added files

2010-08-23 Thread Julian Foad

On Mon, 2010-08-23 at 18:40 +0100, Julian Foad wrote:
 On Mon, 2010-08-23 at 18:43 +0200, Stefan Sperling wrote:
  On Wed, Aug 04, 2010 at 12:32:06PM -0400, Paul Burba wrote:
   On Wed, Aug 4, 2010 at 12:14 PM, Julian Foad julian.f...@wandisco.com 
   wrote:
On Wed, 2010-08-04 at 11:23 -0400, Paul Burba wrote:
On Wed, Aug 4, 2010 at 10:52 AM, Julian Foad 
julian.f...@wandisco.com wrote:
 if the tree conflict detection policy is relaxed, and should be a
 tree conflict if the policy is strict.  (Yes, the tree conflict
 detection policy switch only exists in my head.)
   
I don't follow you there, how would a merge to a file ever be a tree
conflict?  Or do you mean if our merge target is an added directory?
Stefan's patch supports that as well.
   
It's a tree conflict because the incoming change is modification of
file FOO and the local target is a file *named* FOO but ancestrally
unrelated to the source file FOO.  It's similar to the situation that
we'd get if the target branch at one time did have the ancestrally
related FOO line on it but that FOO was then deleted and replaced by a
new file named FOO.
   
- Julian
   
   Got it, thanks Julian.  (Obviously) I'm for the relaxed policy then,
   in this use case.
  
  My take on this is that there is no tree conflict on the target itself.
  
  The user has asked Subversion to merge a set of changes into a target
  in the working copy. The target happens to be locally added.
  The changes being merged might not be ancestrally related to the target,
  but that in itself is no reason for a tree conflict to occur.
  
  Subversion should try its best to apply the changes to the merge target
  as instructed. There can occur tree conflicts *inside* of the merge target
  if it is a locally added directory. But the target itself is just the root
  of the tree the merge operation is applying changes to.
 
 You seem to be talking only about the case where the (locally added)
 target is the root of the whole merge, and saying that lack of ancestral
 relationship between the source node and this target node doesn't
 matter.  Maybe the user performing the merge is fully aware of what
 he/she is doing, which is fine in this simple case.  But what about the
 general case, where the modification to a locally added node may be
 somewhere deep down in a merge that's supposed to be a simple automatic
 merge?  If such a modification encounters a new local node (that perhaps
 replaces an older node that was once related but has since been moved
 away or deleted), some people may want that to be raised as a conflict.
 Others, especially those who have set up the situation knowingly, may
 not.  Hence my talking about this choice of behaviour as the relaxed
 policy option.

But I don't mean to be only negative.  Thanks immensely for implementing
this extension, as well as all the other things you're doing, and I
agree it will be useful.

- Julian

Re: Noise: spicy autoindex httpd.conf workaround #fail

2010-08-23 Thread C. Michael Pilato

Hrm. I'm not seeing the connection here. I'm looking for a way to pull off
a Rewrite condition based on the existence of a given URI. The docs imply
that this can be done with RewriteCond SOME_URI -U, but appear to just be
wrong -- the existence of SOME_URI doesn't appear to tested at all, only
it's accessibility (from an authn/authz standpoint).

On 08/20/2010 12:51 PM, Justin Erenkrantz wrote:
Based on what you describe, I think this is likely what you are looking for:

http://httpd.apache.org/docs/2.0/mod/mod_mime.html#modmimeusepathinfo

http://mail-archives.apache.org/mod_mbox/httpd-dev/200209.mbox/%3c20020904211343.ga16...@apache.org%3e
http://mail-archives.apache.org/mod_mbox/httpd-dev/200306.mbox/%3c2147483647.1054772...@[10.0.1.37]%3e

HTH. -- justin

On Fri, Aug 20, 2010 at 1:27 AM, C. Michael Pilato cmpil...@collab.net
wrote:
[Warning: This matter is far from highly pertinent. One tackles strange
non-problems when in an atypical environment, such as a hotel room in CA.]

I had someone ask me about Subversion autoindex support. So, like, you
point a web browser at
http://svn.apache.org/repos/asf/subversion/site/publish/ and *pow* magically
you are now looking at the index.html inside that directory.

Clearly, this could be done with an hour or two of mod_dav_svn hackery and
some new directives there. But I was trying to come up with an httpd.conf
workaround that did the trick. Here's what I tried. (On my system, all my
Subversion repositories live inside the /repos/ Location.)

# If this is a GET request (but not a subrequest) aimed at my
# collection of Subversion repositories and with a trailing slash, and
# if there exists an index.html file inside that directory, then
# temporarily redirect the browser to the index.html file.
RewriteEngine on
RewriteCond %{IS_SUBREQ} false
RewriteCond %{REQUEST_METHOD} GET
RewriteCond %{REQUEST_URI}^/repos/.*/$
RewriteCond %{REQUEST_URI}index.html -U
RewriteRule /repos/(.*)/$ /repos/$1/index.html [L,R]

The result was that for every directory in which an index.html was found,
that file was served (via a browser redirect). Yay! Unfortunately, the
redirect was transmitted for directories which had no index.html child, too.
Boo!

Sadly, I found that despite the fact that the Apache docs say about that
-U test the following:

'-U' (is existing URL, via subrequest)
Checks whether or not TestString is a valid URL, accessible via all
the server's currently-configured access controls for that path. This
uses an internal subrequest to do the check, so use it with care - it
can impact your server's performance!

In reality validity in this context seems to have nothing to do with
existence. I traced the subrequest that mod_rewrite made into Subversion,
and found that it never enters mod_dav to actually perform an existence get.
I guess I expected that the subrequest would GET all the way into
Subversion, where it would get the appropriate error code (HTTP_NOT_FOUND).
In retrospect, I think I knew that subrequests don't behavior like
full-fledged content-fetching requests. But the documentation quoted above
is pretty misleading, at any rate, IMO.

--
C. Michael Pilato cmpil...@collab.net
CollabNet www.collab.net Distributed Development On Demand

Re: Noise: spicy autoindex httpd.conf workaround #fail

2010-08-23 Thread C. Michael Pilato

Ohohohoh! Now that sounds promising! Will give that a shot.

On 08/23/2010 01:50 PM, Justin Erenkrantz wrote:
Why do you want mod_rewrite at all?Enabling that mod_mime
directive will let mod_autoindex remap / to /index.html when it exists
in the virtual filesystem. -- justin

On Mon, Aug 23, 2010 at 10:48 AM, C. Michael Pilato cmpil...@collab.net
wrote:
Hrm. I'm not seeing the connection here. I'm looking for a way to pull off
a Rewrite condition based on the existence of a given URI. The docs imply
that this can be done with RewriteCond SOME_URI -U, but appear to just be
wrong -- the existence of SOME_URI doesn't appear to tested at all, only
it's accessibility (from an authn/authz standpoint).

On 08/20/2010 12:51 PM, Justin Erenkrantz wrote:
Based on what you describe, I think this is likely what you are looking for:

http://httpd.apache.org/docs/2.0/mod/mod_mime.html#modmimeusepathinfo

HTH. -- justin

I had someone ask me about Subversion autoindex support. So, like, you
point a web browser at
http://svn.apache.org/repos/asf/subversion/site/publish/ and *pow*
magically
you are now looking at the index.html inside that directory.

Sadly, I found that despite the fact that the Apache docs say about that
-U test the following:

In reality validity in this context seems to have nothing to do with
existence. I traced the subrequest that mod_rewrite made into
Subversion,
and found that it never enters mod_dav to actually perform an existence
get.
I guess I expected that the subrequest would GET all the way into
Subversion, where it would get the appropriate error code (HTTP_NOT_FOUND).
In retrospect, I think I knew that subrequests don't behavior like
full-fledged content-fetching requests. But the documentation quoted above
is pretty misleading, at any rate, IMO.

--
C. Michael Pilato cmpil...@collab.net
CollabNet www.collab.net Distributed Development On Demand

signature.asc
Description: OpenPGP digital signature

RE: svn commit: r988074 - in /subversion/trunk/subversion/tests/cmdline: svntest/wc.py upgrade_tests.py

2010-08-23 Thread Bert Huijben

 -Original Message-
 From: phi...@apache.org [mailto:phi...@apache.org]
 Sent: maandag 23 augustus 2010 13:23
 To: comm...@subversion.apache.org
 Subject: svn commit: r988074 - in
 /subversion/trunk/subversion/tests/cmdline: svntest/wc.py
 upgrade_tests.py

 Author: philip
 Date: Mon Aug 23 11:22:51 2010
 New Revision: 988074

 URL: http://svn.apache.org/viewvc?rev=988074view=rev
 Log:
 Verify pristines after upgrade so that the tests fail as expected
 in single-db.

 * subversion/tests/cmdline/upgrade_tests.py
   (check_pristine): New.
   (basic_upgrade, upgrade_with_externals, upgrade_1_5_body): Check
 pristines.

 * subversion/tests/cmdline/upgrade_tests.py
   (text_base_path): Restore MD5 support removed in r960036.

I think the real fix would be to upgrade to SHA1 (and add the mapping in the 
pristines table) in the upgrade step. I expected that this was already handled?

Bert

Re: Performance branch ready for review

2010-08-23 Thread Stefan Fuhrmann


Daniel Shahaf wrote:

Stefan, you did mention Patch by for Johan's patches which you
committed, did you intend to mention Found by or Suggested by for
the other two (quoted below)?

http://subversion.apache.org/docs/community-guide/conventions.html#crediting

Thanks,

Daniel
  

Oh, I just was not aware that there are tons of ... by schemes.
r987868 and r987869 now rightfully mention Johan.

-- Stefan^2.

Re: Performance branch ready for review

2010-08-23 Thread Johan Corveleyn

On Tue, Aug 24, 2010 at 12:22 AM, Stefan Fuhrmann
stefanfuhrm...@alice-dsl.de wrote:
 Daniel Shahaf wrote:

 Stefan, you did mention Patch by for Johan's patches which you
 committed, did you intend to mention Found by or Suggested by for
 the other two (quoted below)?


 http://subversion.apache.org/docs/community-guide/conventions.html#crediting

 Thanks,

 Daniel


 Oh, I just was not aware that there are tons of ... by schemes.
 r987868 and r987869 now rightfully mention Johan.

Thanks. Not terribly important to me, but nice anyway.

I just hope some of your performance-work makes it into 1.7. So
anything I can do to help ...

Cheers,
-- 
Johan

Re: merging into locally added files

2010-08-23 Thread Stefan Sperling

On Mon, Aug 23, 2010 at 06:40:39PM +0100, Julian Foad wrote:
 You seem to be talking only about the case where the (locally added)
 target is the root of the whole merge, and saying that lack of ancestral
 relationship between the source node and this target node doesn't
 matter.  Maybe the user performing the merge is fully aware of what
 he/she is doing, which is fine in this simple case.  But what about the
 general case, where the modification to a locally added node may be
 somewhere deep down in a merge that's supposed to be a simple automatic
 merge?

I don't think locally added nodes somewhere deep within the merge target
are affected by this change. Those should be handled by the regular
tree conflict logic. The change only affects the merge target root,
which can now be a locally added file or directory. Previously,
Subversion just errored out on locally added merge target roots
and didn't merge anything at all.

Stefan

Re: Performance branch ready for review

Performance branch - svnserve crash in fs_history_prev

Re: Performance branch - svnserve crash in fs_history_prev

Re: svn commit: r987592 - /subversion/trunk/subversion/libsvn_ra_neon/util.c

Re: Performance branch - svnserve crash in fs_history_prev

Re: svn commit: r987956 - /subversion/trunk/build/ac-macros/serf.m4

Re: [RFC] 'External' and 'Switched': common ground

Re: [RFC/PATCH] Create a fresh svn_repos_parse_fns3_t

Re: svn commit: r987956 - /subversion/trunk/build/ac-macros/serf.m4

Re: svn commit: r987956 - /subversion/trunk/build/ac-macros/serf.m4

Re: [RFC/PATCH] Create a fresh svn_repos_parse_fns3_t

Re: merging into locally added files

Re: merging into locally added files

RE: [RFC] 'External' and 'Switched': common ground

Re: merging into locally added files

Re: merging into locally added files

Re: merging into locally added files

Re: Noise: spicy autoindex httpd.conf workaround #fail

Re: Noise: spicy autoindex httpd.conf workaround #fail

RE: svn commit: r988074 - in /subversion/trunk/subversion/tests/cmdline: svntest/wc.py upgrade_tests.py

Re: Performance branch ready for review

Re: Performance branch ready for review

Re: merging into locally added files

23 matches

Site Navigation

Mail list logo

Footer information