> On Fri, Jan 7, 2011 at 7:08 PM, Paul Burba <ptburba_at_gmail.com> wrote:
>> Mike suggested to me that the fix for this problem can be made in
>> mod_dav_svn, since the update reporter already knows which properties
>> have changed, *regardless* of the send-all mode, but it discards this
>> information if send-all=FALSE and instead instead sends the client a
>> 'fetch-props' response. As described previously in this thread, this
>> causes the client to fetch all the properties of the path, not just
>> the changes, and *all* the props are pushed through the editor
>> callbacks as if they were all actual changes (which again is the core
>> problem of issue #3657).
>>
>> I'm working on that approach,
>
> I had pushed off the fix-it-in-mod_dav_svn approach till 1.8 since I
> was going in circles with it, but recently found another real-world
> example where this issue causes spurious conflicts on directory
> properties, see
> http://subversion.tigris.org/issues/show_bug.cgi?id=3657#desc13.
>
> This issue has existed before 1.5 so it's nothing new, but given that
> I found another non-theoretical example and given that the presence of
> mergeinfo is only going to make this issue more common[1] I thought
> I'd take another look for 1.7.
>
> I still haven't gotten anywhere with the mod_dav_svn fix, so not much
> to talk about there, but I did commit the fix for the directory
> property conflicts in
> http://svn.apache.org/viewvc?view=revision&revision=1056565. The fix
> here is pretty much the same thing I did in
> http://svn.apache.org/viewvc?view=revision&revision=966822: Filter out
> the phony prop changes in the libsvn_client/repos_diff.c
> svn_delta_editor_t change_[dir|file]_prop() callback implementations.
>
> Just to be clear about one thing, when I committed r966822 Bert asked:
>
>>> (change_file_prop): Only stash true property differences in the file
>>> baton. The DAV RA providers may be sending us all the properties on
>>> a file, not just the differences.
>>
>> Shouldn't this be fixed in the ra layer implementations then?
>>
>> This would be just 'guessing' if it is a change or a complete set.
>
> Why not the RA layer? Two reasons. The first I explained earlier in
> this thread, it reopens issue #2048 'Primary connection (of parallel
> network connections used) in ra-dav diff/merge timeout unnecessarily.'
> I won't go over that again, but even if there is a way to avoid that,
> the RA layer is simply doing what we ask it when using the DAV
> providers. Specifically, the mod_dav_svn update report handler, when
> in 'skelta' mode[2] and describing changes to a path on which a
> property has changed, asks the client to later fetch all properties
> and figure out what has changed itself, i.e. the client asks the RA
> layer to give us *all* the properties, it obliges, so where is the
> problem as far as the RA layer is concerned?
>
> So this ultimately needs to be fixed on the server. Given that, I am
> leaving these libsvn_client filters in place so a 1.7 client will DTRT
> with older servers (or maybe 1.7 servers too, since as I said, I'm not
> having great success there).
>
> As to the 'guessing', that was true. We'd go through the motions of
> filtering over ra_local and ra_svn too. A pretty minor performance
> hit, but in r1056565 I tweaked the filtering so it only applies when
> using ra_serf and ra_neon.
>
> Paul
>
> [1] simply by increasing the potential pool of property changes
> incoming during a merge.
>
> [2] Which merges always are, regardless of whether we use ra_serf or ra_neon.
>
> Paul
Ok, after many false starts and detours I finally have a server-side
fix for this issue. Pursuing a suggestion by cmpilato, I made the
mod_dav_svn update report always send property changes, regardless of
whether the report is in skelta (send-all=false) mode or non-skelta
(send-all=true) mode.
In addition to the normal serf/neon trunk tests, I tested 1.5.10 (dev
build) and 1.6.16 (dev build) against this patched 1.7 server and
there were no unexpected [1] failures.
If any mod_dav_svn experts have time to take a look at this patch I'd
appreciate it.
Thanks,
Paul
[[[
Server-side fix for issue #3657 'dav update report handler in skelta mode can
cause spurious conflicts'.
This change has two parts that are loosely related:
1) It stops mod_dav_svn from ever sending the 'fetch-props' response to a
client, even when in skelta (i.e. send-all=false) mode. The server
already knows what properties have changed so now it simply sends them,
rather than telling the client to ask for them later. This prevents
the svn_delta_editor_t change_[file|dir]_prop callback from receiving
faux property changes when the client asks for *all* the properties.
See http://subversion.tigris.org/issues/show_bug.cgi?id=3657 for all the
gory details.
2) The comments in our code frequently make reference to 'modern' servers
and clients in the context of mod_dav_svn. Unfortunately there is no such
thing as a pre-modern server, they are all modern! The use of the term
'modern' in our code comments predates Subversion 1.0.0, so the comments
are misleading and the special case code for pre-modern servers (or clients)
is unnecessary. The special-case code is interdependent with the changes in
#1 though, so these changes are being made in one commit, despite the fact
one could argue they are logically separate.
* subversion/libsvn_ra_neon/fetch.c
(report_baton_t, start_element): Remove comments about 'modern' servers and
replace as needed with discussion of whether we are in 'send-all' mode or
not.
* subversion/mod_dav_svn/reports/update.c
(struct item_baton_t): Remove changed_props, committed_rev, committed_date,
and last_author members, none of these are necessary.
(close_helper): Simplify check for when we need to send remove-prop.
Remove code which sends 'fetch-props' to the client, we've already sent
them. Stop sending md5-checksum, version-name, creationdate, and
creator-displayname. All four are already sent elsewhere and latter three
were only needed by 'non-modern clients' which don't actually exist!
(upd_change_xxx_prop): Whether we are in send-all mode or not, we have the
property changes already so send them now, rather than telling the client
to 'fetch-props' later. The only time we don't send the props is for
additions while not in send-all mode (i.e. skelta mode). In that case we
do the bare minimum necessary to not break old clients re issue #3711.
]]]
[1] No unexpected failures, but there were these:
Serf 1.5.10 and 1.6.16:
FAIL: prop_tests.py 14: 'test binary property support'
This passes with Passes with r1058722
(http://svn.apache.org/viewvc?view=revision&revision=1058722)
backported (this is nominated for backport to both 1.5.x and 1.6.x).
Neon 1.5.10 and 1.6.16:
FAIL: log_tests.py 26: 'svn log -g target_with_bogus_mergeinfo'
This fails without the patch too, the failure is due to the way the
test is written, see
http://svn.apache.org/viewvc?view=revision&revision=926065, a change
that was (understandably) never backported.
Index: subversion/libsvn_ra_neon/fetch.c
===================================================================
--- subversion/libsvn_ra_neon/fetch.c (revision 1060528)
+++ subversion/libsvn_ra_neon/fetch.c (working copy)
@@ -188,10 +188,11 @@
/* Use an intermediate tmpfile for the REPORT response. */
svn_boolean_t spool_response;
- /* A modern server will understand our "send-all" attribute on the
- update report request, and will put a "send-all" attribute on
- its response. If we see that attribute, we set this to true,
- otherwise, it stays false (i.e., it's not a modern server). */
+ /* If this report is for a switch, update, or status (but not a
+ merge/diff), then we made the update report request with the "send-all"
+ attribute. The server reponds to this by putting a "send-all" attribute
+ in its response. If we see that attribute, we set this to true,
+ otherwise, it stays false. */
svn_boolean_t receiving_all;
/* Hash mapping 'const char *' paths -> 'const char *' lock tokens. */
@@ -1588,19 +1589,16 @@
/* push the new baton onto the directory baton stack */
push_dir(rb, new_dir_baton, pathbuf, subpool);
- /* Property fetching is implied in addition. This flag is only
- for parsing old-style reports; it is ignored when talking to
- a modern server. */
+ /* Property fetching is implied in addition. */
TOP_DIR(rb).fetch_props = TRUE;
bc_url = svn_xml_get_attr_value("bc-url", atts);
- /* In non-modern report responses, we're just told to fetch the
+ /* If we are not in send-all mode, we're just told to fetch the
props later. In that case, we can at least do a pre-emptive
depth-1 propfind on the directory right now; this prevents
individual propfinds on added-files later on, thus reducing
- the number of network turnarounds (though not by as much as
- simply getting a modern report response!). */
+ the number of network turnarounds. */
if ((! rb->receiving_all) && bc_url)
{
apr_hash_t *bc_children;
@@ -1711,9 +1709,7 @@
crev, rb->file_pool,
&rb->file_baton));
- /* Property fetching is implied in addition. This flag is only
- for parsing old-style reports; it is ignored when talking to
- a modern server. */
+ /* Property fetching is implied in addition. */
rb->fetch_props = TRUE;
break;
Index: subversion/mod_dav_svn/reports/update.c
===================================================================
--- subversion/mod_dav_svn/reports/update.c (revision 1060528)
+++ subversion/mod_dav_svn/reports/update.c (working copy)
@@ -98,14 +98,8 @@
svn_boolean_t text_changed; /* Did the file's contents change? */
svn_boolean_t added; /* File added? (Implies text_changed.) */
svn_boolean_t copyfrom; /* File copied? */
- apr_array_header_t *changed_props; /* array of const char * prop names */
apr_array_header_t *removed_props; /* array of const char * prop names */
- /* "entry props" */
- const char *committed_rev;
- const char *committed_date;
- const char *last_author;
-
} item_baton_t;
@@ -413,8 +407,16 @@
return SVN_NO_ERROR;
/* ### ack! binary names won't float here! */
- /* If this is a copied file/dir, we can have removed props. */
- if (baton->removed_props && (! baton->added || baton->copyfrom))
+ /* If this is a copied file/dir, we can have removed props.
+
+ Old features never die: 1.7+ clients don't require this block because
+ they never ask for copyfrom information from the server when adding
+ files created by a copy, but 1.5-1.6 clients will ask for it so we
+ have to keep sending it.
+
+ See http://svn.haxx.se/dev/archive-2010-09/0265.shtml and
+ http://subversion.tigris.org/issues/show_bug.cgi?id=3711. */
+ if (baton->removed_props && baton->copyfrom)
{
const char *qname;
int i;
@@ -429,61 +431,6 @@
}
}
- if ((! baton->uc->send_all) && baton->changed_props && (! baton->added))
- {
- /* Tell the client to fetch all the props */
- SVN_ERR(dav_svn__brigade_puts(baton->uc->bb, baton->uc->output,
- "<S:fetch-props/>" DEBUG_CR));
- }
-
- SVN_ERR(dav_svn__brigade_puts(baton->uc->bb, baton->uc->output, "<S:prop>"));
-
- /* Both modern and non-modern clients need the checksum... */
- if (baton->text_checksum)
- {
- SVN_ERR(dav_svn__brigade_printf(baton->uc->bb, baton->uc->output,
- "<V:md5-checksum>%s</V:md5-checksum>",
- baton->text_checksum));
- }
-
- /* ...but only non-modern clients want the 3 CR-related properties
- sent like here, because they can't handle receiving these special
- props inline like any other prop.
- ### later on, compress via the 'scattered table' solution as
- discussed with gstein. -bmcs */
- if (! baton->uc->send_all)
- {
- /* ### grrr, these DAV: property names are already #defined in
- ra_dav.h, and statically defined in liveprops.c. And now
- they're hardcoded here. Isn't there some header file that both
- sides of the network can share?? */
-
- /* ### special knowledge: svn_repos_dir_delta2 will never send
- *removals* of the commit-info "entry props". */
- if (baton->committed_rev)
- SVN_ERR(dav_svn__brigade_printf(baton->uc->bb, baton->uc->output,
- "<D:version-name>%s</D:version-name>",
- baton->committed_rev));
-
- if (baton->committed_date)
- SVN_ERR(dav_svn__brigade_printf(baton->uc->bb, baton->uc->output,
- "<D:creationdate>%s</D:creationdate>",
- baton->committed_date));
-
- if (baton->last_author)
- SVN_ERR(dav_svn__brigade_printf(baton->uc->bb, baton->uc->output,
- "<D:creator-displayname>%s"
- "</D:creator-displayname>",
- apr_xml_quote_string(baton->pool,
-
baton->last_author,
- 1)));
-
- }
-
- /* Close unconditionally, because we sent checksum unconditionally. */
- SVN_ERR(dav_svn__brigade_puts(baton->uc->bb, baton->uc->output,
- "</S:prop>" DEBUG_CR));
-
if (baton->added)
SVN_ERR(dav_svn__brigade_printf(baton->uc->bb, baton->uc->output,
"</S:add-%s>" DEBUG_CR,
@@ -646,8 +593,10 @@
if (qname == name)
qname = apr_pstrdup(b->pool, name);
-
- if (b->uc->send_all)
+ /* Even if we are not in send-all mode we have the prop changes already,
+ so send them to the client now instead of telling the client to fetch
+ them later. */
+ if (b->uc->send_all || !b->added)
{
if (value)
{
@@ -683,56 +632,16 @@
qname));
}
}
- else /* don't do inline response, just cache prop names for close_helper */
+ else if (!value) /* This is an addition in 'skelta' mode so there is no
+ need for an inline response since property fetching
+ is implied in addition. We shouldn't need to do
+ anything, but we must cache property removals, see
+ note re issue #3711 in close_helper(). */
{
- /* For now, store certain entry props, because we'll need to send
- them later as standard DAV ("D:") props. ### this should go
- away and we should just tunnel those props on through for the
- client to deal with. */
-#define NSLEN (sizeof(SVN_PROP_ENTRY_PREFIX) - 1)
- if (! strncmp(name, SVN_PROP_ENTRY_PREFIX, NSLEN))
- {
- if (strcmp(name, SVN_PROP_ENTRY_COMMITTED_REV) == 0)
- {
- b->committed_rev = value ?
- apr_pstrdup(b->pool, value->data) : NULL;
- }
- else if (strcmp(name, SVN_PROP_ENTRY_COMMITTED_DATE) == 0)
- {
- b->committed_date = value ?
- apr_pstrdup(b->pool, value->data) : NULL;
- }
- else if (strcmp(name, SVN_PROP_ENTRY_LAST_AUTHOR) == 0)
- {
- b->last_author = value ?
- apr_pstrdup(b->pool, value->data) : NULL;
- }
- else if ((strcmp(name, SVN_PROP_ENTRY_LOCK_TOKEN) == 0)
- && (! value))
- {
- /* We only support delete of lock tokens, not add/modify. */
- if (! b->removed_props)
- b->removed_props = apr_array_make(b->pool, 1, sizeof(name));
- APR_ARRAY_PUSH(b->removed_props, const char *) = qname;
- }
- return SVN_NO_ERROR;
- }
-#undef NSLEN
+ if (! b->removed_props)
+ b->removed_props = apr_array_make(b->pool, 1, sizeof(name));
- if (value)
- {
- if (! b->changed_props)
- b->changed_props = apr_array_make(b->pool, 1, sizeof(name));
-
- APR_ARRAY_PUSH(b->changed_props, const char *) = qname;
- }
- else
- {
- if (! b->removed_props)
- b->removed_props = apr_array_make(b->pool, 1, sizeof(name));
-
- APR_ARRAY_PUSH(b->removed_props, const char *) = qname;
- }
+ APR_ARRAY_PUSH(b->removed_props, const char *) = qname;
}
return SVN_NO_ERROR;
