Re: difference between svn copy -r REV SRC and SRC@REV
On Wed, Jan 15, 2014 at 10:19 PM, Ben Reser wrote: > On 1/15/14, 1:05 PM, Mojca Miklavec wrote: >> I was wondering whether "-r REV SRC" and "SRC@REV" in "svn copy" are >> supposed to behave differently or not. > > Yes they have different behavior. You probably want to read about Peg and > Operative Revisions here: > http://svnbook.red-bean.com/en/1.7/svn.advanced.pegrevs.html Thank you very much. I probably asked too fast. I just found the same link on http://stackoverflow.com/questions/6591938/svn-copy-r-revision-number-url-path (I was trying to find the information via "svn help copy" earlier, but the help page wasn't informative enough for this particular case.) Mojca
Re: difference between svn copy -r REV SRC and SRC@REV
On 1/15/14, 1:05 PM, Mojca Miklavec wrote: > I was wondering whether "-r REV SRC" and "SRC@REV" in "svn copy" are > supposed to behave differently or not. Yes they have different behavior. You probably want to read about Peg and Operative Revisions here: http://svnbook.red-bean.com/en/1.7/svn.advanced.pegrevs.html
Re: Locking whole directories
On 1/15/14, 12:44 PM, Kyle Sluder wrote: > Is there foreseeable benefit to moving them into the FS layer? I can't think of any benefit other than a design that seems cleaner. I'm sure Branko can though. > To be completely explicit, the problem as I understand it is that since > touching a file in the FS layer touches all its ancestors, implementing > non-recursive locks in the FS layer would be problematic because > touching an unlocked file might require touching a locked ancestor. That's a theoretical problem of a theoretical fs implementation that doesn't exist (and probably can't exist without Subversion 2.0). Branko's thinking about that because he's been thinking about completely new FS implementations. > How specifically does implementing locks in the RA layer sidestep the > issue? Is it because the bubble-up semantics are confined to the FS > layer, or is it because the RA layer can choose to bend the rules and > say "yes, committing changes to /a/b/c bumps the revision of > non-recursively-locked ancestor /a, but since there are no real > modifications to that directory, I will allow this change"? Does this > have any implications for higher levels of the stack—for example, would > a DAV client be confused that a supposedly-locked collection has > changed? The fact that the bubble up happens is an implementation detail of the FS layer. For instance, Branko actually has thought about a design without the bubble up. The RA layer doesn't really know about that bubble up. It just knows that you're touching /a/b/c, so it checks the lock state on that node (and if we add recursive locks applying whatever sort of inheritance rules apply to the locks that exist). I.E. it's checking the actual operations that you're executing against the locks, not the side effects on the DAG. DAV clients support a 0 depth (non-recursive) and an infinity depth. 0 depth means only the collection itself is locked. Though that does include changing the children of the collection since that's a property of the collection. So a 0 depth lock would not allow you to add or remove files from the collection, but you could change the contents of files already present. Infinity depth is fairly obvious, all children of the collection and their children and so on are locked. If this is what we add support for I don't see any issues for DAV clients. Clients can't presume that their locks can't be removed except by them since the DAV spec says that servers may allow privileged users other than the lock creator to remove locks: http://webdav.org/specs/rfc4918.html#lock-creator I suspect DAV clients in general don't ever try to use locks on directories anyway since they we haven't ever had someone come along complaining that their DAV client doesn't work with auto-versioning due to our lack of directory locks (though I'll admit actual use of auto-versioning is pretty limited). So I'm having a hard time envisioning a scenario with adding support for directory locks (non-recursive or recursive) breaks them.
difference between svn copy -r REV SRC and SRC@REV
Hi, I was wondering whether "-r REV SRC" and "SRC@REV" in "svn copy" are supposed to behave differently or not. I was trying to recover a file that has already been deleted from the repository, but using "-r" syntax failed to work while @REV completed successfully: > svn copy -r 10 ^/trunk/path/to/my/file.txt . svn: E160013: '/repository/name/!svn/bc/25/trunk/path/to/my/file.txt' path not found > svn copy ^/trunk/path/to/my/file.txt@10 . A /here/file.txt On the other hand, if the file still exists, the "-r REV" syntax works just fine and copies the old file (revision 10 in the above example). Is this expected? I'm using SVN 1.8.5 (and also tried with the trunk version which behaves the same), while the server uses SVN 1.6.11. Mojca (PS: If necessary I can provide real URLs.)
Re: Locking whole directories
On Wed, Jan 15, 2014, at 11:27 AM, Ben Reser wrote: > On 1/15/14, 10:34 AM, Branko Čibej wrote: > > We certainly have to hack thinks to map non-recursive directory locks to any > > reasonable locking model in Subversion. This is because of Subversion's > > bubble-up storage model, in which the revisions of all parent directories > > of a > > change are updated by a commit. > > I don't really see that as a problem at all. Locks are enforced at the > RA > layer. For instance in mod_dav's case the locks are actually enforced by > mod_dav itself with it asking mod_dav_svn if there are any applicable > locks. > So the bubble up wouldn't even impact the current implementation if we > extended > it for directories. > > If we wanted to move the enforcement into the filesystems at some point, > I > guess that would become a complication, but until/unless we get rid of > using > mod_dav I think that's not going to be palatable. Is there foreseeable benefit to moving them into the FS layer? To be completely explicit, the problem as I understand it is that since touching a file in the FS layer touches all its ancestors, implementing non-recursive locks in the FS layer would be problematic because touching an unlocked file might require touching a locked ancestor. How specifically does implementing locks in the RA layer sidestep the issue? Is it because the bubble-up semantics are confined to the FS layer, or is it because the RA layer can choose to bend the rules and say "yes, committing changes to /a/b/c bumps the revision of non-recursively-locked ancestor /a, but since there are no real modifications to that directory, I will allow this change"? Does this have any implications for higher levels of the stack—for example, would a DAV client be confused that a supposedly-locked collection has changed? --Kyle Sluder
Re: Locking whole directories
On 1/15/14, 10:34 AM, Branko Čibej wrote: > We certainly have to hack thinks to map non-recursive directory locks to any > reasonable locking model in Subversion. This is because of Subversion's > bubble-up storage model, in which the revisions of all parent directories of a > change are updated by a commit. I don't really see that as a problem at all. Locks are enforced at the RA layer. For instance in mod_dav's case the locks are actually enforced by mod_dav itself with it asking mod_dav_svn if there are any applicable locks. So the bubble up wouldn't even impact the current implementation if we extended it for directories. If we wanted to move the enforcement into the filesystems at some point, I guess that would become a complication, but until/unless we get rid of using mod_dav I think that's not going to be palatable. > Inheritance is also an interesting topic, and the fact that Subversion allows > locks to be created on nonexistent paths (and that our locks don't record the > recursive vs. non-recursive bit) would also make for interesting side effects. Non-existent paths for directories are less of an issue that you might think. The need for locking non-existent files (in generic DAV) is because PUT overwrites a file's content even if it already exists. So two users PUT'ing to the same location will end up having a race with only the last users content being preserved. The old DAV spec implemented this by using the whole lock null stuff where you don't actually create the file but the server keeps track that a lock null resource exists (clients can then pretend like the file exists or not depending on their preferences). Collections don't have this issue because you can't use MKCOL to overwrite an existing collection. You'd have to explicitly remove the existing directory. So in the case of two users both trying to create a collection, they'd race and the second one would fail due to the already existing collection. DAV when it was updated in 2007 deprecated this whole lock null nonsense and replaced it: http://webdav.org/specs/rfc4918.html#lock-null http://webdav.org/specs/rfc4918.html#lock-unmapped-urls As I read the spec, it seems to me that it is required that all LOCKS on non-existent (unmapped in DAV terminology) URLS are treated as file locks, and that creating a lock on a non-existing collection is not allowed. Since you don't NEED the lock on a non-existent directory due to the behavior of MKCOL I believe this makes sense. Additionally, we only allow locks on non-existent paths with auto-versioning, so our support here is to entirely work around the above. There's not even a UI confusion with a svn client since we disallow this behavior from them. One complication if we switch to the new LOCK model for DAV (LOCK actually creates empty files) is that a LOCK on an non-existent file would become a commit and a lock. I suspect we don't want to mess with that since the lock null resource implementation works fine and I don't think there's any need to switch in order to support directory locks, the model doesn't appear to be altered from the RFC 2518 spec. We'd need a format bump to add a depth field, but that doesn't seem to be a huge deal. It'd just be yet another repository controlled capability. Since all existing locks are file locks it's trivial to add the depth field (for that matter it's irrelevant what the depth field is because the behavior for all the values of depth for a file is the same). The biggest effort with respect to the storage is making it efficient to search for inherited locks. Right now create an index for locks but it can be flaky, in order for inherited locks to work well it'd have to be rock solid.
Re: Seeking help for "E000054: Error retrieving REPORT: Connection reset by peer"
On 09.01.2014 20:19, Branko Čibej wrote: > On 09.01.2014 17:09, Mojca Miklavec wrote: >> I'm unable to reproduce the faulty behaviour if I do a checkout from >> the same network where the server is located, no matter what I try >> (upgrading SVN client doesn't "help" triggering the error). Philip >> also said that he had no problem doing a checkout with client version >> 1.8.5 or 1.7. > > This confirms my suspicion that the error is triggered by some part of > the network infrastructure between your server and the outside world. > That's why I asked if there is a load-balancer involved. It could also > be caused by some kind of transparent proxy, or even a packet > analyzer. I doubt that your server is open to the world without some > kind of security measures in place. > > To be clear, I'm not saying that any of these things are configured > incorrectly; only that they may be interacting with Subversion in a > way that we don't handle well. One of the major differences between > 1.7 (which works) and 1.8 (which fails) is that we try to work around > issues with non-standard behaviour of certain "transparent" (sic) > proxies; and we can't claim to have covered all the possibilities. > > I can't see a way to figure out what's going on without help from your > network admins; we need some insight into why the connection is being > reset on the server side, and analyzing the TCP stream on the client > can't tell us that. > > > BTW, if you think it'd help to try a live debugging session, I'm only > about an hour's drive away from IJS. So to wrap this up: they managed to fix the problem themselves, and it was indeed "some part of the network infrastructure;" the specifics are as follows: They have a Cisco ASA 5580 running IOS 9.1(4), and they had HTTP protocol inspection turned on; the configuration was as follows: policy-map type inspect http HTTP-CONTROL parameters protocol-violation action log policy-map global_policy class inspection_default inspect http HTTP-CONTROL The ASA was closing the connections, and their logs contained one of the following two reasons: %ASA-4-415016: policy-map HTTP-CONTROL:Maximum number of unanswered HTTP requests exceeded - Resetting connection from Ext:x.x.x.x/59769 to Int:y.y.y.y/80 %ASA-4-507003: tcp flow from Ext:x.x.x.x/59769 to Int:y.y.y.y/80 terminated by inspection engine, reason - reset unconditionally. The only reasonable explanation we could come up with was that moderately low bandwidth and high latency between client and server, coupled with the fact that some of the files in the repository are rather large and take a while to transfer, caused the 1.8 client to queue up enough pipelined GET requests during checkout that the firewall decided to call quits. A 1.7 client (using serf) did not exhibit this problem, because it also sends PROPGETS, and this apparently changed the timing enough that the number of pipelined requests never exceeded the ASA's configured maximum. Apparently this is not a new problem, having been reported before: https://supportforums.cisco.com/thread/2088590 They fixed the issue by switching off HTTP protocol inspection on the ASA. Interestingly enough, this also fixed a number of intermittent issues with plain ol' Web browsing that they had on occasion, so this is not specific to Subversion (as the link above also suggests), but is rather a bug^Wserious limitation of the ASA and/or IOS. -- Brane -- Branko Čibej | Director of Subversion WANdisco // Non-Stop Data e. br...@wandisco.com
Re: Locking whole directories
On 15.01.2014 19:28, Ben Reser wrote: > On 1/15/14, 10:22 AM, Kyle Sluder wrote: >> To be clear, this isn't implying that `svn lock` of a directory will work >> over DAV, correct? Just that the repository supports storing the fact that >> some other DAV client has LOCKed a directory, and the DAV protocol layer >> supports communicating that information back to the `svn` client? > Not even that much is done. I meant that the DAV RFC includes details on how > locks on collections (DAV's name for directories) should work. DAV handles > the > inheritance by supporting a depth parameter for the lock. > > Basic model for how looking should work that specifies some behavior related > to > depth-infinity locks: > http://webdav.org/specs/rfc4918.html#lock-model > > Specific section on directory locks. > http://webdav.org/specs/rfc4918.html#write.locks.and.collections > > One problem that may arise here is we may not like how DAV implemented > directory locks. If not then we have the issue of our LOCK method support not > really being compatible with DAV. We certainly have to hack thinks to map non-recursive directory locks to any reasonable locking model in Subversion. This is because of Subversion's bubble-up storage model, in which the revisions of all parent directories of a change are updated by a commit. Inheritance is also an interesting topic, and the fact that Subversion allows locks to be created on nonexistent paths (and that our locks don't record the recursive vs. non-recursive bit) would also make for interesting side effects. -- Brane -- Branko Čibej | Director of Subversion WANdisco // Non-Stop Data e. br...@wandisco.com
Re: Locking whole directories
Here is at least one old thread on this topic: http://svn.haxx.se/dev/archive-2004-10/1228.shtml Sadly, I have no recollection of ever participating in this discussion, but there I am :) Mark
Re: Locking whole directories
On 1/15/14, 10:22 AM, Kyle Sluder wrote: > To be clear, this isn't implying that `svn lock` of a directory will work > over DAV, correct? Just that the repository supports storing the fact that > some other DAV client has LOCKed a directory, and the DAV protocol layer > supports communicating that information back to the `svn` client? Not even that much is done. I meant that the DAV RFC includes details on how locks on collections (DAV's name for directories) should work. DAV handles the inheritance by supporting a depth parameter for the lock. Basic model for how looking should work that specifies some behavior related to depth-infinity locks: http://webdav.org/specs/rfc4918.html#lock-model Specific section on directory locks. http://webdav.org/specs/rfc4918.html#write.locks.and.collections One problem that may arise here is we may not like how DAV implemented directory locks. If not then we have the issue of our LOCK method support not really being compatible with DAV.
Re: Locking whole directories
On Wed, Jan 15, 2014 at 1:16 PM, Ben Reser wrote: > On 1/15/14, 9:52 AM, Branko Čibej wrote: > > There's probably a good reason why we didn't implement directory locks > > when the locking feature was developed. In any case it's not a trivial > > feature. > > I don't recall what the reason was at the time, but I'd bet it was just a > pain > to deal with the inheritance and since most people were satisfied with file > locks, everyone just stopped there. > It would not be hard to rattle off half a dozen scenarios where it is not obvious what the behavior should be. Would not be surprised if it was punted to the future so that *something* could be delivered. -- Thanks Mark Phippard http://markphip.blogspot.com/
Re: Locking whole directories
> On Jan 15, 2014, at 9:39 AM, Ben Reser wrote: > >> On 1/13/14, 7:02 PM, Kyle Sluder wrote: >> I understand that implementing this would require all commits to search >> for lock properties on every ancestor of every changed file or >> directory. It essentially amounts to an extension of inheritable >> properties to the RA layer. Which would be pretty neat, IMO. :) > > Well not really. Locks aren't a property so none of the inheritable > properties > work helps here. Inheritable properties also already have RA support (which > is > a big reason to prefer a 1.8 server to be used with a 1.8 client since the RA > support is much more efficient than walking the tree). Oh, well this is a very good thing to learn. It means that a certain tool I want to create will be much easier to write. > > DAV already has support for recursive locks (including locks) on directories. > So we'd shouldn't need any new protocol changes for DAV. Presumably we can > come up with something efficient for svnserve. To be clear, this isn't implying that `svn lock` of a directory will work over DAV, correct? Just that the repository supports storing the fact that some other DAV client has LOCKed a directory, and the DAV protocol layer supports communicating that information back to the `svn` client? > >> Shall I file a feature request? > > Surprised we don't already have a feature request for directory locks already. > So please do so. > > Understand that filing an issue doesn't mean it's going to be immediately > worked on. Will do. I might even hack on this a little bit myself. --Kyle Sluder
Re: Locking whole directories
On 1/15/14, 9:52 AM, Branko Čibej wrote: > There's probably a good reason why we didn't implement directory locks > when the locking feature was developed. In any case it's not a trivial > feature. I don't recall what the reason was at the time, but I'd bet it was just a pain to deal with the inheritance and since most people were satisfied with file locks, everyone just stopped there.
Re: Locking whole directories
On 15.01.2014 18:39, Ben Reser wrote: > On 1/13/14, 7:02 PM, Kyle Sluder wrote: >> I understand that implementing this would require all commits to search >> for lock properties on every ancestor of every changed file or >> directory. It essentially amounts to an extension of inheritable >> properties to the RA layer. Which would be pretty neat, IMO. :) > Well not really. Locks aren't a property so none of the inheritable > properties > work helps here. Inheritable properties also already have RA support (which > is > a big reason to prefer a 1.8 server to be used with a 1.8 client since the RA > support is much more efficient than walking the tree). > > DAV already has support for recursive locks (including locks) on directories. > So we'd shouldn't need any new protocol changes for DAV. Presumably we can > come up with something efficient for svnserve. > >> Shall I file a feature request? > Surprised we don't already have a feature request for directory locks already. > So please do so. > > Understand that filing an issue doesn't mean it's going to be immediately > worked on. There's probably a good reason why we didn't implement directory locks when the locking feature was developed. In any case it's not a trivial feature. -- Brane -- Branko Čibej | Director of Subversion WANdisco // Non-Stop Data e. br...@wandisco.com
Re: Locking whole directories
On 1/13/14, 7:02 PM, Kyle Sluder wrote: > I understand that implementing this would require all commits to search > for lock properties on every ancestor of every changed file or > directory. It essentially amounts to an extension of inheritable > properties to the RA layer. Which would be pretty neat, IMO. :) Well not really. Locks aren't a property so none of the inheritable properties work helps here. Inheritable properties also already have RA support (which is a big reason to prefer a 1.8 server to be used with a 1.8 client since the RA support is much more efficient than walking the tree). DAV already has support for recursive locks (including locks) on directories. So we'd shouldn't need any new protocol changes for DAV. Presumably we can come up with something efficient for svnserve. > Shall I file a feature request? Surprised we don't already have a feature request for directory locks already. So please do so. Understand that filing an issue doesn't mean it's going to be immediately worked on.
Re: Slow VM, svn client drops connection with FIN packet
I replied yesterday, but my post seems to have disappeared. I figured it out, and the key was your reproduction and my realisation I wasn't seeing a FIN from my server. It turns out that in every case the last 392598 71.906863000 10.222.3.88 10.14.11.50 HTTP 1434 Continuation or non-HTTP traffic line from the server is actually also a FIN packet, and Wireshark (or the TCP dissector) is "hiding" that; if I look at the packet I can see the FIN flag, but it's not mentioned in the Info field in the packet list. So the FINs from my client are close handshake responses not initiations, and it is the server timing out after all. Thanks for the help. I'm checking with my IT dept to see if they can bump up the timeout. ...Stu On Tuesday, January 14, 2014 11:55:39 AM UTC-5, Philip Martin wrote: > > Stuart MacDonald > writes: > > > svn: E175002: REPORT of '/svn/repos/deckard-65x/!svn/me': Could not read > > response body: connection was closed by server (http://foundry51.qnx.com) > > > I can provoke this on my local LAN by setting the Apache timeout on the > server to a low value such as 5 seconds. Then I run a checkout that > generates a large REPORT response and I use gdb to interrupt the client > at subversion/libsvn_ra_neon/fetch.c:1709 so that the client is in the > middle of reading the repsonse. I wait for the server timeout to > expire and then resume the client and I get the error: > > svn: E175002: REPORT of '/obj/repo/!svn/me': Could not read response body: > connection was closed by server (http://peri.local:) > > One fix is to increase the server timeout to handle your slow client. > > -- > Philip Martin | Subversion Committer > WANdisco // *Non-Stop Data* >
