RE: one's own posts Re: Can you hear me now?
>From: Stefan Sperling [mailto:s...@elego.de] >Sent: Wednesday, November 03, 2010 3:33 PM > >On Wed, Nov 03, 2010 at 09:20:32PM +0200, Daniel Shahaf wrote: >> Daniel Shahaf wrote on Wed, Nov 03, 2010 at 21:17:09 +0200: >> > Andy Levy wrote on Wed, Nov 03, 2010 at 15:04:29 -0400: >> > > On Wed, Nov 3, 2010 at 14:54, Andersen, Krista > wrote: >> > > > I tried posting a question to this users list last night and >> > > > I still don't see it here (nor any responses). Am I being >> > > > blocked? Am I being too impatient? Will I simply not see my own >> > > > posts until someone replies? >> > > >> > > Your own posts are not sent back to you. >> >> Are you sure? I certainly get all my own posts via the list. > >Andy is using gmail, which hides a user's own posts to mailing lists. I have found that our corporate mail server rejects email sent from me (m...@company.com) that arrives at our SMTP server under the impression that mail from m...@company.com should not originate from outside the company. When I send email to the list, the list is processed, in effect, as a big mail alias, which then gets resent back to me. Thus, it considers my posting as a forged email when it arrives here. I don't have the problem with all lists, so maybe some of them rewrite the from/envelope/return-address/whatever in the distribution process, and so they are not rejected. Google might be doing the same thing. I ended up turning on "send me acks" for those lists that do this. -- -Justin
RE: Granting full access to a directory, readonly access to path to directory, deny access to rest of tree?
> From: Bob Archer [mailto:bob.arc...@amsi.com] > > I'm pretty sure this works... although there was a bug with the group being > able to create a branch in their allowed path if they didn't have read access > to root. However, I think this was fixed in a recent version .10 or newer > perhaps. You can check the change logs. Thanks for the idea. I had assumed that you would need read access to the path leading to the node. I didn't see any mention of a security issue in http://svn.apache.org/repos/asf/subversion/trunk/CHANGES.
Granting full access to a directory, readonly access to path to directory, deny access to rest of tree?
Suppose I have a bunch of projects, and I want to grant full access to a group, but no access to anything else. Please don't call me anti-social. /trunk/proja /trunk/projb /trunk/projc I want to grant full access to proja to groupa, but no access to the others. How can I do this? [repo:/] @groupa = r @others = rw [repo:/trunk/proja] @groupa = rw [repo:/trunk/projb] @groupa = [repo:/trunk/projc] @groupa = However, this does not scale well. When I add projd, I need to make sure that I remove access (@groupa=;) for all the groups that should not have access. That is, I am practicing negative access control (deny access), which is error prone. Is there a way for the permissions to not be recursive, so that I could grant @groupa access to / without it applying to /**? We could reorg the repo (/trunk/secret and /trunk/groupa), but that seems like the tail wagging the dog (security issues dictating repo layout). -- -Justin
RE: Support for filesystem snapshots (?)
> From: Les Mikesell [mailto:lesmikes...@gmail.com] > > A filesystem snapshot should present exactly the same scenario as a machine > that > lost power or crashed for some similar reason at that moment, so the question > boils down to whether subversion can recover sensibly from a crash at any > point. > The fsync's are going to be the critical thing for ordering operations at > the > disk level, however there is no guarantee how much of the fsync operation > might > have completed when a crash or snapshot happens. ...and therefore applications (should) make no assumptions about the order of writing, or, in the case of a crash, what data actually was written. In generally, atomic updates have three phases: prepare, commit, apply. In prepare, all changes are written in a manner that either does not touch the main data (journal), or can be undone. In commit, an atomic write marks the prepared data as committed. In apply, changes written in the prepare phase are applied to the main data if required (ie: journal entries are "replayed" to the main database). The only atomic requirement here is that after prepare, we can flush everything (order is not important) and the commit phase can be written atomically. For example, the db/current file is a integer string. Since that file fits in a single disk block, there is no chance that an update to db/current does not leave either the old or the new contents in the file. -- -Justin
RE: Support for filesystem snapshots (?)
(c) is an automatic hands-off recovery, either by just having the recovery logic know how to clean up (delete stale files) or making the recovery logic transparent (overwrite files). This would be possible where locks are managed by the kernel/filesystem which "knows" when a process dies, and can release and pass a lock on to the next waiting locker. (b) is a case where you have to issue an "svnadmin cleanup-lock" kind of command (don't know if you need to). If the svn repo filesystem uses lock files (touch svnrepo/mylock), then killing the process (kill -9) will not automatically remove the lock, and user intervention will be required to delete the lock file. This is the case for rcs-locks, svn-sandbox locks, lockfile(1) locks. -- -Justin -Original Message- From: Vincent Lefevre [mailto:vincent-...@vinc17.net] Sent: Tuesday, August 03, 2010 8:03 AM To: users@subversion.apache.org Subject: Re: Support for filesystem snapshots (?) On 2010-08-02 14:41:29 -0400, Vallon, Justin wrote: > That is the situation I raised. If the network connection between > the host that is modifying the repository and the filesystem that > houses the repository is lost, will be repository be (a) corrupt, > (b) require cleanup (locks, etc), (c) pristine? > > (c) is great > (b) is ok > (a) means the operations are not transactional > > Further: > > (c) snapshots work > (b) snapshots work, but need a cleanup after restore > (a) snapshots don't work; repository is not transactional, either I don't see how (c) could be possible, as a commit requires several filesystem operations (even for updating "current"). Of course, (b) + automatical clean-up could look like (c). -- Vincent Lefèvre - Web: <http://www.vinc17.net/> 100% accessible validated (X)HTML - Blog: <http://www.vinc17.net/blog/> Work: CR INRIA - computer arithmetic / Arénaire project (LIP, ENS-Lyon)
RE: Support for filesystem snapshots (?)
From: Les Mikesell [mailto:lesmikes...@gmail.com] > On 8/2/2010 3:46 PM, Vallon, Justin wrote: > > That means filesystem snapshots pass the transactional test. > > Maybe - is there a guarantee that the app flushes to disk in the > expected order? Or do snapshots take the current dirty filesystem > buffers into account? If the client or server filesystem buffers are dirty, then the application has not flushed the data. Therefore, the application should make no assumption about whether the data has been written through to its final media. What is critical is that (a) everything gets flushed, then (b) current is updated, then (c) current is flushed. This generalizes to: 1) Do whatever you want so long as it can be discarded and/or be written out of order 2) Flush everything 3) Write a flag saying that everything has been committed 4) Flush again On the topic of whether dirty server-side buffers are in the snapshot, there are two reasons the data could be dirty: a) The client is in the middle of writing some data, the server is about to write it, and the client is waiting. b) The client wrote the data asynchronously, and the server can write it whenever it wants (async write). In either case, it would not be surprising for the data to be included in the snapshot or excluded. However, in neither case should the client have expected that the data be present in the snapshot, since it was either in-progress or a delayed write. All of this can get thrown out the window if the server decides to "lie". See http://nfs.sourceforge.net/nfs-howto/ar01s05.html, section 5.9, where it describes how the Linux async NFS export option can cause the NFS server to lie when asked to fsync or flush-on-close. The problem here is that the server might keep the revs (part 1) in cache, while writing out the current file (part 3), leaving a corrupted repository if the server fails. So, this is a problem in general, not specific to filesystem snapshots. -- -Justin
RE: Support for filesystem snapshots (?)
From: Stefan Sperling [mailto:s...@elego.de] > users asking interesting questions: > http://mail-archives.apache.org/mod_mbox/subversion-users/201008.mbox/%3c6ec02a00cc9f684daf4af4084ca84d5f01c40...@drmbx3.winmail.deshaw.com%3e > i dunno how fsfs behaves in face of an interrupted commit; whether or > not it needs to be rolled back > if you haven't touched current than the rev file will never be > read and will be overwritten > stsp: does that answer your question? > i think so > because the rev file of the following commit will have the same name > to move things into place onto > write lock only for revprop change and commit > :-) > > Now, how does rsync, or a file-system snapshot, know to make sure that > 'current' is always copied first? Even if you copy 'current' first manually, > rsync might later overwrite it. But unless you use packing it's trivial to > fix the backup if it breaks, and all you risk is losing the most recent HEAD > revision, which you may not have gotten with a hotcopy anyway. When I speak of a filesystem snapshot, I mean an instantaneous copy of the volume (ala NetApp, EMC, ZFS). In this case, there is a guarantee that if we snap the new "current", then we will also have the other files (assuming that they have been flushed, etc, by the client). Further, it sounds like (a) subsequent commits will not run into trouble because of the partial commit, and (b) the repository will not be otherwise affected by a partial commit. That means filesystem snapshots pass the transactional test. rsync does not pass this transactional test due to order-of-copying. rsync with some "current"-first logic would pass the test, but then we might as well use the hot-backup script. -- -Justin
RE: Support for filesystem snapshots (?)
> From: Stefan Sperling [mailto:s...@elego.de] > On Mon, Aug 02, 2010 at 12:42:31PM -0400, Vallon, Justin wrote: > > I did see that discussion, but it seems to contradict with the claim > > that the database operations are transactional. > > This is a frequent misunderstanding of the "atomic commit" concept. I suppose that I am assuming that the repository operations would all be transactional. Without transactional guarantees, you can have database corruption if a modification is interrupted. > E.g. Subversion's FSFS needs to create a revision file from the commit's > transaction, and move the finalized revision file into place. > After the revision file has been moved into place successfully, FSFS also > updates the svn:date revision property and moves the revision properties file > into place (or copies revprop data into an sqlite database if you use > revprop packing). Then, it updates the 'current' file which contains the > number of the current HEAD revision. If you use representation sharing to > save disk space, the commit may involve further updates to yet another > sqlite database. > > All these actions need to complete in order to have a consistent state. > > If you're interested in seeing the code that does this, look at the > svn_fs_fs__commit() and commit_body() functions in > http://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_fs_fs/fs_fs.c I see this is executed with a FS write lock. My concern would be focused on the interaction between the commit code and any rollback code. For example, if the commit dies (any any point during the commit), what will be required to insure that the repository behaves as if the commit never started? Will a repo cleanup be required; will the next committer cleanup the partial rev automatically (ie: overwrite stale files); will the repo be hopelessly inconsistent? A partially committed transaction still leaves the repository in a consistent state, so long as subsequent writers (or a special cleanup process) can undo the internal "damage", if any, made the partial commit. > The safest way is creating a hotcopy of the live repository, and taking > a copy of the hotcopy (by taking a snapshot of the filesystem the hotcopy > resides in, or whatever). It's not the most efficient way of doing it, > but the most portable and safest. This solves the problem if you have control over both the hotcopy and the backup. In a large environment, this means that the filesystem backup needs to be synchronized with the hotcopy, or else, the hotcopy itself might run *during* the backup, and would yield an inconsistent hotcopy (hotcopy-in-progress). > You may be lucky and always get a consistent repository state by taking > filesystem-level snapshots of a live repository, but there's no guarantee > that the very latest commit will always be in a consistent state. There will be two outcomes: 1) The in-progress transaction can always be rolled back (transparently or through the use of a recovery command), leaving an intact repository 2) The repository might be corrupted If svn cannot guarantee outcome 1, then we are left with outcome 2, and that we can be left with a corrupt repository after a commit failure. -- -Justin
RE: Support for filesystem snapshots (?)
From: Les Mikesell [mailto:lesmikes...@gmail.com] > On 8/2/2010 11:42 AM, Vallon, Justin wrote: > > So, my follow up is: If I unplug the network cable between svn-commiter > > and filesystem, will the repository be corrupt? > > > > If yes, then the underlying database is not atomic and therefore not > > transactional. > > > > If no, then the server can take a snapshot at any point in time, and the > > snapshot is guaranteed to be consistent. > > A better match for comparison would be what happens if the server > crashes mid-transaction. A backup taken with the server active might > catch the same state. So the more relevant question would be if the > repository will always be consistent after a server crash. That is the situation I raised. If the network connection between the host that is modifying the repository and the filesystem that houses the repository is lost, will be repository be (a) corrupt, (b) require cleanup (locks, etc), (c) pristine? (c) is great (b) is ok (a) means the operations are not transactional Further: (c) snapshots work (b) snapshots work, but need a cleanup after restore (a) snapshots don't work; repository is not transactional, either -- -Justin
RE: Support for filesystem snapshots (?)
I did see that discussion, but it seems to contradict with the claim that the database operations are transactional. So, my follow up is: If I unplug the network cable between svn-commiter and filesystem, will the repository be corrupt? If yes, then the underlying database is not atomic and therefore not transactional. If no, then the server can take a snapshot at any point in time, and the snapshot is guaranteed to be consistent. -- -Justin -Original Message- From: Les Mikesell [mailto:lesmikes...@gmail.com] Sent: Monday, August 02, 2010 12:30 PM To: users@subversion.apache.org Subject: Re: Support for filesystem snapshots (?) On 8/2/2010 11:17 AM, Stefan Sperling wrote: > On Mon, Aug 02, 2010 at 11:38:30AM -0400, Vallon, Justin wrote: >> In the svn book, >> http://svnbook.red-bean.com/en/1.1/ch05s03.html#svn-ch-5-sect-3.6 on >> "Repository Backup" describes some backup methods - hot copy, etc. Given >> that enterprise-level filesystems generally support filesystem snapshots, >> what is SVN's position on whether such snapshots are sufficient for backups? >> >> Suppose our criteria is that we unplug the network cable between the writer >> (svn commit) and the filesystem. If unplugging the network cable could >> yield a repository that is corrupt, then doesn't that mean that there are >> failure modes where the repository is corrupted? If a filesystem insures >> that the snapshots behave in this way (as if you disconnect the network >> cable, make an instant copy, then reconnect), it would seem to imply that >> these volume-level snapshots would result in a consistent view. If the >> snapshots are not consistent, then atomicity is not being insured (somewhere >> between application, client o/s, server o/s, disk controller, media). >> >> My follow up is whether the svn book could include a word about whether it >> blesses (or not) this form of backup? > > By chance, this very topic was discussed earlier today. > See http://svn.haxx.se/users/archive-2010-08/0019.shtml > > Quote: > "Use 'svnadmin hotcopy' to create a copy of a repository that is consistent > from Subversion's point of view. There is no other repository copying > tool that can guarantee a consistent repository snapshot." > > This holds true for filesystem-level snapshots. > The atomicity of Subversion's commits only exists from the application's > point of view. Are there any svnadmin commands that could quiesce/lock the repository in a consistent state for the duration of a filesystem snapshot - or if not, could that be added as an enhancement? -- Les Mikesell lesmikes...@gmail.com
RE: SVN "Relay"
As far as I can tell, from the "vulnerability": <<< GENERATES A FULL SSL CERTIFICATE WHICH THE VICTIM'S BROWSER WILL PROMPT HIM TO ACCEPT: webmitm -dd >>> The user needs to accept the man-in-the-middle certificate. -- -Justin -Original Message- From: Istace Emmanuel [mailto:istace.emman...@hotmail.com] Sent: Monday, August 02, 2010 12:01 PM To: 'Les Mikesell' Cc: users@subversion.apache.org Subject: RE: SVN "Relay" " Can you point me to something specific?" No problem, here a video (the "fun" side) : http://www.youtube.com/watch?v=Aak6-B3JORE An article : http://forums.remote-exploit.org/tutorials-guides/19852-ssl-spoof-using-wire shark-decode-ssl-packets.html "If you are concerned about your service provider maybe you should use someone else - or a service that lets you run your own system images where you could set up a blowfish-based vpn." I haven't choose that :( But as i say, it's a temporary solution -Message d'origine- De : Les Mikesell [mailto:lesmikes...@gmail.com] Envoyé : lundi 2 août 2010 17:07 À : users@subversion.apache.org Objet : Re: SVN "Relay" On 8/2/2010 8:56 AM, Istace Emmanuel wrote: > " Should I be worried about banking transactions or credit card orders?" > Yeah :( > > " You could use any kind of VPN you want with the remote site. Use an > IPSEC tunnel between hosts if you don't trust SSL. Or OpenVPN with blowfish." > No, because the SVN is on a SaaS cloud, so we just have access to the > service and not the system. So we can't install a VPN server and > remember, vpn and ipsec use SSL. Search on google about SSL Spoofing > ;) Can you point me to something specific? I see things about spoofing some other site's certificate and some things about specific implementations being subject to man-in-the-middle attacks but nothing that looks like a generic weakness. If you are concerned about your service provider (who would have the best opportunity to arrange a man-in-the-middle connection), maybe you should use someone else - or a service that lets you run your own system images where you could set up a blowfish-based vpn. -- Les Mikesell lesmikes...@gmail.com
Support for filesystem snapshots (?)
In the svn book, http://svnbook.red-bean.com/en/1.1/ch05s03.html#svn-ch-5-sect-3.6 on "Repository Backup" describes some backup methods - hot copy, etc. Given that enterprise-level filesystems generally support filesystem snapshots, what is SVN's position on whether such snapshots are sufficient for backups? Suppose our criteria is that we unplug the network cable between the writer (svn commit) and the filesystem. If unplugging the network cable could yield a repository that is corrupt, then doesn't that mean that there are failure modes where the repository is corrupted? If a filesystem insures that the snapshots behave in this way (as if you disconnect the network cable, make an instant copy, then reconnect), it would seem to imply that these volume-level snapshots would result in a consistent view. If the snapshots are not consistent, then atomicity is not being insured (somewhere between application, client o/s, server o/s, disk controller, media). My follow up is whether the svn book could include a word about whether it blesses (or not) this form of backup? -- -Justin
