Post commit script examples?
Hello, I am a very new Subversion user and am trying to get a post commit hook script working. My Subversion is running on linux. The script I want to add would add the needs-lock property to every file that didn't have a lock. Any pointers on this would be great. I have searched and not found any good, working, examples on how to do this. Thanks, Greg
Re: Post commit script examples?
On Wednesday 18 Aug 2010, Greg Alexander wrote: > Hello, > I am a very new Subversion user and am trying to get a post commit hook > script working. My Subversion is running on linux. The script I want to add > would add the needs-lock property to every file that didn't have a lock. > Any pointers on this would be great. I have searched and not found any > good, working, examples on how to do this. Thanks, > Greg Sorry for not answering your actual question but I feel that isn't the correct way to tackle the problem. Making changes to the repository during a post-commit means the committers working copy is instantly out of date and is not a recommended way to work as it can also lead to an infinite loop as the change it makes triggers the script again. I think a better way to approach the problem would be to distribute an appropriate configuration file to the clients and implement a pre-commit hook to reject any commits without the properties. The config file on the client should have auto-props enabled (which is not the default) and have the needs lock added for all files. I've never tried adding a property on every file so I'm not sure what to put in for the pattern, would * work? The area to look at in the subversion book is here, search for auto-props: http://svnbook.red-bean.com/nightly/en/svn-book.html#svn.advanced.confarea.opts.config oh, and I can help with the pre-commit hook. Hopefully this works but the logic around testing the property might not be correct. You can test it by calling it directly (make sure it is executable) with the path to the repository and a revision number so long as you change the -t $TXN to -r $TXN. You could also put an exit 1 at the end of the script and place it in a test repository until it works the way you want it to. The exit 1 at the end will refuse all commits and saves on having to keep creating test files :) also I think the script can be cleaned up so their are only two svnlooks. I butchered my own pre-commit that is preventing commits of files in certain locations that didn't have the looped svnlook lookup. #!/bin/sh REPOS="$1" TXN="$2" SVNLOOK=/path/to/svnlook # redirect all stdout to stderr, this is passed back to the subversion client exec 1>&2 TMPFILE=/tmp/svn-pre-commit-$$.tmp # pick out interesting files $SVNLOOK changed -t "$TXN" "$REPOS" | \ perl -n -e 'print "$1\n" if /^A.* +(.*)$/;' > $TMPFILE exec 3<$TMPFILE NOTIFIED= while read -u 3 FILENAME ; do # check file PROPVALUE=$($SVNLOOK propget $REPOS -t $TXN svn:needs-lock $FILENAME 2>/dev/null) if [ -z "$PROPVALUE" ]; then if [ -z "$NOTIFIED" ]; then echo echo "Files need svn:needs-lock property set, use appropriate config file" echo NOTIFIED="yes" fi fi echo "$FILENAME" done rm $TMPFILE if [ ! -z "$NOTIFIED" ]; then exit 1 fi # All checks passed, so allow the commit. exit 0 -- __ Sword Ciboodle is the trading name of ciboodle Limited (a company registered in Scotland with registered number SC143434 and whose registered office is at India of Inchinnan, Renfrewshire, UK, PA4 9LH) which is part of the Sword Group of companies. This email (and any attachments) is intended for the named recipient(s) and is private and confidential. If it is not for you, please inform us and then delete it. If you are not the intended recipient(s), the use, disclosure, copying or distribution of any information contained within this email is prohibited. Messages to and from us may be monitored. If the content is not about the business of the Sword Group then the message is neither from nor sanctioned by us. Internet communications are not secure. You should scan this message and any attachments for viruses. Under no circumstances do we accept liability for any loss or damage which may result from your receipt of this email or any attachment. __
Re: Post commit script examples?
Hello, Thanks for your reply. While I am not certain that my approach is the best, I also know that a pre-commit hook will not suffice. What I am looking to do is to setup a way so that all files in the repository require a lock to be owned by the user before it can be committed the next time. I also want this to be an automatically set property on any new files. I have users used TortiseSVN, some SVN program for Mac (really not sure), and a few accessing the repository command line. Trying to distribute something to all of them may or may not be adopted and we really need this to be a hard and fast rule. We have a situation today where we have many files and many editors of that code and we have people stepping all over each other. What I would really like is a way to have SVN set a lock as soon as a working copy is created, but I don't think that can be done. To me some sort of autolocking solution would be ideal as then it would be clear who was working in what. However, I can't seem to find any case where that can or has been done. Any help or pointers anyone can lend would be greatly appreciated. Thanks, Greg On Thu, 19 Aug 2010 10:27:49 +0100, Campbell Allan wrote: > On Wednesday 18 Aug 2010, Greg Alexander wrote: >> Hello, >> I am a very new Subversion user and am trying to get a post commit >> hook >> script working. My Subversion is running on linux. The script I want to >> add >> would add the needs-lock property to every file that didn't have a lock. >> Any pointers on this would be great. I have searched and not found any >> good, working, examples on how to do this. Thanks, >> Greg > > Sorry for not answering your actual question but I feel that isn't the > correct > way to tackle the problem. Making changes to the repository during a > post-commit means the committers working copy is instantly out of date and > is > not a recommended way to work as it can also lead to an infinite loop as > the > change it makes triggers the script again. > > I think a better way to approach the problem would be to distribute an > appropriate configuration file to the clients and implement a pre-commit > hook > to reject any commits without the properties. The config file on the > client > should have auto-props enabled (which is not the default) and have the > needs > lock added for all files. I've never tried adding a property on every file > so > I'm not sure what to put in for the pattern, would * work? > > The area to look at in the subversion book is here, search for auto-props: > http://svnbook.red-bean.com/nightly/en/svn-book.html#svn.advanced.confarea.opts.config > > oh, and I can help with the pre-commit hook. Hopefully this works but the > logic around testing the property might not be correct. You can test it by > calling it directly (make sure it is executable) with the path to the > repository and a revision number so long as you change the -t $TXN to -r > $TXN. You could also put an exit 1 at the end of the script and place it > in a > test repository until it works the way you want it to. The exit 1 at the > end > will refuse all commits and saves on having to keep creating test files :) > > also I think the script can be cleaned up so their are only two svnlooks. > I > butchered my own pre-commit that is preventing commits of files in certain > locations that didn't have the looped svnlook lookup. > > #!/bin/sh > > REPOS="$1" > TXN="$2" > > SVNLOOK=/path/to/svnlook > > # redirect all stdout to stderr, this is passed back to the subversion > client > exec 1>&2 > > TMPFILE=/tmp/svn-pre-commit-$$.tmp > > # pick out interesting files > $SVNLOOK changed -t "$TXN" "$REPOS" | \ > perl -n -e 'print "$1\n" if /^A.* +(.*)$/;' > $TMPFILE > exec 3<$TMPFILE > > NOTIFIED= > while read -u 3 FILENAME ; do > > # check file > PROPVALUE=$($SVNLOOK propget $REPOS -t $TXN svn:needs-lock $FILENAME > 2>/dev/null) > if [ -z "$PROPVALUE" ]; then > if [ -z "$NOTIFIED" ]; then > echo > echo "Files need svn:needs-lock property set, use appropriate > config > file" > echo > NOTIFIED="yes" > fi > fi > > echo "$FILENAME" > done > > rm $TMPFILE > > if [ ! -z "$NOTIFIED" ]; then > exit 1 > fi > > # All checks passed, so allow the commit. > exit 0
Re: Post commit script examples?
Hi Greg, On Thu, Aug 19, 2010 at 5:28 PM, Greg Alexander wrote: > > Hello, > What I am > looking to do is to setup a way so that all files in the repository require > a lock to be owned by the user before it can be committed the next time. I > also want this to be an automatically set property on any new files. I have > users used TortiseSVN, some SVN program for Mac (really not sure), and a > few accessing the repository command line. Trying to distribute something > to all of them may or may not be adopted and we really need this to be a > hard and fast rule. We have a situation today where we have many files and > many editors of that code and we have people stepping all over each other. It seems to me that you are trying to force Subversion to do something it was not designed to do. Subversion does not use the Lock-Modify-Unlock model but rather the Copy-Modify-Merge model (which it inherits from CVS). See section "Versioning Models" in chapter 1, Fundamental Concepts, in the svn book. (svnbook.red-bean.com) Having said that, it does support locking, at least on per-file basis. See section "Locking" in chapter 3, Advanced Topics. > What I would really like is a way to have SVN set a lock as soon as a > working copy is created, but I don't think that can be done. To me some > sort of autolocking solution would be ideal as then it would be clear who > was working in what. That would prevent everybody from modifying anything, except the first person who created a working copy. -- Life is complex, with real and imaginary parts. "Ok, it boots. Which means it must be bug-free and perfect. " -- Linus Torvalds "People disagree with me. I just ignore them." -- Linus Torvalds
Re: Post commit script examples?
Hi Csaba, Thanks for the response. Yes, I am beginning to have a good feel for what Subversion was designed for, but I would still like to implement something. I saw the page in the book you reference, but I cannot find any examples on how to really implement. Any pointers anyone can provide would be great! Thanks, Greg On Thu, 19 Aug 2010 18:39:56 +0200, Csaba Raduly wrote: > Hi Greg, > > On Thu, Aug 19, 2010 at 5:28 PM, Greg Alexander wrote: >> >> Hello, >> What I am >> looking to do is to setup a way so that all files in the repository >> require >> a lock to be owned by the user before it can be committed the next time. >> I >> also want this to be an automatically set property on any new files. I >> have >> users used TortiseSVN, some SVN program for Mac (really not sure), and a >> few accessing the repository command line. Trying to distribute something >> to all of them may or may not be adopted and we really need this to be a >> hard and fast rule. We have a situation today where we have many files >> and >> many editors of that code and we have people stepping all over each >> other. > > It seems to me that you are trying to force Subversion to do something > it was not designed to do. Subversion does not use the > Lock-Modify-Unlock model but rather the Copy-Modify-Merge model (which > it inherits from CVS). See section "Versioning Models" in chapter 1, > Fundamental Concepts, in the svn book. (svnbook.red-bean.com) > > Having said that, it does support locking, at least on per-file basis. > See section "Locking" in chapter 3, Advanced Topics. > >> What I would really like is a way to have SVN set a lock as soon as a >> working copy is created, but I don't think that can be done. To me some >> sort of autolocking solution would be ideal as then it would be clear who >> was working in what. > > That would prevent everybody from modifying anything, except the first > person who created a working copy. > > -- > Life is complex, with real and imaginary parts. > "Ok, it boots. Which means it must be bug-free and perfect. " -- Linus > Torvalds > "People disagree with me. I just ignore them." -- Linus Torvalds
Re: Post commit script examples?
On Wed, Aug 18, 2010 at 6:56 PM, Greg Alexander wrote: > I am a very new Subversion user and am trying to get a post commit hook script > working. My Subversion is running on linux. The script I want to add would > add the > needs-lock property to every file that didn't have a lock. Any pointers on > this would > be great. I have searched and not found any good, working, examples on how to > do this. A couple of things: First of all, your attempt to modify code in a hook script: Not cool. In Subversion, a hook script runs on the server and doesn't have access to the developer's working directory. You COULD do a checkout, modify the files, then recommit, but that would mean your hook script is modifying code which could cause an error. The whole idea is to have the developers responsible for their code and making sure their code is tested and everything is right. Yet, your hook script modifies the code and never tests it. Also imagine the logs: Bob did a fix, hook script did a fix. Robert did a fix, hook script did a fix. Alice did a fix, hook script did a fix. Every other log entry is your hook script modifying code. If you want to make sure that the code is correctly formatted, or there's a certain property on the file, you use a pre-commit hook to fail the commit if the developer doesn't have things done correctly. After a couple of times, the developers learns to make these checks before a commit. Now, about the svn:needs-lock property. For 20 or so years, developers had to lock their files before editing. This was part of the design of the first version control system, SCCS and later RCS. You were suppose to get all of the files, then mark the ones you want to edit, modify them, and commit them. It was thought this was the only way you could run a version control system. And, it was a pain. People would lock files and forget to unlock them. Development would be delayed while locks had to be broken. CVS changed all of this. In CVS, you made your changes and committed them. If someone committed before you did, you simply updated that file and tried again. Developers loved it. Development was faster and easier, and all modern version control systems followed suit. You simply edit and commit. You can do what you want, but why? It means you, as the administrator will be spending much of your time breaking locks. You'll see developers get into fights about file editing. And for what? CVS has been around now for 20 years and has shown that lockless version control not only works, but is vastly superior to the kind that made you lock files before editing. -- David Weintraub qazw...@gmail.com
Re: Post commit script examples?
On Aug 19, 2010, at 19:25, David Weintraub wrote: > On Wed, Aug 18, 2010 at 6:56 PM, Greg Alexander wrote: >> I am a very new Subversion user and am trying to get a post commit hook >> script >> working. My Subversion is running on linux. The script I want to add would >> add the >> needs-lock property to every file that didn't have a lock. Any pointers on >> this would >> be great. I have searched and not found any good, working, examples on how to >> do this. > > A couple of things: First of all, your attempt to modify code in a > hook script: Not cool. > > In Subversion, a hook script runs on the server and doesn't have > access to the developer's working directory. You COULD do a checkout, > modify the files, then recommit, but that would mean your hook script > is modifying code which could cause an error. The whole idea is to > have the developers responsible for their code and making sure their > code is tested and everything is right. Yet, your hook script modifies > the code and never tests it. Well, no, he's not proposing modifying the content of the files, only adding a property. > Also imagine the logs: Bob did a fix, hook script did a fix. Robert > did a fix, hook script did a fix. Alice did a fix, hook script did a > fix. Every other log entry is your hook script modifying code. Only commits immediately after adding a new file will be a commit by the hook script, to add the property. For commits that just modify existing files, there will be no need for the hook script to commit anything. > If you want to make sure that the code is correctly formatted, or > there's a certain property on the file, you use a pre-commit hook to > fail the commit if the developer doesn't have things done correctly. > After a couple of times, the developers learns to make these checks > before a commit. Yes, I too would recommend he use this strategy instead. Reject bad commits entirely; soon developers will learn how to make good commits on the first try. > Now, about the svn:needs-lock property. For 20 or so years, developers > had to lock their files before editing. This was part of the design of > the first version control system, SCCS and later RCS. You were suppose > to get all of the files, then mark the ones you want to edit, modify > them, and commit them. It was thought this was the only way you could > run a version control system. > > And, it was a pain. People would lock files and forget to unlock them. > Development would be delayed while locks had to be broken. CVS changed > all of this. In CVS, you made your changes and committed them. If > someone committed before you did, you simply updated that file and > tried again. > > Developers loved it. Development was faster and easier, and all modern > version control systems followed suit. You simply edit and commit. > > You can do what you want, but why? It means you, as the administrator > will be spending much of your time breaking locks. You'll see > developers get into fights about file editing. And for what? CVS has > been around now for 20 years and has shown that lockless version > control not only works, but is vastly superior to the kind that made > you lock files before editing. Right. The user should explain to us in more detail, with examples, why he believes locking is necessary.
Re: Post commit script examples?
On Thu, 2010-08-19 at 11:24 -0600, Greg Alexander wrote: > Hi Csaba, > Thanks for the response. Yes, I am beginning to have a good feel for > what Subversion was designed for, but I would still like to implement > something. I saw the page in the book you reference, but I cannot find any > examples on how to really implement. Any pointers anyone can provide would > be great! Really Allan has described the right way to do it: - As a first step, create a script to do a checkout, add "svn:needs-lock" property on files you want and do a commit. So it should be OK for existing content. For file additions after that point: - Distribute auto-props "config" file with "svn:needs-lock" property for binary files that cannot be merged as source text As far as I know, that file is the same for all client/platform. On Linux (and probably MacOS X), command line svn and other clients read ~/.subversion/config On Windows, command line svn and TortoiseSVN read Application Data\Subversion\config - Implement a pre-commit hook to reject commit submitting file without "svn:needs-lock" property when it is required The pre-commit hook can even display an URL with documentation and "config" file to install so that property is properly set at addition. By the way, if work on files is not "segmented" enough (by team, by time slice, by modules - think at splitting content in multiple smaller files) locking will not help; Developers cannot afford to wait for lock releases and will find the work-around: branching Then they may tell you to merge their work ! My opinion, there is only one response: education and proper organization. Regards Yves Martin
Re: Post commit script examples?
On Friday, August 20, 2010 09:42:44 am Ryan Schmidt wrote: > > Also imagine the logs: Bob did a fix, hook script did a fix. Robert > > did a fix, hook script did a fix. Alice did a fix, hook script did a > > fix. Every other log entry is your hook script modifying code. > > Only commits immediately after adding a new file will be a commit by the > hook script, to add the property. For commits that just modify existing > files, there will be no need for the hook script to commit anything. It could be done within the same commit, actually, without a separate commit. Here is a simple pre-commit in Python that will set 'foo:bar' property to 'baz' on all committed files but not directories. [[[ #! /usr/bin/python # vi: set sw=2 : import sys from svn import core, fs, delta, repos def v(pool, repos_path, txn): fs_ptr = repos.fs(repos.open(repos_path, pool)) txn_ptr = fs.open_txn(fs_ptr, txn, pool) txn_root = fs.txn_root(txn_ptr, pool) chg = fs.paths_changed2(txn_root, pool) for f in chg: if chg[f].node_kind == core.svn_node_file: fs.change_node_prop(txn_root, f, "foo:bar", "baz") if __name__ == '__main__': core.run_app(v, sys.argv[1], sys.argv[2]) ]]] Here is how it works. Note the foo:bar property change. [[[ /tmp$ svnadmin create aa /tmp$ cp pre-commit aa/hooks/ /tmp$ svn co file:///tmp/aa awc Checked out revision 0. /tmp$ cd awc /tmp/awc$ echo qwerty > qwerty /tmp/awc$ svn add qwerty A qwerty /tmp/awc$ svn ci -m 1 qwerty Adding qwerty Transmitting file data . Committed revision 1. /tmp/awc$ svn di -c 1 Index: qwerty === --- qwerty (revision 0) +++ qwerty (revision 1) @@ -0,0 +1 @@ +qwerty Property changes on: qwerty ___ Added: foo:bar ## -0,0 +1 ## +baz ]]] The problem with this approach is that the working copy is not aware that the commit was somehow altered on the server, so the client does not see this property on the file: [[[ /tmp/awc$ svn pl qwerty /tmp$ ]]] Therefore, I would only use this approach for properties that client does not care about, and svn:needs-lock is not one of such properties. Regards, Alexey.
Re: Post commit script examples?
On Fri, Aug 20, 2010 at 12:42 PM, Ryan Schmidt wrote: > On Aug 19, 2010, at 19:25, David Weintraub wrote: >> On Wed, Aug 18, 2010 at 6:56 PM, Greg Alexander wrote: >>> I am a very new Subversion user and am trying to get a post commit hook >>> script >>> working. My Subversion is running on linux. The script I want to add would >>> add the >>> needs-lock property to every file that didn't have a lock. Any pointers on >>> this would >>> be great. I have searched and not found any good, working, examples on how >>> to >>> do this. >> >> A couple of things: First of all, your attempt to modify code in a >> hook script: Not cool. >> >> In Subversion, a hook script runs on the server and doesn't have >> access to the developer's working directory. You COULD do a checkout, >> modify the files, then recommit, but that would mean your hook script >> is modifying code which could cause an error. The whole idea is to >> have the developers responsible for their code and making sure their >> code is tested and everything is right. Yet, your hook script modifies >> the code and never tests it. > > Well, no, he's not proposing modifying the content of the files, only adding > a property. "Properties" of file are part of the file content in the necessary Subversion database. They require a revision to commit. Doing that as a post-commit is potentially quite awkward and destructive if a bug creeps into the script and it recurses, spewing hundreds or thousands of revisions into unattended operation that you, as a remote Subversion user, have no graceful way to stop. It's a very dangerous idea. >> Also imagine the logs: Bob did a fix, hook script did a fix. Robert >> did a fix, hook script did a fix. Alice did a fix, hook script did a >> fix. Every other log entry is your hook script modifying code. > > Only commits immediately after adding a new file will be a commit by the hook > script, to add the property. For commits that just modify existing files, > there will be no need for the hook script to commit anything. And you'll be able to detect this, reliably, how? Remember that the data available to a post-commit script has to be parsed, and with multilingual file names or non-7-bit ASCII filenames it can get tricky. >> If you want to make sure that the code is correctly formatted, or >> there's a certain property on the file, you use a pre-commit hook to >> fail the commit if the developer doesn't have things done correctly. >> After a couple of times, the developers learns to make these checks >> before a commit. > > Yes, I too would recommend he use this strategy instead. Reject bad commits > entirely; soon developers will learn how to make good commits on the first > try. Amen. I wouldn't necessarily call them "good" commits, but you need compliant commits for this. >> Now, about the svn:needs-lock property. For 20 or so years, developers >> had to lock their files before editing. This was part of the design of >> the first version control system, SCCS and later RCS. You were suppose >> to get all of the files, then mark the ones you want to edit, modify >> them, and commit them. It was thought this was the only way you could >> run a version control system. >> >> And, it was a pain. People would lock files and forget to unlock them. >> Development would be delayed while locks had to be broken. CVS changed >> all of this. In CVS, you made your changes and committed them. If >> someone committed before you did, you simply updated that file and >> tried again. >> >> Developers loved it. Development was faster and easier, and all modern >> version control systems followed suit. You simply edit and commit. >> >> You can do what you want, but why? It means you, as the administrator >> will be spending much of your time breaking locks. You'll see >> developers get into fights about file editing. And for what? CVS has >> been around now for 20 years and has shown that lockless version >> control not only works, but is vastly superior to the kind that made >> you lock files before editing. > > Right. The user should explain to us in more detail, with examples, why he > believes locking is necessary. As opposed to having tag, or a release tag area, be locked.
