Programming a Watcher File
One of the tech leads wants to be able to program a watch file, so that when a certain set of developers change a particular file, certain other developers get notified. Thus, we need to be able to program what files, what developers do the commit, and what developers receive the notification. I'd like to do this, so this developer can maintain this file himself instead of me having to maintain it which means it can't be a text file on the Subversion server. I'm working with something like this now. I have a configuration file where you setup the various parameters (specifying files via globbing or regex), who does the commit, and who receives the email. The configuration file sits in the source repository, so this tech lead has access to it. I was wondering if anyone already has a post-commit hook script like this already setup. I know there's one included in Subversion, but this one uses a text file that's accessible to the hook script and its setup isn't that flexible. If not, I'll finish up the one I'm working on. -- David Weintraub qazw...@gmail.com
Re: Programming a Watcher File
Greetings, David Weintraub! > One of the tech leads wants to be able to program a watch file, so > that when a certain set of developers change a particular file, > certain other developers get notified. Thus, we need to be able to > program what files, what developers do the commit, and what developers > receive the notification. I'd like to do this, so this developer can > maintain this file himself instead of me having to maintain it which > means it can't be a text file on the Subversion server. Sorry, my head is a bit crippled for now, and your post contains too much cross-references to be understandable in my current state. Could you please put it in simple terms, how you see your potential system works? > I'm working with something like this now. I have a configuration file > where you setup the various parameters (specifying files via globbing > or regex), who does the commit, and who receives the email. The > configuration file sits in the source repository, so this tech lead > has access to it. > I was wondering if anyone already has a post-commit hook script like > this already setup. I know there's one included in Subversion, but > this one uses a text file that's accessible to the hook script and its > setup isn't that flexible. > If not, I'll finish up the one I'm working on. If having a custom (and client-customizable) configuration is all you ever want, there's a hint: hook script can access repository as easy as any other local file. Just do "svn cat" on required file and parse it's content as normal. However, be very wary of access rights on the mentioned configuration file. -- WBR, Andrey Repin (anrdae...@freemail.ru) 03.03.2010, <7:35> Sorry for my terrible english...
Re: Programming a Watcher File
On Wed, Mar 3, 2010 at 5:39 AM, Andrey Repin wrote: > Greetings, David Weintraub! > >> One of the tech leads wants to be able to program a watch file, so >> that when a certain set of developers change a particular file, >> certain other developers get notified. Thus, we need to be able to >> program what files, what developers do the commit, and what developers >> receive the notification. I'd like to do this, so this developer can >> maintain this file himself instead of me having to maintain it which >> means it can't be a text file on the Subversion server. > > Sorry, my head is a bit crippled for now, and your post contains too much > cross-references to be understandable in my current state. > Could you please put it in simple terms, how you see your potential system > works? > >> I'm working with something like this now. I have a configuration file >> where you setup the various parameters (specifying files via globbing >> or regex), who does the commit, and who receives the email. The >> configuration file sits in the source repository, so this tech lead >> has access to it. > >> I was wondering if anyone already has a post-commit hook script like >> this already setup. I know there's one included in Subversion, but >> this one uses a text file that's accessible to the hook script and its >> setup isn't that flexible. > >> If not, I'll finish up the one I'm working on. > > If having a custom (and client-customizable) configuration is all you ever > want, there's a hint: hook script can access repository as easy as any other > local file. Just do "svn cat" on required file and parse it's content as > normal. > > However, be very wary of access rights on the mentioned configuration file. Some time ago, we were thinking of maintaining some notification-configuration inside versioned properties on directories in svn. The idea was simply to be able to put a (comma-separated, or new-line separated or whatever) list of email addresses in some property (let's call it "watchers") on directories. In the simple case, just the top level directories actually. Post-commit hook would then see which top-level directories were affected, and use those email addresses to send commit mails. This then avoids the need to setup and maintain mailinglists ... It might be possible to come up with a more elaborate system to put more configuration in such properties (like people watching only certain authors on certain directories/files etc). I didn't really think it through (there could be some nasty details, performance issues in crawling up the directory structure, ...), because we decided not to implement it. In the end, we decided it would be much easier to just setup some mailinglists, for the simple stuff we needed... Just my .02 € Johan
Re: Programming a Watcher File
On Wed, Mar 3, 2010 at 1:57 AM, Johan Corveleyn wrote: > > Some time ago, we were thinking of maintaining some > notification-configuration inside versioned properties on directories > in svn. The idea was simply to be able to put a (comma-separated, or > new-line separated or whatever) list of email addresses in some > property (let's call it "watchers") on directories. In the simple > case, just the top level directories actually. Post-commit hook would > then see which top-level directories were affected, and use those > email addresses to send commit mails. This then avoids the need to > setup and maintain mailinglists ... It might be possible to come up > with a more elaborate system to put more configuration in such > properties (like people watching only certain authors on certain > directories/files etc). > > I didn't really think it through (there could be some nasty details, > performance issues in crawling up the directory structure, ...), > because we decided not to implement it. In the end, we decided it > would be much easier to just setup some mailinglists, for the simple > stuff we needed... > > Just my .02 € > > Johan > A while ago, I wrote something on top of SVN::Notify with a similar feature, of having a property define who got an email. I threw it up on CPAN at http://search.cpan.org/~larrysh/SVN-Notify-Filter-Watchers-0.09/ It worked pretty good. I have since moved jobs, and we use FishEye, which has a way to setup notifications, and use that instead. So if it has any problems, send me a patch, and I will apply it. -- Larry
Re: Programming a Watcher File
On Tue, Mar 2, 2010 at 11:39 PM, Andrey Repin wrote: > Sorry, my head is a bit crippled for now, and your post contains too much > cross-references to be understandable in my current state. > Could you please put it in simple terms, how you see your potential system > works? We have both development teams in India and in New York. When the Indian development team makes changes, the New York QA team needs to be notified, so they know what to test. When the New York development team checks something in, the Indian QA team needs to be notified. This will currently be for a single project, so I have to specify those files in that project that need to be watched. I decided to make this hook script a bit more general than for this specific circumstance. What if there are separate teams for each branch that need to be notified? What if other teams want their files to be watched? > If having a custom (and client-customizable) configuration is all you ever > want, there's a hint: hook script can access repository as easy as any other > local file. Just do "svn cat" on required file and parse it's content as > normal. > > However, be very wary of access rights on the mentioned configuration file. I am using "svnlook cat" which is a bit faster since I don't have to go through the Subversion server. I will be setting the rights on this configuration file, so only the managers can modify it. Plus, I'll get an email whenever the configuration file itself gets modified. Originally, I was thinking about using properties to set the watch. If I want a particular directory tree watched, I could put a sol:watch property on that directory, and anytime a file in that directory tree gets modified, I'll email the users. However, how do you find all of these properties? If I check in a file svn://host/project/dir/dir2/dir3/file, I'd have to check all the parent directories for the property (that's five different directories to check, plus the file itself). And, I'd have to do that for each file changed. Plus, how do developers find all of their watches? Therefore, I decided it would be better to have a single configuration file for the watches. Another possibility is to have a directory where users can put their own watch definitions. I have an access control script that can easily be setup so that barry.cfg can only be changed by barry and david.cfg can only be changed by david. But, that means getting a list of these watch files (svnlook tree), and then checking opening up each one to construct my watch list. That might take too long. -- David Weintraub qazw...@gmail.com
RE: Programming a Watcher File
> On Tue, Mar 2, 2010 at 11:39 PM, Andrey Repin > wrote: > > Sorry, my head is a bit crippled for now, and your post contains too > much > > cross-references to be understandable in my current state. > > Could you please put it in simple terms, how you see your potential > system > > works? > > We have both development teams in India and in New York. When the > Indian development team makes changes, the New York QA team needs to > be notified, so they know what to test. When the New York development > team checks something in, the Indian QA team needs to be notified. > This will currently be for a single project, so I have to specify > those files in that project that need to be watched. I do this kind of stuff with cruise control. It will watch for check-ins and kick of tasks which can include emails if you want. I tie this in with CI builds... of course, cruise control doesn't have to run a build... it could just do publish/notify tasks if you want. You can also set up a project in cruise control that watches it's own config file that you put into svn. When this config file is changed cruise control will update the WC it is in and then re-intialize with the new config info. The config file is where email addresses and such would be stored. I would rather go this way than writing my own script. BOb
Re: Programming a Watcher File
On Wed, Mar 3, 2010 at 10:09 AM, Larry Shatzer, Jr. wrote:> > A while ago, I wrote something on top of SVN::Notify with a similar feature, > of having a property define who got an email. I threw it up on CPAN > at http://search.cpan.org/~larrysh/SVN-Notify-Filter-Watchers-0.09/ It > worked pretty good. I have since moved jobs, and we use FishEye, which has a > way to setup notifications, and use that instead. So if it has any problems, > send me a patch, and I will apply it. Thanks for the link. I'll take a look at it. Something like Fisheye may be the best way to go. I need to see what might be available in the open source world. That might be cleaner and easier than some roll your own setup. It just occurred to me that this is a bit unique. This isn't a developer saying what they're watching. It's their manager saying "You're watching these files!". That's pretty unique. -- David Weintraub qazw...@gmail.com
Re: Programming a Watcher File
On Wed, Mar 3, 2010 at 12:06 PM, Bob Archer wrote: > I do this kind of stuff with cruise control. It will watch for check-ins and > kick of tasks > which can include emails if you want. I tie this in with CI builds... of > course, cruise control > doesn't have to run a build... it could just do publish/notify tasks if you > want. We have Hudson, and I could do something similar. However, the problem is that for the same file, I have to notify Group "C" if Group "A" does a check in, and Group "D" if Group "B" does a checkin. That makes things a bit more difficult. Plus, it might get to the point where certain sets of files in a single project will send notifications to a particular user. -- David Weintraub qazw...@gmail.com
Re: Programming a Watcher File
On 3/3/2010 11:06 AM, Bob Archer wrote: On Tue, Mar 2, 2010 at 11:39 PM, Andrey Repin wrote: Sorry, my head is a bit crippled for now, and your post contains too much cross-references to be understandable in my current state. Could you please put it in simple terms, how you see your potential system works? We have both development teams in India and in New York. When the Indian development team makes changes, the New York QA team needs to be notified, so they know what to test. When the New York development team checks something in, the Indian QA team needs to be notified. This will currently be for a single project, so I have to specify those files in that project that need to be watched. I do this kind of stuff with cruise control. It will watch for check-ins and kick of tasks which can include emails if you want. I tie this in with CI builds... of course, cruise control doesn't have to run a build... it could just do publish/notify tasks if you want. You can also set up a project in cruise control that watches it's own config file that you put into svn. When this config file is changed cruise control will update the WC it is in and then re-intialize with the new config info. The config file is where email addresses and such would be stored. I would rather go this way than writing my own script. Hudson would be able to do this sort of thing too. And probably do whatever action you wanted the person to take after noticing the file changed... -- Les Mikesell lesmikes...@gmail.com
Re: Programming a Watcher File
Greetings, David Weintraub! > On Tue, Mar 2, 2010 at 11:39 PM, Andrey Repin wrote: >> Sorry, my head is a bit crippled for now, and your post contains too much >> cross-references to be understandable in my current state. >> Could you please put it in simple terms, how you see your potential system >> works? > We have both development teams in India and in New York. When the > Indian development team makes changes, the New York QA team needs to > be notified, so they know what to test. When the New York development > team checks something in, the Indian QA team needs to be notified. > This will currently be for a single project, so I have to specify > those files in that project that need to be watched. > I decided to make this hook script a bit more general than for this > specific circumstance. What if there are separate teams for each > branch that need to be notified? What if other teams want their files > to be watched? Having generalized hook script is always a nice option, if you ask me. Given sane configuration scheme, you never know, how unexpectedly nice setups your users would be able to build. >> If having a custom (and client-customizable) configuration is all you ever >> want, there's a hint: hook script can access repository as easy as any other >> local file. Just do "svn cat" on required file and parse it's content as >> normal. >> >> However, be very wary of access rights on the mentioned configuration file. > I am using "svnlook cat" which is a bit faster Right, and thanks for a note. > since I don't have to go through the Subversion server. I will be setting > the rights on this configuration file, so only the managers can modify it. > Plus, I'll get an email whenever the configuration file itself gets modified. > Originally, I was thinking about using properties to set the watch. If > I want a particular directory tree watched, I could put a sol:watch > property on that directory, and anytime a file in that directory tree > gets modified, I'll email the users. I didn't though about properties, but I clearly see a perfomance issue you've mentioned below. You have to enumerate whole tree back up to the repository root in attempt to locate the properties, and you can't just stop in the middle after first such property found. > However, how do you find all of these properties? If I check in a file > svn://host/project/dir/dir2/dir3/file, I'd have to check all the > parent directories for the property (that's five different directories > to check, plus the file itself). And, I'd have to do that for each > file changed. Plus, how do developers find all of their watches? > Therefore, I decided it would be better to have a single configuration > file for the watches. > Another possibility is to have a directory where users can put their > own watch definitions. I have an access control script that can easily > be setup so that barry.cfg can only be changed by barry and david.cfg > can only be changed by david. But, that means getting a list of these > watch files (svnlook tree), and then checking opening up each one to > construct my watch list. That might take too long. You're missing one option - put a post-commit hook on this directory, which would compile single configuration file from all user files. I.e. \watch\watchlist (global options) \watch\watchlist.d\.list \watch\watchlist.d\.list Every time your post-commit (or pre-commit, if you decide) hook spot a write operation to the \watch\watchlist.d, it run compilation process, and notify repository administrators of the results. You can store compiled config out of repository, or inside it - as you decide. I'd go with off-site approach, as it could be reconstructed at any time, whenever the repository moves. -- WBR, Andrey Repin (anrdae...@freemail.ru) 04.03.2010, <17:27> Sorry for my terrible english...
Re: Programming a Watcher File
I'm almost finished my second notification scheme. Each user their own watch file. In my access-control trigger, I can easily set it up, so that users can only change their own watch files: [FILE Lock Watch directory as read-only] file = ^/watchers/.* access = read-only users = @ALL [FILE Now allow users to edit their own watch file] file = ^/watchers/.cfg access = read-write users = Watch files will be composed of three lines. The first is the email line: email = qazw...@gmail.com Users can have as many email lines as their heart desires, and I'll email each and every one of them. If the user doesn't have an email line in their file, I'll email to @ which is pretty much what you expect as the default. There are two other lines in the file: file = /foo/trunk/mail/src/java/**/*.java and regex = ^foo/.*\.java$ The "file" line is for Ant glob type expressions where "*" is a series of any characters except "/" and "**" is any series of characters. It is automatically anchored at the root of repository it is expected to match the entire file name. The "regex" is an extended regular expression that matches the file name. It is not anchored unless you use the "^" and "$" characters. Of course, I don't notify the user of self changes. I am still testing, but I'm willing to give this to anyone who wants to try it. The other watch file trigger is for a manager to specify who gets notified of what type of change and by whom. It's a lot more specific to our circumstance where we want QA people in India to be notified of changes developers in NY do, and QA people in New York to be notified of developer changes in India. -- David Weintraub qazw...@gmail.com
Re: Programming a Watcher File
On Wed, Mar 3, 2010 at 2:26 AM, David Weintraub wrote: > One of the tech leads wants to be able to program a watch file, so > that when a certain set of developers change a particular file, > certain other developers get notified. Thus, we need to be able to > program what files, what developers do the commit, and what developers > receive the notification. I'd like to do this, so this developer can > maintain this file himself instead of me having to maintain it which > means it can't be a text file on the Subversion server. > > I'm working with something like this now. I have a configuration file > where you setup the various parameters (specifying files via globbing > or regex), who does the commit, and who receives the email. The > configuration file sits in the source repository, so this tech lead > has access to it. > > I was wondering if anyone already has a post-commit hook script like > this already setup. I know there's one included in Subversion, but > this one uses a text file that's accessible to the hook script and its > setup isn't that flexible. > > If not, I'll finish up the one I'm working on. > > Probably a bit too late now, but have you considered using a tool such as CommitMonitor, or SVNNotifier? Cheers, Daniel B.
Re: Programming a Watcher File
CommitNotifier uses Tortoise and SVN Notifier uses MS .NET platform. Both are Windows specific, and we have a really mixed office. We have people on Linux, Windows, and Mac. On Mon, Mar 8, 2010 at 5:49 PM, Daniel Becroft wrote: > On Wed, Mar 3, 2010 at 2:26 AM, David Weintraub wrote: >> >> One of the tech leads wants to be able to program a watch file, so >> that when a certain set of developers change a particular file, >> certain other developers get notified. Thus, we need to be able to >> program what files, what developers do the commit, and what developers >> receive the notification. I'd like to do this, so this developer can >> maintain this file himself instead of me having to maintain it which >> means it can't be a text file on the Subversion server. >> >> I'm working with something like this now. I have a configuration file >> where you setup the various parameters (specifying files via globbing >> or regex), who does the commit, and who receives the email. The >> configuration file sits in the source repository, so this tech lead >> has access to it. >> >> I was wondering if anyone already has a post-commit hook script like >> this already setup. I know there's one included in Subversion, but >> this one uses a text file that's accessible to the hook script and its >> setup isn't that flexible. >> >> If not, I'll finish up the one I'm working on. >> > > Probably a bit too late now, but have you considered using a tool such as > CommitMonitor, or SVNNotifier? > > Cheers, > Daniel B. > > > -- David Weintraub qazw...@gmail.com
Re: Programming a Watcher File
On 3/8/2010 4:56 PM, David Weintraub wrote: CommitNotifier uses Tortoise and SVN Notifier uses MS .NET platform. Both are Windows specific, and we have a really mixed office. We have people on Linux, Windows, and Mac. Hudson should run on about anything. It does have to poll for changes but it can do about anything in the job that it kicks off, even on remote slave machines. -- Les Mikesell lesmikes...@gmail.com
Re: Programming a Watcher File
The idea is to allow users to specify exactly which they want to watch. It might be a few configuration files, images, etc. Hudson will notify if any files in the entire project are changed, and when you do a dozen builds each day, the developers start to ignore these build notices. This is something that most other version control systems allow and is usually built in. Third party clients like SVN Notifier and Commit Notifier is that they must be user installed and running on the user's machine. If you aren't on that machine, you don't get notified. Something like Fisheye is good because that allows users to set notifications and is not dependent upon the user's own system. However, we don't have Fisheye. Subversion comes with a post-commit notification script written in Perl, but this script requires a configuration file that sits on the server. That means developers have to ask the administrator to set and change notifications. By putting the notification configurations inside the repository, I allow users to set their own notifications. Since it is the server that's running it, the notifications aren't machine dependent. -- David Weintraub qazw...@gmail.com
Re: Programming a Watcher File
On Tue, Mar 9, 2010 at 1:15 AM, David Weintraub wrote: > The idea is to allow users to specify exactly which they want to > watch. It might be a few configuration files, images, etc. > > Hudson will notify if any files in the entire project are changed, and > when you do a dozen builds each day, the developers start to ignore > these build notices. > > This is something that most other version control systems allow and is > usually built in. Third party clients like SVN Notifier and Commit > Notifier is that they must be user installed and running on the user's > machine. If you aren't on that machine, you don't get notified. > > Something like Fisheye is good because that allows users to set > notifications and is not dependent upon the user's own system. > However, we don't have Fisheye. > > Subversion comes with a post-commit notification script written in > Perl, but this script requires a configuration file that sits on the > server. That means developers have to ask the administrator to set and > change notifications. By putting the notification configurations > inside the repository, I allow users to set their own notifications. > Since it is the server that's running it, the notifications aren't > machine dependent. I think it's great functionality that you're building here. I for one would be quite interested in something like this, so if you could share it, that would be super (maybe even put it in "contrib" if it's ready?). Having developers manage their own notifications (all within the repos, and cross-platform, independent of client machine), specifying which files/paths they are interested in, is very useful functionality. Just a thought: it would be nice if you could make the coupling with the actual notification system a "loose coupling". To separate "how to determine who is notified" (what you're building now) from the "mechanism that sends the notifications". I'm currently using commit-email.pl, but am also looking at mailer.py (once we upgrade our server, including python bindings) and SVN::Notify (http://search.cpan.org/dist/SVN-Notify/). It would be nice to still be able to choose whichever notification system works best ... Johan
Re: Programming a Watcher File
I'm looking at the SVN:Notify, and it appears to require sendmail. Is that correct? I would rather use NET::SMTP which is more platform neutral. What does the NET::SMTP notification look like? Does it do a diff between files? Does it tell you all the files modified, or just the ones you're interested in? I'll try to separate out the notification system and make it optional. That's actually a nice idea. The question is how do I pass the notification stuff. I can't use STDOUT because Subversion captures that. On Tue, Mar 9, 2010 at 5:42 AM, Johan Corveleyn wrote: > On Tue, Mar 9, 2010 at 1:15 AM, David Weintraub wrote: >> The idea is to allow users to specify exactly which they want to >> watch. It might be a few configuration files, images, etc. >> >> Hudson will notify if any files in the entire project are changed, and >> when you do a dozen builds each day, the developers start to ignore >> these build notices. >> >> This is something that most other version control systems allow and is >> usually built in. Third party clients like SVN Notifier and Commit >> Notifier is that they must be user installed and running on the user's >> machine. If you aren't on that machine, you don't get notified. >> >> Something like Fisheye is good because that allows users to set >> notifications and is not dependent upon the user's own system. >> However, we don't have Fisheye. >> >> Subversion comes with a post-commit notification script written in >> Perl, but this script requires a configuration file that sits on the >> server. That means developers have to ask the administrator to set and >> change notifications. By putting the notification configurations >> inside the repository, I allow users to set their own notifications. >> Since it is the server that's running it, the notifications aren't >> machine dependent. > > I think it's great functionality that you're building here. I for one > would be quite interested in something like this, so if you could > share it, that would be super (maybe even put it in "contrib" if it's > ready?). Having developers manage their own notifications (all within > the repos, and cross-platform, independent of client machine), > specifying which files/paths they are interested in, is very useful > functionality. > > Just a thought: it would be nice if you could make the coupling with > the actual notification system a "loose coupling". To separate "how to > determine who is notified" (what you're building now) from the > "mechanism that sends the notifications". I'm currently using > commit-email.pl, but am also looking at mailer.py (once we upgrade our > server, including python bindings) and SVN::Notify > (http://search.cpan.org/dist/SVN-Notify/). It would be nice to still > be able to choose whichever notification system works best ... > > Johan > -- David Weintraub qazw...@gmail.com
Re: Programming a Watcher File
On Tue, Mar 9, 2010 at 10:27 AM, David Weintraub wrote: > I'm looking at the SVN:Notify, and it appears to require sendmail. Is > that correct? I would rather use NET::SMTP which is more platform > neutral. > > What does the NET::SMTP notification look like? Does it do a diff > between files? Does it tell you all the files modified, or just the > ones you're interested in? > > I'll try to separate out the notification system and make it optional. > That's actually a nice idea. The question is how do I pass the > notification stuff. I can't use STDOUT because Subversion captures > that. > If you specify a smtp config, it will use that over sendmail. See http://search.cpan.org/dist/SVN-Notify/lib/SVN/Notify.pm#smtp -- Larry
Re: Programming a Watcher File
After playing around with snvnotify, I notice all it does is use the SNV::Notify class. I'll just add the option to use SNV::Notify if you happen to have the module installed. -- David Weintraub qazw...@gmail.com