On 11/16/2010 05:57 AM, Vincent Massol wrote: > Sounds good (I like experiments ;)) but before going too far I'd like to see: > > 1) the needs we have regarding hudson mails. I think we need to start seeing the reports before we can know exactly what we want.
> > For example, IMO we need: > - immediate notification after a commit that makes the build fail RegressionAlertReport should cater to this need. > - no false positives This is rather difficult. Reporter looks back in history to see if a test failed before and if yes then it disregards it is a flicker. This is limited by the number of builds which hudson keeps in history. > - know who broke the build very visibly (with the commit that broke it) This is implemented, it is used in the RegressionAlertReport, a report which should run very often and only send mail if there are new regressions. Adding it to other reports is trivial. > - console logs in the mail not to have to click on the link to get details This is handled, /dev/stderr is included in the reports, stack traces are in the TestCase class so adding them to a report is trivial. > - short and clear mail subjects (right now they're long and all look the same > which makes it hard to see which module is broken) Option 1 (which is currently implemented) is one message goes out which includes information for the entire array of jobs and reports on them all. Option 2 is a message goes out for each job which contains a regression. Changing to this can be done in a few hours of coding. I think it's best to start seeing mail before we decide which is best. > - no duplicate mails. We receive one for the top level project and one for > the each module right now > - no mails when build is successful; only when it fails Both done. > > Do we agree about those needs? Any more? I would add that I think a daily report including all of the failures (not just regressions) is necessary so we can get a perspective of the overall health of the code. > > 2) how you report is addressing these needs in 1) See above. > and whether it's supposed to replace the existing mails sent or come in > addition to them. Replace all of the test report mail. Hudson does a very good job sending compilation failure mail and the reporting tool is designed to work with test reports which don't exist if the code does not compile. > Also I'd like to know how frequently it is sent? On each commit? On a time > basis? I imagined a time based system, another option is to run some reports after every Hudson build, per commit makes no sense because the test reports are not available until after the build. If this runs as a hudson job (as I would like) then it can be run by any hudson trigger we choose. IMO something to fine tune later. Caleb > > Thanks > -Vincent > > On Nov 15, 2010, at 8:50 PM, Caleb James DeLisle wrote: > >> I would like to get reports going out so that we can get a feel for the >> periodic reports, and work >> out bugs and make improvements in real time. >> >> I'm +1 to getting reports going. >> >> >> A technical challenge is that unless one of the build agents has access to a >> mail relay, it will not >> be able to send mail. If this is a problem, it can be fixed by signing up >> with an email service and >> placing the password in a file on the filesystem of the agent which the job >> is bound to. The file >> can be loaded by the configuration script. >> >> Caleb > _______________________________________________ > devs mailing list > [email protected] > http://lists.xwiki.org/mailman/listinfo/devs > _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
