automatic bug filing by test robot

2008-08-27 Thread Ian Jackson
Now that gdebi-core is in back in lenny I've got my automatic package testing robot working on lenny again. (I got stalled last time due to leaving Canonical and being busy for a while.) Mostly what it does right now is build a package, discover that the package doesn't have any autopkgtest tests

Re: automatic bug filing by test robot

2008-08-27 Thread Gonéri Le Bouder
On Wed, Aug 27, 2008 at 10:10:13PM +0100, Ian Jackson wrote: > Now that gdebi-core is in back in lenny I've got my automatic package > testing robot working on lenny again. (I got stalled last time due to > leaving Canonical and being busy for a while.) > > Mostly what it does right now is build

Re: automatic bug filing by test robot

2008-08-27 Thread Ian Jackson
Gonéri Le Bouder writes ("Re: automatic bug filing by test robot"): > Sorry for promoting my pet toy but I think your script would be a > great input. Right ... > I'm working improving svnbuildstat to be able to support different input. The > idea is: > -a databas

Re: automatic bug filing by test robot

2008-08-28 Thread Raphael Hertzog
On Wed, 27 Aug 2008, Ian Jackson wrote: > I'm hoping to shortly turn on the automatic bug filing mechanism. > I'm writing now to give people a chance to object :-). [...] > very few other packages. Most autopkgtest reports are FTBFS problems. I object filing FTBFS automatically. In some cases, th

Re: automatic bug filing by test robot

2008-08-28 Thread Martin Michlmayr
* Raphael Hertzog <[EMAIL PROTECTED]> [2008-08-28 09:19]: > I object filing FTBFS automatically. In some cases, the FTBFS is a failure ... > Also, how would you check that a bug has not already been filed > manually by another user? Totally agreed. -- Martin Michlmayr http://www.cyrius.com/ --

Re: automatic bug filing by test robot

2008-08-28 Thread Holger Levsen
Hi Ian, nice work! but... On Wednesday 27 August 2008 23:10, Ian Jackson wrote: > I'm hoping to shortly turn on the automatic bug filing mechanism. > I'm writing now to give people a chance to object :-). /me objects / suggests to first enable mails to some mailinglist (this one?), instead of a

Re: automatic bug filing by test robot

2008-08-28 Thread Brian May
Holger Levsen wrote: /me objects / suggests to first enable mails to some mailinglist (this one?), instead of actually filing bugs. Then we can review the reports and then decide whether it's working as well as it should. Maybe send the report to @package.debian.org instead? As a maintainer

Re: automatic bug filing by test robot

2008-08-28 Thread Jonathan Wiltshire
On Thu, Aug 28, 2008 at 09:38:54PM +1000, Brian May wrote: > Maybe send the report to @package.debian.org instead? > > As a maintainer I would like to know when a package fails to build for > any reason, even if it is due to factors outside my control. > I'm inclined to agree. I would find it ve

Re: automatic bug filing by test robot

2008-08-28 Thread Ian Jackson
Jonathan Wiltshire writes ("Re: automatic bug filing by test robot"): > I'm inclined to agree. I would find it very useful to get reports, but > not as a bug against my package. That seems to be the consensus. Fair enough. I shall see about getting a PTS tag for these repo

Re: automatic bug filing by test robot

2008-08-28 Thread Gonéri Le Bouder
On Thu, Aug 28, 2008 at 07:11:51PM +0100, Ian Jackson wrote: > Jonathan Wiltshire writes ("Re: automatic bug filing by test robot"): > > I'm inclined to agree. I would find it very useful to get reports, but > > not as a bug against my package. > > That seems t

Re: automatic bug filing by test robot

2008-08-29 Thread Steve Greenland
On 28-Aug-08, 02:19 (CDT), Raphael Hertzog <[EMAIL PROTECTED]> wrote: > Also, how would you check that a bug has not already been filed > manually by another user? Why, the same way all the manual duplicate bug filers[1] check :-) Steve [1] yes, I've filed a few duplicates in my life, no need to

Re: automatic bug filing by test robot

2008-09-01 Thread Lucas Nussbaum
On 28/08/08 at 09:19 +0200, Raphael Hertzog wrote: > On Wed, 27 Aug 2008, Ian Jackson wrote: > > I'm hoping to shortly turn on the automatic bug filing mechanism. > > I'm writing now to give people a chance to object :-). > [...] > > very few other packages. Most autopkgtest reports are FTBFS prob

Re: automatic bug filing by test robot

2008-09-01 Thread Lucas Nussbaum
On 27/08/08 at 22:10 +0100, Ian Jackson wrote: > Also, if you have any comments about the format of the email, > supporting references, etc., that would be helpful. I think that we have a problem with the way we export data from automated tests. Consensus is (I think) that bugs should not be filed

Re: automatic bug filing by test robot

2008-09-04 Thread Stefano Zacchiroli
On Wed, Aug 27, 2008 at 10:10:13PM +0100, Ian Jackson wrote: > it also does test installs. This is useful as it tests buildability > and installability Can you expand what do you mean by "test installs"? The main points which are not clear to me are: are dependencies taken into consideration? wh

Re: automatic bug filing by test robot

2008-09-07 Thread Ian Jackson
Lucas Nussbaum writes ("Re: automatic bug filing by test robot"): > I think that we have a problem with the way we export data from > automated tests. Consensus is (I think) that bugs should not be filed > automatically, unless the person filing the bugs is reasonably sure of &g

Re: automatic bug filing by test robot

2008-09-07 Thread Ian Jackson
Stefano Zacchiroli writes ("Re: automatic bug filing by test robot"): > Can you expand what do you mean by "test installs"? The equivalent of apt-get install (without a controlling terminal and with a noninteractive debconf frontend). > The main points which ar

Re: automatic bug filing by test robot

2008-09-07 Thread Gonéri Le Bouder
On Sun, Sep 07, 2008 at 06:35:12PM +0100, Ian Jackson wrote: > Lucas Nussbaum writes ("Re: automatic bug filing by test robot"): > > I think that we have a problem with the way we export data from > > automated tests. Consensus is (I think) that bugs should not be filed

Re: automatic bug filing by test robot

2008-09-08 Thread Stefano Zacchiroli
On Sun, Sep 07, 2008 at 06:35:12PM +0100, Ian Jackson wrote: > > Maybe you should focus on providing the data in a machine-parseable > > format, and write a seperate tool that would allow to gather data from > > all the tests we are running (I can think of lintian and piuparts, at > > least), and d