Re: autopkgtest verision deployed on production infrastructure updated

2018-12-11 Thread Iain Lane
On Tue, Dec 11, 2018 at 12:16:26PM -0200, Andreas Hasenack wrote:
> On Mon, Dec 10, 2018 at 8:11 AM Iain Lane  wrote:
> >
> > +skippable
> > +The test might need to be skipped for reasons that cannot be
> > +described by an existing restriction such as isolation-machine or
> > +breaks-testbed, but must instead be detected at runtime. If the
> > +test exits with status 77 (a convention borrowed from Automake), it
> > +will be treated as though it had been skipped. If it exits with any
> > +other status, its success or failure will be derived from the exit
> > +status and stderr as usual. Test authors must be careful to ensure
> > +that ``skippable`` tests never exit with status 77 for reasons that
> > +should be treated as a failure.
> >
> 
> Is it ok to use skippable for arches where the test is known to not
> work? AFAIK there is no arch restriction support in d/t/control
> 
> I would then mark the test as skippable, detect the arch at runtime,
> and  if it's one we know it won't work, exit 77. I understand care
> must be taken to check the test and make sure it doesn't exit 77 for
> other reasons in that case.

That sounds fine, but consider first if you can encode *what* about the
system makes the test fail, if that can be done in a better way than the
architecture itself. You might be able to use skip-not-installable if
this is expressable via dependencies.

Cheers,

-- 
Iain Lane  [ i...@orangesquash.org.uk ]
Debian Developer   [ la...@debian.org ]
Ubuntu Developer   [ la...@ubuntu.com ]


signature.asc
Description: PGP signature
-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel


Re: autopkgtest verision deployed on production infrastructure updated

2018-12-11 Thread Andreas Hasenack
On Mon, Dec 10, 2018 at 8:11 AM Iain Lane  wrote:
>
> +skippable
> +The test might need to be skipped for reasons that cannot be
> +described by an existing restriction such as isolation-machine or
> +breaks-testbed, but must instead be detected at runtime. If the
> +test exits with status 77 (a convention borrowed from Automake), it
> +will be treated as though it had been skipped. If it exits with any
> +other status, its success or failure will be derived from the exit
> +status and stderr as usual. Test authors must be careful to ensure
> +that ``skippable`` tests never exit with status 77 for reasons that
> +should be treated as a failure.
>

Is it ok to use skippable for arches where the test is known to not
work? AFAIK there is no arch restriction support in d/t/control

I would then mark the test as skippable, detect the arch at runtime,
and  if it's one we know it won't work, exit 77. I understand care
must be taken to check the test and make sure it doesn't exit 77 for
other reasons in that case.

-- 
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel