On Tue, Jul 25, 2017 at 12:16:13PM -0400, Cleber Rosa wrote: > On 07/25/2017 11:49 AM, Stefan Hajnoczi wrote: > > On Fri, Jul 21, 2017 at 10:21:24AM -0400, Cleber Rosa wrote: > >> On 07/21/2017 10:01 AM, Daniel P. Berrange wrote: > >>> On Fri, Jul 21, 2017 at 01:33:25PM +0100, Stefan Hajnoczi wrote: > >>>> On Thu, Jul 20, 2017 at 11:47:27PM -0400, Cleber Rosa wrote: > >> Without the static capabilities defined, the dynamic check would be > >> influenced by the run time environment. It would really mean "qemu-io > >> running on this environment (filesystem?) can do native aio". Again, > >> that's not the best type of information to depend on when writing tests. > > > > Can you explain this more? > > > > It seems logical to me that if qemu-io in this environment cannot do > > aio=native then we must skip those tests. > > > > Stefan > > > > OK, let's abstract a bit more. Let's take this part of your statement: > > "if qemu-io in this environment cannot do aio=native" > > Let's call that a feature check. Depending on how the *feature check* > is written, a negative result may hide a test failure, because it would > now be skipped.
You are saying a pass->skip transition can hide a failure but ./check tracks skipped tests. See tests/qemu-iotests/check.log for a pass/fail/skip history. It is the job of the CI system to flag up pass/fail/skip transitions. You're no worse off using feature tests. Stefan
signature.asc
Description: PGP signature
