On Sat, Jun 20, 2020 at 11:03 PM Frank Rowand wrote:
>
> On 2020-06-20 01:44, David Gow wrote:
> > On Sat, Jun 20, 2020 at 1:58 AM Frank Rowand wrote:
> >>
> >> On 2020-06-16 07:08, Paolo Bonzini wrote:
> >>> On 15/06/20 21:07, Bird, Tim wrote:
> >
> >> Finally,
> >> - Should a SKIP res
On 2020-06-20 01:44, David Gow wrote:
> On Sat, Jun 20, 2020 at 1:58 AM Frank Rowand wrote:
>>
>> On 2020-06-16 07:08, Paolo Bonzini wrote:
>>> On 15/06/20 21:07, Bird, Tim wrote:
>
>> Finally,
>> - Should a SKIP result be 'ok' (TAP13 spec) or 'not ok' (current
>> kselftest practic
On 2020-06-19 17:58, Paolo Bonzini wrote:
> On 19/06/20 20:47, Frank Rowand wrote:
>> Or if the entire test depends on the missing config then Bail out might
>> be appropriate.
>
> No, in that case you want
>
> 1..0 # SKIP: unsupported configuration
>
> The spec is not clear if "Bail out!"
On Sat, Jun 20, 2020 at 1:58 AM Frank Rowand wrote:
>
> On 2020-06-16 07:08, Paolo Bonzini wrote:
> > On 15/06/20 21:07, Bird, Tim wrote:
> Finally,
> - Should a SKIP result be 'ok' (TAP13 spec) or 'not ok' (current
> kselftest practice)?
> See https://testanything.org/tap-
Just a quick note that there's been a lot of good discussion.
I have an updated draft of the document, but I need to review
the flurry of comments today, and I'm busy getting my slides
ready for a conference. So I just wanted to give a heads up
that I'll be working on this (responding to comments
On 19/06/20 20:47, Frank Rowand wrote:
> Or if the entire test depends on the missing config then Bail out might
> be appropriate.
No, in that case you want
1..0 # SKIP: unsupported configuration
The spec is not clear if "Bail out!" is an error condition or just a
warning that only part
On 2020-06-16 23:05, David Gow wrote:
> On Wed, Jun 17, 2020 at 11:36 AM Kees Cook wrote:
>>
>> On Wed, Jun 17, 2020 at 02:30:45AM +, Bird, Tim wrote:
>>> Agreed. You only need machine-parsable data if you expect the CI
>>> system to do something more with the data than just present it.
>>> W
On Tue, Jun 16, 2020 at 4:52 PM Kees Cook wrote:
>
> On Mon, Jun 15, 2020 at 07:07:34PM +, Bird, Tim wrote:
> > From: Kees Cook
> > > Note: making the plan line required differs from TAP13 and TAP14. I
> > > think it's the right choice, but we should be clear.
> >
> > [...]
> > With regards t
On 2020-06-15 14:07, Bird, Tim wrote:
> Kees,
>
> Thanks for the great feedback. See comments inline below.
>
>> -Original Message-
>> From: Kees Cook
>>
>> On Wed, Jun 10, 2020 at 06:11:06PM +, Bird, Tim wrote:
>>> The kernel test result format consists of 5 major elements,
>>> 4 o
On Tue, Jun 16, 2020 at 9:06 PM David Gow wrote:
>
> On Wed, Jun 17, 2020 at 11:36 AM Kees Cook wrote:
> >
> > On Wed, Jun 17, 2020 at 02:30:45AM +, Bird, Tim wrote:
> > > Agreed. You only need machine-parsable data if you expect the CI
> > > system to do something more with the data than ju
On Tue, Jun 16, 2020 at 2:16 PM Bird, Tim wrote:
>
>
>
> > -Original Message-
> > From: Brendan Higgins
> >
> > On Wed, Jun 10, 2020 at 06:11:06PM +, Bird, Tim wrote:
> > > Some months ago I started work on a document to formalize how
> > > kselftest implements the TAP specification.
On Tue, Jun 16, 2020 at 1:37 PM Bird, Tim wrote:
>
> > -Original Message-
> > From: Brendan Higgins
> >
> > On Mon, Jun 15, 2020 at 10:34 AM Bird, Tim wrote:
> > >
> > > > -Original Message-
> > > > From: David Gow
> > > >
> > > > On Thu, Jun 11, 2020 at 2:11 AM Bird, Tim wrote
On Fri, Jun 19, 2020 at 01:47:29PM -0500, Frank Rowand wrote:
> On 2020-06-16 18:58, Kees Cook wrote:
> > I proposed fixing that recently[1]. seccomp uses XFAIL for "I have
> > detected you lack the config to test this, so I can't say it's working
> > or not, because it only looks like a failure wi
On 2020-06-16 18:52, Kees Cook wrote:
> On Mon, Jun 15, 2020 at 07:07:34PM +, Bird, Tim wrote:
>> From: Kees Cook
>>> Note: making the plan line required differs from TAP13 and TAP14. I
>>> think it's the right choice, but we should be clear.
>>
>> [...]
>> With regards to making it optional o
On 2020-06-16 18:58, Kees Cook wrote:
> On Tue, Jun 16, 2020 at 12:44:28PM -0700, Brendan Higgins wrote:
>> On Tue, Jun 16, 2020 at 9:42 AM Bird, Tim wrote:
From: Paolo Bonzini
On 15/06/20 21:07, Bird, Tim wrote:
>> Note: making the plan line required differs from TAP13 and TAP
On 2020-06-16 11:42, Bird, Tim wrote:
>
>
>> -Original Message-
>> From: Paolo Bonzini
>>
>> On 15/06/20 21:07, Bird, Tim wrote:
Note: making the plan line required differs from TAP13 and TAP14. I
think it's the right choice, but we should be clear.
>>
>> As an aside, where is
On 2020-06-16 15:03, Brendan Higgins wrote:
> On Mon, Jun 15, 2020 at 10:34 AM Bird, Tim wrote:
>>
>>> -Original Message-
>>> From: David Gow
>>>
>>> On Thu, Jun 11, 2020 at 2:11 AM Bird, Tim wrote:
> [...]
>>> KUnit is currently outputting "TAP version 14", as we were hoping some
>>> of
On 2020-06-16 07:08, Paolo Bonzini wrote:
> On 15/06/20 21:07, Bird, Tim wrote:
>>> Note: making the plan line required differs from TAP13 and TAP14. I
>>> think it's the right choice, but we should be clear.
>
> As an aside, where is TAP14?
>
>> With regards to making it optional or not, I don't
On 2020-06-10 13:11, Bird, Tim wrote:
> Some months ago I started work on a document to formalize how
> kselftest implements the TAP specification. However, I didn't finish
> that work. Maybe it's time to do so now.
>
> kselftest has developed a few differences from the original
> TAP specificat
On Wed, Jun 17, 2020 at 11:36 AM Kees Cook wrote:
>
> On Wed, Jun 17, 2020 at 02:30:45AM +, Bird, Tim wrote:
> > Agreed. You only need machine-parsable data if you expect the CI
> > system to do something more with the data than just present it.
> > What that would be, that would be common fo
On Wed, Jun 17, 2020 at 02:30:45AM +, Bird, Tim wrote:
> Agreed. You only need machine-parsable data if you expect the CI
> system to do something more with the data than just present it.
> What that would be, that would be common for all tests (or at least
> many test), is unclear. Maybe the
> -Original Message-
> From: Kees Cook
>
> On Tue, Jun 16, 2020 at 09:16:01PM +, Bird, Tim wrote:
> > So far, most of the CI systems don't parse out diagnostic data, so it
> > doesn't
> > really matter what the format is. If it's useful for humans, it's valuable
> > as is.
> > Ho
On Tue, Jun 16, 2020 at 09:16:01PM +, Bird, Tim wrote:
> So far, most of the CI systems don't parse out diagnostic data, so it doesn't
> really matter what the format is. If it's useful for humans, it's valuable
> as is.
> However, it would be nice if that could change. But without some
> f
On Tue, Jun 16, 2020 at 08:37:18PM +, Bird, Tim wrote:
> One thing that needs to be rationalized between KUnit and selftest
> is the syntax for subtests. KUnit follows the TAP14 spec, and starts
> subtests with indented "# Subtest: name of the child test"
> and selftests just indents the outpu
On Tue, Jun 16, 2020 at 12:44:28PM -0700, Brendan Higgins wrote:
> On Tue, Jun 16, 2020 at 9:42 AM Bird, Tim wrote:
> > > From: Paolo Bonzini
> > >
> > > On 15/06/20 21:07, Bird, Tim wrote:
> > > >> Note: making the plan line required differs from TAP13 and TAP14. I
> > > >> think it's the right
On Mon, Jun 15, 2020 at 07:07:34PM +, Bird, Tim wrote:
> From: Kees Cook
> > Note: making the plan line required differs from TAP13 and TAP14. I
> > think it's the right choice, but we should be clear.
>
> [...]
> With regards to making it optional or not, I don't have a strong
> preference.
> -Original Message-
> From: Bird, Tim
>
> > Maybe in Documentation/dev-tools/ ?
> I'm leaning towards Documentation/dev-tools/test-results_format.rst
Ummm. Make that "test-results-format.rst"
(with a dash not an underline. I'm not insane, I swear...)
-- Tim
> -Original Message-
> From: Brendan Higgins
>
> On Wed, Jun 10, 2020 at 06:11:06PM +, Bird, Tim wrote:
> > Some months ago I started work on a document to formalize how
> > kselftest implements the TAP specification. However, I didn't finish
> > that work. Maybe it's time to do s
On Wed, Jun 10, 2020 at 06:11:06PM +, Bird, Tim wrote:
> Some months ago I started work on a document to formalize how
> kselftest implements the TAP specification. However, I didn't finish
> that work. Maybe it's time to do so now.
>
> kselftest has developed a few differences from the orig
> -Original Message-
> From: Brendan Higgins
>
> On Mon, Jun 15, 2020 at 10:34 AM Bird, Tim wrote:
> >
> > > -Original Message-
> > > From: David Gow
> > >
> > > On Thu, Jun 11, 2020 at 2:11 AM Bird, Tim wrote:
> [...]
> > > KUnit is currently outputting "TAP version 14", as we
> -Original Message-
> From: Brendan Higgins
>
> On Tue, Jun 16, 2020 at 9:42 AM Bird, Tim wrote:
>
> Apologies for taking so long to get to this. I have been busy with
> some stuff internally at Google.
>
> > > -Original Message-
> > > From: Paolo Bonzini
> > >
> > > On 15/0
On Mon, Jun 15, 2020 at 10:34 AM Bird, Tim wrote:
>
> > -Original Message-
> > From: David Gow
> >
> > On Thu, Jun 11, 2020 at 2:11 AM Bird, Tim wrote:
[...]
> > KUnit is currently outputting "TAP version 14", as we were hoping some
> > of our changes would get into the TAP14 spec. (Any
On Tue, Jun 16, 2020 at 9:42 AM Bird, Tim wrote:
Apologies for taking so long to get to this. I have been busy with
some stuff internally at Google.
> > -Original Message-
> > From: Paolo Bonzini
> >
> > On 15/06/20 21:07, Bird, Tim wrote:
> > >> Note: making the plan line required diff
> -Original Message-
> From: Paolo Bonzini
>
> On 15/06/20 21:07, Bird, Tim wrote:
> >> Note: making the plan line required differs from TAP13 and TAP14. I
> >> think it's the right choice, but we should be clear.
>
> As an aside, where is TAP14?
By TAP14, I was referring to the curren
On 15/06/20 21:07, Bird, Tim wrote:
>> Note: making the plan line required differs from TAP13 and TAP14. I
>> think it's the right choice, but we should be clear.
As an aside, where is TAP14?
> With regards to making it optional or not, I don't have a strong
> preference. The extra info seems he
Kees,
Thanks for the great feedback. See comments inline below.
> -Original Message-
> From: Kees Cook
>
> On Wed, Jun 10, 2020 at 06:11:06PM +, Bird, Tim wrote:
> > The kernel test result format consists of 5 major elements,
> > 4 of which are line-based:
> > * the output version
On Mon, Jun 15, 2020 at 05:45:28PM +, Bird, Tim wrote:
> Personally, as a human I find the space prefix a bit easier to read.
> However, I think that in "normal" kernel log output it is unusual for
> a line to be prefixed with a hash (#), so this might be easier to
> both visually distinguish a
gins ;
> linux-kernel-ment...@lists.linuxfoundation.org;
> li...@rasmusvillemoes.dk
> Subject: Re: RFC - kernel selftest result documentation (KTAP)
>
> On Sat, Jun 13, 2020 at 02:51:17PM +0800, David Gow wrote:
> > On Sat, Jun 13, 2020 at 6:36 AM Kees Cook wrote:
> > > Re
> -Original Message-
> From: David Gow
>
> On Thu, Jun 11, 2020 at 2:11 AM Bird, Tim wrote:
> >
> > Some months ago I started work on a document to formalize how
> > kselftest implements the TAP specification. However, I didn't finish
> > that work. Maybe it's time to do so now.
> >
>
On Wed, Jun 10, 2020 at 06:11:06PM +, Bird, Tim wrote:
> The kernel test result format consists of 5 major elements,
> 4 of which are line-based:
> * the output version line
> * the plan line
Note: making the plan line required differs from TAP13 and TAP14. I
think it's the right choice, but
On Sat, Jun 13, 2020 at 02:51:17PM +0800, David Gow wrote:
> On Sat, Jun 13, 2020 at 6:36 AM Kees Cook wrote:
> > Regarding output:
> >
> > [ 36.611358] TAP version 14
> > [ 36.611953] # Subtest: overflow
> > [ 36.611954] 1..3
> > ...
> > [ 36.622914] # overflow_calculation_tes
On Thu, Jun 11, 2020 at 2:11 AM Bird, Tim wrote:
>
> Some months ago I started work on a document to formalize how
> kselftest implements the TAP specification. However, I didn't finish
> that work. Maybe it's time to do so now.
>
> kselftest has developed a few differences from the original
> T
42 matches
Mail list logo
