Re: [RFC v3 14/19] Documentation: kunit: add documentation for KUnit

[Brendan Higgins via dri-devel]

On Thu, Feb 14, 2019 at 9:26 AM Luis Chamberlain  wrote:
>
> On Wed, Feb 13, 2019 at 04:17:13PM -0800, Brendan Higgins wrote:
> > On Wed, Feb 13, 2019 at 1:55 PM Kieran Bingham
> >  wrote:
> > Oh, yep, you are right. Does that mean we should bother at all with a 
> > defconfig?
>
> If one wanted a qemu enabled type of kernel and also for kuniut one
> could imply run:
>
> make kvmconfig
> make kunitconfig
>
> That would get what you suggest above of default "bells and whistles"
> and keep the kuniut as a fragment.
>
> Hm, actually the kvmconfig doesn't really enable the required fragments
> for qemu, so perhaps one would be good. It would have the serial stuff
> for instance.
>
> > Luis, I know you said you wanted one. I am thinking just stick with
> > the UML one? The downside there is we then get stuck having to
> > maintain the fragment and the defconfig. I right now (in the new
> > revision I am working on) have the Python kunit_tool copy the
> > defconfig if no kunitconfig is provided and a flag is set. It would be
> > pretty straightforward to make it merge in the fragment instead.
>
> Up to you in the end.

I don't really have any opinions on the matter; I don't really use
defconfigs in any of my workflows. So, I just want whatever is easier
for people. The thing that makes the most sense to me would be to
provide a "merge-kunitconfig" option similar to what kselftest does,
but I don't intend on doing that in the initial patchset, unless
someone really thinks that I should do it. So in the meantime, I guess
provide both since that gives people options?

In anycase, I just (finally) sent out v4, so I suggest we continue the
discussion over there.

Cheers
___
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [RFC v3 14/19] Documentation: kunit: add documentation for KUnit

[Luis Chamberlain via dri-devel]

On Wed, Feb 13, 2019 at 04:17:13PM -0800, Brendan Higgins wrote:
> On Wed, Feb 13, 2019 at 1:55 PM Kieran Bingham
>  wrote:
> Oh, yep, you are right. Does that mean we should bother at all with a 
> defconfig?

If one wanted a qemu enabled type of kernel and also for kuniut one
could imply run:

make kvmconfig
make kunitconfig

That would get what you suggest above of default "bells and whistles"
and keep the kuniut as a fragment.

Hm, actually the kvmconfig doesn't really enable the required fragments
for qemu, so perhaps one would be good. It would have the serial stuff
for instance.

> Luis, I know you said you wanted one. I am thinking just stick with
> the UML one? The downside there is we then get stuck having to
> maintain the fragment and the defconfig. I right now (in the new
> revision I am working on) have the Python kunit_tool copy the
> defconfig if no kunitconfig is provided and a flag is set. It would be
> pretty straightforward to make it merge in the fragment instead.

Up to you in the end.

  Luis
___
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [RFC v3 14/19] Documentation: kunit: add documentation for KUnit

[Brendan Higgins via dri-devel]

On Wed, Feb 13, 2019 at 1:55 PM Kieran Bingham
 wrote:
>
> Hi Brendan,
>
> On 12/02/2019 22:10, Brendan Higgins wrote:
> > On Mon, Feb 11, 2019 at 4:16 AM Kieran Bingham
> >  wrote:
> >>
> >> Hi Brendan,
> >>
> >> On 09/02/2019 00:56, Brendan Higgins wrote:
> >>> On Thu, Dec 6, 2018 at 4:16 AM Kieran Bingham
> >>>  wrote:
> 
>  Hi Brendan,
> 
>  On 03/12/2018 23:53, Brendan Higgins wrote:
> > On Thu, Nov 29, 2018 at 7:45 PM Luis Chamberlain  
> > wrote:
> >>
> >> On Thu, Nov 29, 2018 at 01:56:37PM +, Kieran Bingham wrote:
> >>> Hi Brendan,
> >>>
> >>> Please excuse the top posting, but I'm replying here as I'm following
> >>> the section "Creating a kunitconfig" in Documentation/kunit/start.rst.
> >>>
> >>> Could the three line kunitconfig file live under say
> >>>arch/um/configs/kunit_defconfig?
> 
> 
>  Further consideration to this topic - I mentioned putting it in
>    arch/um/configs
> 
>  - but I think this is wrong.
> 
>  We now have a location for config-fragments, which is essentially what
>  this is, under kernel/configs
> 
>  So perhaps an addition as :
> 
>   kernel/configs/kunit.config
> 
>  Would be more appropriate - and less (UM) architecture specific.
> >>>
> >>> Sorry for the long radio silence.
> >>>
> >>> I just got around to doing this and I found that there are some
> >>> configs that are desirable to have when running KUnit under x86 in a
> >>> VM, but not UML.
> >>
> >> Should this behaviour you mention be handled by the KCONFIG depends flags?
> >>
> >> depends on (KUMIT & UML)
> >> or
> >> depends on (KUNIT & !UML)
> >>
> >> or such?
> >
> > Not really. Anything that is strictly necessary to run KUnit on an
> > architectures should of course be turned on as a dependency like you
> > suggest, but I am talking about stuff that you would probably want to
> > get yourself going, but is by no means necessary.
> >
> >>
> >> An example of which configs you are referring to would help to
> >> understand the issue perhaps.
> >>
> >
> > For example, you might want to enable a serial console that is known
> > to work with a fairly generic qemu setup when building for x86:
> > CONFIG_SERIAL_8250=y
> > CONFIG_SERIAL_8250_CONSOLE=y
> >
> > Obviously not a dependency, and not even particularly useful to people
> > who know what they are doing, but to someone who is new or just wants
> > something to work out of the box would probably want that.
>
> It sounds like that would be a config fragment for qemu ?
>
> Although - perhaps this is already covered by the following fragment:
>kernel/configs/kvm_guest.config
>

Oh, yep, you are right. Does that mean we should bother at all with a defconfig?

Luis, I know you said you wanted one. I am thinking just stick with
the UML one? The downside there is we then get stuck having to
maintain the fragment and the defconfig. I right now (in the new
revision I am working on) have the Python kunit_tool copy the
defconfig if no kunitconfig is provided and a flag is set. It would be
pretty straightforward to make it merge in the fragment instead.

All that being said, I think I am going to drop the arch/x86
defconfig, since I think we all agree that it is not very useful, but
keep the UML defconfig and the fragment. That will at least given
something concrete to discuss.

>
> >>> So should we have one that goes in with
> >>> config-fragments and others that go into architectures? Another idea,
> >>> it would be nice to have a KUnit config that runs all known tests
> >>
> >> This might also be a config option added to the tests directly like
> >> COMPILE_TEST perhaps?
> >
> > That just allows a bunch of drivers to be compiled, it does not
> > actually go through and turn the configs on, right? I mean, there is
> > no a priori way to know that there is a configuration which spans all
> > possible options available under COMPILE_TEST, right? Maybe I
> > misunderstand what you are suggesting...
>
> Bah - you're right of course. I was mis-remembering the functionality of
> COMPILE_TEST as if it were some sort of 'select' but it's just an enable..
>
> Sorry for the confusion.
>

No problem, I thought for a second that was a good example too (and I
wish it were. It would make my life so much easier!). I remember
getting emails with a COMPILE_TEST config attached that demonstrates
an invalid build caused by my changes, presumably that email bot just
tries random configs with a new change until it finds one that breaks.

>
> >> (Not sure what that would be called though ... KUNIT_RUNTIME_TEST?)
> >>
> >> I think that might be more maintainable as otherwise each new test would
> >> have to modify the {min,def}{config,fragment} ...
> >>
> >
> > Looking at kselftest-merge, they just start out with a set of
> > fragments in which the union should contain all tests and then merge
> > it with a base .config (probably intended to be $(ARCH)_defc

Re: [RFC v3 14/19] Documentation: kunit: add documentation for KUnit

[Brendan Higgins via dri-devel]

On Mon, Feb 11, 2019 at 4:16 AM Kieran Bingham
 wrote:
>
> Hi Brendan,
>
> On 09/02/2019 00:56, Brendan Higgins wrote:
> > On Thu, Dec 6, 2018 at 4:16 AM Kieran Bingham
> >  wrote:
> >>
> >> Hi Brendan,
> >>
> >> On 03/12/2018 23:53, Brendan Higgins wrote:
> >>> On Thu, Nov 29, 2018 at 7:45 PM Luis Chamberlain  
> >>> wrote:
> 
>  On Thu, Nov 29, 2018 at 01:56:37PM +, Kieran Bingham wrote:
> > Hi Brendan,
> >
> > Please excuse the top posting, but I'm replying here as I'm following
> > the section "Creating a kunitconfig" in Documentation/kunit/start.rst.
> >
> > Could the three line kunitconfig file live under say
> >arch/um/configs/kunit_defconfig?
> >>
> >>
> >> Further consideration to this topic - I mentioned putting it in
> >>   arch/um/configs
> >>
> >> - but I think this is wrong.
> >>
> >> We now have a location for config-fragments, which is essentially what
> >> this is, under kernel/configs
> >>
> >> So perhaps an addition as :
> >>
> >>  kernel/configs/kunit.config
> >>
> >> Would be more appropriate - and less (UM) architecture specific.
> >
> > Sorry for the long radio silence.
> >
> > I just got around to doing this and I found that there are some
> > configs that are desirable to have when running KUnit under x86 in a
> > VM, but not UML.
>
> Should this behaviour you mention be handled by the KCONFIG depends flags?
>
> depends on (KUMIT & UML)
> or
> depends on (KUNIT & !UML)
>
> or such?

Not really. Anything that is strictly necessary to run KUnit on an
architectures should of course be turned on as a dependency like you
suggest, but I am talking about stuff that you would probably want to
get yourself going, but is by no means necessary.

>
> An example of which configs you are referring to would help to
> understand the issue perhaps.
>

For example, you might want to enable a serial console that is known
to work with a fairly generic qemu setup when building for x86:
CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_CONSOLE=y

Obviously not a dependency, and not even particularly useful to people
who know what they are doing, but to someone who is new or just wants
something to work out of the box would probably want that.

>
> > So should we have one that goes in with
> > config-fragments and others that go into architectures? Another idea,
> > it would be nice to have a KUnit config that runs all known tests
>
> This might also be a config option added to the tests directly like
> COMPILE_TEST perhaps?

That just allows a bunch of drivers to be compiled, it does not
actually go through and turn the configs on, right? I mean, there is
no a priori way to know that there is a configuration which spans all
possible options available under COMPILE_TEST, right? Maybe I
misunderstand what you are suggesting...

>
> (Not sure what that would be called though ... KUNIT_RUNTIME_TEST?)
>
> I think that might be more maintainable as otherwise each new test would
> have to modify the {min,def}{config,fragment} ...
>

Looking at kselftest-merge, they just start out with a set of
fragments in which the union should contain all tests and then merge
it with a base .config (probably intended to be $(ARCH)_defconfig).
However, I don't know if that is the state of the art.

>
> > (this probably won't work in practice once we start testing mutually
> > exclusive things or things with lots of ifdeffery, but it probably
> > something we should try to maintain as best as we can?); this probably
> > shouldn't go in with the fragments, right?
>
> Sounds like we agree there :)

Totally. Long term we will need something a lot more sophisticated
than anything under discussion here. I was talking about this with
Luis on another thread:
https://groups.google.com/forum/#!topic/kunit-dev/EQ1x0SzrUus (feel
free to chime in!). Nevertheless, that's a really hard problem and I
figure some variant of defconfigs and config fragments will work well
enough until we reach that point.

>
> >
> > I will be sending another revision out soon, but I figured I might be
> > able to catch you before I did so.
>
> Thanks for thinking of me.

How can I forget? You have been super helpful!

> I hope I managed to reply in time to help and not hinder your progress.

Yep, no trouble at all. You are the one helping me :-)

Thanks!
___
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [RFC v3 14/19] Documentation: kunit: add documentation for KUnit

[Kieran Bingham]

Hi Brendan,

On 12/02/2019 22:10, Brendan Higgins wrote:
> On Mon, Feb 11, 2019 at 4:16 AM Kieran Bingham
>  wrote:
>>
>> Hi Brendan,
>>
>> On 09/02/2019 00:56, Brendan Higgins wrote:
>>> On Thu, Dec 6, 2018 at 4:16 AM Kieran Bingham
>>>  wrote:

 Hi Brendan,

 On 03/12/2018 23:53, Brendan Higgins wrote:
> On Thu, Nov 29, 2018 at 7:45 PM Luis Chamberlain  
> wrote:
>>
>> On Thu, Nov 29, 2018 at 01:56:37PM +, Kieran Bingham wrote:
>>> Hi Brendan,
>>>
>>> Please excuse the top posting, but I'm replying here as I'm following
>>> the section "Creating a kunitconfig" in Documentation/kunit/start.rst.
>>>
>>> Could the three line kunitconfig file live under say
>>>arch/um/configs/kunit_defconfig?


 Further consideration to this topic - I mentioned putting it in
   arch/um/configs

 - but I think this is wrong.

 We now have a location for config-fragments, which is essentially what
 this is, under kernel/configs

 So perhaps an addition as :

  kernel/configs/kunit.config

 Would be more appropriate - and less (UM) architecture specific.
>>>
>>> Sorry for the long radio silence.
>>>
>>> I just got around to doing this and I found that there are some
>>> configs that are desirable to have when running KUnit under x86 in a
>>> VM, but not UML.
>>
>> Should this behaviour you mention be handled by the KCONFIG depends flags?
>>
>> depends on (KUMIT & UML)
>> or
>> depends on (KUNIT & !UML)
>>
>> or such?
> 
> Not really. Anything that is strictly necessary to run KUnit on an
> architectures should of course be turned on as a dependency like you
> suggest, but I am talking about stuff that you would probably want to
> get yourself going, but is by no means necessary.
> 
>>
>> An example of which configs you are referring to would help to
>> understand the issue perhaps.
>>
> 
> For example, you might want to enable a serial console that is known
> to work with a fairly generic qemu setup when building for x86:
> CONFIG_SERIAL_8250=y
> CONFIG_SERIAL_8250_CONSOLE=y
> 
> Obviously not a dependency, and not even particularly useful to people
> who know what they are doing, but to someone who is new or just wants
> something to work out of the box would probably want that.

It sounds like that would be a config fragment for qemu ?

Although - perhaps this is already covered by the following fragment:
   kernel/configs/kvm_guest.config


>>> So should we have one that goes in with
>>> config-fragments and others that go into architectures? Another idea,
>>> it would be nice to have a KUnit config that runs all known tests
>>
>> This might also be a config option added to the tests directly like
>> COMPILE_TEST perhaps?
> 
> That just allows a bunch of drivers to be compiled, it does not
> actually go through and turn the configs on, right? I mean, there is
> no a priori way to know that there is a configuration which spans all
> possible options available under COMPILE_TEST, right? Maybe I
> misunderstand what you are suggesting...

Bah - you're right of course. I was mis-remembering the functionality of
COMPILE_TEST as if it were some sort of 'select' but it's just an enable..

Sorry for the confusion.



>> (Not sure what that would be called though ... KUNIT_RUNTIME_TEST?)
>>
>> I think that might be more maintainable as otherwise each new test would
>> have to modify the {min,def}{config,fragment} ...
>>
> 
> Looking at kselftest-merge, they just start out with a set of
> fragments in which the union should contain all tests and then merge
> it with a base .config (probably intended to be $(ARCH)_defconfig).
> However, I don't know if that is the state of the art.
> 
>>
>>> (this probably won't work in practice once we start testing mutually
>>> exclusive things or things with lots of ifdeffery, but it probably
>>> something we should try to maintain as best as we can?); this probably
>>> shouldn't go in with the fragments, right?
>>
>> Sounds like we agree there :)
> 
> Totally. Long term we will need something a lot more sophisticated
> than anything under discussion here. I was talking about this with
> Luis on another thread:
> https://groups.google.com/forum/#!topic/kunit-dev/EQ1x0SzrUus (feel
> free to chime in!). Nevertheless, that's a really hard problem and I
> figure some variant of defconfigs and config fragments will work well
> enough until we reach that point.
> 
>>
>>>
>>> I will be sending another revision out soon, but I figured I might be
>>> able to catch you before I did so.
>>
>> Thanks for thinking of me.
> 
> How can I forget? You have been super helpful!
> 
>> I hope I managed to reply in time to help and not hinder your progress.
> 
> Yep, no trouble at all. You are the one helping me :-)
> 
> Thanks!
> 

-- 
Regards
--
Kieran
___
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailma

Re: [RFC v3 14/19] Documentation: kunit: add documentation for KUnit

[Kieran Bingham]

Hi Brendan,

On 09/02/2019 00:56, Brendan Higgins wrote:
> On Thu, Dec 6, 2018 at 4:16 AM Kieran Bingham
>  wrote:
>>
>> Hi Brendan,
>>
>> On 03/12/2018 23:53, Brendan Higgins wrote:
>>> On Thu, Nov 29, 2018 at 7:45 PM Luis Chamberlain  wrote:

 On Thu, Nov 29, 2018 at 01:56:37PM +, Kieran Bingham wrote:
> Hi Brendan,
>
> Please excuse the top posting, but I'm replying here as I'm following
> the section "Creating a kunitconfig" in Documentation/kunit/start.rst.
>
> Could the three line kunitconfig file live under say
>arch/um/configs/kunit_defconfig?
>>
>>
>> Further consideration to this topic - I mentioned putting it in
>>   arch/um/configs
>>
>> - but I think this is wrong.
>>
>> We now have a location for config-fragments, which is essentially what
>> this is, under kernel/configs
>>
>> So perhaps an addition as :
>>
>>  kernel/configs/kunit.config
>>
>> Would be more appropriate - and less (UM) architecture specific.
> 
> Sorry for the long radio silence.
> 
> I just got around to doing this and I found that there are some
> configs that are desirable to have when running KUnit under x86 in a
> VM, but not UML. 

Should this behaviour you mention be handled by the KCONFIG depends flags?

depends on (KUMIT & UML)
or
depends on (KUNIT & !UML)

or such?

An example of which configs you are referring to would help to
understand the issue perhaps.


> So should we have one that goes in with
> config-fragments and others that go into architectures? Another idea,
> it would be nice to have a KUnit config that runs all known tests

This might also be a config option added to the tests directly like
COMPILE_TEST perhaps?

(Not sure what that would be called though ... KUNIT_RUNTIME_TEST?)

I think that might be more maintainable as otherwise each new test would
have to modify the {min,def}{config,fragment} ...


> (this probably won't work in practice once we start testing mutually
> exclusive things or things with lots of ifdeffery, but it probably
> something we should try to maintain as best as we can?); this probably
> shouldn't go in with the fragments, right?

Sounds like we agree there :)

> 
> I will be sending another revision out soon, but I figured I might be
> able to catch you before I did so.

Thanks for thinking of me.
I hope I managed to reply in time to help and not hinder your progress.

-- 
Regards
--
Kieran
___
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [RFC v3 14/19] Documentation: kunit: add documentation for KUnit

[Brendan Higgins]

On Thu, Dec 6, 2018 at 4:16 AM Kieran Bingham
 wrote:
>
> Hi Brendan,
>
> On 03/12/2018 23:53, Brendan Higgins wrote:
> > On Thu, Nov 29, 2018 at 7:45 PM Luis Chamberlain  wrote:
> >>
> >> On Thu, Nov 29, 2018 at 01:56:37PM +, Kieran Bingham wrote:
> >>> Hi Brendan,
> >>>
> >>> Please excuse the top posting, but I'm replying here as I'm following
> >>> the section "Creating a kunitconfig" in Documentation/kunit/start.rst.
> >>>
> >>> Could the three line kunitconfig file live under say
> >>>arch/um/configs/kunit_defconfig?
>
>
> Further consideration to this topic - I mentioned putting it in
>   arch/um/configs
>
> - but I think this is wrong.
>
> We now have a location for config-fragments, which is essentially what
> this is, under kernel/configs
>
> So perhaps an addition as :
>
>  kernel/configs/kunit.config
>
> Would be more appropriate - and less (UM) architecture specific.

Sorry for the long radio silence.

I just got around to doing this and I found that there are some
configs that are desirable to have when running KUnit under x86 in a
VM, but not UML. So should we have one that goes in with
config-fragments and others that go into architectures? Another idea,
it would be nice to have a KUnit config that runs all known tests
(this probably won't work in practice once we start testing mutually
exclusive things or things with lots of ifdeffery, but it probably
something we should try to maintain as best as we can?); this probably
shouldn't go in with the fragments, right?

I will be sending another revision out soon, but I figured I might be
able to catch you before I did so.
___
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [RFC v3 14/19] Documentation: kunit: add documentation for KUnit

[Kieran Bingham]

Hi Brendan,

On 03/12/2018 23:53, Brendan Higgins wrote:
> On Thu, Nov 29, 2018 at 7:45 PM Luis Chamberlain  wrote:
>>
>> On Thu, Nov 29, 2018 at 01:56:37PM +, Kieran Bingham wrote:
>>> Hi Brendan,
>>>
>>> Please excuse the top posting, but I'm replying here as I'm following
>>> the section "Creating a kunitconfig" in Documentation/kunit/start.rst.
>>>
>>> Could the three line kunitconfig file live under say
>>>arch/um/configs/kunit_defconfig?


Further consideration to this topic - I mentioned putting it in
  arch/um/configs

- but I think this is wrong.

We now have a location for config-fragments, which is essentially what
this is, under kernel/configs

So perhaps an addition as :

 kernel/configs/kunit.config

Would be more appropriate - and less (UM) architecture specific.



>>>
>>> So that it's always provided? And could even be extended with tests
>>> which people would expect to be run by default? (say in distributions)
>>
>> Indeed, and then a top level 'make kunitconfig' could use it as well.
> 
> Yep, I totally agree.
> 

-- 
Regards
--
Kieran
___
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [RFC v3 14/19] Documentation: kunit: add documentation for KUnit

[Brendan Higgins]

On Thu, Nov 29, 2018 at 7:45 PM Luis Chamberlain  wrote:
>
> On Thu, Nov 29, 2018 at 01:56:37PM +, Kieran Bingham wrote:
> > Hi Brendan,
> >
> > Please excuse the top posting, but I'm replying here as I'm following
> > the section "Creating a kunitconfig" in Documentation/kunit/start.rst.
> >
> > Could the three line kunitconfig file live under say
> >arch/um/configs/kunit_defconfig?
> >
> > So that it's always provided? And could even be extended with tests
> > which people would expect to be run by default? (say in distributions)
>
> Indeed, and then a top level 'make kunitconfig' could use it as well.

Yep, I totally agree.
___
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [RFC v3 14/19] Documentation: kunit: add documentation for KUnit

[Luis Chamberlain]

On Thu, Nov 29, 2018 at 01:56:37PM +, Kieran Bingham wrote:
> Hi Brendan,
> 
> Please excuse the top posting, but I'm replying here as I'm following
> the section "Creating a kunitconfig" in Documentation/kunit/start.rst.
> 
> Could the three line kunitconfig file live under say
>arch/um/configs/kunit_defconfig?
> 
> So that it's always provided? And could even be extended with tests
> which people would expect to be run by default? (say in distributions)

Indeed, and then a top level 'make kunitconfig' could use it as well.

  Luis
___
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [RFC v3 14/19] Documentation: kunit: add documentation for KUnit

[Kieran Bingham]

Hi Brendan,

Please excuse the top posting, but I'm replying here as I'm following
the section "Creating a kunitconfig" in Documentation/kunit/start.rst.

Could the three line kunitconfig file live under say
 arch/um/configs/kunit_defconfig?

So that it's always provided? And could even be extended with tests
which people would expect to be run by default? (say in distributions)

--
Kieran




On 28/11/2018 19:36, Brendan Higgins wrote:
> Add documentation for KUnit, the Linux kernel unit testing framework.
> - Add intro and usage guide for KUnit
> - Add API reference
> 
> Signed-off-by: Felix Guo 
> Signed-off-by: Brendan Higgins 
> ---
>  Documentation/index.rst   |   1 +
>  Documentation/kunit/api/index.rst |  16 ++
>  Documentation/kunit/api/test.rst  |  15 +
>  Documentation/kunit/faq.rst   |  46 +++
>  Documentation/kunit/index.rst |  80 ++
>  Documentation/kunit/start.rst | 180 
>  Documentation/kunit/usage.rst | 447 ++
>  7 files changed, 785 insertions(+)
>  create mode 100644 Documentation/kunit/api/index.rst
>  create mode 100644 Documentation/kunit/api/test.rst
>  create mode 100644 Documentation/kunit/faq.rst
>  create mode 100644 Documentation/kunit/index.rst
>  create mode 100644 Documentation/kunit/start.rst
>  create mode 100644 Documentation/kunit/usage.rst
> 
> diff --git a/Documentation/index.rst b/Documentation/index.rst
> index 5db7e87c7cb1d..275ef4db79f61 100644
> --- a/Documentation/index.rst
> +++ b/Documentation/index.rst
> @@ -68,6 +68,7 @@ merged much easier.
> kernel-hacking/index
> trace/index
> maintainer/index
> +   kunit/index
>  
>  Kernel API documentation
>  
> diff --git a/Documentation/kunit/api/index.rst 
> b/Documentation/kunit/api/index.rst
> new file mode 100644
> index 0..c31c530088153
> --- /dev/null
> +++ b/Documentation/kunit/api/index.rst
> @@ -0,0 +1,16 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +=
> +API Reference
> +=
> +.. toctree::
> +
> + test
> +
> +This section documents the KUnit kernel testing API. It is divided into 3
> +sections:
> +
> += 
> ==
> +:doc:`test`   documents all of the standard testing API
> +  excluding mocking or mocking related 
> features.
> += 
> ==
> diff --git a/Documentation/kunit/api/test.rst 
> b/Documentation/kunit/api/test.rst
> new file mode 100644
> index 0..7c926014f047c
> --- /dev/null
> +++ b/Documentation/kunit/api/test.rst
> @@ -0,0 +1,15 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +
> +Test API
> +
> +
> +This file documents all of the standard testing API excluding mocking or 
> mocking
> +related features.
> +
> +.. kernel-doc:: include/kunit/test.h
> +   :internal:
> +
> +.. kernel-doc:: include/kunit/kunit-stream.h
> +   :internal:
> +
> diff --git a/Documentation/kunit/faq.rst b/Documentation/kunit/faq.rst
> new file mode 100644
> index 0..cb8e4fb2257a0
> --- /dev/null
> +++ b/Documentation/kunit/faq.rst
> @@ -0,0 +1,46 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +=
> +Frequently Asked Questions
> +=
> +
> +How is this different from Autotest, kselftest, etc?
> +
> +KUnit is a unit testing framework. Autotest, kselftest (and some others) are
> +not.
> +
> +A `unit test `_ is supposed to
> +test a single unit of code in isolation, hence the name. A unit test should 
> be
> +the finest granularity of testing and as such should allow all possible code
> +paths to be tested in the code under test; this is only possible if the code
> +under test is very small and does not have any external dependencies outside 
> of
> +the test's control like hardware.
> +
> +There are no testing frameworks currently available for the kernel that do 
> not
> +require installing the kernel on a test machine or in a VM and all require
> +tests to be written in userspace and run on the kernel under test; this is 
> true
> +for Autotest, kselftest, and some others, disqualifying any of them from 
> being
> +considered unit testing frameworks.
> +
> +What is the difference between a unit test and these other kinds of tests?
> +==
> +Most existing tests for the Linux kernel would be categorized as an 
> integration
> +test, or an end-to-end test.
> +
> +- A unit test is supposed to test a single unit of code in isolation, hence 
> the
> +  name. A unit test should be the finest granularity of testing and as such
> +  should allow all possible code paths to be tested in the code under test; 
>