Re: Test for clock_nanosleep with CLOCK_MONOTONIC (# 3890)

2020-04-09 Thread Utkarsh Rai 
Thank you for the clarification, I will proceed accordingly.

On Thu, Apr 9, 2020 at 10:36 PM Joel Sherrill  wrote:

>
>
> On Thu, Apr 9, 2020 at 11:56 AM Utkarsh Rai 
> wrote:
>
>> Thank you, under psxtests/psxhdrs/time we have a test for clock_nanosleep
>> for CLOCK_REALTIME, would it be a good idea to add test for CLOCK_MONOTONIC
>> under the same test, or should I add a different test using RTEMS Test
>> framework?
>>
>
> psxhdr tests are NOT functionality tests. They only test that the method
> interface is prototyped as required by POSIX. For example, the following
> says you should only have to include  and then make any call to
> clock_nanosleep():
>
>
> https://pubs.opengroup.org/onlinepubs/9699919799/functions/clock_nanosleep.html
>
>
> That's what psxhdr tests verify. It would make sense to add a second
> clock_nanosleep psxhdr test which uses the CLOCK_MONOTONIC enumeration.
>
> You will need to add a functional test which is focused on ensuring using
> clock_nanosleep() with CLOCK_MONOTONIC works as expected. For example, does
> it delay the correct minimum length of time based on
> clock_gettime(CLOCK_MONOTONIC)? If you change CLOCK_REALTIME, does it not
> have any impact on the delay period of a CLOCK_MONOTONIC nanosleep call?
>
> The functional test should cover the code in clock_nanosleep that is
> related to CLOCK_MONOTONIC and not tested when you delay using
> CLOCK_REALTIME.
>
> --joel
>
>>
>> On Thu, Apr 9, 2020 at 6:43 PM Sebastian Huber <
>> sebastian.hu...@embedded-brains.de> wrote:
>>
>>>
>>> On 09/04/2020 15:04, Joel Sherrill wrote:
>>>
>>> On Thu, Apr 9, 2020, 7:43 AM Sebastian Huber <
>>> sebastian.hu...@embedded-brains.de> wrote:
>>>
 On 09/04/2020 14:40, Joel Sherrill wrote:

 On Thu, Apr 9, 2020, 7:28 AM Utkarsh Rai 
 wrote:

> Hi,
> I am willing to add tests for clock_nanosleep with CLOCK_MONOTONIC.
> What is the standard way of adding test for an already present  API but
> with different configuration? For eg. should I add  
> 'psxtmclocknanosleep04/
> 05/ 06' in the testsuite?
>

 Yes. That is the pattern.

 We should try to reduce the count of test programs since on boards with
 a long reboot time, more tests programs means much more test time (compared
 to the new test cases alone).

>>>
>>> And there is the competing factor that you end up with test executables
>>> that are overly complex, do not have clean environments for many of the
>>> test cases, and are too large to fit on many target boards.
>>>
>>> The RTEMS Test framework has checks to ensure that the environment is
>>> sane before a new test case is executed. It decouples the test cases from
>>> the runner. This could be used to group test cases to test suites depending
>>> on the target memory size.
>>>
>>>
>>> I know you have seen how long the list is of tests that you can't run on
>>> many boards.  That's a bad quality attribute
>>>
>>> I don't think that tests for clock_nanosleep() will result in big
>>> executables. The executable size is mostly defined by the feature to be
>>> tested.
>>>
>>
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: Test for clock_nanosleep with CLOCK_MONOTONIC (# 3890)

2020-04-09 Thread Joel Sherrill 
On Thu, Apr 9, 2020 at 11:56 AM Utkarsh Rai  wrote:

> Thank you, under psxtests/psxhdrs/time we have a test for clock_nanosleep
> for CLOCK_REALTIME, would it be a good idea to add test for CLOCK_MONOTONIC
> under the same test, or should I add a different test using RTEMS Test
> framework?
>

psxhdr tests are NOT functionality tests. They only test that the method
interface is prototyped as required by POSIX. For example, the following
says you should only have to include  and then make any call to
clock_nanosleep():

https://pubs.opengroup.org/onlinepubs/9699919799/functions/clock_nanosleep.html


That's what psxhdr tests verify. It would make sense to add a second
clock_nanosleep psxhdr test which uses the CLOCK_MONOTONIC enumeration.

You will need to add a functional test which is focused on ensuring using
clock_nanosleep() with CLOCK_MONOTONIC works as expected. For example, does
it delay the correct minimum length of time based on
clock_gettime(CLOCK_MONOTONIC)? If you change CLOCK_REALTIME, does it not
have any impact on the delay period of a CLOCK_MONOTONIC nanosleep call?

The functional test should cover the code in clock_nanosleep that is
related to CLOCK_MONOTONIC and not tested when you delay using
CLOCK_REALTIME.

--joel

>
> On Thu, Apr 9, 2020 at 6:43 PM Sebastian Huber <
> sebastian.hu...@embedded-brains.de> wrote:
>
>>
>> On 09/04/2020 15:04, Joel Sherrill wrote:
>>
>> On Thu, Apr 9, 2020, 7:43 AM Sebastian Huber <
>> sebastian.hu...@embedded-brains.de> wrote:
>>
>>> On 09/04/2020 14:40, Joel Sherrill wrote:
>>>
>>> On Thu, Apr 9, 2020, 7:28 AM Utkarsh Rai 
>>> wrote:
>>>
 Hi,
 I am willing to add tests for clock_nanosleep with CLOCK_MONOTONIC.
 What is the standard way of adding test for an already present  API but
 with different configuration? For eg. should I add  'psxtmclocknanosleep04/
 05/ 06' in the testsuite?

>>>
>>> Yes. That is the pattern.
>>>
>>> We should try to reduce the count of test programs since on boards with
>>> a long reboot time, more tests programs means much more test time (compared
>>> to the new test cases alone).
>>>
>>
>> And there is the competing factor that you end up with test executables
>> that are overly complex, do not have clean environments for many of the
>> test cases, and are too large to fit on many target boards.
>>
>> The RTEMS Test framework has checks to ensure that the environment is
>> sane before a new test case is executed. It decouples the test cases from
>> the runner. This could be used to group test cases to test suites depending
>> on the target memory size.
>>
>>
>> I know you have seen how long the list is of tests that you can't run on
>> many boards.  That's a bad quality attribute
>>
>> I don't think that tests for clock_nanosleep() will result in big
>> executables. The executable size is mostly defined by the feature to be
>> tested.
>>
>
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: Test for clock_nanosleep with CLOCK_MONOTONIC (# 3890)

2020-04-09 Thread Utkarsh Rai 
Thank you, under psxtests/psxhdrs/time we have a test for clock_nanosleep
for CLOCK_REALTIME, would it be a good idea to add test for CLOCK_MONOTONIC
under the same test, or should I add a different test using RTEMS Test
framework?

On Thu, Apr 9, 2020 at 6:43 PM Sebastian Huber <
sebastian.hu...@embedded-brains.de> wrote:

>
> On 09/04/2020 15:04, Joel Sherrill wrote:
>
> On Thu, Apr 9, 2020, 7:43 AM Sebastian Huber <
> sebastian.hu...@embedded-brains.de> wrote:
>
>> On 09/04/2020 14:40, Joel Sherrill wrote:
>>
>> On Thu, Apr 9, 2020, 7:28 AM Utkarsh Rai  wrote:
>>
>>> Hi,
>>> I am willing to add tests for clock_nanosleep with CLOCK_MONOTONIC. What
>>> is the standard way of adding test for an already present  API but with
>>> different configuration? For eg. should I add  'psxtmclocknanosleep04/ 05/
>>> 06' in the testsuite?
>>>
>>
>> Yes. That is the pattern.
>>
>> We should try to reduce the count of test programs since on boards with a
>> long reboot time, more tests programs means much more test time (compared
>> to the new test cases alone).
>>
>
> And there is the competing factor that you end up with test executables
> that are overly complex, do not have clean environments for many of the
> test cases, and are too large to fit on many target boards.
>
> The RTEMS Test framework has checks to ensure that the environment is sane
> before a new test case is executed. It decouples the test cases from the
> runner. This could be used to group test cases to test suites depending on
> the target memory size.
>
>
> I know you have seen how long the list is of tests that you can't run on
> many boards.  That's a bad quality attribute
>
> I don't think that tests for clock_nanosleep() will result in big
> executables. The executable size is mostly defined by the feature to be
> tested.
>
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: Test for clock_nanosleep with CLOCK_MONOTONIC (# 3890)

2020-04-09 Thread Sebastian Huber 


On 09/04/2020 15:04, Joel Sherrill wrote:
On Thu, Apr 9, 2020, 7:43 AM Sebastian Huber 
> wrote:


On 09/04/2020 14:40, Joel Sherrill wrote:


On Thu, Apr 9, 2020, 7:28 AM Utkarsh Rai mailto:utkarsh.ra...@gmail.com>> wrote:

Hi,
I am willing to add tests for clock_nanosleep with
CLOCK_MONOTONIC. What is the standard way of adding test for
an already present  API but with different configuration? For
eg. should I add  'psxtmclocknanosleep04/ 05/ 06' in the
testsuite?


Yes. That is the pattern.

We should try to reduce the count of test programs since on boards
with a long reboot time, more tests programs means much more test
time (compared to the new test cases alone).


And there is the competing factor that you end up with test 
executables that are overly complex, do not have clean environments 
for many of the test cases, and are too large to fit on many target 
boards.
The RTEMS Test framework has checks to ensure that the environment is 
sane before a new test case is executed. It decouples the test cases 
from the runner. This could be used to group test cases to test suites 
depending on the target memory size.


I know you have seen how long the list is of tests that you can't run 
on many boards.  That's a bad quality attribute


I don't think that tests for clock_nanosleep() will result in big 
executables. The executable size is mostly defined by the feature to be 
tested.


___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: Test for clock_nanosleep with CLOCK_MONOTONIC (# 3890)

2020-04-09 Thread Joel Sherrill 
On Thu, Apr 9, 2020, 7:43 AM Sebastian Huber <
sebastian.hu...@embedded-brains.de> wrote:

> On 09/04/2020 14:40, Joel Sherrill wrote:
>
> On Thu, Apr 9, 2020, 7:28 AM Utkarsh Rai  wrote:
>
>> Hi,
>> I am willing to add tests for clock_nanosleep with CLOCK_MONOTONIC. What
>> is the standard way of adding test for an already present  API but with
>> different configuration? For eg. should I add  'psxtmclocknanosleep04/ 05/
>> 06' in the testsuite?
>>
>
> Yes. That is the pattern.
>
> We should try to reduce the count of test programs since on boards with a
> long reboot time, more tests programs means much more test time (compared
> to the new test cases alone).
>

And there is the competing factor that you end up with test executables
that are overly complex, do not have clean environments for many of the
test cases, and are too large to fit on many target boards.

I know you have seen how long the list is of tests that you can't run on
many boards.  That's a bad quality attribute

--jo

>
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: Test for clock_nanosleep with CLOCK_MONOTONIC (# 3890)

2020-04-09 Thread Sebastian Huber 

On 09/04/2020 14:40, Joel Sherrill wrote:

On Thu, Apr 9, 2020, 7:28 AM Utkarsh Rai > wrote:


Hi,
I am willing to add tests for clock_nanosleep with
CLOCK_MONOTONIC. What is the standard way of adding test for an
already present  API but with different configuration? For eg.
should I add 'psxtmclocknanosleep04/ 05/ 06' in the testsuite?


Yes. That is the pattern.
We should try to reduce the count of test programs since on boards with 
a long reboot time, more tests programs means much more test time 
(compared to the new test cases alone).
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: Test for clock_nanosleep with CLOCK_MONOTONIC (# 3890)

2020-04-09 Thread Sebastian Huber 

Hello Utkarsh Rai,

On 09/04/2020 14:28, Utkarsh Rai wrote:
I am willing to add tests for clock_nanosleep with CLOCK_MONOTONIC. 
What is the standard way of adding test for an already present API but 
with different configuration? For eg. should I add 
'psxtmclocknanosleep04/ 05/ 06' in the testsuite?


the psxtm* tests are timing/performance tests. Functionality should be 
tested in testsuites/psxtests. I would add new test cases to an existing 
test program, e.g. in separate functions which are called by the main 
thread.


An alternative would be to add the tests using the new RTEMS Test framework:

https://docs.rtems.org/branches/master/eng/test-framework.html#the-rtems-test-framework

The new test framework has also some support for performance tests which 
give more useful results on modern hardware:


https://docs.rtems.org/branches/master/eng/test-framework.html#code-runtime-measurements

___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: Test for clock_nanosleep with CLOCK_MONOTONIC (# 3890)

2020-04-09 Thread Joel Sherrill 
On Thu, Apr 9, 2020, 7:28 AM Utkarsh Rai  wrote:

> Hi,
> I am willing to add tests for clock_nanosleep with CLOCK_MONOTONIC. What
> is the standard way of adding test for an already present  API but with
> different configuration? For eg. should I add  'psxtmclocknanosleep04/ 05/
> 06' in the testsuite?
>

Yes. That is the pattern.

One of the existing clock_nanosleep tests may be a good guide except how it
checks time passage will use CLOCK_MONOTONIC

 --joel

> ___
> devel mailing list
> devel@rtems.org
> http://lists.rtems.org/mailman/listinfo/devel
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel