Re: [tools] tester: Remove hard coded time limits for SIS
On 5/7/2022 8:50 pm, Jiri Gaisler wrote: > > On 7/5/22 12:45, Sebastian Huber wrote: >> >> >> On 05/07/2022 12:31, Chris Johns wrote: >>> On 5 Jul 2022, at 6:23 pm, Sebastian Huber wrote: On 05/07/2022 10:21, Chris Johns wrote: >> On 5/7/2022 4:29 pm, Sebastian Huber wrote: >> On 05/07/2022 08:23, Chris Johns wrote: >>> On 5/7/2022 4:02 pm, Sebastian Huber wrote: On 05/07/2022 07:14, Chris Johns wrote: > On 5/7/2022 2:58 pm, Sebastian Huber wrote: >> On 05/07/2022 03:08, Chris Johns wrote: >>> On 5/7/2022 9:44 am, Joel Sherrill wrote: The limit removed in sis and tsim is the simulated cpu time used. If not using that, the behavior of the tester is to let the simulator run for so much real processor time. Replacing these with a command line argument is probably good but just removing these mean these simulators will just run much longer before being killed. How best to capture the distinction between target run time and host run time? >>> Thank you for the explanation. I was not sure how the option >>> effected >>> things >>> and >>> yes it does matter we have this set correctly. >>> >>> Options can be set in the $HOME/.rtemstesterrc is via the >>> --user-config >>> option. >>> Maybe this can be used to control the time out for specific user >>> tests? This is the only way we currently have and it works ok for hardware type testing with site specific overrides. I think we need an sis.cfg to test for a user defined time limit or use a default. >> I would not make this more complicated than necessary. We have a >> --timeout >> command line option and the default timeout value can be set by *.ini >> files. The >> simulator speed is just a detail similar to running a target at >> 100MHz or >> 1GHz. > It is actually simpler to have this option and to measure time against > the cpu > time. The work loads on SMP hosts with qemu shows simulation timeouts > are > difficult to get right. I don't know what is wrong with the patch. Overruling command line options is just bad. >>> It does not work that way. When simulating the timeout in the tester is >>> a >>> catch >>> all. It may triggered if the simulator locks up. With real hardware it >>> is >>> the >>> timeout but that is a different use case. A simulator timeout is >>> preferred when >>> available. >> Ok, good. Who will fix this? > I am sorry I am not following. The tests have valid times for the default > optimisation. What is broken? What is broken is that the --timeout command line option doesn't work with SIS because it uses hard coded values. >>> The timeout option is correct and your understanding of it’s purpose is >>> wrong. Joining them as you would like would break it. >> >> It would be nice if someone could offer me a way to run tests which exceed >> the >> hard coded SIS timeout values? I will post a patch. I have only tested the erc32-sis bsp. I think it is important we do not join the host's realtime clock to the simulated CPU's realtime clock if we can. The problem with the host's realtime clock controlling the simulated test's time limit is the performance of hosts users differ and what works for one person may not work for someone else. Using extra long timeouts on the host tends to skew the testing time by having the tester wait for the stuck tests to time out and be killed. Note, testing on hardware does not have this problem because the host's and hardware's realtime clocks are the same. > sis accepts several -tlim options. The last one will be the active one. So > rtems-test could add an extra -tlim option at the end of the sis parameters > which would override the default one. I think adding a `sis.cfg` and then letting the user pass the option via the $HOME/.rtemstesterrc or the --user-config option is the best approach. It provides an easy to manage site specific configuration. Chris ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: [tools] tester: Remove hard coded time limits for SIS
On 7/5/22 12:45, Sebastian Huber wrote: On 05/07/2022 12:31, Chris Johns wrote: On 5 Jul 2022, at 6:23 pm, Sebastian Huber wrote: On 05/07/2022 10:21, Chris Johns wrote: On 5/7/2022 4:29 pm, Sebastian Huber wrote: On 05/07/2022 08:23, Chris Johns wrote: On 5/7/2022 4:02 pm, Sebastian Huber wrote: On 05/07/2022 07:14, Chris Johns wrote: On 5/7/2022 2:58 pm, Sebastian Huber wrote: On 05/07/2022 03:08, Chris Johns wrote: On 5/7/2022 9:44 am, Joel Sherrill wrote: The limit removed in sis and tsim is the simulated cpu time used. If not using that, the behavior of the tester is to let the simulator run for so much real processor time. Replacing these with a command line argument is probably good but just removing these mean these simulators will just run much longer before being killed. How best to capture the distinction between target run time and host run time? Thank you for the explanation. I was not sure how the option effected things and yes it does matter we have this set correctly. Options can be set in the $HOME/.rtemstesterrc is via the --user-config option. Maybe this can be used to control the time out for specific user tests? I would not make this more complicated than necessary. We have a --timeout command line option and the default timeout value can be set by *.ini files. The simulator speed is just a detail similar to running a target at 100MHz or 1GHz. It is actually simpler to have this option and to measure time against the cpu time. The work loads on SMP hosts with qemu shows simulation timeouts are difficult to get right. I don't know what is wrong with the patch. Overruling command line options is just bad. It does not work that way. When simulating the timeout in the tester is a catch all. It may triggered if the simulator locks up. With real hardware it is the timeout but that is a different use case. A simulator timeout is preferred when available. Ok, good. Who will fix this? I am sorry I am not following. The tests have valid times for the default optimisation. What is broken? What is broken is that the --timeout command line option doesn't work with SIS because it uses hard coded values. The timeout option is correct and your understanding of it’s purpose is wrong. Joining them as you would like would break it. It would be nice if someone could offer me a way to run tests which exceed the hard coded SIS timeout values? sis accepts several -tlim options. The last one will be the active one. So rtems-test could add an extra -tlim option at the end of the sis parameters which would override the default one. ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: [tools] tester: Remove hard coded time limits for SIS
On 05/07/2022 12:31, Chris Johns wrote: On 5 Jul 2022, at 6:23 pm, Sebastian Huber wrote: On 05/07/2022 10:21, Chris Johns wrote: On 5/7/2022 4:29 pm, Sebastian Huber wrote: On 05/07/2022 08:23, Chris Johns wrote: On 5/7/2022 4:02 pm, Sebastian Huber wrote: On 05/07/2022 07:14, Chris Johns wrote: On 5/7/2022 2:58 pm, Sebastian Huber wrote: On 05/07/2022 03:08, Chris Johns wrote: On 5/7/2022 9:44 am, Joel Sherrill wrote: The limit removed in sis and tsim is the simulated cpu time used. If not using that, the behavior of the tester is to let the simulator run for so much real processor time. Replacing these with a command line argument is probably good but just removing these mean these simulators will just run much longer before being killed. How best to capture the distinction between target run time and host run time? Thank you for the explanation. I was not sure how the option effected things and yes it does matter we have this set correctly. Options can be set in the $HOME/.rtemstesterrc is via the --user-config option. Maybe this can be used to control the time out for specific user tests? I would not make this more complicated than necessary. We have a --timeout command line option and the default timeout value can be set by *.ini files. The simulator speed is just a detail similar to running a target at 100MHz or 1GHz. It is actually simpler to have this option and to measure time against the cpu time. The work loads on SMP hosts with qemu shows simulation timeouts are difficult to get right. I don't know what is wrong with the patch. Overruling command line options is just bad. It does not work that way. When simulating the timeout in the tester is a catch all. It may triggered if the simulator locks up. With real hardware it is the timeout but that is a different use case. A simulator timeout is preferred when available. Ok, good. Who will fix this? I am sorry I am not following. The tests have valid times for the default optimisation. What is broken? What is broken is that the --timeout command line option doesn't work with SIS because it uses hard coded values. The timeout option is correct and your understanding of it’s purpose is wrong. Joining them as you would like would break it. It would be nice if someone could offer me a way to run tests which exceed the hard coded SIS timeout values? -- embedded brains GmbH Herr Sebastian HUBER Dornierstr. 4 82178 Puchheim Germany email: sebastian.hu...@embedded-brains.de phone: +49-89-18 94 741 - 16 fax: +49-89-18 94 741 - 08 Registergericht: Amtsgericht München Registernummer: HRB 157899 Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler Unsere Datenschutzerklärung finden Sie hier: https://embedded-brains.de/datenschutzerklaerung/ ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: [tools] tester: Remove hard coded time limits for SIS
On 7/5/22 12:31, Chris Johns wrote: On 5 Jul 2022, at 6:23 pm, Sebastian Huber wrote: On 05/07/2022 10:21, Chris Johns wrote: On 5/7/2022 4:29 pm, Sebastian Huber wrote: On 05/07/2022 08:23, Chris Johns wrote: On 5/7/2022 4:02 pm, Sebastian Huber wrote: On 05/07/2022 07:14, Chris Johns wrote: On 5/7/2022 2:58 pm, Sebastian Huber wrote: On 05/07/2022 03:08, Chris Johns wrote: On 5/7/2022 9:44 am, Joel Sherrill wrote: The limit removed in sis and tsim is the simulated cpu time used. If not using that, the behavior of the tester is to let the simulator run for so much real processor time. Replacing these with a command line argument is probably good but just removing these mean these simulators will just run much longer before being killed. How best to capture the distinction between target run time and host run time? Thank you for the explanation. I was not sure how the option effected things and yes it does matter we have this set correctly. Options can be set in the $HOME/.rtemstesterrc is via the --user-config option. Maybe this can be used to control the time out for specific user tests? I would not make this more complicated than necessary. We have a --timeout command line option and the default timeout value can be set by *.ini files. The simulator speed is just a detail similar to running a target at 100MHz or 1GHz. It is actually simpler to have this option and to measure time against the cpu time. The work loads on SMP hosts with qemu shows simulation timeouts are difficult to get right. I don't know what is wrong with the patch. Overruling command line options is just bad. It does not work that way. When simulating the timeout in the tester is a catch all. It may triggered if the simulator locks up. With real hardware it is the timeout but that is a different use case. A simulator timeout is preferred when available. Ok, good. Who will fix this? I am sorry I am not following. The tests have valid times for the default optimisation. What is broken? What is broken is that the --timeout command line option doesn't work with SIS because it uses hard coded values. The timeout option is correct and your understanding of it’s purpose is wrong. Joining them as you would like would break it. I think that Sebastian has a point, a hard-coded value in the sis script prevents changing the time-out value from rtems-test. The timeout value in sis is in simulated CPU time, not host time. I am not sure how it works on the other simulators. If you would prefer host time time-outs, let me know and I will modify sis for this. Jiri. ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: [tools] tester: Remove hard coded time limits for SIS
> On 5 Jul 2022, at 6:23 pm, Sebastian Huber > wrote: > > On 05/07/2022 10:21, Chris Johns wrote: >>> On 5/7/2022 4:29 pm, Sebastian Huber wrote: >>> On 05/07/2022 08:23, Chris Johns wrote: On 5/7/2022 4:02 pm, Sebastian Huber wrote: > On 05/07/2022 07:14, Chris Johns wrote: >> On 5/7/2022 2:58 pm, Sebastian Huber wrote: >>> On 05/07/2022 03:08, Chris Johns wrote: On 5/7/2022 9:44 am, Joel Sherrill wrote: > The limit removed in sis and tsim is the simulated cpu time used. If > not > using > that, the behavior of the tester is to let the simulator run for so > much > real > processor time. > > Replacing these with a command line argument is probably good but just > removing > these mean these simulators will just run much longer before being > killed. > > How best to capture the distinction between target run time and host > run > time? Thank you for the explanation. I was not sure how the option effected things and yes it does matter we have this set correctly. Options can be set in the $HOME/.rtemstesterrc is via the --user-config option. Maybe this can be used to control the time out for specific user tests? >>> I would not make this more complicated than necessary. We have a >>> --timeout >>> command line option and the default timeout value can be set by *.ini >>> files. The >>> simulator speed is just a detail similar to running a target at 100MHz >>> or >>> 1GHz. >> It is actually simpler to have this option and to measure time against >> the cpu >> time. The work loads on SMP hosts with qemu shows simulation timeouts are >> difficult to get right. > I don't know what is wrong with the patch. Overruling command line > options is > just bad. It does not work that way. When simulating the timeout in the tester is a catch all. It may triggered if the simulator locks up. With real hardware it is the timeout but that is a different use case. A simulator timeout is preferred when available. >>> >>> Ok, good. Who will fix this? >> I am sorry I am not following. The tests have valid times for the default >> optimisation. What is broken? > > What is broken is that the --timeout command line option doesn't work with > SIS because it uses hard coded values. The timeout option is correct and your understanding of it’s purpose is wrong. Joining them as you would like would break it. Chris ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: [tools] tester: Remove hard coded time limits for SIS
On 05/07/2022 10:21, Chris Johns wrote: On 5/7/2022 4:29 pm, Sebastian Huber wrote: On 05/07/2022 08:23, Chris Johns wrote: On 5/7/2022 4:02 pm, Sebastian Huber wrote: On 05/07/2022 07:14, Chris Johns wrote: On 5/7/2022 2:58 pm, Sebastian Huber wrote: On 05/07/2022 03:08, Chris Johns wrote: On 5/7/2022 9:44 am, Joel Sherrill wrote: The limit removed in sis and tsim is the simulated cpu time used. If not using that, the behavior of the tester is to let the simulator run for so much real processor time. Replacing these with a command line argument is probably good but just removing these mean these simulators will just run much longer before being killed. How best to capture the distinction between target run time and host run time? Thank you for the explanation. I was not sure how the option effected things and yes it does matter we have this set correctly. Options can be set in the $HOME/.rtemstesterrc is via the --user-config option. Maybe this can be used to control the time out for specific user tests? I would not make this more complicated than necessary. We have a --timeout command line option and the default timeout value can be set by *.ini files. The simulator speed is just a detail similar to running a target at 100MHz or 1GHz. It is actually simpler to have this option and to measure time against the cpu time. The work loads on SMP hosts with qemu shows simulation timeouts are difficult to get right. I don't know what is wrong with the patch. Overruling command line options is just bad. It does not work that way. When simulating the timeout in the tester is a catch all. It may triggered if the simulator locks up. With real hardware it is the timeout but that is a different use case. A simulator timeout is preferred when available. Ok, good. Who will fix this? I am sorry I am not following. The tests have valid times for the default optimisation. What is broken? What is broken is that the --timeout command line option doesn't work with SIS because it uses hard coded values. -- embedded brains GmbH Herr Sebastian HUBER Dornierstr. 4 82178 Puchheim Germany email: sebastian.hu...@embedded-brains.de phone: +49-89-18 94 741 - 16 fax: +49-89-18 94 741 - 08 Registergericht: Amtsgericht München Registernummer: HRB 157899 Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler Unsere Datenschutzerklärung finden Sie hier: https://embedded-brains.de/datenschutzerklaerung/ ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: [tools] tester: Remove hard coded time limits for SIS
On 5/7/2022 4:29 pm, Sebastian Huber wrote: > On 05/07/2022 08:23, Chris Johns wrote: >> On 5/7/2022 4:02 pm, Sebastian Huber wrote: >>> On 05/07/2022 07:14, Chris Johns wrote: On 5/7/2022 2:58 pm, Sebastian Huber wrote: > On 05/07/2022 03:08, Chris Johns wrote: >> On 5/7/2022 9:44 am, Joel Sherrill wrote: >>> The limit removed in sis and tsim is the simulated cpu time used. If not >>> using >>> that, the behavior of the tester is to let the simulator run for so much >>> real >>> processor time. >>> >>> Replacing these with a command line argument is probably good but just >>> removing >>> these mean these simulators will just run much longer before being >>> killed. >>> >>> How best to capture the distinction between target run time and host run >>> time? >> Thank you for the explanation. I was not sure how the option effected >> things >> and >> yes it does matter we have this set correctly. >> >> Options can be set in the $HOME/.rtemstesterrc is via the --user-config >> option. >> Maybe this can be used to control the time out for specific user tests? > I would not make this more complicated than necessary. We have a --timeout > command line option and the default timeout value can be set by *.ini > files. The > simulator speed is just a detail similar to running a target at 100MHz or > 1GHz. It is actually simpler to have this option and to measure time against the cpu time. The work loads on SMP hosts with qemu shows simulation timeouts are difficult to get right. >>> I don't know what is wrong with the patch. Overruling command line options >>> is >>> just bad. >> It does not work that way. When simulating the timeout in the tester is a >> catch >> all. It may triggered if the simulator locks up. With real hardware it is the >> timeout but that is a different use case. A simulator timeout is preferred >> when >> available. > > Ok, good. Who will fix this? I am sorry I am not following. The tests have valid times for the default optimisation. What is broken? Chris ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: [tools] tester: Remove hard coded time limits for SIS
On 05/07/2022 08:23, Chris Johns wrote: On 5/7/2022 4:02 pm, Sebastian Huber wrote: On 05/07/2022 07:14, Chris Johns wrote: On 5/7/2022 2:58 pm, Sebastian Huber wrote: On 05/07/2022 03:08, Chris Johns wrote: On 5/7/2022 9:44 am, Joel Sherrill wrote: The limit removed in sis and tsim is the simulated cpu time used. If not using that, the behavior of the tester is to let the simulator run for so much real processor time. Replacing these with a command line argument is probably good but just removing these mean these simulators will just run much longer before being killed. How best to capture the distinction between target run time and host run time? Thank you for the explanation. I was not sure how the option effected things and yes it does matter we have this set correctly. Options can be set in the $HOME/.rtemstesterrc is via the --user-config option. Maybe this can be used to control the time out for specific user tests? I would not make this more complicated than necessary. We have a --timeout command line option and the default timeout value can be set by *.ini files. The simulator speed is just a detail similar to running a target at 100MHz or 1GHz. It is actually simpler to have this option and to measure time against the cpu time. The work loads on SMP hosts with qemu shows simulation timeouts are difficult to get right. I don't know what is wrong with the patch. Overruling command line options is just bad. It does not work that way. When simulating the timeout in the tester is a catch all. It may triggered if the simulator locks up. With real hardware it is the timeout but that is a different use case. A simulator timeout is preferred when available. Ok, good. Who will fix this? -- embedded brains GmbH Herr Sebastian HUBER Dornierstr. 4 82178 Puchheim Germany email: sebastian.hu...@embedded-brains.de phone: +49-89-18 94 741 - 16 fax: +49-89-18 94 741 - 08 Registergericht: Amtsgericht München Registernummer: HRB 157899 Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler Unsere Datenschutzerklärung finden Sie hier: https://embedded-brains.de/datenschutzerklaerung/ ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: [tools] tester: Remove hard coded time limits for SIS
On 5/7/2022 4:02 pm, Sebastian Huber wrote: > On 05/07/2022 07:14, Chris Johns wrote: >> On 5/7/2022 2:58 pm, Sebastian Huber wrote: >>> On 05/07/2022 03:08, Chris Johns wrote: On 5/7/2022 9:44 am, Joel Sherrill wrote: > The limit removed in sis and tsim is the simulated cpu time used. If not > using > that, the behavior of the tester is to let the simulator run for so much > real > processor time. > > Replacing these with a command line argument is probably good but just > removing > these mean these simulators will just run much longer before being killed. > > How best to capture the distinction between target run time and host run > time? Thank you for the explanation. I was not sure how the option effected things and yes it does matter we have this set correctly. Options can be set in the $HOME/.rtemstesterrc is via the --user-config option. Maybe this can be used to control the time out for specific user tests? >>> >>> I would not make this more complicated than necessary. We have a --timeout >>> command line option and the default timeout value can be set by *.ini >>> files. The >>> simulator speed is just a detail similar to running a target at 100MHz or >>> 1GHz. >> >> It is actually simpler to have this option and to measure time against the >> cpu >> time. The work loads on SMP hosts with qemu shows simulation timeouts are >> difficult to get right. > > I don't know what is wrong with the patch. Overruling command line options is > just bad. It does not work that way. When simulating the timeout in the tester is a catch all. It may triggered if the simulator locks up. With real hardware it is the timeout but that is a different use case. A simulator timeout is preferred when available. Sebastian, are some of the standard testsuite test's failing because of this setting? >>> >>> Yes, with -O0 and code coverage enabled the tests run longer than usual. >>> >> >> Looks to me like the timeout may need to be adjusted? > > One of the performance tests needed 14 minutes on a fast computer. I would > keep > the timeouts as is for now. There is more work to do before the gcov coverage > is > generally usable. I think it should stay and a user config used to control the setting but I will let Joel or Jiri make the call. Chtis ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: [tools] tester: Remove hard coded time limits for SIS
On 05/07/2022 07:14, Chris Johns wrote: On 5/7/2022 2:58 pm, Sebastian Huber wrote: On 05/07/2022 03:08, Chris Johns wrote: On 5/7/2022 9:44 am, Joel Sherrill wrote: The limit removed in sis and tsim is the simulated cpu time used. If not using that, the behavior of the tester is to let the simulator run for so much real processor time. Replacing these with a command line argument is probably good but just removing these mean these simulators will just run much longer before being killed. How best to capture the distinction between target run time and host run time? Thank you for the explanation. I was not sure how the option effected things and yes it does matter we have this set correctly. Options can be set in the $HOME/.rtemstesterrc is via the --user-config option. Maybe this can be used to control the time out for specific user tests? I would not make this more complicated than necessary. We have a --timeout command line option and the default timeout value can be set by *.ini files. The simulator speed is just a detail similar to running a target at 100MHz or 1GHz. It is actually simpler to have this option and to measure time against the cpu time. The work loads on SMP hosts with qemu shows simulation timeouts are difficult to get right. I don't know what is wrong with the patch. Overruling command line options is just bad. Sebastian, are some of the standard testsuite test's failing because of this setting? Yes, with -O0 and code coverage enabled the tests run longer than usual. Looks to me like the timeout may need to be adjusted? One of the performance tests needed 14 minutes on a fast computer. I would keep the timeouts as is for now. There is more work to do before the gcov coverage is generally usable. -- embedded brains GmbH Herr Sebastian HUBER Dornierstr. 4 82178 Puchheim Germany email: sebastian.hu...@embedded-brains.de phone: +49-89-18 94 741 - 16 fax: +49-89-18 94 741 - 08 Registergericht: Amtsgericht München Registernummer: HRB 157899 Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler Unsere Datenschutzerklärung finden Sie hier: https://embedded-brains.de/datenschutzerklaerung/ ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: [tools] tester: Remove hard coded time limits for SIS
On 5/7/2022 2:58 pm, Sebastian Huber wrote: > On 05/07/2022 03:08, Chris Johns wrote: >> On 5/7/2022 9:44 am, Joel Sherrill wrote: >>> The limit removed in sis and tsim is the simulated cpu time used. If not >>> using >>> that, the behavior of the tester is to let the simulator run for so much >>> real >>> processor time. >>> >>> Replacing these with a command line argument is probably good but just >>> removing >>> these mean these simulators will just run much longer before being killed. >>> >>> How best to capture the distinction between target run time and host run >>> time? >> >> Thank you for the explanation. I was not sure how the option effected things >> and >> yes it does matter we have this set correctly. >> >> Options can be set in the $HOME/.rtemstesterrc is via the --user-config >> option. >> Maybe this can be used to control the time out for specific user tests? > > I would not make this more complicated than necessary. We have a --timeout > command line option and the default timeout value can be set by *.ini files. > The > simulator speed is just a detail similar to running a target at 100MHz or > 1GHz. It is actually simpler to have this option and to measure time against the cpu time. The work loads on SMP hosts with qemu shows simulation timeouts are difficult to get right. >> Sebastian, are some of the standard testsuite test's failing because of this >> setting? > > Yes, with -O0 and code coverage enabled the tests run longer than usual. > Looks to me like the timeout may need to be adjusted? Chris ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: [tools] tester: Remove hard coded time limits for SIS
On 05/07/2022 03:08, Chris Johns wrote: On 5/7/2022 9:44 am, Joel Sherrill wrote: The limit removed in sis and tsim is the simulated cpu time used. If not using that, the behavior of the tester is to let the simulator run for so much real processor time. Replacing these with a command line argument is probably good but just removing these mean these simulators will just run much longer before being killed. How best to capture the distinction between target run time and host run time? Thank you for the explanation. I was not sure how the option effected things and yes it does matter we have this set correctly. Options can be set in the $HOME/.rtemstesterrc is via the --user-config option. Maybe this can be used to control the time out for specific user tests? I would not make this more complicated than necessary. We have a --timeout command line option and the default timeout value can be set by *.ini files. The simulator speed is just a detail similar to running a target at 100MHz or 1GHz. Sebastian, are some of the standard testsuite test's failing because of this setting? Yes, with -O0 and code coverage enabled the tests run longer than usual. -- embedded brains GmbH Herr Sebastian HUBER Dornierstr. 4 82178 Puchheim Germany email: sebastian.hu...@embedded-brains.de phone: +49-89-18 94 741 - 16 fax: +49-89-18 94 741 - 08 Registergericht: Amtsgericht München Registernummer: HRB 157899 Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler Unsere Datenschutzerklärung finden Sie hier: https://embedded-brains.de/datenschutzerklaerung/ ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: [tools] tester: Remove hard coded time limits for SIS
On 5/7/2022 9:44 am, Joel Sherrill wrote: > The limit removed in sis and tsim is the simulated cpu time used. If not using > that, the behavior of the tester is to let the simulator run for so much real > processor time. > > Replacing these with a command line argument is probably good but just > removing > these mean these simulators will just run much longer before being killed. > > How best to capture the distinction between target run time and host run time? Thank you for the explanation. I was not sure how the option effected things and yes it does matter we have this set correctly. Options can be set in the $HOME/.rtemstesterrc is via the --user-config option. Maybe this can be used to control the time out for specific user tests? Sebastian, are some of the standard testsuite test's failing because of this setting? Chris ___ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
Re: [tools] tester: Remove hard coded time limits for SIS
The limit removed in sis and tsim is the simulated cpu time used. If not
using that, the behavior of the tester is to let the simulator run for so
much real processor time.
Replacing these with a command line argument is probably good but just
removing these mean these simulators will just run much longer before being
killed.
How best to capture the distinction between target run time and host run
time?
On Mon, Jul 4, 2022, 5:38 PM Chris Johns wrote:
> OK
>
> Chris
>
> On 4/7/2022 6:52 pm, Sebastian Huber wrote:
> > Remove the hard coded time limits in the SIS configurations which would
> > overrule the general tester settings (for example the--timeout command
> line
> > option).
> > ---
> > tester/rtems/testing/bsps/erc32-sis.ini | 2 +-
> > tester/rtems/testing/bsps/gr740-sis.ini | 2 +-
> > tester/rtems/testing/bsps/griscv-sis-cov.ini | 2 +-
> > tester/rtems/testing/bsps/griscv-sis.ini | 2 +-
> > tester/rtems/testing/bsps/leon2-sis.ini | 2 +-
> > tester/rtems/testing/bsps/leon3-sis-cov.ini | 2 +-
> > tester/rtems/testing/bsps/leon3-sis.ini | 2 +-
> > 7 files changed, 7 insertions(+), 7 deletions(-)
> >
> > diff --git a/tester/rtems/testing/bsps/erc32-sis.ini
> b/tester/rtems/testing/bsps/erc32-sis.ini
> > index fca2122..fe1eaa0 100644
> > --- a/tester/rtems/testing/bsps/erc32-sis.ini
> > +++ b/tester/rtems/testing/bsps/erc32-sis.ini
> > @@ -36,4 +36,4 @@ bsp = erc32
> > arch = sparc
> > tester = %{_rtscripts}/run.cfg
> > bsp_run_cmd = %{rtems_tools}/%{bsp_arch}-rtems%{rtems_version}-sis
> > -bsp_run_opts = -nouartrx -r -tlim 600 s
> > +bsp_run_opts = -nouartrx -r
> > diff --git a/tester/rtems/testing/bsps/gr740-sis.ini
> b/tester/rtems/testing/bsps/gr740-sis.ini
> > index b71048c..5f8dea7 100644
> > --- a/tester/rtems/testing/bsps/gr740-sis.ini
> > +++ b/tester/rtems/testing/bsps/gr740-sis.ini
> > @@ -35,4 +35,4 @@ bsp = gr740
> > arch = sparc
> > tester = %{_rtscripts}/run.cfg
> > bsp_run_cmd = %{rtems_tools}/%{bsp_arch}-rtems%{rtems_version}-sis
> > -bsp_run_opts = -gr740 -nouartrx -r -tlim 200 s -m 4
> > +bsp_run_opts = -gr740 -nouartrx -r -m 4
> > diff --git a/tester/rtems/testing/bsps/griscv-sis-cov.ini
> b/tester/rtems/testing/bsps/griscv-sis-cov.ini
> > index 9ab37a8..e356a4d 100644
> > --- a/tester/rtems/testing/bsps/griscv-sis-cov.ini
> > +++ b/tester/rtems/testing/bsps/griscv-sis-cov.ini
> > @@ -36,5 +36,5 @@ bsp= griscv-sis
> > arch = riscv
> > tester = %{_rtscripts}/run.cfg
> > bsp_run_cmd= %{rtems_tools}/%{bsp_arch}-rtems%{rtems_version}-sis
> > -bsp_run_opts = -nouartrx -r -tlim 300 s -m 4 -cov
> > +bsp_run_opts = -nouartrx -r -m 4 -cov
> > bsp_covoar_cmd = -S %{bsp_symbol_path} -E %{cov_explanations} -f TSIM
> > diff --git a/tester/rtems/testing/bsps/griscv-sis.ini
> b/tester/rtems/testing/bsps/griscv-sis.ini
> > index b21cba1..8676ea5 100644
> > --- a/tester/rtems/testing/bsps/griscv-sis.ini
> > +++ b/tester/rtems/testing/bsps/griscv-sis.ini
> > @@ -36,4 +36,4 @@ bsp = griscv
> > arch = riscv
> > tester = %{_rtscripts}/run.cfg
> > bsp_run_cmd = %{rtems_tools}/%{bsp_arch}-rtems%{rtems_version}-sis
> > -bsp_run_opts = -nouartrx -r -tlim 300 s -m 4
> > +bsp_run_opts = -nouartrx -r -m 4
> > diff --git a/tester/rtems/testing/bsps/leon2-sis.ini
> b/tester/rtems/testing/bsps/leon2-sis.ini
> > index 61205ad..844665c 100644
> > --- a/tester/rtems/testing/bsps/leon2-sis.ini
> > +++ b/tester/rtems/testing/bsps/leon2-sis.ini
> > @@ -36,4 +36,4 @@ bsp = leon2
> > arch = sparc
> > tester = %{_rtscripts}/run.cfg
> > bsp_run_cmd = %{rtems_tools}/%{bsp_arch}-rtems%{rtems_version}-sis
> > -bsp_run_opts = -leon2 -nouartrx -r -tlim 200 s
> > +bsp_run_opts = -leon2 -nouartrx -r
> > diff --git a/tester/rtems/testing/bsps/leon3-sis-cov.ini
> b/tester/rtems/testing/bsps/leon3-sis-cov.ini
> > index d8ffe28..59d415f 100644
> > --- a/tester/rtems/testing/bsps/leon3-sis-cov.ini
> > +++ b/tester/rtems/testing/bsps/leon3-sis-cov.ini
> > @@ -36,5 +36,5 @@ bsp= leon3-sis
> > arch = sparc
> > tester = %{_rtscripts}/run.cfg
> > bsp_run_cmd= %{rtems_tools}/%{bsp_arch}-rtems%{rtems_version}-sis
> > -bsp_run_opts = -leon3 -nouartrx -r -tlim 200 s -cov
> > +bsp_run_opts = -leon3 -nouartrx -r -cov
> > bsp_covoar_cmd = -S %{bsp_symbol_path} -E %{cov_explanations} -f TSIM
> > diff --git a/tester/rtems/testing/bsps/leon3-sis.ini
> b/tester/rtems/testing/bsps/leon3-sis.ini
> > index 2f933a7..afa9082 100644
> > --- a/tester/rtems/testing/bsps/leon3-sis.ini
> > +++ b/tester/rtems/testing/bsps/leon3-sis.ini
> > @@ -36,4 +36,4 @@ bsp = leon3
> > arch = sparc
> > tester = %{_rtscripts}/run.cfg
> > bsp_run_cmd = %{rtems_tools}/%{bsp_arch}-rtems%{rtems_version}-sis
> > -bsp_run_opts = -leon3 -nouartrx -r -tlim 200 s -m 4
> > +bsp_run_opts = -leon3 -nouartrx -r -m 4
>
Re: [tools] tester: Remove hard coded time limits for SIS
OK
Chris
On 4/7/2022 6:52 pm, Sebastian Huber wrote:
> Remove the hard coded time limits in the SIS configurations which would
> overrule the general tester settings (for example the--timeout command line
> option).
> ---
> tester/rtems/testing/bsps/erc32-sis.ini | 2 +-
> tester/rtems/testing/bsps/gr740-sis.ini | 2 +-
> tester/rtems/testing/bsps/griscv-sis-cov.ini | 2 +-
> tester/rtems/testing/bsps/griscv-sis.ini | 2 +-
> tester/rtems/testing/bsps/leon2-sis.ini | 2 +-
> tester/rtems/testing/bsps/leon3-sis-cov.ini | 2 +-
> tester/rtems/testing/bsps/leon3-sis.ini | 2 +-
> 7 files changed, 7 insertions(+), 7 deletions(-)
>
> diff --git a/tester/rtems/testing/bsps/erc32-sis.ini
> b/tester/rtems/testing/bsps/erc32-sis.ini
> index fca2122..fe1eaa0 100644
> --- a/tester/rtems/testing/bsps/erc32-sis.ini
> +++ b/tester/rtems/testing/bsps/erc32-sis.ini
> @@ -36,4 +36,4 @@ bsp = erc32
> arch = sparc
> tester = %{_rtscripts}/run.cfg
> bsp_run_cmd = %{rtems_tools}/%{bsp_arch}-rtems%{rtems_version}-sis
> -bsp_run_opts = -nouartrx -r -tlim 600 s
> +bsp_run_opts = -nouartrx -r
> diff --git a/tester/rtems/testing/bsps/gr740-sis.ini
> b/tester/rtems/testing/bsps/gr740-sis.ini
> index b71048c..5f8dea7 100644
> --- a/tester/rtems/testing/bsps/gr740-sis.ini
> +++ b/tester/rtems/testing/bsps/gr740-sis.ini
> @@ -35,4 +35,4 @@ bsp = gr740
> arch = sparc
> tester = %{_rtscripts}/run.cfg
> bsp_run_cmd = %{rtems_tools}/%{bsp_arch}-rtems%{rtems_version}-sis
> -bsp_run_opts = -gr740 -nouartrx -r -tlim 200 s -m 4
> +bsp_run_opts = -gr740 -nouartrx -r -m 4
> diff --git a/tester/rtems/testing/bsps/griscv-sis-cov.ini
> b/tester/rtems/testing/bsps/griscv-sis-cov.ini
> index 9ab37a8..e356a4d 100644
> --- a/tester/rtems/testing/bsps/griscv-sis-cov.ini
> +++ b/tester/rtems/testing/bsps/griscv-sis-cov.ini
> @@ -36,5 +36,5 @@ bsp= griscv-sis
> arch = riscv
> tester = %{_rtscripts}/run.cfg
> bsp_run_cmd= %{rtems_tools}/%{bsp_arch}-rtems%{rtems_version}-sis
> -bsp_run_opts = -nouartrx -r -tlim 300 s -m 4 -cov
> +bsp_run_opts = -nouartrx -r -m 4 -cov
> bsp_covoar_cmd = -S %{bsp_symbol_path} -E %{cov_explanations} -f TSIM
> diff --git a/tester/rtems/testing/bsps/griscv-sis.ini
> b/tester/rtems/testing/bsps/griscv-sis.ini
> index b21cba1..8676ea5 100644
> --- a/tester/rtems/testing/bsps/griscv-sis.ini
> +++ b/tester/rtems/testing/bsps/griscv-sis.ini
> @@ -36,4 +36,4 @@ bsp = griscv
> arch = riscv
> tester = %{_rtscripts}/run.cfg
> bsp_run_cmd = %{rtems_tools}/%{bsp_arch}-rtems%{rtems_version}-sis
> -bsp_run_opts = -nouartrx -r -tlim 300 s -m 4
> +bsp_run_opts = -nouartrx -r -m 4
> diff --git a/tester/rtems/testing/bsps/leon2-sis.ini
> b/tester/rtems/testing/bsps/leon2-sis.ini
> index 61205ad..844665c 100644
> --- a/tester/rtems/testing/bsps/leon2-sis.ini
> +++ b/tester/rtems/testing/bsps/leon2-sis.ini
> @@ -36,4 +36,4 @@ bsp = leon2
> arch = sparc
> tester = %{_rtscripts}/run.cfg
> bsp_run_cmd = %{rtems_tools}/%{bsp_arch}-rtems%{rtems_version}-sis
> -bsp_run_opts = -leon2 -nouartrx -r -tlim 200 s
> +bsp_run_opts = -leon2 -nouartrx -r
> diff --git a/tester/rtems/testing/bsps/leon3-sis-cov.ini
> b/tester/rtems/testing/bsps/leon3-sis-cov.ini
> index d8ffe28..59d415f 100644
> --- a/tester/rtems/testing/bsps/leon3-sis-cov.ini
> +++ b/tester/rtems/testing/bsps/leon3-sis-cov.ini
> @@ -36,5 +36,5 @@ bsp= leon3-sis
> arch = sparc
> tester = %{_rtscripts}/run.cfg
> bsp_run_cmd= %{rtems_tools}/%{bsp_arch}-rtems%{rtems_version}-sis
> -bsp_run_opts = -leon3 -nouartrx -r -tlim 200 s -cov
> +bsp_run_opts = -leon3 -nouartrx -r -cov
> bsp_covoar_cmd = -S %{bsp_symbol_path} -E %{cov_explanations} -f TSIM
> diff --git a/tester/rtems/testing/bsps/leon3-sis.ini
> b/tester/rtems/testing/bsps/leon3-sis.ini
> index 2f933a7..afa9082 100644
> --- a/tester/rtems/testing/bsps/leon3-sis.ini
> +++ b/tester/rtems/testing/bsps/leon3-sis.ini
> @@ -36,4 +36,4 @@ bsp = leon3
> arch = sparc
> tester = %{_rtscripts}/run.cfg
> bsp_run_cmd = %{rtems_tools}/%{bsp_arch}-rtems%{rtems_version}-sis
> -bsp_run_opts = -leon3 -nouartrx -r -tlim 200 s -m 4
> +bsp_run_opts = -leon3 -nouartrx -r -m 4
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel
