Re: [Piglit] [PATCH RFC] framework: Add --skip-missing option

2017-08-11 Thread Dylan Baker
Quoting Arkadiusz Hiler (2017-07-31 05:15:26) > On Thu, Jul 27, 2017 at 09:54:13AM -0700, Dylan Baker wrote: > > Quoting Arkadiusz Hiler (2017-07-27 02:24:33) > > > > > > > > > > __all__ = [ > > > > > 'RegexFilter', > > > > > @@ -314,6 +316,9 @@ class TestProfile(object): > > > > >

Re: [Piglit] [PATCH RFC] framework: Add --skip-missing option

2017-07-31 Thread Arkadiusz Hiler
On Thu, Jul 27, 2017 at 09:54:13AM -0700, Dylan Baker wrote: > Quoting Arkadiusz Hiler (2017-07-27 02:24:33) > > > > > > > > __all__ = [ > > > > 'RegexFilter', > > > > @@ -314,6 +316,9 @@ class TestProfile(object): > > > > if self.forced_test_list: > > > > opts = colle

Re: [Piglit] [PATCH RFC] framework: Add --skip-missing option

2017-07-27 Thread Dylan Baker
Quoting Arkadiusz Hiler (2017-07-27 02:24:33) > > > > > > __all__ = [ > > > 'RegexFilter', > > > @@ -314,6 +316,9 @@ class TestProfile(object): > > > if self.forced_test_list: > > > opts = collections.OrderedDict() > > > for n in self.forced_test_list: > >

Re: [Piglit] [PATCH RFC] framework: Add --skip-missing option

2017-07-27 Thread Arkadiusz Hiler
On Wed, Jul 26, 2017 at 11:56:47AM -0700, Dylan Baker wrote: > Quoting Arkadiusz Hiler (2017-07-26 00:46:21) > > Currently, if a test from provided testlist fails to be discovered by > > the framework, piglit blows up with an exception. > > Thank you for keeping the default behavior, it's importan

Re: [Piglit] [PATCH RFC] framework: Add --skip-missing option

2017-07-26 Thread Dylan Baker
Quoting Arkadiusz Hiler (2017-07-26 00:46:21) > Currently, if a test from provided testlist fails to be discovered by > the framework, piglit blows up with an exception. Thank you for keeping the default behavior, it's important to have the "run these and exactly these or fail" option. > > This

[Piglit] [PATCH RFC] framework: Add --skip-missing option

2017-07-26 Thread Arkadiusz Hiler
Currently, if a test from provided testlist fails to be discovered by the framework, piglit blows up with an exception. This is both good - for consistency/early errors - and bad - for handling some CI/automation scenarios (e.g autobisecting the tests). So let's keep the current default, but allo