Re: [Piglit] [PATCH v3] khr_texture_compression_astc: Enable subtest reports

[Nanley Chery]

On Tue, Dec 01, 2015 at 03:02:08PM -0800, Dylan Baker wrote:
> On Tue, Dec 01, 2015 at 02:14:04PM -0500, Ilia Mirkin wrote:
> > On Tue, Dec 1, 2015 at 1:58 PM, Nanley Chery  wrote:
> > > On Tue, Dec 01, 2015 at 10:21:23AM -0500, Ilia Mirkin wrote:
> > >> On Tue, Dec 1, 2015 at 1:31 AM, Nanley Chery  
> > >> wrote:
> > >> > From: Nanley Chery 
> > >> >
> > >> > Enable Jenkins to report the result of each individual subtest
> > >> > for the array and miptree ASTC tests. Modify the miptree test
> > >> > to only run one subset of ASTC formats at a time.
> > >> >
> > >> > v2. Modify miptree test to only check the given subtest.
> > >> > Use the -subtest option to run a specific subtest.
> > >> > v3. Indent function arguments and misc cleanups (Ilia).
> > >> > Initialize texture objects to 0 (Ilia).
> > >> >
> > >> > Signed-off-by: Nanley Chery 
> > >> > ---
> > >> >  tests/all.py   | 12 ++--
> > >> >  .../khr_compressed_astc-miptree.c  | 84 
> > >> > +-
> > >> >  2 files changed, 42 insertions(+), 54 deletions(-)
> > >> >
> > >> > diff --git a/tests/all.py b/tests/all.py
> > >> > index cfafa71..29ee6a9 100644
> > >> > --- a/tests/all.py
> > >> > +++ b/tests/all.py
> > >> > @@ -4223,12 +4223,16 @@ with profile.group_manager(
> > >> >   PiglitGLTest,
> > >> >   grouptools.join('spec', 'khr_texture_compression_astc')) as 
> > >> > g:
> > >> >  g(['arb_texture_compression-invalid-formats', 'astc'], 'invalid 
> > >> > formats')
> > >> > -g(['khr_compressed_astc-array_gl'], 'array-gl')
> > >> > -g(['khr_compressed_astc-array_gles3'], 'array-gles')
> > >> >  g(['khr_compressed_astc-basic_gl'], 'basic-gl')
> > >> >  g(['khr_compressed_astc-basic_gles2'], 'basic-gles')
> > >> > -g(['khr_compressed_astc-miptree_gl'], 'miptree-gl')
> > >> > -g(['khr_compressed_astc-miptree_gles2'], 'miptree-gles')
> > >> > +
> > >> > +for subtest in ('odd', 'even'):
> > >> > +g(['khr_compressed_astc-array_gl', '-subtest', subtest])
> > >> > +g(['khr_compressed_astc-array_gles3', '-subtest', subtest])
> > >> > +
> > >> > +for subtest in ('ldr', 'srgb', 'hdr'):
> > >> > +g(['khr_compressed_astc-miptree_gl', '-subtest', subtest])
> > >> > +g(['khr_compressed_astc-miptree_gles2', '-subtest', subtest])
> > >>
> > >> I still don't understand why you need this... what's wrong with the
> > >> current way? All the subtests should get run, and be properly
> > >> reported, without needing to be included one-by-one here. Does that
> > >> not happen?
> > >
> > > This enables the Jenkins CI system to report which specific subtest is
> > > failing - without having to inspect the error message to determine the
> > 
> > Wow, that seems like a pretty big failing for the jenkins integration
> > -- So basically if you use jenkins, subtests are useless? Should we be
> > discouraging them? I don't suppose fixing jenkins is an option?
> 
> I just need to fix the junit backend. I've been meaning to and keep
> forgetting. I'm working on it now.
> 

Sounds good.

Thanks,
Nanley

> > 
> > > cause after every run. One known failure is the sRGB miptree subtest.
> > > It's unresolved whether or not the issue lies within HW or the codec-
> > > provided texture. I would like to be notified when the other modes are
> > > failing as that would indicate something has broken in mesa.
> > 
> > Hm, well my experience with Qualcomm Adreno A420 is that (a) I get
> > black instead of the error color when loading HDR textures, and (b) I
> > need much wider color tolerances -- IIRC I said color had 6 bits of
> > resolution. Once I did that, the ldr and srgb tests both passed. Both
> > are attributable to incorrect programming of the hardware though.
> > There's no easy way (currently) to run piglit tests against android
> > userspace, and even if there were, their drivers tend to be pretty
> > weak in terms of ... working.
> > 
> > I couldn't get the array tests to work since I think they implicitly
> > depended on some higher GL *and* GLES version than what the freedreno
> > driver provides [GL 3.1, GLES 3.0].
> > 
> >   -ilia
> > ___
> > Piglit mailing list
> > Piglit@lists.freedesktop.org
> > http://lists.freedesktop.org/mailman/listinfo/piglit


___
Piglit mailing list
Piglit@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/piglit

Re: [Piglit] [PATCH v3] khr_texture_compression_astc: Enable subtest reports

[Nanley Chery]

On Tue, Dec 01, 2015 at 10:21:23AM -0500, Ilia Mirkin wrote:
> On Tue, Dec 1, 2015 at 1:31 AM, Nanley Chery  wrote:
> > From: Nanley Chery 
> >
> > Enable Jenkins to report the result of each individual subtest
> > for the array and miptree ASTC tests. Modify the miptree test
> > to only run one subset of ASTC formats at a time.
> >
> > v2. Modify miptree test to only check the given subtest.
> > Use the -subtest option to run a specific subtest.
> > v3. Indent function arguments and misc cleanups (Ilia).
> > Initialize texture objects to 0 (Ilia).
> >
> > Signed-off-by: Nanley Chery 
> > ---
> >  tests/all.py   | 12 ++--
> >  .../khr_compressed_astc-miptree.c  | 84 
> > +-
> >  2 files changed, 42 insertions(+), 54 deletions(-)
> >
> > diff --git a/tests/all.py b/tests/all.py
> > index cfafa71..29ee6a9 100644
> > --- a/tests/all.py
> > +++ b/tests/all.py
> > @@ -4223,12 +4223,16 @@ with profile.group_manager(
> >   PiglitGLTest,
> >   grouptools.join('spec', 'khr_texture_compression_astc')) as g:
> >  g(['arb_texture_compression-invalid-formats', 'astc'], 'invalid 
> > formats')
> > -g(['khr_compressed_astc-array_gl'], 'array-gl')
> > -g(['khr_compressed_astc-array_gles3'], 'array-gles')
> >  g(['khr_compressed_astc-basic_gl'], 'basic-gl')
> >  g(['khr_compressed_astc-basic_gles2'], 'basic-gles')
> > -g(['khr_compressed_astc-miptree_gl'], 'miptree-gl')
> > -g(['khr_compressed_astc-miptree_gles2'], 'miptree-gles')
> > +
> > +for subtest in ('odd', 'even'):
> > +g(['khr_compressed_astc-array_gl', '-subtest', subtest])
> > +g(['khr_compressed_astc-array_gles3', '-subtest', subtest])
> > +
> > +for subtest in ('ldr', 'srgb', 'hdr'):
> > +g(['khr_compressed_astc-miptree_gl', '-subtest', subtest])
> > +g(['khr_compressed_astc-miptree_gles2', '-subtest', subtest])
> 
> I still don't understand why you need this... what's wrong with the
> current way? All the subtests should get run, and be properly
> reported, without needing to be included one-by-one here. Does that
> not happen?

This enables the Jenkins CI system to report which specific subtest is
failing - without having to inspect the error message to determine the
cause after every run. One known failure is the sRGB miptree subtest.
It's unresolved whether or not the issue lies within HW or the codec-
provided texture. I would like to be notified when the other modes are
failing as that would indicate something has broken in mesa.

-Nanley
___
Piglit mailing list
Piglit@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/piglit

Re: [Piglit] [PATCH v3] khr_texture_compression_astc: Enable subtest reports

[Nanley Chery]

On Tue, Dec 01, 2015 at 10:58:29AM -0800, Nanley Chery wrote:
> On Tue, Dec 01, 2015 at 10:21:23AM -0500, Ilia Mirkin wrote:
> > On Tue, Dec 1, 2015 at 1:31 AM, Nanley Chery  wrote:
> > > From: Nanley Chery 
> > >
> > > Enable Jenkins to report the result of each individual subtest
> > > for the array and miptree ASTC tests. Modify the miptree test
> > > to only run one subset of ASTC formats at a time.
> > >
> > > v2. Modify miptree test to only check the given subtest.
> > > Use the -subtest option to run a specific subtest.
> > > v3. Indent function arguments and misc cleanups (Ilia).
> > > Initialize texture objects to 0 (Ilia).
> > >
> > > Signed-off-by: Nanley Chery 
> > > ---
> > >  tests/all.py   | 12 ++--
> > >  .../khr_compressed_astc-miptree.c  | 84 
> > > +-
> > >  2 files changed, 42 insertions(+), 54 deletions(-)
> > >
> > > diff --git a/tests/all.py b/tests/all.py
> > > index cfafa71..29ee6a9 100644
> > > --- a/tests/all.py
> > > +++ b/tests/all.py
> > > @@ -4223,12 +4223,16 @@ with profile.group_manager(
> > >   PiglitGLTest,
> > >   grouptools.join('spec', 'khr_texture_compression_astc')) as g:
> > >  g(['arb_texture_compression-invalid-formats', 'astc'], 'invalid 
> > > formats')
> > > -g(['khr_compressed_astc-array_gl'], 'array-gl')
> > > -g(['khr_compressed_astc-array_gles3'], 'array-gles')
> > >  g(['khr_compressed_astc-basic_gl'], 'basic-gl')
> > >  g(['khr_compressed_astc-basic_gles2'], 'basic-gles')
> > > -g(['khr_compressed_astc-miptree_gl'], 'miptree-gl')
> > > -g(['khr_compressed_astc-miptree_gles2'], 'miptree-gles')
> > > +
> > > +for subtest in ('odd', 'even'):
> > > +g(['khr_compressed_astc-array_gl', '-subtest', subtest])
> > > +g(['khr_compressed_astc-array_gles3', '-subtest', subtest])
> > > +
> > > +for subtest in ('ldr', 'srgb', 'hdr'):
> > > +g(['khr_compressed_astc-miptree_gl', '-subtest', subtest])
> > > +g(['khr_compressed_astc-miptree_gles2', '-subtest', subtest])
> > 
> > I still don't understand why you need this... what's wrong with the
> > current way? All the subtests should get run, and be properly
> > reported, without needing to be included one-by-one here. Does that
> > not happen?
> 
> This enables the Jenkins CI system to report which specific subtest is
> failing - without having to inspect the error message to determine the
> cause after every run. One known failure is the sRGB miptree subtest.
> It's unresolved whether or not the issue lies within HW or the codec-
> provided texture. I would like to be notified when the other modes are
> failing as that would indicate something has broken in mesa.
> 

To answer the last part of your question, yes, each subtest does get
run in Piglit. The purpose of this hunk is to improve reporting in the
Jenkins CI system.

> -Nanley
___
Piglit mailing list
Piglit@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/piglit

Re: [Piglit] [PATCH v3] khr_texture_compression_astc: Enable subtest reports

[Ilia Mirkin]

On Tue, Dec 1, 2015 at 1:58 PM, Nanley Chery  wrote:
> On Tue, Dec 01, 2015 at 10:21:23AM -0500, Ilia Mirkin wrote:
>> On Tue, Dec 1, 2015 at 1:31 AM, Nanley Chery  wrote:
>> > From: Nanley Chery 
>> >
>> > Enable Jenkins to report the result of each individual subtest
>> > for the array and miptree ASTC tests. Modify the miptree test
>> > to only run one subset of ASTC formats at a time.
>> >
>> > v2. Modify miptree test to only check the given subtest.
>> > Use the -subtest option to run a specific subtest.
>> > v3. Indent function arguments and misc cleanups (Ilia).
>> > Initialize texture objects to 0 (Ilia).
>> >
>> > Signed-off-by: Nanley Chery 
>> > ---
>> >  tests/all.py   | 12 ++--
>> >  .../khr_compressed_astc-miptree.c  | 84 
>> > +-
>> >  2 files changed, 42 insertions(+), 54 deletions(-)
>> >
>> > diff --git a/tests/all.py b/tests/all.py
>> > index cfafa71..29ee6a9 100644
>> > --- a/tests/all.py
>> > +++ b/tests/all.py
>> > @@ -4223,12 +4223,16 @@ with profile.group_manager(
>> >   PiglitGLTest,
>> >   grouptools.join('spec', 'khr_texture_compression_astc')) as g:
>> >  g(['arb_texture_compression-invalid-formats', 'astc'], 'invalid 
>> > formats')
>> > -g(['khr_compressed_astc-array_gl'], 'array-gl')
>> > -g(['khr_compressed_astc-array_gles3'], 'array-gles')
>> >  g(['khr_compressed_astc-basic_gl'], 'basic-gl')
>> >  g(['khr_compressed_astc-basic_gles2'], 'basic-gles')
>> > -g(['khr_compressed_astc-miptree_gl'], 'miptree-gl')
>> > -g(['khr_compressed_astc-miptree_gles2'], 'miptree-gles')
>> > +
>> > +for subtest in ('odd', 'even'):
>> > +g(['khr_compressed_astc-array_gl', '-subtest', subtest])
>> > +g(['khr_compressed_astc-array_gles3', '-subtest', subtest])
>> > +
>> > +for subtest in ('ldr', 'srgb', 'hdr'):
>> > +g(['khr_compressed_astc-miptree_gl', '-subtest', subtest])
>> > +g(['khr_compressed_astc-miptree_gles2', '-subtest', subtest])
>>
>> I still don't understand why you need this... what's wrong with the
>> current way? All the subtests should get run, and be properly
>> reported, without needing to be included one-by-one here. Does that
>> not happen?
>
> This enables the Jenkins CI system to report which specific subtest is
> failing - without having to inspect the error message to determine the

Wow, that seems like a pretty big failing for the jenkins integration
-- So basically if you use jenkins, subtests are useless? Should we be
discouraging them? I don't suppose fixing jenkins is an option?

> cause after every run. One known failure is the sRGB miptree subtest.
> It's unresolved whether or not the issue lies within HW or the codec-
> provided texture. I would like to be notified when the other modes are
> failing as that would indicate something has broken in mesa.

Hm, well my experience with Qualcomm Adreno A420 is that (a) I get
black instead of the error color when loading HDR textures, and (b) I
need much wider color tolerances -- IIRC I said color had 6 bits of
resolution. Once I did that, the ldr and srgb tests both passed. Both
are attributable to incorrect programming of the hardware though.
There's no easy way (currently) to run piglit tests against android
userspace, and even if there were, their drivers tend to be pretty
weak in terms of ... working.

I couldn't get the array tests to work since I think they implicitly
depended on some higher GL *and* GLES version than what the freedreno
driver provides [GL 3.1, GLES 3.0].

  -ilia
___
Piglit mailing list
Piglit@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/piglit

Re: [Piglit] [PATCH v3] khr_texture_compression_astc: Enable subtest reports

[Ilia Mirkin]

On Tue, Dec 1, 2015 at 1:31 AM, Nanley Chery  wrote:
> From: Nanley Chery 
>
> Enable Jenkins to report the result of each individual subtest
> for the array and miptree ASTC tests. Modify the miptree test
> to only run one subset of ASTC formats at a time.
>
> v2. Modify miptree test to only check the given subtest.
> Use the -subtest option to run a specific subtest.
> v3. Indent function arguments and misc cleanups (Ilia).
> Initialize texture objects to 0 (Ilia).
>
> Signed-off-by: Nanley Chery 
> ---
>  tests/all.py   | 12 ++--
>  .../khr_compressed_astc-miptree.c  | 84 
> +-
>  2 files changed, 42 insertions(+), 54 deletions(-)
>
> diff --git a/tests/all.py b/tests/all.py
> index cfafa71..29ee6a9 100644
> --- a/tests/all.py
> +++ b/tests/all.py
> @@ -4223,12 +4223,16 @@ with profile.group_manager(
>   PiglitGLTest,
>   grouptools.join('spec', 'khr_texture_compression_astc')) as g:
>  g(['arb_texture_compression-invalid-formats', 'astc'], 'invalid formats')
> -g(['khr_compressed_astc-array_gl'], 'array-gl')
> -g(['khr_compressed_astc-array_gles3'], 'array-gles')
>  g(['khr_compressed_astc-basic_gl'], 'basic-gl')
>  g(['khr_compressed_astc-basic_gles2'], 'basic-gles')
> -g(['khr_compressed_astc-miptree_gl'], 'miptree-gl')
> -g(['khr_compressed_astc-miptree_gles2'], 'miptree-gles')
> +
> +for subtest in ('odd', 'even'):
> +g(['khr_compressed_astc-array_gl', '-subtest', subtest])
> +g(['khr_compressed_astc-array_gles3', '-subtest', subtest])
> +
> +for subtest in ('ldr', 'srgb', 'hdr'):
> +g(['khr_compressed_astc-miptree_gl', '-subtest', subtest])
> +g(['khr_compressed_astc-miptree_gles2', '-subtest', subtest])

I still don't understand why you need this... what's wrong with the
current way? All the subtests should get run, and be properly
reported, without needing to be included one-by-one here. Does that
not happen?
___
Piglit mailing list
Piglit@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/piglit

[Piglit] [PATCH v3] khr_texture_compression_astc: Enable subtest reports

[Nanley Chery]

From: Nanley Chery 

Enable Jenkins to report the result of each individual subtest
for the array and miptree ASTC tests. Modify the miptree test
to only run one subset of ASTC formats at a time.

v2. Modify miptree test to only check the given subtest.
Use the -subtest option to run a specific subtest.
v3. Indent function arguments and misc cleanups (Ilia).
Initialize texture objects to 0 (Ilia).

Signed-off-by: Nanley Chery 
---
 tests/all.py   | 12 ++--
 .../khr_compressed_astc-miptree.c  | 84 +-
 2 files changed, 42 insertions(+), 54 deletions(-)

diff --git a/tests/all.py b/tests/all.py
index cfafa71..29ee6a9 100644
--- a/tests/all.py
+++ b/tests/all.py
@@ -4223,12 +4223,16 @@ with profile.group_manager(
  PiglitGLTest,
  grouptools.join('spec', 'khr_texture_compression_astc')) as g:
 g(['arb_texture_compression-invalid-formats', 'astc'], 'invalid formats')
-g(['khr_compressed_astc-array_gl'], 'array-gl')
-g(['khr_compressed_astc-array_gles3'], 'array-gles')
 g(['khr_compressed_astc-basic_gl'], 'basic-gl')
 g(['khr_compressed_astc-basic_gles2'], 'basic-gles')
-g(['khr_compressed_astc-miptree_gl'], 'miptree-gl')
-g(['khr_compressed_astc-miptree_gles2'], 'miptree-gles')
+
+for subtest in ('odd', 'even'):
+g(['khr_compressed_astc-array_gl', '-subtest', subtest])
+g(['khr_compressed_astc-array_gles3', '-subtest', subtest])
+
+for subtest in ('ldr', 'srgb', 'hdr'):
+g(['khr_compressed_astc-miptree_gl', '-subtest', subtest])
+g(['khr_compressed_astc-miptree_gles2', '-subtest', subtest])
 
 with profile.group_manager(
  PiglitGLTest,
diff --git 
a/tests/spec/khr_texture_compression_astc/khr_compressed_astc-miptree.c 
b/tests/spec/khr_texture_compression_astc/khr_compressed_astc-miptree.c
index 20f2415..6429c2e 100644
--- a/tests/spec/khr_texture_compression_astc/khr_compressed_astc-miptree.c
+++ b/tests/spec/khr_texture_compression_astc/khr_compressed_astc-miptree.c
@@ -222,10 +222,9 @@ bool draw_compare_levels(bool check_error, bool check_srgb,
 enum piglit_result
 test_miptrees(void* input_type)
 {
-   int subtest =  0;
-   enum test_type * type = (enum test_type*) input_type;
-   bool is_srgb_test = *type == TEST_TYPE_SRGB;
-   bool is_hdr_test  = *type == TEST_TYPE_HDR;
+   const enum test_type subtest = *(enum test_type*) input_type;
+   const bool is_srgb_test = subtest == TEST_TYPE_SRGB;
+   const bool is_hdr_test  = subtest == TEST_TYPE_HDR;
 
static const char * tests[3] = {"hdr", "ldrl", "ldrs"};
static const char * block_dim_str[14] = {
@@ -245,62 +244,47 @@ test_miptrees(void* input_type)
"12x12"
};
 
-   bool has_hdr = piglit_is_extension_supported(
-   "GL_KHR_texture_compression_astc_hdr");
-
-   /* If testing sRGB mode, fast-forward to the srgb test. */
-   if (is_srgb_test) {
-   subtest =  TEST_TYPE_SRGB;
-   } else {
-   /* Skip if on an HDR system not running the HDR test
-* or if on an LDR system running the HDR test.
-*/
-   if (has_hdr != is_hdr_test)
-   return PIGLIT_SKIP;
+   if (!is_srgb_test)
piglit_require_extension("GL_EXT_texture_sRGB_decode");
 
-   }
-
GLint pixel_offset_loc = glGetUniformLocation(prog, "pixel_offset");
GLint level_pixel_size_loc = glGetUniformLocation(prog,
"level_pixel_size");
 
-   /* Test each submode */
-   for (; subtest < ARRAY_SIZE(tests); ++subtest) {
-
-   /*  Check for error color if an LDR-only sys reading an HDR
-*  texture. No need to draw a reference mipmap in this case.
-*/
-   int check_error = !has_hdr && subtest == TEST_TYPE_HDR;
-   int block_dims = 0;
-   for (; block_dims < ARRAY_SIZE(block_dim_str); ++block_dims) {
-
-   /* Texture objects. */
-   GLuint tex_compressed;
-   GLuint tex_decompressed;
-
-   /* Load texture for current submode and block size */
-   load_texture("compressed", tests[subtest],
+   /*  Check for error color if an LDR-only sys reading an HDR
+*  texture. No need to draw a reference mipmap in this case.
+*/
+   const bool has_hdr = piglit_is_extension_supported(
+   "GL_KHR_texture_compression_astc_hdr");
+   const bool check_error = is_hdr_test && !has_hdr;
+   int block_dims;
+   for (block_dims = 0; block_dims < ARRAY_SIZE(block_dim_str); 
++block_dims) {
+
+   /* Texture objects. */
+   GLuint tex_compressed = 0;
+   GLuint