Re: [Intel-gfx] [PATCH i-g-t] tests: make color management tests part of the BAT

2016-07-19 Thread Daniel Vetter 
On Tue, Jul 19, 2016 at 12:42:36PM +0100, Chris Wilson wrote:
> On Tue, Jul 19, 2016 at 08:48:11AM +0200, Daniel Vetter wrote:
> > On Fri, Jul 15, 2016 at 03:00:59PM +0100, Lionel Landwerlin wrote:
> > > Signed-off-by: Lionel Landwerlin 
> > 
> > Do we have the time for those in the BAT budget?
> 
> Do we not? It has been demonstrated that people notice when gamma is
> broken, can we afford to risk repeating this bug?
> 
> (Or in other news, where are all the new QA bugs from failing tests?
> Seems like we are missing some bug reports from igt added to show off
> bugs.)

We don't run enough test, don't report the bugs we do catch and don't
handle it. It's all sad, but unfortunately just piling even more onto the
BAT pile is not going to help anyone - as is the results are already
pretty close to useless. The idea behind BAT really is that it runs real
fast, and that it's a gate for a more extended testcase. If we add tests
until it's real slow and still broken, no one wins at all.


Unfortunately I can't tell you how to get out of this mess either :(
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
___
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [Intel-gfx] [PATCH i-g-t] tests: make color management tests part of the BAT

2016-07-19 Thread Lionel Landwerlin 

On 19/07/16 13:49, Chris Wilson wrote:

On Tue, Jul 19, 2016 at 01:21:23PM +0100, Lionel Landwerlin wrote:

On 19/07/16 12:42, Chris Wilson wrote:

On Tue, Jul 19, 2016 at 08:48:11AM +0200, Daniel Vetter wrote:

On Fri, Jul 15, 2016 at 03:00:59PM +0100, Lionel Landwerlin wrote:

Signed-off-by: Lionel Landwerlin 

Do we have the time for those in the BAT budget?

Do we not? It has been demonstrated that people notice when gamma is
broken, can we afford to risk repeating this bug?

(Or in other news, where are all the new QA bugs from failing tests?
Seems like we are missing some bug reports from igt added to show off
bugs.)
-Chris


It's about 35s to run this test :
real0m34.352s
user0m0.972s
sys0m1.626s

Knowing that we repeat the same tests across different pipes (so for
it would only take a third of that time if we were to just test pipe
A).

If one test is likely to catch 99.999% of the bugs, then just add that
one test to bat.
  

I don't have a sense of the budget, is that too much already?

Oh, we've overshot the budget by 200%. Deciding which tests are more
important than others, or whether that budget is unrealistic requires
holistic knowledge i.e. our maintainer overlords.
-Chris


Right,


Let me resend a patch for just pipe-A then.

I haven't seen failures on one pipe but not the other.


-

Lionel

___
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [Intel-gfx] [PATCH i-g-t] tests: make color management tests part of the BAT

2016-07-19 Thread Chris Wilson 
On Tue, Jul 19, 2016 at 01:21:23PM +0100, Lionel Landwerlin wrote:
> On 19/07/16 12:42, Chris Wilson wrote:
> >On Tue, Jul 19, 2016 at 08:48:11AM +0200, Daniel Vetter wrote:
> >>On Fri, Jul 15, 2016 at 03:00:59PM +0100, Lionel Landwerlin wrote:
> >>>Signed-off-by: Lionel Landwerlin 
> >>Do we have the time for those in the BAT budget?
> >Do we not? It has been demonstrated that people notice when gamma is
> >broken, can we afford to risk repeating this bug?
> >
> >(Or in other news, where are all the new QA bugs from failing tests?
> >Seems like we are missing some bug reports from igt added to show off
> >bugs.)
> >-Chris
> >
> It's about 35s to run this test :
> real0m34.352s
> user0m0.972s
> sys0m1.626s
> 
> Knowing that we repeat the same tests across different pipes (so for
> it would only take a third of that time if we were to just test pipe
> A).

If one test is likely to catch 99.999% of the bugs, then just add that
one test to bat.
 
> I don't have a sense of the budget, is that too much already?

Oh, we've overshot the budget by 200%. Deciding which tests are more
important than others, or whether that budget is unrealistic requires
holistic knowledge i.e. our maintainer overlords.
-Chris

-- 
Chris Wilson, Intel Open Source Technology Centre
___
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [Intel-gfx] [PATCH i-g-t] tests: make color management tests part of the BAT

2016-07-19 Thread Lionel Landwerlin 

On 19/07/16 12:42, Chris Wilson wrote:

On Tue, Jul 19, 2016 at 08:48:11AM +0200, Daniel Vetter wrote:

On Fri, Jul 15, 2016 at 03:00:59PM +0100, Lionel Landwerlin wrote:

Signed-off-by: Lionel Landwerlin 

Do we have the time for those in the BAT budget?

Do we not? It has been demonstrated that people notice when gamma is
broken, can we afford to risk repeating this bug?

(Or in other news, where are all the new QA bugs from failing tests?
Seems like we are missing some bug reports from igt added to show off
bugs.)
-Chris


It's about 35s to run this test :
real0m34.352s
user0m0.972s
sys0m1.626s

Knowing that we repeat the same tests across different pipes (so for it 
would only take a third of that time if we were to just test pipe A).


I don't have a sense of the budget, is that too much already?

Thanks,
-
Lionel
___
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [Intel-gfx] [PATCH i-g-t] tests: make color management tests part of the BAT

2016-07-19 Thread Chris Wilson 
On Tue, Jul 19, 2016 at 08:48:11AM +0200, Daniel Vetter wrote:
> On Fri, Jul 15, 2016 at 03:00:59PM +0100, Lionel Landwerlin wrote:
> > Signed-off-by: Lionel Landwerlin 
> 
> Do we have the time for those in the BAT budget?

Do we not? It has been demonstrated that people notice when gamma is
broken, can we afford to risk repeating this bug?

(Or in other news, where are all the new QA bugs from failing tests?
Seems like we are missing some bug reports from igt added to show off
bugs.)
-Chris

-- 
Chris Wilson, Intel Open Source Technology Centre
___
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [Intel-gfx] [PATCH i-g-t] tests: make color management tests part of the BAT

2016-07-19 Thread Daniel Vetter 
On Fri, Jul 15, 2016 at 03:00:59PM +0100, Lionel Landwerlin wrote:
> Signed-off-by: Lionel Landwerlin 

Do we have the time for those in the BAT budget?
-Daniel

> ---
>  tests/Makefile.sources   |2 +-
>  tests/kms_pipe_color.c   | 1184 
> --
>  tests/kms_pipe_color_basic.c | 1184 
> ++
>  3 files changed, 1185 insertions(+), 1185 deletions(-)
>  delete mode 100644 tests/kms_pipe_color.c
>  create mode 100644 tests/kms_pipe_color_basic.c
> 
> diff --git a/tests/Makefile.sources b/tests/Makefile.sources
> index 8a9a7ec..53afe3c 100644
> --- a/tests/Makefile.sources
> +++ b/tests/Makefile.sources
> @@ -98,7 +98,7 @@ TESTS_progs_M = \
>   kms_legacy_colorkey \
>   kms_mmio_vs_cs_flip \
>   kms_pipe_b_c_ivb \
> - kms_pipe_color \
> + kms_pipe_color_basic \
>   kms_pipe_crc_basic \
>   kms_plane \
>   kms_psr_sink_crc \
> diff --git a/tests/kms_pipe_color.c b/tests/kms_pipe_color.c
> deleted file mode 100644
> index 9f7ac7e..000
> --- a/tests/kms_pipe_color.c
> +++ /dev/null
> @@ -1,1184 +0,0 @@
> -/*
> - * Copyright © 2015 Intel Corporation
> - *
> -  * Permission is hereby granted, free of charge, to any person obtaining a
> - * copy of this software and associated documentation files (the "Software"),
> - * to deal in the Software without restriction, including without limitation
> - * the rights to use, copy, modify, merge, publish, distribute, sublicense,
> - * and/or sell copies of the Software, and to permit persons to whom the
> - * Software is furnished to do so, subject to the following conditions:
> - *
> - * The above copyright notice and this permission notice (including the next
> - * paragraph) shall be included in all copies or substantial portions of the
> - * Software.
> - *
> - * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> - * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> - * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
> - * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> - * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
> - * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER 
> DEALINGS
> - * IN THE SOFTWARE.
> - *
> - */
> -
> -#include 
> -#include 
> -
> -#include "drm.h"
> -#include "drmtest.h"
> -#include "igt.h"
> -
> -IGT_TEST_DESCRIPTION("Test Color Features at Pipe level");
> -
> -/* Data structures for gamma/degamma ramps & ctm matrix. */
> -struct _drm_color_ctm {
> - /* Transformation matrix in S31.32 format. */
> - __s64 matrix[9];
> -};
> -
> -struct _drm_color_lut {
> - /*
> -  * Data is U0.16 fixed point format.
> -  */
> - __u16 red;
> - __u16 green;
> - __u16 blue;
> - __u16 reserved;
> -};
> -
> -/* Internal */
> -typedef struct {
> - double r, g, b;
> -} color_t;
> -
> -typedef struct {
> - int drm_fd;
> - uint32_t devid;
> - igt_display_t display;
> - igt_pipe_crc_t *pipe_crc;
> -
> - uint32_t color_depth;
> - uint64_t degamma_lut_size;
> - uint64_t gamma_lut_size;
> -} data_t;
> -
> -
> -static void paint_gradient_rectangles(data_t *data,
> -   drmModeModeInfo *mode,
> -   color_t *colors,
> -   struct igt_fb *fb)
> -{
> - cairo_t *cr = igt_get_cairo_ctx(data->drm_fd, fb);
> - int i, l = mode->hdisplay / 3;
> -
> - /* Paint 3 gradient rectangles with red/green/blue between 1.0 and
> -  * 0.5. We want to avoid 0 so each max LUTs only affect their own
> -  * rectangle.
> -  */
> - for (i = 0 ; i < 3; i++) {
> - igt_paint_color_gradient_range(cr, i * l, 0, l, mode->vdisplay,
> -colors[i].r != 0 ? 0.2 : 0,
> -colors[i].g != 0 ? 0.2 : 0,
> -colors[i].b != 0 ? 0.2 : 0,
> -colors[i].r,
> -colors[i].g,
> -colors[i].b);
> - }
> -
> - cairo_destroy(cr);
> -}
> -
> -static void paint_rectangles(data_t *data,
> -  drmModeModeInfo *mode,
> -  color_t *colors,
> -  struct igt_fb *fb)
> -{
> - cairo_t *cr = igt_get_cairo_ctx(data->drm_fd, fb);
> - int i, l = mode->hdisplay / 3;
> -
> - /* Paint 3 solid rectangles. */
> - for (i = 0 ; i < 3; i++) {
> - igt_paint_color(cr, i * l, 0, l, mode->vdisplay,
> - colors[i].r, colors[i].g, colors[i].b);
> - }
> -
> - cairo_destroy(cr);
> -}
> -
> -static double *generate_table(uint32_t lut_size, double exp)
> -{
> - double

[Intel-gfx] [PATCH i-g-t] tests: make color management tests part of the BAT

2016-07-15 Thread Lionel Landwerlin 
Signed-off-by: Lionel Landwerlin 
---
 tests/Makefile.sources   |2 +-
 tests/kms_pipe_color.c   | 1184 --
 tests/kms_pipe_color_basic.c | 1184 ++
 3 files changed, 1185 insertions(+), 1185 deletions(-)
 delete mode 100644 tests/kms_pipe_color.c
 create mode 100644 tests/kms_pipe_color_basic.c

diff --git a/tests/Makefile.sources b/tests/Makefile.sources
index 8a9a7ec..53afe3c 100644
--- a/tests/Makefile.sources
+++ b/tests/Makefile.sources
@@ -98,7 +98,7 @@ TESTS_progs_M = \
kms_legacy_colorkey \
kms_mmio_vs_cs_flip \
kms_pipe_b_c_ivb \
-   kms_pipe_color \
+   kms_pipe_color_basic \
kms_pipe_crc_basic \
kms_plane \
kms_psr_sink_crc \
diff --git a/tests/kms_pipe_color.c b/tests/kms_pipe_color.c
deleted file mode 100644
index 9f7ac7e..000
--- a/tests/kms_pipe_color.c
+++ /dev/null
@@ -1,1184 +0,0 @@
-/*
- * Copyright © 2015 Intel Corporation
- *
-  * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the "Software"),
- * to deal in the Software without restriction, including without limitation
- * the rights to use, copy, modify, merge, publish, distribute, sublicense,
- * and/or sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice (including the next
- * paragraph) shall be included in all copies or substantial portions of the
- * Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
- * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
- * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
- * IN THE SOFTWARE.
- *
- */
-
-#include 
-#include 
-
-#include "drm.h"
-#include "drmtest.h"
-#include "igt.h"
-
-IGT_TEST_DESCRIPTION("Test Color Features at Pipe level");
-
-/* Data structures for gamma/degamma ramps & ctm matrix. */
-struct _drm_color_ctm {
-   /* Transformation matrix in S31.32 format. */
-   __s64 matrix[9];
-};
-
-struct _drm_color_lut {
-   /*
-* Data is U0.16 fixed point format.
-*/
-   __u16 red;
-   __u16 green;
-   __u16 blue;
-   __u16 reserved;
-};
-
-/* Internal */
-typedef struct {
-   double r, g, b;
-} color_t;
-
-typedef struct {
-   int drm_fd;
-   uint32_t devid;
-   igt_display_t display;
-   igt_pipe_crc_t *pipe_crc;
-
-   uint32_t color_depth;
-   uint64_t degamma_lut_size;
-   uint64_t gamma_lut_size;
-} data_t;
-
-
-static void paint_gradient_rectangles(data_t *data,
- drmModeModeInfo *mode,
- color_t *colors,
- struct igt_fb *fb)
-{
-   cairo_t *cr = igt_get_cairo_ctx(data->drm_fd, fb);
-   int i, l = mode->hdisplay / 3;
-
-   /* Paint 3 gradient rectangles with red/green/blue between 1.0 and
-* 0.5. We want to avoid 0 so each max LUTs only affect their own
-* rectangle.
-*/
-   for (i = 0 ; i < 3; i++) {
-   igt_paint_color_gradient_range(cr, i * l, 0, l, mode->vdisplay,
-  colors[i].r != 0 ? 0.2 : 0,
-  colors[i].g != 0 ? 0.2 : 0,
-  colors[i].b != 0 ? 0.2 : 0,
-  colors[i].r,
-  colors[i].g,
-  colors[i].b);
-   }
-
-   cairo_destroy(cr);
-}
-
-static void paint_rectangles(data_t *data,
-drmModeModeInfo *mode,
-color_t *colors,
-struct igt_fb *fb)
-{
-   cairo_t *cr = igt_get_cairo_ctx(data->drm_fd, fb);
-   int i, l = mode->hdisplay / 3;
-
-   /* Paint 3 solid rectangles. */
-   for (i = 0 ; i < 3; i++) {
-   igt_paint_color(cr, i * l, 0, l, mode->vdisplay,
-   colors[i].r, colors[i].g, colors[i].b);
-   }
-
-   cairo_destroy(cr);
-}
-
-static double *generate_table(uint32_t lut_size, double exp)
-{
-   double *coeffs = malloc(sizeof(double) * lut_size);
-   uint32_t i;
-
-   for (i = 0; i < lut_size; i++)
-   coeffs[i] = powf((double) i * 1.0 / (double) (lut_size - 1), 
exp);
-
-   return coeffs;
-}
-
-static double *generate_table_max(uint32_t lut_size)
-{
-   double *coeffs =