Re: [Intel-gfx] [PATCH i-g-t] tests/kms_plane_multiple: CRC based atomic correctness test

[Mika Kahola]

On Mon, 2016-10-17 at 16:30 +0200, Daniel Vetter wrote:
> On Mon, Oct 17, 2016 at 02:28:37PM +0300, Mika Kahola wrote:
> > 
> > +   for (int i = 0; i < iterations; i++) {
> > +   igt_info("%d/%d: Testing connector %s using pipe
> > %s with %d planes\n",
> > +    i+1, iterations, igt_output_name(output),
> > +    kmstest_pipe_name(pipe), max_planes);
> > +
> > +   test_init(data, pipe);
> > +
> > +   test_grab_crc(data, output, pipe, , tiling,
> > +     _crc);
> > +
> > +   test_planes(data, pipe, , tiling, max_planes,
> > output);
> > +
> > +   if (test_atomic) {
> > +   igt_display_commit_atomic(>display,
> > +     DRM_MODE_PAGE_FL
> > IP_EVENT,
> > +     >display);
> > +   } else
> > +   igt_display_commit2(>display,
> > COMMIT_LEGACY);
> > +
> > +   igt_pipe_crc_start(data->pipe_crc);
> > +   n = igt_pipe_crc_get_crcs(data->pipe_crc, 1,
> > );
> > +   igt_assert_eq(n, 1);
> > +   igt_pipe_crc_stop(data->pipe_crc);
> Comment on testing method here: With atomic we don't just require
> that the
> result looks good at the end, but also that _every_ frame is perfect.
> That
> means you need a slightly different test sequence:
> 
> 1. Enable crc capture.
> 
> 2. Create a new atomic state (randomized, whatever) which should in
> the
> end still result in the same screen contents using the punchout box
> trick.
> This depends upon the exact subtest.
> 
> 3. Commit the state from step 2.
> 
> 4. Wait to make sure the atomic commit has completed. If you do an
> async
> commit, that means waiting for the flip_event to get signalled
> (didn't see
> code for that anywhere).
> 
> 5. Fetch all the crc values (if there's not a bug in your code or in
> the
> kernel it should be just 1) and make sure they are _all_ the right
> value.
> Your code here only grabs 1 crc after the atomic commit completed,
> which
> means if there's tearing or underruns you'll miss them. Which means
> it
> won't actually validate the crucial feature for which we've created
> atomic!
That's true, I missed that one. I'll make modification on the test
sequence and spin another round of this test.

Cheers,
Mika

> 
> 6. Go back to 2.
> 
> 7. After enough loops, stop crc capturing.
> 
> Note that this is the loop for ALLOW_MODESET==false atomic commits,
> i.e.
> where every atomic commit should take at most 1 vblank interval. If
> any of
> the commits take longer than that, there's a bug in either the kernel
> or
> your testcase. Note that crc_start alone has a few vblank waits (due
> to
> crc bugs on some platforms) which will break this.
> 
> Cheers, Daniel
> 
> > 
> > +
> > +   igt_assert_crc_equal(_crc, crc);
> > +
> > +   test_fini(data, output, max_planes);
> > +   }
> > +}
___
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Re: [Intel-gfx] [PATCH i-g-t] tests/kms_plane_multiple: CRC based atomic correctness test

[Daniel Vetter]

On Mon, Oct 17, 2016 at 02:28:37PM +0300, Mika Kahola wrote:
> + for (int i = 0; i < iterations; i++) {
> + igt_info("%d/%d: Testing connector %s using pipe %s with %d 
> planes\n",
> +  i+1, iterations, igt_output_name(output),
> +  kmstest_pipe_name(pipe), max_planes);
> +
> + test_init(data, pipe);
> +
> + test_grab_crc(data, output, pipe, , tiling,
> +   _crc);
> +
> + test_planes(data, pipe, , tiling, max_planes, output);
> +
> + if (test_atomic) {
> + igt_display_commit_atomic(>display,
> +   DRM_MODE_PAGE_FLIP_EVENT,
> +   >display);
> + } else
> + igt_display_commit2(>display, COMMIT_LEGACY);
> +
> + igt_pipe_crc_start(data->pipe_crc);
> + n = igt_pipe_crc_get_crcs(data->pipe_crc, 1, );
> + igt_assert_eq(n, 1);
> + igt_pipe_crc_stop(data->pipe_crc);

Comment on testing method here: With atomic we don't just require that the
result looks good at the end, but also that _every_ frame is perfect. That
means you need a slightly different test sequence:

1. Enable crc capture.

2. Create a new atomic state (randomized, whatever) which should in the
end still result in the same screen contents using the punchout box trick.
This depends upon the exact subtest.

3. Commit the state from step 2.

4. Wait to make sure the atomic commit has completed. If you do an async
commit, that means waiting for the flip_event to get signalled (didn't see
code for that anywhere).

5. Fetch all the crc values (if there's not a bug in your code or in the
kernel it should be just 1) and make sure they are _all_ the right value.
Your code here only grabs 1 crc after the atomic commit completed, which
means if there's tearing or underruns you'll miss them. Which means it
won't actually validate the crucial feature for which we've created
atomic!

6. Go back to 2.

7. After enough loops, stop crc capturing.

Note that this is the loop for ALLOW_MODESET==false atomic commits, i.e.
where every atomic commit should take at most 1 vblank interval. If any of
the commits take longer than that, there's a bug in either the kernel or
your testcase. Note that crc_start alone has a few vblank waits (due to
crc bugs on some platforms) which will break this.

Cheers, Daniel

> +
> + igt_assert_crc_equal(_crc, crc);
> +
> + test_fini(data, output, max_planes);
> + }
> +}
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
___
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

[Intel-gfx] [PATCH i-g-t] tests/kms_plane_multiple: CRC based atomic correctness test

[Mika Kahola]

This is a testcase with multiple planes. The idea here is the following

 - draw a uniform frame with blue color
 - grab crc for reference
 - put planes randomly on top with the same blue color
 - punch holes with black color into the primary framebuffer
 - ideally the planes should cover these holes so that the output is the
   identical to reference crc
 - composite all with one ioctl call
 - grab crc and verify that the reference crc is equal
 - repeat this for dozen iterations to maximize coverage

v3: Cleanup by removing separate plane array
For atomic, pass DRM_MODE_PAGE_FLIP_EVENT
Grab crc by using igt_pipe_crc_get_crc instead of igt_pipe_crc_collect_crc
Rename nplanes variable to max_planes
To optimize test execution, run iterations after the modeset

v2: Keep a logfile on random number seeds per subtest that are not skipped
due to unmet test requirements

Signed-off-by: Mika Kahola 
---
 tests/Makefile.sources |   1 +
 tests/kms_plane_multiple.c | 423 +
 2 files changed, 424 insertions(+)
 create mode 100644 tests/kms_plane_multiple.c

diff --git a/tests/Makefile.sources b/tests/Makefile.sources
index 6d081c3..ffd59c1 100644
--- a/tests/Makefile.sources
+++ b/tests/Makefile.sources
@@ -105,6 +105,7 @@ TESTS_progs_M = \
kms_pipe_color \
kms_pipe_crc_basic \
kms_plane \
+   kms_plane_multiple \
kms_properties \
kms_psr_sink_crc \
kms_render \
diff --git a/tests/kms_plane_multiple.c b/tests/kms_plane_multiple.c
new file mode 100644
index 000..0cb7552
--- /dev/null
+++ b/tests/kms_plane_multiple.c
@@ -0,0 +1,423 @@
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#include "igt.h"
+#include "drmtest.h"
+#include 
+#include 
+#include 
+#include 
+#include 
+
+IGT_TEST_DESCRIPTION("Test atomic mode setting with multiple planes ");
+
+#define SIZE 128
+
+typedef struct {
+   float red;
+   float green;
+   float blue;
+} color_t;
+
+typedef struct {
+   int drm_fd;
+   igt_display_t display;
+   igt_pipe_crc_t *pipe_crc;
+   igt_plane_t *plane[IGT_MAX_PLANES];
+   struct igt_fb fb[IGT_MAX_PLANES];
+} data_t;
+
+typedef struct {
+   data_t *data;
+   igt_crc_t reference_crc;
+} test_position_t;
+
+/* Command line parameters. */
+struct {
+   bool user_seed;
+   int seed;
+   bool user_logfile;
+   char logfile[SIZE];
+} opt = {
+   .user_seed = false,
+   .seed = 1,
+   .user_logfile = false,
+   .logfile = "kms_plane_multiple.log",
+};
+
+
+static int logwrite(const char *testname)
+{
+   time_t curr_time;
+   FILE *fid;
+   char *time_str;
+
+   fid = fopen(opt.logfile, "a");
+
+   if (fid == NULL) {
+   igt_debug("Could not open file %s\n", opt.logfile);
+   return -1;
+   }
+
+   curr_time = time(NULL);
+
+   time_str = ctime(_time);
+   time_str[strlen(time_str)-1] = '\0';
+
+   fprintf(fid, "%s: kms_plane_multiple --run-subtest %s --seed %d\n",
+   time_str, testname, opt.seed);
+
+   fclose(fid);
+
+   return 0;
+}
+
+/*
+ * Common code across all tests, acting on data_t
+ */
+static void test_init(data_t *data, enum pipe pipe)
+{
+   data->pipe_crc = igt_pipe_crc_new(pipe, INTEL_PIPE_CRC_SOURCE_AUTO);
+}
+
+static void test_fini(data_t *data, igt_output_t *output, int max_planes)
+{
+   igt_plane_set_fb(data->plane[IGT_PLANE_PRIMARY], NULL);
+
+   for (int i = IGT_PLANE_2; i <= max_planes; i++)
+   igt_plane_set_fb(data->plane[i], NULL);
+
+   /* reset the constraint on the pipe */
+   igt_output_set_pipe(output, PIPE_ANY);
+
+   igt_pipe_crc_free(data->pipe_crc);
+}
+
+static bool test_is_atomic(const char *testname)
+{
+   if (strncmp(testname, 

Re: [Intel-gfx] [PATCH i-g-t] tests/kms_plane_multiple: CRC based atomic correctness test

[Mika Kahola]

On Wed, 2016-10-12 at 14:54 +0200, Maarten Lankhorst wrote:
> Op 07-10-16 om 13:45 schreef Mika Kahola:
> > 
> > This is a testcase with multiple planes. The idea here is the
> > following
> > 
> >  - draw a uniform frame with blue color
> >  - grab crc for reference
> >  - put planes randomly on top with the same blue color
> >  - punch holes with black color into the primary framebuffer
> >  - ideally the planes should cover these holes so that the output
> > is the
> >    identical to reference crc
> >  - composite all with one ioctl call
> >  - grab crc and verify that the reference crc is equal
> >  - repeat this for dozen iterations to maximize coverage
> > 
> > Signed-off-by: Mika Kahola 
> > ---
> >  tests/Makefile.sources |   1 +
> >  tests/kms_plane_multiple.c | 332
> > +
> >  2 files changed, 333 insertions(+)
> >  create mode 100644 tests/kms_plane_multiple.c
> > 
> > diff --git a/tests/Makefile.sources b/tests/Makefile.sources
> > index 598ec6f..aed0f3a 100644
> > --- a/tests/Makefile.sources
> > +++ b/tests/Makefile.sources
> > @@ -105,6 +105,7 @@ TESTS_progs_M = \
> >     kms_pipe_color \
> >     kms_pipe_crc_basic \
> >     kms_plane \
> > +   kms_plane_multiple \
> >     kms_properties \
> >     kms_psr_sink_crc \
> >     kms_render \
> > diff --git a/tests/kms_plane_multiple.c
> > b/tests/kms_plane_multiple.c
> > new file mode 100644
> > index 000..153d6d1
> > --- /dev/null
> > +++ b/tests/kms_plane_multiple.c
> > @@ -0,0 +1,332 @@
> > +/*
> > + * Copyright © 2016 Intel Corporation
> > + *
> > + * Permission is hereby granted, free of charge, to any person
> > obtaining a
> > + * copy of this software and associated documentation files (the
> > "Software"),
> > + * to deal in the Software without restriction, including without
> > limitation
> > + * the rights to use, copy, modify, merge, publish, distribute,
> > sublicense,
> > + * and/or sell copies of the Software, and to permit persons to
> > whom the
> > + * Software is furnished to do so, subject to the following
> > conditions:
> > + *
> > + * The above copyright notice and this permission notice
> > (including the next
> > + * paragraph) shall be included in all copies or substantial
> > portions of the
> > + * Software.
> > + *
> > + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
> > EXPRESS OR
> > + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
> > MERCHANTABILITY,
> > + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO
> > EVENT SHALL
> > + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
> > DAMAGES OR OTHER
> > + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
> > ARISING
> > + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
> > OTHER DEALINGS
> > + * IN THE SOFTWARE.
> > + *
> > + */
> > +
> > +#include "igt.h"
> > +#include "drmtest.h"
> > +#include 
> > +#include 
> > +#include 
> > +#include 
> > +#include 
> > +
> > +#define SIZE 128
> > +
> > +typedef struct {
> > +   float red;
> > +   float green;
> > +   float blue;
> > +} color_t;
> > +
> > +typedef struct {
> > +   int drm_fd;
> > +   igt_display_t display;
> > +   igt_pipe_crc_t *pipe_crc;
> > +   igt_plane_t *primary;
> > +   igt_plane_t *sprite[IGT_MAX_PLANES-1];
> > +   struct igt_fb primary_fb;
> > +   struct igt_fb sprite_fb[IGT_MAX_PLANES-1];
> Single array, instead of primary/sprite separate? See also below for
> index change..
Indeed, it does look cleaner that way.
> > 
> > +} data_t;
> > +
> > +typedef struct {
> > +   data_t *data;
> > +   igt_crc_t reference_crc;
> > +} test_position_t;
> > +
> > +/*
> > + * Common code across all tests, acting on data_t
> > + */
> > +static void test_init(data_t *data, enum pipe pipe)
> > +{
> > +   data->pipe_crc = igt_pipe_crc_new(pipe,
> > INTEL_PIPE_CRC_SOURCE_AUTO);
> > +}
> > +
> > +static void test_fini(data_t *data, igt_output_t *output, int
> > nplanes)
> > +{
> > +   igt_plane_set_fb(data->primary, NULL);
> > +
> > +   for (int i = 0; i < nplanes; i++)
> > +   igt_plane_set_fb(data->sprite[i], NULL);
> > +
> > +   /* reset the constraint on the pipe */
> > +   igt_output_set_pipe(output, PIPE_ANY);
> > +
> > +   igt_pipe_crc_free(data->pipe_crc);
> > +}
> > +
> > +static void
> > +test_grab_crc(data_t *data, igt_output_t *output, enum pipe pipe,
> > +     color_t *color, uint64_t tiling, int commit,
> > +     igt_crc_t *crc /* out */)
> > +{
> > +   struct igt_fb fb;
> > +   drmModeModeInfo *mode;
> > +   igt_plane_t *primary;
> > +
> > +   igt_output_set_pipe(output, pipe);
> > +
> > +   primary = igt_output_get_plane(output, IGT_PLANE_PRIMARY);
> > +
> > +   mode = igt_output_get_mode(output);
> > +
> > +   igt_create_color_fb(data->drm_fd, mode->hdisplay, mode-
> > >vdisplay,
> > +   DRM_FORMAT_XRGB,
> > +   LOCAL_DRM_FORMAT_MOD_NONE,
> > +   color->red, 

Re: [Intel-gfx] [PATCH i-g-t] tests/kms_plane_multiple: CRC based atomic correctness test

[Maarten Lankhorst]

Op 07-10-16 om 13:45 schreef Mika Kahola:
> This is a testcase with multiple planes. The idea here is the following
>
>  - draw a uniform frame with blue color
>  - grab crc for reference
>  - put planes randomly on top with the same blue color
>  - punch holes with black color into the primary framebuffer
>  - ideally the planes should cover these holes so that the output is the
>identical to reference crc
>  - composite all with one ioctl call
>  - grab crc and verify that the reference crc is equal
>  - repeat this for dozen iterations to maximize coverage
>
> Signed-off-by: Mika Kahola 
> ---
>  tests/Makefile.sources |   1 +
>  tests/kms_plane_multiple.c | 332 
> +
>  2 files changed, 333 insertions(+)
>  create mode 100644 tests/kms_plane_multiple.c
>
> diff --git a/tests/Makefile.sources b/tests/Makefile.sources
> index 598ec6f..aed0f3a 100644
> --- a/tests/Makefile.sources
> +++ b/tests/Makefile.sources
> @@ -105,6 +105,7 @@ TESTS_progs_M = \
>   kms_pipe_color \
>   kms_pipe_crc_basic \
>   kms_plane \
> + kms_plane_multiple \
>   kms_properties \
>   kms_psr_sink_crc \
>   kms_render \
> diff --git a/tests/kms_plane_multiple.c b/tests/kms_plane_multiple.c
> new file mode 100644
> index 000..153d6d1
> --- /dev/null
> +++ b/tests/kms_plane_multiple.c
> @@ -0,0 +1,332 @@
> +/*
> + * Copyright © 2016 Intel Corporation
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining a
> + * copy of this software and associated documentation files (the "Software"),
> + * to deal in the Software without restriction, including without limitation
> + * the rights to use, copy, modify, merge, publish, distribute, sublicense,
> + * and/or sell copies of the Software, and to permit persons to whom the
> + * Software is furnished to do so, subject to the following conditions:
> + *
> + * The above copyright notice and this permission notice (including the next
> + * paragraph) shall be included in all copies or substantial portions of the
> + * Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
> + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
> + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER 
> DEALINGS
> + * IN THE SOFTWARE.
> + *
> + */
> +
> +#include "igt.h"
> +#include "drmtest.h"
> +#include 
> +#include 
> +#include 
> +#include 
> +#include 
> +
> +#define SIZE 128
> +
> +typedef struct {
> + float red;
> + float green;
> + float blue;
> +} color_t;
> +
> +typedef struct {
> + int drm_fd;
> + igt_display_t display;
> + igt_pipe_crc_t *pipe_crc;
> + igt_plane_t *primary;
> + igt_plane_t *sprite[IGT_MAX_PLANES-1];
> + struct igt_fb primary_fb;
> + struct igt_fb sprite_fb[IGT_MAX_PLANES-1];
Single array, instead of primary/sprite separate? See also below for index 
change..
> +} data_t;
> +
> +typedef struct {
> + data_t *data;
> + igt_crc_t reference_crc;
> +} test_position_t;
> +
> +/*
> + * Common code across all tests, acting on data_t
> + */
> +static void test_init(data_t *data, enum pipe pipe)
> +{
> + data->pipe_crc = igt_pipe_crc_new(pipe, INTEL_PIPE_CRC_SOURCE_AUTO);
> +}
> +
> +static void test_fini(data_t *data, igt_output_t *output, int nplanes)
> +{
> + igt_plane_set_fb(data->primary, NULL);
> +
> + for (int i = 0; i < nplanes; i++)
> + igt_plane_set_fb(data->sprite[i], NULL);
> +
> + /* reset the constraint on the pipe */
> + igt_output_set_pipe(output, PIPE_ANY);
> +
> + igt_pipe_crc_free(data->pipe_crc);
> +}
> +
> +static void
> +test_grab_crc(data_t *data, igt_output_t *output, enum pipe pipe,
> +   color_t *color, uint64_t tiling, int commit,
> +   igt_crc_t *crc /* out */)
> +{
> + struct igt_fb fb;
> + drmModeModeInfo *mode;
> + igt_plane_t *primary;
> +
> + igt_output_set_pipe(output, pipe);
> +
> + primary = igt_output_get_plane(output, IGT_PLANE_PRIMARY);
> +
> + mode = igt_output_get_mode(output);
> +
> + igt_create_color_fb(data->drm_fd, mode->hdisplay, mode->vdisplay,
> + DRM_FORMAT_XRGB,
> + LOCAL_DRM_FORMAT_MOD_NONE,
> + color->red, color->green, color->blue,
> + );
> +
> + igt_plane_set_fb(primary, );
> +
> + igt_display_commit2(>display, commit);
> +
> + igt_wait_for_vblank(data->drm_fd, pipe);
> +
> + igt_pipe_crc_collect_crc(data->pipe_crc, crc);
> +
> + igt_plane_set_fb(primary, NULL);
> +
> + igt_display_commit2(>display, commit);
> +
> + 

[Intel-gfx] [PATCH i-g-t] tests/kms_plane_multiple: CRC based atomic correctness test

[Mika Kahola]

This is a testcase with multiple planes. The idea here is the following

 - draw a uniform frame with blue color
 - grab crc for reference
 - put planes randomly on top with the same blue color
 - punch holes with black color into the primary framebuffer
 - ideally the planes should cover these holes so that the output is the
   identical to reference crc
 - composite all with one ioctl call
 - grab crc and verify that the reference crc is equal
 - repeat this for dozen iterations to maximize coverage

v2: Keep a logfile on random number seeds per subtest that are not skipped
due to unmet test requirements

Signed-off-by: Mika Kahola 
---
 tests/kms_plane_multiple.c | 421 +
 1 file changed, 421 insertions(+)
 create mode 100644 tests/kms_plane_multiple.c

diff --git a/tests/kms_plane_multiple.c b/tests/kms_plane_multiple.c
new file mode 100644
index 000..cafd409
--- /dev/null
+++ b/tests/kms_plane_multiple.c
@@ -0,0 +1,421 @@
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#include "igt.h"
+#include "drmtest.h"
+#include 
+#include 
+#include 
+#include 
+#include 
+
+IGT_TEST_DESCRIPTION("Test atomic mode setting with multiple planes ");
+
+#define SIZE 128
+
+typedef struct {
+   float red;
+   float green;
+   float blue;
+} color_t;
+
+typedef struct {
+   int drm_fd;
+   igt_display_t display;
+   igt_pipe_crc_t *pipe_crc;
+   igt_plane_t *primary;
+   igt_plane_t *sprite[IGT_MAX_PLANES-1];
+   struct igt_fb primary_fb;
+   struct igt_fb sprite_fb[IGT_MAX_PLANES-1];
+} data_t;
+
+typedef struct {
+   data_t *data;
+   igt_crc_t reference_crc;
+} test_position_t;
+
+/* Command line parameters. */
+struct {
+   bool user_seed;
+   int seed;
+   bool user_logfile;
+   char logfile[SIZE];
+} opt = {
+   .user_seed = false,
+   .seed = 1,
+   .user_logfile = false,
+   .logfile = "kms_plane_multiple.log",
+};
+
+
+static int logwrite(const char *testname)
+{
+   time_t curr_time;
+   FILE *fid;
+   char *time_str;
+
+   fid = fopen(opt.logfile, "a");
+
+   if (fid == NULL) {
+   igt_debug("Could not open file %s\n", opt.logfile);
+   return -1;
+   }
+
+   curr_time = time(NULL);
+
+   time_str = ctime(_time);
+   time_str[strlen(time_str)-1] = '\0';
+
+   fprintf(fid, "%s: kms_plane_multiple --run-subtest %s --seed %d\n",
+   time_str, testname, opt.seed);
+
+   fclose(fid);
+
+   return 0;
+}
+
+/*
+ * Common code across all tests, acting on data_t
+ */
+static void test_init(data_t *data, enum pipe pipe)
+{
+   data->pipe_crc = igt_pipe_crc_new(pipe, INTEL_PIPE_CRC_SOURCE_AUTO);
+}
+
+static void test_fini(data_t *data, igt_output_t *output, int nplanes)
+{
+   igt_plane_set_fb(data->primary, NULL);
+
+   for (int i = 0; i < nplanes; i++)
+   igt_plane_set_fb(data->sprite[i], NULL);
+
+   /* reset the constraint on the pipe */
+   igt_output_set_pipe(output, PIPE_ANY);
+
+   igt_pipe_crc_free(data->pipe_crc);
+}
+
+static void
+test_grab_crc(data_t *data, igt_output_t *output, enum pipe pipe,
+ color_t *color, uint64_t tiling, int commit,
+ igt_crc_t *crc /* out */)
+{
+   struct igt_fb fb;
+   drmModeModeInfo *mode;
+   igt_plane_t *primary;
+
+   igt_output_set_pipe(output, pipe);
+
+   primary = igt_output_get_plane(output, IGT_PLANE_PRIMARY);
+
+   mode = igt_output_get_mode(output);
+
+   igt_create_color_fb(data->drm_fd, mode->hdisplay, mode->vdisplay,
+   DRM_FORMAT_XRGB,
+   LOCAL_DRM_FORMAT_MOD_NONE,
+   color->red, color->green, color->blue,
+   );
+
+   

[Intel-gfx] [PATCH i-g-t] tests/kms_plane_multiple: CRC based atomic correctness test

[Mika Kahola]

This is a testcase with multiple planes. The idea here is the following

 - draw a uniform frame with blue color
 - grab crc for reference
 - put planes randomly on top with the same blue color
 - punch holes with black color into the primary framebuffer
 - ideally the planes should cover these holes so that the output is the
   identical to reference crc
 - composite all with one ioctl call
 - grab crc and verify that the reference crc is equal
 - repeat this for dozen iterations to maximize coverage

Signed-off-by: Mika Kahola 
---
 tests/Makefile.sources |   1 +
 tests/kms_plane_multiple.c | 332 +
 2 files changed, 333 insertions(+)
 create mode 100644 tests/kms_plane_multiple.c

diff --git a/tests/Makefile.sources b/tests/Makefile.sources
index 598ec6f..aed0f3a 100644
--- a/tests/Makefile.sources
+++ b/tests/Makefile.sources
@@ -105,6 +105,7 @@ TESTS_progs_M = \
kms_pipe_color \
kms_pipe_crc_basic \
kms_plane \
+   kms_plane_multiple \
kms_properties \
kms_psr_sink_crc \
kms_render \
diff --git a/tests/kms_plane_multiple.c b/tests/kms_plane_multiple.c
new file mode 100644
index 000..153d6d1
--- /dev/null
+++ b/tests/kms_plane_multiple.c
@@ -0,0 +1,332 @@
+/*
+ * Copyright © 2016 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#include "igt.h"
+#include "drmtest.h"
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#define SIZE 128
+
+typedef struct {
+   float red;
+   float green;
+   float blue;
+} color_t;
+
+typedef struct {
+   int drm_fd;
+   igt_display_t display;
+   igt_pipe_crc_t *pipe_crc;
+   igt_plane_t *primary;
+   igt_plane_t *sprite[IGT_MAX_PLANES-1];
+   struct igt_fb primary_fb;
+   struct igt_fb sprite_fb[IGT_MAX_PLANES-1];
+} data_t;
+
+typedef struct {
+   data_t *data;
+   igt_crc_t reference_crc;
+} test_position_t;
+
+/*
+ * Common code across all tests, acting on data_t
+ */
+static void test_init(data_t *data, enum pipe pipe)
+{
+   data->pipe_crc = igt_pipe_crc_new(pipe, INTEL_PIPE_CRC_SOURCE_AUTO);
+}
+
+static void test_fini(data_t *data, igt_output_t *output, int nplanes)
+{
+   igt_plane_set_fb(data->primary, NULL);
+
+   for (int i = 0; i < nplanes; i++)
+   igt_plane_set_fb(data->sprite[i], NULL);
+
+   /* reset the constraint on the pipe */
+   igt_output_set_pipe(output, PIPE_ANY);
+
+   igt_pipe_crc_free(data->pipe_crc);
+}
+
+static void
+test_grab_crc(data_t *data, igt_output_t *output, enum pipe pipe,
+ color_t *color, uint64_t tiling, int commit,
+ igt_crc_t *crc /* out */)
+{
+   struct igt_fb fb;
+   drmModeModeInfo *mode;
+   igt_plane_t *primary;
+
+   igt_output_set_pipe(output, pipe);
+
+   primary = igt_output_get_plane(output, IGT_PLANE_PRIMARY);
+
+   mode = igt_output_get_mode(output);
+
+   igt_create_color_fb(data->drm_fd, mode->hdisplay, mode->vdisplay,
+   DRM_FORMAT_XRGB,
+   LOCAL_DRM_FORMAT_MOD_NONE,
+   color->red, color->green, color->blue,
+   );
+
+   igt_plane_set_fb(primary, );
+
+   igt_display_commit2(>display, commit);
+
+   igt_wait_for_vblank(data->drm_fd, pipe);
+
+   igt_pipe_crc_collect_crc(data->pipe_crc, crc);
+
+   igt_plane_set_fb(primary, NULL);
+
+   igt_display_commit2(>display, commit);
+
+   igt_remove_fb(data->drm_fd, );
+}
+
+/*
+ * Multiple plane position test.
+ *   - We start by grabbing a reference CRC of a full blue fb being scanned
+ * out on the primary plane
+ *   - Then we scannout number of planes:
+ *  * the primary plane uses a blue fb with a black rectangle hole
+ *  * planes, on top of the primary plane, with a