On Thu, Oct 06, 2016 at 09:31:32AM +0300, Petri Latvala wrote:
> See tests/intel-ci/README for rationale and explanation.
>
> Signed-off-by: Petri Latvala
> ---
>
>
> The plans for CI are to use explicit test lists in the future, as laid
> out in this commit. Comments, acks, nacks, flames, etc would be very
> welcome.
>
> The only list so far is a fattened version of the currently run set,
> also known as the BAT set. The tests added are such that they all pass
> in preliminary test runs, and do it sufficiently quickly to still meet
> the time requirement of 15 minute runs.
>
> Having said that, the limit is getting a bit tight and some of these
> should be dropped before deploying to production. Comments on test
> overlap and such also appreciated.
I think we should split this into 2 patches: First switch the BAT test
list over to this file, then in a 2nd patch extend it.
-Daniel
>
>
>
>
>
> tests/intel-ci/README | 34 ++
> tests/intel-ci/fast-feedback.testlist | 611
> ++
> 2 files changed, 645 insertions(+)
> create mode 100644 tests/intel-ci/README
> create mode 100644 tests/intel-ci/fast-feedback.testlist
>
> diff --git a/tests/intel-ci/README b/tests/intel-ci/README
> new file mode 100644
> index 000..dc7e5c8
> --- /dev/null
> +++ b/tests/intel-ci/README
> @@ -0,0 +1,34 @@
> +This directory contains test lists that are used by Intel's CI. The
> +files are passed to piglit with the --test-list parameter directly.
> +
> +The test lists are contained in the IGT repository for several
> +reasons:
> +
> +- The lists stay synchronized with the IGT codebase.
> +- Public availability. Kernel developers can see what tests are run,
> + and can see what changes are done to the set, when, and why.
> +- Explicit test lists in general make it possible to implement a new
> + test without having it run in CI (accidentally or on purpose) before
> + CI is ready for it, and for CI to stop running a test without
> + removing the test or changing its name. The previous method of test
> + selection was based on name patterns.
> +
> +Changing the test lists should only happen with approval from the CI
> +people. If you make a new test that should be tested by CI regularly,
> +ask a maintainer (see MAINTAINERS).
> +
> +==
> +fast-feedback.testlist
> +==
> +
> +Fast-feedback is the testlist that is run for Patchwork and Trybot
> +testing. It is also run on various drm-intel-* branches.
> +
> +Time limit: Full run must complete in 15 minutes on all tested
> +platforms.
> +
> +Feature coverage: Should test all working features, with at least 1
> +test per feature.
> +
> +The string "basic" in a test name means the test probably belongs in
> +this list.
> diff --git a/tests/intel-ci/fast-feedback.testlist
> b/tests/intel-ci/fast-feedback.testlist
> new file mode 100644
> index 000..6f9349c
> --- /dev/null
> +++ b/tests/intel-ci/fast-feedback.testlist
> @@ -0,0 +1,611 @@
> +igt@core_auth@basic-auth
> +igt@core_auth@many-magics
> +igt@core_getclient
> +igt@core_get_client_auth@master-drop
> +igt@core_get_client_auth@simple
> +igt@core_getstats
> +igt@core_getversion
> +igt@core_prop_blob@basic
> +igt@core_prop_blob@blob-multiple
> +igt@core_prop_blob@blob-prop-core
> +igt@core_prop_blob@blob-prop-lifetime
> +igt@core_prop_blob@blob-prop-validate
> +igt@core_prop_blob@invalid-get-prop
> +igt@core_prop_blob@invalid-get-prop-any
> +igt@core_prop_blob@invalid-set-prop
> +igt@core_prop_blob@invalid-set-prop-any
> +igt@core_setmaster_vs_auth
> +igt@debugfs_emon_crash
> +igt@drm_import_export@import-close-race-flink
> +igt@drm_import_export@import-close-race-prime
> +igt@drm_read@empty-nonblock
> +igt@drm_read@fault-buffer
> +igt@drm_read@invalid-buffer
> +igt@drm_read@short-buffer-block
> +igt@drm_read@short-buffer-nonblock
> +igt@drm_vma_limiter
> +igt@drm_vma_limiter_cpu
> +igt@drv_debugfs_reader
> +igt@drv_getparams_basic@basic-eu-total
> +igt@drv_getparams_basic@basic-subslice-total
> +igt@drv_hangman@error-state-basic
> +igt@drv_hangman@error-state-debugfs-entry
> +igt@drv_hangman@error-state-sysfs-entry
> +igt@drv_module_reload_basic
> +igt@gem_bad_length
> +igt@gem_bad_reloc@negative-reloc-blt
> +igt@gem_bad_reloc@negative-reloc-bltcopy
> +igt@gem_bad_reloc@negative-reloc-bsd
> +igt@gem_bad_reloc@negative-reloc-bsd1
> +igt@gem_bad_reloc@negative-reloc-bsd2
> +igt@gem_bad_reloc@negative-reloc-default
> +igt@gem_bad_reloc@negative-reloc-lut-blt
> +igt@gem_bad_reloc@negative-reloc-lut-bsd
> +igt@gem_bad_reloc@negative-reloc-lut-bsd1
> +igt@gem_bad_reloc@negative-reloc-lut-bsd2
> +igt@gem_bad_reloc@negative-reloc-lut-default
> +igt@gem_bad_reloc@negative-reloc-lut-render
> +igt@gem_bad_reloc@negative-reloc-lut-vebox
> +igt@gem_bad_reloc@negative-reloc-render
> +igt@gem_bad_reloc@negative-reloc-vebox
> +igt@gem_basic@bad-close
> +igt@gem_basic@create-close
> +igt@gem_basic@create-fd-close
> +igt@gem_busy@basic-bus
