Hi Chad,
See comments below...
On Sunday, December 07, 2014 08:54:06 PM Chad Versace wrote:
> Google opensourced the drawElements Quality Product Testsuite (dEQP) as
> part of the Android Lollipop release. It's git repo lives in the Android
> tree at [https://android.googlesource.com/platform/external/deqp/].
>
> This patch adds a new module, deqp_gles3.py, that runs the dEQP-GLES3
> tests. It queries the 'deqp-gles3' executable at runtime for the list
> of testcases. The module attempts queries the 'deqp-gles3' executable
> only if the environment variable PIGLIT_DEQP_GLES3_EXE or the
> piglit.conf option deqp-gles3.exe is set.
>
> Why do we need to use Pigit as the testrunner for dEQP? (After all, dEQP has
> its own testrunner). Because dEQP runs all tests in a single process. If test
> 17530 of 55409 crashes, then the dEQP testrunner crashes and the remaining
> tests never run. Piglit doesn't suffer from that problem, because it runs each
> test as a separate process.
>
> Tested on Intel Ivybridge by running the command
> piglit run -t dEQP-GLES3/info/vendor \
> tests/deqp_gles3.py tests/quick.py results
> with and without PIGLIT_DEQP_GLES3_EXE set and with and without
> piglit.conf:deqp-gles.exe set.
>
> Cc: Dylan Baker
> Cc: Daniel Kurtz
> Cc: Ilja H. Friedel
> Signed-off-by: Chad Versace
> ---
>
> You can find this patch on top of my 'deqp-gles3' branch.
>
> git://github.com/chadversary/piglit refs/heads/deqp-gles3
>
> Daniel and Ilja, I CC'd you just so you would be aware of the dEQP/Piglit work
> Intel is doing. It would be nice to see it go into Chrome OS enventually.
>
>
> piglit.conf.example | 7 +++
> tests/deqp_gles3.py | 137
>
> 2 files changed, 144 insertions(+)
> create mode 100644 tests/deqp_gles3.py
>
> diff --git a/piglit.conf.example b/piglit.conf.example
> index a864c01..0ac0f4f 100644
> --- a/piglit.conf.example
> +++ b/piglit.conf.example
> @@ -33,6 +33,13 @@ bindir=/home/usr/oclconform
> testA
> testB
>
> +;[deqp-gles3]
> +;
> +; Path to the deqp-gles3 executable. You can also set this with the
> environment
> +; variable PIGLIT_DEQP_GLES3_EXE. Piglit will run the dEQP-GLES3 tests if and
> +; only if this option is set.
> +;exe=/home/knuth/deqp/modules/gles3/deqp-gles3
> +
> ; Section for specific oclconform test. One of these sections is required
> for
> ; each test list in the oclconform section and must be called:
> ; oclconform-$testname
> diff --git a/tests/deqp_gles3.py b/tests/deqp_gles3.py
> new file mode 100644
> index 000..422d1fa
> --- /dev/null
> +++ b/tests/deqp_gles3.py
> @@ -0,0 +1,137 @@
> +# Copyright 2014 Intel Corporation
> +#
> +# Permission is hereby granted, free of charge, to any person obtaining a
> copy
> +# of this software and associated documentation files (the "Software"), to
> deal
> +# in the Software without restriction, including without limitation the
> rights
> +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
> +# copies of the Software, and to permit persons to whom the Software is
> +# furnished to do so, subject to the following conditions:
> +#
> +# The above copyright notice and this permission notice shall be included in
> +# all copies or substantial portions of the Software.
> +#
> +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
> +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
> FROM,
> +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
> THE
> +# SOFTWARE.
> +
> +import ConfigParser
> +import os
> +import shutil
> +import subprocess
> +import sys
> +import tempfile
> +
> +import framework.profile
> +
> +
> +__all__ = ['profile']
> +
> +
> +_deqp_gles3_exe = None
> +
> +
> +def get_option(env_varname, config_option):
> +"""Query the given environment variable and then piglit.conf for the
> +option. Return None if the option is unset.
> +"""
> +
> +opt = os.environ.get(env_varname, None)
> +if opt is not None:
> +return opt
> +
> +try:
> +opt = framework.core.PIGLIT_CONFIG.get(config_option[0],
> + config_option[1])
> +except ConfigParser.NoSectionError, ConfigParser.NoOptionError:
> +pass
> +
> +return opt
> +
> +
> +def get_deqp_gles3_exe():
> +"""Get path to the deqp-gles3 executable. Idempotent."""
> +global _deqp_gles3_exe
> +
> +if _deqp_gles3_exe is None:
> +_deqp_gles3_exe = get_option('PIGLIT_DEQP_GLES3_EXE',
> + ('deqp-gles3', 'exe'))
> +
> +return _deqp_gles3_exe
> +
> +
> +def gen_caselist_txt():
> +"""Generate dEQP-GLES
