Re: [Piglit] [PATCH] Add test for external sampler with 2D tex

2019-02-02 Thread Mark Janes
FWIW, the skqp test is run constantly in mesa ci: https://mesa-ci.01.org/mesa_master/builds/15138/results/174306816 Aditya Swarup writes: > Commit 649c149dbb96ac367 removed the test for intel external sampler > with dma only because of failure due to commit a5c39ed974402c - > ("i965: Lift rest

Re: [Piglit] [PATCH v2] miptree: test ability to use upside down miptree

2018-12-03 Thread Mark Janes
It's preferable to push piglit tests first, and produce the CI test failure. CI staff will track the failure, and attribute it to the piglit patch. When the subsequent mesa patch is pushed, we track the resolution. The process gives us artifacts in git that help to analyze driver differences bet

Re: [Piglit] [PATCH] arb_texture_view: Test interaction with ARB_shader_image_load_store

2018-11-16 Thread Mark Janes
Thanks for writing this test. I checked it in CI and pushed it. -Mark Danylo Piliaiev writes: > Hello, > > Since the patch which fixes the issue got pushed in > f9fd0cf4790cb2a530e75d1a2206dbb9d8af7cb2 > could this test be reviewed/pushed? > > Thanks! > > On 10/24/18 2:17 PM, Danylo Piliaiev w

Re: [Piglit] [PATCH] vulkan: Prefix group names for VkRunner tests with “vulkan”

2018-11-13 Thread Mark Janes
Let me know when this is pushed, and I'll enable the tests in i965 CI. Dylan Baker writes: > Quoting Neil Roberts (2018-11-13 12:50:16) >> This will make it easier to distinguish tests written for Vulkan. It >> makes a bit of an inconsistency because the GL and CL tests don\u2019t have >> any pr

Re: [Piglit] [PATCH] vulkan: Prefix group names for VkRunner tests with “vulkan”

2018-11-13 Thread Mark Janes
Reviewed-by: Mark Janes Tested-by: Mark Janes Neil Roberts writes: > This will make it easier to distinguish tests written for Vulkan. It > makes a bit of an inconsistency because the GL and CL tests don’t have > any prefix. Ideally maybe we would add a prefix for those too, but &

Re: [Piglit] [PATCH v1] opengl/intel/ppgtt: memory alignment test.

2018-10-24 Thread Mark Janes
Tested-by: Mark Janes Sergii Romantsov writes: > Intel ppgtt-model requires memory addresses to be aligned by page size. > Test explores cases with cached memory-buckets and sizes above cached. > > CC: Kenneth Graunke > CC: Lionel G Landwerlin > CC: Chris Wilson

Re: [Piglit] [PATCH] Cmake: Install shader_source files

2018-08-28 Thread Mark Janes
Reviewed-by: Mark Janes Tested-by: Mark Janes Dylan Baker writes: > Otherwise these tests fail in odd ways when running from an installed > instance. > > Cc: Alejandro Piñeiro > Cc: Mark Janes > Cc: Clayton Craft > --- > CMakeLists.txt | 2 +- > 1 file changed,

Re: [Piglit] [PATCH 1/2] multisample-accuracy: Add an error margin for lit/unlit pixels

2018-07-20 Thread Mark Janes
Hi Neil, We still get intermittent failures with this threshold, and have historically disabled the multisample accuracy tests in CI. Can we relax the threshold a bit more? -Mark Neil Roberts writes: > The unlit and totally_lit stats are supposed to count the pixels where > either a primitive

Re: [Piglit] [PATCH 00/35] Serialize profiles into XML at build time

2018-04-10 Thread Mark Janes
Dylan Baker writes: > Quoting Marek Olšák (2018-04-10 14:22:10) >> On Tue, Apr 10, 2018 at 2:15 PM, Dylan Baker wrote: >> >> Quoting Eric Anholt (2018-04-09 17:10:35) >> > Marek Olšák writes: >> > >> > > Is this use case affected? >> > > >> > > piglit run --deqp-mustpas

Re: [Piglit] [PATCH] tests: Added a new GLES test profile for the Khronos CTS runner

2018-02-14 Thread Mark Janes
to run the renamed > tests in the master branch of the opensourced Khronos CTS tests at: > https://github.com/KhronosGroup/VK-GL-CTS > > Cc: Mark Janes > Cc: Dylan Baker > Cc: Juan A. Suarez Romero > Signed-off-by: Andres Gomez > --- > tests/khr_gles.py | 88 > +++

Re: [Piglit] [PATCH v2] tests: Added a couple of new test profiles for the Khronos CTS runner

2017-06-13 Thread Mark Janes
Tested-by: Mark Janes Andres Gomez writes: > After: > https://github.com/KhronosGroup/VK-GL-CTS/commit/af8c22a343ee2c230488f6de71b36dc3070b2024 > > OpenGL GL*-CTS case lists were renamed to KHR-GL*. > > We want to keep the existing profiles so we are able to keep running &g

Re: [Piglit] [PATCH 0/2] Rewind the changes to the Khronos GL CTS profiles

2017-06-06 Thread Mark Janes
Tested-by: Mark Janes Andres Gomez writes: > Recently I added some changes in the cts_gl* profiles to adapt to > renames in the opensourced Khronos CTS at: > https://github.com/KhronosGroup/VK-GL-CTS > > Unfortunately, I didn't foresee that this change would break backward

Re: [Piglit] [PATCH 6/8] fbo: Use renderbuffers instead of textures

2017-06-02 Thread Mark Janes
Hi Ian, This test fails on g33. Was that expected? -Mark Ian Romanick writes: > From: Ian Romanick > > This allows fbo-blit-stretch to run on platforms that support > ARB_framebuffer_object but not ARB_texture_non_power_of_two. > > Coneptually similar to 7b3f6d5. > > Signed-off-by: Ian Roman

Re: [Piglit] [PATCH 1/2] shaders: set missing blue color to glsl-const-builtin-distance.shader_test

2017-05-22 Thread Mark Janes
Series is: Tested-by: Mark Janes Samuel Pitoiset writes: > Since b741bc770 ("parser_utils: do not overwrite value when no > digits are found "), the blue color has to be explicitly initialized > to zero. > > Cc: Mark Janes > Signed-off-by: Samuel Pitoiset >

Re: [Piglit] [PATCH] getteximage-formats: Disable dithering during glDrawPixels

2017-01-30 Thread Mark Janes
Tested-by: Mark Janes Chad Versace writes: > The test generates a mipmap by "copying" a client-array of pixels to > miplevel 0 with glDrawPixels, then calling glGenerateMipmap. The test > expects the rendered level 0 to match the client-array source. Dithering > during

[Piglit] [PATCH] deqp: Search stdout and stderr for X connection failure

2017-01-06 Thread Mark Janes
Recent versions of the GL CTS report X connection errors on standard out. Support all variants by checking both stdout and stderr for the error string. --- framework/test/deqp.py | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/framework/test/deqp.py b/framework/test/deqp

Re: [Piglit] [PATCH] arb_shader_image_load_store/shader-mem-barrier: Add explicit memoryBarrier to volatile subtest.

2016-12-30 Thread Mark Janes
Tested-by: Mark Janes Francisco Jerez writes: > This makes sure that reads and writes of the volatile-qualified images > occur in the expected order. It's unclear whether the GLSL volatile > qualifier was intended to have the required memory ordering > implications as it

Re: [Piglit] [PATCH] fbo-maxsize: Try to avoid allocating miplevels.

2016-12-10 Thread Mark Janes
This patch allows the test to complete on a Braswell system with 1.5GB memory. Without it, the OOM killer is dispatched. Tested-by: Mark Janes Kenneth Graunke writes: > A 16384x16384 RGBA buffer takes 1GB of RAM. If the driver allocates > mipmap levels, that increases our s

Re: [Piglit] [PATCH] util: link with WAYLAND_LIBRARIES when building for Wayland

2016-11-30 Thread Mark Janes
Tested-by: Mark Janes Tapani Pälli writes: > Signed-off-by: Tapani Pälli > Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=98769 > --- > tests/util/CMakeLists.txt | 5 + > 1 file changed, 5 insertions(+) > > diff --git a/tests/util/CMakeLists.txt b/test

Re: [Piglit] [PATCH v2 0/2] eventloop for Wayland platform

2016-11-18 Thread Mark Janes
This series breaks my compilation: gcc -m64 -Wall -std=gnu99 -Werror=vla -Werror=pointer-arith -Werror=variadic-macros -g target_api/gl/tests/bugs/CMakeFiles/point-sprite.dir/point-sprite.c.o -o bin/point-sprite -rdynamic lib/libpiglitutil_gl.so.0 -lGL lib/libpiglitutil.so.0 -lrt -ldl -lx

Re: [Piglit] [PATCH] framework/profile: don't deepcopy when copying profiles

2016-11-15 Thread Mark Janes
Tested-by: Mark Janes Reviewed-by: Mark Janes Dylan Baker writes: > This reduces memory consumption by a lot (about 60% on my system), which > fixes some tests that fail sporadically on low memory systems. > > cc: Mark Janes > Signed-off-by: Dylan Baker > --- > fra

[Piglit] [PATCH v2] framework/backends/json: support non-piglit junit files

2016-10-26 Thread Mark Janes
The junit loader is unnecessarily strict with the input that it accepts. It expects input generated by piglit, but can be made to handle junit from other test suites like crucible. v2: Remove unnecessary initializers Conditionally parse command for piglit tests --- framework/backends/junit.p

Re: [Piglit] [PATCH] framework/backends/json: support non-piglit junit files

2016-10-26 Thread Mark Janes
Dylan Baker writes: > Quoting Mark Janes (2016-10-26 16:30:12) >> The junit loader is unnecessarily strict with the input that it >> accepts. It expects input generated by piglit, but can be made to >> handle junit from other test suites like crucible. >> --- >

[Piglit] [PATCH] framework/backends/json: support non-piglit junit files

2016-10-26 Thread Mark Janes
The junit loader is unnecessarily strict with the input that it accepts. It expects input generated by piglit, but can be made to handle junit from other test suites like crucible. --- framework/backends/junit.py | 28 1 file changed, 20 insertions(+), 8 deletions(-)

Re: [Piglit] [PATCH 0/3] Fix summary breakage

2016-10-25 Thread Mark Janes
; Mark and Michel, > I'd appreciate if you could test this and make sure it fixes your respective > problems. Tested-by: Mark Janes > Dylan Baker (3): > piglit: Only catch parsed.func AttributeError > framework: Properly load JSON into internal representation >

Re: [Piglit] Newbie question: command to run tests that passed before?

2016-09-07 Thread Mark Janes
Dan Kegel writes: > Hrm. A solution that involved Jenkins would be way overkill. > I'm just looking for the little core idiom for running just the > test of tests that passed in the, um, past. Even if you don't want to set up a CI with jenkins, you can refer to the automation that I've written

Re: [Piglit] [PATCH v3 0/4] Add support for subtests in JUnit as nested testsuite elements

2016-08-31 Thread Mark Janes
. I don't need to run a single subtest. Series is Reviewed-by: Mark Janes Tested-by: Mark Janes Dylan Baker writes: > This series refactors the JUnit backend so that the _write attribute is > actually a callable class, instead of a method. This gives greater > flexibility to h

Re: [Piglit] [PATCH] Tests: Add integration for dEQP-EGL tests

2016-08-09 Thread Mark Janes
Some minor nits on the comments below. Reviewed-by: Mark Janes Tested-by: Mark Janes Dylan Baker writes: > This adds the necessary bits to make dEQP-EGL work. > > Signed-off-by: Dylan Baker > CC: Mark Janes > --- > piglit.conf.example | 10 ++ > tes

Re: [Piglit] [PATCH 2/2] shader_runner: Fix get_ints on 32-bit systems.

2016-06-06 Thread Mark Janes
) strtoll(*text, &endptr, 0); > - if (errno == ERANGE) { > + long long value = strtoll(*text, &endptr, 0); > + if (errno == ERANGE || (unsigned long long) value > > 0xull) { ^^^

Re: [Piglit] [PATCH] tests: Add integration with Khronos CTS OpenGL runner

2016-06-02 Thread Mark Janes
deqp.iter_deqp_test_cases( >> - deqp.gen_caselist_txt(_CTS_BIN, 'ESEXT-CTS-cases.txt', >> - _EXTRA_ARGS)), >> +deqp.gen_caselist_txt(_CTS_BIN, 'GL45-CTS-cases.txt', >> _EXTRA_ARGS)), >> ),

Re: [Piglit] Nearly finished: shader_runner running THOUSANDS of tests per process

2016-05-27 Thread Mark Janes
Marek Olšák writes: > On Fri, May 27, 2016 at 3:18 AM, Mark Janes wrote: >> Marek Olšák writes: >> >>> On Mon, Apr 18, 2016 at 6:45 PM, Dylan Baker >>> wrote: >>>> Quoting Marek Olšák (2016-04-16 15:16:34) >>>>> Hi, >>>&

Re: [Piglit] Nearly finished: shader_runner running THOUSANDS of tests per process

2016-05-26 Thread Mark Janes
Marek Olšák writes: > On Mon, Apr 18, 2016 at 6:45 PM, Dylan Baker wrote: >> Quoting Marek Olšák (2016-04-16 15:16:34) >>> Hi, >>> >>> This makes shader_runner very fast. The expected result is 40% >>> decrease in quick.py running time, or a 12x faster piglit run if you >>> run shader tests alon

Re: [Piglit] [PATCH] framework/backends/junit: Fix invalid JUnit output

2016-05-06 Thread Mark Janes
Reviewed-by: Mark Janes Dylan Baker writes: > In commit b0d05323e code was added to produce clearer error messages for > tests who's status changed from crash to fail and vice versa. There is a > not so subtle bug in that patch, it adds a "crash" element, but that >

Re: [Piglit] [PATCH 2/2] framework/glslparsertest: Don't add exclude extensions to the required list

2016-04-13 Thread Mark Janes
Reviewed-by: Mark Janes Dylan Baker writes: > glslparsertest has a syntax for the 'require_extensions' option that is > "!", when this is specified it means "This extensions is > not supported". There is a bug in the fast skip code in > glsl_

Re: [Piglit] [PATCH] deqp: Rerun if tests fail to connect to X

2016-03-28 Thread Mark Janes
Reviewed-by: Mark Janes Dylan Baker writes: > A pretty common error it seems is to get a failure to open display when > using X11. This adds an extra method to the DEQPBaseTest class that > reruns the test is it finds that error in the stderr. This affects only > deqp based suite

Re: [Piglit] [PATCH] framework: Add handler for failure and error when expecting the other

2016-03-19 Thread Mark Janes
I don't need to see a v2 Acked-by: Mark Janes Dylan Baker writes: > Quoting Mark Janes (2016-03-15 17:40:16) >> Dylan Baker writes: >> >> > Right now the JUnit backend knows what to do with a failure when it >> > expects one (or a crash), but no

Re: [Piglit] [PATCH] deqp-integration: Handle ResourceError.

2016-03-15 Thread Mark Janes
Mark Janes writes: > This patch doesn't fix the issue I see on HSW with Resource errors. I take back this statement. I was confused because the tests went from fail to crash, but there was no indication of this discrepancy in the output. Thanks for your recent patch to make this mo

Re: [Piglit] [PATCH] framework: Add handler for failure and error when expecting the other

2016-03-15 Thread Mark Janes
Dylan Baker writes: > Right now the JUnit backend knows what to do with a failure when it > expects one (or a crash), but not what to do when it expects a failure > and gets a crash (or vice versa). > > This patch teaches it what to do in that case, mark the test as a > failure and give a message

Re: [Piglit] [PATCH] deqp-integration: Handle ResourceError.

2016-03-15 Thread Mark Janes
This patch doesn't fix the issue I see on HSW with Resource errors. Dylan Baker writes: > This is a previously unseen error type generated on by the Vulkan CTS > (which is dEQP based). > > Signed-off-by: Dylan Baker > --- > > Mark, I think this will fix the that jenkins bug we're seeing. > > f

Re: [Piglit] [PATCH] Fix invalid glShaderSource call in pipeline_stats_comp.

2016-02-10 Thread Mark Janes
Reviewed-by: Mark Janes Kenneth Graunke writes: > The test was passing a pointer to an uninitialized integer value as the > glShaderSource length parameter. This is meant to be an array > containing the length of each shader string. > > Instead, just pass NULL, which makes th

Re: [Piglit] [PATCH] piglit-fbo: check the proper extension dependency when attaching a color texture

2016-01-15 Thread Mark Janes
Alejandro Piñeiro writes: > On 15/01/16 13:07, Mark Janes wrote: >> This patch causes piglit.spec.ext_framebuffer_multisample.accuracy to >> fail on g33 and g965. Is that what you expected? > > No, I was not expecting any regression. It seemed a trivial change, > ba

Re: [Piglit] [PATCH] piglit-fbo: check the proper extension dependency when attaching a color texture

2016-01-15 Thread Mark Janes
This patch causes piglit.spec.ext_framebuffer_multisample.accuracy to fail on g33 and g965. Is that what you expected? Alejandro Piñeiro writes: > attach_color_texture uses TEXTURE_RECTANGLE, so it should check > for GL_ARB_texture_rectangle. > > attach_multisample_color_texture uses multisampl

Re: [Piglit] [PATCH] igt: Restore handling of special igt error codes

2015-12-16 Thread Mark Janes
Reviewed-by: Mark Janes Daniel Vetter writes: > In > > commit 5cbc1834fd47e7f475afc85e1638762e5b221a81 > Author: Dylan Baker > Date: Mon Dec 14 15:34:11 2015 -0800 > > framework/test/base.py: Handle fail cases for tests. > > the Test baseclass was made

Re: [Piglit] [PATCH 2/2] framework/test/base.py: Handle fail cases for tests.

2015-12-14 Thread Mark Janes
Reviewed-by: Mark Janes baker.dyla...@gmail.com writes: > From: Dylan Baker > > I'm going to admit I'm a bit puzzled how this could have slipped through > without being caught (I'm guessing an unrelated change uncovered this). > But basically if a test doesn

Re: [Piglit] [PATCH 0/2] Handle subtests in JUnit output

2015-12-10 Thread Mark Janes
There are a few things with this patch that complicate my use of junit: 1) the stdout for the parent test is in the stored in the testsuite tag in the junit xml. However, I can't access this data from the jenkins ui. If a subtest fails, I won't be able to write up a bug from the Jen

Re: [Piglit] [PATCH] arb_draw_indirect: require extension in shared-binding test

2015-11-25 Thread Mark Janes
Reviewed-by: Mark Janes Tapani Palli writes: > Signed-off-by: Tapani Pälli > Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=93093 > --- > tests/spec/arb_draw_indirect/draw-arrays-shared-binding.c | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/tests/s

Re: [Piglit] [PATCH 00/13] Add more arb_shader_storage_buffer_object tests

2015-10-16 Thread Mark Janes
Emil Velikov writes: > On 25 September 2015 at 16:40, Mark Janes wrote: >> With the push of SSBO support in mesa, one of these tests continues to >> fail, but only on broadwell: >> >> piglit.spec.arb_shader_storage_buffer_object.array-ssbo-binding >> >>

Re: [Piglit] [PATCH 1/3] framework/backends/junit.py: restore time for stderr

2015-10-15 Thread Mark Janes
ames The transformation is not reversible, eg: test/foo.bar/baz -> test.foo.bar.baz -> test/foo/bar/baz This doesn't affect my use case, though. Reviewed-by: Mark Janes > @@ -264,6 +267,15 @@ def _load(results_file): > result.command = out[0] > resu

Re: [Piglit] [PATCH 2/3] framework/backends/junit.py: add calculate_group_totals() to load

2015-10-15 Thread Mark Janes
Reviewed-by: Mark Janes baker.dyla...@gmail.com writes: > From: Dylan Baker > > This calculates the group totals on load (this is stored in the json), > which is required to get proper totals information. > --- > framework/backends/junit.py | 2 ++

Re: [Piglit] Allow junit to be used for summary generation

2015-10-09 Thread Mark Janes
Jose Fonseca writes: > On 09/10/15 01:21, baker.dyla...@gmail.com wrote: >> This series updates the junit backend to allow it to properly load junit >> and convert it back into piglit's internal representation, thus allowing >> it to be summarized using the piglit summary tools >> >> There is sti

Re: [Piglit] Allow junit to be used for summary generation

2015-10-09 Thread Mark Janes
Dylan Baker writes: > On Thu, Oct 8, 2015 at 6:09 PM, Mark Janes wrote: > >> baker.dyla...@gmail.com writes: >> >> > This series updates the junit backend to allow it to properly load junit >> > and convert it back into piglit's internal representatio

Re: [Piglit] Allow junit to be used for summary generation

2015-10-08 Thread Mark Janes
baker.dyla...@gmail.com writes: > This series updates the junit backend to allow it to properly load junit > and convert it back into piglit's internal representation, thus allowing > it to be summarized using the piglit summary tools > > There is still some work that needs to be done beyond this,

Re: [Piglit] [RESEND PATCH 1/4] framework: Add a TimeAttribute class

2015-10-08 Thread Mark Janes
Series is Reviewed-by: Mark Janes dylanx.c.ba...@intel.com writes: > From: Dylan Baker > > This class will provide a new interface for time, one that stores the > start and stop times, rather than the total time. It then provides > methods for getting the total and the delta

Re: [Piglit] [PATCH] glsl-es: Verify restrictions on global variable initializers

2015-10-07 Thread Mark Janes
Reviewed-by: Mark Janes Tested-by: Mark Janes Ian Romanick writes: > From: Ian Romanick > > Section 4.3 (Storage Qualifiers) of the OpenGL ES 1.00.17 spec says: > > "Declarations of globals without a storage qualifier, or with just > the const qualifier, may i

Re: [Piglit] [PATCH 1/2] framework: add exception back to TestResult class

2015-10-01 Thread Mark Janes
patch 1 is Reviewed-by: Mark Janes Tested-by: Mark Janes Dylan Baker writes: > Like command, this somehow didn't get moved to the new TestResult class. > Generally this isn't a problem since this is a very uncommon path, but > in some cases this does get hit, and it would

Re: [Piglit] [PATCH] tests: Add integration for khronos CTS runner.

2015-09-28 Thread Mark Janes
Reviewed-by: Mark Janes Dylan Baker writes: > This adds support for running Khronos' deqp-based conformance suite with > piglit. This gives access to all of the powerful features of pigilt, > per-process tests, the junit and json backends, the summary tools, and > the familia

Re: [Piglit] [PATCH 00/13] Add more arb_shader_storage_buffer_object tests

2015-09-25 Thread Mark Janes
Emil Velikov writes: > On 25 September 2015 at 16:40, Mark Janes wrote: >> With the push of SSBO support in mesa, one of these tests continues to >> fail, but only on broadwell: >> >> piglit.spec.arb_shader_storage_buffer_object.array-ssbo-binding A subsequent run o

Re: [Piglit] [PATCH 00/13] Add more arb_shader_storage_buffer_object tests

2015-09-25 Thread Mark Janes
With the push of SSBO support in mesa, one of these tests continues to fail, but only on broadwell: piglit.spec.arb_shader_storage_buffer_object.array-ssbo-binding Standard Output /tmp/build_root/m64/lib/piglit/bin/arb_shader_storage_buffer_object-array-ssbo-binding -auto -fbo pigl

Re: [Piglit] [PATCH] gl-3.2: fix layered-rendering test

2015-08-18 Thread Mark Janes
Tested-by: Mark Janes Timothy Arceri writes: > Samples must be greater than 0 and color formats must match. > > Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=91670 > --- > tests/spec/gl-3.2/layered-rendering/framebuffertexture.c | 4 ++-- > 1 file changed, 2 insertio

Re: [Piglit] [PATCH v2] arb_program_interface_query: fix getprogramresourceiv errors

2015-08-12 Thread Mark Janes
Tested-by: Mark Janes Tapani Pälli writes: > Move IS_PER_PATCH as part of tessellation queries, require > GL_ARB_compute_shader for compute shader enum queries. > > Strictly speaking, spec would allow to query GL_REFERENCED_BY_COMPUTE_SHADER > without having compute sh

Re: [Piglit] [PATCH 2/2] arb_gpu_shader5: Fix rounding instability in UBO and sampler indexing vs-nonuniform-control-flow tests.

2015-06-30 Thread Mark Janes
Tested-by: Mark Janes Francisco Jerez writes: > Fixes a rounding instability that would cause shader_runner to probe > pixels offset by one for some points close to the right edge of the > window on systems using x87 floating point arithmetic with certain > compiler versions (the

[Piglit] [PATCH v2] Ignore piglit warning status in JUnit

2015-06-26 Thread Mark Janes
JUnit has no concept of "warning". It supports the following statuses: - skip - success - fail - error A test which emits a warning is more accurately represented as "success" in JUnit. v2: Continue to report failure for "dmesg-warn", which is more serious than "warn". (from Ilia Mirki

Re: [Piglit] [PATCH] Interpret dEQP "QualityWarning" as pass

2015-06-26 Thread Mark Janes
Ilia Mirkin writes: > On Fri, Jun 26, 2015 at 2:44 PM, Dylan Baker wrote: >>> That's fine, but then that's the justification, not "junit interprets >>> warn as fail". Note that some (regular) piglit tests will also emit >>> "warn", and they don't mean "fail". I believe that junit needs to be >>>

Re: [Piglit] [PATCH] Ignore piglit warning status in JUnit

2015-06-26 Thread Mark Janes
45, G965: !opengl 1_1.teximage-colors gl_r32f !opengl 1_1.teximage-colors gl_rg32f !opengl 1_1.teximage-colors gl_rgb32f !opengl 1_1.teximage-colors gl_rgba32f No output was provided for these warnings, so they may have have previously been dmesg-warn. > > On Fri, Jun 26, 2015 at 1:40

[Piglit] [PATCH] Ignore piglit warning status in JUnit

2015-06-26 Thread Mark Janes
JUnit has no concept of "warning". It supports the following statuses: - skip - success - fail - error dEQP has been found to intermittently emit warnings for passed tests, and this status is accurately represented in piglit json. However, current JUnit transforms them into failures. A tes

Re: [Piglit] [PATCH] Interpret dEQP "QualityWarning" as pass

2015-06-26 Thread Mark Janes
Ilia Mirkin writes: > On Thu, Jun 25, 2015 at 8:52 PM, Mark Janes wrote: >> Many tests in the functional.fbo.completeness.attachment_combinations >> report "QualityWarning" intermittently. This particular warning is >> emitted because dEQP attempted to cr

[Piglit] [PATCH] Interpret dEQP "QualityWarning" as pass

2015-06-25 Thread Mark Janes
Many tests in the functional.fbo.completeness.attachment_combinations report "QualityWarning" intermittently. This particular warning is emitted because dEQP attempted to create an unsupported framebuffer. "warning" is interpreted as "failure" in junit, but this output from dEQP is clearly not a

Re: [Piglit] [PATCH] framework/test/deqp: handle deqp extra args properly in more cases

2015-06-22 Thread Mark Janes
Reviewed-by: Mark Janes Dylan Baker writes: > Currently we blindly pass deqp extra args to each test case, and only to > the test case. > > There are two cases where this is problematic: > 1) if the extra args need to be passed to the deqp command that >generates the t

Re: [Piglit] [PATCH] glsl-es-3.10: dont run gles tests where restrictions apply

2015-06-17 Thread Mark Janes
Tested-by: Mark Janes Timothy Arceri writes: > In GLES vertex shader outputs cant be arrays of structs or structs that > contain arrays or structs. > > The GLSL ES 3.0 spec says: > > "Vertex output variables output per-vertex data and are declared > using the ou

Re: [Piglit] [PATCH] dispatch: Fix lookup for core OpenGL symbols

2015-04-30 Thread Mark Janes
Tested-by: Mark Janes Chad Versace writes: > The following commit regressed lookup of some new OpenGL symbols: > > commit cb334db3f2bb7ddf9b2f43641f898896f99c69b1 > Author: Daniel Kurtz > Date: Thu Oct 16 21:36:01 2014 +0800 > Subject: dispatch: Use dl

Re: [Piglit] [PATCH] glsl-deriv-varyings: Migrated to shader_runner and corrected test expectations on deriv + abs cases

2015-04-28 Thread Mark Janes
Tested-by: Mark Janes Andres Gomez writes: > Old glsl-deriv-varyings test has been migrated to > shader_runner and, also, splitted in four > different tests: > * glsl-derivs-varyings > * glsl-derivs-sign > * glsl-derivs-abs > * glsl-derivs-abs-sign > > In addition,

Re: [Piglit] [PATCH] junit.py: Escape `api`/`search` in test's classname too.

2015-04-27 Thread Mark Janes
Reviewed-by: Mark Janes Jose Fonseca writes: > If `api` appears in the middle of the test classname, like it happens > with all `spec/glsl-1.20/api/getactiveattrib *` tests, then Jenkins will > intercept it making it impossible to see individual tests results. > Therefore, j

Re: [Piglit] [PATCH 13/14] dispatch: Use dlsym to lookup core symbols for EGL

2015-04-27 Thread Mark Janes
Adding the list. Mark Janes writes: > Hi Chad, > > This patch was recently pushed, and it caused piglit failures for > arb_pipeline_statistics_query: > > > Standard Output > > /tmp/build_root/m32/lib/piglit/bin/arb_pipeline_statistics_q

Re: [Piglit] Add incomplete file for tests as they start

2015-04-15 Thread Mark Janes
Series is Reviewed-by: Mark Janes Dylan Baker writes: > One of the more interesting problems for piglit, especially with > hardware bring-up where bad code may result in the system crashing, is > to know what test actually crashed the system. > > This series is meant to addre

Re: [Piglit] [PATCH] glsl-1.50: check read only interfaces are read only

2015-03-27 Thread Mark Janes
Reviewed-by: Mark Janes Timothy Arceri writes: > Test results: > > AMD Radeon HD 6670 - Catalyst 14.501.1003 OpenGL 4.4 > > interface-block-instance-name-input-read-only.frag - pass > interface-block-input-read-only.frag - pass > interface-block-instance-name-uniform-

Re: [Piglit] [PATCH] tests/all.py: Fix typo that switched arb with ext

2015-03-05 Thread Mark Janes
Reviewed-by: Mark Janes Dylan Baker writes: > This fixes several tests on platforms that support > arb_transform_feedback3. > > Signed-off-by: Dylan Baker > --- > tests/all.py | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/tests/all.py b/t

Re: [Piglit] [PATCH 1/2] varying-packing: Use logical-OR instead of if statement to set failed.

2015-03-04 Thread Mark Janes
series Reviewed-by: Mark Janes Matt Turner writes: > By flattening the if statements we cut 288 instructions from the > fragment shader (806 -> 518, or 35%), and we remove all register > spilling on i965. > > Reduces runtime of "varying-packing-simple int separate"

Re: [Piglit] [PATCH] framework/backends/junit: Report expected failures/crashes as skipped.

2015-03-03 Thread Mark Janes
distribution, > and fail when performance goes below a given percentile. I'm curious to know which benchmarks you use. We experience variability in a lot of benchmarks, especially if they are cpu-heavy and running on an under-powered system. It would be great to have workloads that produc

Re: [Piglit] [PATCH] framework/backends/junit: Report expected failures/crashes as skipped.

2015-03-03 Thread Mark Janes
over 350k unskipped tests for each build. We filter out skipped tests due to the memory consumption on jenkins when displaying this many tests. I am interested in learning more about your test system, and sharing lessons learned / techniques. -Mark Reviewed-by: Mark Janes Jose Fonseca

Re: [Piglit] [PATCH 1/2] glsl-1.50: check array sizes when instance name differ

2015-03-02 Thread Mark Janes
series is Reviewed-by: Mark Janes Timothy Arceri writes: > Uniform interface blocks can have different instance names across shaders. > These tests check that array rules are applied correctly. > > Test results: > > Intel Ivybridge - Mesa 10.6 > > uniform-block-arra

Re: [Piglit] [PATCH 1/2] profile_tests.py: Convert generator test into a class test

2015-02-24 Thread Mark Janes
k_mixed_flatten(tests, testlist): > nt.assert_dict_equal(profile_.test_list, baseline) > > > -def generate_prepare_test_list_test_test_matches(): > -""" Generate tests for TestProfile.perpare_test_list filtering """ typo:

[Piglit] [PATCH] junit.py: Fix handling of special test names.

2015-02-20 Thread Mark Janes
In junit.py, the testname variable is used by a closure within the write_test() scope. Modifying it breaks the filtering of expected failures. --- framework/backends/junit.py | 9 + 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/framework/backends/junit.py b/framework/backe

Re: [Piglit] [PATCH] junit.py: Avoid special test names.

2015-02-20 Thread Mark Janes
Jose Fonseca writes: > On 20/02/15 20:50, Mark Janes wrote: >> We support using the test names as displayed in jenkins to start bisect >> jobs. If a test fails, you can copy/paste the name into a job, and it >> will find the commit that broke it. >> >> With this

Re: [Piglit] [PATCH] junit.py: Avoid special test names.

2015-02-20 Thread Mark Janes
We support using the test names as displayed in jenkins to start bisect jobs. If a test fails, you can copy/paste the name into a job, and it will find the commit that broke it. With this change, special-cased test names can't be used with --include-tests anymore. Instead of adding an underscore

Re: [Piglit] [Mesa-dev] [PATCH] arb_occlusion_query2: expect an error when target mismatch in glBeginQuery()

2015-02-18 Thread Mark Janes
Hi Eduardo, I track piglit regressions for the mesa team at Intel. It would really help me if you put a note in your commit message when a new test is known to fail on any platform. thanks, Mark Eduardo Lima Mitev writes: > As a heads-up, with this patch piglit fails the test for current Mes

Re: [Piglit] [Patch v2] cmake: Fix ee8579b452f0 problem

2015-01-19 Thread Mark Janes
${Waffle_LDFLAGS} > ) > IF(PIGLIT_BUILD_GLX_TESTS) > list(APPEND UTIL_GL_LIBS > -- > 2.2.1 > > ___ > Piglit mailing list > Piglit@lists.freedesktop.org > http://lists.freedesktop.org/mailman/listinfo/piglit Tested-by: Mark Janes ___ Piglit mailing list Piglit@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/piglit

Re: [Piglit] [PATCH] texture-packed-formats: Don't try bother with ABGR_EXT formats

2015-01-12 Thread Mark Janes
Tested-by: Mark Janes Jason Ekstrand writes: > Since GL_ABGR_EXT was extension number 1 to the GL spec, it didn't take > packed formats into account. As far as I can tell from the way the packed > formats extensions are written, packed formats with GL_ABGR_EXT isn't &g

Re: [Piglit] [PATCH] tests/all.py: fix typo in recent list conversion

2015-01-09 Thread Mark Janes
n( > 'glsl-{}'.format(version), 'execution', 'texelFetchOffset', > '{}-{}'.format(stage, sampler))] = PiglitGLTest( > -['texelFetch', 'offset,' '140', stage, sampler], > +

Re: [Piglit] [PATCH] generate-cl-store-tests.py: fix copy and paste error from 68829470

2014-12-19 Thread Mark Janes
Reviewed-by: Mark Janes Dylan Baker writes: > bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=87513 > Signed-off-by: Dylan Baker > --- > generated_tests/generate-cl-store-tests.py | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/genera

Re: [Piglit] [Patch v3] generated_tests: Actually catch exception in os.makedirs

2014-12-19 Thread Mark Janes
gt; module. > > v2: - remove accidentally included hunk (Mark) > v3: - Actually include the modules directory in the commit (Mark) Reviewed-by: Mark Janes ___ Piglit mailing list Piglit@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/piglit

Re: [Piglit] [Patch v2] generated_tests: Actually catch exception in os.makedirs

2014-12-19 Thread Mark Janes
It looks like this patch omits the utils.py module which is being called. This would have been hard to spot in a build-test, because the pyc from your previous patch was still in place. -Mark Dylan Baker writes: > Patch b59ff71eb was supposed to fix os.makedirs exceptions, but falls > short bec

Re: [Piglit] [PATCH] generated_tests: Actually catch exception in os.makedirs

2014-12-17 Thread Mark Janes
It seems like you unintentionally included an extra file: generated_tests/genclbuiltins.py The deltas are not related to os.makedirs. Dylan Baker writes: > generated_tests/genclbuiltins.py | 54 > +- > diff --git a/generated_tests/genclbuiltins.py > b/ge

Re: [Piglit] [Patch v2] generated_tests: Catch exception in os.makedirs

2014-12-12 Thread Mark Janes
Thanks, this is a superior implementation imho. -Mark Dylan Baker writes: > + > +def safe_makedirs(dirs): > +"""A safe function for creating a directory tree. > + > +This function wraps os.makedirs, and provides a couple of sanity checks, > +first, if the directory already exists it

Re: [Piglit] [PATCH] generated_tests: Except error for os.makedirs

2014-12-12 Thread Mark Janes
Thanks for making this fix. Dylan Baker writes: > -os.makedirs(dirname) > +try: > +os.makedirs(dirname) > +except OSError as e: > +if e.errno == 17: # file exists > +pass > +raise This works

Re: [Piglit] [PATCH v2] util/gl: link against OPENGL_gl_LIBRARY on waffle aware systems

2014-11-26 Thread Mark Janes
> v2: Try to handle both gl and gles*. > > Cc: Mark Janes > Reported-by: Mark Janes > Signed-off-by: Emil Velikov > --- > tests/util/CMakeLists.txt | 5 + > 1 file changed, 5 insertions(+) > > diff --git a/tests/util/CMakeLists.txt b/tests/util/CMakeLists.txt > ind

Re: [Piglit] [PATCH] util/gl: link against OPENGL_gl_LIBRARY on waffle aware systems

2014-11-25 Thread Mark Janes
instruction that should be used, instead of linking the gles tests against GL? -Mark Emil Velikov writes: > Similar to the earlier patch for waffle-less (glut) builds. The need > for this patch became apparent as we removed the libGL link dependency > in waffle. > > Cc: Mark Janes >

Re: [Piglit] [PATCH 2/2] framework/backends/junit: Prepend command line to stdout.

2014-11-21 Thread Mark Janes
Dylan Baker writes: > On Friday, November 21, 2014 07:37:20 PM Jose Fonseca wrote: >> On 20/11/14 12:06, jfons...@vmware.com wrote: >> > From: José Fonseca >> > >> > Showing the command line of the test is quite useful, specially when >> > diagnosing failures. > > If Marks okay with it. I like

Re: [Piglit] [PATCH v2] Remove ARB_timer_query / EXT_timer_query from quick.py

2014-11-20 Thread Mark Janes
Ben Widawsky writes: > On Tue, Nov 18, 2014 at 02:48:09PM -0800, Mark Janes wrote: >> Ilia Mirkin writes: >> >> > FWIW, on nv50 and nvc0, EXT_timer_query time-elapsed and >> > ARB_timer_query query GL_TIMESTAMP reliably fail, and the rest >> > r

Re: [Piglit] [PATCH v2] Remove ARB_timer_query / EXT_timer_query from quick.py

2014-11-18 Thread Mark Janes
Ilia Mirkin writes: > FWIW, on nv50 and nvc0, EXT_timer_query time-elapsed and > ARB_timer_query query GL_TIMESTAMP reliably fail, and the rest > reliably pass. I don't remember ever seeing inconsistent behaviour. > FWIW I don't really know what these extensions do, or what the tests > check for,

[Piglit] [PATCH v2] Remove ARB_timer_query / EXT_timer_query from quick.py

2014-11-18 Thread Mark Janes
EXT_timer_query and ARB_timer_query tests fail intermittently, causing confusion for developers running quick.py to find regressions. These tests have always been intermittent, and people generally know to ignore them when they fail. However, if everyone ignores a test, there is no point in runni

  1   2   >