Re: [Piglit] [PATCH 00/40] Replace vpfp-generic with shader_runner.
On Saturday, June 06, 2015 07:11:26 PM Kenneth Graunke wrote:
> Hi all,
>
> This patch series ports all vpfp-generic tests to shader_runner,
> and then deletes vpfp-generic.
>
> A bit of history:
> - vpfp-generic was introduced by Nicolai Hähnle in 2009, as a generic
> ARB_vertex|fragment_program test runner.
> - shader_runner was introduced by Ian Romanick in 2010, as a generic
> GLSL shader runner.
> - shader_runner gained ARB program support in 2011 (courtesy of Eric Anholt).
>
> At this point, vpfp-generic is fairly redundant - shader_runner can do
> everything we need, and is much more widespread (12000+ tests). I've
> been meaning to delete it for a few years, but never got around to it.
>
> One difference is that the new tests don't glClear() before drawing. Since
> they draw the entire window, it's pretty unnecessary, and just makes the
> tests harder to debug. Many shader_runner tests don't bother clearing.
I've gone ahead and added the clears back in, based on everyone's
feedback.
The original tests actually inspected the expected color - if a
component was > 0.5, it used 0.0 as the clear color; otherwise it used
0.0. That ensured that the clear color was always different than the
expected value. Which seems useful, and perhaps hard to replicate in
a completely generic mechanism in shader_runner.
The new shader_runner tests should be functionally equivalent to the
old ones. I've gone ahead and pushed them since I don't think anybody
actually wants to review these boring changes.
Here is the updated python script that handles clears, too:
#! /usr/bin/python3
import re
import sys
template = '''{comments}[require]
GL >= 1.3
ARB_vertex_program
ARB_fragment_program
[vertex program]
{vptext}
[fragment program]
{fptext}
[test]
ortho 0 1 0 1
{script}'''
def convert(infile):
with open(infile, 'r') as f:
vpfp = f.read()
vpfp = re.sub('^;', '#', vpfp, flags=re.MULTILINE)
doublebang = vpfp.find('!!')
comments = vpfp[0:doublebang]
vpfp = vpfp[doublebang:]
parse =
re.match('(^!!ARBvp1.0$.*^END$)\s*(^!!ARBfp1.0$.*^END$)\s*!!test\n(.*)', vpfp,
re.MULTILINE | re.DOTALL)
vptext, fptext, vpfp_test = parse.groups()
script = ''
for line in vpfp_test.splitlines():
line = ' '.join(line.split()) # collapse whitespace
if line == '' or line == '!!test':
continue
parse = re.match('^expected ([^\s]+) ([^\s]+) ([^\s]+) ([^\s]+)', line)
if parse is not None:
params = parse.groups()
clear_color = ["0.0" if float(x) > 0.5 else "1.0" for x in params]
script += 'clear color {} {} {} {}\nclear\n'.format(*clear_color)
script += 'draw rect 0 0 1 1\nprobe all rgba {} {} {}
{}\n'.format(*params)
continue
parse = re.match('^(fragment|vertex).(local|environment)\[([0-9]+)\]
([^\s]+) ([^\s]+) ([^\s]+) ([^\s]+)', line)
if parse is not None:
params = parse.groups()
script += 'parameter {}_{}p {} ({}, {}, {},
{})\n'.format(params[1].replace('environment', 'env'), params[0][0],
*params[2:])
continue
parse = re.match('^texcoord\[([0-9]+)\] ([^\s]+) ([^\s]+) ([^\s]+)
([^\s]+)', line)
if parse is not None:
script += 'texcoord {} ({}, {}, {}, {})\n'.format(*parse.groups())
continue
raise Exception("unknown line: " + line)
print(template.format(comments=comments, vptext=vptext, fptext=fptext,
script=script))
if __name__ == '__main__':
convert(sys.argv[1])
signature.asc
Description: This is a digitally signed message part.
___
Piglit mailing list
Piglit@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/piglit
Re: [Piglit] [PATCH 00/40] Replace vpfp-generic with shader_runner.
Yes, that's what I'm suggesting as well. On Tue, Jun 9, 2015 at 7:36 AM, Marek Olšák wrote: > The clearing could be done by shader_runner automatically, tests > shouldn't have to do it manually. > > Marek > > On Tue, Jun 9, 2015 at 1:06 PM, Ilia Mirkin wrote: >> On Mon, Jun 8, 2015 at 9:48 PM, Michel Dänzer wrote: >>> On 08.06.2015 19:35, Ilia Mirkin wrote: This happens all the time. Vs gets messed up and nothing gets drawn, and the test passes. This is always a huge source of wasted time for me. I'd be highly in favor of having a way to pre clear... Perhaps via cmdline option so that only people actively working on driver dev incur the extra cost. >>> >>> Is the extra cost actually significant? >> >> I have to imagine it's non-0, and I wanted to avoid the discussion >> from people saying that it'll slow down their piglit runs for no >> reason. I would also be in favor of just always doing it. >> >> -ilia ___ Piglit mailing list Piglit@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/piglit
Re: [Piglit] [PATCH 00/40] Replace vpfp-generic with shader_runner.
The clearing could be done by shader_runner automatically, tests shouldn't have to do it manually. Marek On Tue, Jun 9, 2015 at 1:06 PM, Ilia Mirkin wrote: > On Mon, Jun 8, 2015 at 9:48 PM, Michel Dänzer wrote: >> On 08.06.2015 19:35, Ilia Mirkin wrote: >>> This happens all the time. Vs gets messed up and nothing gets drawn, and >>> the test passes. This is always a huge source of wasted time for me. I'd >>> be highly in favor of having a way to pre clear... Perhaps via cmdline >>> option so that only people actively working on driver dev incur the >>> extra cost. >> >> Is the extra cost actually significant? > > I have to imagine it's non-0, and I wanted to avoid the discussion > from people saying that it'll slow down their piglit runs for no > reason. I would also be in favor of just always doing it. > > -ilia ___ Piglit mailing list Piglit@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/piglit
Re: [Piglit] [PATCH 00/40] Replace vpfp-generic with shader_runner.
On Mon, Jun 8, 2015 at 9:48 PM, Michel Dänzer wrote: > On 08.06.2015 19:35, Ilia Mirkin wrote: >> This happens all the time. Vs gets messed up and nothing gets drawn, and >> the test passes. This is always a huge source of wasted time for me. I'd >> be highly in favor of having a way to pre clear... Perhaps via cmdline >> option so that only people actively working on driver dev incur the >> extra cost. > > Is the extra cost actually significant? I have to imagine it's non-0, and I wanted to avoid the discussion from people saying that it'll slow down their piglit runs for no reason. I would also be in favor of just always doing it. -ilia ___ Piglit mailing list Piglit@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/piglit
Re: [Piglit] [PATCH 00/40] Replace vpfp-generic with shader_runner.
On 08.06.2015 19:35, Ilia Mirkin wrote: > This happens all the time. Vs gets messed up and nothing gets drawn, and > the test passes. This is always a huge source of wasted time for me. I'd > be highly in favor of having a way to pre clear... Perhaps via cmdline > option so that only people actively working on driver dev incur the > extra cost. Is the extra cost actually significant? -- Earthling Michel Dänzer | http://www.amd.com Libre software enthusiast | Mesa and X developer ___ Piglit mailing list Piglit@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/piglit
Re: [Piglit] [PATCH 00/40] Replace vpfp-generic with shader_runner.
This happens all the time. Vs gets messed up and nothing gets drawn, and the test passes. This is always a huge source of wasted time for me. I'd be highly in favor of having a way to pre clear... Perhaps via cmdline option so that only people actively working on driver dev incur the extra cost. On Jun 8, 2015 1:01 PM, "Marek Olšák" wrote: > I recently ran into this. A tessellation test passed when it > shouldn't. That was pretty weird. The driver or hw was dropping draw > calls for some reason, yet most of the time the test passed. The > solution: > > > http://cgit.freedesktop.org/piglit/commit/?id=98c92d7bb2b80954912d9f8004e7810e7d21ff20 > > Marek > > On Sun, Jun 7, 2015 at 10:29 PM, Dave Airlie wrote: > > On 7 June 2015 at 12:11, Kenneth Graunke wrote: > >> Hi all, > >> > >> This patch series ports all vpfp-generic tests to shader_runner, > >> and then deletes vpfp-generic. > >> > >> A bit of history: > >> - vpfp-generic was introduced by Nicolai Hähnle in 2009, as a generic > >> ARB_vertex|fragment_program test runner. > >> - shader_runner was introduced by Ian Romanick in 2010, as a generic > >> GLSL shader runner. > >> - shader_runner gained ARB program support in 2011 (courtesy of Eric > Anholt). > >> > >> At this point, vpfp-generic is fairly redundant - shader_runner can do > >> everything we need, and is much more widespread (12000+ tests). I've > >> been meaning to delete it for a few years, but never got around to it. > >> > >> One difference is that the new tests don't glClear() before drawing. > Since > >> they draw the entire window, it's pretty unnecessary, and just makes the > >> tests harder to debug. Many shader_runner tests don't bother clearing. > > > > This is actually annoying feature, esp if all tests use the same color > > for success, > > > > because we render one test, it passes, we render another test, it > > doesn't draw anything > > but it has gotten the back buffer from the previous tests, and it > > magically passes. > > > > This happens a lot more often on GPUs with VRAM. > > > > Dave. > > ___ > > Piglit mailing list > > Piglit@lists.freedesktop.org > > http://lists.freedesktop.org/mailman/listinfo/piglit > ___ > Piglit mailing list > Piglit@lists.freedesktop.org > http://lists.freedesktop.org/mailman/listinfo/piglit > ___ Piglit mailing list Piglit@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/piglit
Re: [Piglit] [PATCH 00/40] Replace vpfp-generic with shader_runner.
I recently ran into this. A tessellation test passed when it shouldn't. That was pretty weird. The driver or hw was dropping draw calls for some reason, yet most of the time the test passed. The solution: http://cgit.freedesktop.org/piglit/commit/?id=98c92d7bb2b80954912d9f8004e7810e7d21ff20 Marek On Sun, Jun 7, 2015 at 10:29 PM, Dave Airlie wrote: > On 7 June 2015 at 12:11, Kenneth Graunke wrote: >> Hi all, >> >> This patch series ports all vpfp-generic tests to shader_runner, >> and then deletes vpfp-generic. >> >> A bit of history: >> - vpfp-generic was introduced by Nicolai Hähnle in 2009, as a generic >> ARB_vertex|fragment_program test runner. >> - shader_runner was introduced by Ian Romanick in 2010, as a generic >> GLSL shader runner. >> - shader_runner gained ARB program support in 2011 (courtesy of Eric Anholt). >> >> At this point, vpfp-generic is fairly redundant - shader_runner can do >> everything we need, and is much more widespread (12000+ tests). I've >> been meaning to delete it for a few years, but never got around to it. >> >> One difference is that the new tests don't glClear() before drawing. Since >> they draw the entire window, it's pretty unnecessary, and just makes the >> tests harder to debug. Many shader_runner tests don't bother clearing. > > This is actually annoying feature, esp if all tests use the same color > for success, > > because we render one test, it passes, we render another test, it > doesn't draw anything > but it has gotten the back buffer from the previous tests, and it > magically passes. > > This happens a lot more often on GPUs with VRAM. > > Dave. > ___ > Piglit mailing list > Piglit@lists.freedesktop.org > http://lists.freedesktop.org/mailman/listinfo/piglit ___ Piglit mailing list Piglit@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/piglit
Re: [Piglit] [PATCH 00/40] Replace vpfp-generic with shader_runner.
On Monday, June 08, 2015 03:42:36 PM Dave Airlie wrote: > On 8 June 2015 at 15:31, Kenneth Graunke wrote: > > On Monday, June 08, 2015 06:29:26 AM Dave Airlie wrote: > >> On 7 June 2015 at 12:11, Kenneth Graunke wrote: > >> > Hi all, > >> > > >> > This patch series ports all vpfp-generic tests to shader_runner, > >> > and then deletes vpfp-generic. > >> > > >> > A bit of history: > >> > - vpfp-generic was introduced by Nicolai Hähnle in 2009, as a generic > >> > ARB_vertex|fragment_program test runner. > >> > - shader_runner was introduced by Ian Romanick in 2010, as a generic > >> > GLSL shader runner. > >> > - shader_runner gained ARB program support in 2011 (courtesy of Eric > >> > Anholt). > >> > > >> > At this point, vpfp-generic is fairly redundant - shader_runner can do > >> > everything we need, and is much more widespread (12000+ tests). I've > >> > been meaning to delete it for a few years, but never got around to it. > >> > > >> > One difference is that the new tests don't glClear() before drawing. > >> > Since > >> > they draw the entire window, it's pretty unnecessary, and just makes the > >> > tests harder to debug. Many shader_runner tests don't bother clearing. > >> > >> This is actually annoying feature, esp if all tests use the same color > >> for success, > >> > >> because we render one test, it passes, we render another test, it > >> doesn't draw anything > >> but it has gotten the back buffer from the previous tests, and it > >> magically passes. > >> > >> This happens a lot more often on GPUs with VRAM. > >> > >> Dave. > > > > I don't know...the tests probe the entire window...so the only failure > > mode that will bite you like that is "the driver didn't render anything > > at all." And the assumption is that, even with such a broken driver, > > clearing will actually succeed at drawing... > > Yes but what happens if all the tests run and don't bother clearing, > so drawing fails in test 1, test 2 passes because it doesn't clear, > and it gets test1 result frame, where it passed, It looks like > test2 passes when it clearly hasn't. > > You've actually said it, clearing would succeed, but the problem > is the tests don't clear. > > and yes there are many reasons things don't render, the main > one I see if where an earlier test has locked up the GPU but > not totally. > > Dave. So your thinking is that Test 2 exercises some functionality (i.e. complex geometry shaders) which hang the GPU, but the clear is simple, so it's likely to succeed, and is done first? I suppose that makes some sense. Still, if your GPU is locking up, then it seems like tracking that down (and either fixing or blacklisting those tests) ought to be your top priority. Finding the tests that hang is pretty straightforward: $ piglit run --dmesg -1 -v quick results/hangs I always assume Piglit results are invalid (or dubious at best) if a run has hung the GPU. We should probably check dmesg at least once, even on a normal run, and log a message indicating that there was a GPU hang. It wouldn't tell you which hung, but you'd know your results were suspect and to re-run with --dmesg to pinpoint it... At any rate, are you asking me to rework my patches to preserve the existing color clear? There are a lot of Piglit tests done both ways. --Ken signature.asc Description: This is a digitally signed message part. ___ Piglit mailing list Piglit@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/piglit
Re: [Piglit] [PATCH 00/40] Replace vpfp-generic with shader_runner.
On 8 June 2015 at 15:31, Kenneth Graunke wrote: > On Monday, June 08, 2015 06:29:26 AM Dave Airlie wrote: >> On 7 June 2015 at 12:11, Kenneth Graunke wrote: >> > Hi all, >> > >> > This patch series ports all vpfp-generic tests to shader_runner, >> > and then deletes vpfp-generic. >> > >> > A bit of history: >> > - vpfp-generic was introduced by Nicolai Hähnle in 2009, as a generic >> > ARB_vertex|fragment_program test runner. >> > - shader_runner was introduced by Ian Romanick in 2010, as a generic >> > GLSL shader runner. >> > - shader_runner gained ARB program support in 2011 (courtesy of Eric >> > Anholt). >> > >> > At this point, vpfp-generic is fairly redundant - shader_runner can do >> > everything we need, and is much more widespread (12000+ tests). I've >> > been meaning to delete it for a few years, but never got around to it. >> > >> > One difference is that the new tests don't glClear() before drawing. Since >> > they draw the entire window, it's pretty unnecessary, and just makes the >> > tests harder to debug. Many shader_runner tests don't bother clearing. >> >> This is actually annoying feature, esp if all tests use the same color >> for success, >> >> because we render one test, it passes, we render another test, it >> doesn't draw anything >> but it has gotten the back buffer from the previous tests, and it >> magically passes. >> >> This happens a lot more often on GPUs with VRAM. >> >> Dave. > > I don't know...the tests probe the entire window...so the only failure > mode that will bite you like that is "the driver didn't render anything > at all." And the assumption is that, even with such a broken driver, > clearing will actually succeed at drawing... Yes but what happens if all the tests run and don't bother clearing, so drawing fails in test 1, test 2 passes because it doesn't clear, and it gets test1 result frame, where it passed, It looks like test2 passes when it clearly hasn't. You've actually said it, clearing would succeed, but the problem is the tests don't clear. and yes there are many reasons things don't render, the main one I see if where an earlier test has locked up the GPU but not totally. Dave. ___ Piglit mailing list Piglit@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/piglit
Re: [Piglit] [PATCH 00/40] Replace vpfp-generic with shader_runner.
On Monday, June 08, 2015 06:29:26 AM Dave Airlie wrote: > On 7 June 2015 at 12:11, Kenneth Graunke wrote: > > Hi all, > > > > This patch series ports all vpfp-generic tests to shader_runner, > > and then deletes vpfp-generic. > > > > A bit of history: > > - vpfp-generic was introduced by Nicolai Hähnle in 2009, as a generic > > ARB_vertex|fragment_program test runner. > > - shader_runner was introduced by Ian Romanick in 2010, as a generic > > GLSL shader runner. > > - shader_runner gained ARB program support in 2011 (courtesy of Eric > > Anholt). > > > > At this point, vpfp-generic is fairly redundant - shader_runner can do > > everything we need, and is much more widespread (12000+ tests). I've > > been meaning to delete it for a few years, but never got around to it. > > > > One difference is that the new tests don't glClear() before drawing. Since > > they draw the entire window, it's pretty unnecessary, and just makes the > > tests harder to debug. Many shader_runner tests don't bother clearing. > > This is actually annoying feature, esp if all tests use the same color > for success, > > because we render one test, it passes, we render another test, it > doesn't draw anything > but it has gotten the back buffer from the previous tests, and it > magically passes. > > This happens a lot more often on GPUs with VRAM. > > Dave. I don't know...the tests probe the entire window...so the only failure mode that will bite you like that is "the driver didn't render anything at all." And the assumption is that, even with such a broken driver, clearing will actually succeed at drawing... Is that really so common? i965 will usually either draw something, or fail so spectacularly that even clear doesn't work... signature.asc Description: This is a digitally signed message part. ___ Piglit mailing list Piglit@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/piglit
Re: [Piglit] [PATCH 00/40] Replace vpfp-generic with shader_runner.
On 7 June 2015 at 12:11, Kenneth Graunke wrote: > Hi all, > > This patch series ports all vpfp-generic tests to shader_runner, > and then deletes vpfp-generic. > > A bit of history: > - vpfp-generic was introduced by Nicolai Hähnle in 2009, as a generic > ARB_vertex|fragment_program test runner. > - shader_runner was introduced by Ian Romanick in 2010, as a generic > GLSL shader runner. > - shader_runner gained ARB program support in 2011 (courtesy of Eric Anholt). > > At this point, vpfp-generic is fairly redundant - shader_runner can do > everything we need, and is much more widespread (12000+ tests). I've > been meaning to delete it for a few years, but never got around to it. > > One difference is that the new tests don't glClear() before drawing. Since > they draw the entire window, it's pretty unnecessary, and just makes the > tests harder to debug. Many shader_runner tests don't bother clearing. This is actually annoying feature, esp if all tests use the same color for success, because we render one test, it passes, we render another test, it doesn't draw anything but it has gotten the back buffer from the previous tests, and it magically passes. This happens a lot more often on GPUs with VRAM. Dave. ___ Piglit mailing list Piglit@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/piglit
