Re: [Piglit] [PATCH] framework/summary.py: Always print number of changes in console mode
On Wed, Sep 09, 2015 at 01:23:22PM -0700, Dylan Baker wrote: > On Wed, Sep 09, 2015 at 03:57:55PM -0400, Tom Stellard wrote: > > On Wed, Sep 09, 2015 at 12:52:22PM -0700, Dylan Baker wrote: > > > > I have a buildbot seti up to run piglit and the way it detects whether > > > > or no there have been piglit regressions is by running piglit summary > > > > console and then parsing the output. > > > > > > > > The CSV summary seems to only accept a single result file, unless I'm > > > > doing something wrong: > > > > > > > > tstellar@localhost ~/piglit $ ./piglit summary csv -o out.csv > > > > results/master--150727-/ results/master-BONAIRE-150223-/ > > > > usage: piglit [-h] [-o ] > > > > piglit: error: unrecognized arguments: results/master-BONAIRE-150223-/ > > > > > > > > -Tom > > > > > > Are you just wanting to see if there are regressions, or do you want to > > > get a count and the tests that regressed as well? > > > > What I want is a list of changes, and yes/no for regressions. The > > number of regressions does not matter. `piglit/piglit summary console > > -d` gives me a list of changes and prints the summary so I can check > > if there were regressions. > > > > Another solution I considered was to add an exit status to piglit > > summary to indicate whether or not there were regressions. This would > > be the best solution, but I wasn't sure if this would break other > > people's use cases. > > > > -Tom > > I think with my changes it would be trivial to add a '-r/--regressions' > mode to piglit summary that would print only regressions in the change > list, and could have an exit status for whether there were regressions > or not. > For me I would still like to see all changes: fixes and regressions but have the exit code based on whether or not there are regressions. So, I'm not sure a -r flag would be that useful for me. -Tom > I'll add an extra patch to my series. > > Dylan ___ Piglit mailing list Piglit@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/piglit
Re: [Piglit] [PATCH] framework/summary.py: Always print number of changes in console mode
On Wed, Sep 09, 2015 at 03:57:55PM -0400, Tom Stellard wrote: > On Wed, Sep 09, 2015 at 12:52:22PM -0700, Dylan Baker wrote: > > > I have a buildbot seti up to run piglit and the way it detects whether > > > or no there have been piglit regressions is by running piglit summary > > > console and then parsing the output. > > > > > > The CSV summary seems to only accept a single result file, unless I'm > > > doing something wrong: > > > > > > tstellar@localhost ~/piglit $ ./piglit summary csv -o out.csv > > > results/master--150727-/ results/master-BONAIRE-150223-/ > > > usage: piglit [-h] [-o ] > > > piglit: error: unrecognized arguments: results/master-BONAIRE-150223-/ > > > > > > -Tom > > > > Are you just wanting to see if there are regressions, or do you want to > > get a count and the tests that regressed as well? > > What I want is a list of changes, and yes/no for regressions. The > number of regressions does not matter. `piglit/piglit summary console > -d` gives me a list of changes and prints the summary so I can check > if there were regressions. > > Another solution I considered was to add an exit status to piglit > summary to indicate whether or not there were regressions. This would > be the best solution, but I wasn't sure if this would break other > people's use cases. > > -Tom I think with my changes it would be trivial to add a '-r/--regressions' mode to piglit summary that would print only regressions in the change list, and could have an exit status for whether there were regressions or not. I'll add an extra patch to my series. Dylan signature.asc Description: Digital signature ___ Piglit mailing list Piglit@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/piglit
Re: [Piglit] [PATCH] framework/summary.py: Always print number of changes in console mode
On Wed, Sep 09, 2015 at 12:52:22PM -0700, Dylan Baker wrote: > > I have a buildbot seti up to run piglit and the way it detects whether > > or no there have been piglit regressions is by running piglit summary > > console and then parsing the output. > > > > The CSV summary seems to only accept a single result file, unless I'm > > doing something wrong: > > > > tstellar@localhost ~/piglit $ ./piglit summary csv -o out.csv > > results/master--150727-/ results/master-BONAIRE-150223-/ > > usage: piglit [-h] [-o ] > > piglit: error: unrecognized arguments: results/master-BONAIRE-150223-/ > > > > -Tom > > Are you just wanting to see if there are regressions, or do you want to > get a count and the tests that regressed as well? What I want is a list of changes, and yes/no for regressions. The number of regressions does not matter. `piglit/piglit summary console -d` gives me a list of changes and prints the summary so I can check if there were regressions. Another solution I considered was to add an exit status to piglit summary to indicate whether or not there were regressions. This would be the best solution, but I wasn't sure if this would break other people's use cases. -Tom ___ Piglit mailing list Piglit@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/piglit
Re: [Piglit] [PATCH] framework/summary.py: Always print number of changes in console mode
> I have a buildbot seti up to run piglit and the way it detects whether > or no there have been piglit regressions is by running piglit summary > console and then parsing the output. > > The CSV summary seems to only accept a single result file, unless I'm > doing something wrong: > > tstellar@localhost ~/piglit $ ./piglit summary csv -o out.csv > results/master--150727-/ results/master-BONAIRE-150223-/ > usage: piglit [-h] [-o ] > piglit: error: unrecognized arguments: results/master-BONAIRE-150223-/ > > -Tom Are you just wanting to see if there are regressions, or do you want to get a count and the tests that regressed as well? signature.asc Description: Digital signature ___ Piglit mailing list Piglit@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/piglit
Re: [Piglit] [PATCH] framework/summary.py: Always print number of changes in console mode
On Wed, Sep 09, 2015 at 03:25:50PM -0400, Tom Stellard wrote: > On Wed, Sep 09, 2015 at 11:05:55AM -0700, Dylan Baker wrote: > > Reviewed-by: Dylan Baker > > > > On a related note, I have a series that seriously overhauls the summary > > code, including fixing the console/text summary, producing a column for > > each result. It does include changes, fixes, and regressions in all > > cases like this patch. I'm hoping to have that cleaned up and on the > > list today. > > > > On a tangent, why are you parsing the output of this, it seems like the > > CSV summary would be easier to pass to another script than this, which > > is volatile. > > > > I have a buildbot seti up to run piglit and the way it detects whether > or no there have been piglit regressions is by running piglit summary > console and then parsing the output. > > The CSV summary seems to only accept a single result file, unless I'm > doing something wrong: > > tstellar@localhost ~/piglit $ ./piglit summary csv -o out.csv > results/master--150727-/ results/master-BONAIRE-150223-/ > usage: piglit [-h] [-o ] > piglit: error: unrecognized arguments: results/master-BONAIRE-150223-/ > > -Tom You're right. I'll fix that, it seems useful to get more than one result out of it. > > > > Dylan > > > > On Wed, Sep 09, 2015 at 12:28:21PM +, Tom Stellard wrote: > > > The counts for changes, fixes, and regressions are now reported as zero > > > when no changes are detect rather than being omitted from the output. > > > > > > This makes it easier for scripts to parse the output of piglit summary > > > console, becuase now when there are no regressions piglit always > > > outputs: > > > > > > regressions: 0 > > > > > > Before when there where no regressions, piglit would either print no > > > regression line or it would print 'regressions: 0' > > > --- > > > framework/summary.py | 5 - > > > 1 file changed, 4 insertions(+), 1 deletion(-) > > > > > > diff --git a/framework/summary.py b/framework/summary.py > > > index 80cb181..e56c0f2 100644 > > > --- a/framework/summary.py > > > +++ b/framework/summary.py > > > @@ -604,7 +604,10 @@ class Summary: > > >" fixes: {fixes}\n" > > >"regressions: {regressions}".format( > > >**{k: len(v) for k, v in > > > self.tests.iteritems()})) > > > - > > > +else: > > > +print("changes: 0\n" > > > + " fixes: 0\n" > > > + "regressions: 0") > > > print(" total: > > > {}".format(sum(self.totals.itervalues( > > > > > > # Print the name of the test and the status from each test run > > > -- > > > 2.0.4 > > > > > > ___ > > > Piglit mailing list > > > Piglit@lists.freedesktop.org > > > http://lists.freedesktop.org/mailman/listinfo/piglit > > > > > ___ > > Piglit mailing list > > Piglit@lists.freedesktop.org > > http://lists.freedesktop.org/mailman/listinfo/piglit > signature.asc Description: Digital signature ___ Piglit mailing list Piglit@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/piglit
Re: [Piglit] [PATCH] framework/summary.py: Always print number of changes in console mode
On Wed, Sep 09, 2015 at 11:05:55AM -0700, Dylan Baker wrote: > Reviewed-by: Dylan Baker > > On a related note, I have a series that seriously overhauls the summary > code, including fixing the console/text summary, producing a column for > each result. It does include changes, fixes, and regressions in all > cases like this patch. I'm hoping to have that cleaned up and on the > list today. > > On a tangent, why are you parsing the output of this, it seems like the > CSV summary would be easier to pass to another script than this, which > is volatile. > I have a buildbot seti up to run piglit and the way it detects whether or no there have been piglit regressions is by running piglit summary console and then parsing the output. The CSV summary seems to only accept a single result file, unless I'm doing something wrong: tstellar@localhost ~/piglit $ ./piglit summary csv -o out.csv results/master--150727-/ results/master-BONAIRE-150223-/ usage: piglit [-h] [-o ] piglit: error: unrecognized arguments: results/master-BONAIRE-150223-/ -Tom > Dylan > > On Wed, Sep 09, 2015 at 12:28:21PM +, Tom Stellard wrote: > > The counts for changes, fixes, and regressions are now reported as zero > > when no changes are detect rather than being omitted from the output. > > > > This makes it easier for scripts to parse the output of piglit summary > > console, becuase now when there are no regressions piglit always > > outputs: > > > > regressions: 0 > > > > Before when there where no regressions, piglit would either print no > > regression line or it would print 'regressions: 0' > > --- > > framework/summary.py | 5 - > > 1 file changed, 4 insertions(+), 1 deletion(-) > > > > diff --git a/framework/summary.py b/framework/summary.py > > index 80cb181..e56c0f2 100644 > > --- a/framework/summary.py > > +++ b/framework/summary.py > > @@ -604,7 +604,10 @@ class Summary: > >" fixes: {fixes}\n" > >"regressions: {regressions}".format( > >**{k: len(v) for k, v in > > self.tests.iteritems()})) > > - > > +else: > > +print("changes: 0\n" > > + " fixes: 0\n" > > + "regressions: 0") > > print(" total: {}".format(sum(self.totals.itervalues( > > > > # Print the name of the test and the status from each test run > > -- > > 2.0.4 > > > > ___ > > Piglit mailing list > > Piglit@lists.freedesktop.org > > http://lists.freedesktop.org/mailman/listinfo/piglit > ___ > Piglit mailing list > Piglit@lists.freedesktop.org > http://lists.freedesktop.org/mailman/listinfo/piglit ___ Piglit mailing list Piglit@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/piglit
Re: [Piglit] [PATCH] framework/summary.py: Always print number of changes in console mode
Reviewed-by: Dylan Baker On a related note, I have a series that seriously overhauls the summary code, including fixing the console/text summary, producing a column for each result. It does include changes, fixes, and regressions in all cases like this patch. I'm hoping to have that cleaned up and on the list today. On a tangent, why are you parsing the output of this, it seems like the CSV summary would be easier to pass to another script than this, which is volatile. Dylan On Wed, Sep 09, 2015 at 12:28:21PM +, Tom Stellard wrote: > The counts for changes, fixes, and regressions are now reported as zero > when no changes are detect rather than being omitted from the output. > > This makes it easier for scripts to parse the output of piglit summary > console, becuase now when there are no regressions piglit always > outputs: > > regressions: 0 > > Before when there where no regressions, piglit would either print no > regression line or it would print 'regressions: 0' > --- > framework/summary.py | 5 - > 1 file changed, 4 insertions(+), 1 deletion(-) > > diff --git a/framework/summary.py b/framework/summary.py > index 80cb181..e56c0f2 100644 > --- a/framework/summary.py > +++ b/framework/summary.py > @@ -604,7 +604,10 @@ class Summary: >" fixes: {fixes}\n" >"regressions: {regressions}".format( >**{k: len(v) for k, v in self.tests.iteritems()})) > - > +else: > +print("changes: 0\n" > + " fixes: 0\n" > + "regressions: 0") > print(" total: {}".format(sum(self.totals.itervalues( > > # Print the name of the test and the status from each test run > -- > 2.0.4 > > ___ > Piglit mailing list > Piglit@lists.freedesktop.org > http://lists.freedesktop.org/mailman/listinfo/piglit signature.asc Description: Digital signature ___ Piglit mailing list Piglit@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/piglit
[Piglit] [PATCH] framework/summary.py: Always print number of changes in console mode
The counts for changes, fixes, and regressions are now reported as zero when no changes are detect rather than being omitted from the output. This makes it easier for scripts to parse the output of piglit summary console, becuase now when there are no regressions piglit always outputs: regressions: 0 Before when there where no regressions, piglit would either print no regression line or it would print 'regressions: 0' --- framework/summary.py | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/framework/summary.py b/framework/summary.py index 80cb181..e56c0f2 100644 --- a/framework/summary.py +++ b/framework/summary.py @@ -604,7 +604,10 @@ class Summary: " fixes: {fixes}\n" "regressions: {regressions}".format( **{k: len(v) for k, v in self.tests.iteritems()})) - +else: +print("changes: 0\n" + " fixes: 0\n" + "regressions: 0") print(" total: {}".format(sum(self.totals.itervalues( # Print the name of the test and the status from each test run -- 2.0.4 ___ Piglit mailing list Piglit@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/piglit