[Piglit] [PATCH] framework/summary.py: Always print number of changes in console mode

[Tom Stellard]

The counts for changes, fixes, and regressions are now reported as zero
when no changes are detect rather than being omitted from the output.

This makes it easier for scripts to parse the output of piglit summary
console, becuase now when there are no regressions piglit always
outputs:

regressions: 0

Before when there where no regressions, piglit would either print no
regression line or it would print 'regressions: 0'
---
 framework/summary.py | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/framework/summary.py b/framework/summary.py
index 80cb181..e56c0f2 100644
--- a/framework/summary.py
+++ b/framework/summary.py
@@ -604,7 +604,10 @@ class Summary:
   "  fixes: {fixes}\n"
   "regressions: {regressions}".format(
   **{k: len(v) for k, v in self.tests.iteritems()}))
-
+else:
+print("changes: 0\n"
+  "  fixes: 0\n"
+  "regressions: 0")
 print("  total: {}".format(sum(self.totals.itervalues(
 
 # Print the name of the test and the status from each test run
-- 
2.0.4

___
Piglit mailing list
Piglit@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/piglit

Re: [Piglit] [PATCH] framework/summary.py: Always print number of changes in console mode

[Tom Stellard]

On Wed, Sep 09, 2015 at 01:23:22PM -0700, Dylan Baker wrote:
> On Wed, Sep 09, 2015 at 03:57:55PM -0400, Tom Stellard wrote:
> > On Wed, Sep 09, 2015 at 12:52:22PM -0700, Dylan Baker wrote:
> > > > I have a buildbot seti up to run piglit and the way it detects whether
> > > > or no there have been piglit regressions is by running piglit summary
> > > > console and then parsing the output.
> > > > 
> > > > The CSV summary seems to only accept a single result file, unless I'm
> > > > doing something wrong:
> > > > 
> > > > tstellar@localhost ~/piglit $ ./piglit summary csv  -o out.csv 
> > > > results/master--150727-/ results/master-BONAIRE-150223-/
> > > > usage: piglit [-h] [-o ] 
> > > > piglit: error: unrecognized arguments: results/master-BONAIRE-150223-/
> > > > 
> > > > -Tom
> > > 
> > > Are you just wanting to see if there are regressions, or do you want to
> > > get a count and the tests that regressed as well?
> > 
> > What I want is a list of changes, and yes/no for regressions.  The
> > number of regressions does not matter. `piglit/piglit summary console
> > -d` gives me a list of changes and prints the summary so I can check
> > if there were regressions.
> > 
> > Another solution I considered was to add an exit status to piglit
> > summary to indicate whether or not there were regressions.  This would
> > be the best solution, but I wasn't sure if this would break other
> > people's use cases.
> > 
> > -Tom
> 
> I think with my changes it would be trivial to add a '-r/--regressions'
> mode to piglit summary that would print only regressions in the change
> list, and could have an exit status for whether there were regressions
> or not.
> 

For me I would still like to see all changes: fixes and regressions but
have the exit code based on whether or not there are regressions.
So, I'm not sure a -r flag would be that useful for me.

-Tom

> I'll add an extra patch to my series.
> 
> Dylan


___
Piglit mailing list
Piglit@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/piglit

Re: [Piglit] [PATCH] framework/summary.py: Always print number of changes in console mode

[Dylan Baker]

Reviewed-by: Dylan Baker 

On a related note, I have a series that seriously overhauls the summary
code, including fixing the console/text summary, producing a column for
each result. It does include changes, fixes, and regressions in all
cases like this patch. I'm hoping to have that cleaned up and on the
list today.

On a tangent, why are you parsing the output of this, it seems like the
CSV summary would be easier to pass to another script than this, which
is volatile.

Dylan

On Wed, Sep 09, 2015 at 12:28:21PM +, Tom Stellard wrote:
> The counts for changes, fixes, and regressions are now reported as zero
> when no changes are detect rather than being omitted from the output.
> 
> This makes it easier for scripts to parse the output of piglit summary
> console, becuase now when there are no regressions piglit always
> outputs:
> 
> regressions: 0
> 
> Before when there where no regressions, piglit would either print no
> regression line or it would print 'regressions: 0'
> ---
>  framework/summary.py | 5 -
>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/framework/summary.py b/framework/summary.py
> index 80cb181..e56c0f2 100644
> --- a/framework/summary.py
> +++ b/framework/summary.py
> @@ -604,7 +604,10 @@ class Summary:
>"  fixes: {fixes}\n"
>"regressions: {regressions}".format(
>**{k: len(v) for k, v in self.tests.iteritems()}))
> -
> +else:
> +print("changes: 0\n"
> +  "  fixes: 0\n"
> +  "regressions: 0")
>  print("  total: {}".format(sum(self.totals.itervalues(
>  
>  # Print the name of the test and the status from each test run
> -- 
> 2.0.4
> 
> ___
> Piglit mailing list
> Piglit@lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/piglit


signature.asc
Description: Digital signature
___
Piglit mailing list
Piglit@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/piglit

Re: [Piglit] [PATCH] framework/summary.py: Always print number of changes in console mode

[Tom Stellard]

On Wed, Sep 09, 2015 at 11:05:55AM -0700, Dylan Baker wrote:
> Reviewed-by: Dylan Baker 
> 
> On a related note, I have a series that seriously overhauls the summary
> code, including fixing the console/text summary, producing a column for
> each result. It does include changes, fixes, and regressions in all
> cases like this patch. I'm hoping to have that cleaned up and on the
> list today.
> 
> On a tangent, why are you parsing the output of this, it seems like the
> CSV summary would be easier to pass to another script than this, which
> is volatile.
> 

I have a buildbot seti up to run piglit and the way it detects whether
or no there have been piglit regressions is by running piglit summary
console and then parsing the output.

The CSV summary seems to only accept a single result file, unless I'm
doing something wrong:

tstellar@localhost ~/piglit $ ./piglit summary csv  -o out.csv 
results/master--150727-/ results/master-BONAIRE-150223-/
usage: piglit [-h] [-o ] 
piglit: error: unrecognized arguments: results/master-BONAIRE-150223-/

-Tom


> Dylan
> 
> On Wed, Sep 09, 2015 at 12:28:21PM +, Tom Stellard wrote:
> > The counts for changes, fixes, and regressions are now reported as zero
> > when no changes are detect rather than being omitted from the output.
> > 
> > This makes it easier for scripts to parse the output of piglit summary
> > console, becuase now when there are no regressions piglit always
> > outputs:
> > 
> > regressions: 0
> > 
> > Before when there where no regressions, piglit would either print no
> > regression line or it would print 'regressions: 0'
> > ---
> >  framework/summary.py | 5 -
> >  1 file changed, 4 insertions(+), 1 deletion(-)
> > 
> > diff --git a/framework/summary.py b/framework/summary.py
> > index 80cb181..e56c0f2 100644
> > --- a/framework/summary.py
> > +++ b/framework/summary.py
> > @@ -604,7 +604,10 @@ class Summary:
> >"  fixes: {fixes}\n"
> >"regressions: {regressions}".format(
> >**{k: len(v) for k, v in 
> > self.tests.iteritems()}))
> > -
> > +else:
> > +print("changes: 0\n"
> > +  "  fixes: 0\n"
> > +  "regressions: 0")
> >  print("  total: {}".format(sum(self.totals.itervalues(
> >  
> >  # Print the name of the test and the status from each test run
> > -- 
> > 2.0.4
> > 
> > ___
> > Piglit mailing list
> > Piglit@lists.freedesktop.org
> > http://lists.freedesktop.org/mailman/listinfo/piglit



> ___
> Piglit mailing list
> Piglit@lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/piglit

___
Piglit mailing list
Piglit@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/piglit

Re: [Piglit] [PATCH] framework/summary.py: Always print number of changes in console mode

[Dylan Baker]

On Wed, Sep 09, 2015 at 03:25:50PM -0400, Tom Stellard wrote:
> On Wed, Sep 09, 2015 at 11:05:55AM -0700, Dylan Baker wrote:
> > Reviewed-by: Dylan Baker 
> > 
> > On a related note, I have a series that seriously overhauls the summary
> > code, including fixing the console/text summary, producing a column for
> > each result. It does include changes, fixes, and regressions in all
> > cases like this patch. I'm hoping to have that cleaned up and on the
> > list today.
> > 
> > On a tangent, why are you parsing the output of this, it seems like the
> > CSV summary would be easier to pass to another script than this, which
> > is volatile.
> > 
> 
> I have a buildbot seti up to run piglit and the way it detects whether
> or no there have been piglit regressions is by running piglit summary
> console and then parsing the output.
> 
> The CSV summary seems to only accept a single result file, unless I'm
> doing something wrong:
> 
> tstellar@localhost ~/piglit $ ./piglit summary csv  -o out.csv 
> results/master--150727-/ results/master-BONAIRE-150223-/
> usage: piglit [-h] [-o ] 
> piglit: error: unrecognized arguments: results/master-BONAIRE-150223-/
> 
> -Tom

You're right. I'll fix that, it seems useful to get more than one result
out of it.

> 
> 
> > Dylan
> > 
> > On Wed, Sep 09, 2015 at 12:28:21PM +, Tom Stellard wrote:
> > > The counts for changes, fixes, and regressions are now reported as zero
> > > when no changes are detect rather than being omitted from the output.
> > > 
> > > This makes it easier for scripts to parse the output of piglit summary
> > > console, becuase now when there are no regressions piglit always
> > > outputs:
> > > 
> > > regressions: 0
> > > 
> > > Before when there where no regressions, piglit would either print no
> > > regression line or it would print 'regressions: 0'
> > > ---
> > >  framework/summary.py | 5 -
> > >  1 file changed, 4 insertions(+), 1 deletion(-)
> > > 
> > > diff --git a/framework/summary.py b/framework/summary.py
> > > index 80cb181..e56c0f2 100644
> > > --- a/framework/summary.py
> > > +++ b/framework/summary.py
> > > @@ -604,7 +604,10 @@ class Summary:
> > >"  fixes: {fixes}\n"
> > >"regressions: {regressions}".format(
> > >**{k: len(v) for k, v in 
> > > self.tests.iteritems()}))
> > > -
> > > +else:
> > > +print("changes: 0\n"
> > > +  "  fixes: 0\n"
> > > +  "regressions: 0")
> > >  print("  total: 
> > > {}".format(sum(self.totals.itervalues(
> > >  
> > >  # Print the name of the test and the status from each test run
> > > -- 
> > > 2.0.4
> > > 
> > > ___
> > > Piglit mailing list
> > > Piglit@lists.freedesktop.org
> > > http://lists.freedesktop.org/mailman/listinfo/piglit
> 
> 
> 
> > ___
> > Piglit mailing list
> > Piglit@lists.freedesktop.org
> > http://lists.freedesktop.org/mailman/listinfo/piglit
> 


signature.asc
Description: Digital signature
___
Piglit mailing list
Piglit@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/piglit

Re: [Piglit] [PATCH] framework/summary.py: Always print number of changes in console mode

[Dylan Baker]

> I have a buildbot seti up to run piglit and the way it detects whether
> or no there have been piglit regressions is by running piglit summary
> console and then parsing the output.
> 
> The CSV summary seems to only accept a single result file, unless I'm
> doing something wrong:
> 
> tstellar@localhost ~/piglit $ ./piglit summary csv  -o out.csv 
> results/master--150727-/ results/master-BONAIRE-150223-/
> usage: piglit [-h] [-o ] 
> piglit: error: unrecognized arguments: results/master-BONAIRE-150223-/
> 
> -Tom

Are you just wanting to see if there are regressions, or do you want to
get a count and the tests that regressed as well?


signature.asc
Description: Digital signature
___
Piglit mailing list
Piglit@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/piglit

Re: [Piglit] [PATCH] framework/summary.py: Always print number of changes in console mode

[Tom Stellard]

On Wed, Sep 09, 2015 at 12:52:22PM -0700, Dylan Baker wrote:
> > I have a buildbot seti up to run piglit and the way it detects whether
> > or no there have been piglit regressions is by running piglit summary
> > console and then parsing the output.
> > 
> > The CSV summary seems to only accept a single result file, unless I'm
> > doing something wrong:
> > 
> > tstellar@localhost ~/piglit $ ./piglit summary csv  -o out.csv 
> > results/master--150727-/ results/master-BONAIRE-150223-/
> > usage: piglit [-h] [-o ] 
> > piglit: error: unrecognized arguments: results/master-BONAIRE-150223-/
> > 
> > -Tom
> 
> Are you just wanting to see if there are regressions, or do you want to
> get a count and the tests that regressed as well?

What I want is a list of changes, and yes/no for regressions.  The
number of regressions does not matter. `piglit/piglit summary console
-d` gives me a list of changes and prints the summary so I can check
if there were regressions.

Another solution I considered was to add an exit status to piglit
summary to indicate whether or not there were regressions.  This would
be the best solution, but I wasn't sure if this would break other
people's use cases.

-Tom
___
Piglit mailing list
Piglit@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/piglit