Re: [Piglit] [PATCH] framework: Add handler for failure and error when expecting the other
Quoting Mark Janes (2016-03-16 09:41:18) > I don't need to see a v2 > > Acked-by: Mark Janes > > Dylan Baker writes: > > > Quoting Mark Janes (2016-03-15 17:40:16) > >> Dylan Baker writes: > >> > >> > Right now the JUnit backend knows what to do with a failure when it > >> > expects one (or a crash), but not what to do when it expects a failure > >> > and gets a crash (or vice versa). > >> > > >> > This patch teaches it what to do in that case, mark the test as a > >> > failure and give a message explaining why. > >> > > >> > cc: mark.a.ja...@intel.com > >> > signed-off-by: Dylan Baker > >> > --- > >> > framework/backends/junit.py | 12 > >> > 1 file changed, 12 insertions(+) > >> > > >> > diff --git a/framework/backends/junit.py b/framework/backends/junit.py > >> > index f9eec66..761b201 100644 > >> > --- a/framework/backends/junit.py > >> > +++ b/framework/backends/junit.py > >> > @@ -151,6 +151,12 @@ class JUnitBackend(FileBackend): > >> > err.text += "\n\nWARN: passing test as an expected > >> > failure" > >> > res = etree.SubElement(element, 'skipped', > >> > message='expected failure') > >> > +elif expected_result == 'error': > >> > +err.text += \ > >> > +"\n\nERROR: Test should have been crash but was > >> > failure" > >> > +res = etree.SubElement(element, 'failure', > >> > + message='expected crash, but > >> > got ' > >> > + 'failure') > >> > else: > >> > res = etree.SubElement(element, 'failure') > >> > > >> > @@ -159,6 +165,12 @@ class JUnitBackend(FileBackend): > >> > err.text += "\n\nWARN: passing test as an expected > >> > crash" > >> > res = etree.SubElement(element, 'skipped', > >> > message='expected crash') > >> > +elif expected_result == 'failure': > >> > +err.text += \ > >> > +"\n\nERROR: Test should have been failure but > >> > was crash" > >> > +res = etree.SubElement(element, 'failure', > >> I think you want to insert an 'error' tag here ^^^ > >> > + message='expected failure, > >> > but got ' > >> > + 'crash') > >> > else: > >> > res = etree.SubElement(element, 'error') > >> > > >> > -- > >> > 2.7.3 > > > > After discussing this offline, you are right. > > > > I've made the change locally, do you want to see a v2? > > > > Dylan Thanks Mark, pushed. signature.asc Description: signature ___ Piglit mailing list Piglit@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/piglit
Re: [Piglit] [PATCH] framework: Add handler for failure and error when expecting the other
Quoting Mark Janes (2016-03-15 17:40:16) > Dylan Baker writes: > > > Right now the JUnit backend knows what to do with a failure when it > > expects one (or a crash), but not what to do when it expects a failure > > and gets a crash (or vice versa). > > > > This patch teaches it what to do in that case, mark the test as a > > failure and give a message explaining why. > > > > cc: mark.a.ja...@intel.com > > signed-off-by: Dylan Baker > > --- > > framework/backends/junit.py | 12 > > 1 file changed, 12 insertions(+) > > > > diff --git a/framework/backends/junit.py b/framework/backends/junit.py > > index f9eec66..761b201 100644 > > --- a/framework/backends/junit.py > > +++ b/framework/backends/junit.py > > @@ -151,6 +151,12 @@ class JUnitBackend(FileBackend): > > err.text += "\n\nWARN: passing test as an expected > > failure" > > res = etree.SubElement(element, 'skipped', > > message='expected failure') > > +elif expected_result == 'error': > > +err.text += \ > > +"\n\nERROR: Test should have been crash but was > > failure" > > +res = etree.SubElement(element, 'failure', > > + message='expected crash, but > > got ' > > + 'failure') > > else: > > res = etree.SubElement(element, 'failure') > > > > @@ -159,6 +165,12 @@ class JUnitBackend(FileBackend): > > err.text += "\n\nWARN: passing test as an expected > > crash" > > res = etree.SubElement(element, 'skipped', > > message='expected crash') > > +elif expected_result == 'failure': > > +err.text += \ > > +"\n\nERROR: Test should have been failure but was > > crash" > > +res = etree.SubElement(element, 'failure', > I think you want to insert an 'error' tag here ^^^ > > + message='expected failure, but > > got ' > > + 'crash') > > else: > > res = etree.SubElement(element, 'error') > > > > -- > > 2.7.3 After discussing this offline, you are right. I've made the change locally, do you want to see a v2? Dylan signature.asc Description: signature ___ Piglit mailing list Piglit@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/piglit
Re: [Piglit] [PATCH] framework: Add handler for failure and error when expecting the other
I don't need to see a v2 Acked-by: Mark Janes Dylan Baker writes: > Quoting Mark Janes (2016-03-15 17:40:16) >> Dylan Baker writes: >> >> > Right now the JUnit backend knows what to do with a failure when it >> > expects one (or a crash), but not what to do when it expects a failure >> > and gets a crash (or vice versa). >> > >> > This patch teaches it what to do in that case, mark the test as a >> > failure and give a message explaining why. >> > >> > cc: mark.a.ja...@intel.com >> > signed-off-by: Dylan Baker >> > --- >> > framework/backends/junit.py | 12 >> > 1 file changed, 12 insertions(+) >> > >> > diff --git a/framework/backends/junit.py b/framework/backends/junit.py >> > index f9eec66..761b201 100644 >> > --- a/framework/backends/junit.py >> > +++ b/framework/backends/junit.py >> > @@ -151,6 +151,12 @@ class JUnitBackend(FileBackend): >> > err.text += "\n\nWARN: passing test as an expected >> > failure" >> > res = etree.SubElement(element, 'skipped', >> > message='expected failure') >> > +elif expected_result == 'error': >> > +err.text += \ >> > +"\n\nERROR: Test should have been crash but was >> > failure" >> > +res = etree.SubElement(element, 'failure', >> > + message='expected crash, but >> > got ' >> > + 'failure') >> > else: >> > res = etree.SubElement(element, 'failure') >> > >> > @@ -159,6 +165,12 @@ class JUnitBackend(FileBackend): >> > err.text += "\n\nWARN: passing test as an expected >> > crash" >> > res = etree.SubElement(element, 'skipped', >> > message='expected crash') >> > +elif expected_result == 'failure': >> > +err.text += \ >> > +"\n\nERROR: Test should have been failure but was >> > crash" >> > +res = etree.SubElement(element, 'failure', >> I think you want to insert an 'error' tag here ^^^ >> > + message='expected failure, but >> > got ' >> > + 'crash') >> > else: >> > res = etree.SubElement(element, 'error') >> > >> > -- >> > 2.7.3 > > After discussing this offline, you are right. > > I've made the change locally, do you want to see a v2? > > Dylan ___ Piglit mailing list Piglit@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/piglit
Re: [Piglit] [PATCH] framework: Add handler for failure and error when expecting the other
Quoting Mark Janes (2016-03-15 17:40:16) > Dylan Baker writes: > > > Right now the JUnit backend knows what to do with a failure when it > > expects one (or a crash), but not what to do when it expects a failure > > and gets a crash (or vice versa). > > > > This patch teaches it what to do in that case, mark the test as a > > failure and give a message explaining why. > > > > cc: mark.a.ja...@intel.com > > signed-off-by: Dylan Baker > > --- > > framework/backends/junit.py | 12 > > 1 file changed, 12 insertions(+) > > > > diff --git a/framework/backends/junit.py b/framework/backends/junit.py > > index f9eec66..761b201 100644 > > --- a/framework/backends/junit.py > > +++ b/framework/backends/junit.py > > @@ -151,6 +151,12 @@ class JUnitBackend(FileBackend): > > err.text += "\n\nWARN: passing test as an expected > > failure" > > res = etree.SubElement(element, 'skipped', > > message='expected failure') > > +elif expected_result == 'error': > > +err.text += \ > > +"\n\nERROR: Test should have been crash but was > > failure" > > +res = etree.SubElement(element, 'failure', > > + message='expected crash, but > > got ' > > + 'failure') > > else: > > res = etree.SubElement(element, 'failure') > > > > @@ -159,6 +165,12 @@ class JUnitBackend(FileBackend): > > err.text += "\n\nWARN: passing test as an expected > > crash" > > res = etree.SubElement(element, 'skipped', > > message='expected crash') > > +elif expected_result == 'failure': > > +err.text += \ > > +"\n\nERROR: Test should have been failure but was > > crash" > > +res = etree.SubElement(element, 'failure', > I think you want to insert an 'error' tag here ^^^ I think they should have the same tag, whichever makes more sense, since it's a framework error (we expected A but got B) rather than a test error. Does that make sense or am I crazy? > > + message='expected failure, but > > got ' > > + 'crash') > > else: > > res = etree.SubElement(element, 'error') > > > > -- > > 2.7.3 signature.asc Description: signature ___ Piglit mailing list Piglit@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/piglit
Re: [Piglit] [PATCH] framework: Add handler for failure and error when expecting the other
Dylan Baker writes: > Right now the JUnit backend knows what to do with a failure when it > expects one (or a crash), but not what to do when it expects a failure > and gets a crash (or vice versa). > > This patch teaches it what to do in that case, mark the test as a > failure and give a message explaining why. > > cc: mark.a.ja...@intel.com > signed-off-by: Dylan Baker > --- > framework/backends/junit.py | 12 > 1 file changed, 12 insertions(+) > > diff --git a/framework/backends/junit.py b/framework/backends/junit.py > index f9eec66..761b201 100644 > --- a/framework/backends/junit.py > +++ b/framework/backends/junit.py > @@ -151,6 +151,12 @@ class JUnitBackend(FileBackend): > err.text += "\n\nWARN: passing test as an expected > failure" > res = etree.SubElement(element, 'skipped', > message='expected failure') > +elif expected_result == 'error': > +err.text += \ > +"\n\nERROR: Test should have been crash but was > failure" > +res = etree.SubElement(element, 'failure', > + message='expected crash, but got ' > + 'failure') > else: > res = etree.SubElement(element, 'failure') > > @@ -159,6 +165,12 @@ class JUnitBackend(FileBackend): > err.text += "\n\nWARN: passing test as an expected crash" > res = etree.SubElement(element, 'skipped', > message='expected crash') > +elif expected_result == 'failure': > +err.text += \ > +"\n\nERROR: Test should have been failure but was > crash" > +res = etree.SubElement(element, 'failure', I think you want to insert an 'error' tag here ^^^ > + message='expected failure, but > got ' > + 'crash') > else: > res = etree.SubElement(element, 'error') > > -- > 2.7.3 ___ Piglit mailing list Piglit@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/piglit
[Piglit] [PATCH] framework: Add handler for failure and error when expecting the other
Right now the JUnit backend knows what to do with a failure when it expects one (or a crash), but not what to do when it expects a failure and gets a crash (or vice versa). This patch teaches it what to do in that case, mark the test as a failure and give a message explaining why. cc: mark.a.ja...@intel.com signed-off-by: Dylan Baker --- framework/backends/junit.py | 12 1 file changed, 12 insertions(+) diff --git a/framework/backends/junit.py b/framework/backends/junit.py index f9eec66..761b201 100644 --- a/framework/backends/junit.py +++ b/framework/backends/junit.py @@ -151,6 +151,12 @@ class JUnitBackend(FileBackend): err.text += "\n\nWARN: passing test as an expected failure" res = etree.SubElement(element, 'skipped', message='expected failure') +elif expected_result == 'error': +err.text += \ +"\n\nERROR: Test should have been crash but was failure" +res = etree.SubElement(element, 'failure', + message='expected crash, but got ' + 'failure') else: res = etree.SubElement(element, 'failure') @@ -159,6 +165,12 @@ class JUnitBackend(FileBackend): err.text += "\n\nWARN: passing test as an expected crash" res = etree.SubElement(element, 'skipped', message='expected crash') +elif expected_result == 'failure': +err.text += \ +"\n\nERROR: Test should have been failure but was crash" +res = etree.SubElement(element, 'failure', + message='expected failure, but got ' + 'crash') else: res = etree.SubElement(element, 'error') -- 2.7.3 ___ Piglit mailing list Piglit@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/piglit
