[
https://issues.apache.org/jira/browse/QPIDIT-93?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16341138#comment-16341138
]
Kim van der Riet commented on QPIDIT-93:
----------------------------------------
With the above commit, I have taken the following approach:
As all the test information and results are contained within the
{{unittest.TestCase}} and {{unittest.TestResult}} objects, I created a new
{{xunit_log}} module which takes these and generates an xUnit-formatted report
using the standard Python {{xml.etree.cElementTree}} module to construct the
XML. To activate the xUnit logging, the {{--xunit-log}} parameter flag must be
present when running the test(s).
The xUnit XML is based on this specification found at
[GitHub|https://gist.github.com/erikd/4192748] (attached), a sample of the
output is also attached.
The only information valid to a test report, but which does not appear in the
xUnit XSD is a description of the broker (and optionally its config) and a full
list of parameter options (for example {{--no-skip --broker-type None
--xunit-log}}) used in the test. For example, the broker could be one of
several and which would produce different results. Additionally, for some
configurations (for example, using the [Qpid Dispatch
Router|https://qpid.apache.org/components/dispatch-router/index.html]), the
configuration of nodes and brokers (if any) is also relevant. It feels to me
as though there needs to be broker name, broker configuration and test
parameters elements and/or attributes in the schema somewhere. This particular
schema does not seem to be extensible. Am I missing something?
Ideas welcome.
> Optionally produce xUnit XML report with test results
> -----------------------------------------------------
>
> Key: QPIDIT-93
> URL: https://issues.apache.org/jira/browse/QPIDIT-93
> Project: Apache QPID Interoperability Test Suite
> Issue Type: Improvement
> Affects Versions: 0.1.0
> Reporter: Jiri Daněk
> Assignee: Kim van der Riet
> Priority: Minor
> Attachments: amqp_types_test.2018-01-26T09-24-24.xml, xunit.xsd
>
>
> Something like the following should do the job. Instead of
> https://pypi.python.org/pypi/unittest-xml-reporting, it might be possible to
> maybe switch from unittest package to py.test, which has xUnit reports built
> in.
> {noformat}
> diff --git a/src/python/qpid_interop_test/jms_messages_test.py
> b/src/python/qpid_interop_test/jms_messages_test.py
> index 3b54510..3b94f3a 100755
> --- a/src/python/qpid_interop_test/jms_messages_test.py
> +++ b/src/python/qpid_interop_test/jms_messages_test.py
> @@ -24,19 +24,17 @@ Module to test JMS message types across different APIs
> from json import dumps
> from os import getenv, path
> +import xmlrunner
> from proton import symbol
> from qpid_interop_test.test_type_map import TestTypeMap
> @@ -302,7 +300,7 @@ class TestOptions(object):
> Class controlling command-line arguments used to control the test.
> """
> def __init__(self, shim_map):
> - parser = argparse.ArgumentParser(description='Qpid-interop AMQP
> client interoparability test suite '
> + parser = argparse.ArgumentParser(description='Qpid-interop AMQP
> client interoperability test suite '
> 'for JMS message types')
> parser.add_argument('--sender', action='store',
> default='localhost:5672', metavar='IP-ADDR:PORT',
> help='Node to which test suite will send
> messages.')
> @@ -313,6 +311,8 @@ class TestOptions(object):
> parser.add_argument('--broker-type', action='store',
> metavar='BROKER_NAME',
> help='Disable test of broker type (using
> connection properties) by specifying the broker' +
> ' name, or "None".')
> + parser.add_argument('--xunit-report-dir', action='store',
> metavar='REPORTS_DIRECTORY',
> + help='Generate xUnit report into
> REPORTS_DIRECTORY/TEST-<test_name>-<timestamp>.xml')
> type_group = parser.add_mutually_exclusive_group()
> type_group.add_argument('--include-type', action='append',
> metavar='JMS_MESSAGE-TYPE',
> help='Name of JMS message type to include.
> Supported types:\n%s' %
> @@ -421,6 +421,7 @@ if __name__ == '__main__':
> TEST_SUITE.addTest(unittest.makeSuite(test_case_class))
>
> # Finally, run all the dynamically created tests
> - RES = unittest.TextTestRunner(verbosity=2).run(TEST_SUITE)
> +# RES = unittest.TextTestRunner(verbosity=2).run(TEST_SUITE) # type:
> unittest.TextTestResult
> + RES = xmlrunner.XMLTestRunner(verbosity=2).run(TEST_SUITE) # type:
> unittest.TextTestResult
> if not RES.wasSuccessful():
> sys.exit(1)
> {noformat}
> The main "problem" seems that each test has completely separate main method
> and builds up options from scratch. Adding this to every test would mean some
> unnecessary duplication. So maybe refactor this somewhat at first? On the
> other hand, there are good reasons to keep the tests simple...
> What do you think?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
