[OE-core] [PATCH 2/2] script/lib/testcasemgmt/manualexecution.py : manual helper script with bare-minimum function
From: Mazliana
Manual execution script is a helper script to execute all manual test cases in
baseline command,
which consists of user guideline steps and the expected results. The last step
will ask user to
provide their input to execute result. The input options are
passed/failed/blocked/skipped status.
The result given will be written in testresults.json including log error from
the user input
and configuration if there is any. The output test result for json file is
created by using
OEQA library.
The configuration part is manually key-in by the user. The system allow user to
specify how many
configuration they want to add and they need to define the required
configuration name and value
pair. In QA perspective, "configuration" means the test environments and
parameters used during
QA setup before testing can be carry out. Example of configurations: image used
for boot up, host
machine distro used, poky configurations, etc.
The purpose of adding the configuration is to standardize the output test
result format between
automation and manual execution.
scripts/test-case-mgmt: add "manualexecution" as a tool
Integrated the test-case-mgmt "store", "report" with "manual execution".Manual
test execution
is one of an alternative test case management tool of Testopia. This script has
only a
bare-minimum function. Bare-minimum function refer to function where the user
can only execute all of the
test cases that component have.
To use these scripts, first source oe environment, then run the entry point
script to look for help.
$ test-case-mgmt
To execute manual test cases, execute the below
$ test-case-mgmt manualexecution
By default testresults.json store in /tmp/log/manual
[YOCTO #12651]
Signed-off-by: Mazliana
---
scripts/lib/testcasemgmt/manualexecution.py | 142
scripts/test-case-mgmt | 11 ++-
2 files changed, 152 insertions(+), 1 deletion(-)
create mode 100644 scripts/lib/testcasemgmt/manualexecution.py
diff --git a/scripts/lib/testcasemgmt/manualexecution.py
b/scripts/lib/testcasemgmt/manualexecution.py
new file mode 100644
index 000..c6c450f
--- /dev/null
+++ b/scripts/lib/testcasemgmt/manualexecution.py
@@ -0,0 +1,142 @@
+# test case management tool - manual execution from testopia test cases
+#
+# Copyright (c) 2018, Intel Corporation.
+#
+# This program is free software; you can redistribute it and/or modify it
+# under the terms and conditions of the GNU General Public License,
+# version 2, as published by the Free Software Foundation.
+#
+# This program is distributed in the hope it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+# more details.
+#
+import argparse
+import json
+import os
+import sys
+import datetime
+import re
+from oeqa.core.runner import OETestResultJSONHelper
+
+class ManualTestRunner(object):
+def __init__(self):
+self.jdata = ''
+self.test_module = ''
+self.test_suite = ''
+self.test_case = ''
+self.configuration = ''
+self.starttime = ''
+self.result_id = ''
+self.write_dir = ''
+
+def _read_json(self, file):
+self.jdata = json.load(open('%s' % file))
+self.test_case = []
+self.test_module = self.jdata[0]['test']['@alias'].split('.', 2)[0]
+self.test_suite = self.jdata[0]['test']['@alias'].split('.', 2)[1]
+for i in range(0, len(self.jdata)):
+self.test_case.append(self.jdata[i]['test']['@alias'].split('.',
2)[2])
+
+def _get_input(self, config):
+while True:
+output = input('{} = '.format(config))
+if re.match('^[a-zA-Z0-9_]+$', output):
+break
+print('Only alphanumeric and underscore are allowed. Please try
again')
+return output
+
+def _create_config(self):
+self.configuration = {}
+while True:
+try:
+conf_total = int(input('\nPlease provide how many
configuration you want to save \n'))
+break
+except ValueError:
+print('Invalid input. Please provide input as a number not
character.')
+for i in range(conf_total):
+print('-')
+print('This is configuration #%s ' % (i + 1) + '. Please provide
configuration name and its value')
+print('-')
+name_conf = self._get_input('Configuration Name')
+value_conf = self._get_input('Configuration Value')
+print('-\n')
+self.configuration[name_conf.upper()] = value_conf
+current_datetime = datetime.datetime.now()
+self.starttime = current_datetime.strftime('%Y%m%d%H%M%S')
+
Re: [OE-core] [PATCH 2/2] script/lib/testcasemgmt/manualexecution.py : manual helper script with bare-minimum function
Please ignore this patch
-Original Message-
From: openembedded-core-boun...@lists.openembedded.org
[mailto:openembedded-core-boun...@lists.openembedded.org] On Behalf Of
mazliana.moha...@intel.com
Sent: Thursday, December 6, 2018 6:59 PM
To: openembedded-core@lists.openembedded.org
Subject: [OE-core] [PATCH 2/2] script/lib/testcasemgmt/manualexecution.py :
manual helper script with bare-minimum function
From: Mazliana
Manual execution is a helper script to execute all manual test cases in
baseline command. Basically, this script will show the steps and expected
results. Then, in the end of the steps, user need to give their input for the
result.
The input based on the passed/failed/skipped status. The result given will be
stored in testresults.json and it will stored the log error given by user input
& the configuration. The output test result for json file was created by using
OEQA library.
The configuration part is manually key-in by user. The system allow user to
specify how many configuration they want to add and they needs to define the
configuration name and value pair needed. The configuration part was added
because we want to standardize the output test result format between automation
and manual execution.
[YOCTO #12651]
Signed-off-by: Mazliana
---
scripts/lib/testcasemgmt/manualexecution.py | 141
1 file changed, 141 insertions(+)
create mode 100644 scripts/lib/testcasemgmt/manualexecution.py
diff --git a/scripts/lib/testcasemgmt/manualexecution.py
b/scripts/lib/testcasemgmt/manualexecution.py
new file mode 100644
index 000..2911c1e
--- /dev/null
+++ b/scripts/lib/testcasemgmt/manualexecution.py
@@ -0,0 +1,141 @@
+# test case management tool - manual execution from testopia test cases
+# # Copyright (c) 2018, Intel Corporation.
+#
+# This program is free software; you can redistribute it and/or modify
+it # under the terms and conditions of the GNU General Public License,
+# version 2, as published by the Free Software Foundation.
+#
+# This program is distributed in the hope it will be useful, but
+WITHOUT # ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+General Public License for # more details.
+#
+import argparse
+import json
+import os
+import sys
+import datetime
+import scriptpath
+import re
+scriptpath.add_oe_lib_path()
+scriptpath.add_bitbake_lib_path()
+import bb.utils
+from oeqa.core.runner import OETestResultJSONHelper
+
+class ManualTestRunner(object):
+def __init__(self):
+self.jdata = ''
+self.testmodule = ''
+self.testsuite = ''
+self.testcase = ''
+self.configuration = ''
+self.host_distro = ''
+self.host_name = ''
+self.machine = ''
+self.starttime = ''
+self.result_id = ''
+
+def read_json(self, file):
+self.jdata = json.load(open('%s' % file))
+self.testcase = []
+self.testmodule = self.jdata[0]['test']['@alias'].split('.', 2)[0]
+self.testsuite = self.jdata[0]['test']['@alias'].split('.', 2)[1]
+for i in range(0, len(self.jdata)):
+self.testcase.append(self.jdata[i]['test']['@alias'].split('.',
2)[2])
+return self.jdata, self.testmodule, self.testsuite,
+ self.testcase
+
+def get_input(self, config):
+while True:
+output = input('{} = '.format(config))
+if re.match('^[a-zA-Z0-9_]+$', output):
+break
+print('Only alphanumeric and underscore are allowed. Please try
again')
+return output
+
+def create_config(self):
+self.configuration = {}
+while True:
+try:
+conf_total = int(input('\nPlease provide how many
configuration you want to save \n'))
+break
+except ValueError:
+print('Invalid input. Please provide input as a number not
character.')
+for i in range(conf_total):
+print('-')
+print('This is your %s ' % (i + 1) + 'configuration. Please
provide configuration name and its value')
+print('-')
+self.name_conf = self.get_input('Configuration Name')
+self.value_conf = self.get_input('Configuration Value')
+print('-\n')
+self.configuration[self.name_conf.upper()] = self.value_conf
+self.currentDT = datetime.datetime.now()
+self.starttime = self.currentDT.strftime('%Y%m%d%H%M%S')
+self.test_type = self.testmodule
+self.configuration['STARTTIME'] = self.starttime
+self.configuration['TEST_TYPE'] = self.test_type
+return self.configuration
+
+def create_result_id(self):
+self.result_id = 'manual_' + self.test_type + '_' + self.start
[OE-core] [PATCH 2/2] script/lib/testcasemgmt/manualexecution.py : manual helper script with bare-minimum function
From: Mazliana
Manual execution is a helper script to execute all manual test cases in
baseline command. Basically, this script
will show the steps and expected results. Then, in the end of the steps, user
need to give their input for the result.
The input based on the passed/failed/skipped status. The result given will be
stored in testresults.json and it will
stored the log error given by user input & the configuration. The output test
result for json file was created by using
OEQA library.
The configuration part is manually key-in by user. The system allow user to
specify how many configuration they want to
add and they needs to define the configuration name and value pair needed. The
configuration part was added because we
want to standardize the output test result format between automation and manual
execution.
[YOCTO #12651]
Signed-off-by: Mazliana
---
scripts/lib/testcasemgmt/manualexecution.py | 141
1 file changed, 141 insertions(+)
create mode 100644 scripts/lib/testcasemgmt/manualexecution.py
diff --git a/scripts/lib/testcasemgmt/manualexecution.py
b/scripts/lib/testcasemgmt/manualexecution.py
new file mode 100644
index 000..2911c1e
--- /dev/null
+++ b/scripts/lib/testcasemgmt/manualexecution.py
@@ -0,0 +1,141 @@
+# test case management tool - manual execution from testopia test cases
+#
+# Copyright (c) 2018, Intel Corporation.
+#
+# This program is free software; you can redistribute it and/or modify it
+# under the terms and conditions of the GNU General Public License,
+# version 2, as published by the Free Software Foundation.
+#
+# This program is distributed in the hope it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+# more details.
+#
+import argparse
+import json
+import os
+import sys
+import datetime
+import scriptpath
+import re
+scriptpath.add_oe_lib_path()
+scriptpath.add_bitbake_lib_path()
+import bb.utils
+from oeqa.core.runner import OETestResultJSONHelper
+
+class ManualTestRunner(object):
+def __init__(self):
+self.jdata = ''
+self.testmodule = ''
+self.testsuite = ''
+self.testcase = ''
+self.configuration = ''
+self.host_distro = ''
+self.host_name = ''
+self.machine = ''
+self.starttime = ''
+self.result_id = ''
+
+def read_json(self, file):
+self.jdata = json.load(open('%s' % file))
+self.testcase = []
+self.testmodule = self.jdata[0]['test']['@alias'].split('.', 2)[0]
+self.testsuite = self.jdata[0]['test']['@alias'].split('.', 2)[1]
+for i in range(0, len(self.jdata)):
+self.testcase.append(self.jdata[i]['test']['@alias'].split('.',
2)[2])
+return self.jdata, self.testmodule, self.testsuite, self.testcase
+
+def get_input(self, config):
+while True:
+output = input('{} = '.format(config))
+if re.match('^[a-zA-Z0-9_]+$', output):
+break
+print('Only alphanumeric and underscore are allowed. Please try
again')
+return output
+
+def create_config(self):
+self.configuration = {}
+while True:
+try:
+conf_total = int(input('\nPlease provide how many
configuration you want to save \n'))
+break
+except ValueError:
+print('Invalid input. Please provide input as a number not
character.')
+for i in range(conf_total):
+print('-')
+print('This is your %s ' % (i + 1) + 'configuration. Please
provide configuration name and its value')
+print('-')
+self.name_conf = self.get_input('Configuration Name')
+self.value_conf = self.get_input('Configuration Value')
+print('-\n')
+self.configuration[self.name_conf.upper()] = self.value_conf
+self.currentDT = datetime.datetime.now()
+self.starttime = self.currentDT.strftime('%Y%m%d%H%M%S')
+self.test_type = self.testmodule
+self.configuration['STARTTIME'] = self.starttime
+self.configuration['TEST_TYPE'] = self.test_type
+return self.configuration
+
+def create_result_id(self):
+self.result_id = 'manual_' + self.test_type + '_' + self.starttime
+return self.result_id
+
+def execute_test_steps(self, testID):
+temp = {}
+testcaseID = self.testmodule + '.' + self.testsuite + '.' +
self.testcase[testID]
+
print('')
+print('Executing test case:' + '' '' + self.testcase[testID])
+
