[OE-core] [PATCH] testimage.bbclass: Add kernel provider and version to testresult

2019-10-02 Thread Yeoh Ee Peng
configuration section. Example of virtual/kernel and version added to testresult configuration: - "KERNEL_PROVIDER_VERSION": "linux-intel_4.19" Signed-off-by: Yeoh Ee Peng --- meta/classes/testimage.bbclass | 20 ++-- 1 file changed, 18 insertions(+), 2 deletions(-

[OE-core] [PATCH] resulttool/store.py: Enable add extra test environment data

2019-10-28 Thread Yeoh Ee Peng
environment used for bitbake Signed-off-by: Yeoh Ee Peng --- scripts/lib/resulttool/store.py | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/scripts/lib/resulttool/store.py b/scripts/lib/resulttool/store.py index 79c83dd..e0951f0 100644 --- a/scripts/lib/resulttool/store.py +++ b

[OE-core] [PATCH] scripts/oe-pkgdata-util: Enable list-pkgs to print ordered packages

2019-10-31 Thread Yeoh Ee Peng
The list-pkgs currently print packages in unordered format. Enable list-pkgs to print ordered packages that will ease viewing. Signed-off-by: Yeoh Ee Peng --- scripts/oe-pkgdata-util | 17 - 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/scripts/oe-pkgdata-util b

[OE-core] [PATCH 0/4] Resulttool minor enhancements

2019-11-07 Thread Yeoh Ee Peng
Resulttool minor enhancements - Enable report to use regression_map - Enable report to output raw test results - Enable report to print total test result statistic - Enable store to add extra test environment config data Yeoh Ee Peng (4): scripts/resulttool/report: Enable report to use

[OE-core] [PATCH 1/4] scripts/resulttool/report: Enable report to use regression_map

2019-11-07 Thread Yeoh Ee Peng
only showing partial result set. Signed-off-by: Yeoh Ee Peng --- scripts/lib/resulttool/report.py | 16 +++- scripts/lib/resulttool/resultutils.py | 4 ++-- 2 files changed, 13 insertions(+), 7 deletions(-) diff --git a/scripts/lib/resulttool/report.py b/scripts/lib/resulttool

[OE-core] [PATCH 2/4] scripts/resulttool/report: Enable output raw test results

2019-11-07 Thread Yeoh Ee Peng
In case of debugging, report user need to acccess the raw test result. Instead of going back to source file/directory/URL to manually pull out the raw result, provide alternative way to let report showing raw test results by providing the result id (optional). Signed-off-by: Yeoh Ee Peng

[OE-core] [PATCH 4/4] resulttool/store.py: Enable add extra test environment data

2019-11-07 Thread Yeoh Ee Peng
environment used for bitbake Signed-off-by: Yeoh Ee Peng --- scripts/lib/resulttool/store.py | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/scripts/lib/resulttool/store.py b/scripts/lib/resulttool/store.py index 79c83dd..e0951f0 100644 --- a/scripts/lib/resulttool/store.py +++ b

[OE-core] [PATCH 3/4] scripts/resulttool/report: Add total statistic to test result.

2019-11-07 Thread Yeoh Ee Peng
Add total passed, failed, and skipped statistic to test result. Signed-off-by: Yeoh Ee Peng --- scripts/lib/resulttool/report.py | 5 + scripts/lib/resulttool/template/test_report_full_text.txt | 3 ++- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a

[OE-core] [PATCH] oe-selftest: crosstap: add tests for crosstap script

2018-03-20 Thread Yeoh Ee Peng
tive on the host machine. Finally this test will boot the image with qemu and then execute crosstap script to print hello world on qemu. Signed-off-by: Yeoh Ee Peng --- meta/lib/oeqa/files/trace_begin_hello.stp | 5 +++ meta/lib/oeqa/selftest/cases/crosstap.py

Re: [OE-core] Openembedded-core Digest, Vol 86, Issue 203

2018-03-20 Thread Yeoh, Ee Peng
Status WW12?18 (Burton, Ross) -- Message: 1 Date: Tue, 20 Mar 2018 11:58:04 +0200 From: Alexander Kanavin To: Yeoh Ee Peng , openembedded-core@lists.openembedded.org Cc: Paul Eggleton Subject: Re: [OE-core] [PATCH] oe-selftest: crosstap: add tests for

Re: [OE-core] Openembedded-core Digest, Vol 86, Issue 203

2018-03-21 Thread Yeoh, Ee Peng
stap testcase as oe-selftest instead of runtime/testimage as the extra efforts required and the additional step required to build systemtap-native on the host machine. Please let me know your input and suggestion. Thank you very much! Thanks, Ee Peng -Original Message- From: Yeoh, Ee

Re: [OE-core] [PATCH] oe-selftest: crosstap: add tests for crosstap script

2018-03-21 Thread Yeoh, Ee Peng
anavin [mailto:alexander.kana...@linux.intel.com] Sent: Wednesday, March 21, 2018 9:45 PM To: Yeoh, Ee Peng ; openembedded-core@lists.openembedded.org Subject: Re: [OE-core] [PATCH] oe-selftest: crosstap: add tests for crosstap script On 03/20/2018 01:59 AM, Yeoh Ee Peng wrote: > +def start_qemu_with_i

Re: [OE-core] [PATCH] oe-selftest: crosstap: add tests for crosstap script

2018-03-22 Thread Yeoh, Ee Peng
Peng ; openembedded-core@lists.openembedded.org Cc: Paul Eggleton Subject: Re: [OE-core] [PATCH] oe-selftest: crosstap: add tests for crosstap script On 03/22/2018 02:50 AM, Yeoh, Ee Peng wrote: > From my understanding, oeqa.utils.commands.runqemu will start and stop qemu > in a singl

[OE-core] [PATCH] oe-selftest: crosstap: add tests for crosstap script

2018-03-27 Thread Yeoh Ee Peng
tive on the host machine. Finally this test will boot the image with qemu and then execute crosstap script to print hello world on qemu. Signed-off-by: Yeoh Ee Peng --- meta/lib/oeqa/files/trace_begin_hello.stp | 5 + meta/lib/oeqa/selftest/cases/crosstap.py | 25

Re: [OE-core] [PATCH] oe-selftest: crosstap: add tests for crosstap script

2018-03-27 Thread Yeoh, Ee Peng
To: Yeoh, Ee Peng ; openembedded-core@lists.openembedded.org Subject: Re: [OE-core] [PATCH] oe-selftest: crosstap: add tests for crosstap script On 03/27/2018 04:37 AM, Yeoh Ee Peng wrote: > QA team were testing crosstap script manually. Add automated tests and > systemtap file to tes

[OE-core] [PATCH] oe-selftest: qemutest: add tests for qemu boot and shutdown

2018-04-05 Thread Yeoh Ee Peng
image (iso and hddimg) can boot but this live image was not supported on other qemu architechture. Signed-off-by: Yeoh Ee Peng --- meta/lib/oeqa/selftest/cases/qemutest.py | 44 1 file changed, 44 insertions(+) create mode 100644 meta/lib/oeqa/selftest/cases

Re: [OE-core] [PATCH] oe-selftest: qemutest: add tests for qemu boot and shutdown

2018-04-05 Thread Yeoh, Ee Peng
herefore, this test was developed as independent selftest testcase. Please let me know your inputs and feedback. Thanks, Yeoh Ee Peng -Original Message- From: openembedded-core-boun...@lists.openembedded.org [mailto:openembedded-core-boun...@lists.openembedded.org] On Behalf Of Yeoh

Re: [OE-core] [PATCH] oe-selftest: qemutest: add tests for qemu boot and shutdown

2018-04-05 Thread Yeoh, Ee Peng
Noted, let me rework on this. Thank you for your feedback! -Original Message- From: Alexander Kanavin [mailto:alexander.kana...@linux.intel.com] Sent: Thursday, April 5, 2018 5:56 PM To: Yeoh, Ee Peng ; openembedded-core@lists.openembedded.org Cc: Eggleton, Paul Subject: Re: [OE-core

[OE-core] [PATCH] oe-selftest: runqemu: add tests for qemu boot and shutdown

2018-04-09 Thread Yeoh Ee Peng
QA team were testing qemu boot image and shutdown on each qemu architecture manually. Add automated test to test qemu boot on ext4 and nfs, finally check that it can shutdown properly. Signed-off-by: Yeoh Ee Peng --- meta/lib/oeqa/selftest/cases/runqemu.py | 61

Re: [OE-core] [PATCH] oe-selftest: runqemu: add tests for qemu boot and shutdown

2018-04-09 Thread Yeoh, Ee Peng
Noted, will add comment and resend the patch. Thank you for your inputs. -Original Message- From: Alexander Kanavin [mailto:alexander.kana...@linux.intel.com] Sent: Monday, April 9, 2018 8:34 PM To: Yeoh, Ee Peng ; openembedded-core@lists.openembedded.org Subject: Re: [OE-core] [PATCH

[OE-core] [PATCH] oe-selftest: runqemu: add tests for qemu boot and shutdown

2018-04-09 Thread Yeoh Ee Peng
-by: Yeoh Ee Peng --- meta/lib/oeqa/selftest/cases/runqemu.py | 61 +++-- 1 file changed, 59 insertions(+), 2 deletions(-) diff --git a/meta/lib/oeqa/selftest/cases/runqemu.py b/meta/lib/oeqa/selftest/cases/runqemu.py index 47d41f5..7288ab2 100644 --- a/meta/lib/oeqa

Re: [OE-core] [PATCH] oe-selftest: runqemu: add tests for qemu boot and shutdown

2018-04-09 Thread Yeoh, Ee Peng
...@lists.openembedded.org] On Behalf Of Yeoh Ee Peng Sent: Tuesday, April 10, 2018 1:44 AM To: openembedded-core@lists.openembedded.org Subject: [OE-core] [PATCH] oe-selftest: runqemu: add tests for qemu boot and shutdown QA team were testing qemu boot image and shutdown on each qemu architecture

Re: [OE-core] [PATCH] oe-selftest: runqemu: add tests for qemu boot and shutdown

2018-04-10 Thread Yeoh, Ee Peng
reason behind this. For the skipTest, I agreed with you, it shall be replace by assertion to highlight environment setup failure. Please let me know if you have any more input. Thank you very much! Thanks, Yeoh Ee Peng -Original Message- From: Mittal, Anuj Sent: Tuesday, April 10, 2018

Re: [OE-core] [PATCH] oe-selftest: runqemu: add tests for qemu boot and shutdown

2018-04-10 Thread Yeoh, Ee Peng
Noted, I understood it now. Let me enhance the comment in the source code itself. It make sense. Thank you for your inputs! -Original Message- From: Alexander Kanavin [mailto:alexander.kana...@linux.intel.com] Sent: Tuesday, April 10, 2018 3:30 PM To: Yeoh, Ee Peng ; openembedded

[OE-core] [PATCH] oe-selftest: runqemu: add tests for qemu boot and shutdown

2018-04-10 Thread Yeoh Ee Peng
QA team were testing qemu boot image and shutdown on each qemu architecture manually. Add automated test to test qemu boot on ext4 and nfs, finally check that it can shutdown properly. Signed-off-by: Yeoh Ee Peng --- meta/lib/oeqa/selftest/cases/runqemu.py | 70

Re: [OE-core] [PATCH] oe-selftest: runqemu: add tests for qemu boot and shutdown

2018-04-10 Thread Yeoh, Ee Peng
, Yeoh Ee Peng -Original Message- From: Richard Purdie [mailto:richard.pur...@linuxfoundation.org] Sent: Tuesday, April 10, 2018 5:41 PM To: Yeoh, Ee Peng ; openembedded-core@lists.openembedded.org Subject: Re: [OE-core] [PATCH] oe-selftest: runqemu: add tests for qemu boot and shutdown On

[OE-core] [PATCH] oe-selftest: crosstap: add tests for crosstap script

2018-04-10 Thread Yeoh Ee Peng
tive on the host machine. Finally this test will boot the image with qemu and then execute crosstap script to print hello world on qemu. Signed-off-by: Yeoh Ee Peng --- meta/lib/oeqa/files/trace_begin_hello.stp | 5 + meta/lib/oeqa/selftest/cases/crosstap.py | 25

Re: [OE-core] [PATCH] oe-selftest: runqemu: add tests for qemu boot and shutdown

2018-04-20 Thread Yeoh, Ee Peng
Richard, Noted, thank you very much! It was fine, QA team shall be able to manually trigger this selftest, at least it was more efficient than manually executing the testcase. Cheers, Yeoh Ee Peng -Original Message- From: Richard Purdie [mailto:richard.pur...@linuxfoundation.org

[OE-core] [PATCH] scripts/test-result-log: Store test result & log and view summary report

2018-07-11 Thread Yeoh Ee Peng
sult & log in GIT, user can view text-based summary report, track the history of test, as well as uing GIT DIFF commmand to perform regresssion comparison. Signed-off-by: Yeoh Ee Peng --- scripts/lib/testresultlog/__init__.py | 0 .../template/test_report_full_text.txt

[OE-core] [PATCH] scripts/test-result-log: Store test result & log and view summary report

2018-07-12 Thread Yeoh Ee Peng
t-result-log --help Signed-off-by: Yeoh Ee Peng --- scripts/lib/testresultlog/__init__.py | 0 .../template/test_report_full_text.txt | 33 ++ scripts/lib/testresultlog/testlogparser.py | 97 ++ scripts/lib/testresultlog/testresultgitstore.py

[OE-core] [PATCH] test-result-log: testcase management tool to store and view test result

2018-07-31 Thread Yeoh Ee Peng
elow $ test-result-log view Signed-off-by: Yeoh Ee Peng --- scripts/lib/testresultlog/gitstore.py | 387 + scripts/lib/testresultlog/oeqalogparser.py | 97 ++ scripts/lib/testresultlog/oeqatestdiscover.py | 65 scripts/lib/testresul

Re: [OE-core] [PATCH] test-result-log: testcase management tool to store and view test result

2018-07-31 Thread Yeoh, Ee Peng
testcase execution and view test summary report. There are another patch planned to enable manual test execution and store/view test result. Please review this patch and provide us your valuable inputs and feedbacks. Thank you very much for your attention! Best regards, Yeoh Ee Peng -Original

[OE-core] [PATCH] test-result-log: testcase management tool to store test result

2018-08-10 Thread Yeoh Ee Peng
run the entry point script to look for help information. $ test-result-log To store test result for OEQA automated testcase, execute the below $ test-result-log store-auto / Signed-off-by: Yeoh Ee Peng --- scripts/lib/testresultlog/gitstore.py |

Re: [OE-core] [PATCH] test-result-log: testcase management tool to store test result

2018-08-10 Thread Yeoh, Ee Peng
regards, Yeoh Ee Peng -Original Message- From: Yeoh, Ee Peng Sent: Friday, August 10, 2018 4:23 PM To: openembedded-core@lists.openembedded.org Cc: Yeoh, Ee Peng Subject: [PATCH] test-result-log: testcase management tool to store test result These scripts were developed as an

Re: [OE-core] [PATCH] test-result-log: testcase management tool to store test result

2018-08-10 Thread Yeoh, Ee Peng
Same codes were available in github with better viewing and reviewing. https://github.com/epyeoh/openembedded-core/tree/test-result-log/scripts/lib/testresultlog -Original Message- From: Yeoh, Ee Peng Sent: Friday, August 10, 2018 4:42 PM To: openembedded-core@lists.openembedded.org Cc

[OE-core] [PATCH] test-result-log: testcase management tool to store test result

2018-08-28 Thread Yeoh Ee Peng
run the entry point script to look for help. $ test-result-log To store test result for OEQA automated testcase, execute the below $ test-result-log store-auto / Signed-off-by: Yeoh Ee Peng --- scripts/lib/testresultlog/__init__.py | 1 + scripts/lib/testresul

[OE-core] [PATCH] test-result-log: testcase management tool to store test result

2018-08-29 Thread Yeoh Ee Peng
run the entry point script to look for help. $ test-result-log To store test result for OEQA automated testcase, execute the below $ test-result-log store-auto / Signed-off-by: Yeoh Ee Peng --- scripts/lib/testresultlog/__init__.py | 1 + scripts/lib/testresul

[OE-core] [PATCH] oeqa/runtime/dnf: Make sure test_dnf_install does not skipped

2018-08-29 Thread Yeoh Ee Peng
. Signed-off-by: Yeoh Ee Peng --- meta/lib/oeqa/runtime/cases/dnf.py | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/meta/lib/oeqa/runtime/cases/dnf.py b/meta/lib/oeqa/runtime/cases/dnf.py index 67484bc..e93a1ce 100644 --- a/meta/lib/oeqa/runtime/cases/dnf.py +++ b/meta

[OE-core] [PATCH] insane: add qa check for lowercase recipe name

2017-08-15 Thread Yeoh Ee Peng
recipe name is not lowercase. [YOCTO# 11592] Signed-off-by: Yeoh Ee Peng --- meta/classes/insane.bbclass | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass index b7177c9..8e062d1 100644 --- a/meta/classes

[OE-core] [PATCH] insane: add qa check for uppercase recipe name

2017-08-17 Thread Yeoh Ee Peng
recipe name is uppercase. [YOCTO# 11592] Signed-off-by: Yeoh Ee Peng --- meta/classes/insane.bbclass | 8 ++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass index b7177c9..86237a8 100644 --- a/meta/classes/insane.bbclass

[OE-core] [PATCH] insane: add qa check for uppercase recipe name

2017-08-28 Thread Yeoh Ee Peng
recipe name is uppercase. [YOCTO# 11592] Signed-off-by: Yeoh Ee Peng --- meta/classes/insane.bbclass | 9 +++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass index b7177c9..fe0d744 100644 --- a/meta/classes

Re: [OE-core] [PATCH] test-result-log: testcase management tool to store test result

2018-09-12 Thread Yeoh, Ee Peng
Hi Richard, Great thanks for your feedbacks! I will work on the improvement to address your concerns. Best regards, Ee Peng -Original Message- From: richard.pur...@linuxfoundation.org [mailto:richard.pur...@linuxfoundation.org] Sent: Tuesday, September 11, 2018 11:09 PM To: Yeoh, Ee

Re: [OE-core] [PATCH] test-result-log: testcase management tool to store test result

2018-09-19 Thread Yeoh, Ee Peng
as expected. Please let me know if you have any more feedback. Thank you very much! Best regards, Yeoh Ee Peng -Original Message- From: richard.pur...@linuxfoundation.org [mailto:richard.pur...@linuxfoundation.org] Sent: Tuesday, September 11, 2018 11:09 PM To: Yeoh, Ee Peng

[OE-core] [PATCH 1/2] oeqa/core/runner: enable oeqa to write testresult into json file

2018-09-19 Thread Yeoh Ee Peng
management tools to write testresult for manual test case executed into json files. Signed-off-by: Yeoh Ee Peng --- meta/lib/oeqa/core/runner.py | 120 +++ 1 file changed, 109 insertions(+), 11 deletions(-) diff --git a/meta/lib/oeqa/core/runner.py b/meta

[OE-core] [PATCH 2/2] oeqa/selftest/context: enable selftest to write json testresult

2018-09-19 Thread Yeoh Ee Peng
To enable selftest to write testresult into json files, where these json files will be used by future test case management tools for test reporting. Signed-off-by: Yeoh Ee Peng --- meta/lib/oeqa/selftest/context.py | 12 ++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git

[OE-core] [PATCH 1/4] oeqa/runner: write testresult to json files

2018-10-02 Thread Yeoh Ee Peng
. message from unit test assertion) will be created for storing. Also the library class inside this patch will be reused by the future test-case-management tools to write json testresult for manual test case executed. Signed-off-by: Yeoh Ee Peng --- meta/lib/oeqa/core/runner.py | 137

[OE-core] [PATCH 4/4] testsdk.bbclass: write testresult to json files

2018-10-02 Thread Yeoh Ee Peng
disable this, specify OEQA_SKIP_OUTPUT_JSON="1" as configuration. Signed-off-by: Yeoh Ee Peng --- meta/classes/testsdk.bbclass | 24 ++-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git a/meta/classes/testsdk.bbclass b/meta/classes/testsdk.bbclass ind

[OE-core] [PATCH 3/4] testimage.bbclass: write testresult to json files

2018-10-02 Thread Yeoh Ee Peng
this, specify OEQA_SKIP_OUTPUT_JSON="1" as configuration. Signed-off-by: Yeoh Ee Peng --- meta/classes/testimage.bbclass | 12 1 file changed, 12 insertions(+) diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass index 39de191..0adaf60 100644 --- a/me

[OE-core] [PATCH 2/4] selftest/context: write testresult to json files

2018-10-02 Thread Yeoh Ee Peng
. To disable this, provide '-s' argument to oe-selftest execution. Signed-off-by: Yeoh Ee Peng --- meta/lib/oeqa/selftest/context.py | 12 ++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/meta/lib/oeqa/selftest/context.py b/meta/lib/oeqa/selftest/context.py ind

[OE-core] [PATCH] scripts/test-case-mgmt: store test result & log into git repository

2018-10-05 Thread Yeoh Ee Peng
nt, then run the entry point script to look for help. $ test-case-mgmt To store test result for OEQA automated testcase, execute the below $ test-case-mgmt store Signed-off-by: Yeoh Ee Peng --- scripts/lib/testcasemgmt/__init__.py | 0 scripts/lib/testcasemgmt/gitstore.

[OE-core] [PATCH] oeqa/manual: transfer manual test cases from testopia to oeqa

2018-10-11 Thread Yeoh Ee Peng
test execution. Any reader interested in understanding manual test cases will open and read these files. Signed-off-by: Yeoh Ee Peng --- meta/lib/oeqa/manual/abat.patch | 64 ++ meta/lib/oeqa/manual/bsp-hw.json | 1200 + meta/lib/oeqa/manual/bsp

[OE-core] [PATCH 4/4] testsdk.bbclass: write testresult to json files

2018-10-11 Thread Yeoh Ee Peng
disable this, specify OEQA_SKIP_OUTPUT_JSON="1" as configuration. Signed-off-by: Yeoh Ee Peng --- meta/classes/testsdk.bbclass | 22 ++ 1 file changed, 22 insertions(+) diff --git a/meta/classes/testsdk.bbclass b/meta/classes/testsdk.bbclass index d3f475d..6f81

[OE-core] [PATCH 3/4] testimage.bbclass: write testresult to json files

2018-10-11 Thread Yeoh Ee Peng
this, specify OEQA_SKIP_OUTPUT_JSON="1" as configuration. Signed-off-by: Yeoh Ee Peng --- meta/classes/testimage.bbclass | 12 1 file changed, 12 insertions(+) diff --git a/meta/classes/testimage.bbclass b/meta/classes/testimage.bbclass index 0e07afa..2901e10 100644 --- a/me

[OE-core] [PATCH 1/4] oeqa/core/runner: write testresult to json files

2018-10-11 Thread Yeoh Ee Peng
. message from unit test assertion) will be created for storing. Also the library class inside this patch will be reused by the future test-case-management tools to write json testresult for manual test case executed. Signed-off-by: Yeoh Ee Peng --- meta/lib/oeqa/core/runner.py | 132

[OE-core] [PATCH 2/4] oeqa/selftest/context: write testresult to json files

2018-10-11 Thread Yeoh Ee Peng
. To disable this, provide '-s' argument to oe-selftest execution. Signed-off-by: Yeoh Ee Peng --- meta/lib/oeqa/selftest/context.py | 12 ++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/meta/lib/oeqa/selftest/context.py b/meta/lib/oeqa/selftest/context.py ind

[OE-core] [PATCH 2/5] oeqa/core/runner: write testresult to json files

2018-10-15 Thread Yeoh Ee Peng
. message from unit test assertion) will be created for storing. Also the library class inside this patch will be reused by the future test-case-management tools to write json testresult for manual test case executed. Signed-off-by: Yeoh Ee Peng --- meta/lib/oeqa/core/runner.py | 64

[OE-core] [PATCH 1/5] oeqa/core/runner: refactor for OEQA to write json testresult

2018-10-15 Thread Yeoh Ee Peng
. Signed-off-by: Yeoh Ee Peng --- meta/lib/oeqa/core/runner.py | 70 ++-- 1 file changed, 29 insertions(+), 41 deletions(-) diff --git a/meta/lib/oeqa/core/runner.py b/meta/lib/oeqa/core/runner.py index eeb625b..5ee 100644 --- a/meta/lib/oeqa/core

[OE-core] [PATCH 3/5] oeqa/selftest/context: write testresult to json files

2018-10-15 Thread Yeoh Ee Peng
As part of the solution to replace Testopia to store testresult, OEQA selftest need to output testresult into json files, where these json testresult files will be stored into git repository by the future test-case-management tools. Signed-off-by: Yeoh Ee Peng --- meta/lib/oeqa/selftest

[OE-core] [PATCH 4/5] testimage.bbclass: write testresult to json files

2018-10-15 Thread Yeoh Ee Peng
As part of the solution to replace Testopia to store testresult, OEQA testimage need to output testresult into json files, where these json testresult files will be stored into git repository by the future test-case-management tools. Signed-off-by: Yeoh Ee Peng --- meta/classes

[OE-core] [PATCH 5/5] testsdk.bbclass: write testresult to json files

2018-10-15 Thread Yeoh Ee Peng
As part of the solution to replace Testopia to store testresult, OEQA sdk and sdkext need to output testresult into json files, where these json testresult files will be stored into git repository by the future test-case-management tools. Signed-off-by: Yeoh Ee Peng --- meta/classes

Re: [OE-core] [PATCH 1/4] oeqa/core/runner: write testresult to json files

2018-10-15 Thread Yeoh, Ee Peng
keeping test logs separate from testresult json file. Please let me know your thoughts and inputs. Thank you very much! Best regards, Yeoh Ee Peng [OE-core] [PATCH 1/5] oeqa/core/runner: refactor for OEQA to write json testresult http://lists.openembedded.org/pipermail/openembedded-core/2018

Re: [OE-core] [PATCH 1/5] oeqa/core/runner: refactor for OEQA to write json testresult

2018-10-15 Thread Yeoh, Ee Peng
Hi Richard, Thank you very much for the sharing! I will make the require enhancement asap. Thanks, Yeoh Ee Peng -Original Message- From: richard.pur...@linuxfoundation.org [mailto:richard.pur...@linuxfoundation.org] Sent: Monday, October 15, 2018 4:25 PM To: Yeoh, Ee Peng

Re: [OE-core] [PATCH 1/4] oeqa/core/runner: write testresult to json files

2018-10-15 Thread Yeoh, Ee Peng
. I agreed that these environments need to be store in the testresult json file as well. Let me work on the adding these environments information into the testresult json file. Thanks, Yeoh Ee Peng -Original Message- From: richard.pur...@linuxfoundation.org [mailto:richard.pur

Re: [OE-core] [PATCH 2/5] oeqa/core/runner: write testresult to json files

2018-10-15 Thread Yeoh, Ee Peng
thinking now. Attached sample data format Two section : 1. Test environments: storing all key-value pair information related to test environments 2. Test cases: storing test result and log Please let me know your inputs and thoughts. Thanks, Yeoh Ee Peng -Original Message- From

[OE-core] [PATCH 1/5] oeqa/core/runner: refactor for OEQA to write json testresult

2018-10-18 Thread Yeoh Ee Peng
. Signed-off-by: Yeoh Ee Peng --- meta/lib/oeqa/core/runner.py | 70 ++-- 1 file changed, 29 insertions(+), 41 deletions(-) diff --git a/meta/lib/oeqa/core/runner.py b/meta/lib/oeqa/core/runner.py index eeb625b..f1dd080 100644 --- a/meta/lib/oeqa/core

[OE-core] [PATCH 2/5] oeqa/core/runner: write testresult to json files

2018-10-18 Thread Yeoh Ee Peng
. message from unit test assertion) will be created for storing. Also the library class inside this patch will be reused by the future test-case-management tools to write json testresult for manual test case executed. Signed-off-by: Yeoh Ee Peng --- meta/lib/oeqa/core/runner.py | 30

[OE-core] [PATCH 4/5] testimage.bbclass: write testresult to json files

2018-10-18 Thread Yeoh Ee Peng
As part of the solution to replace Testopia to store testresult, OEQA testimage need to output testresult into json files, where these json testresult files will be stored into git repository by the future test-case-management tools. Signed-off-by: Yeoh Ee Peng --- meta/classes

[OE-core] [PATCH 3/5] oeqa/selftest/context: write testresult to json files

2018-10-18 Thread Yeoh Ee Peng
As part of the solution to replace Testopia to store testresult, OEQA selftest need to output testresult into json files, where these json testresult files will be stored into git repository by the future test-case-management tools. Signed-off-by: Yeoh Ee Peng --- meta/lib/oeqa/selftest

[OE-core] [PATCH 5/5] testsdk.bbclass: write testresult to json files

2018-10-18 Thread Yeoh Ee Peng
As part of the solution to replace Testopia to store testresult, OEQA sdk and sdkext need to output testresult into json files, where these json testresult files will be stored into git repository by the future test-case-management tools. Signed-off-by: Yeoh Ee Peng --- meta/classes

Re: [OE-core] [PATCH 2/5] oeqa/core/runner: write testresult to json files

2018-10-18 Thread Yeoh, Ee Peng
Hi Richard, Thank you very much for your great feedbacks and inputs! I had completed the refactor and resubmitted the patches as below. Please let me know if any more feedbacks and inputs! Thanks, Yeoh Ee Peng [1] http://lists.openembedded.org/pipermail/openembedded-core/2018-October/156835

Re: [OE-core] [PATCH 2/5] oeqa/core/runner: write testresult to json files

2018-10-19 Thread Yeoh, Ee Peng
Hi Richard, Thank you for your inputs! I am now working on the new features that you requested. Best regards, Yeoh Ee Peng -Original Message- From: richard.pur...@linuxfoundation.org [mailto:richard.pur...@linuxfoundation.org] Sent: Friday, October 19, 2018 5:56 AM To: Yeoh, Ee

[OE-core] [PATCH 2/4] oeqa/selftest/context: write testresult to json files

2018-10-22 Thread Yeoh Ee Peng
testresult to a single testresult file, user will define the variable "OEQA_JSON_RESULT_COMMON_DIR" with the common directory for writing json testresult. Signed-off-by: Yeoh Ee Peng --- meta/lib/oeqa/selftest/context.py | 35 --- 1 file changed, 32 insert

[OE-core] [PATCH 4/4] testsdk.bbclass: write testresult to json files

2018-10-22 Thread Yeoh Ee Peng
testresult to a single testresult file, user will define the variable "OEQA_JSON_RESULT_COMMON_DIR" with the common directory for writing json testresult. Signed-off-by: Yeoh Ee Peng --- meta/classes/testsdk.bbclass | 40 1 file changed, 36 insert

[OE-core] [PATCH 3/4] testimage.bbclass: write testresult to json files

2018-10-22 Thread Yeoh Ee Peng
testresult to a single testresult file, user will define the variable "OEQA_JSON_RESULT_COMMON_DIR" with the common directory for writing json testresult. Signed-off-by: Yeoh Ee Peng --- meta/classes/testimage.bbclass | 34 -- 1 file changed, 32 insert

[OE-core] [PATCH 1/4] oeqa/core/runner: write testresult to json files

2018-10-22 Thread Yeoh Ee Peng
write json testresult to a single testresult file, using locking a lockfile alongside the results file directory to prevent races. Also the library class inside this patch will be reused by the future test-case-management tools to write json testresult for manual test case executed. Signed-off-by: Ye

Re: [OE-core] [PATCH 2/5] oeqa/core/runner: write testresult to json files

2018-10-22 Thread Yeoh, Ee Peng
Hi Richard, I had done making the changes to match your inputs and suggestions. Below was the patches resubmitted. Please let me know if you have any more inputs or questions. Best regards, Yeoh Ee Peng http://lists.openembedded.org/pipermail/openembedded-core/2018-October/156926.html http

Re: [OE-core] [PATCH 2/5] oeqa/core/runner: write testresult to json files

2018-10-22 Thread Yeoh, Ee Peng
for the inconvenience. http://lists.openembedded.org/pipermail/openembedded-core/2018-October/156928.html http://lists.openembedded.org/pipermail/openembedded-core/2018-October/156929.html Best regards, Yeoh Ee Peng -Original Message- From: Yeoh, Ee Peng Sent: Monday, October 22, 2018

Re: [OE-core] [PATCH 1/4] oeqa/core/runner: write testresult to json files

2018-10-22 Thread Yeoh, Ee Peng
class responsibility as simple as to consume the configuration, results, result_id information provided by the test classes. Hope this explain the reason behind current design. Thank you very much for your attention & knowledge sharing! Thanks, Yeoh Ee Peng -Original Message- From:

Re: [OE-core] [PATCH 3/4] testimage.bbclass: write testresult to json files

2018-10-22 Thread Yeoh, Ee Peng
etVar("OEQA_JSON_RESULT_COMMON_DIR") > + if oeqa_json_result_common_dir: > +json_result_dir = oeqa_json_result_common_dir Best regards, Yeoh Ee Peng -Original Message- From: richard.pur...@linuxfoundation.org [mailto:richard.pur...@linuxfoundation.org] Sent: Monday, Oc

Re: [OE-core] [PATCH 1/4] oeqa/core/runner: write testresult to json files

2018-10-22 Thread Yeoh, Ee Peng
think we shall have OEQA_JSON_RESULT_DIR variable for user to define custom result_dir? Thanks, Yeoh Ee Peng -Original Message- From: richard.pur...@linuxfoundation.org [mailto:richard.pur...@linuxfoundation.org] Sent: Monday, October 22, 2018 5:34 PM To: Yeoh, Ee Peng ; openembedded-core@lists.open

[OE-core] [PATCH 1/4] oeqa/core/runner: write testresult to json files

2018-10-22 Thread Yeoh Ee Peng
write json testresult to a single testresult file, using locking a lockfile alongside the results file directory to prevent races. Also the library class inside this patch will be reused by the future test-case-management tools to write json testresult for manual test case executed. Signed-off-by: Ye

[OE-core] [PATCH 4/4] testsdk.bbclass: write testresult to json files

2018-10-22 Thread Yeoh Ee Peng
testresult to a single testresult file at custom direcotry, user will define the variable "OEQA_JSON_RESULT_DIR" with the custom directory for writing json testresult. Signed-off-by: Yeoh Ee Peng --- meta/classes/testsdk.bbclass | 36 1 file c

[OE-core] [PATCH 3/4] testimage.bbclass: write testresult to json files

2018-10-22 Thread Yeoh Ee Peng
testresult to a single testresult file at custom direcotry, user will define the variable "OEQA_JSON_RESULT_DIR" with the custom directory for writing json testresult. Signed-off-by: Yeoh Ee Peng --- meta/classes/testimage.bbclass | 31 +-- 1 file changed, 29

[OE-core] [PATCH 2/4] oeqa/selftest/context: write testresult to json files

2018-10-22 Thread Yeoh Ee Peng
testresult to a single testresult file at custom direcotry, user will define the variable "OEQA_JSON_RESULT_DIR" with the custom directory for writing json testresult. Signed-off-by: Yeoh Ee Peng --- meta/lib/oeqa/selftest/context.py | 36 +--- 1 file c

Re: [OE-core] [PATCH 1/4] oeqa/core/runner: write testresult to json files

2018-10-22 Thread Yeoh, Ee Peng
-core/2018-October/156946.html http://lists.openembedded.org/pipermail/openembedded-core/2018-October/156947.html http://lists.openembedded.org/pipermail/openembedded-core/2018-October/156948.html Thanks, Yeoh Ee Peng -Original Message- From: richard.pur...@linuxfoundation.org

[OE-core] [PATCH 2/4] oeqa/selftest/context: write testresult to json files

2018-10-22 Thread Yeoh Ee Peng
;oeqa" directory under the oe-selftest log directory. To configure multiple instances of bitbake to write json testresult to a single testresult file at custom directory, user will define the variable "OEQA_JSON_RESULT_DIR" with the custom directory for json testresult. Signed-off-by: Yeo

[OE-core] [PATCH 3/4] testimage.bbclass: write testresult to json files

2018-10-22 Thread Yeoh Ee Peng
;oeqa" directory under the "WORKDIR" directory. To configure multiple instances of bitbake to write json testresult to a single testresult file at custom directory, user will define the variable "OEQA_JSON_RESULT_DIR" with the custom directory for json testresult. Signed-o

[OE-core] [PATCH 1/4] oeqa/core/runner: write testresult to json files

2018-10-22 Thread Yeoh Ee Peng
write json testresult to a single testresult file, using locking a lockfile alongside the results file directory to prevent races. Also the library class inside this patch will be reused by the future test-case-management tools to write json testresult for manual test case executed. Signed-off-by: Ye

[OE-core] [PATCH 4/4] testsdk.bbclass: write testresult to json files

2018-10-22 Thread Yeoh Ee Peng
;oeqa" directory under the "WORKDIR" directory. To configure multiple instances of bitbake to write json testresult to a single testresult file at custom directory, user will define the variable "OEQA_JSON_RESULT_DIR" with the custom directory for json testresult. Signed-o

Re: [OE-core] [PATCH 1/4] oeqa/core/runner: write testresult to json files

2018-10-22 Thread Yeoh, Ee Peng
Hi Richard, I submitted the revised patches below. Sorry for the missing "version #" in the patch title. After this, I will add the "version #" into the patch title. Please let me know if any question or inputs. Thank you very much for your attention & sharing! Be

[OE-core] [PATCH v2] scripts/test-case-mgmt: store test result & log into git repository

2018-10-25 Thread Yeoh Ee Peng
nt, then run the entry point script to look for help. $ test-case-mgmt To store test result for OEQA automated testcase, execute the below $ test-case-mgmt store Signed-off-by: Yeoh Ee Peng --- scripts/lib/testcasemgmt/__init__.py | 0 scripts/lib/testcasemgmt/gitstore.

Re: [OE-core] [PATCH 1/4] oeqa/core/runner: write testresult to json files

2018-10-30 Thread Yeoh, Ee Peng
you again for sharing your inputs with us! Best regards, Yeoh Ee Peng -Original Message- From: richard.pur...@linuxfoundation.org [mailto:richard.pur...@linuxfoundation.org] Sent: Monday, October 29, 2018 9:59 PM To: Yeoh, Ee Peng ; openembedded-core@lists.openembedded.org Subject:

[OE-core] [PATCH] oeqa/selftest: Standardize json logging output directory

2018-11-06 Thread Yeoh Ee Peng
Currently sdk & sdkext will output json file to LOG_DIR, while selftest will output json file to TOPDIR/log. Standardize selftest json output file to LOG_DIR. Signed-off-by: Yeoh Ee Peng --- meta/lib/oeqa/selftest/context.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --g

[OE-core] [PATCH v3] scripts/test-case-mgmt: store test result file in git repository

2018-11-07 Thread Yeoh Ee Peng
nt, then run the entry point script to look for help. $ test-case-mgmt To store test result for OEQA automated testcase, execute the below $ test-case-mgmt store Signed-off-by: Yeoh Ee Peng --- scripts/lib/testcasemgmt/__init__.py | 0 scripts/lib/testcasemgmt/gitstore.

[OE-core] [PATCH] oeqa/runtime: Add tests for gstreamer CLI

2018-11-08 Thread Yeoh Ee Peng
dline tools (gst-inspect-1.0 & gst-launch-1.0). [YOCTO #11418] Signed-off-by: Yeoh Ee Peng --- meta/lib/oeqa/runtime/cases/gstreamer.py | 11 +++ 1 file changed, 11 insertions(+) create mode 100644 meta/lib/oeqa/runtime/cases/gstreamer.py diff --git a/meta/lib/oeqa/runt

[OE-core] [PATCH v2] oeqa/runtime: Add tests for gstreamer CLI

2018-11-08 Thread Yeoh Ee Peng
dline tools (gst-inspect-1.0 & gst-launch-1.0). [YOCTO #11418] Signed-off-by: Yeoh Ee Peng --- meta/lib/oeqa/runtime/cases/gstreamer.py | 14 ++ 1 file changed, 14 insertions(+) create mode 100644 meta/lib/oeqa/runtime/cases/gstreamer.py diff --git a/meta/lib/oeqa/runt

Re: [OE-core] [PATCH] oeqa/runtime: Add tests for gstreamer CLI

2018-11-08 Thread Yeoh, Ee Peng
sday, November 8, 2018 4:33 PM To: Yeoh, Ee Peng ; openembedded-core@lists.openembedded.org Subject: Re: [OE-core] [PATCH] oeqa/runtime: Add tests for gstreamer CLI On Thu, 2018-11-08 at 16:02 +0800, Yeoh Ee Peng wrote: > We should not be finding GStreamer issues right around release time >

Re: [OE-core] [PATCH 01/12] build-appliance: Drop test which is already automated

2018-11-11 Thread Yeoh, Ee Peng
uot;$ {AUTOREV} " 4. Run "bitbake build-appliance-image" I am sorry for the confusion it created. Best regards, Yeoh Ee Peng -Original Message- From: openembedded-core-boun...@lists.openembedded.org [mailto:openembedded-core-boun...@lists.openembedded.org] On Behalf Of

[OE-core] [PATCH 1/4] runtime/context: qemu do not need ip and server ip inputs from external

2018-11-13 Thread Yeoh Ee Peng
Qemu do not use the ip and server ip inputs from external. It will assign ip and server ip after initialize qemu. Signed-off-by: Yeoh Ee Peng --- meta/lib/oeqa/runtime/context.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/lib/oeqa/runtime/context.py b/meta/lib/oeqa

[OE-core] [PATCH 3/4] oeqa/qemu: Add support for slirp

2018-11-13 Thread Yeoh Ee Peng
Enable qemu for slirp. Initialize Qemurunner with slirp. Setup ip and port attribute to enable connection with qemu running with slirp. [YOCTO#10713] Signed-off-by: Yeoh Ee Peng --- meta/lib/oeqa/core/target/qemu.py | 13 +++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff

  1   2   3   >