On 4/18/23 03:10, Thomas Huth wrote:
On 17/04/2023 15.43, Alex Bennée wrote:
From: Kautuk Consul <kcon...@linux.vnet.ibm.com>
Avocado version 101.0 has a fix to re-compute the checksum
of an asset file if the algorithm used in the *-CHECKSUM
file isn't the same as the one being passed to it by the
avocado user (i.e. the avocado_qemu python module).
In the earlier avocado versions this fix wasn't there due
to which if the checksum wouldn't match the earlier
checksum (calculated by a different algorithm), the avocado
code would start downloading a fresh image from the internet
URL thus making the test-cases take longer to execute.
Bump up the avocado-framework version to 101.0.
Signed-off-by: Kautuk Consul <kcon...@linux.vnet.ibm.com>
Tested-by: Hariharan T S <hariharan...@linux.vnet.ibm.com>
Message-Id: <20230327115030.3418323-2-kcon...@linux.vnet.ibm.com>
Message-Id: <20230330101141.30199-10-alex.ben...@linaro.org>
---
v2
- limit --max-parallel-tasks $(JOBS_OPTION:-j%=%)
---
tests/Makefile.include | 18 +++++++++++-------
tests/requirements.txt | 2 +-
2 files changed, 12 insertions(+), 8 deletions(-)
diff --git a/tests/Makefile.include b/tests/Makefile.include
index 9422ddaece..a4de0ad5a2 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -138,14 +138,18 @@ get-vm-image-fedora-31-%: check-venv
# download all vm images, according to defined targets
get-vm-images: check-venv $(patsubst %,get-vm-image-fedora-31-%,
$(FEDORA_31_DOWNLOAD))
+JOBS_OPTION=$(lastword -j1 $(filter-out -j, $(filter
-j%,$(MAKEFLAGS))))
+
check-avocado: check-venv $(TESTS_RESULTS_DIR) get-vm-images
- $(call quiet-command, \
- $(TESTS_PYTHON) -m avocado \
- --show=$(AVOCADO_SHOW) run
--job-results-dir=$(TESTS_RESULTS_DIR) \
- $(if $(AVOCADO_TAGS),, --filter-by-tags-include-empty \
- --filter-by-tags-include-empty-key) \
- $(AVOCADO_CMDLINE_TAGS) \
- $(if $(GITLAB_CI),,--failfast) $(AVOCADO_TESTS), \
+ $(call quiet-command, \
+ $(TESTS_PYTHON) -m avocado \
+ --show=$(AVOCADO_SHOW) run
--job-results-dir=$(TESTS_RESULTS_DIR) \
+ $(if $(AVOCADO_TAGS),, \
+ --filter-by-tags-include-empty \
+ --filter-by-tags-include-empty-key) \
+ --max-parallel-tasks $(JOBS_OPTION:-j%=%) \
+ $(AVOCADO_CMDLINE_TAGS) \
+ $(if $(GITLAB_CI),,--failfast)
$(AVOCADO_TESTS), \
We might need to revisit this --failfast logic, too. If I've got that
right, failfast is now the default with the new system? So we might
want to disable it by default again if GITLAB_CI is not set?
Thomas
Hi Thomas,
I must be missing something, because under Avocado 101.0, I'm getting
the following behavior without the --failfast flag:
$ avocado run --max-parallel-tasks=1 -- /bin/true /bin/false /bin/true
JOB ID : 646f476f01b8d5599a57530606de543f2d9a5366
JOB LOG :
/root/avocado/job-results/job-2023-04-21T04.15-646f476/job.log
(1/3) /bin/true: STARTED
(1/3) /bin/true: PASS (0.01 s)
(2/3) /bin/false: STARTED
(2/3) /bin/false: FAIL (0.01 s)
(3/3) /bin/true: STARTED
(3/3) /bin/true: PASS (0.01 s)
RESULTS : PASS 2 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT
0 | CANCEL 0
JOB TIME : 1.01 s
Test summary:
/bin/false: FAIL
And this with --failfast:
$ avocado run --failfast --max-parallel-tasks=1 -- /bin/true
/bin/false /bin/true
JOB ID : ae4894607a42194a7382efa545eccaccf7495fa3
JOB LOG :
/root/avocado/job-results/job-2023-04-21T04.17-ae48946/job.log
(1/3) /bin/true: STARTED
(1/3) /bin/true: PASS (0.01 s)
(2/3) /bin/false: STARTED
(2/3) /bin/false: FAIL (0.01 s)
Interrupting job (failfast).
RESULTS : PASS 1 | ERROR 0 | FAIL 1 | SKIP 1 | WARN 0 | INTERRUPT
0 | CANCEL 0
JOB TIME : 0.75 s
Test summary:
/bin/false: FAIL
Maybe it's something in the Makefile I'm missing, or something specific
to these tests, but that would be a long shot.
Thanks,
- Cleber.