Fabiano Rosas <faro...@suse.de> writes: > Alex Bennée <alex.ben...@linaro.org> writes: > >> Juan Quintela <quint...@redhat.com> writes: >> >>> From: Fabiano Rosas <faro...@suse.de> >>> >>> Add a smoke test that migrates to a file and gives it to the >>> script. It should catch the most annoying errors such as changes in >>> the ram flags. >>> >>> After code has been merged it becomes way harder to figure out what is >>> causing the script to fail, the person making the change is the most >>> likely to know right away what the problem is. >>> >>> Signed-off-by: Fabiano Rosas <faro...@suse.de> >>> Acked-by: Thomas Huth <th...@redhat.com> >>> Reviewed-by: Juan Quintela <quint...@redhat.com> >>> Signed-off-by: Juan Quintela <quint...@redhat.com> >>> Message-ID: <20231009184326.15777-7-faro...@suse.de> >> >> I bisected the failures I'm seeing on s390x to the introduction of this >> script. I don't know if its simply a timeout on a relatively slow VM: > > What's the range of your bisect? That test has been disabled and then > reenabled on s390x. It could be tripping the bisect. > > 04131e0009 ("tests/qtest/migration-test: Disable the analyze-migration.py > test on s390x") > 81c2c9dd5d ("tests/qtest/migration-test: Fix analyze-migration.py for > s390x")
I ran between v8.1.0 and master. But it is still failing @ HEAD 01782d6b294f95bcde334386f0aaac593cd28c0d as you can see from the run I did at the end. > > I don't think that test itself could be timing out. It's a very simple > test. It runs a migration and then uses the output to validate the > script. > > I don't have a Z machine at hand and the migration tests only run with > KVM for s390x, so it would be useful to take a look at meson's > testlog.txt so we can see which test is failing and hopefully in what > way it is failing. > > If you're up for it, running this in a loop is usually the best way to > catch any intermittent issues: > > QTEST_QEMU_BINARY=./qemu-system-x86_64 ./tests/qtest/migration-test > > And once you figure out which test, there's this monstrosity: > > QTEST_QEMU_BINARY='gdb -q --ex "set pagination off" \ > --ex "set print thread-events off" \ > --ex "handle SIGUSR1 noprint" \ > --ex "handle SIGPIPE noprint" \ > --ex "run" --ex "quit \$_exitcode" \ > --args ./qemu-system-x86_64' \ > gdb -q --ex "set prompt (qtest) " \ > --ex "handle SIGPIPE noprint" \ > --args ./tests/qtest/migration-test -p > /x86_64/migration/<some>/<test> > >> Summary of Failures: >> >> 36/546 qemu:qtest+qtest-s390x / qtest-s390x/migration-test >> ERROR 93.51s killed by signal 6 SIGABRT >> >> It seems to be unstable as we pass sometimes: >> >> 11:26:42 [ajb@qemu01:~/l/q/b/system] master|… + ./pyvenv/bin/meson test >> --repeat 100 qtest-s390x/migration-test >> ninja: Entering directory `/home/ajb/lsrc/qemu.git/builds/system' >> [1/9] Generating qemu-version.h with a custom command (wrapped by meson to >> capture output) >> 1/100 qemu:qtest+qtest-s390x / qtest-s390x/migration-test ERROR >> 251.98s killed by signal 6 SIGABRT >>>>> MALLOC_PERTURB_=9 >>>>> PYTHON=/home/ajb/lsrc/qemu.git/builds/system/pyvenv/bin/python3 >>>>> G_TEST_DBUS_DAEMON=/home/ajb/lsrc/qemu.git/tests/dbus-vmstate-daemon.sh >>>>> QTEST_QEMU_BINARY=./qemu-system-s390x QTEST_QEMU_IMG=./qemu-img >>>>> QTEST_QEMU_STORAGE_DAEMON_BINARY=./storage-daemon/qemu-storage-daemon >>>>> /home/ajb/lsrc/qemu.git/builds/system/tests/qtest/migration-test >>>>> --tap -k >> >> 2/100 qemu:qtest+qtest-s390x / qtest-s390x/migration-test ERROR >> 258.71s killed by signal 6 SIGABRT >>>>> PYTHON=/home/ajb/lsrc/qemu.git/builds/system/pyvenv/bin/python3 >>>>> MALLOC_PERTURB_=205 >>>>> G_TEST_DBUS_DAEMON=/home/ajb/lsrc/qemu.git/tests/dbus-vmstate-daemon.sh >>>>> QTEST_QEMU_BINARY=./qemu-system-s390x QTEST_QEMU_IMG=./qemu-img >>>>> QTEST_QEMU_STORAGE_DAEMON_BINARY=./storage-daemon/qemu-storage-daemon >>>>> /home/ajb/lsrc/qemu.git/builds/system/tests/qtest/migration-test >>>>> --tap -k >> >> 3/100 qemu:qtest+qtest-s390x / qtest-s390x/migration-test OK >> 302.53s 46 subtests passed >> 4/100 qemu:qtest+qtest-s390x / qtest-s390x/migration-test OK >> 319.56s 46 subtests passed >> 5/100 qemu:qtest+qtest-s390x / qtest-s390x/migration-test OK >> 320.11s 46 subtests passed >> 6/100 qemu:qtest+qtest-s390x / qtest-s390x/migration-test OK >> 328.40s 46 subtests passed >> >> Ok: 4 >> Expected Fail: 0 >> Fail: 2 >> Unexpected Pass: 0 >> Skipped: 0 >> Timeout: 0 >> >>> --- >>> tests/qtest/migration-test.c | 60 ++++++++++++++++++++++++++++++++++++ >>> tests/qtest/meson.build | 2 ++ >>> 2 files changed, 62 insertions(+) >>> >>> diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c >>> index 8eb2053dbb..cef5081f8c 100644 >>> --- a/tests/qtest/migration-test.c >>> +++ b/tests/qtest/migration-test.c >>> @@ -66,6 +66,8 @@ static bool got_dst_resume; >>> */ >>> #define DIRTYLIMIT_TOLERANCE_RANGE 25 /* MB/s */ >>> >>> +#define ANALYZE_SCRIPT "scripts/analyze-migration.py" >>> + >>> #if defined(__linux__) >>> #include <sys/syscall.h> >>> #include <sys/vfs.h> >>> @@ -1501,6 +1503,61 @@ static void test_baddest(void) >>> test_migrate_end(from, to, false); >>> } >>> >>> +#ifndef _WIN32 >>> +static void test_analyze_script(void) >>> +{ >>> + MigrateStart args = { >>> + .opts_source = "-uuid 11111111-1111-1111-1111-111111111111", >>> + }; >>> + QTestState *from, *to; >>> + g_autofree char *uri = NULL; >>> + g_autofree char *file = NULL; >>> + int pid, wstatus; >>> + const char *python = g_getenv("PYTHON"); >>> + >>> + if (!python) { >>> + g_test_skip("PYTHON variable not set"); >>> + return; >>> + } >>> + >>> + /* dummy url */ >>> + if (test_migrate_start(&from, &to, "tcp:127.0.0.1:0", &args)) { >>> + return; >>> + } >>> + >>> + /* >>> + * Setting these two capabilities causes the "configuration" >>> + * vmstate to include subsections for them. The script needs to >>> + * parse those subsections properly. >>> + */ >>> + migrate_set_capability(from, "validate-uuid", true); >>> + migrate_set_capability(from, "x-ignore-shared", true); >>> + >>> + file = g_strdup_printf("%s/migfile", tmpfs); >>> + uri = g_strdup_printf("exec:cat > %s", file); >>> + >>> + migrate_ensure_converge(from); >>> + migrate_qmp(from, uri, "{}"); >>> + wait_for_migration_complete(from); >>> + >>> + pid = fork(); >>> + if (!pid) { >>> + close(1); >>> + open("/dev/null", O_WRONLY); >>> + execl(python, python, ANALYZE_SCRIPT, "-f", file, NULL); >>> + g_assert_not_reached(); >>> + } >>> + >>> + g_assert(waitpid(pid, &wstatus, 0) == pid); >>> + if (WIFEXITED(wstatus) && WEXITSTATUS(wstatus) != 0) { >>> + g_test_message("Failed to analyze the migration stream"); >>> + g_test_fail(); >>> + } >>> + test_migrate_end(from, to, false); >>> + cleanup("migfile"); >>> +} >>> +#endif >>> + >>> static void test_precopy_common(MigrateCommon *args) >>> { >>> QTestState *from, *to; >>> @@ -2837,6 +2894,9 @@ int main(int argc, char **argv) >>> } >>> >>> qtest_add_func("/migration/bad_dest", test_baddest); >>> +#ifndef _WIN32 >>> + qtest_add_func("/migration/analyze-script", test_analyze_script); >>> +#endif >>> qtest_add_func("/migration/precopy/unix/plain", >>> test_precopy_unix_plain); >>> qtest_add_func("/migration/precopy/unix/xbzrle", >>> test_precopy_unix_xbzrle); >>> /* >>> diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build >>> index 66795cfcd2..d6022ebd64 100644 >>> --- a/tests/qtest/meson.build >>> +++ b/tests/qtest/meson.build >>> @@ -357,6 +357,8 @@ foreach dir : target_dirs >>> test_deps += [qsd] >>> endif >>> >>> + qtest_env.set('PYTHON', python.full_path()) >>> + >>> foreach test : target_qtests >>> # Executables are shared across targets, declare them only the first >>> time we >>> # encounter them -- Alex Bennée Virtualisation Tech Lead @ Linaro