Re: [PATCH net-next v8] selftests/vsock: add initial vmtest.sh for vsock
On Tue, May 27, 2025 at 05:55:13PM +0200, Stefano Garzarella wrote: > On Tue, May 27, 2025 at 07:30:29AM -0700, Bobby Eshleman wrote: > > On Mon, May 26, 2025 at 01:18:18PM +0200, Stefano Garzarella wrote: > > > On Thu, May 22, 2025 at 09:59:07PM -0700, Bobby Eshleman wrote: > > Yes, that would be great, but anyway for now I would say let's go with this > since we're supposed to support these current versions I guess, and this > hack at the end I think is doable. > sgtm! > BTW, thanks again for this useful work! > Stefano > No problem, happy to help! Best, Bobby
Re: [PATCH net-next v8] selftests/vsock: add initial vmtest.sh for vsock
On Tue, May 27, 2025 at 07:30:29AM -0700, Bobby Eshleman wrote: On Mon, May 26, 2025 at 01:18:18PM +0200, Stefano Garzarella wrote: On Thu, May 22, 2025 at 09:59:07PM -0700, Bobby Eshleman wrote: > This commit introduces a new vmtest.sh runner for vsock. > > It uses virtme-ng/qemu to run tests in a VM. The tests validate G2H, > H2G, and loopback. The testing tools from tools/testing/vsock/ are > reused. Currently, only vsock_test is used. > > VMCI and hyperv support is included in the config file to be built with > the -b option, though not used in the tests. > > Only tested on x86. > > To run: > > $ make -C tools/testing/selftests TARGETS=vsock > $ tools/testing/selftests/vsock/vmtest.sh > > or > > $ make -C tools/testing/selftests TARGETS=vsock run_tests > > Example runs (after make -C tools/testing/selftests TARGETS=vsock): > > $ ./tools/testing/selftests/vsock/vmtest.sh > 1..3 > ok 0 vm_server_host_client > ok 1 vm_client_host_server > ok 2 vm_loopback > SUMMARY: PASS=3 SKIP=0 FAIL=0 > Log: /tmp/vsock_vmtest_m7DI.log > > $ ./tools/testing/selftests/vsock/vmtest.sh vm_loopback > 1..1 > ok 0 vm_loopback > SUMMARY: PASS=1 SKIP=0 FAIL=0 > Log: /tmp/vsock_vmtest_a1IO.log > > $ mkdir -p ~/scratch > $ make -C tools/testing/selftests install TARGETS=vsock INSTALL_PATH=~/scratch > [... omitted ...] > $ cd ~/scratch > $ ./run_kselftest.sh > TAP version 13 > 1..1 > # timeout set to 300 > # selftests: vsock: vmtest.sh > # 1..3 > # ok 0 vm_server_host_client > # ok 1 vm_client_host_server > # ok 2 vm_loopback > # SUMMARY: PASS=3 SKIP=0 FAIL=0 > # Log: /tmp/vsock_vmtest_svEl.log > ok 1 selftests: vsock: vmtest.sh > > Future work can include vsock_diag_test. > > Because vsock requires a VM to test anything other than loopback, this > patch adds vmtest.sh as a kselftest itself. This is different than other > systems that have a "vmtest.sh", where it is used as a utility script to > spin up a VM to run the selftests as a guest (but isn't hooked into > kselftest). > > Signed-off-by: Bobby Eshleman > --- > Changes in v8: > - remove NIPA comment from commit msg > - remove tap_* functions and TAP_PREFIX > - add -b for building kernel > - Link to v7: https://lore.kernel.org/r/[email protected] > > Changes in v7: > - fix exit code bug when ran is kselftest: use cnt_total instead of KSFT_NUM_TESTS > - updated commit message with updated output > - updated commit message with commands for installing/running as > kselftest > - Link to v6: https://lore.kernel.org/r/[email protected] > > Changes in v6: > - add make cmd in commit message in vmtest.sh example (Stefano) > - check nonzero size of QEMU_PIDFILE using -s conditional (Stefano) > - display log file path after tests so it is easier to find amongst other random names > - cleanup qemu pidfile if qemu is unable to remove it > - make oops/warning failures more obvious with 'FAIL' prefix in log > (simply saying 'detected' wasn't clear enough to identify failing > condition) > - Link to v5: https://lore.kernel.org/r/[email protected] > > Changes in v5: > - make log file a tmpfile (Paolo) > - make sure both default and user defined QEMU gets handled by the dependency check (Paolo) > - increased VM boot up timeout from 1m to 3m for slow hosts (Paolo) > - rename vm_setup -> vm_start (Paolo) > - derive wait_for_listener from selftests/net/net_helper.sh to removes ss usage > - Remove unused 'unset IFS' line (Paolo) > - leave space after variable declarations (Paolo) > - make QEMU_PIDFILE a tmp file (Paolo) > - make everything readonly that is only read (Paolo) > - source ktap_helpers.sh for KSFT_PASS and friends (Paolo) > - don't check for timeout util (Paolo) > - add missing usage string for -q qemu arg > - add tap prefix to SUMMARY line since it isn't part of TAP protocol > - exit with the correct status code based on failure/pass counts > - Link to v4: https://lore.kernel.org/r/[email protected] > > Changes in v4: > - do not use special tab delimiter for help string parsing (Stefano + Paolo) > - fix paths for when installing kselftest and running out-of-tree (Paolo) > - change vng to using running kernel instead of compiled kernel (Paolo) > - use multi-line string for QEMU_OPTS (Stefano) > - change timeout to 300s (Paolo) > - skip if tools are not found and use kselftests status codes (Paolo) > - remove build from vmtest.sh (Paolo) > - change -> SSH_HOST_PORT (Stefano) > - add tap-format output > - add vmtest.log to gitignore > - check for vsock_test binary and remind user to build it if missing > - create a proper build in makefile > - style fixes > - add ssh, timeout, and pkill to dependency check, just in case > - fix numerical comparison in conditionals > - check qemu pidfile exists before proceeding (avoid wasting time waiting for ssh) > - fix tracking of pass/fail bug > - fix stderr redirection bug > - Link to v3: https://lo
Re: [PATCH net-next v8] selftests/vsock: add initial vmtest.sh for vsock
On Mon, May 26, 2025 at 01:18:18PM +0200, Stefano Garzarella wrote: > On Thu, May 22, 2025 at 09:59:07PM -0700, Bobby Eshleman wrote: > > This commit introduces a new vmtest.sh runner for vsock. > > > > It uses virtme-ng/qemu to run tests in a VM. The tests validate G2H, > > H2G, and loopback. The testing tools from tools/testing/vsock/ are > > reused. Currently, only vsock_test is used. > > > > VMCI and hyperv support is included in the config file to be built with > > the -b option, though not used in the tests. > > > > Only tested on x86. > > > > To run: > > > > $ make -C tools/testing/selftests TARGETS=vsock > > $ tools/testing/selftests/vsock/vmtest.sh > > > > or > > > > $ make -C tools/testing/selftests TARGETS=vsock run_tests > > > > Example runs (after make -C tools/testing/selftests TARGETS=vsock): > > > > $ ./tools/testing/selftests/vsock/vmtest.sh > > 1..3 > > ok 0 vm_server_host_client > > ok 1 vm_client_host_server > > ok 2 vm_loopback > > SUMMARY: PASS=3 SKIP=0 FAIL=0 > > Log: /tmp/vsock_vmtest_m7DI.log > > > > $ ./tools/testing/selftests/vsock/vmtest.sh vm_loopback > > 1..1 > > ok 0 vm_loopback > > SUMMARY: PASS=1 SKIP=0 FAIL=0 > > Log: /tmp/vsock_vmtest_a1IO.log > > > > $ mkdir -p ~/scratch > > $ make -C tools/testing/selftests install TARGETS=vsock > > INSTALL_PATH=~/scratch > > [... omitted ...] > > $ cd ~/scratch > > $ ./run_kselftest.sh > > TAP version 13 > > 1..1 > > # timeout set to 300 > > # selftests: vsock: vmtest.sh > > # 1..3 > > # ok 0 vm_server_host_client > > # ok 1 vm_client_host_server > > # ok 2 vm_loopback > > # SUMMARY: PASS=3 SKIP=0 FAIL=0 > > # Log: /tmp/vsock_vmtest_svEl.log > > ok 1 selftests: vsock: vmtest.sh > > > > Future work can include vsock_diag_test. > > > > Because vsock requires a VM to test anything other than loopback, this > > patch adds vmtest.sh as a kselftest itself. This is different than other > > systems that have a "vmtest.sh", where it is used as a utility script to > > spin up a VM to run the selftests as a guest (but isn't hooked into > > kselftest). > > > > Signed-off-by: Bobby Eshleman > > --- > > Changes in v8: > > - remove NIPA comment from commit msg > > - remove tap_* functions and TAP_PREFIX > > - add -b for building kernel > > - Link to v7: > > https://lore.kernel.org/r/[email protected] > > > > Changes in v7: > > - fix exit code bug when ran is kselftest: use cnt_total instead of > > KSFT_NUM_TESTS > > - updated commit message with updated output > > - updated commit message with commands for installing/running as > > kselftest > > - Link to v6: > > https://lore.kernel.org/r/[email protected] > > > > Changes in v6: > > - add make cmd in commit message in vmtest.sh example (Stefano) > > - check nonzero size of QEMU_PIDFILE using -s conditional (Stefano) > > - display log file path after tests so it is easier to find amongst other > > random names > > - cleanup qemu pidfile if qemu is unable to remove it > > - make oops/warning failures more obvious with 'FAIL' prefix in log > > (simply saying 'detected' wasn't clear enough to identify failing > > condition) > > - Link to v5: > > https://lore.kernel.org/r/[email protected] > > > > Changes in v5: > > - make log file a tmpfile (Paolo) > > - make sure both default and user defined QEMU gets handled by the > > dependency check (Paolo) > > - increased VM boot up timeout from 1m to 3m for slow hosts (Paolo) > > - rename vm_setup -> vm_start (Paolo) > > - derive wait_for_listener from selftests/net/net_helper.sh to removes ss > > usage > > - Remove unused 'unset IFS' line (Paolo) > > - leave space after variable declarations (Paolo) > > - make QEMU_PIDFILE a tmp file (Paolo) > > - make everything readonly that is only read (Paolo) > > - source ktap_helpers.sh for KSFT_PASS and friends (Paolo) > > - don't check for timeout util (Paolo) > > - add missing usage string for -q qemu arg > > - add tap prefix to SUMMARY line since it isn't part of TAP protocol > > - exit with the correct status code based on failure/pass counts > > - Link to v4: > > https://lore.kernel.org/r/[email protected] > > > > Changes in v4: > > - do not use special tab delimiter for help string parsing (Stefano + Paolo) > > - fix paths for when installing kselftest and running out-of-tree (Paolo) > > - change vng to using running kernel instead of compiled kernel (Paolo) > > - use multi-line string for QEMU_OPTS (Stefano) > > - change timeout to 300s (Paolo) > > - skip if tools are not found and use kselftests status codes (Paolo) > > - remove build from vmtest.sh (Paolo) > > - change -> SSH_HOST_PORT (Stefano) > > - add tap-format output > > - add vmtest.log to gitignore > > - check for vsock_test binary and remind user to build it if missing > > - create a proper build in makefile > > - style fixes > > - add ssh, timeout, and pkill to dependency check, just in ca
Re: [PATCH net-next v8] selftests/vsock: add initial vmtest.sh for vsock
On Thu, May 22, 2025 at 09:59:07PM -0700, Bobby Eshleman wrote: This commit introduces a new vmtest.sh runner for vsock. It uses virtme-ng/qemu to run tests in a VM. The tests validate G2H, H2G, and loopback. The testing tools from tools/testing/vsock/ are reused. Currently, only vsock_test is used. VMCI and hyperv support is included in the config file to be built with the -b option, though not used in the tests. Only tested on x86. To run: $ make -C tools/testing/selftests TARGETS=vsock $ tools/testing/selftests/vsock/vmtest.sh or $ make -C tools/testing/selftests TARGETS=vsock run_tests Example runs (after make -C tools/testing/selftests TARGETS=vsock): $ ./tools/testing/selftests/vsock/vmtest.sh 1..3 ok 0 vm_server_host_client ok 1 vm_client_host_server ok 2 vm_loopback SUMMARY: PASS=3 SKIP=0 FAIL=0 Log: /tmp/vsock_vmtest_m7DI.log $ ./tools/testing/selftests/vsock/vmtest.sh vm_loopback 1..1 ok 0 vm_loopback SUMMARY: PASS=1 SKIP=0 FAIL=0 Log: /tmp/vsock_vmtest_a1IO.log $ mkdir -p ~/scratch $ make -C tools/testing/selftests install TARGETS=vsock INSTALL_PATH=~/scratch [... omitted ...] $ cd ~/scratch $ ./run_kselftest.sh TAP version 13 1..1 # timeout set to 300 # selftests: vsock: vmtest.sh # 1..3 # ok 0 vm_server_host_client # ok 1 vm_client_host_server # ok 2 vm_loopback # SUMMARY: PASS=3 SKIP=0 FAIL=0 # Log: /tmp/vsock_vmtest_svEl.log ok 1 selftests: vsock: vmtest.sh Future work can include vsock_diag_test. Because vsock requires a VM to test anything other than loopback, this patch adds vmtest.sh as a kselftest itself. This is different than other systems that have a "vmtest.sh", where it is used as a utility script to spin up a VM to run the selftests as a guest (but isn't hooked into kselftest). Signed-off-by: Bobby Eshleman --- Changes in v8: - remove NIPA comment from commit msg - remove tap_* functions and TAP_PREFIX - add -b for building kernel - Link to v7: https://lore.kernel.org/r/[email protected] Changes in v7: - fix exit code bug when ran is kselftest: use cnt_total instead of KSFT_NUM_TESTS - updated commit message with updated output - updated commit message with commands for installing/running as kselftest - Link to v6: https://lore.kernel.org/r/[email protected] Changes in v6: - add make cmd in commit message in vmtest.sh example (Stefano) - check nonzero size of QEMU_PIDFILE using -s conditional (Stefano) - display log file path after tests so it is easier to find amongst other random names - cleanup qemu pidfile if qemu is unable to remove it - make oops/warning failures more obvious with 'FAIL' prefix in log (simply saying 'detected' wasn't clear enough to identify failing condition) - Link to v5: https://lore.kernel.org/r/[email protected] Changes in v5: - make log file a tmpfile (Paolo) - make sure both default and user defined QEMU gets handled by the dependency check (Paolo) - increased VM boot up timeout from 1m to 3m for slow hosts (Paolo) - rename vm_setup -> vm_start (Paolo) - derive wait_for_listener from selftests/net/net_helper.sh to removes ss usage - Remove unused 'unset IFS' line (Paolo) - leave space after variable declarations (Paolo) - make QEMU_PIDFILE a tmp file (Paolo) - make everything readonly that is only read (Paolo) - source ktap_helpers.sh for KSFT_PASS and friends (Paolo) - don't check for timeout util (Paolo) - add missing usage string for -q qemu arg - add tap prefix to SUMMARY line since it isn't part of TAP protocol - exit with the correct status code based on failure/pass counts - Link to v4: https://lore.kernel.org/r/[email protected] Changes in v4: - do not use special tab delimiter for help string parsing (Stefano + Paolo) - fix paths for when installing kselftest and running out-of-tree (Paolo) - change vng to using running kernel instead of compiled kernel (Paolo) - use multi-line string for QEMU_OPTS (Stefano) - change timeout to 300s (Paolo) - skip if tools are not found and use kselftests status codes (Paolo) - remove build from vmtest.sh (Paolo) - change -> SSH_HOST_PORT (Stefano) - add tap-format output - add vmtest.log to gitignore - check for vsock_test binary and remind user to build it if missing - create a proper build in makefile - style fixes - add ssh, timeout, and pkill to dependency check, just in case - fix numerical comparison in conditionals - check qemu pidfile exists before proceeding (avoid wasting time waiting for ssh) - fix tracking of pass/fail bug - fix stderr redirection bug - Link to v3: https://lore.kernel.org/r/[email protected] Changes in v3: - use common conditional syntax for checking variables - use return value instead of global rc - fix typo TEST_HOST_LISTENER_PORT -> TEST_HOST_PORT_LISTENER - use SIGTERM instead of SIGKILL on cleanup - use peer-cid=1 for loopback - change sleep delay times into globals - fix tes
