Re: [Qemu-devel] [PATCH for-2.12] memfd: fix vhost-user-test on non-memfd capable host
Hi, This series failed docker-build@min-glib build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. Type: series Message-id: 20180328121804.16203-1-marcandre.lur...@redhat.com Subject: [Qemu-devel] [PATCH for-2.12] memfd: fix vhost-user-test on non-memfd capable host === TEST SCRIPT BEGIN === #!/bin/bash set -e git submodule update --init dtc # Let docker tests dump environment info export SHOW_ENV=1 export J=8 time make docker-test-build@min-glib === TEST SCRIPT END === Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384 Switched to a new branch 'test' 12b7026b1b memfd: fix vhost-user-test on non-memfd capable host === OUTPUT BEGIN === Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc' Cloning into '/var/tmp/patchew-tester-tmp-14znopuf/src/dtc'... Submodule path 'dtc': checked out 'e54388015af1fb4bf04d0bca99caba1074d9cc42' BUILD min-glib make[1]: Entering directory '/var/tmp/patchew-tester-tmp-14znopuf/src' GEN /var/tmp/patchew-tester-tmp-14znopuf/src/docker-src.2018-03-31-02.57.01.19306/qemu.tar Cloning into '/var/tmp/patchew-tester-tmp-14znopuf/src/docker-src.2018-03-31-02.57.01.19306/qemu.tar.vroot'... done. Checking out files: 42% (2549/6066) Checking out files: 43% (2609/6066) Checking out files: 44% (2670/6066) Checking out files: 45% (2730/6066) Checking out files: 45% (2740/6066) Checking out files: 46% (2791/6066) Checking out files: 46% (2796/6066) Checking out files: 47% (2852/6066) Checking out files: 48% (2912/6066) Checking out files: 49% (2973/6066) Checking out files: 50% (3033/6066) Checking out files: 51% (3094/6066) Checking out files: 52% (3155/6066) Checking out files: 53% (3215/6066) Checking out files: 54% (3276/6066) Checking out files: 55% (3337/6066) Checking out files: 56% (3397/6066) Checking out files: 57% (3458/6066) Checking out files: 58% (3519/6066) Checking out files: 59% (3579/6066) Checking out files: 60% (3640/6066) Checking out files: 61% (3701/6066) Checking out files: 62% (3761/6066) Checking out files: 63% (3822/6066) Checking out files: 64% (3883/6066) Checking out files: 65% (3943/6066) Checking out files: 66% (4004/6066) Checking out files: 67% (4065/6066) Checking out files: 68% (4125/6066) Checking out files: 69% (4186/6066) Checking out files: 70% (4247/6066) Checking out files: 71% (4307/6066) Checking out files: 72% (4368/6066) Checking out files: 73% (4429/6066) Checking out files: 74% (4489/6066) Checking out files: 75% (4550/6066) Checking out files: 76% (4611/6066) Checking out files: 77% (4671/6066) Checking out files: 78% (4732/6066) Checking out files: 78% (4784/6066) Checking out files: 79% (4793/6066) Checking out files: 80% (4853/6066) Checking out files: 81% (4914/6066) Checking out files: 82% (4975/6066) Checking out files: 83% (5035/6066) Checking out files: 84% (5096/6066) Checking out files: 85% (5157/6066) Checking out files: 86% (5217/6066) Checking out files: 87% (5278/6066) Checking out files: 88% (5339/6066) Checking out files: 89% (5399/6066) Checking out files: 90% (5460/6066) Checking out files: 91% (5521/6066) Checking out files: 92% (5581/6066) Checking out files: 93% (5642/6066) Checking out files: 94% (5703/6066) Checking out files: 95% (5763/6066) Checking out files: 96% (5824/6066) Checking out files: 97% (5885/6066) Checking out files: 98% (5945/6066) Checking out files: 99% (6006/6066) Checking out files: 100% (6066/6066) Checking out files: 100% (6066/6066), done. Your branch is up-to-date with 'origin/test'. Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc' Cloning into '/var/tmp/patchew-tester-tmp-14znopuf/src/docker-src.2018-03-31-02.57.01.19306/qemu.tar.vroot/dtc'... Submodule path 'dtc': checked out 'e54388015af1fb4bf04d0bca99caba1074d9cc42' Submodule 'ui/keycodemapdb' (git://git.qemu.org/keycodemapdb.git) registered for path 'ui/keycodemapdb' Cloning into '/var/tmp/patchew-tester-tmp-14znopuf/src/docker-src.2018-03-31-02.57.01.19306/qemu.tar.vroot/ui/keycodemapdb'... Submodule path 'ui/keycodemapdb': checked out '6b3d716e2b6472eb7189d3220552280ef3d832ce' tar: /var/tmp/patchew-tester-tmp-14znopuf/src/docker-src.2018-03-31-02.57.01.19306/qemu.tar: Wrote only 2048 of 10240 bytes tar: Error is not recoverable: exiting now failed to create tar file COPYRUNNER RUN test-build in qemu:min-glib tar: Unexpected EOF in archive tar: Unexpected EOF in archive tar: Error is not recoverable: exiting now /var/tmp/qemu/run: line 32: prep_fail: command not found Environment variables: HOSTNAME=4b68edf62d68 MAKEFLAGS= -j8 J=8 CCACHE_DIR=/var/tmp/ccache EXTRA_CONFIGURE_OPTS= V= SHOW_ENV=1 PATH=/usr/lib/ccache:/usr/lib64/ccache
Re: [Qemu-devel] [PULL 0/1] RISC-V: Critical fixes for QEMU 2.12
Hi, This series failed docker-mingw@fedora build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. Type: series Message-id: 1522344417-60114-1-git-send-email-...@sifive.com Subject: [Qemu-devel] [PULL 0/1] RISC-V: Critical fixes for QEMU 2.12 === TEST SCRIPT BEGIN === #!/bin/bash set -e git submodule update --init dtc # Let docker tests dump environment info export SHOW_ENV=1 export J=8 time make docker-test-mingw@fedora === TEST SCRIPT END === Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384 Switched to a new branch 'test' 237fb031f4 RISC-V: Workaround for critical mstatus.FS bug === OUTPUT BEGIN === Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc' Cloning into '/var/tmp/patchew-tester-tmp-ysxzxo5z/src/dtc'... Submodule path 'dtc': checked out 'e54388015af1fb4bf04d0bca99caba1074d9cc42' BUILD fedora make[1]: Entering directory '/var/tmp/patchew-tester-tmp-ysxzxo5z/src' GEN /var/tmp/patchew-tester-tmp-ysxzxo5z/src/docker-src.2018-03-31-02.33.56.26135/qemu.tar Cloning into '/var/tmp/patchew-tester-tmp-ysxzxo5z/src/docker-src.2018-03-31-02.33.56.26135/qemu.tar.vroot'... done. Checking out files: 25% (1548/6066) Checking out files: 26% (1578/6066) Checking out files: 27% (1638/6066) Checking out files: 28% (1699/6066) Checking out files: 29% (1760/6066) Checking out files: 30% (1820/6066) Checking out files: 31% (1881/6066) Checking out files: 32% (1942/6066) Checking out files: 33% (2002/6066) Checking out files: 34% (2063/6066) Checking out files: 35% (2124/6066) Checking out files: 36% (2184/6066) Checking out files: 37% (2245/6066) Checking out files: 38% (2306/6066) Checking out files: 39% (2366/6066) Checking out files: 40% (2427/6066) Checking out files: 41% (2488/6066) Checking out files: 42% (2548/6066) Checking out files: 43% (2609/6066) Checking out files: 44% (2670/6066) Checking out files: 45% (2730/6066) Checking out files: 46% (2791/6066) Checking out files: 47% (2852/6066) Checking out files: 48% (2912/6066) Checking out files: 49% (2973/6066) Checking out files: 50% (3033/6066) Checking out files: 51% (3094/6066) Checking out files: 52% (3155/6066) Checking out files: 53% (3215/6066) Checking out files: 54% (3276/6066) Checking out files: 55% (3337/6066) Checking out files: 56% (3397/6066) Checking out files: 57% (3458/6066) Checking out files: 58% (3519/6066) Checking out files: 59% (3579/6066) Checking out files: 60% (3640/6066) Checking out files: 61% (3701/6066) Checking out files: 62% (3761/6066) Checking out files: 63% (3822/6066) Checking out files: 64% (3883/6066) Checking out files: 65% (3943/6066) Checking out files: 66% (4004/6066) Checking out files: 67% (4065/6066) Checking out files: 68% (4125/6066) Checking out files: 69% (4186/6066) Checking out files: 70% (4247/6066) Checking out files: 71% (4307/6066) Checking out files: 72% (4368/6066) Checking out files: 73% (4429/6066) Checking out files: 74% (4489/6066) Checking out files: 75% (4550/6066) Checking out files: 76% (4611/6066) Checking out files: 77% (4671/6066) Checking out files: 78% (4732/6066) Checking out files: 79% (4793/6066) Checking out files: 80% (4853/6066) Checking out files: 81% (4914/6066) Checking out files: 82% (4975/6066) Checking out files: 83% (5035/6066) Checking out files: 84% (5096/6066) Checking out files: 85% (5157/6066) Checking out files: 86% (5217/6066) Checking out files: 87% (5278/6066) Checking out files: 88% (5339/6066) Checking out files: 89% (5399/6066) Checking out files: 90% (5460/6066) Checking out files: 91% (5521/6066) Checking out files: 92% (5581/6066) Checking out files: 93% (5642/6066) Checking out files: 93% (5655/6066) Checking out files: 93% (5690/6066) Checking out files: 94% (5703/6066) Checking out files: 95% (5763/6066) Checking out files: 96% (5824/6066) Checking out files: 97% (5885/6066) Checking out files: 98% (5945/6066) Checking out files: 99% (6006/6066) Checking out files: 100% (6066/6066) Checking out files: 100% (6066/6066), done. Your branch is up-to-date with 'origin/test'. Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc' Cloning into '/var/tmp/patchew-tester-tmp-ysxzxo5z/src/docker-src.2018-03-31-02.33.56.26135/qemu.tar.vroot/dtc'... Submodule path 'dtc': checked out 'e54388015af1fb4bf04d0bca99caba1074d9cc42' Submodule 'ui/keycodemapdb' (git://git.qemu.org/keycodemapdb.git) registered for path 'ui/keycodemapdb' Cloning into '/var/tmp/patchew-tester-tmp-ysxzxo5z/src/docker-src.2018-03-31-02.33.56.26135/qemu.tar.vroot/ui/keycodemapdb'... Submodule path 'ui/keycodemapdb': checked out '6b3d716e2b6472e
Re: [Qemu-devel] [PULL 0/4] Block patches for 2.12.0-rc1
Hi, This series failed docker-build@min-glib build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. Type: series Message-id: 20180326202039.21070-1-mre...@redhat.com Subject: [Qemu-devel] [PULL 0/4] Block patches for 2.12.0-rc1 === TEST SCRIPT BEGIN === #!/bin/bash set -e git submodule update --init dtc # Let docker tests dump environment info export SHOW_ENV=1 export J=8 time make docker-test-build@min-glib === TEST SCRIPT END === Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384 Switched to a new branch 'test' 88d49f9d64 Merge remote-tracking branch 'remotes/rth/tags/pull-tcg-20180328' into staging d9d2cb9840 tcg: Mark muluh_i64 and mulsh_i64 as 64-bit ops === OUTPUT BEGIN === Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc' Cloning into '/var/tmp/patchew-tester-tmp-w6rnif1s/src/dtc'... Submodule path 'dtc': checked out 'e54388015af1fb4bf04d0bca99caba1074d9cc42' BUILD min-glib make[1]: Entering directory '/var/tmp/patchew-tester-tmp-w6rnif1s/src' GEN /var/tmp/patchew-tester-tmp-w6rnif1s/src/docker-src.2018-03-31-02.50.30.10766/qemu.tar Cloning into '/var/tmp/patchew-tester-tmp-w6rnif1s/src/docker-src.2018-03-31-02.50.30.10766/qemu.tar.vroot'... done. Checking out files: 39% (2385/6066) Checking out files: 40% (2427/6066) Checking out files: 41% (2488/6066) Checking out files: 42% (2548/6066) Checking out files: 43% (2609/6066) Checking out files: 44% (2670/6066) Checking out files: 44% (2673/6066) Checking out files: 44% (2708/6066) Checking out files: 45% (2730/6066) Checking out files: 46% (2791/6066) Checking out files: 47% (2852/6066) Checking out files: 48% (2912/6066) Checking out files: 49% (2973/6066) Checking out files: 50% (3033/6066) Checking out files: 50% (3034/6066) Checking out files: 51% (3094/6066) Checking out files: 52% (3155/6066) Checking out files: 53% (3215/6066) Checking out files: 54% (3276/6066) Checking out files: 55% (3337/6066) Checking out files: 56% (3397/6066) Checking out files: 57% (3458/6066) Checking out files: 58% (3519/6066) Checking out files: 59% (3579/6066) Checking out files: 60% (3640/6066) Checking out files: 61% (3701/6066) Checking out files: 62% (3761/6066) Checking out files: 63% (3822/6066) Checking out files: 64% (3883/6066) Checking out files: 65% (3943/6066) Checking out files: 66% (4004/6066) Checking out files: 67% (4065/6066) Checking out files: 68% (4125/6066) Checking out files: 69% (4186/6066) Checking out files: 70% (4247/6066) Checking out files: 71% (4307/6066) Checking out files: 72% (4368/6066) Checking out files: 73% (4429/6066) Checking out files: 74% (4489/6066) Checking out files: 75% (4550/6066) Checking out files: 76% (4611/6066) Checking out files: 77% (4671/6066) Checking out files: 78% (4732/6066) Checking out files: 79% (4793/6066) Checking out files: 80% (4853/6066) Checking out files: 81% (4914/6066) Checking out files: 82% (4975/6066) Checking out files: 83% (5035/6066) Checking out files: 84% (5096/6066) Checking out files: 85% (5157/6066) Checking out files: 86% (5217/6066) Checking out files: 87% (5278/6066) Checking out files: 88% (5339/6066) Checking out files: 89% (5399/6066) Checking out files: 90% (5460/6066) Checking out files: 91% (5521/6066) Checking out files: 92% (5581/6066) Checking out files: 93% (5642/6066) Checking out files: 94% (5703/6066) Checking out files: 95% (5763/6066) Checking out files: 96% (5824/6066) Checking out files: 97% (5885/6066) Checking out files: 98% (5945/6066) Checking out files: 98% (5988/6066) Checking out files: 99% (6006/6066) Checking out files: 100% (6066/6066) Checking out files: 100% (6066/6066), done. Your branch is up-to-date with 'origin/test'. Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc' Cloning into '/var/tmp/patchew-tester-tmp-w6rnif1s/src/docker-src.2018-03-31-02.50.30.10766/qemu.tar.vroot/dtc'... Submodule path 'dtc': checked out 'e54388015af1fb4bf04d0bca99caba1074d9cc42' Submodule 'ui/keycodemapdb' (git://git.qemu.org/keycodemapdb.git) registered for path 'ui/keycodemapdb' Cloning into '/var/tmp/patchew-tester-tmp-w6rnif1s/src/docker-src.2018-03-31-02.50.30.10766/qemu.tar.vroot/ui/keycodemapdb'... Submodule path 'ui/keycodemapdb': checked out '6b3d716e2b6472eb7189d3220552280ef3d832ce' tar: /var/tmp/patchew-tester-tmp-w6rnif1s/src/docker-src.2018-03-31-02.50.30.10766/qemu.tar: Wrote only 2048 of 10240 bytes tar: Error is not recoverable: exiting now failed to create tar file COPYRUNNER RUN test-build in qemu:min-glib tar: Unexpected EOF in archive tar: rmtlseek not stopped at a record boundary tar: Error is not recoverable: exiting now /
Re: [Qemu-devel] [PATCH for-2.12] dump: Fix build with newer gcc
Hi, This series failed docker-build@min-glib build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. Type: series Message-id: 20180327202152.1799131-1-ebl...@redhat.com Subject: [Qemu-devel] [PATCH for-2.12] dump: Fix build with newer gcc === TEST SCRIPT BEGIN === #!/bin/bash set -e git submodule update --init dtc # Let docker tests dump environment info export SHOW_ENV=1 export J=8 time make docker-test-build@min-glib === TEST SCRIPT END === Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384 Switched to a new branch 'test' cc8ef6a6be dump: Fix build with newer gcc === OUTPUT BEGIN === Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc' Cloning into '/var/tmp/patchew-tester-tmp-vs3sdjpj/src/dtc'... Submodule path 'dtc': checked out 'e54388015af1fb4bf04d0bca99caba1074d9cc42' BUILD min-glib make[1]: Entering directory '/var/tmp/patchew-tester-tmp-vs3sdjpj/src' GEN /var/tmp/patchew-tester-tmp-vs3sdjpj/src/docker-src.2018-03-31-02.48.49.8700/qemu.tar Cloning into '/var/tmp/patchew-tester-tmp-vs3sdjpj/src/docker-src.2018-03-31-02.48.49.8700/qemu.tar.vroot'... done. Checking out files: 38% (2340/6066) Checking out files: 39% (2366/6066) Checking out files: 40% (2427/6066) Checking out files: 41% (2488/6066) Checking out files: 42% (2548/6066) Checking out files: 43% (2609/6066) Checking out files: 44% (2670/6066) Checking out files: 45% (2730/6066) Checking out files: 46% (2791/6066) Checking out files: 47% (2852/6066) Checking out files: 48% (2912/6066) Checking out files: 49% (2973/6066) Checking out files: 50% (3033/6066) Checking out files: 51% (3094/6066) Checking out files: 52% (3155/6066) Checking out files: 52% (3178/6066) Checking out files: 53% (3215/6066) Checking out files: 54% (3276/6066) Checking out files: 55% (3337/6066) Checking out files: 56% (3397/6066) Checking out files: 57% (3458/6066) Checking out files: 58% (3519/6066) Checking out files: 58% (3526/6066) Checking out files: 59% (3579/6066) Checking out files: 60% (3640/6066) Checking out files: 61% (3701/6066) Checking out files: 62% (3761/6066) Checking out files: 63% (3822/6066) Checking out files: 64% (3883/6066) Checking out files: 65% (3943/6066) Checking out files: 66% (4004/6066) Checking out files: 67% (4065/6066) Checking out files: 68% (4125/6066) Checking out files: 69% (4186/6066) Checking out files: 70% (4247/6066) Checking out files: 71% (4307/6066) Checking out files: 72% (4368/6066) Checking out files: 73% (4429/6066) Checking out files: 74% (4489/6066) Checking out files: 75% (4550/6066) Checking out files: 76% (4611/6066) Checking out files: 77% (4671/6066) Checking out files: 78% (4732/6066) Checking out files: 79% (4793/6066) Checking out files: 80% (4853/6066) Checking out files: 81% (4914/6066) Checking out files: 82% (4975/6066) Checking out files: 83% (5035/6066) Checking out files: 84% (5096/6066) Checking out files: 85% (5157/6066) Checking out files: 86% (5217/6066) Checking out files: 87% (5278/6066) Checking out files: 88% (5339/6066) Checking out files: 89% (5399/6066) Checking out files: 90% (5460/6066) Checking out files: 91% (5521/6066) Checking out files: 92% (5581/6066) Checking out files: 93% (5642/6066) Checking out files: 94% (5703/6066) Checking out files: 95% (5763/6066) Checking out files: 96% (5824/6066) Checking out files: 97% (5885/6066) Checking out files: 98% (5945/6066) Checking out files: 99% (6006/6066) Checking out files: 100% (6066/6066) Checking out files: 100% (6066/6066), done. Your branch is up-to-date with 'origin/test'. Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc' Cloning into '/var/tmp/patchew-tester-tmp-vs3sdjpj/src/docker-src.2018-03-31-02.48.49.8700/qemu.tar.vroot/dtc'... Submodule path 'dtc': checked out 'e54388015af1fb4bf04d0bca99caba1074d9cc42' Submodule 'ui/keycodemapdb' (git://git.qemu.org/keycodemapdb.git) registered for path 'ui/keycodemapdb' Cloning into '/var/tmp/patchew-tester-tmp-vs3sdjpj/src/docker-src.2018-03-31-02.48.49.8700/qemu.tar.vroot/ui/keycodemapdb'... Submodule path 'ui/keycodemapdb': checked out '6b3d716e2b6472eb7189d3220552280ef3d832ce' tar: /var/tmp/patchew-tester-tmp-vs3sdjpj/src/docker-src.2018-03-31-02.48.49.8700/qemu.tar: Wrote only 2048 of 10240 bytes tar: Error is not recoverable: exiting now failed to create tar file COPYRUNNER RUN test-build in qemu:min-glib tar: Unexpected EOF in archive tar: Unexpected EOF in archive tar: Error is not recoverable: exiting now /var/tmp/qemu/run: line 32: prep_fail: command not found Environment variables: HOSTNAME=1014623e0fbf MAKEFLAGS= -j8 J=8 CCACHE_DIR=/var/tmp/ccache EXTRA_CONFIG
Re: [Qemu-devel] [PATCH for-2.13] pc-bios/s390-ccw: Increase virtio timeout to 30 seconds
Hi, This series failed docker-quick@centos6 build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. Type: series Message-id: 1522316251-16399-1-git-send-email-th...@redhat.com Subject: [Qemu-devel] [PATCH for-2.13] pc-bios/s390-ccw: Increase virtio timeout to 30 seconds === TEST SCRIPT BEGIN === #!/bin/bash set -e git submodule update --init dtc # Let docker tests dump environment info export SHOW_ENV=1 export J=8 time make docker-test-quick@centos6 === TEST SCRIPT END === Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384 Switched to a new branch 'test' eab5813eba pc-bios/s390-ccw: Increase virtio timeout to 30 seconds === OUTPUT BEGIN === Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc' Cloning into '/var/tmp/patchew-tester-tmp-btm39b6h/src/dtc'... Submodule path 'dtc': checked out 'e54388015af1fb4bf04d0bca99caba1074d9cc42' BUILD centos6 make[1]: Entering directory '/var/tmp/patchew-tester-tmp-btm39b6h/src' GEN /var/tmp/patchew-tester-tmp-btm39b6h/src/docker-src.2018-03-31-02.27.43.19655/qemu.tar Cloning into '/var/tmp/patchew-tester-tmp-btm39b6h/src/docker-src.2018-03-31-02.27.43.19655/qemu.tar.vroot'... done. Checking out files: 41% (2535/6066) Checking out files: 42% (2548/6066) Checking out files: 43% (2609/6066) Checking out files: 44% (2670/6066) Checking out files: 45% (2730/6066) Checking out files: 46% (2791/6066) Checking out files: 47% (2852/6066) Checking out files: 48% (2912/6066) Checking out files: 49% (2973/6066) Checking out files: 50% (3033/6066) Checking out files: 51% (3094/6066) Checking out files: 52% (3155/6066) Checking out files: 53% (3215/6066) Checking out files: 54% (3276/6066) Checking out files: 55% (3337/6066) Checking out files: 56% (3397/6066) Checking out files: 57% (3458/6066) Checking out files: 58% (3519/6066) Checking out files: 59% (3579/6066) Checking out files: 60% (3640/6066) Checking out files: 60% (3652/6066) Checking out files: 61% (3701/6066) Checking out files: 62% (3761/6066) Checking out files: 63% (3822/6066) Checking out files: 64% (3883/6066) Checking out files: 65% (3943/6066) Checking out files: 66% (4004/6066) Checking out files: 67% (4065/6066) Checking out files: 68% (4125/6066) Checking out files: 69% (4186/6066) Checking out files: 69% (4194/6066) Checking out files: 70% (4247/6066) Checking out files: 71% (4307/6066) Checking out files: 72% (4368/6066) Checking out files: 73% (4429/6066) Checking out files: 73% (4481/6066) Checking out files: 74% (4489/6066) Checking out files: 75% (4550/6066) Checking out files: 76% (4611/6066) Checking out files: 77% (4671/6066) Checking out files: 78% (4732/6066) Checking out files: 79% (4793/6066) Checking out files: 80% (4853/6066) Checking out files: 81% (4914/6066) Checking out files: 82% (4975/6066) Checking out files: 83% (5035/6066) Checking out files: 84% (5096/6066) Checking out files: 85% (5157/6066) Checking out files: 86% (5217/6066) Checking out files: 87% (5278/6066) Checking out files: 88% (5339/6066) Checking out files: 89% (5399/6066) Checking out files: 90% (5460/6066) Checking out files: 91% (5521/6066) Checking out files: 92% (5581/6066) Checking out files: 93% (5642/6066) Checking out files: 94% (5703/6066) Checking out files: 95% (5763/6066) Checking out files: 96% (5824/6066) Checking out files: 97% (5885/6066) Checking out files: 98% (5945/6066) Checking out files: 99% (6006/6066) Checking out files: 100% (6066/6066) Checking out files: 100% (6066/6066), done. Your branch is up-to-date with 'origin/test'. Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc' Cloning into '/var/tmp/patchew-tester-tmp-btm39b6h/src/docker-src.2018-03-31-02.27.43.19655/qemu.tar.vroot/dtc'... Submodule path 'dtc': checked out 'e54388015af1fb4bf04d0bca99caba1074d9cc42' Submodule 'ui/keycodemapdb' (git://git.qemu.org/keycodemapdb.git) registered for path 'ui/keycodemapdb' Cloning into '/var/tmp/patchew-tester-tmp-btm39b6h/src/docker-src.2018-03-31-02.27.43.19655/qemu.tar.vroot/ui/keycodemapdb'... Submodule path 'ui/keycodemapdb': checked out '6b3d716e2b6472eb7189d3220552280ef3d832ce' tar: /var/tmp/patchew-tester-tmp-btm39b6h/src/docker-src.2018-03-31-02.27.43.19655/qemu.tar: Wrote only 4096 of 10240 bytes tar: Error is not recoverable: exiting now failed to create tar file COPYRUNNER RUN test-quick in qemu:centos6 tar: Unexpected EOF in archive tar: Unexpected EOF in archive tar: Error is not recoverable: exiting now /var/tmp/qemu/run: line 32: prep_fail: command not found Packages installed: SDL-devel-1.2.14-7.el6_7.1.x86_64 bison-2.4.1-5.el6.x86_64 bzip2-devel-1.0.5-7.el6_0.x86_64 ccache-3.1.
Re: [Qemu-devel] [PATCH 0/3] fix memory leaks when using object_property_get_str()
Hi, This series failed docker-quick@centos6 build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. Type: series Message-id: 152231456507.69730.15601462044394150786.st...@bahia.lan Subject: [Qemu-devel] [PATCH 0/3] fix memory leaks when using object_property_get_str() === TEST SCRIPT BEGIN === #!/bin/bash set -e git submodule update --init dtc # Let docker tests dump environment info export SHOW_ENV=1 export J=8 time make docker-test-quick@centos6 === TEST SCRIPT END === Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384 Switched to a new branch 'test' 15abdd963a sev/i386: fix memory leak in sev_guest_init() 20c477c6c5 hw/s390x: fix memory leak in s390_init_ipl_dev() 1937ef1647 exec: fix memory leak in find_max_supported_pagesize() === OUTPUT BEGIN === Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc' Cloning into '/var/tmp/patchew-tester-tmp-5hf354ro/src/dtc'... Submodule path 'dtc': checked out 'e54388015af1fb4bf04d0bca99caba1074d9cc42' BUILD centos6 make[1]: Entering directory '/var/tmp/patchew-tester-tmp-5hf354ro/src' GEN /var/tmp/patchew-tester-tmp-5hf354ro/src/docker-src.2018-03-31-02.38.14.30460/qemu.tar Cloning into '/var/tmp/patchew-tester-tmp-5hf354ro/src/docker-src.2018-03-31-02.38.14.30460/qemu.tar.vroot'... done. Checking out files: 37% (2296/6066) Checking out files: 38% (2306/6066) Checking out files: 39% (2366/6066) Checking out files: 40% (2427/6066) Checking out files: 40% (2434/6066) Checking out files: 41% (2488/6066) Checking out files: 42% (2548/6066) Checking out files: 43% (2609/6066) Checking out files: 44% (2670/6066) Checking out files: 45% (2730/6066) Checking out files: 46% (2791/6066) Checking out files: 47% (2852/6066) Checking out files: 48% (2912/6066) Checking out files: 49% (2973/6066) Checking out files: 50% (3033/6066) Checking out files: 51% (3094/6066) Checking out files: 52% (3155/6066) Checking out files: 53% (3215/6066) Checking out files: 54% (3276/6066) Checking out files: 55% (3337/6066) Checking out files: 56% (3397/6066) Checking out files: 57% (3458/6066) Checking out files: 58% (3519/6066) Checking out files: 59% (3579/6066) Checking out files: 60% (3640/6066) Checking out files: 61% (3701/6066) Checking out files: 62% (3761/6066) Checking out files: 63% (3822/6066) Checking out files: 64% (3883/6066) Checking out files: 65% (3943/6066) Checking out files: 66% (4004/6066) Checking out files: 67% (4065/6066) Checking out files: 68% (4125/6066) Checking out files: 69% (4186/6066) Checking out files: 70% (4247/6066) Checking out files: 71% (4307/6066) Checking out files: 72% (4368/6066) Checking out files: 73% (4429/6066) Checking out files: 74% (4489/6066) Checking out files: 75% (4550/6066) Checking out files: 76% (4611/6066) Checking out files: 77% (4671/6066) Checking out files: 78% (4732/6066) Checking out files: 79% (4793/6066) Checking out files: 80% (4853/6066) Checking out files: 81% (4914/6066) Checking out files: 82% (4975/6066) Checking out files: 83% (5035/6066) Checking out files: 84% (5096/6066) Checking out files: 85% (5157/6066) Checking out files: 86% (5217/6066) Checking out files: 87% (5278/6066) Checking out files: 88% (5339/6066) Checking out files: 89% (5399/6066) Checking out files: 90% (5460/6066) Checking out files: 91% (5521/6066) Checking out files: 92% (5581/6066) Checking out files: 93% (5642/6066) Checking out files: 94% (5703/6066) Checking out files: 95% (5763/6066) Checking out files: 96% (5824/6066) Checking out files: 97% (5885/6066) Checking out files: 98% (5945/6066) Checking out files: 99% (6006/6066) Checking out files: 99% (6053/6066) Checking out files: 100% (6066/6066) Checking out files: 100% (6066/6066), done. Your branch is up-to-date with 'origin/test'. Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc' Cloning into '/var/tmp/patchew-tester-tmp-5hf354ro/src/docker-src.2018-03-31-02.38.14.30460/qemu.tar.vroot/dtc'... Submodule path 'dtc': checked out 'e54388015af1fb4bf04d0bca99caba1074d9cc42' Submodule 'ui/keycodemapdb' (git://git.qemu.org/keycodemapdb.git) registered for path 'ui/keycodemapdb' Cloning into '/var/tmp/patchew-tester-tmp-5hf354ro/src/docker-src.2018-03-31-02.38.14.30460/qemu.tar.vroot/ui/keycodemapdb'... Submodule path 'ui/keycodemapdb': checked out '6b3d716e2b6472eb7189d3220552280ef3d832ce' tar: /var/tmp/patchew-tester-tmp-5hf354ro/src/docker-src.2018-03-31-02.38.14.30460/qemu.tar: Wrote only 4096 of 10240 bytes tar: Error is not recoverable: exiting now failed to create tar file COPYRUNNER RUN test-quick in qemu:centos6 tar: Unexpected EOF in archive tar: Unexpected EOF
Re: [Qemu-devel] [PATCH for-2.13] s390x: introduce 2.13 compat machine
Hi, This series failed docker-quick@centos6 build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. Type: series Message-id: 20180329113320.8053-1-coh...@redhat.com Subject: [Qemu-devel] [PATCH for-2.13] s390x: introduce 2.13 compat machine === TEST SCRIPT BEGIN === #!/bin/bash set -e git submodule update --init dtc # Let docker tests dump environment info export SHOW_ENV=1 export J=8 time make docker-test-quick@centos6 === TEST SCRIPT END === Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384 Switched to a new branch 'test' cf76da366a s390x: introduce 2.13 compat machine === OUTPUT BEGIN === Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc' Cloning into '/var/tmp/patchew-tester-tmp-le7ucg4x/src/dtc'... Submodule path 'dtc': checked out 'e54388015af1fb4bf04d0bca99caba1074d9cc42' BUILD centos6 make[1]: Entering directory '/var/tmp/patchew-tester-tmp-le7ucg4x/src' GEN /var/tmp/patchew-tester-tmp-le7ucg4x/src/docker-src.2018-03-31-02.20.20.11479/qemu.tar Cloning into '/var/tmp/patchew-tester-tmp-le7ucg4x/src/docker-src.2018-03-31-02.20.20.11479/qemu.tar.vroot'... done. Checking out files: 45% (2731/6066) Checking out files: 46% (2791/6066) Checking out files: 47% (2852/6066) Checking out files: 48% (2912/6066) Checking out files: 49% (2973/6066) Checking out files: 50% (3033/6066) Checking out files: 51% (3094/6066) Checking out files: 52% (3155/6066) Checking out files: 53% (3215/6066) Checking out files: 54% (3276/6066) Checking out files: 54% (3296/6066) Checking out files: 55% (3337/6066) Checking out files: 56% (3397/6066) Checking out files: 57% (3458/6066) Checking out files: 58% (3519/6066) Checking out files: 59% (3579/6066) Checking out files: 60% (3640/6066) Checking out files: 61% (3701/6066) Checking out files: 62% (3761/6066) Checking out files: 63% (3822/6066) Checking out files: 64% (3883/6066) Checking out files: 65% (3943/6066) Checking out files: 66% (4004/6066) Checking out files: 67% (4065/6066) Checking out files: 68% (4125/6066) Checking out files: 69% (4186/6066) Checking out files: 70% (4247/6066) Checking out files: 71% (4307/6066) Checking out files: 72% (4368/6066) Checking out files: 72% (4417/6066) Checking out files: 73% (4429/6066) Checking out files: 74% (4489/6066) Checking out files: 75% (4550/6066) Checking out files: 76% (4611/6066) Checking out files: 77% (4671/6066) Checking out files: 78% (4732/6066) Checking out files: 79% (4793/6066) Checking out files: 80% (4853/6066) Checking out files: 81% (4914/6066) Checking out files: 82% (4975/6066) Checking out files: 83% (5035/6066) Checking out files: 84% (5096/6066) Checking out files: 85% (5157/6066) Checking out files: 86% (5217/6066) Checking out files: 87% (5278/6066) Checking out files: 88% (5339/6066) Checking out files: 89% (5399/6066) Checking out files: 90% (5460/6066) Checking out files: 91% (5521/6066) Checking out files: 92% (5581/6066) Checking out files: 93% (5642/6066) Checking out files: 94% (5703/6066) Checking out files: 95% (5763/6066) Checking out files: 96% (5824/6066) Checking out files: 97% (5885/6066) Checking out files: 98% (5945/6066) Checking out files: 99% (6006/6066) Checking out files: 100% (6066/6066) Checking out files: 100% (6066/6066), done. Your branch is up-to-date with 'origin/test'. Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc' Cloning into '/var/tmp/patchew-tester-tmp-le7ucg4x/src/docker-src.2018-03-31-02.20.20.11479/qemu.tar.vroot/dtc'... Submodule path 'dtc': checked out 'e54388015af1fb4bf04d0bca99caba1074d9cc42' Submodule 'ui/keycodemapdb' (git://git.qemu.org/keycodemapdb.git) registered for path 'ui/keycodemapdb' Cloning into '/var/tmp/patchew-tester-tmp-le7ucg4x/src/docker-src.2018-03-31-02.20.20.11479/qemu.tar.vroot/ui/keycodemapdb'... Submodule path 'ui/keycodemapdb': checked out '6b3d716e2b6472eb7189d3220552280ef3d832ce' COPYRUNNER RUN test-quick in qemu:centos6 Packages installed: SDL-devel-1.2.14-7.el6_7.1.x86_64 bison-2.4.1-5.el6.x86_64 bzip2-devel-1.0.5-7.el6_0.x86_64 ccache-3.1.6-2.el6.x86_64 csnappy-devel-0-6.20150729gitd7bc683.el6.x86_64 flex-2.5.35-9.el6.x86_64 gcc-4.4.7-18.el6.x86_64 gettext-0.17-18.el6.x86_64 git-1.7.1-9.el6_9.x86_64 glib2-devel-2.28.8-9.el6.x86_64 libepoxy-devel-1.2-3.el6.x86_64 libfdt-devel-1.4.0-1.el6.x86_64 librdmacm-devel-1.0.21-0.el6.x86_64 lzo-devel-2.03-3.1.el6_5.1.x86_64 make-3.81-23.el6.x86_64 mesa-libEGL-devel-11.0.7-4.el6.x86_64 mesa-libgbm-devel-11.0.7-4.el6.x86_64 package g++ is not installed pixman-devel-0.32.8-1.el6.x86_64 spice-glib-devel-0.26-8.el6.x86_64 spice-server-devel-0.12.4-16.el6.x86_64 tar-1.23-15.el6_8.x86_64 vte-devel-0.25.
Re: [Qemu-devel] [PATCH v3 0/4] RFC: simplify qobject refcount
Hi, This series failed docker-mingw@fedora build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. Type: series Message-id: 20180329154833.566-1-marcandre.lur...@redhat.com Subject: [Qemu-devel] [PATCH v3 0/4] RFC: simplify qobject refcount === TEST SCRIPT BEGIN === #!/bin/bash set -e git submodule update --init dtc # Let docker tests dump environment info export SHOW_ENV=1 export J=8 time make docker-test-mingw@fedora === TEST SCRIPT END === Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384 Switched to a new branch 'test' 71c700b6c1 qobject: modify qobject_ref() to assert on NULL and return obj 4aa8f064ce qobject: replace qobject_incref/QINCREF qobject_decref/QDECREF b9012447f8 qobject: introduce QObjectCommon 069ffe6078 qobject: ensure base is at offset 0 === OUTPUT BEGIN === Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc' Cloning into '/var/tmp/patchew-tester-tmp-nrkq3fro/src/dtc'... Submodule path 'dtc': checked out 'e54388015af1fb4bf04d0bca99caba1074d9cc42' BUILD fedora make[1]: Entering directory '/var/tmp/patchew-tester-tmp-nrkq3fro/src' GEN /var/tmp/patchew-tester-tmp-nrkq3fro/src/docker-src.2018-03-31-02.25.24.17497/qemu.tar Cloning into '/var/tmp/patchew-tester-tmp-nrkq3fro/src/docker-src.2018-03-31-02.25.24.17497/qemu.tar.vroot'... done. Checking out files: 19% (1187/6066) Checking out files: 20% (1214/6066) Checking out files: 21% (1274/6066) Checking out files: 22% (1335/6066) Checking out files: 23% (1396/6066) Checking out files: 24% (1456/6066) Checking out files: 25% (1517/6066) Checking out files: 26% (1578/6066) Checking out files: 26% (1626/6066) Checking out files: 27% (1638/6066) Checking out files: 28% (1699/6066) Checking out files: 29% (1760/6066) Checking out files: 30% (1820/6066) Checking out files: 31% (1881/6066) Checking out files: 32% (1942/6066) Checking out files: 33% (2002/6066) Checking out files: 34% (2063/6066) Checking out files: 35% (2124/6066) Checking out files: 36% (2184/6066) Checking out files: 37% (2245/6066) Checking out files: 38% (2306/6066) Checking out files: 38% (2347/6066) Checking out files: 39% (2366/6066) Checking out files: 40% (2427/6066) Checking out files: 41% (2488/6066) Checking out files: 42% (2548/6066) Checking out files: 43% (2609/6066) Checking out files: 44% (2670/6066) Checking out files: 45% (2730/6066) Checking out files: 46% (2791/6066) Checking out files: 47% (2852/6066) Checking out files: 48% (2912/6066) Checking out files: 49% (2973/6066) Checking out files: 50% (3033/6066) Checking out files: 51% (3094/6066) Checking out files: 52% (3155/6066) Checking out files: 53% (3215/6066) Checking out files: 54% (3276/6066) Checking out files: 55% (3337/6066) Checking out files: 56% (3397/6066) Checking out files: 57% (3458/6066) Checking out files: 58% (3519/6066) Checking out files: 59% (3579/6066) Checking out files: 60% (3640/6066) Checking out files: 61% (3701/6066) Checking out files: 61% (3756/6066) Checking out files: 62% (3761/6066) Checking out files: 63% (3822/6066) Checking out files: 64% (3883/6066) Checking out files: 65% (3943/6066) Checking out files: 66% (4004/6066) Checking out files: 67% (4065/6066) Checking out files: 68% (4125/6066) Checking out files: 69% (4186/6066) Checking out files: 70% (4247/6066) Checking out files: 71% (4307/6066) Checking out files: 72% (4368/6066) Checking out files: 73% (4429/6066) Checking out files: 74% (4489/6066) Checking out files: 75% (4550/6066) Checking out files: 76% (4611/6066) Checking out files: 77% (4671/6066) Checking out files: 78% (4732/6066) Checking out files: 79% (4793/6066) Checking out files: 80% (4853/6066) Checking out files: 81% (4914/6066) Checking out files: 82% (4975/6066) Checking out files: 83% (5035/6066) Checking out files: 84% (5096/6066) Checking out files: 85% (5157/6066) Checking out files: 86% (5217/6066) Checking out files: 87% (5278/6066) Checking out files: 88% (5339/6066) Checking out files: 88% (5395/6066) Checking out files: 89% (5399/6066) Checking out files: 90% (5460/6066) Checking out files: 91% (5521/6066) Checking out files: 92% (5581/6066) Checking out files: 93% (5642/6066) Checking out files: 94% (5703/6066) Checking out files: 95% (5763/6066) Checking out files: 96% (5824/6066) Checking out files: 97% (5885/6066) Checking out files: 98% (5945/6066) Checking out files: 99% (6006/6066) Checking out files: 100% (6066/6066) Checking out files: 100% (6066/6066), done. Your branch is up-to-date with 'origin/test'. Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for p
[Qemu-devel] [Bug 1760262] Re: cmsdk-apb-uart doesn't appear to clear interrupt flags
** Patch added: "0001-hw-char-cmsdk-apb-uart-fix-clearing-of-interrupt-fla.patch" https://bugs.launchpad.net/qemu/+bug/1760262/+attachment/5096702/+files/0001-hw-char-cmsdk-apb-uart-fix-clearing-of-interrupt-fla.patch -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1760262 Title: cmsdk-apb-uart doesn't appear to clear interrupt flags Status in QEMU: New Bug description: I have been writing a small operating system and using QEMU emulating the mps2-an385 board for some of my testing. During development of the uart driver I observed some odd behaviour with the TX interrupt -- writing a '1' to bit 0 of the INTCLEAR register doesn't clear the TX interrupt flag, and the interrupt fires continuously. It's possible that I have an error somewhere in my code, but after inspecting the QEMU source it does appear to be a QEMU bug. I applied the following patch and it solved my issue: From 9875839c144fa60a3772f16ae44d32685f9328aa Mon Sep 17 00:00:00 2001 From: Patrick Oppenlander Date: Sat, 31 Mar 2018 15:10:28 +1100 Subject: [PATCH] hw/char/cmsdk-apb-uart: fix clearing of interrupt flags --- hw/char/cmsdk-apb-uart.c | 1 + 1 file changed, 1 insertion(+) diff --git a/hw/char/cmsdk-apb-uart.c b/hw/char/cmsdk-apb-uart.c index 1ad1e14295..64991bd9d7 100644 --- a/hw/char/cmsdk-apb-uart.c +++ b/hw/char/cmsdk-apb-uart.c @@ -274,6 +274,7 @@ static void uart_write(void *opaque, hwaddr offset, uint64_t value, * is then reflected into the intstatus value by the update function). */ s->state &= ~(value & (R_INTSTATUS_TXO_MASK | R_INTSTATUS_RXO_MASK)); +s->intstatus &= ~(value & ~(R_INTSTATUS_TXO_MASK | R_INTSTATUS_RXO_MASK)); cmsdk_apb_uart_update(s); break; case A_BAUDDIV: -- 2.16.2 To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1760262/+subscriptions
[Qemu-devel] [Bug 1760262] [NEW] cmsdk-apb-uart doesn't appear to clear interrupt flags
Public bug reported: I have been writing a small operating system and using QEMU emulating the mps2-an385 board for some of my testing. During development of the uart driver I observed some odd behaviour with the TX interrupt -- writing a '1' to bit 0 of the INTCLEAR register doesn't clear the TX interrupt flag, and the interrupt fires continuously. It's possible that I have an error somewhere in my code, but after inspecting the QEMU source it does appear to be a QEMU bug. I applied the following patch and it solved my issue: >From 9875839c144fa60a3772f16ae44d32685f9328aa Mon Sep 17 00:00:00 2001 From: Patrick Oppenlander Date: Sat, 31 Mar 2018 15:10:28 +1100 Subject: [PATCH] hw/char/cmsdk-apb-uart: fix clearing of interrupt flags --- hw/char/cmsdk-apb-uart.c | 1 + 1 file changed, 1 insertion(+) diff --git a/hw/char/cmsdk-apb-uart.c b/hw/char/cmsdk-apb-uart.c index 1ad1e14295..64991bd9d7 100644 --- a/hw/char/cmsdk-apb-uart.c +++ b/hw/char/cmsdk-apb-uart.c @@ -274,6 +274,7 @@ static void uart_write(void *opaque, hwaddr offset, uint64_t value, * is then reflected into the intstatus value by the update function). */ s->state &= ~(value & (R_INTSTATUS_TXO_MASK | R_INTSTATUS_RXO_MASK)); +s->intstatus &= ~(value & ~(R_INTSTATUS_TXO_MASK | R_INTSTATUS_RXO_MASK)); cmsdk_apb_uart_update(s); break; case A_BAUDDIV: -- 2.16.2 ** Affects: qemu Importance: Undecided Status: New -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1760262 Title: cmsdk-apb-uart doesn't appear to clear interrupt flags Status in QEMU: New Bug description: I have been writing a small operating system and using QEMU emulating the mps2-an385 board for some of my testing. During development of the uart driver I observed some odd behaviour with the TX interrupt -- writing a '1' to bit 0 of the INTCLEAR register doesn't clear the TX interrupt flag, and the interrupt fires continuously. It's possible that I have an error somewhere in my code, but after inspecting the QEMU source it does appear to be a QEMU bug. I applied the following patch and it solved my issue: From 9875839c144fa60a3772f16ae44d32685f9328aa Mon Sep 17 00:00:00 2001 From: Patrick Oppenlander Date: Sat, 31 Mar 2018 15:10:28 +1100 Subject: [PATCH] hw/char/cmsdk-apb-uart: fix clearing of interrupt flags --- hw/char/cmsdk-apb-uart.c | 1 + 1 file changed, 1 insertion(+) diff --git a/hw/char/cmsdk-apb-uart.c b/hw/char/cmsdk-apb-uart.c index 1ad1e14295..64991bd9d7 100644 --- a/hw/char/cmsdk-apb-uart.c +++ b/hw/char/cmsdk-apb-uart.c @@ -274,6 +274,7 @@ static void uart_write(void *opaque, hwaddr offset, uint64_t value, * is then reflected into the intstatus value by the update function). */ s->state &= ~(value & (R_INTSTATUS_TXO_MASK | R_INTSTATUS_RXO_MASK)); +s->intstatus &= ~(value & ~(R_INTSTATUS_TXO_MASK | R_INTSTATUS_RXO_MASK)); cmsdk_apb_uart_update(s); break; case A_BAUDDIV: -- 2.16.2 To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1760262/+subscriptions
[Qemu-devel] [Bug 1760262] Re: cmsdk-apb-uart doesn't appear to clear interrupt flags
Found in v2.12.0-rc1. -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1760262 Title: cmsdk-apb-uart doesn't appear to clear interrupt flags Status in QEMU: New Bug description: I have been writing a small operating system and using QEMU emulating the mps2-an385 board for some of my testing. During development of the uart driver I observed some odd behaviour with the TX interrupt -- writing a '1' to bit 0 of the INTCLEAR register doesn't clear the TX interrupt flag, and the interrupt fires continuously. It's possible that I have an error somewhere in my code, but after inspecting the QEMU source it does appear to be a QEMU bug. I applied the following patch and it solved my issue: From 9875839c144fa60a3772f16ae44d32685f9328aa Mon Sep 17 00:00:00 2001 From: Patrick Oppenlander Date: Sat, 31 Mar 2018 15:10:28 +1100 Subject: [PATCH] hw/char/cmsdk-apb-uart: fix clearing of interrupt flags --- hw/char/cmsdk-apb-uart.c | 1 + 1 file changed, 1 insertion(+) diff --git a/hw/char/cmsdk-apb-uart.c b/hw/char/cmsdk-apb-uart.c index 1ad1e14295..64991bd9d7 100644 --- a/hw/char/cmsdk-apb-uart.c +++ b/hw/char/cmsdk-apb-uart.c @@ -274,6 +274,7 @@ static void uart_write(void *opaque, hwaddr offset, uint64_t value, * is then reflected into the intstatus value by the update function). */ s->state &= ~(value & (R_INTSTATUS_TXO_MASK | R_INTSTATUS_RXO_MASK)); +s->intstatus &= ~(value & ~(R_INTSTATUS_TXO_MASK | R_INTSTATUS_RXO_MASK)); cmsdk_apb_uart_update(s); break; case A_BAUDDIV: -- 2.16.2 To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1760262/+subscriptions
Re: [Qemu-devel] [RFC 00/18] QEMU validator: A method to specify QEMU crash-test cases
Hi,
This series seems to have some coding style problems. See output below for
more information:
Type: series
Message-id: 20180329213857.15499-1-ehabk...@redhat.com
Subject: [Qemu-devel] [RFC 00/18] QEMU validator: A method to specify QEMU
crash-test cases
=== TEST SCRIPT BEGIN ===
#!/bin/bash
BASE=base
n=1
total=$(git log --oneline $BASE.. | wc -l)
failed=0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
commits="$(git log --format=%H --reverse $BASE..)"
for c in $commits; do
echo "Checking PATCH $n/$total: $(git log -n 1 --format=%s $c)..."
if ! git show $c --format=email | ./scripts/checkpatch.pl --mailback -; then
failed=1
echo
fi
n=$((n+1))
done
exit $failed
=== TEST SCRIPT END ===
Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
t [tag update]patchew/20180330170209.20627-1-rka...@virtuozzo.com
-> patchew/20180330170209.20627-1-rka...@virtuozzo.com
* [new tag] patchew/20180330195928.15607-1-jcmvb...@gmail.com ->
patchew/20180330195928.15607-1-jcmvb...@gmail.com
Switched to a new branch 'test'
0fdb1a2f05 Collection of validator.py test cases
962c7b9a45 validator.py script
11125125d8 qemu.py: is_launched() method
e7b9360fc7 qemu.py: qmp_obj() method
04f1d03e56 qemu.py: Don't try to quit cleanly on exceptions
b504207869 qemu.py: 'force' parameter on shutdown()
9cfedb7337 qemu.py: Only wait for process if it's still running
8c7e843d6e qemu.py: Log crashes inside _post_shutdown()
992c1b6bb2 qemu.py: Set _launched = False on _post_shutdown
9e4bcb4215 qemu.py: Make monitor optional
4d78dabfe2 qemu.py: Close _qmp inside _post_shutdown()
3937ce28e2 qemu.py: Use wait() logic inside shutdown()
c2b5140e3d qemu.py: Move _load_io_log() call to _post_shutdown()
a20254c9e0 qemu.py: Split _base_args()
bb5e609a6b qemu.py: Make _vm_monitor a method
a571feea0c qmp.py: Cleanly handle unexpectedly closed socket
9a92759c0c qmp.py: Fix error handling for Python 3
1f8f56095d qmp.py: Make it safe to call close() any time
=== OUTPUT BEGIN ===
Checking PATCH 1/18: qmp.py: Make it safe to call close() any time...
Checking PATCH 2/18: qmp.py: Fix error handling for Python 3...
Checking PATCH 3/18: qmp.py: Cleanly handle unexpectedly closed socket...
Checking PATCH 4/18: qemu.py: Make _vm_monitor a method...
Checking PATCH 5/18: qemu.py: Split _base_args()...
Checking PATCH 6/18: qemu.py: Move _load_io_log() call to _post_shutdown()...
Checking PATCH 7/18: qemu.py: Use wait() logic inside shutdown()...
Checking PATCH 8/18: qemu.py: Close _qmp inside _post_shutdown()...
Checking PATCH 9/18: qemu.py: Make monitor optional...
Checking PATCH 10/18: qemu.py: Set _launched = False on _post_shutdown...
Checking PATCH 11/18: qemu.py: Log crashes inside _post_shutdown()...
Checking PATCH 12/18: qemu.py: Only wait for process if it's still running...
Checking PATCH 13/18: qemu.py: 'force' parameter on shutdown()...
Checking PATCH 14/18: qemu.py: Don't try to quit cleanly on exceptions...
Checking PATCH 15/18: qemu.py: qmp_obj() method...
Checking PATCH 16/18: qemu.py: is_launched() method...
Checking PATCH 17/18: validator.py script...
WARNING: line over 80 characters
#188: FILE: scripts/validator.py:169:
+self.alldevs = qom_type_names(vm, implements='device',
abstract=False)
WARNING: line over 80 characters
#191: FILE: scripts/validator.py:172:
+self.no_user_devs = [d['name'] for d in info_qdm(vm, ) if
d['no-user']]
WARNING: line over 80 characters
#193: FILE: scripts/validator.py:174:
+self.user_devs = [dev for dev in self.alldevs if dev not in
self.no_user_devs]
WARNING: line over 80 characters
#195: FILE: scripts/validator.py:176:
+self.cpu_models = [c['name'] for c in
vm.command('query-cpu-definitions')]
ERROR: line over 90 characters
#301: FILE: scripts/validator.py:282:
+>>> list(tc) == [{'a':[i], 'b':[j], 'c':[100, 200, 300]} for i in [1,2]
for j in [10, 20]]
ERROR: line over 90 characters
#362: FILE: scripts/validator.py:343:
+self._vars = dict((v, getattr(BuiltinVars, v)()) for v in
dir(BuiltinVars) if not v.startswith('_'))
WARNING: line over 80 characters
#372: FILE: scripts/validator.py:353:
+Default values override the values returned by
VariableDefinition.enumerate()
WARNING: line over 80 characters
#392: FILE: scripts/validator.py:373:
+"""Return full list of variables, including dependencies in the right
order
ERROR: line over 90 characters
#411: FILE: scripts/validator.py:392:
+raise Exception("Variable dependency cycle: %s" % (' ->
'.join(vars.keys(
ERROR: line over 90 characters
#530: FILE: scripts/validator.py:511:
+vars = vars_for_template(self.get('command-line')) +
vars_for_template(self.get('monitor-commands'))
WARNING: line over 80 characters
#548: FILE: scripts/validator.py:529:
+return ' '.join(
[Qemu-devel] [Bug 1760176] [NEW] optical drive doesn't open in Lubuntu 18.04 on '12 MacPro
Public bug reported: Running an install to HD of Lubuntu 18.04 and after running update/upgrade this morning now the optical drive door doesn't respond to keyboard "eject" key on a Mac keyboard for '12 MacPro . . . . I tried to use "xfburn" to get the door to open, nothing seems to get it to work that I know of. Yesterday there was no issue . . . . Tried to run "apt -f install" and it showed some old kernels to "autoremove" . . . which I did; didn't try to reboot into old kernel to test, perhaps now old kernel is gone? F ** Affects: qemu Importance: Undecided Status: New -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1760176 Title: optical drive doesn't open in Lubuntu 18.04 on '12 MacPro Status in QEMU: New Bug description: Running an install to HD of Lubuntu 18.04 and after running update/upgrade this morning now the optical drive door doesn't respond to keyboard "eject" key on a Mac keyboard for '12 MacPro . . . . I tried to use "xfburn" to get the door to open, nothing seems to get it to work that I know of. Yesterday there was no issue . . . . Tried to run "apt -f install" and it showed some old kernels to "autoremove" . . . which I did; didn't try to reboot into old kernel to test, perhaps now old kernel is gone? F To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1760176/+subscriptions
[Qemu-devel] [PATCH] target/xtensa: linux-user: rewind pc for restarted syscall
In case of syscall restart request set pc back to the syscall instruction. Signed-off-by: Max Filippov --- linux-user/main.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/linux-user/main.c b/linux-user/main.c index ba09b7d0c873..8907a8411411 100644 --- a/linux-user/main.c +++ b/linux-user/main.c @@ -4006,6 +4006,9 @@ void cpu_loop(CPUXtensaState *env) break; case -TARGET_ERESTARTSYS: +env->pc -= 3; +break; + case -TARGET_QEMU_ESIGRETURN: break; } -- 2.11.0
Re: [Qemu-devel] [PATCH for-2.12 v3 2/2] i386/hyperv: error out if features requested but unsupported
On Fri, Mar 30, 2018 at 08:02:09PM +0300, Roman Kagan wrote: > In order to guarantee compatibility on migration, QEMU should have > complete control over the features it announces to the guest via CPUID. > > However, for a number of Hyper-V-related cpu properties, if the > corresponding feature is not supported by the underlying KVM, the > propery is silently ignored and the feature is not announced to the > guest. > > Refuse to start with an error instead. > > Signed-off-by: Roman Kagan Reviewed-by: Eduardo Habkost -- Eduardo
[Qemu-devel] [Bug 1728256] Re: Memory corruption in Windows 10 guest / amd64
Got similar behavior with windows server 2012r2 VMs. Environment: uname -a Linux ubuntu-q87 4.13.0-37-generic #42~16.04.1-Ubuntu SMP Wed Mar 7 16:03:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux ii linux-image-4.13.0-36-generic 4.13.0-36.40~16.04.1 amd64 Linux kernel image for version 4.13.0 on 64 bit x86 SMP apt policy qemu qemu: Installed: (none) Candidate: 1:2.11+dfsg-1ubuntu5~cloud0 Version table: 1:2.11+dfsg-1ubuntu5~cloud0 500 Windows VMs error out and create a memory dump: The computer has rebooted from a bugcheck. The bugcheck was: 0x0109 (0xa3a01f5891f186c5, 0xb3b72bdee47188a0, 0x0320, 0x0017). A dump was saved in: C:\Windows\MEMORY.DMP. Report Id: 033018-31234-01. Based on microsoft docs: https://docs.microsoft.com/en-us/windows-hardware/drivers/debugger/bug-check-0x109---critical-structure-corruption CRITICAL_STRUCTURE_CORRUPTION Parameters Parameter Description 1 Reserved 2 Reserved 3 Reserved 4 The type of the corrupted region. (See the following table later on this page.) ... 0x17 Local APIC modification <--- this Which is the same as with other reports out there: https://www.spinics.net/lists/kvm/msg159977.html https://forum.proxmox.com/threads/new-windows-vm-keeps-dying.39145/#post-193639 >From what I see the change was backported but there was no new build yet. http://kernel.ubuntu.com/git/ubuntu/ubuntu-xenial.git/commit/arch/x86/kvm/x86.c?h=hwe&id=78d2542b88d16 See https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1738972 I suggest this is marked as a duplicate to 1738972 -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1728256 Title: Memory corruption in Windows 10 guest / amd64 Status in QEMU: New Status in linux package in Ubuntu: Confirmed Bug description: I have a Win 10 Pro x64 guest inside a qemu/kvm running on an Arch x86_64 host. The VM has a physical GPU passed through, as well as the physical USB controllers, as well as a dedicated SSD attached via SATA; you can find the complete libvirt xml here: https://pastebin.com/U1ZAXBNg I built qemu from source using the qemu-minimal-git AUR package; you can find the build script here: https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=qemu-minimal-git (if you aren't familiar with Arch, this is essentially a bash script where build() and package() are run to build the files, and then install them into the $pkgdir to later tar them up.) Starting with qemu v2.10.0, Windows crashes randomly with a bluescreen about CRITICAL_STRUCTURE_CORRUPTION. I also tested the git heads f90ea7ba7c, 861cd431c9 and e822e81e35, before I went back to v2.9.0, which is running stable for over 50 hours right now. During my tests I found that locking the memory pages alleviates the problem somewhat, but never completely avoids it. However, with the crashes occuring randomly, that could as well be false conclusions; I had crashes within minutes after boot with that too. I will now start `git bisect`ing; if you have any other suggestions on what I could try or possible patches feel free to leave them with me. To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1728256/+subscriptions
[Qemu-devel] [Bug 1728256] Re: Memory corruption in Windows 10 guest / amd64
Status changed to 'Confirmed' because the bug affects multiple users. ** Changed in: linux (Ubuntu) Status: New => Confirmed -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1728256 Title: Memory corruption in Windows 10 guest / amd64 Status in QEMU: New Status in linux package in Ubuntu: Confirmed Bug description: I have a Win 10 Pro x64 guest inside a qemu/kvm running on an Arch x86_64 host. The VM has a physical GPU passed through, as well as the physical USB controllers, as well as a dedicated SSD attached via SATA; you can find the complete libvirt xml here: https://pastebin.com/U1ZAXBNg I built qemu from source using the qemu-minimal-git AUR package; you can find the build script here: https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=qemu-minimal-git (if you aren't familiar with Arch, this is essentially a bash script where build() and package() are run to build the files, and then install them into the $pkgdir to later tar them up.) Starting with qemu v2.10.0, Windows crashes randomly with a bluescreen about CRITICAL_STRUCTURE_CORRUPTION. I also tested the git heads f90ea7ba7c, 861cd431c9 and e822e81e35, before I went back to v2.9.0, which is running stable for over 50 hours right now. During my tests I found that locking the memory pages alleviates the problem somewhat, but never completely avoids it. However, with the crashes occuring randomly, that could as well be false conclusions; I had crashes within minutes after boot with that too. I will now start `git bisect`ing; if you have any other suggestions on what I could try or possible patches feel free to leave them with me. To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1728256/+subscriptions
[Qemu-devel] [Bug 1728256] Re: Memory corruption in Windows 10 guest / amd64
** Also affects: linux (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1728256 Title: Memory corruption in Windows 10 guest / amd64 Status in QEMU: New Status in linux package in Ubuntu: Confirmed Bug description: I have a Win 10 Pro x64 guest inside a qemu/kvm running on an Arch x86_64 host. The VM has a physical GPU passed through, as well as the physical USB controllers, as well as a dedicated SSD attached via SATA; you can find the complete libvirt xml here: https://pastebin.com/U1ZAXBNg I built qemu from source using the qemu-minimal-git AUR package; you can find the build script here: https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=qemu-minimal-git (if you aren't familiar with Arch, this is essentially a bash script where build() and package() are run to build the files, and then install them into the $pkgdir to later tar them up.) Starting with qemu v2.10.0, Windows crashes randomly with a bluescreen about CRITICAL_STRUCTURE_CORRUPTION. I also tested the git heads f90ea7ba7c, 861cd431c9 and e822e81e35, before I went back to v2.9.0, which is running stable for over 50 hours right now. During my tests I found that locking the memory pages alleviates the problem somewhat, but never completely avoids it. However, with the crashes occuring randomly, that could as well be false conclusions; I had crashes within minutes after boot with that too. I will now start `git bisect`ing; if you have any other suggestions on what I could try or possible patches feel free to leave them with me. To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1728256/+subscriptions
[Qemu-devel] Qemu IRIX 6.5 MIPS R4000 ?
Is there way to emulate SGI IRIX 6.5 MIPS R4000 in QEMU? I need to transfer my work from my software Softimage 3d 4.0 to IRIX for a modern machine and qemu is great for this, I saw many on YouTube and Google working perfectly Debian (no graphic) DEC Alpha and up to Windows NT 4.0 MIPS 4000 in Qemu, how can it help? Is that possible? Do I need to send the copy of the software? help ? Download any free irix iso here: https://winworldpc.com/product/irix/65
Re: [Qemu-devel] [PULL 0/1] RISC-V: Critical fixes for QEMU 2.12
Hi Peter, I had tested Richard's proper fix but we didn't have a PR or the required Reviewed-by and Signed-off-by so I made the PR for the conservative fix, assuming we can test Richard's more correct fix and include it in the QEMU 2.13 timeframe. I've tested Richard's fix with the simple scheduling test case, so if he makes a PR, I'm happy for that to be included versus this workaround. The workaround is, of course, the most conservative fix as it will always cause FP state to be saved, assuming we missed a case in Richard's patch. I'll leave it up to you and Richard, but please include either this workaround or Richard's slightly larger change in QEMU 2.12, if possible. In any case, I believe vendors have patches they can include in their packages... but it would be nice that upstream QEMU 2.12 has working FP for RISC-V with SMP Linux. Thanks, Michael. On Thu, Mar 29, 2018 at 10:26 AM, Michael Clark wrote: > The following changes since commit 47d3b60858d90ac8a0cc3a72af7f95 > c96781125a: > > Merge remote-tracking branch > 'remotes/riscv/tags/riscv-qemu-2.12-important-fixes' > into staging (2018-03-28 22:13:38 +0100) > > are available in the git repository at: > > https://github.com/riscv/riscv-qemu.git tags/riscv-qemu-2.12-critical- > fixes > > for you to fetch changes up to b02403363f1056421d120c8e974fdf9c76a84f95: > > RISC-V: Workaround for critical mstatus.FS bug (2018-03-29 10:22:26 > -0700) > > > RISC-V: Critical fixes for QEMU 2.12 > > This series includes changes that are considered release critical, > such as floating point register file corruption under SMP Linux > due to incorrect handling of mstatus.FS. > > This workaround will be replaced with a more comprehensive fix > for mstatus.FS handling in QEMU 2.13. > > > Michael Clark (1): > RISC-V: Workaround for critical mstatus.FS bug > > target/riscv/op_helper.c | 17 +++-- > 1 file changed, 15 insertions(+), 2 deletions(-) >
[Qemu-devel] [PATCH for-2.12 v3 0/2] i386/hyperv: fully control Hyper-V features in CPUID
In order to guarantee compatibility on migration, QEMU should have complete control over the features it announces to the guest via CPUID. However, a number of Hyper-V-related features happen to depend on the support in the underlying KVM, with no regard to QEMU configuration. Make QEMU regain control over what Hyper-V features it announces to the guest. Note #1: the patches are also being proposed[*] for stable-2.11, even though one of them introduces a new cpu property. This is done to minimize the number of published QEMU releases where the behavior of the features is unpredictable, with potentially fatal consequences for the guest. Note #2: there are other problems in the surrounding code, like ugly error reporting or inconsistent population of MSRs. I think this can be put off to post-2.12. [*] for the stable branch the second patch will have error returns replaced with warnings; I'll post a separate series. v2 -> v3: - include the fix for 'hv-time' missed previously v1 -> v2: - indicate what flag requested the feature that can't be enabled in the error message - fix a typo in the error message for VP_RUNTIME Roman Kagan (2): i386/hyperv: add hv-frequencies cpu property i386/hyperv: error out if features requested but unsupported target/i386/cpu.h | 1 + target/i386/cpu.c | 1 + target/i386/kvm.c | 56 ++- 3 files changed, 45 insertions(+), 13 deletions(-) -- 2.14.3
[Qemu-devel] [PATCH for-2.12 v3 2/2] i386/hyperv: error out if features requested but unsupported
In order to guarantee compatibility on migration, QEMU should have
complete control over the features it announces to the guest via CPUID.
However, for a number of Hyper-V-related cpu properties, if the
corresponding feature is not supported by the underlying KVM, the
propery is silently ignored and the feature is not announced to the
guest.
Refuse to start with an error instead.
Signed-off-by: Roman Kagan
---
v2 -> v3:
- include the fix for 'hv-time' missed previously
v1 -> v2:
- indicate what flag requested the feature that can't be enabled in the
error message
- fix a typo in the error message for VP_RUNTIME
target/i386/kvm.c | 43 ++-
1 file changed, 34 insertions(+), 9 deletions(-)
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index b35623ae24..6c49954e68 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -632,11 +632,6 @@ static int hyperv_handle_properties(CPUState *cs)
X86CPU *cpu = X86_CPU(cs);
CPUX86State *env = &cpu->env;
-if (cpu->hyperv_time &&
-kvm_check_extension(cs->kvm_state, KVM_CAP_HYPERV_TIME) <= 0) {
-cpu->hyperv_time = false;
-}
-
if (cpu->hyperv_relaxed_timing) {
env->features[FEAT_HYPERV_EAX] |= HV_HYPERCALL_AVAILABLE;
}
@@ -645,6 +640,12 @@ static int hyperv_handle_properties(CPUState *cs)
env->features[FEAT_HYPERV_EAX] |= HV_APIC_ACCESS_AVAILABLE;
}
if (cpu->hyperv_time) {
+if (kvm_check_extension(cs->kvm_state, KVM_CAP_HYPERV_TIME) <= 0) {
+fprintf(stderr, "Hyper-V clocksources "
+"(requested by 'hv-time' cpu flag) "
+"are not supported by kernel\n");
+return -ENOSYS;
+}
env->features[FEAT_HYPERV_EAX] |= HV_HYPERCALL_AVAILABLE;
env->features[FEAT_HYPERV_EAX] |= HV_TIME_REF_COUNT_AVAILABLE;
env->features[FEAT_HYPERV_EAX] |= HV_REFERENCE_TSC_AVAILABLE;
@@ -659,17 +660,41 @@ static int hyperv_handle_properties(CPUState *cs)
env->features[FEAT_HYPERV_EAX] |= HV_ACCESS_FREQUENCY_MSRS;
env->features[FEAT_HYPERV_EDX] |= HV_FREQUENCY_MSRS_AVAILABLE;
}
-if (cpu->hyperv_crash && has_msr_hv_crash) {
+if (cpu->hyperv_crash) {
+if (!has_msr_hv_crash) {
+fprintf(stderr, "Hyper-V crash MSRs "
+"(requested by 'hv-crash' cpu flag) "
+"are not supported by kernel\n");
+return -ENOSYS;
+}
env->features[FEAT_HYPERV_EDX] |= HV_GUEST_CRASH_MSR_AVAILABLE;
}
env->features[FEAT_HYPERV_EDX] |= HV_CPU_DYNAMIC_PARTITIONING_AVAILABLE;
-if (cpu->hyperv_reset && has_msr_hv_reset) {
+if (cpu->hyperv_reset) {
+if (!has_msr_hv_reset) {
+fprintf(stderr, "Hyper-V reset MSR "
+"(requested by 'hv-reset' cpu flag) "
+"is not supported by kernel\n");
+return -ENOSYS;
+}
env->features[FEAT_HYPERV_EAX] |= HV_RESET_AVAILABLE;
}
-if (cpu->hyperv_vpindex && has_msr_hv_vpindex) {
+if (cpu->hyperv_vpindex) {
+if (!has_msr_hv_vpindex) {
+fprintf(stderr, "Hyper-V VP_INDEX MSR "
+"(requested by 'hv-vpindex' cpu flag) "
+"is not supported by kernel\n");
+return -ENOSYS;
+}
env->features[FEAT_HYPERV_EAX] |= HV_VP_INDEX_AVAILABLE;
}
-if (cpu->hyperv_runtime && has_msr_hv_runtime) {
+if (cpu->hyperv_runtime) {
+if (!has_msr_hv_runtime) {
+fprintf(stderr, "Hyper-V VP_RUNTIME MSR "
+"(requested by 'hv-runtime' cpu flag) "
+"is not supported by kernel\n");
+return -ENOSYS;
+}
env->features[FEAT_HYPERV_EAX] |= HV_VP_RUNTIME_AVAILABLE;
}
if (cpu->hyperv_synic) {
--
2.14.3
[Qemu-devel] [PATCH for-2.12 v3 1/2] i386/hyperv: add hv-frequencies cpu property
In order to guarantee compatibility on migration, QEMU should have
complete control over the features it announces to the guest via CPUID.
However, the availability of Hyper-V frequency MSRs
(HV_X64_MSR_TSC_FREQUENCY and HV_X64_MSR_APIC_FREQUENCY) depends solely
on the support for them in the underlying KVM.
Introduce "hv-frequencies" cpu property (off by default) which gives
QEMU full control over whether these MSRs are announced.
While at this, drop the redundant check of the cpu tsc frequency, and
decouple this feature from hv-time.
Signed-off-by: Roman Kagan
Reviewed-by: Eduardo Habkost
---
v2 -> v3:
- no changes
v1 -> v2:
- indicate what flag requested the feature that can't be enabled in the
error message
target/i386/cpu.h | 1 +
target/i386/cpu.c | 1 +
target/i386/kvm.c | 13 +
3 files changed, 11 insertions(+), 4 deletions(-)
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 78db1b833a..1b219fafc4 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -1296,6 +1296,7 @@ struct X86CPU {
bool hyperv_runtime;
bool hyperv_synic;
bool hyperv_stimer;
+bool hyperv_frequencies;
bool check_cpuid;
bool enforce_cpuid;
bool expose_kvm;
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 555ae79d29..1a6b082b6f 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -4761,6 +4761,7 @@ static Property x86_cpu_properties[] = {
DEFINE_PROP_BOOL("hv-runtime", X86CPU, hyperv_runtime, false),
DEFINE_PROP_BOOL("hv-synic", X86CPU, hyperv_synic, false),
DEFINE_PROP_BOOL("hv-stimer", X86CPU, hyperv_stimer, false),
+DEFINE_PROP_BOOL("hv-frequencies", X86CPU, hyperv_frequencies, false),
DEFINE_PROP_BOOL("check", X86CPU, check_cpuid, true),
DEFINE_PROP_BOOL("enforce", X86CPU, enforce_cpuid, false),
DEFINE_PROP_BOOL("kvm", X86CPU, expose_kvm, true),
diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index d23fff12f5..b35623ae24 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -648,11 +648,16 @@ static int hyperv_handle_properties(CPUState *cs)
env->features[FEAT_HYPERV_EAX] |= HV_HYPERCALL_AVAILABLE;
env->features[FEAT_HYPERV_EAX] |= HV_TIME_REF_COUNT_AVAILABLE;
env->features[FEAT_HYPERV_EAX] |= HV_REFERENCE_TSC_AVAILABLE;
-
-if (has_msr_hv_frequencies && tsc_is_stable_and_known(env)) {
-env->features[FEAT_HYPERV_EAX] |= HV_ACCESS_FREQUENCY_MSRS;
-env->features[FEAT_HYPERV_EDX] |= HV_FREQUENCY_MSRS_AVAILABLE;
+}
+if (cpu->hyperv_frequencies) {
+if (!has_msr_hv_frequencies) {
+fprintf(stderr, "Hyper-V frequency MSRs "
+"(requested by 'hv-frequencies' cpu flag) "
+"are not supported by kernel\n");
+return -ENOSYS;
}
+env->features[FEAT_HYPERV_EAX] |= HV_ACCESS_FREQUENCY_MSRS;
+env->features[FEAT_HYPERV_EDX] |= HV_FREQUENCY_MSRS_AVAILABLE;
}
if (cpu->hyperv_crash && has_msr_hv_crash) {
env->features[FEAT_HYPERV_EDX] |= HV_GUEST_CRASH_MSR_AVAILABLE;
--
2.14.3
Re: [Qemu-devel] [PULL 0/2] M68k for 2.12 patches
Le 30/03/2018 à 18:54, Rob Landley a écrit : > On 03/20/2018 04:08 AM, Laurent Vivier wrote:> This series of patches is > needed > to fix a problem >> in the m68k translator that can crash QEMU when translation >> cache has too many instructions: >> >> qemu-m68k: tcg/tcg.c:883: tcg_temp_alloc: Assertion `n < 512' failed. >> qemu: uncaught target signal 11 (Segmentation fault) - core dumped >> >> I have reproduced it in linux user mode, with "ghc", and in >> system mode with the debian-installer for unstable distro >> from debian-ports. > > If someone wanted to follow along with your "boot linux on qemu-system-m68k" > work on https://github.com/vivier/qemu-m68k, which of the 51 branches should > qemu-system-m68k with like -M q800 or whatever you had working be built from? The branch to use is q800-dev > > Also, "git pull" of the last tree I had from there exploded into a fireball of > automerge conflicts. Is there something I should know? As I rebase the branch regularly, a simple "git pull" will not work. If the branch already exist locally git checkout q800-dev git reset --hard vivier/q800-dev [I guess the remote name is "vivier"] otherwise, a simple "git checkout q800-dev" should create the branch from scrash. Thanks, Laurent
Re: [Qemu-devel] [PATCH v1 1/1] RISC-V: Workaround for critical mstatus.FS MTTCG bug
On Fri, Mar 30, 2018 at 12:11 AM, Alex Bennée
wrote:
>
> Michael Clark writes:
>
> > On Tue, Mar 27, 2018 at 3:17 PM, Philippe Mathieu-Daudé >
> > wrote:
> >
> >> Cc'ing Alex and Richard.
> >>
> >> On 03/27/2018 04:54 PM, Michael Clark wrote:
> >> > This change is a workaround for a bug where mstatus.FS
> >> > is not correctly reporting dirty when MTTCG and SMP are
> >> > enabled which results in the floating point register file
> >> > not being saved during context switches. This a critical
> >> > bug for RISC-V in QEMU as it results in floating point
> >> > register file corruption when running SMP Linux in the
> >> > RISC-V 'virt' machine.
> >> >
> >> > This workaround will return dirty if mstatus.FS is
> >> > switched from off to initial or clean. We have checked
> >> > the specification and it is legal for an implementation
> >> > to return either off, or dirty, if set to initial or clean.
> >> >
> >> > This workaround will result in unnecessary floating point
> >> > save restore. When mstatus.FS is off, floating point
> >> > instruction trap to indicate the process is using the FPU.
> >> > The OS can then save floating-point state of the previous
> >> > process using the FPU and set mstatus.FS to initial or
> >> > clean. With this workaround, mstatus.FS will always return
> >> > dirty if set to a non-zero value, indicating floating point
> >> > save restore is necessary, versus misreporting mstatus.FS
> >> > resulting in floating point register file corruption.
> >> >
> >> > Cc: Palmer Dabbelt
> >> > Cc: Sagar Karandikar
> >> > Cc: Bastian Koppelmann
> >> > Cc: Peter Maydell
> >> > Tested-by: Richard W.M. Jones
> >> > Signed-off-by: Michael Clark
> >> > ---
> >> > target/riscv/op_helper.c | 19 +--
> >> > 1 file changed, 17 insertions(+), 2 deletions(-)
> >> >
> >> > diff --git a/target/riscv/op_helper.c b/target/riscv/op_helper.c
> >> > index e34715d..7281b98 100644
> >> > --- a/target/riscv/op_helper.c
> >> > +++ b/target/riscv/op_helper.c
> >> > @@ -144,8 +144,23 @@ void csr_write_helper(CPURISCVState *env,
> >> target_ulong val_to_write,
> >> > }
> >> >
> >> > mstatus = (mstatus & ~mask) | (val_to_write & mask);
> >> > -int dirty = (mstatus & MSTATUS_FS) == MSTATUS_FS;
> >> > -dirty |= (mstatus & MSTATUS_XS) == MSTATUS_XS;
> >> > +
> >> > +/* Note: this is a workaround for an issue where mstatus.FS
> >> > + does not report dirty when SMP and MTTCG is enabled. This
> >> > + workaround is technically compliant with the RISC-V
> >> Privileged
> >> > + specification as it is legal to return only off, or dirty,
> >> > + however this may cause unnecessary saves of floating point
> >> state.
> >> > + Without this workaround, floating point state is not saved
> >> and
> >> > + restored correctly when SMP and MTTCG is enabled, */
> >>
> >
> > On looking at this again, I think we may need to remove the
> > qemu_tcg_mttcg_enabled conditional and always return dirty if the state
> is
> > initial or clean, but not off.
> >
> > While testing on uniprocessor worked okay, it's likely because we were
> > lucky and there was no task migration or multiple FPU tasks working. This
> > would mean we would revert to Richard W.M. Jones initial patch.
> >
> >> +if (qemu_tcg_mttcg_enabled()) {
> >> > +/* FP is always dirty or off */
> >> > +if (mstatus & MSTATUS_FS) {
> >> > +mstatus |= MSTATUS_FS;
> >> > +}
> >> > +}
>
>
> I'm confused. If mstatus is a per-vCPU variable why does the enabling or
> not of MTTCG matter here?
It was an incorrect inference from the available data at the time given the
problem showed up on SMP but not uniprocessor.
After inspecting the logic we concluded we were just lucky in the
uniprocessor case because our test case only have once process using the
CPU, whereas with SMP the process was rescheduled to another core which
made the corruption visible on SMP.
We removed the MTTCG condition from the subsequent version of the patch,
updated the patch description and comment and removed references to MTTCG.
> >> > +
> >> > +int dirty = ((mstatus & MSTATUS_FS) == MSTATUS_FS) |
> >> > +((mstatus & MSTATUS_XS) == MSTATUS_XS);
> >> > mstatus = set_field(mstatus, MSTATUS_SD, dirty);
> >> > env->mstatus = mstatus;
> >> > break;
> >> >
> >>
> >
> > The text from the specification that allows us to always return dirty if
> > set to initial or clean, is below i.e. Dirty implies state has
> > "potentially" been modified, so that gives us wriggle room.
> >
> > "
> > When an extension's status is set to Off , any instruction that attempts
> to
> > read or write the corresponding
> > state will cause an exception. When the status is Initial, the
> > corresponding state should
> > have an initial constant value. When the status is Clean, the
> corresponding
>
Re: [Qemu-devel] [PATCH for-2.12] nbd/client: Correctly handle bad server REP_META_CONTEXT
30.03.2018 02:18, Eric Blake wrote:
It's never a good idea to blindly read for size bytes as
returned by the server without first validating that the size
is within bounds; a malicious or buggy server could cause us
to hang or get out of sync from reading further messages.
It may be smarter to try and teach the client to cope with
unexpected context ids by silently ignoring them instead of
hanging up on the server, but for now, if the server doesn't
reply with exactly the one context we expect, it's easier to
just give up - however, if we give up for any reason other
than an I/O failure, we might as well try to politely tell
the server we are quitting rather than continuing.
Fix some typos in the process.
Signed-off-by: Eric Blake
---
nbd/client.c | 19 +--
1 file changed, 17 insertions(+), 2 deletions(-)
diff --git a/nbd/client.c b/nbd/client.c
index 9b9b7f0ea29..4ee1d9a4a2c 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -599,8 +599,8 @@ static QIOChannel *nbd_receive_starttls(QIOChannel *ioc,
* Set one meta context. Simple means that reply must contain zero (not
* negotiated) or one (negotiated) contexts. More contexts would be considered
* as a protocol error. It's also implied that meta-data query equals queried
- * context name, so, if server replies with something different then @context,
- * it considered as error too.
+ * context name, so, if server replies with something different than @context,
+ * it is considered an error too.
* return 1 for successful negotiation, context_id is set
*0 if operation is unsupported,
*-1 with errp set for any other error
@@ -651,6 +651,14 @@ static int nbd_negotiate_simple_meta_context(QIOChannel
*ioc,
char *name;
size_t len;
+if (reply.length != sizeof(received_id) + context_len) {
+error_setg(errp, "Failed to negotiate meta context '%s', server "
+ "answered with unexpected length %u", context,
uint32_t, is it worth PRIu32 ? Or %u is absolutely portable in this case?
+ reply.length);
+nbd_send_opt_abort(ioc);
+return -1;
+}
hmm, after this check, len variable is not actually needed, we can use
context_len
+
if (nbd_read(ioc, &received_id, sizeof(received_id), errp) < 0) {
return -1;
}
@@ -668,6 +676,7 @@ static int nbd_negotiate_simple_meta_context(QIOChannel
*ioc,
"answered with different context '%s'", context,
name);
g_free(name);
+nbd_send_opt_abort(ioc);
return -1;
}
g_free(name);
@@ -690,6 +699,12 @@ static int nbd_negotiate_simple_meta_context(QIOChannel
*ioc,
if (reply.type != NBD_REP_ACK) {
error_setg(errp, "Unexpected reply type %" PRIx32 " expected %x",
reply.type, NBD_REP_ACK);
+nbd_send_opt_abort(ioc);
+return -1;
+}
+if (reply.length) {
this check is very common for REP_ACK, it may be better to move it to
nbd_handle_reply_err... (and rename this function? and combine it
somehow with _option_request() and _option_reply()?)
+error_setg(errp, "Unexpected length to ACK response");
+nbd_send_opt_abort(ioc);
hmm, looks like we want nbd_send_opt_abort() before most of return -1.
Looks like it lacks some generalization, may be want to send it at some
common point..
return -1;
}
mostly, just ideas for future refactoring, so:
Reviewed-by: Vladimir Sementsov-Ogievskiy
--
Best regards,
Vladimir
Re: [Qemu-devel] [PULL 0/2] M68k for 2.12 patches
On 03/20/2018 04:08 AM, Laurent Vivier wrote:> This series of patches is needed to fix a problem > in the m68k translator that can crash QEMU when translation > cache has too many instructions: > > qemu-m68k: tcg/tcg.c:883: tcg_temp_alloc: Assertion `n < 512' failed. > qemu: uncaught target signal 11 (Segmentation fault) - core dumped > > I have reproduced it in linux user mode, with "ghc", and in > system mode with the debian-installer for unstable distro > from debian-ports. If someone wanted to follow along with your "boot linux on qemu-system-m68k" work on https://github.com/vivier/qemu-m68k, which of the 51 branches should qemu-system-m68k with like -M q800 or whatever you had working be built from? Also, "git pull" of the last tree I had from there exploded into a fireball of automerge conflicts. Is there something I should know? Rob
Re: [Qemu-devel] [PATCH for-2.12] nbd: trace meta context negotiation
30.03.2018 16:09, Eric Blake wrote:
Having a more detailed log of the interaction between client and
server is invaluable in debugging how meta context negotiation
actually works.
Signed-off-by: Eric Blake
---
nbd/client.c | 2 ++
nbd/server.c | 8
nbd/trace-events | 6 ++
3 files changed, 16 insertions(+)
diff --git a/nbd/client.c b/nbd/client.c
index 4ee1d9a4a2c..478b6085df7 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -623,6 +623,7 @@ static int nbd_negotiate_simple_meta_context(QIOChannel
*ioc,
char *data = g_malloc(data_len);
char *p = data;
+trace_nbd_opt_meta_request(context, export);
stl_be_p(p, export_len);
memcpy(p += sizeof(export_len), export, export_len);
stl_be_p(p += export_len, 1);
@@ -681,6 +682,7 @@ static int nbd_negotiate_simple_meta_context(QIOChannel
*ioc,
}
g_free(name);
+trace_nbd_opt_meta_reply(context, received_id);
received = true;
/* receive NBD_REP_ACK */
diff --git a/nbd/server.c b/nbd/server.c
index cea158913ba..9e1f2271784 100644
--- a/nbd/server.c
+++ b/nbd/server.c
@@ -726,6 +726,7 @@ static int nbd_negotiate_send_meta_context(NBDClient
*client,
context_id = 0;
}
+trace_nbd_negotiate_meta_query_reply(context, context_id);
set_be_option_rep(&opt.h, client->opt, NBD_REP_META_CONTEXT,
sizeof(opt) - sizeof(opt.h) + iov[1].iov_len);
stl_be_p(&opt.context_id, context_id);
@@ -752,10 +753,12 @@ static int nbd_meta_base_query(NBDClient *client,
NBDExportMetaContexts *meta,
if (client->opt == NBD_OPT_LIST_META_CONTEXT) {
meta->base_allocation = true;
}
+trace_nbd_negotiate_meta_query_parse("base:");
return 1;
}
if (len != alen) {
+trace_nbd_negotiate_meta_query_skip("not base:allocation");
return nbd_opt_skip(client, len, errp);
}
@@ -768,6 +771,7 @@ static int nbd_meta_base_query(NBDClient *client,
NBDExportMetaContexts *meta,
meta->base_allocation = true;
}
+trace_nbd_negotiate_meta_query_parse("base:allocation");
return 1;
}
@@ -800,6 +804,7 @@ static int nbd_negotiate_meta_query(NBDClient *client,
/* The only supported namespace for now is 'base'. So query should start
* with 'base:'. Otherwise, we can ignore it and skip the remainder. */
if (len < baselen) {
+trace_nbd_negotiate_meta_query_skip("length too short");
return nbd_opt_skip(client, len, errp);
}
@@ -809,6 +814,7 @@ static int nbd_negotiate_meta_query(NBDClient *client,
return ret;
}
if (strncmp(query, "base:", baselen) != 0) {
+trace_nbd_negotiate_meta_query_skip("not for base: namespace");
Hmm, reasonable example of using same trace-point in several places in
the code. Is it a precedent?
return nbd_opt_skip(client, len, errp);
}
@@ -858,6 +864,8 @@ static int nbd_negotiate_meta_queries(NBDClient *client,
return ret;
}
cpu_to_be32s(&nb_queries);
+trace_nbd_negotiate_meta_context(nbd_opt_lookup(client->opt),
+ meta->export_name, nb_queries);
if (client->opt == NBD_OPT_LIST_META_CONTEXT && !nb_queries) {
/* enable all known contexts */
diff --git a/nbd/trace-events b/nbd/trace-events
index 0d03edc967d..cfdbe04b447 100644
--- a/nbd/trace-events
+++ b/nbd/trace-events
@@ -10,6 +10,8 @@ nbd_receive_query_exports_start(const char *wantname)
"Querying export list for
nbd_receive_query_exports_success(const char *wantname) "Found desired export name
'%s'"
nbd_receive_starttls_new_client(void) "Setting up TLS"
nbd_receive_starttls_tls_handshake(void) "Starting TLS handshake"
+nbd_opt_meta_request(const char *context, const char *export) "Requesting to set
meta context %s for export %s"
+nbd_opt_meta_reply(const char *context, int id) "Received mapping of context %s to
id %d"
actual type is uint32_t, I'd prefer to reflect it here.
nbd_receive_negotiate(void *tlscreds, const char *hostname) "Receiving negotiation
tlscreds=%p hostname=%s"
nbd_receive_negotiate_magic(uint64_t magic) "Magic is 0x%" PRIx64
nbd_receive_negotiate_server_flags(uint32_t globalflags) "Global flags are
0x%" PRIx32
@@ -44,6 +46,10 @@ nbd_negotiate_handle_info_request(int request, const char *name)
"Client request
nbd_negotiate_handle_info_block_size(uint32_t minimum, uint32_t preferred, uint32_t maximum)
"advertising minimum 0x%" PRIx32 ", preferred 0x%" PRIx32 ", maximum 0x%" PRIx32
nbd_negotiate_handle_starttls(void) "Setting up TLS"
nbd_negotiate_handle_starttls_handshake(void) "Starting TLS handshake"
+nbd_negotiate_meta_context(const char *optname, const char *export, int queries)
"Client requested %s for export %s, with %d queries"
and here
+nbd_negotiate_meta_query_skip(const char *reason) "Skipping meta query: %s"
+nbd_negotiate_meta_query_pars
Re: [Qemu-devel] [PATCH] iotests: fix 169
30.03.2018 19:10, Vladimir Sementsov-Ogievskiy wrote:
Use MIGRATION events instead of RESUME. Also, make a TODO: enable
dirty-bitmaps capability for offline case.
This (likely) fixes racy faults at least of the following types:
- timeout on waiting for RESUME event
- sha256 mismatch on 136 (138 after this patch)
136 line I mean
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
This patch is a true change for the test anyway. But I don't understand,
why (and do really) it fixes the things. And I'm not sure about do we
really have a bug in bitmap migration or persistence. So, it's up to you,
take it into 2.12...
It was already discussed, that "STOP" event is bad for tests. What about
"RESUME"? How can we miss it? And sha256 mismatch is really something
strange.
Max, please check, do it fix 169 for you.
tests/qemu-iotests/169 | 44 +++-
1 file changed, 23 insertions(+), 21 deletions(-)
diff --git a/tests/qemu-iotests/169 b/tests/qemu-iotests/169
index 153b10b6e7..5e525ab9d5 100755
--- a/tests/qemu-iotests/169
+++ b/tests/qemu-iotests/169
@@ -31,6 +31,8 @@ disk_a = os.path.join(iotests.test_dir, 'disk_a')
disk_b = os.path.join(iotests.test_dir, 'disk_b')
size = '1M'
mig_file = os.path.join(iotests.test_dir, 'mig_file')
+mig_cmd = 'exec: cat > ' + mig_file
+incoming_cmd = 'exec: cat ' + mig_file
class TestDirtyBitmapMigration(iotests.QMPTestCase):
@@ -49,7 +51,6 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase):
self.vm_a.launch()
self.vm_b = iotests.VM(path_suffix='b')
-self.vm_b.add_incoming("exec: cat '" + mig_file + "'")
def add_bitmap(self, vm, granularity, persistent):
params = {'node': 'drive0',
@@ -86,36 +87,30 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase):
(0xa0201, 0x1000))
should_migrate = migrate_bitmaps or persistent and shared_storage
+mig_caps = [{'capability': 'events', 'state': True}]
+if migrate_bitmaps:
+mig_caps.append({'capability': 'dirty-bitmaps', 'state': True})
+result = self.vm_a.qmp('migrate-set-capabilities',
+ capabilities=mig_caps)
+self.assert_qmp(result, 'return', {})
+
+self.vm_b.add_incoming(incoming_cmd if online else "defer")
self.vm_b.add_drive(disk_a if shared_storage else disk_b)
if online:
os.mkfifo(mig_file)
self.vm_b.launch()
+result = self.vm_b.qmp('migrate-set-capabilities',
+ capabilities=mig_caps)
+self.assert_qmp(result, 'return', {})
self.add_bitmap(self.vm_a, granularity, persistent)
for r in regions:
self.vm_a.hmp_qemu_io('drive0', 'write %d %d' % r)
sha256 = self.get_bitmap_hash(self.vm_a)
-if migrate_bitmaps:
-capabilities = [{'capability': 'dirty-bitmaps', 'state': True}]
-
-result = self.vm_a.qmp('migrate-set-capabilities',
- capabilities=capabilities)
-self.assert_qmp(result, 'return', {})
-
-if online:
-result = self.vm_b.qmp('migrate-set-capabilities',
- capabilities=capabilities)
-self.assert_qmp(result, 'return', {})
-
-result = self.vm_a.qmp('migrate-set-capabilities',
- capabilities=[{'capability': 'events',
- 'state': True}])
-self.assert_qmp(result, 'return', {})
-
-result = self.vm_a.qmp('migrate', uri='exec:cat>' + mig_file)
+result = self.vm_a.qmp('migrate', uri=mig_cmd)
while True:
event = self.vm_a.event_wait('MIGRATION')
if event['data']['status'] == 'completed':
@@ -124,9 +119,16 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase):
if not online:
self.vm_a.shutdown()
self.vm_b.launch()
-# TODO enable bitmap capability for vm_b in this case
+result = self.vm_b.qmp('migrate-set-capabilities',
+ capabilities=mig_caps)
+self.assert_qmp(result, 'return', {})
+result = self.vm_b.qmp('migrate-incoming', uri=incoming_cmd)
+self.assert_qmp(result, 'return', {})
-self.vm_b.event_wait("RESUME", timeout=10.0)
+while True:
+event = self.vm_b.event_wait('MIGRATION')
+if event['data']['status'] == 'completed':
+break
self.check_bitmap(self.vm_b, sha256 if should_migrate else False)
--
Best regards,
Vladimir
[Qemu-devel] [PATCH] iotests: fix 169
Use MIGRATION events instead of RESUME. Also, make a TODO: enable
dirty-bitmaps capability for offline case.
This (likely) fixes racy faults at least of the following types:
- timeout on waiting for RESUME event
- sha256 mismatch on 136 (138 after this patch)
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
This patch is a true change for the test anyway. But I don't understand,
why (and do really) it fixes the things. And I'm not sure about do we
really have a bug in bitmap migration or persistence. So, it's up to you,
take it into 2.12...
It was already discussed, that "STOP" event is bad for tests. What about
"RESUME"? How can we miss it? And sha256 mismatch is really something
strange.
Max, please check, do it fix 169 for you.
tests/qemu-iotests/169 | 44 +++-
1 file changed, 23 insertions(+), 21 deletions(-)
diff --git a/tests/qemu-iotests/169 b/tests/qemu-iotests/169
index 153b10b6e7..5e525ab9d5 100755
--- a/tests/qemu-iotests/169
+++ b/tests/qemu-iotests/169
@@ -31,6 +31,8 @@ disk_a = os.path.join(iotests.test_dir, 'disk_a')
disk_b = os.path.join(iotests.test_dir, 'disk_b')
size = '1M'
mig_file = os.path.join(iotests.test_dir, 'mig_file')
+mig_cmd = 'exec: cat > ' + mig_file
+incoming_cmd = 'exec: cat ' + mig_file
class TestDirtyBitmapMigration(iotests.QMPTestCase):
@@ -49,7 +51,6 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase):
self.vm_a.launch()
self.vm_b = iotests.VM(path_suffix='b')
-self.vm_b.add_incoming("exec: cat '" + mig_file + "'")
def add_bitmap(self, vm, granularity, persistent):
params = {'node': 'drive0',
@@ -86,36 +87,30 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase):
(0xa0201, 0x1000))
should_migrate = migrate_bitmaps or persistent and shared_storage
+mig_caps = [{'capability': 'events', 'state': True}]
+if migrate_bitmaps:
+mig_caps.append({'capability': 'dirty-bitmaps', 'state': True})
+result = self.vm_a.qmp('migrate-set-capabilities',
+ capabilities=mig_caps)
+self.assert_qmp(result, 'return', {})
+
+self.vm_b.add_incoming(incoming_cmd if online else "defer")
self.vm_b.add_drive(disk_a if shared_storage else disk_b)
if online:
os.mkfifo(mig_file)
self.vm_b.launch()
+result = self.vm_b.qmp('migrate-set-capabilities',
+ capabilities=mig_caps)
+self.assert_qmp(result, 'return', {})
self.add_bitmap(self.vm_a, granularity, persistent)
for r in regions:
self.vm_a.hmp_qemu_io('drive0', 'write %d %d' % r)
sha256 = self.get_bitmap_hash(self.vm_a)
-if migrate_bitmaps:
-capabilities = [{'capability': 'dirty-bitmaps', 'state': True}]
-
-result = self.vm_a.qmp('migrate-set-capabilities',
- capabilities=capabilities)
-self.assert_qmp(result, 'return', {})
-
-if online:
-result = self.vm_b.qmp('migrate-set-capabilities',
- capabilities=capabilities)
-self.assert_qmp(result, 'return', {})
-
-result = self.vm_a.qmp('migrate-set-capabilities',
- capabilities=[{'capability': 'events',
- 'state': True}])
-self.assert_qmp(result, 'return', {})
-
-result = self.vm_a.qmp('migrate', uri='exec:cat>' + mig_file)
+result = self.vm_a.qmp('migrate', uri=mig_cmd)
while True:
event = self.vm_a.event_wait('MIGRATION')
if event['data']['status'] == 'completed':
@@ -124,9 +119,16 @@ class TestDirtyBitmapMigration(iotests.QMPTestCase):
if not online:
self.vm_a.shutdown()
self.vm_b.launch()
-# TODO enable bitmap capability for vm_b in this case
+result = self.vm_b.qmp('migrate-set-capabilities',
+ capabilities=mig_caps)
+self.assert_qmp(result, 'return', {})
+result = self.vm_b.qmp('migrate-incoming', uri=incoming_cmd)
+self.assert_qmp(result, 'return', {})
-self.vm_b.event_wait("RESUME", timeout=10.0)
+while True:
+event = self.vm_b.event_wait('MIGRATION')
+if event['data']['status'] == 'completed':
+break
self.check_bitmap(self.vm_b, sha256 if should_migrate else False)
--
2.11.1
[Qemu-devel] [Bug 1756519] Re: qemu linux-user crash in QOM path canonicalization during do_fork() call to cpu_create
The following patch should fix that: http://lists.nongnu.org/archive/html/qemu-devel/2018-03/msg07437.html -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1756519 Title: qemu linux-user crash in QOM path canonicalization during do_fork() call to cpu_create Status in QEMU: New Bug description: qemu-riscv64 version 2.11.50 (v2.11.0-2491-g2bb39a657a) crashes running gcc libgomp.c/sort-1.c testsuite test case with the following message: (process:11683): GLib-CRITICAL **: g_hash_table_iter_next: assertion 'ri->version == ri->hash_table->version' failed ** ERROR:qom/object.c:1665:object_get_canonical_path_component: code should not be reached qemu:handle_cpu_signal received signal outside vCPU context @ pc=0x60139c16 Backtrace obtained via gdb: #0 raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51 #1 0x60139b21 in abort () at abort.c:79 #2 0x60100505 in g_assertion_message (domain=domain@entry=0x0, file=file@entry=0x60213ca1 "qom/object.c", line=line@entry=1665, func=func@entry=0x60214420 <__func__.18106> "object_get_canonical_path_component", message=message@entry=0x7fffe8000cd0 "code should not be reached") at gtestutils.c:2430 #3 0x60100586 in g_assertion_message_expr (domain=0x0, file=0x60213ca1 "qom/object.c", line=1665, func=0x60214420 <__func__.18106> "object_get_canonical_path_component", expr=) at gtestutils.c:2453 #4 0x60098334 in object_get_canonical_path_component (obj=0x7fffe81340b0) at qom/object.c:1665 #5 0x60098366 in object_get_canonical_path (obj=0x7fffe81340b0) at qom/object.c:1675 #6 0x6008e152 in device_set_realized (obj=0x7fffe81340b0, value=true, errp=0x7762fe68) at hw/core/qdev.c:874 #7 0x60098bf4 in property_set_bool (obj=0x7fffe81340b0, v=0x7fffe80fd3c0, name=0x60213694 "realized", opaque=0x7fffe80fd140, errp=0x7762fe68) at qom/object.c:1926 #8 0x60096fee in object_property_set (obj=0x7fffe81340b0, v=0x7fffe80fd3c0, name=0x60213694 "realized", errp=0x7762fe68) at qom/object.c:1122 #9 0x60099ebd in object_property_set_qobject (obj=0x7fffe81340b0, value=0x7fffe80fd310, name=0x60213694 "realized", errp=0x7762fe68) at qom/qom-qobject.c:27 #10 0x60097274 in object_property_set_bool (obj=0x7fffe81340b0, value=true, name=0x60213694 "realized", errp=0x7762fe68) at qom/object.c:1191 #11 0x60092ec5 in cpu_create (typename=0x6250e1a0 "any-riscv-cpu") at qom/cpu.c:61 #12 0x6009301a in cpu_generic_init (typename=0x601dd58f "riscv-cpu", cpu_model=0x601dd527 "any") at qom/cpu.c:98 #13 0x6004cb61 in cpu_copy (env=0x7008cd60) at /opt/qemu/linux-user/main.c:3881 #14 0x6005b79a in do_fork (env=0x7008cd60, flags=4001536, newsp=275531880704, parent_tidptr=275531882704, newtls=275531884288, child_tidptr=275531882704) at /opt/qemu/linux-user/syscall.c:6348 #15 0x60063e56 in do_syscall (cpu_env=0x7008cd60, num=220, arg1=4001536, arg2=275531880704, arg3=275531882704, arg4=275531884288, arg5=275531882704, arg6=275531884288, arg7=0, arg8=0) at /opt/qemu/linux-user/syscall.c:10001 #16 0x6004c89f in cpu_loop (env=0x7008cd60) at /opt/qemu/linux-user/main.c:3600 #17 0x6005b68f in clone_func (arg=0x77775050) at /opt/qemu/linux-user/syscall.c:6311 #18 0x60121797 in start_thread (arg=0x77632700) at pthread_create.c:463 #19 0x6019b4fb in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 Attached is a test case source code extracted from libgomp test suite. Note that it is a multi-threaded and requires 5 or more threads to fail. Number of launched threads is controlled by OMP_NUM_THREADS evironment variable, defaulting to number of hardware threads. Changing constants in the test case makes it fail with different numbers of threads. I will attach statically linked riscv64 binary executable if size limits permit. To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1756519/+subscriptions
Re: [Qemu-devel] [PATCH v4 for 2.12 0/3] fix bitmaps migration through shared storage
30.03.2018 16:31, Vladimir Sementsov-Ogievskiy wrote:
29.03.2018 18:09, Vladimir Sementsov-Ogievskiy wrote:
29.03.2018 17:03, Max Reitz wrote:
On 2018-03-29 10:08, Vladimir Sementsov-Ogievskiy wrote:
28.03.2018 17:53, Max Reitz wrote:
On 2018-03-27 12:11, Vladimir Sementsov-Ogievskiy wrote:
[...]
isn't it because a lot of cat processes? will check, update loop to
i=0; while check -qcow2 169; do ((i++)); echo $i OK; killall -9 cat;
done
Hmm... I know I tried to kill all of the cats, but for some
reason that
didn't really help yesterday. Seems to help now, for 2.12.0-rc0 at
least (that is, before this series).
reproduced with killing... (without these series, just on master)
After the whole series, I still get a lot of failures in 169
(mismatching bitmap hash, mostly).
And interestingly, if I add an abort():
diff --git a/block/qcow2.c b/block/qcow2.c
index 486f3e83b7..9204c1c0ac 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -1481,6 +1481,7 @@ static int coroutine_fn
qcow2_do_open(BlockDriverState *bs, QDict *options, }
if (bdrv_dirty_bitmap_next(bs, NULL)) {
+ abort();
/* It's some kind of reopen with already existing dirty
bitmaps. There
* are no known cases where we need loading bitmaps in
such
situation,
* so it's safer don't load them.
Then this fires for a couple of test cases of 169 even without the
third
patch of this series.
I guess bdrv_dirty_bitmap_next() reacts to some bitmaps that
migration
adds or something? Then this would be the wrong condition, because I
guess we still want to load the bitmaps that are in the qcow2 file.
I'm not sure whether bdrv_has_readonly_bitmaps() is the correct
condition then, either, though. Maybe let's take a step back: We
want
to load all the bitmaps from the file exactly once, and that is
when it
is opened the first time. Or that's what I would have thought... Is
that even correct?
Why do we load the bitmaps when the device is inactive anyway?
Shouldn't we load them only once the device is activated?
Hmm, not sure. May be, we don't need. But we anyway need to load them,
when opening read-only, and we should correspondingly reopen in
this case.
Yeah, well, yeah, but the current state seems just wrong. Apparently
there are cases where a qcow2 node may have bitmaps before we try to
load them from the file, so the current condition doesn't work.
Furthermore, it seems like the current "state machine" is too
complex so
we don't know which cases are possible anymore and what to do when.
So the first thing we could do is add a field to the BDRVQCow2State
that
tells us whether the bitmaps have been loaded already or not. If not,
we invoke qcow2_load_dirty_bitmaps() and set the value to true. If the
value was true already and the BDS is active and R/W now, we call
qcow2_reopen_bitmaps_rw_hint(). That should solve one problem.
good idea, will do.
The other problem of course is the question whether we should call
qcow2_load_dirty_bitmaps() at all while the drive is still inactive.
You know the migration model better than me, so I'm asking this
question
to you. We can phrase it differently: Do we need to load the bitmaps
before the drive is activated?
Now I think that we don't need. At least, we don't have such cases in
Virtuozzo (I hope :).
Why did I doubt:
1. We have cases, when we want to start vm as inactive, to be able to
export it's drive as NBD export, push some data to it and then start
the VM (which involves activating)
2. We have cases, when we want to start vm stopped and operate on
dirty bitmaps.
If just open all images in inactive mode until vm start, it looks
like we need bitmaps in inactive mode (for 2.). But it looks like
wrong approach anyway.
Firstly, I tried to solve (1.) by simply inactivate_all() in case of
start vm in paused mode, but it breaks at least (2.), so finally, I
solved (1.) by an approach similar with "-incoming defer". So, we
have inactive mode in two cases:
- incoming migration
- push data to vm before start
and, in these cases, we don't need to load dirty-bitmaps.
Also, inconsistency: now, we remove persistent bitmaps on inactivate.
So, it is inconsistent to load the in inactive mode.
Ok, I'll try to respin.
finally, what cases we actually have for qcow2_do_open?
1. INACTIVE -> ACTIVE (through invalidate_cache, we obviously should
load bitmaps, if we decided that we have no persistent bitmaps in
INACTIVE mode)
2. creating new bdrv state (first open of the image) in INACTIVE mode
(will not load bitmaps)
3. creating new bdrv state (first open of the image) in ACTIVE mode
(will load bitmaps, maybe read-only if disk is RO)
If only these three cases, it would be enough to just load bitmaps if
!INACTIVE and do nothing otherwise.
Or, we have some of the following cases too?
1?. ACTIVE -> ACTIVE (through invalidate_cache, some kind of no-op, we
should not reload bitmaps)
2?. RO -> RW (we should reopen_bit
[Qemu-devel] [PATCH 2/3] iotests.py: support unsupported_fmts in main()
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
tests/qemu-iotests/iotests.py | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 83c454d..89fe446 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -553,7 +553,8 @@ def verify_quorum():
if not supports_quorum():
notrun('quorum support missing')
-def main(supported_fmts=[], supported_oses=['linux'],
supported_cache_modes=[]):
+def main(supported_fmts=[], supported_oses=['linux'], supported_cache_modes=[],
+ unsupported_fmts=[]):
'''Run tests'''
global debug
@@ -568,7 +569,7 @@ def main(supported_fmts=[], supported_oses=['linux'],
supported_cache_modes=[]):
debug = '-d' in sys.argv
verbosity = 1
-verify_image_format(supported_fmts)
+verify_image_format(supported_fmts, unsupported_fmts)
verify_platform(supported_oses)
verify_cache_mode(supported_cache_modes)
--
2.7.4
[Qemu-devel] [PATCH 1/3] iotests.py: improve verify_image_format helper
Add an assert (we don't want set both arguments) and remove
duplication.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
tests/qemu-iotests/iotests.py | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index b5d7945..83c454d 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -532,9 +532,9 @@ def notrun(reason):
sys.exit(0)
def verify_image_format(supported_fmts=[], unsupported_fmts=[]):
-if supported_fmts and (imgfmt not in supported_fmts):
-notrun('not suitable for this image format: %s' % imgfmt)
-if unsupported_fmts and (imgfmt in unsupported_fmts):
+assert not (supported_fmts and unsupported_fmts)
+not_sup = supported_fmts and (imgfmt not in supported_fmts)
+if not_sup or (imgfmt in unsupported_fmts):
notrun('not suitable for this image format: %s' % imgfmt)
def verify_platform(supported_oses=['linux']):
--
2.7.4
Re: [Qemu-devel] [PATCH for 2.12 0/3] iotests: blacklist bochs and cloop for 205 and 208
for 2.12 30.03.2018 18:16, Vladimir Sementsov-Ogievskiy wrote: Blacklist these formats, as they don't support image creation, as they say: > ./qemu-img create -f bochs x 1m qemu-img: x: Format driver 'bochs' does not support image creation > ./qemu-img create -f cloop x 1m qemu-img: x: Format driver 'cloop' does not support image creation Vladimir Sementsov-Ogievskiy (3): iotests.py: improve verify_image_format helper iotests.py: support unsupported_fmts in main() iotests: blacklist bochs and cloop for 205 and 208 tests/qemu-iotests/205| 2 +- tests/qemu-iotests/208| 2 ++ tests/qemu-iotests/iotests.py | 11 ++- 3 files changed, 9 insertions(+), 6 deletions(-) -- Best regards, Vladimir
[Qemu-devel] [PATCH 0/3] iotests: blacklist bochs and cloop for 205 and 208
Blacklist these formats, as they don't support image creation, as they say: > ./qemu-img create -f bochs x 1m qemu-img: x: Format driver 'bochs' does not support image creation > ./qemu-img create -f cloop x 1m qemu-img: x: Format driver 'cloop' does not support image creation Vladimir Sementsov-Ogievskiy (3): iotests.py: improve verify_image_format helper iotests.py: support unsupported_fmts in main() iotests: blacklist bochs and cloop for 205 and 208 tests/qemu-iotests/205| 2 +- tests/qemu-iotests/208| 2 ++ tests/qemu-iotests/iotests.py | 11 ++- 3 files changed, 9 insertions(+), 6 deletions(-) -- 2.7.4
[Qemu-devel] [PATCH 3/3] iotests: blacklist bochs and cloop for 205 and 208
Blacklist these formats, as they don't support image creation, as they
say:
> ./qemu-img create -f bochs x 1m
qemu-img: x: Format driver 'bochs' does not support image creation
> ./qemu-img create -f cloop x 1m
qemu-img: x: Format driver 'cloop' does not support image creation
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
tests/qemu-iotests/205 | 2 +-
tests/qemu-iotests/208 | 2 ++
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/tests/qemu-iotests/205 b/tests/qemu-iotests/205
index e7b2eae..c41b929 100755
--- a/tests/qemu-iotests/205
+++ b/tests/qemu-iotests/205
@@ -153,4 +153,4 @@ class TestNbdServerRemove(iotests.QMPTestCase):
if __name__ == '__main__':
-iotests.main()
+iotests.main(unsupported_fmts=['bochs', 'cloop'])
diff --git a/tests/qemu-iotests/208 b/tests/qemu-iotests/208
index 18f59ad..3bbfc9d 100755
--- a/tests/qemu-iotests/208
+++ b/tests/qemu-iotests/208
@@ -22,6 +22,8 @@
import iotests
+iotests.verify_image_format(unsupported_fmts=['bochs', 'cloop'])
+
with iotests.FilePath('disk.img') as disk_img_path, \
iotests.FilePath('disk-snapshot.img') as disk_snapshot_img_path, \
iotests.FilePath('nbd.sock') as nbd_sock_path, \
--
2.7.4
[Qemu-devel] [PATCH for 2.12] iotests: fix 208 for luks format
Support luks images creatins like in 205
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
tests/qemu-iotests/208 | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tests/qemu-iotests/208 b/tests/qemu-iotests/208
index 4e82b96..18f59ad 100755
--- a/tests/qemu-iotests/208
+++ b/tests/qemu-iotests/208
@@ -28,7 +28,7 @@ with iotests.FilePath('disk.img') as disk_img_path, \
iotests.VM() as vm:
img_size = '10M'
-iotests.qemu_img_pipe('create', '-f', iotests.imgfmt, disk_img_path,
img_size)
+iotests.qemu_img_create('-f', iotests.imgfmt, disk_img_path, img_size)
iotests.log('Launching VM...')
(vm.add_drive(disk_img_path, 'node-name=drive0-node', interface='none')
--
2.7.4
Re: [Qemu-devel] [PATCH] linux-user: call cpu_copy under clone_lock
Le 30/03/2018 à 15:35, Max Filippov a écrit : > cpu_copy adds newly created CPU object to container/machine/unattached, > but does it w/o proper locking. As a result when multiple threads are > created rapidly QEMU may abort with the following message: > > GLib-CRITICAL **: g_hash_table_iter_next: assertion > 'ri->version == ri->hash_table->version' failed > > ERROR:qemu/qom/object.c:1663:object_get_canonical_path_component: > code should not be reached Also reported in https://bugs.launchpad.net/qemu/+bug/1756519 > Move cpu_copy invocation under clone_lock to fix that. > > Signed-off-by: Max Filippov > --- > linux-user/syscall.c | 7 --- > 1 file changed, 4 insertions(+), 3 deletions(-) > > diff --git a/linux-user/syscall.c b/linux-user/syscall.c > index 889abbda1e65..18ea79140f16 100644 > --- a/linux-user/syscall.c > +++ b/linux-user/syscall.c > @@ -6346,6 +6346,10 @@ static int do_fork(CPUArchState *env, unsigned int > flags, abi_ulong newsp, > > ts = g_new0(TaskState, 1); > init_task_state(ts); > + > +/* Grab a mutex so that thread setup appears atomic. */ > +pthread_mutex_lock(&clone_lock); > + > /* we create a new CPU instance. */ > new_env = cpu_copy(env); > /* Init regs that differ from the parent. */ > @@ -6364,9 +6368,6 @@ static int do_fork(CPUArchState *env, unsigned int > flags, abi_ulong newsp, > cpu_set_tls (new_env, newtls); > } > > -/* Grab a mutex so that thread setup appears atomic. */ > -pthread_mutex_lock(&clone_lock); > - > memset(&info, 0, sizeof(info)); > pthread_mutex_init(&info.mutex, NULL); > pthread_mutex_lock(&info.mutex); > Reviewed-by: Laurent Vivier
[Qemu-devel] [PATCH] linux-user: call cpu_copy under clone_lock
cpu_copy adds newly created CPU object to container/machine/unattached, but does it w/o proper locking. As a result when multiple threads are created rapidly QEMU may abort with the following message: GLib-CRITICAL **: g_hash_table_iter_next: assertion 'ri->version == ri->hash_table->version' failed ERROR:qemu/qom/object.c:1663:object_get_canonical_path_component: code should not be reached Move cpu_copy invocation under clone_lock to fix that. Signed-off-by: Max Filippov --- linux-user/syscall.c | 7 --- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/linux-user/syscall.c b/linux-user/syscall.c index 889abbda1e65..18ea79140f16 100644 --- a/linux-user/syscall.c +++ b/linux-user/syscall.c @@ -6346,6 +6346,10 @@ static int do_fork(CPUArchState *env, unsigned int flags, abi_ulong newsp, ts = g_new0(TaskState, 1); init_task_state(ts); + +/* Grab a mutex so that thread setup appears atomic. */ +pthread_mutex_lock(&clone_lock); + /* we create a new CPU instance. */ new_env = cpu_copy(env); /* Init regs that differ from the parent. */ @@ -6364,9 +6368,6 @@ static int do_fork(CPUArchState *env, unsigned int flags, abi_ulong newsp, cpu_set_tls (new_env, newtls); } -/* Grab a mutex so that thread setup appears atomic. */ -pthread_mutex_lock(&clone_lock); - memset(&info, 0, sizeof(info)); pthread_mutex_init(&info.mutex, NULL); pthread_mutex_lock(&info.mutex); -- 2.11.0
Re: [Qemu-devel] [PATCH v4 for 2.12 0/3] fix bitmaps migration through shared storage
29.03.2018 18:09, Vladimir Sementsov-Ogievskiy wrote:
29.03.2018 17:03, Max Reitz wrote:
On 2018-03-29 10:08, Vladimir Sementsov-Ogievskiy wrote:
28.03.2018 17:53, Max Reitz wrote:
On 2018-03-27 12:11, Vladimir Sementsov-Ogievskiy wrote:
[...]
isn't it because a lot of cat processes? will check, update loop to
i=0; while check -qcow2 169; do ((i++)); echo $i OK; killall -9 cat;
done
Hmm... I know I tried to kill all of the cats, but for some reason
that
didn't really help yesterday. Seems to help now, for 2.12.0-rc0 at
least (that is, before this series).
reproduced with killing... (without these series, just on master)
After the whole series, I still get a lot of failures in 169
(mismatching bitmap hash, mostly).
And interestingly, if I add an abort():
diff --git a/block/qcow2.c b/block/qcow2.c
index 486f3e83b7..9204c1c0ac 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -1481,6 +1481,7 @@ static int coroutine_fn
qcow2_do_open(BlockDriverState *bs, QDict *options, }
if (bdrv_dirty_bitmap_next(bs, NULL)) {
+ abort();
/* It's some kind of reopen with already existing dirty
bitmaps. There
* are no known cases where we need loading bitmaps in such
situation,
* so it's safer don't load them.
Then this fires for a couple of test cases of 169 even without the
third
patch of this series.
I guess bdrv_dirty_bitmap_next() reacts to some bitmaps that migration
adds or something? Then this would be the wrong condition, because I
guess we still want to load the bitmaps that are in the qcow2 file.
I'm not sure whether bdrv_has_readonly_bitmaps() is the correct
condition then, either, though. Maybe let's take a step back: We want
to load all the bitmaps from the file exactly once, and that is
when it
is opened the first time. Or that's what I would have thought... Is
that even correct?
Why do we load the bitmaps when the device is inactive anyway?
Shouldn't we load them only once the device is activated?
Hmm, not sure. May be, we don't need. But we anyway need to load them,
when opening read-only, and we should correspondingly reopen in this
case.
Yeah, well, yeah, but the current state seems just wrong. Apparently
there are cases where a qcow2 node may have bitmaps before we try to
load them from the file, so the current condition doesn't work.
Furthermore, it seems like the current "state machine" is too complex so
we don't know which cases are possible anymore and what to do when.
So the first thing we could do is add a field to the BDRVQCow2State that
tells us whether the bitmaps have been loaded already or not. If not,
we invoke qcow2_load_dirty_bitmaps() and set the value to true. If the
value was true already and the BDS is active and R/W now, we call
qcow2_reopen_bitmaps_rw_hint(). That should solve one problem.
good idea, will do.
The other problem of course is the question whether we should call
qcow2_load_dirty_bitmaps() at all while the drive is still inactive.
You know the migration model better than me, so I'm asking this question
to you. We can phrase it differently: Do we need to load the bitmaps
before the drive is activated?
Now I think that we don't need. At least, we don't have such cases in
Virtuozzo (I hope :).
Why did I doubt:
1. We have cases, when we want to start vm as inactive, to be able to
export it's drive as NBD export, push some data to it and then start
the VM (which involves activating)
2. We have cases, when we want to start vm stopped and operate on
dirty bitmaps.
If just open all images in inactive mode until vm start, it looks like
we need bitmaps in inactive mode (for 2.). But it looks like wrong
approach anyway.
Firstly, I tried to solve (1.) by simply inactivate_all() in case of
start vm in paused mode, but it breaks at least (2.), so finally, I
solved (1.) by an approach similar with "-incoming defer". So, we have
inactive mode in two cases:
- incoming migration
- push data to vm before start
and, in these cases, we don't need to load dirty-bitmaps.
Also, inconsistency: now, we remove persistent bitmaps on inactivate.
So, it is inconsistent to load the in inactive mode.
Ok, I'll try to respin.
finally, what cases we actually have for qcow2_do_open?
1. INACTIVE -> ACTIVE (through invalidate_cache, we obviously should
load bitmaps, if we decided that we have no persistent bitmaps in
INACTIVE mode)
2. creating new bdrv state (first open of the image) in INACTIVE mode
(will not load bitmaps)
3. creating new bdrv state (first open of the image) in ACTIVE mode
(will load bitmaps, maybe read-only if disk is RO)
If only these three cases, it would be enough to just load bitmaps if
!INACTIVE and do nothing otherwise.
Or, we have some of the following cases too?
1?. ACTIVE -> ACTIVE (through invalidate_cache, some kind of no-op, we
should not reload bitmaps)
2?. RO -> RW (we should reopen_bitmaps_rw) (or it is possible only
through bdrv_reopen, which will
[Qemu-devel] [PATCH for-2.12] nbd: trace meta context negotiation
Having a more detailed log of the interaction between client and
server is invaluable in debugging how meta context negotiation
actually works.
Signed-off-by: Eric Blake
---
nbd/client.c | 2 ++
nbd/server.c | 8
nbd/trace-events | 6 ++
3 files changed, 16 insertions(+)
diff --git a/nbd/client.c b/nbd/client.c
index 4ee1d9a4a2c..478b6085df7 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -623,6 +623,7 @@ static int nbd_negotiate_simple_meta_context(QIOChannel
*ioc,
char *data = g_malloc(data_len);
char *p = data;
+trace_nbd_opt_meta_request(context, export);
stl_be_p(p, export_len);
memcpy(p += sizeof(export_len), export, export_len);
stl_be_p(p += export_len, 1);
@@ -681,6 +682,7 @@ static int nbd_negotiate_simple_meta_context(QIOChannel
*ioc,
}
g_free(name);
+trace_nbd_opt_meta_reply(context, received_id);
received = true;
/* receive NBD_REP_ACK */
diff --git a/nbd/server.c b/nbd/server.c
index cea158913ba..9e1f2271784 100644
--- a/nbd/server.c
+++ b/nbd/server.c
@@ -726,6 +726,7 @@ static int nbd_negotiate_send_meta_context(NBDClient
*client,
context_id = 0;
}
+trace_nbd_negotiate_meta_query_reply(context, context_id);
set_be_option_rep(&opt.h, client->opt, NBD_REP_META_CONTEXT,
sizeof(opt) - sizeof(opt.h) + iov[1].iov_len);
stl_be_p(&opt.context_id, context_id);
@@ -752,10 +753,12 @@ static int nbd_meta_base_query(NBDClient *client,
NBDExportMetaContexts *meta,
if (client->opt == NBD_OPT_LIST_META_CONTEXT) {
meta->base_allocation = true;
}
+trace_nbd_negotiate_meta_query_parse("base:");
return 1;
}
if (len != alen) {
+trace_nbd_negotiate_meta_query_skip("not base:allocation");
return nbd_opt_skip(client, len, errp);
}
@@ -768,6 +771,7 @@ static int nbd_meta_base_query(NBDClient *client,
NBDExportMetaContexts *meta,
meta->base_allocation = true;
}
+trace_nbd_negotiate_meta_query_parse("base:allocation");
return 1;
}
@@ -800,6 +804,7 @@ static int nbd_negotiate_meta_query(NBDClient *client,
/* The only supported namespace for now is 'base'. So query should start
* with 'base:'. Otherwise, we can ignore it and skip the remainder. */
if (len < baselen) {
+trace_nbd_negotiate_meta_query_skip("length too short");
return nbd_opt_skip(client, len, errp);
}
@@ -809,6 +814,7 @@ static int nbd_negotiate_meta_query(NBDClient *client,
return ret;
}
if (strncmp(query, "base:", baselen) != 0) {
+trace_nbd_negotiate_meta_query_skip("not for base: namespace");
return nbd_opt_skip(client, len, errp);
}
@@ -858,6 +864,8 @@ static int nbd_negotiate_meta_queries(NBDClient *client,
return ret;
}
cpu_to_be32s(&nb_queries);
+trace_nbd_negotiate_meta_context(nbd_opt_lookup(client->opt),
+ meta->export_name, nb_queries);
if (client->opt == NBD_OPT_LIST_META_CONTEXT && !nb_queries) {
/* enable all known contexts */
diff --git a/nbd/trace-events b/nbd/trace-events
index 0d03edc967d..cfdbe04b447 100644
--- a/nbd/trace-events
+++ b/nbd/trace-events
@@ -10,6 +10,8 @@ nbd_receive_query_exports_start(const char *wantname)
"Querying export list for
nbd_receive_query_exports_success(const char *wantname) "Found desired export
name '%s'"
nbd_receive_starttls_new_client(void) "Setting up TLS"
nbd_receive_starttls_tls_handshake(void) "Starting TLS handshake"
+nbd_opt_meta_request(const char *context, const char *export) "Requesting to
set meta context %s for export %s"
+nbd_opt_meta_reply(const char *context, int id) "Received mapping of context
%s to id %d"
nbd_receive_negotiate(void *tlscreds, const char *hostname) "Receiving
negotiation tlscreds=%p hostname=%s"
nbd_receive_negotiate_magic(uint64_t magic) "Magic is 0x%" PRIx64
nbd_receive_negotiate_server_flags(uint32_t globalflags) "Global flags are
0x%" PRIx32
@@ -44,6 +46,10 @@ nbd_negotiate_handle_info_request(int request, const char
*name) "Client request
nbd_negotiate_handle_info_block_size(uint32_t minimum, uint32_t preferred,
uint32_t maximum) "advertising minimum 0x%" PRIx32 ", preferred 0x%" PRIx32 ",
maximum 0x%" PRIx32
nbd_negotiate_handle_starttls(void) "Setting up TLS"
nbd_negotiate_handle_starttls_handshake(void) "Starting TLS handshake"
+nbd_negotiate_meta_context(const char *optname, const char *export, int
queries) "Client requested %s for export %s, with %d queries"
+nbd_negotiate_meta_query_skip(const char *reason) "Skipping meta query: %s"
+nbd_negotiate_meta_query_parse(const char *query) "Parsed meta query '%s'"
+nbd_negotiate_meta_query_reply(const char *context, unsigned int id) "Replying
with meta context '%s' id %d"
nbd_negotiate_options_flags(uint32_t flags) "Received client flags 0x%" PRIx32
nbd_negotiate_options_check_magi
[Qemu-devel] [PATCH v6 5/5] migration: use the free page hint feature from balloon
Start the free page optimization after the migration bitmap is
synchronized. This can't be used in the stop© phase since the guest
is paused. Make sure the guest reporting has stopped before
synchronizing the migration dirty bitmap. Currently, the optimization is
added to precopy only.
Signed-off-by: Wei Wang
CC: Dr. David Alan Gilbert
CC: Juan Quintela
CC: Michael S. Tsirkin
---
migration/ram.c | 26 +-
1 file changed, 25 insertions(+), 1 deletion(-)
diff --git a/migration/ram.c b/migration/ram.c
index 0147548..1d85ffa 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -51,6 +51,7 @@
#include "qemu/rcu_queue.h"
#include "migration/colo.h"
#include "migration/block.h"
+#include "sysemu/balloon.h"
/***/
/* ram save/restore */
@@ -213,6 +214,8 @@ struct RAMState {
uint32_t last_version;
/* We are in the first round */
bool ram_bulk_stage;
+/* The free pages optimization feature is supported */
+bool free_page_support;
/* How many times we have dirty too many pages */
int dirty_rate_high_cnt;
/* these variables are used for bitmap sync */
@@ -841,6 +844,10 @@ static void migration_bitmap_sync(RAMState *rs)
int64_t end_time;
uint64_t bytes_xfer_now;
+if (rs->free_page_support) {
+balloon_free_page_stop();
+}
+
ram_counters.dirty_sync_count++;
if (!rs->time_last_bitmap_sync) {
@@ -907,6 +914,10 @@ static void migration_bitmap_sync(RAMState *rs)
if (migrate_use_events()) {
qapi_event_send_migration_pass(ram_counters.dirty_sync_count, NULL);
}
+
+if (rs->free_page_support) {
+balloon_free_page_start();
+}
}
/**
@@ -1663,7 +1674,17 @@ static void ram_state_reset(RAMState *rs)
rs->last_sent_block = NULL;
rs->last_page = 0;
rs->last_version = ram_list.version;
-rs->ram_bulk_stage = true;
+rs->free_page_support = balloon_free_page_support() && !migrate_postcopy();
+if (rs->free_page_support) {
+/*
+ * When the free page optimization is used, not all the pages are
+ * treated as dirty pages (via migration_bitmap_find_dirty), which need
+ * to be sent. So disable ram_bulk_stage in this case.
+ */
+rs->ram_bulk_stage = false;
+} else {
+rs->ram_bulk_stage = true;
+}
}
#define MAX_WAIT 50 /* ms, half buffered_file limit */
@@ -2369,6 +2390,9 @@ out:
ret = qemu_file_get_error(f);
if (ret < 0) {
+if (rs->free_page_support) {
+balloon_free_page_stop();
+}
return ret;
}
--
1.8.3.1
Re: [Qemu-devel] [virtio-dev] Re: [PATCH v5 4/5] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT
On 03/20/2018 11:24 AM, Michael S. Tsirkin wrote: On Tue, Mar 20, 2018 at 11:18:23AM +0800, Wei Wang wrote: On 03/20/2018 10:59 AM, Michael S. Tsirkin wrote: On Tue, Mar 20, 2018 at 10:16:09AM +0800, Wei Wang wrote: On 03/20/2018 06:55 AM, Michael S. Tsirkin wrote: On Mon, Mar 19, 2018 at 05:01:38PM +0800, Wei Wang wrote: On 03/19/2018 12:24 PM, Michael S. Tsirkin wrote: On Sun, Mar 18, 2018 at 06:36:20PM +0800, Wei Wang wrote: On 03/16/2018 11:16 PM, Michael S. Tsirkin wrote: On Fri, Mar 16, 2018 at 06:48:28PM +0800, Wei Wang wrote: OTOH it seems that if thread stops nothing will wake it up whem vm is restarted. Such bahaviour change across vmstop/vmstart is unexpected. I do not understand why we want to increment the counter on vm stop though. It does make sense to stop the thread but why not resume where we left off when vm is resumed? I'm not sure which counter we incremented. But it would be clear if we have a high level view of how it works (it is symmetric actually). Basically, we start the optimization when each round starts and stop it at the end of each round (i.e. before we do the bitmap sync), as shown below: 1) 1st Round starts --> free_page_start 2) 1st Round in progress.. 3) 1st Round ends --> free_page_stop 4) 2nd Round starts --> free_page_start 5) 2nd Round in progress.. 6) 2nd Round ends --> free_page_stop .. For example, in 2), the VM is stopped. virtio_balloon_poll_free_page_hints finds the vq is empty (i.e. elem == NULL) and the runstate is stopped, the optimization thread exits immediately. That is, this optimization thread is gone forever (the optimization we can do for this round is done). We won't know when would the VM be woken up: A) If the VM is woken up very soon when the migration thread is still in progress of 2), then in 4) a new optimization thread (not the same one for the first round) will be created and start the optimization for the 2nd round as usual (If you have questions about 3) in this case, that free_page_stop will do nothing than just return, since the optimization thread has exited) ; B) If the VM is woken up after the whole migration has ended, there is still no point in resuming the optimization. I think this would be the simple design for the first release of this optimization. There are possibilities to improve case A) above by continuing optimization for the 1st Round as it is still in progress, but I think adding that complexity for this rare case wouldn't be worthwhile (at least for now). What would you think? Best, Wei In my opinion this just makes the patch very messy. E.g. attempts to attach a debugger to the guest will call vmstop and then behaviour changes. This is a receipe for heisenbugs which are then extremely painful to debug. It is not really hard to make things symmetrical: e.g. if you stop on vmstop then you should start on vmstart, etc. And stopping thread should not involve a bunch of state changes, just stop it and that's it. "stop it" - do you mean to 1) make the thread exit (i.e.make virtio_balloon_poll_free_page_hints exit the while loop and return NULL); or 2) keep the thread staying in the while loop but yield running (e.g. sleep(1) or block on a mutex)? (or please let me know if you suggested a different implementation about stopping the thread) I would say it makes more sense to make it block on something. BTW I still think you are engaging in premature optimization here. What you are doing here is a "data plane for balloon". I would make the feature work first by processing this in a BH. Creating threads immediately opens up questions of isolation, cgroups etc. Could you please share more about how creating threads affects isolation and cgroup? When threads are coming and going, they are hard to control. Consider the rich API libvirt exposes for controlling the io threads: https://libvirt.org/formatdomain.html#elementsIOThreadsAllocation and how does BH solve it? Thanks. Best, Wei It's late at night so I don't remember whether it's the emulator or the io thread that runs the BH, but the point is it's already controlled by libvirt. Thanks all for reviewing the patches. I've changed to use iothread in the new version. Please have a check if that would be good enough. Best, Wei
[Qemu-devel] [PATCH v6 4/5] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT
The new feature enables the virtio-balloon device to receive hints of
guest free pages from the free page vq.
balloon_free_page_start - start guest free page hint reporting.
balloon_free_page_stop - stop guest free page hint reporting.
Note: balloon will report pages which were free at the time
of this call. As the reporting happens asynchronously, dirty bit logging
must be enabled before this call is made. Guest reporting must be
disabled before the migration dirty bitmap is synchronized.
TODO:
- handle the case when page poisoning is in use
Signed-off-by: Wei Wang
Signed-off-by: Liang Li
CC: Michael S. Tsirkin
CC: Dr. David Alan Gilbert
CC: Juan Quintela
---
balloon.c | 58 +-
hw/virtio/virtio-balloon.c | 240 ++--
include/hw/virtio/virtio-balloon.h | 27 ++-
include/standard-headers/linux/virtio_balloon.h | 7 +
include/sysemu/balloon.h| 15 +-
5 files changed, 318 insertions(+), 29 deletions(-)
diff --git a/balloon.c b/balloon.c
index 6bf0a96..87a0410 100644
--- a/balloon.c
+++ b/balloon.c
@@ -36,6 +36,9 @@
static QEMUBalloonEvent *balloon_event_fn;
static QEMUBalloonStatus *balloon_stat_fn;
+static QEMUBalloonFreePageSupport *balloon_free_page_support_fn;
+static QEMUBalloonFreePageStart *balloon_free_page_start_fn;
+static QEMUBalloonFreePageStop *balloon_free_page_stop_fn;
static void *balloon_opaque;
static bool balloon_inhibited;
@@ -64,19 +67,51 @@ static bool have_balloon(Error **errp)
return true;
}
-int qemu_add_balloon_handler(QEMUBalloonEvent *event_func,
- QEMUBalloonStatus *stat_func, void *opaque)
+bool balloon_free_page_support(void)
{
-if (balloon_event_fn || balloon_stat_fn || balloon_opaque) {
-/* We're already registered one balloon handler. How many can
- * a guest really have?
- */
-return -1;
+return balloon_free_page_support_fn &&
+ balloon_free_page_support_fn(balloon_opaque);
+}
+
+/*
+ * Balloon will report pages which were free at the time of this call. As the
+ * reporting happens asynchronously, dirty bit logging must be enabled before
+ * this call is made.
+ */
+void balloon_free_page_start(void)
+{
+balloon_free_page_start_fn(balloon_opaque);
+}
+
+/*
+ * Guest reporting must be disabled before the migration dirty bitmap is
+ * synchronized.
+ */
+void balloon_free_page_stop(void)
+{
+balloon_free_page_stop_fn(balloon_opaque);
+}
+
+void qemu_add_balloon_handler(QEMUBalloonEvent *event_fn,
+ QEMUBalloonStatus *stat_fn,
+ QEMUBalloonFreePageSupport *free_page_support_fn,
+ QEMUBalloonFreePageStart *free_page_start_fn,
+ QEMUBalloonFreePageStop *free_page_stop_fn,
+ void *opaque)
+{
+if (balloon_event_fn || balloon_stat_fn || balloon_free_page_support_fn ||
+balloon_free_page_start_fn || balloon_free_page_stop_fn ||
+balloon_opaque) {
+/* We already registered one balloon handler. */
+return;
}
-balloon_event_fn = event_func;
-balloon_stat_fn = stat_func;
+
+balloon_event_fn = event_fn;
+balloon_stat_fn = stat_fn;
+balloon_free_page_support_fn = free_page_support_fn;
+balloon_free_page_start_fn = free_page_start_fn;
+balloon_free_page_stop_fn = free_page_stop_fn;
balloon_opaque = opaque;
-return 0;
}
void qemu_remove_balloon_handler(void *opaque)
@@ -86,6 +121,9 @@ void qemu_remove_balloon_handler(void *opaque)
}
balloon_event_fn = NULL;
balloon_stat_fn = NULL;
+balloon_free_page_support_fn = NULL;
+balloon_free_page_start_fn = NULL;
+balloon_free_page_stop_fn = NULL;
balloon_opaque = NULL;
}
diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c
index f456cea..2710c6e 100644
--- a/hw/virtio/virtio-balloon.c
+++ b/hw/virtio/virtio-balloon.c
@@ -31,6 +31,7 @@
#include "hw/virtio/virtio-bus.h"
#include "hw/virtio/virtio-access.h"
+#include "migration/misc.h"
#define BALLOON_PAGE_SIZE (1 << VIRTIO_BALLOON_PFN_SHIFT)
@@ -308,6 +309,124 @@ out:
}
}
+static void virtio_balloon_poll_free_page_hints(void *opaque)
+{
+VirtQueueElement *elem;
+VirtIOBalloon *dev = opaque;
+VirtIODevice *vdev = VIRTIO_DEVICE(dev);
+VirtQueue *vq = dev->free_page_vq;
+uint32_t id;
+size_t size;
+
+while (1) {
+qemu_mutex_lock(&dev->free_page_lock);
+while (dev->block_iothread) {
+qemu_cond_wait(&dev->free_page_cond, &dev->free_page_lock);
+}
+
+/*
+ * If the migration thread actively stops the reporting, exit
+ * immediately.
+ */
+if (dev->free_page_report_status == FREE_PAGE_REPORT_S_STOP) {
+qemu_mutex_unlock(&dev->free_page_lock);
+break;
+
[Qemu-devel] [PATCH v6 2/5] migration: use bitmap_mutex in migration_bitmap_clear_dirty
The bitmap mutex is used to synchronize threads to update the dirty
bitmap and the migration_dirty_pages counter. This patch makes
migration_bitmap_clear_dirty update the bitmap and counter under the
mutex.
Signed-off-by: Wei Wang
CC: Dr. David Alan Gilbert
CC: Juan Quintela
CC: Michael S. Tsirkin
---
migration/ram.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/migration/ram.c b/migration/ram.c
index 0e90efa..9a72b1a 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -795,11 +795,14 @@ static inline bool migration_bitmap_clear_dirty(RAMState
*rs,
{
bool ret;
+qemu_mutex_lock(&rs->bitmap_mutex);
ret = test_and_clear_bit(page, rb->bmap);
if (ret) {
rs->migration_dirty_pages--;
}
+qemu_mutex_unlock(&rs->bitmap_mutex);
+
return ret;
}
--
1.8.3.1
[Qemu-devel] [PATCH v6 3/5] migration: API to clear bits of guest free pages from the dirty bitmap
This patch adds an API to clear bits corresponding to guest free pages
from the dirty bitmap. Spilt the free page block if it crosses the QEMU
RAMBlock boundary.
Signed-off-by: Wei Wang
CC: Dr. David Alan Gilbert
CC: Juan Quintela
CC: Michael S. Tsirkin
---
include/migration/misc.h | 2 ++
migration/ram.c | 44
2 files changed, 46 insertions(+)
diff --git a/include/migration/misc.h b/include/migration/misc.h
index 4ebf24c..113320e 100644
--- a/include/migration/misc.h
+++ b/include/migration/misc.h
@@ -14,11 +14,13 @@
#ifndef MIGRATION_MISC_H
#define MIGRATION_MISC_H
+#include "exec/cpu-common.h"
#include "qemu/notify.h"
/* migration/ram.c */
void ram_mig_init(void);
+void qemu_guest_free_page_hint(void *addr, size_t len);
/* migration/block.c */
diff --git a/migration/ram.c b/migration/ram.c
index 9a72b1a..0147548 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -2198,6 +2198,50 @@ static int ram_init_all(RAMState **rsp)
}
/*
+ * This function clears bits of the free pages reported by the caller from the
+ * migration dirty bitmap. @addr is the host address corresponding to the
+ * start of the continuous guest free pages, and @len is the total bytes of
+ * those pages.
+ */
+void qemu_guest_free_page_hint(void *addr, size_t len)
+{
+RAMBlock *block;
+ram_addr_t offset;
+size_t used_len, start, npages;
+
+for (; len > 0; len -= used_len) {
+block = qemu_ram_block_from_host(addr, false, &offset);
+if (unlikely(!block)) {
+return;
+}
+
+/*
+ * This handles the case that the RAMBlock is resized after the free
+ * page hint is reported.
+ */
+if (unlikely(offset > block->used_length)) {
+return;
+}
+
+if (len <= block->used_length - offset) {
+used_len = len;
+} else {
+used_len = block->used_length - offset;
+addr += used_len;
+}
+
+start = offset >> TARGET_PAGE_BITS;
+npages = used_len >> TARGET_PAGE_BITS;
+
+qemu_mutex_lock(&ram_state->bitmap_mutex);
+ram_state->migration_dirty_pages -=
+ bitmap_count_one_with_offset(block->bmap, start, npages);
+bitmap_clear(block->bmap, start, npages);
+qemu_mutex_unlock(&ram_state->bitmap_mutex);
+}
+}
+
+/*
* Each of ram_save_setup, ram_save_iterate and ram_save_complete has
* long-running RCU critical section. When rcu-reclaims in the code
* start to become numerous it will be necessary to reduce the
--
1.8.3.1
[Qemu-devel] [PATCH v6 1/5] bitmap: bitmap_count_one_with_offset
Count the number of 1s in a bitmap starting from an offset.
Signed-off-by: Wei Wang
CC: Dr. David Alan Gilbert
CC: Juan Quintela
CC: Michael S. Tsirkin
Reviewed-by: Dr. David Alan Gilbert
---
include/qemu/bitmap.h | 13 +
1 file changed, 13 insertions(+)
diff --git a/include/qemu/bitmap.h b/include/qemu/bitmap.h
index 509eedd..e3f31f1 100644
--- a/include/qemu/bitmap.h
+++ b/include/qemu/bitmap.h
@@ -228,6 +228,19 @@ static inline long bitmap_count_one(const unsigned long
*bitmap, long nbits)
}
}
+static inline long bitmap_count_one_with_offset(const unsigned long *bitmap,
+long offset, long nbits)
+{
+long aligned_offset = QEMU_ALIGN_DOWN(offset, BITS_PER_LONG);
+long redundant_bits = offset - aligned_offset;
+long bits_to_count = nbits + redundant_bits;
+const unsigned long *bitmap_start = bitmap +
+aligned_offset / BITS_PER_LONG;
+
+return bitmap_count_one(bitmap_start, bits_to_count) -
+ bitmap_count_one(bitmap_start, redundant_bits);
+}
+
void bitmap_set(unsigned long *map, long i, long len);
void bitmap_set_atomic(unsigned long *map, long i, long len);
void bitmap_clear(unsigned long *map, long start, long nr);
--
1.8.3.1
[Qemu-devel] [PATCH v6 0/5] virtio-balloon: free page hint reporting support
This is the deivce part implementation to add a new feature,
VIRTIO_BALLOON_F_FREE_PAGE_HINT to the virtio-balloon device. The device
receives the guest free page hints from the driver and clears the
corresponding bits in the dirty bitmap, so that those free pages are
not transferred by the migration thread to the destination.
- Test Environment
Host: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz
Guest: 8G RAM, 4 vCPU
Migration setup: migrate_set_speed 100G, migrate_set_downtime 2 second
- Test Results
- Idle Guest Live Migration Time (results are averaged over 10 runs):
- Optimization v.s. Legacy = 271ms vs 1769ms --> ~86% reduction
- Guest with Linux Compilation Workload (make bzImage -j4):
- Live Migration Time (average)
Optimization v.s. Legacy = 1265ms v.s. 2634ms --> ~51% reduction
- Linux Compilation Time
Optimization v.s. Legacy = 4min56s v.s. 5min3s
--> no obvious difference
- Source Code
- QEMU: https://github.com/wei-w-wang/qemu-free-page-lm.git
- Linux: https://github.com/wei-w-wang/linux-free-page-lm.git
ChangeLog:
v5->v6:
virtio-balloon:
- use an iothread to get free page hint;
- block the iothread when the vm is stoppedm and
unblock it when it is woken up;
- remove FREE_PAGE_REPORT_S_EXIT which is not needed now.
v4->v5:
1) migration:
- bitmap_clear_dirty: update the dirty bitmap and dirty page
count under the bitmap mutex as what other functions are doing;
- qemu_guest_free_page_hint:
- add comments for this function;
- check the !block case;
- check "offset > block->used_length" before proceed;
- assign used_len inside the for{} body;
- update the dirty bitmap and dirty page counter under the
bitmap mutex;
- ram_state_reset:
- rs->free_page_support: && with use "migrate_postcopy"
instead of migration_in_postcopy;
- clear the ram_bulk_stage flag if free_page_support is true;
2) balloon:
- add the usage documentation of balloon_free_page_start and
balloon_free_page_stop in code;
- the optimization thread is named "balloon_fpo" to meet the
requirement of "less than 14 characters";
- virtio_balloon_poll_free_page_hints:
- run on condition when runstate_is_running() is true;
- add a qemu spin lock to synchronize accesses to the free
page reporting related fields shared among the migration
thread and the optimization thread;
- virtio_balloon_free_page_start: just return if
runstate_is_running is false;
- virtio_balloon_free_page_stop: access to the free page
reporting related fields under a qemu spin lock;
- virtio_balloon_device_unrealize/reset: call
virtio_balloon_free_page_stop is the free page hint feature is
used;
- virtio_balloon_set_status: call irtio_balloon_free_page_stop
in case the guest is stopped by qmp when the optimization is
running;
v3->v4:
1) bitmap: add a new API to count 1s starting from an offset of a
bitmap
2) migration:
- qemu_guest_free_page_hint: calculate
ram_state->migration_dirty_pages by counting how many bits of
free pages are truely cleared. If some of the bits were
already 0, they shouldn't be deducted by
ram_state->migration_dirty_pages. This wasn't needed for
previous versions since we optimized bulk stage only,
where all bits are guaranteed to be set. It's needed now
because we extened the usage of this optimizaton to all stages
except the last stop© stage. From 2nd stage onward, there
are possibilities that some bits of free pages are already 0.
3) virtio-balloon:
- virtio_balloon_free_page_report_status: introduce a new status,
FREE_PAGE_REPORT_S_EXIT. This status indicates that the
optimization thread has exited. FREE_PAGE_REPORT_S_STOP means
the reporting is stopped, but the optimization thread still needs
to be joined by the migration thread.
v2->v3:
1) virtio-balloon
- virtio_balloon_free_page_start: poll the hints using a new
thread;
- use cmd id between [0x8000, UINT_MAX];
- virtio_balloon_poll_free_page_hints:
- stop the optimization only when it has started;
- don't skip free pages when !poison_val;
- add poison_val to vmsd to migrate;
- virtio_balloon_get_features: add the F_PAGE_POISON feature when
host has F_FREE_PAGE_HINT;
- remove the timer patch which is not needed now.
2) migration
- new api, qemu_guest_free_page_hint;
- rs->free_page_support set only in the precopy c
[Qemu-devel] [RFC] About 9pfs support "O_DIRECT + aio"?
Hi everyone, Currently, I found virtio-9p in VirtFS don't support "O_DIRECT + aio" mode, both v9fs and qemu. So when user use "O_DIRECT + aio" mode and increase iodepths, they can't get higher IOPS. I want to know why v9fs don't implement this mode? And I will try to implement this mode from now on. Thanks, Yiwen.
[Qemu-devel] [PATCH v3 10/10] migration: remove ram_save_compressed_page()
From: Xiao Guangrong
Now, we can reuse the path in ram_save_page() to post the page out
as normal, then the only thing remained in ram_save_compressed_page()
is compression that we can move it out to the caller
Reviewed-by: Peter Xu
Reviewed-by: Dr. David Alan Gilbert
Signed-off-by: Xiao Guangrong
---
migration/ram.c | 45 -
1 file changed, 8 insertions(+), 37 deletions(-)
diff --git a/migration/ram.c b/migration/ram.c
index 2eb4c0bf49..912810c18e 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -1184,41 +1184,6 @@ static int compress_page_with_multi_thread(RAMState *rs,
RAMBlock *block,
return pages;
}
-/**
- * ram_save_compressed_page: compress the given page and send it to the stream
- *
- * Returns the number of pages written.
- *
- * @rs: current RAM state
- * @block: block that contains the page we want to send
- * @offset: offset inside the block for the page
- * @last_stage: if we are at the completion stage
- */
-static int ram_save_compressed_page(RAMState *rs, PageSearchStatus *pss,
-bool last_stage)
-{
-int pages = -1;
-uint8_t *p;
-RAMBlock *block = pss->block;
-ram_addr_t offset = pss->page << TARGET_PAGE_BITS;
-
-p = block->host + offset;
-
-if (block != rs->last_sent_block) {
-/*
- * Make sure the first page is sent out before other pages.
- *
- * we post it as normal page as compression will take much
- * CPU resource.
- */
-pages = save_normal_page(rs, block, offset, p, true);
-} else {
-pages = compress_page_with_multi_thread(rs, block, offset);
-}
-
-return pages;
-}
-
/**
* find_dirty_block: find the next dirty page and update any state
* associated with the search process.
@@ -1518,8 +1483,14 @@ static int ram_save_target_page(RAMState *rs,
PageSearchStatus *pss,
return res;
}
-if (save_page_use_compression(rs)) {
-return ram_save_compressed_page(rs, pss, last_stage);
+/*
+ * Make sure the first page is sent out before other pages.
+ *
+ * we post it as normal page as compression will take much
+ * CPU resource.
+ */
+if (block == rs->last_sent_block && save_page_use_compression(rs)) {
+res = compress_page_with_multi_thread(rs, block, offset);
}
return ram_save_page(rs, pss, last_stage);
--
2.14.3
[Qemu-devel] [PATCH v3 05/10] migration: introduce control_save_page()
From: Xiao Guangrong
Abstract the common function control_save_page() to cleanup the code,
no logic is changed
Reviewed-by: Peter Xu
Reviewed-by: Dr. David Alan Gilbert
Signed-off-by: Xiao Guangrong
---
migration/ram.c | 174 +---
1 file changed, 89 insertions(+), 85 deletions(-)
diff --git a/migration/ram.c b/migration/ram.c
index 72cb8dfb66..79c7958993 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -974,6 +974,44 @@ static void ram_release_pages(const char *rbname, uint64_t
offset, int pages)
ram_discard_range(rbname, offset, pages << TARGET_PAGE_BITS);
}
+/*
+ * @pages: the number of pages written by the control path,
+ *< 0 - error
+ *> 0 - number of pages written
+ *
+ * Return true if the pages has been saved, otherwise false is returned.
+ */
+static bool control_save_page(RAMState *rs, RAMBlock *block, ram_addr_t offset,
+ int *pages)
+{
+uint64_t bytes_xmit = 0;
+int ret;
+
+*pages = -1;
+ret = ram_control_save_page(rs->f, block->offset, offset, TARGET_PAGE_SIZE,
+&bytes_xmit);
+if (ret == RAM_SAVE_CONTROL_NOT_SUPP) {
+return false;
+}
+
+if (bytes_xmit) {
+ram_counters.transferred += bytes_xmit;
+*pages = 1;
+}
+
+if (ret == RAM_SAVE_CONTROL_DELAYED) {
+return true;
+}
+
+if (bytes_xmit > 0) {
+ram_counters.normal++;
+} else if (bytes_xmit == 0) {
+ram_counters.duplicate++;
+}
+
+return true;
+}
+
/**
* ram_save_page: send the given page to the stream
*
@@ -990,56 +1028,36 @@ static void ram_release_pages(const char *rbname,
uint64_t offset, int pages)
static int ram_save_page(RAMState *rs, PageSearchStatus *pss, bool last_stage)
{
int pages = -1;
-uint64_t bytes_xmit;
-ram_addr_t current_addr;
uint8_t *p;
-int ret;
bool send_async = true;
RAMBlock *block = pss->block;
ram_addr_t offset = pss->page << TARGET_PAGE_BITS;
+ram_addr_t current_addr = block->offset + offset;
p = block->host + offset;
trace_ram_save_page(block->idstr, (uint64_t)offset, p);
-/* In doubt sent page as normal */
-bytes_xmit = 0;
-ret = ram_control_save_page(rs->f, block->offset,
- offset, TARGET_PAGE_SIZE, &bytes_xmit);
-if (bytes_xmit) {
-ram_counters.transferred += bytes_xmit;
-pages = 1;
+if (control_save_page(rs, block, offset, &pages)) {
+return pages;
}
XBZRLE_cache_lock();
-
-current_addr = block->offset + offset;
-
-if (ret != RAM_SAVE_CONTROL_NOT_SUPP) {
-if (ret != RAM_SAVE_CONTROL_DELAYED) {
-if (bytes_xmit > 0) {
-ram_counters.normal++;
-} else if (bytes_xmit == 0) {
-ram_counters.duplicate++;
-}
-}
-} else {
-pages = save_zero_page(rs, block, offset);
-if (pages > 0) {
-/* Must let xbzrle know, otherwise a previous (now 0'd) cached
- * page would be stale
+pages = save_zero_page(rs, block, offset);
+if (pages > 0) {
+/* Must let xbzrle know, otherwise a previous (now 0'd) cached
+ * page would be stale
+ */
+xbzrle_cache_zero_page(rs, current_addr);
+ram_release_pages(block->idstr, offset, pages);
+} else if (!rs->ram_bulk_stage &&
+ !migration_in_postcopy() && migrate_use_xbzrle()) {
+pages = save_xbzrle_page(rs, &p, current_addr, block,
+ offset, last_stage);
+if (!last_stage) {
+/* Can't send this cached data async, since the cache page
+ * might get updated before it gets to the wire
*/
-xbzrle_cache_zero_page(rs, current_addr);
-ram_release_pages(block->idstr, offset, pages);
-} else if (!rs->ram_bulk_stage &&
- !migration_in_postcopy() && migrate_use_xbzrle()) {
-pages = save_xbzrle_page(rs, &p, current_addr, block,
- offset, last_stage);
-if (!last_stage) {
-/* Can't send this cached data async, since the cache page
- * might get updated before it gets to the wire
- */
-send_async = false;
-}
+send_async = false;
}
}
@@ -1174,63 +1192,49 @@ static int ram_save_compressed_page(RAMState *rs,
PageSearchStatus *pss,
bool last_stage)
{
int pages = -1;
-uint64_t bytes_xmit = 0;
uint8_t *p;
-int ret;
RAMBlock *block = pss->block;
ram_addr_t offset = pss->page << TARGET_PAGE_BITS;
p = block->host + offset;
-ret = ram_control_save_page(rs->f, block->offset,
-offset, TARGET_PAGE_SIZE, &bytes_xmit);
-if (bytes_xmit
[Qemu-devel] [PATCH v3 09/10] migration: introduce save_normal_page()
From: Xiao Guangrong
It directly sends the page to the stream neither checking zero nor
using xbzrle or compression
Reviewed-by: Peter Xu
Reviewed-by: Dr. David Alan Gilbert
Signed-off-by: Xiao Guangrong
---
migration/ram.c | 50 ++
1 file changed, 30 insertions(+), 20 deletions(-)
diff --git a/migration/ram.c b/migration/ram.c
index 97917542c5..2eb4c0bf49 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -1012,6 +1012,34 @@ static bool control_save_page(RAMState *rs, RAMBlock
*block, ram_addr_t offset,
return true;
}
+/*
+ * directly send the page to the stream
+ *
+ * Returns the number of pages written.
+ *
+ * @rs: current RAM state
+ * @block: block that contains the page we want to send
+ * @offset: offset inside the block for the page
+ * @buf: the page to be sent
+ * @async: send to page asyncly
+ */
+static int save_normal_page(RAMState *rs, RAMBlock *block, ram_addr_t offset,
+uint8_t *buf, bool async)
+{
+ram_counters.transferred += save_page_header(rs, rs->f, block,
+ offset | RAM_SAVE_FLAG_PAGE);
+if (async) {
+qemu_put_buffer_async(rs->f, buf, TARGET_PAGE_SIZE,
+ migrate_release_ram() &
+ migration_in_postcopy());
+} else {
+qemu_put_buffer(rs->f, buf, TARGET_PAGE_SIZE);
+}
+ram_counters.transferred += TARGET_PAGE_SIZE;
+ram_counters.normal++;
+return 1;
+}
+
/**
* ram_save_page: send the given page to the stream
*
@@ -1052,18 +1080,7 @@ static int ram_save_page(RAMState *rs, PageSearchStatus
*pss, bool last_stage)
/* XBZRLE overflow or normal page */
if (pages == -1) {
-ram_counters.transferred +=
-save_page_header(rs, rs->f, block, offset | RAM_SAVE_FLAG_PAGE);
-if (send_async) {
-qemu_put_buffer_async(rs->f, p, TARGET_PAGE_SIZE,
- migrate_release_ram() &
- migration_in_postcopy());
-} else {
-qemu_put_buffer(rs->f, p, TARGET_PAGE_SIZE);
-}
-ram_counters.transferred += TARGET_PAGE_SIZE;
-pages = 1;
-ram_counters.normal++;
+pages = save_normal_page(rs, block, offset, p, send_async);
}
XBZRLE_cache_unlock();
@@ -1194,14 +1211,7 @@ static int ram_save_compressed_page(RAMState *rs,
PageSearchStatus *pss,
* we post it as normal page as compression will take much
* CPU resource.
*/
-ram_counters.transferred += save_page_header(rs, rs->f, block,
-offset | RAM_SAVE_FLAG_PAGE);
-qemu_put_buffer_async(rs->f, p, TARGET_PAGE_SIZE,
- migrate_release_ram() &
- migration_in_postcopy());
-ram_counters.transferred += TARGET_PAGE_SIZE;
-ram_counters.normal++;
-pages = 1;
+pages = save_normal_page(rs, block, offset, p, true);
} else {
pages = compress_page_with_multi_thread(rs, block, offset);
}
--
2.14.3
[Qemu-devel] [PATCH v3 02/10] migration: stop compression to allocate and free memory frequently
From: Xiao Guangrong
Current code uses compress2() to compress memory which manages memory
internally, that causes huge memory is allocated and freed very
frequently
More worse, frequently returning memory to kernel will flush TLBs
and trigger invalidation callbacks on mmu-notification which
interacts with KVM MMU, that dramatically reduce the performance
of VM
So, we maintain the memory by ourselves and reuse it for each
compression
Reviewed-by: Peter Xu
Reviewed-by: Jiang Biao
Signed-off-by: Xiao Guangrong
---
migration/qemu-file.c | 39 ---
migration/qemu-file.h | 6 --
migration/ram.c | 41 -
3 files changed, 68 insertions(+), 18 deletions(-)
diff --git a/migration/qemu-file.c b/migration/qemu-file.c
index bb63c779cc..bafe3a0c0d 100644
--- a/migration/qemu-file.c
+++ b/migration/qemu-file.c
@@ -658,8 +658,32 @@ uint64_t qemu_get_be64(QEMUFile *f)
return v;
}
-/* Compress size bytes of data start at p with specific compression
- * level and store the compressed data to the buffer of f.
+/* return the size after compression, or negative value on error */
+static int qemu_compress_data(z_stream *stream, uint8_t *dest, size_t dest_len,
+ const uint8_t *source, size_t source_len)
+{
+int err;
+
+err = deflateReset(stream);
+if (err != Z_OK) {
+return -1;
+}
+
+stream->avail_in = source_len;
+stream->next_in = (uint8_t *)source;
+stream->avail_out = dest_len;
+stream->next_out = dest;
+
+err = deflate(stream, Z_FINISH);
+if (err != Z_STREAM_END) {
+return -1;
+}
+
+return stream->next_out - dest;
+}
+
+/* Compress size bytes of data start at p and store the compressed
+ * data to the buffer of f.
*
* When f is not writable, return -1 if f has no space to save the
* compressed data.
@@ -667,9 +691,8 @@ uint64_t qemu_get_be64(QEMUFile *f)
* do fflush first, if f still has no space to save the compressed
* data, return -1.
*/
-
-ssize_t qemu_put_compression_data(QEMUFile *f, const uint8_t *p, size_t size,
- int level)
+ssize_t qemu_put_compression_data(QEMUFile *f, z_stream *stream,
+ const uint8_t *p, size_t size)
{
ssize_t blen = IO_BUF_SIZE - f->buf_index - sizeof(int32_t);
@@ -683,8 +706,10 @@ ssize_t qemu_put_compression_data(QEMUFile *f, const
uint8_t *p, size_t size,
return -1;
}
}
-if (compress2(f->buf + f->buf_index + sizeof(int32_t), (uLongf *)&blen,
- (Bytef *)p, size, level) != Z_OK) {
+
+blen = qemu_compress_data(stream, f->buf + f->buf_index + sizeof(int32_t),
+ blen, p, size);
+if (blen < 0) {
error_report("Compress Failed!");
return 0;
}
diff --git a/migration/qemu-file.h b/migration/qemu-file.h
index f4f356ab12..2ccfcfb2a8 100644
--- a/migration/qemu-file.h
+++ b/migration/qemu-file.h
@@ -25,6 +25,8 @@
#ifndef MIGRATION_QEMU_FILE_H
#define MIGRATION_QEMU_FILE_H
+#include
+
/* Read a chunk of data from a file at the given position. The pos argument
* can be ignored if the file is only be used for streaming. The number of
* bytes actually read should be returned.
@@ -132,8 +134,8 @@ bool qemu_file_is_writable(QEMUFile *f);
size_t qemu_peek_buffer(QEMUFile *f, uint8_t **buf, size_t size, size_t
offset);
size_t qemu_get_buffer_in_place(QEMUFile *f, uint8_t **buf, size_t size);
-ssize_t qemu_put_compression_data(QEMUFile *f, const uint8_t *p, size_t size,
- int level);
+ssize_t qemu_put_compression_data(QEMUFile *f, z_stream *stream,
+ const uint8_t *p, size_t size);
int qemu_put_qemu_file(QEMUFile *f_des, QEMUFile *f_src);
/*
diff --git a/migration/ram.c b/migration/ram.c
index 409c847a76..a21514a469 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -269,6 +269,7 @@ struct CompressParam {
QemuCond cond;
RAMBlock *block;
ram_addr_t offset;
+z_stream stream;
};
typedef struct CompressParam CompressParam;
@@ -299,7 +300,7 @@ static QemuThread *decompress_threads;
static QemuMutex decomp_done_lock;
static QemuCond decomp_done_cond;
-static int do_compress_ram_page(QEMUFile *f, RAMBlock *block,
+static int do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *block,
ram_addr_t offset);
static void *do_data_compress(void *opaque)
@@ -316,7 +317,7 @@ static void *do_data_compress(void *opaque)
param->block = NULL;
qemu_mutex_unlock(¶m->mutex);
-do_compress_ram_page(param->file, block, offset);
+do_compress_ram_page(param->file, ¶m->stream, block, offset);
qemu_mutex_lock(&comp_done_lock);
param->done = true;
@@ -357,10 +358,19 @@ static void compress_threads_save_cleanup(void)
[Qemu-devel] [PATCH v3 03/10] migration: stop decompression to allocate and free memory frequently
From: Xiao Guangrong
Current code uses uncompress() to decompress memory which manages
memory internally, that causes huge memory is allocated and freed
very frequently, more worse, frequently returning memory to kernel
will flush TLBs
So, we maintain the memory by ourselves and reuse it for each
decompression
Reviewed-by: Peter Xu
Reviewed-by: Jiang Biao
Signed-off-by: Xiao Guangrong
---
migration/ram.c | 112 +---
1 file changed, 82 insertions(+), 30 deletions(-)
diff --git a/migration/ram.c b/migration/ram.c
index a21514a469..fb24b2f32f 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -281,6 +281,7 @@ struct DecompressParam {
void *des;
uint8_t *compbuf;
int len;
+z_stream stream;
};
typedef struct DecompressParam DecompressParam;
@@ -2524,6 +2525,31 @@ void ram_handle_compressed(void *host, uint8_t ch,
uint64_t size)
}
}
+/* return the size after decompression, or negative value on error */
+static int
+qemu_uncompress_data(z_stream *stream, uint8_t *dest, size_t dest_len,
+ const uint8_t *source, size_t source_len)
+{
+int err;
+
+err = inflateReset(stream);
+if (err != Z_OK) {
+return -1;
+}
+
+stream->avail_in = source_len;
+stream->next_in = (uint8_t *)source;
+stream->avail_out = dest_len;
+stream->next_out = dest;
+
+err = inflate(stream, Z_NO_FLUSH);
+if (err != Z_STREAM_END) {
+return -1;
+}
+
+return stream->total_out;
+}
+
static void *do_data_decompress(void *opaque)
{
DecompressParam *param = opaque;
@@ -2540,13 +2566,13 @@ static void *do_data_decompress(void *opaque)
qemu_mutex_unlock(¶m->mutex);
pagesize = TARGET_PAGE_SIZE;
-/* uncompress() will return failed in some case, especially
- * when the page is dirted when doing the compression, it's
- * not a problem because the dirty page will be retransferred
+/* qemu_uncompress_data() will return failed in some case,
+ * especially when the page is dirtied when doing the compression,
+ * it's not a problem because the dirty page will be retransferred
* and uncompress() won't break the data in other pages.
*/
-uncompress((Bytef *)des, &pagesize,
- (const Bytef *)param->compbuf, len);
+qemu_uncompress_data(¶m->stream, des, pagesize, param->compbuf,
+ len);
qemu_mutex_lock(&decomp_done_lock);
param->done = true;
@@ -2581,30 +2607,6 @@ static void wait_for_decompress_done(void)
qemu_mutex_unlock(&decomp_done_lock);
}
-static void compress_threads_load_setup(void)
-{
-int i, thread_count;
-
-if (!migrate_use_compression()) {
-return;
-}
-thread_count = migrate_decompress_threads();
-decompress_threads = g_new0(QemuThread, thread_count);
-decomp_param = g_new0(DecompressParam, thread_count);
-qemu_mutex_init(&decomp_done_lock);
-qemu_cond_init(&decomp_done_cond);
-for (i = 0; i < thread_count; i++) {
-qemu_mutex_init(&decomp_param[i].mutex);
-qemu_cond_init(&decomp_param[i].cond);
-decomp_param[i].compbuf = g_malloc0(compressBound(TARGET_PAGE_SIZE));
-decomp_param[i].done = true;
-decomp_param[i].quit = false;
-qemu_thread_create(decompress_threads + i, "decompress",
- do_data_decompress, decomp_param + i,
- QEMU_THREAD_JOINABLE);
-}
-}
-
static void compress_threads_load_cleanup(void)
{
int i, thread_count;
@@ -2614,16 +2616,30 @@ static void compress_threads_load_cleanup(void)
}
thread_count = migrate_decompress_threads();
for (i = 0; i < thread_count; i++) {
+/*
+ * we use it as a indicator which shows if the thread is
+ * properly init'd or not
+ */
+if (!decomp_param[i].compbuf) {
+break;
+}
+
qemu_mutex_lock(&decomp_param[i].mutex);
decomp_param[i].quit = true;
qemu_cond_signal(&decomp_param[i].cond);
qemu_mutex_unlock(&decomp_param[i].mutex);
}
for (i = 0; i < thread_count; i++) {
+if (!decomp_param[i].compbuf) {
+break;
+}
+
qemu_thread_join(decompress_threads + i);
qemu_mutex_destroy(&decomp_param[i].mutex);
qemu_cond_destroy(&decomp_param[i].cond);
+inflateEnd(&decomp_param[i].stream);
g_free(decomp_param[i].compbuf);
+decomp_param[i].compbuf = NULL;
}
g_free(decompress_threads);
g_free(decomp_param);
@@ -2631,6 +2647,39 @@ static void compress_threads_load_cleanup(void)
decomp_param = NULL;
}
+static int compress_threads_load_setup(void)
+{
+int i, thread_count;
+
+if (!migrate_use_compression()) {
+return 0;
+}
+
+th
[Qemu-devel] [PATCH v3 07/10] migration: move calling control_save_page to the common place
From: Xiao Guangrong
The function is called by both ram_save_page and ram_save_target_page,
so move it to the common caller to cleanup the code
Reviewed-by: Peter Xu
Signed-off-by: Xiao Guangrong
---
migration/ram.c | 16
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/migration/ram.c b/migration/ram.c
index c3628b020e..e0caf7182b 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -1037,10 +1037,6 @@ static int ram_save_page(RAMState *rs, PageSearchStatus
*pss, bool last_stage)
p = block->host + offset;
trace_ram_save_page(block->idstr, (uint64_t)offset, p);
-if (control_save_page(rs, block, offset, &pages)) {
-return pages;
-}
-
XBZRLE_cache_lock();
pages = save_zero_page(rs, block, offset);
if (pages > 0) {
@@ -1198,10 +1194,6 @@ static int ram_save_compressed_page(RAMState *rs,
PageSearchStatus *pss,
p = block->host + offset;
-if (control_save_page(rs, block, offset, &pages)) {
-return pages;
-}
-
/* When starting the process of a new block, the first page of
* the block should be sent out before other pages in the same
* block, and all the pages in last block should have been sent
@@ -1489,6 +1481,14 @@ err:
static int ram_save_target_page(RAMState *rs, PageSearchStatus *pss,
bool last_stage)
{
+RAMBlock *block = pss->block;
+ram_addr_t offset = pss->page << TARGET_PAGE_BITS;
+int res;
+
+if (control_save_page(rs, block, offset, &res)) {
+return res;
+}
+
/*
* If xbzrle is on, stop using the data compression after first
* round of migration even if compression is enabled. In theory,
--
2.14.3
[Qemu-devel] [PATCH v3 08/10] migration: move calling save_zero_page to the common place
From: Xiao Guangrong
save_zero_page() is always our first approach to try, move it to
the common place before calling ram_save_compressed_page
and ram_save_page
Reviewed-by: Peter Xu
Reviewed-by: Dr. David Alan Gilbert
Signed-off-by: Xiao Guangrong
---
migration/ram.c | 105 +++-
1 file changed, 59 insertions(+), 46 deletions(-)
diff --git a/migration/ram.c b/migration/ram.c
index e0caf7182b..97917542c5 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -1038,15 +1038,8 @@ static int ram_save_page(RAMState *rs, PageSearchStatus
*pss, bool last_stage)
trace_ram_save_page(block->idstr, (uint64_t)offset, p);
XBZRLE_cache_lock();
-pages = save_zero_page(rs, block, offset);
-if (pages > 0) {
-/* Must let xbzrle know, otherwise a previous (now 0'd) cached
- * page would be stale
- */
-xbzrle_cache_zero_page(rs, current_addr);
-ram_release_pages(block->idstr, offset, pages);
-} else if (!rs->ram_bulk_stage &&
- !migration_in_postcopy() && migrate_use_xbzrle()) {
+if (!rs->ram_bulk_stage && !migration_in_postcopy() &&
+migrate_use_xbzrle()) {
pages = save_xbzrle_page(rs, &p, current_addr, block,
offset, last_stage);
if (!last_stage) {
@@ -1194,40 +1187,23 @@ static int ram_save_compressed_page(RAMState *rs,
PageSearchStatus *pss,
p = block->host + offset;
-/* When starting the process of a new block, the first page of
- * the block should be sent out before other pages in the same
- * block, and all the pages in last block should have been sent
- * out, keeping this order is important, because the 'cont' flag
- * is used to avoid resending the block name.
- */
if (block != rs->last_sent_block) {
-flush_compressed_data(rs);
-pages = save_zero_page(rs, block, offset);
-if (pages > 0) {
-ram_release_pages(block->idstr, offset, pages);
-} else {
-/*
- * Make sure the first page is sent out before other pages.
- *
- * we post it as normal page as compression will take much
- * CPU resource.
- */
-ram_counters.transferred += save_page_header(rs, rs->f, block,
-offset | RAM_SAVE_FLAG_PAGE);
-qemu_put_buffer_async(rs->f, p, TARGET_PAGE_SIZE,
- migrate_release_ram() &
- migration_in_postcopy());
-ram_counters.transferred += TARGET_PAGE_SIZE;
-ram_counters.normal++;
-pages = 1;
-}
+/*
+ * Make sure the first page is sent out before other pages.
+ *
+ * we post it as normal page as compression will take much
+ * CPU resource.
+ */
+ram_counters.transferred += save_page_header(rs, rs->f, block,
+offset | RAM_SAVE_FLAG_PAGE);
+qemu_put_buffer_async(rs->f, p, TARGET_PAGE_SIZE,
+ migrate_release_ram() &
+ migration_in_postcopy());
+ram_counters.transferred += TARGET_PAGE_SIZE;
+ram_counters.normal++;
+pages = 1;
} else {
-pages = save_zero_page(rs, block, offset);
-if (pages == -1) {
-pages = compress_page_with_multi_thread(rs, block, offset);
-} else {
-ram_release_pages(block->idstr, offset, pages);
-}
+pages = compress_page_with_multi_thread(rs, block, offset);
}
return pages;
@@ -1469,6 +1445,24 @@ err:
return -1;
}
+static bool save_page_use_compression(RAMState *rs)
+{
+if (!migrate_use_compression()) {
+return false;
+}
+
+/*
+ * If xbzrle is on, stop using the data compression after first
+ * round of migration even if compression is enabled. In theory,
+ * xbzrle can do better than compression.
+ */
+if (rs->ram_bulk_stage || !migrate_use_xbzrle()) {
+return true;
+}
+
+return false;
+}
+
/**
* ram_save_target_page: save one target page
*
@@ -1490,12 +1484,31 @@ static int ram_save_target_page(RAMState *rs,
PageSearchStatus *pss,
}
/*
- * If xbzrle is on, stop using the data compression after first
- * round of migration even if compression is enabled. In theory,
- * xbzrle can do better than compression.
+ * When starting the process of a new block, the first page of
+ * the block should be sent out before other pages in the same
+ * block, and all the pages in last block should have been sent
+ * out, keeping this order is important, because the 'cont' flag
+ * is used to avoid resending the block name.
*/
-if (migrate_use_compression() &&
-(rs->ram_bulk_stage || !migrate_use_xbzrle())) {
+if (
[Qemu-devel] [PATCH v3 04/10] migration: detect compression and decompression errors
From: Xiao Guangrong
Currently the page being compressed is allowed to be updated by
the VM on the source QEMU, correspondingly the destination QEMU
just ignores the decompression error. However, we completely miss
the chance to catch real errors, then the VM is corrupted silently
To make the migration more robuster, we copy the page to a buffer
first to avoid it being written by VM, then detect and handle the
errors of both compression and decompression errors properly
Reviewed-by: Peter Xu
Signed-off-by: Xiao Guangrong
---
migration/qemu-file.c | 4 ++--
migration/ram.c | 56 +++
2 files changed, 41 insertions(+), 19 deletions(-)
diff --git a/migration/qemu-file.c b/migration/qemu-file.c
index bafe3a0c0d..0463f4c321 100644
--- a/migration/qemu-file.c
+++ b/migration/qemu-file.c
@@ -710,9 +710,9 @@ ssize_t qemu_put_compression_data(QEMUFile *f, z_stream
*stream,
blen = qemu_compress_data(stream, f->buf + f->buf_index + sizeof(int32_t),
blen, p, size);
if (blen < 0) {
-error_report("Compress Failed!");
-return 0;
+return -1;
}
+
qemu_put_be32(f, blen);
if (f->ops->writev_buffer) {
add_to_iovec(f, f->buf + f->buf_index, blen, false);
diff --git a/migration/ram.c b/migration/ram.c
index fb24b2f32f..72cb8dfb66 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -269,7 +269,10 @@ struct CompressParam {
QemuCond cond;
RAMBlock *block;
ram_addr_t offset;
+
+/* internally used fields */
z_stream stream;
+uint8_t *originbuf;
};
typedef struct CompressParam CompressParam;
@@ -296,13 +299,14 @@ static QemuCond comp_done_cond;
/* The empty QEMUFileOps will be used by file in CompressParam */
static const QEMUFileOps empty_ops = { };
+static QEMUFile *decomp_file;
static DecompressParam *decomp_param;
static QemuThread *decompress_threads;
static QemuMutex decomp_done_lock;
static QemuCond decomp_done_cond;
static int do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *block,
-ram_addr_t offset);
+ram_addr_t offset, uint8_t *source_buf);
static void *do_data_compress(void *opaque)
{
@@ -318,7 +322,8 @@ static void *do_data_compress(void *opaque)
param->block = NULL;
qemu_mutex_unlock(¶m->mutex);
-do_compress_ram_page(param->file, ¶m->stream, block, offset);
+do_compress_ram_page(param->file, ¶m->stream, block, offset,
+ param->originbuf);
qemu_mutex_lock(&comp_done_lock);
param->done = true;
@@ -370,6 +375,7 @@ static void compress_threads_save_cleanup(void)
qemu_mutex_destroy(&comp_param[i].mutex);
qemu_cond_destroy(&comp_param[i].cond);
deflateEnd(&comp_param[i].stream);
+g_free(comp_param[i].originbuf);
qemu_fclose(comp_param[i].file);
comp_param[i].file = NULL;
}
@@ -394,8 +400,14 @@ static int compress_threads_save_setup(void)
qemu_cond_init(&comp_done_cond);
qemu_mutex_init(&comp_done_lock);
for (i = 0; i < thread_count; i++) {
+comp_param[i].originbuf = g_try_malloc(TARGET_PAGE_SIZE);
+if (!comp_param[i].originbuf) {
+goto exit;
+}
+
if (deflateInit(&comp_param[i].stream,
migrate_compress_level()) != Z_OK) {
+g_free(comp_param[i].originbuf);
goto exit;
}
@@ -1053,7 +1065,7 @@ static int ram_save_page(RAMState *rs, PageSearchStatus
*pss, bool last_stage)
}
static int do_compress_ram_page(QEMUFile *f, z_stream *stream, RAMBlock *block,
-ram_addr_t offset)
+ram_addr_t offset, uint8_t *source_buf)
{
RAMState *rs = ram_state;
int bytes_sent, blen;
@@ -1061,7 +1073,14 @@ static int do_compress_ram_page(QEMUFile *f, z_stream
*stream, RAMBlock *block,
bytes_sent = save_page_header(rs, f, block, offset |
RAM_SAVE_FLAG_COMPRESS_PAGE);
-blen = qemu_put_compression_data(f, stream, p, TARGET_PAGE_SIZE);
+
+/*
+ * copy it to a internal buffer to avoid it being modified by VM
+ * so that we can catch up the error during compression and
+ * decompression
+ */
+memcpy(source_buf, p, TARGET_PAGE_SIZE);
+blen = qemu_put_compression_data(f, stream, source_buf, TARGET_PAGE_SIZE);
if (blen < 0) {
bytes_sent = 0;
qemu_file_set_error(migrate_get_current()->to_dst_file, blen);
@@ -2555,7 +2574,7 @@ static void *do_data_decompress(void *opaque)
DecompressParam *param = opaque;
unsigned long pagesize;
uint8_t *des;
-int len;
+int len, ret;
qemu_mutex_lock(¶m->mutex);
while (!param->quit) {
@@ -2566,13 +2585,13 @@ static void *do_data_decompress(void *opaque)
q
[Qemu-devel] [PATCH v3 06/10] migration: move some code to ram_save_host_page
From: Xiao Guangrong
Move some code from ram_save_target_page() to ram_save_host_page()
to make it be more readable for latter patches that dramatically
clean ram_save_target_page() up
Reviewed-by: Peter Xu
Signed-off-by: Xiao Guangrong
---
migration/ram.c | 43 +++
1 file changed, 19 insertions(+), 24 deletions(-)
diff --git a/migration/ram.c b/migration/ram.c
index 79c7958993..c3628b020e 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -1483,38 +1483,23 @@ err:
* Returns the number of pages written
*
* @rs: current RAM state
- * @ms: current migration state
* @pss: data about the page we want to send
* @last_stage: if we are at the completion stage
*/
static int ram_save_target_page(RAMState *rs, PageSearchStatus *pss,
bool last_stage)
{
-int res = 0;
-
-/* Check the pages is dirty and if it is send it */
-if (migration_bitmap_clear_dirty(rs, pss->block, pss->page)) {
-/*
- * If xbzrle is on, stop using the data compression after first
- * round of migration even if compression is enabled. In theory,
- * xbzrle can do better than compression.
- */
-if (migrate_use_compression() &&
-(rs->ram_bulk_stage || !migrate_use_xbzrle())) {
-res = ram_save_compressed_page(rs, pss, last_stage);
-} else {
-res = ram_save_page(rs, pss, last_stage);
-}
-
-if (res < 0) {
-return res;
-}
-if (pss->block->unsentmap) {
-clear_bit(pss->page, pss->block->unsentmap);
-}
+/*
+ * If xbzrle is on, stop using the data compression after first
+ * round of migration even if compression is enabled. In theory,
+ * xbzrle can do better than compression.
+ */
+if (migrate_use_compression() &&
+(rs->ram_bulk_stage || !migrate_use_xbzrle())) {
+return ram_save_compressed_page(rs, pss, last_stage);
}
-return res;
+return ram_save_page(rs, pss, last_stage);
}
/**
@@ -1543,12 +1528,22 @@ static int ram_save_host_page(RAMState *rs,
PageSearchStatus *pss,
qemu_ram_pagesize(pss->block) >> TARGET_PAGE_BITS;
do {
+/* Check the pages is dirty and if it is send it */
+if (!migration_bitmap_clear_dirty(rs, pss->block, pss->page)) {
+pss->page++;
+continue;
+}
+
tmppages = ram_save_target_page(rs, pss, last_stage);
if (tmppages < 0) {
return tmppages;
}
pages += tmppages;
+if (pss->block->unsentmap) {
+clear_bit(pss->page, pss->block->unsentmap);
+}
+
pss->page++;
} while ((pss->page & (pagesize_bits - 1)) &&
offset_in_ramblock(pss->block, pss->page << TARGET_PAGE_BITS));
--
2.14.3
[Qemu-devel] [PATCH v3 00/10] migration: improve and cleanup compression
From: Xiao Guangrong Changelog in v3: Following changes are from Peter's review: 1) use comp_param[i].file and decomp_param[i].compbuf to indicate if the thread is properly init'd or not 2) save the file which is used by ram loader to the global variable instead it is cached per decompression thread Changelog in v2: Thanks to the review from Dave, Peter, Wei and Jiang Biao, the changes in this version are: 1) include the performance number in the cover letter 2）add some comments to explain how to use z_stream->opaque in the patchset 3) allocate a internal buffer for per thread to store the data to be compressed 4) add a new patch that moves some code to ram_save_host_page() so that 'goto' can be omitted gracefully 5) split the optimization of compression and decompress into two separated patches 6) refine and correct code styles This is the first part of our work to improve compression to make it be more useful in the production. The first patch resolves the problem that the migration thread spends too much CPU resource to compression memory if it jumps to a new block that causes the network is used very deficient. The second patch fixes the performance issue that too many VM-exits happen during live migration if compression is being used, it is caused by huge memory returned to kernel frequently as the memory is allocated and freed for every signal call to compress2() The remaining patches clean the code up dramatically Performance numbers: We have tested it on my desktop, i7-4790 + 16G, by locally live migrate the VM which has 8 vCPUs + 6G memory and the max-bandwidth is limited to 350. During the migration, a workload which has 8 threads repeatedly written total 6G memory in the VM. Before this patchset, its bandwidth is ~25 mbps, after applying, the bandwidth is ~50 mbp. We also collected the perf data for patch 2 and 3 on our production, before the patchset: + 57.88% kqemu [kernel.kallsyms][k] queued_spin_lock_slowpath + 10.55% kqemu [kernel.kallsyms][k] __lock_acquire + 4.83% kqemu [kernel.kallsyms][k] flush_tlb_func_common - 1.16% kqemu [kernel.kallsyms][k] lock_acquire ▒ - lock_acquire ▒ - 15.68% _raw_spin_lock ▒ + 29.42% __schedule ▒ + 29.14% perf_event_context_sched_out ▒ + 23.60% tdp_page_fault ▒ + 10.54% do_anonymous_page ▒ + 2.07% kvm_mmu_notifier_invalidate_range_start ▒ + 1.83% zap_pte_range ▒ + 1.44% kvm_mmu_notifier_invalidate_range_end apply our work: + 51.92% kqemu [kernel.kallsyms][k] queued_spin_lock_slowpath + 14.82% kqemu [kernel.kallsyms][k] __lock_acquire + 1.47% kqemu [kernel.kallsyms][k] mark_lock.clone.0 + 1.46% kqemu [kernel.kallsyms][k] native_sched_clock + 1.31% kqemu [kernel.kallsyms][k] lock_acquire + 1.24% kqemu libc-2.12.so [.] __memset_sse2 - 14.82% kqemu [kernel.kallsyms][k] __lock_acquire ▒ - __lock_acquire ▒ - 99.75% lock_acquire ▒ - 18.38% _raw_spin_lock ▒ + 39.62% tdp_page_fault ▒ + 31.32% __schedule ▒ + 27.53% perf_event_context_sched_out ▒ + 0.58% hrtimer_interrupt We can see the TLB flush and mmu-lock contention have gone. Xiao Guangrong (10): migration: stop compressing page in migration thread migration: stop compression to allocate and free memory frequently migration: stop decompression to allocate and free memory frequently migration: detect compression and decompression errors migration: introduce control_save_page() migration: move some code to ram_save_host_page migration: move calling control_save_page to the common place migration: move calling save_zero_page to the common place migration: introduce save_normal_page() migration: remove ram_save_compressed_page() migration/qemu-file.c | 43 - migration/qemu-file.h | 6 +- migration/ram.c | 482 ++
[Qemu-devel] [PATCH v3 01/10] migration: stop compressing page in migration thread
From: Xiao Guangrong
As compression is a heavy work, do not do it in migration thread,
instead, we post it out as a normal page
Reviewed-by: Wei Wang
Reviewed-by: Peter Xu
Reviewed-by: Dr. David Alan Gilbert
Signed-off-by: Xiao Guangrong
---
migration/ram.c | 32
1 file changed, 16 insertions(+), 16 deletions(-)
diff --git a/migration/ram.c b/migration/ram.c
index 0e90efa092..409c847a76 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -1137,7 +1137,7 @@ static int ram_save_compressed_page(RAMState *rs,
PageSearchStatus *pss,
int pages = -1;
uint64_t bytes_xmit = 0;
uint8_t *p;
-int ret, blen;
+int ret;
RAMBlock *block = pss->block;
ram_addr_t offset = pss->page << TARGET_PAGE_BITS;
@@ -1167,23 +1167,23 @@ static int ram_save_compressed_page(RAMState *rs,
PageSearchStatus *pss,
if (block != rs->last_sent_block) {
flush_compressed_data(rs);
pages = save_zero_page(rs, block, offset);
-if (pages == -1) {
-/* Make sure the first page is sent out before other pages */
-bytes_xmit = save_page_header(rs, rs->f, block, offset |
- RAM_SAVE_FLAG_COMPRESS_PAGE);
-blen = qemu_put_compression_data(rs->f, p, TARGET_PAGE_SIZE,
- migrate_compress_level());
-if (blen > 0) {
-ram_counters.transferred += bytes_xmit + blen;
-ram_counters.normal++;
-pages = 1;
-} else {
-qemu_file_set_error(rs->f, blen);
-error_report("compressed data failed!");
-}
-}
if (pages > 0) {
ram_release_pages(block->idstr, offset, pages);
+} else {
+/*
+ * Make sure the first page is sent out before other pages.
+ *
+ * we post it as normal page as compression will take much
+ * CPU resource.
+ */
+ram_counters.transferred += save_page_header(rs, rs->f, block,
+offset | RAM_SAVE_FLAG_PAGE);
+qemu_put_buffer_async(rs->f, p, TARGET_PAGE_SIZE,
+ migrate_release_ram() &
+ migration_in_postcopy());
+ram_counters.transferred += TARGET_PAGE_SIZE;
+ram_counters.normal++;
+pages = 1;
}
} else {
pages = save_zero_page(rs, block, offset);
--
2.14.3
Re: [Qemu-devel] [PATCH v1 1/1] RISC-V: Workaround for critical mstatus.FS MTTCG bug
Michael Clark writes:
> On Tue, Mar 27, 2018 at 3:17 PM, Philippe Mathieu-Daudé
> wrote:
>
>> Cc'ing Alex and Richard.
>>
>> On 03/27/2018 04:54 PM, Michael Clark wrote:
>> > This change is a workaround for a bug where mstatus.FS
>> > is not correctly reporting dirty when MTTCG and SMP are
>> > enabled which results in the floating point register file
>> > not being saved during context switches. This a critical
>> > bug for RISC-V in QEMU as it results in floating point
>> > register file corruption when running SMP Linux in the
>> > RISC-V 'virt' machine.
>> >
>> > This workaround will return dirty if mstatus.FS is
>> > switched from off to initial or clean. We have checked
>> > the specification and it is legal for an implementation
>> > to return either off, or dirty, if set to initial or clean.
>> >
>> > This workaround will result in unnecessary floating point
>> > save restore. When mstatus.FS is off, floating point
>> > instruction trap to indicate the process is using the FPU.
>> > The OS can then save floating-point state of the previous
>> > process using the FPU and set mstatus.FS to initial or
>> > clean. With this workaround, mstatus.FS will always return
>> > dirty if set to a non-zero value, indicating floating point
>> > save restore is necessary, versus misreporting mstatus.FS
>> > resulting in floating point register file corruption.
>> >
>> > Cc: Palmer Dabbelt
>> > Cc: Sagar Karandikar
>> > Cc: Bastian Koppelmann
>> > Cc: Peter Maydell
>> > Tested-by: Richard W.M. Jones
>> > Signed-off-by: Michael Clark
>> > ---
>> > target/riscv/op_helper.c | 19 +--
>> > 1 file changed, 17 insertions(+), 2 deletions(-)
>> >
>> > diff --git a/target/riscv/op_helper.c b/target/riscv/op_helper.c
>> > index e34715d..7281b98 100644
>> > --- a/target/riscv/op_helper.c
>> > +++ b/target/riscv/op_helper.c
>> > @@ -144,8 +144,23 @@ void csr_write_helper(CPURISCVState *env,
>> target_ulong val_to_write,
>> > }
>> >
>> > mstatus = (mstatus & ~mask) | (val_to_write & mask);
>> > -int dirty = (mstatus & MSTATUS_FS) == MSTATUS_FS;
>> > -dirty |= (mstatus & MSTATUS_XS) == MSTATUS_XS;
>> > +
>> > +/* Note: this is a workaround for an issue where mstatus.FS
>> > + does not report dirty when SMP and MTTCG is enabled. This
>> > + workaround is technically compliant with the RISC-V
>> Privileged
>> > + specification as it is legal to return only off, or dirty,
>> > + however this may cause unnecessary saves of floating point
>> state.
>> > + Without this workaround, floating point state is not saved
>> and
>> > + restored correctly when SMP and MTTCG is enabled, */
>>
>
> On looking at this again, I think we may need to remove the
> qemu_tcg_mttcg_enabled conditional and always return dirty if the state is
> initial or clean, but not off.
>
> While testing on uniprocessor worked okay, it's likely because we were
> lucky and there was no task migration or multiple FPU tasks working. This
> would mean we would revert to Richard W.M. Jones initial patch.
>
>> +if (qemu_tcg_mttcg_enabled()) {
>> > +/* FP is always dirty or off */
>> > +if (mstatus & MSTATUS_FS) {
>> > +mstatus |= MSTATUS_FS;
>> > +}
>> > +}
I'm confused. If mstatus is a per-vCPU variable why does the enabling or
not of MTTCG matter here?
>> > +
>> > +int dirty = ((mstatus & MSTATUS_FS) == MSTATUS_FS) |
>> > +((mstatus & MSTATUS_XS) == MSTATUS_XS);
>> > mstatus = set_field(mstatus, MSTATUS_SD, dirty);
>> > env->mstatus = mstatus;
>> > break;
>> >
>>
>
> The text from the specification that allows us to always return dirty if
> set to initial or clean, is below i.e. Dirty implies state has
> "potentially" been modified, so that gives us wriggle room.
>
> "
> When an extension's status is set to Off , any instruction that attempts to
> read or write the corresponding
> state will cause an exception. When the status is Initial, the
> corresponding state should
> have an initial constant value. When the status is Clean, the corresponding
> state is potentially
> di fferent from the initial value, but matches the last value stored on a
> context swap. When the
> status is Dirty, the corresponding state has potentially been modif ed
> since the last context save.
> "
>
> I think the problem is Linux is setting the state to clean after saving
> fpu register state [1], but we have no code in the QEMU FPU operations to
> set the state to dirty, if is clean or initial, only code to cause an
> exception if the floating point extension state is set to off e.g.
>
> static void gen_fp_store(DisasContext *ctx, uint32_t opc, int rs1,
> int rs2, target_long imm)
> {
> TCGv t0;
>
> if (!(ctx->flags & TB_FLAGS_FP_ENABLE)) {
> gen_exception_illegal(ctx);
> return;
> }
>
> t0 = tcg_temp_ne
