Re: [PATCH v2 3/3] migration: Support responsive CPU throttle

2024-10-01 Thread Yong Huang
On Tue, Oct 1, 2024 at 4:47 AM Peter Xu wrote: > On Mon, Sep 30, 2024 at 01:14:28AM +0800, yong.hu...@smartx.com wrote: > > From: Hyman Huang > > > > Currently, the convergence algorithm determines that the migration > > cannot converge according to the following principle: > > The dirty pages g

Re: [PATCH v2 1/3] migration: Support background ramblock dirty sync

2024-09-30 Thread Yong Huang
On Tue, Oct 1, 2024 at 4:41 AM Peter Xu wrote: > On Mon, Sep 30, 2024 at 01:14:26AM +0800, yong.hu...@smartx.com wrote: > > From: Hyman Huang > > > > When VM is configured with huge memory, the current throttle logic > > doesn't look like to scale, because migration_trigger_throttle() > > is onl

[PATCH v2 2/3] qapi/migration: Introduce cpu-throttle-responsive parameter

2024-09-29 Thread yong . huang
From: Hyman Huang To enable the responsive throttle that will be implemented in the next commit, introduce the cpu-responsive-throttle parameter. Signed-off-by: Hyman Huang --- migration/migration-hmp-cmds.c | 8 migration/options.c| 20 migration/opt

[PATCH v2 3/3] migration: Support responsive CPU throttle

2024-09-29 Thread yong . huang
From: Hyman Huang Currently, the convergence algorithm determines that the migration cannot converge according to the following principle: The dirty pages generated in current iteration exceed a specific percentage (throttle-trigger-threshold, 50 by default) of the number of transmissions. Let's

[PATCH v2 0/3] migration: auto-converge refinements for huge VM

2024-09-29 Thread yong . huang
From: Hyman Huang v2: 1. background sync - Throw out the idea of "not updating the bitmap" when the RAMBlock of the RAM list is iterated during migration; re-implement the background RAM dirty sync using Peter's updated method. 2. responsive throttle - Rename the "cpu-responsive throttl

[PATCH v2 1/3] migration: Support background ramblock dirty sync

2024-09-29 Thread yong . huang
From: Hyman Huang When VM is configured with huge memory, the current throttle logic doesn't look like to scale, because migration_trigger_throttle() is only called for each iteration, so it won't be invoked for a long time if one iteration can take a long time. The background dirty sync aim to

Re: [PATCH v1 1/7] migration: Introduce structs for background sync

2024-09-27 Thread Yong Huang
On Fri, Sep 27, 2024 at 11:35 PM Peter Xu wrote: > On Fri, Sep 27, 2024 at 10:50:01AM +0800, Yong Huang wrote: > > On Fri, Sep 27, 2024 at 3:55 AM Peter Xu wrote: > > > > > On Fri, Sep 27, 2024 at 02:13:47AM +0800, Yong Huang wrote: > > > > On Thu, Sep

Re: [PATCH v1 1/7] migration: Introduce structs for background sync

2024-09-26 Thread Yong Huang
On Fri, Sep 27, 2024 at 3:55 AM Peter Xu wrote: > On Fri, Sep 27, 2024 at 02:13:47AM +0800, Yong Huang wrote: > > On Thu, Sep 26, 2024 at 3:17 AM Peter Xu wrote: > > > > > On Fri, Sep 20, 2024 at 10:43:31AM +0800, Yong Huang wrote: > > > > Yes, invok

Re: [PATCH v1 1/7] migration: Introduce structs for background sync

2024-09-26 Thread Yong Huang
On Thu, Sep 26, 2024 at 3:17 AM Peter Xu wrote: > On Fri, Sep 20, 2024 at 10:43:31AM +0800, Yong Huang wrote: > > Yes, invoke migration_bitmap_sync_precopy more frequently is also my > > first idea but it involves bitmap updating and interfere with the > behavior > >

Re: [PATCH v1 1/7] migration: Introduce structs for background sync

2024-09-19 Thread Yong Huang
On Fri, Sep 20, 2024 at 2:45 AM Peter Xu wrote: > On Tue, Sep 17, 2024 at 02:48:03PM +0800, Yong Huang wrote: > > On Tue, Sep 17, 2024 at 5:11 AM Fabiano Rosas wrote: > > > > > Hyman Huang writes: > > > > > > > shadow_bmap, iter_bmap and iter_dirt

Re: [PATCH v1 1/7] migration: Introduce structs for background sync

2024-09-19 Thread Yong Huang
On Fri, Sep 20, 2024 at 2:45 AM Peter Xu wrote: > On Tue, Sep 17, 2024 at 02:48:03PM +0800, Yong Huang wrote: > > On Tue, Sep 17, 2024 at 5:11 AM Fabiano Rosas wrote: > > > > > Hyman Huang writes: > > > > > > > shadow_bmap, iter_bmap and iter_dirt

Re: [PATCH v1 1/7] migration: Introduce structs for background sync

2024-09-19 Thread Yong Huang
On Fri, Sep 20, 2024 at 2:45 AM Peter Xu wrote: > On Tue, Sep 17, 2024 at 02:48:03PM +0800, Yong Huang wrote: > > On Tue, Sep 17, 2024 at 5:11 AM Fabiano Rosas wrote: > > > > > Hyman Huang writes: > > > > > > > shadow_bmap, iter_bmap and iter_dirt

Re: [PATCH v1 3/7] qapi/migration: Introduce the iteration-count

2024-09-18 Thread Yong Huang
On Tue, Sep 17, 2024 at 4:35 AM Fabiano Rosas wrote: > Hyman Huang writes: > > > The original migration information dirty-sync-count could > > no longer reflect iteration count due to the introduction > > of background synchronization in the next commit; > > add the iteration count to compensate

Re: [PATCH v1 6/7] qapi/migration: Introduce cpu-responsive-throttle parameter

2024-09-16 Thread Yong Huang
On Tue, Sep 17, 2024 at 4:55 AM Fabiano Rosas wrote: > Hyman Huang writes: > > > To enable the responsive throttle that will be implemented > > in the next commit, introduce the cpu-responsive-throttle > > parameter. > > > > Signed-off-by: Hyman Huang > > --- > > migration/migration-hmp-cmds.c

Re: [PATCH v1 3/7] qapi/migration: Introduce the iteration-count

2024-09-16 Thread Yong Huang
On Tue, Sep 17, 2024 at 4:35 AM Fabiano Rosas wrote: > Hyman Huang writes: > > > The original migration information dirty-sync-count could > > no longer reflect iteration count due to the introduction > > of background synchronization in the next commit; > > add the iteration count to compensate

Re: [PATCH v1 1/7] migration: Introduce structs for background sync

2024-09-16 Thread Yong Huang
On Tue, Sep 17, 2024 at 5:11 AM Fabiano Rosas wrote: > Hyman Huang writes: > > > shadow_bmap, iter_bmap and iter_dirty_pages are introduced > > to satisfy the need for background sync. > > > > Meanwhile, introduce enumeration of sync method. > > > > Signed-off-by: Hyman Huang > > --- > > inclu

Re: [PATCH RESEND RFC 08/10] migration: Introduce cpu-responsive-throttle parameter

2024-09-09 Thread Yong Huang
In order to reduce the maintenance work of QMP. Given that the focus of this patchset is huge VM migration, is it possible to enable or disable these two features with a single parameter, such as "cpu-aggressive-throttle"? Yong On Mon, Sep 9, 2024 at 10:26 PM Hyman Huang wrote: > To enable the

Re: [PATCH RESEND RFC 03/10] qapi/migration: Introduce periodic CPU throttling parameters

2024-09-09 Thread Yong Huang
On Tue, Sep 10, 2024 at 5:30 AM Peter Xu wrote: > On Mon, Sep 09, 2024 at 10:25:36PM +0800, Hyman Huang wrote: > > To activate the periodic CPU throttleing feature, introduce > > the cpu-periodic-throttle. > > > > To control the frequency of throttling, introduce the > > cpu-periodic-throttle-int

Re: [PATCH RFC 10/10] tests/migration-tests: Add test case for responsive CPU throttle

2024-09-09 Thread Yong Huang
On Mon, Sep 9, 2024 at 10:03 PM Peter Maydell wrote: > On Mon, 9 Sept 2024 at 14:51, Hyman Huang wrote: > > > > Despite the fact that the responsive CPU throttle is enabled, > > the dirty sync count may not always increase because this is > > an optimization that might not happen in any situatio

Re: [PATCH v1 2/5] tests/migration: Make initrd-stress.img built by default

2024-08-09 Thread Yong Huang
On Fri, Aug 9, 2024 at 4:34 PM Daniel P. Berrangé wrote: > On Fri, Aug 09, 2024 at 03:43:33PM +0800, yong.hu...@smartx.com wrote: > > From: Hyman Huang > > > > The initrd-stress.img was compiled by specifying the > > target, to make it easier for developers to play the > > guestperf tool, make i

[PATCH v1 5/5] guestperf: Introduce multifd compression option

2024-08-09 Thread yong . huang
From: Hyman Huang Guestperf tool does not cover the multifd compression option currently, it is worth supporting so that developers can analysis the migration performance with different compression algorithms. Multifd support 4 compression algorithms currently: zlib, zstd, qpl, uadk To request

[PATCH v1 3/5] guestperf: Support deferred migration for multifd

2024-08-09 Thread yong . huang
From: Hyman Huang The way to enable multifd migration has been changed by commit, 82137e6c8c (migration: enforce multifd and postcopy preempt to be set before incoming), and guestperf has not made the necessary changes. If multifd migration had been enabled in the previous manner, the following e

[PATCH v1 1/5] tests/migration: Move the guestperf tool to scripts directory

2024-08-09 Thread yong . huang
From: Hyman Huang Guestperf was designed to test the performance of migration, with a loose connection to the fundamental test cases of QEMU. To improve the repository's structure, move it to the scripts directory. Signed-off-by: Hyman Huang --- MAINTAINERS

[PATCH v1 2/5] tests/migration: Make initrd-stress.img built by default

2024-08-09 Thread yong . huang
From: Hyman Huang The initrd-stress.img was compiled by specifying the target, to make it easier for developers to play the guestperf tool, make it built by default. Signed-off-by: Hyman Huang --- tests/migration/meson.build | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --gi

[PATCH v1 4/5] guestperf: Nitpick the inconsistent parameters

2024-08-09 Thread yong . huang
From: Hyman Huang Signed-off-by: Hyman Huang Reviewed-by: Fabiano Rosas --- scripts/migration/guestperf/comparison.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/migration/guestperf/comparison.py b/scripts/migration/guestperf/comparison.py index 42cc0372d1..40e

[PATCH v1 0/5] Guestperf: miscellaneous refinement and enrichment

2024-08-09 Thread yong . huang
From: Hyman Huang The previous patchset: https://lore.kernel.org/qemu-devel/cover.1722957352.git.yong.hu...@smartx.com/ does not made the necessary changes and tests for the upstream version. This patchset works for that: 1. Move the guestperf to scripts directory suggested by Fabiano Rosas 2. M

Re: [PATCH] migration: Refine the convergence detection

2024-08-07 Thread Yong Huang
On Wed, Aug 7, 2024 at 8:59 PM Peter Xu wrote: > On Wed, Aug 07, 2024 at 02:46:29PM +0800, Yong Huang wrote: > > I still think dropping the dirty_rate_high_cnt is a refinement. If > migration > > has a precise detection of convergence, what do you think of it? > > Maybe;

Re: [PATCH] migration: Refine the convergence detection

2024-08-06 Thread Yong Huang
On Wed, Aug 7, 2024 at 12:45 AM Peter Xu wrote: > On Mon, Aug 05, 2024 at 03:03:27PM +0800, Yong Huang wrote: > > Sorry for the late reply. > > > > On Wed, Jul 31, 2024 at 4:01 AM Peter Xu wrote: > > > > > On Wed, Jul 24, 2024 at 07:39:29PM +0800, Hyman

Re: [PATCH 2/2] tests/migration: Introduce multifd compression into guestperf

2024-08-06 Thread Yong Huang
On Wed, Aug 7, 2024 at 3:22 AM Fabiano Rosas wrote: > yong.hu...@smartx.com writes: > > > From: Hyman Huang > > > > Guestperf tool does not cover the multifd compression option > > currently, it is worth supporting so that developers can > > analysis the migration performance with different > >

[PATCH 2/2] tests/migration: Introduce multifd compression into guestperf

2024-08-06 Thread yong . huang
From: Hyman Huang Guestperf tool does not cover the multifd compression option currently, it is worth supporting so that developers can analysis the migration performance with different compression algorithms. Multifd support 4 compression algorithms currently: zlib, zstd, qpl, uadk To request

[PATCH 0/2] Support multifd compression test for guestperf

2024-08-06 Thread yong . huang
From: Hyman Huang The first patch just nitpick the inconsistent parameters for multifd test. The second patch support multifd compression test. Please review, thanks, Yong Hyman Huang (2): tests/migration: Nitpick the inconsistent parameters tests/migration: Introduce multifd compression

[PATCH 1/2] tests/migration: Nitpick the inconsistent parameters

2024-08-06 Thread yong . huang
From: Hyman Huang Signed-off-by: Hyman Huang --- tests/migration/guestperf/comparison.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/migration/guestperf/comparison.py b/tests/migration/guestperf/comparison.py index 42cc0372d1..40e9d2eb1d 100644 --- a/tests/migrati

Re: [PATCH] migration: Refine the convergence detection

2024-08-05 Thread Yong Huang
Sorry for the late reply. On Wed, Jul 31, 2024 at 4:01 AM Peter Xu wrote: > On Wed, Jul 24, 2024 at 07:39:29PM +0800, Hyman Huang wrote: > > Currently, the convergence algorithm determines that the migration > > cannot converge according to the following principle: > > The dirty pages generated

Re: [PATCH] e1000: Fix the unexpected assumption that the receive buffer is full

2024-07-16 Thread Yong Huang
On Wed, Jul 17, 2024 at 9:29 AM Jason Wang wrote: > On Wed, Jul 17, 2024 at 9:24 AM Yong Huang wrote: > > > > > > > > On Fri, Jul 12, 2024 at 10:01 AM Jason Wang wrote: > >> > >> On Wed, Jul 10, 2024 at 5:05 PM Yong Huang > wrote: > >&g

Re: [PATCH] e1000: Fix the unexpected assumption that the receive buffer is full

2024-07-16 Thread Yong Huang
On Fri, Jul 12, 2024 at 10:01 AM Jason Wang wrote: > On Wed, Jul 10, 2024 at 5:05 PM Yong Huang wrote: > > > > > > > > On Wed, Jul 10, 2024 at 3:36 PM Jason Wang wrote: > >> > >> On Wed, Jul 10, 2024 at 2:26 PM Yong Huang > wrote: > >>

Re: [PATCH] e1000: Fix the unexpected assumption that the receive buffer is full

2024-07-10 Thread Yong Huang
On Wed, Jul 10, 2024 at 3:36 PM Jason Wang wrote: > On Wed, Jul 10, 2024 at 2:26 PM Yong Huang wrote: > > > > > > > > On Wed, Jul 10, 2024 at 11:44 AM Jason Wang wrote: > >> > >> On Tue, Jul 9, 2024 at 10:56 AM Yong Huang > wrote: > >&g

Re: [PATCH] e1000: Fix the unexpected assumption that the receive buffer is full

2024-07-09 Thread Yong Huang
On Wed, Jul 10, 2024 at 11:44 AM Jason Wang wrote: > On Tue, Jul 9, 2024 at 10:56 AM Yong Huang wrote: > > > > > > > > On Tue, Jul 9, 2024 at 10:41 AM Jason Wang wrote: > >> > >> On Mon, Jul 8, 2024 at 1:17 PM Yong Huang > wrote: > >&g

Re: [PATCH] docs/devel: Add introduction to LUKS volume with detached header

2024-07-09 Thread Yong Huang
On Mon, May 6, 2024 at 2:26 PM Markus Armbruster wrote: > Looks like this fell through the cracks. Is anyone familiar with LUKS > willing to review it? > :) Ping? > > Hyman Huang writes: > > > Signed-off-by: Hyman Huang > > --- > > MAINTAINERS | 1 + > > docs/devel

[PATCH v2 0/1] Fix the overrun check of the receive ring buffer

2024-07-08 Thread yong . huang
From: Hyman Huang The version 2 fix the corner case in e1000_has_rxbufs: Handle the scenario of RDH==RDT while total_size > s->rxbuf_size. Hyman Huang (1): e1000: Fix the overrun check of the receive ring buffer hw/net/e1000.c | 16 ++-- 1 file changed, 14 insertions(+), 2 deleti

[PATCH v2 1/1] e1000: Fix the overrun check of the receive ring buffer

2024-07-08 Thread yong . huang
From: Hyman Huang Unexpected work by certain Windows guests equipped with the e1000 interface can cause the network to go down and never come back up again unless the guest's interface is reset. To reproduce the failure: 1. Set up two guests with a Windows 2016 or 2019 server operating system

Re: [PATCH] e1000: Fix the unexpected assumption that the receive buffer is full

2024-07-08 Thread Yong Huang
On Tue, Jul 9, 2024 at 10:41 AM Jason Wang wrote: > On Mon, Jul 8, 2024 at 1:17 PM Yong Huang wrote: > > > > > > > > On Mon, Jul 8, 2024 at 11:21 AM Jason Wang wrote: > >> > >> On Sat, Jul 6, 2024 at 4:30 AM Hyman Huang > wrote: > >> &g

Re: [PATCH] e1000: Fix the unexpected assumption that the receive buffer is full

2024-07-07 Thread Yong Huang
On Sat, Jul 6, 2024 at 4:30 AM Hyman Huang wrote: > Unexpected work by certain Windows guests equipped with the e1000 > interface can cause the network to go down and never come back up > again unless the guest's interface is reset. > > To reproduce the failure: > 1. Set up two guests with a Wind

Re: [PATCH] e1000: Fix the unexpected assumption that the receive buffer is full

2024-07-07 Thread Yong Huang
On Mon, Jul 8, 2024 at 11:21 AM Jason Wang wrote: > On Sat, Jul 6, 2024 at 4:30 AM Hyman Huang wrote: > > > > Unexpected work by certain Windows guests equipped with the e1000 > > interface can cause the network to go down and never come back up > > again unless the guest's interface is reset. >

Re: [PATCH] hw: Fix crash that happens when introspecting scsi-block on older machine types

2024-07-03 Thread Yong Huang
On Wed, Jul 3, 2024 at 5:09 PM Thomas Huth wrote: > "make check SPEED=slow" is currently failing the device-introspect-test on > older machine types since introspecting "scsi-block" is causing an abort: > > $ ./qemu-system-x86_64 -M pc-q35-8.0 -monitor stdio > QEMU 9.0.50 monitor - type 'help'

Re: [PATCH v2] scsi-disk: Fix crash for VM configured with USB CDROM after live migration

2024-06-24 Thread Yong Huang
On Mon, Jun 24, 2024 at 6:06 PM Thomas Huth wrote: > On 10/06/2024 19.02, Paolo Bonzini wrote: > > From: Hyman Huang > > > > For VMs configured with the USB CDROM device: > > > > -drive > file=/path/to/local/file,id=drive-usb-disk0,media=cdrom,readonly=on... > > -device usb-storage,drive=drive-u

Re: [PATCH 2/2] scsi-disk: Fix crash for VM configured with USB CDROM after live migration

2024-05-24 Thread Yong Huang
On Fri, May 24, 2024 at 6:01 PM Prasad Pandit wrote: > Hello Hyman, > > * Is this the same patch series as sent before..? > -> > https://lists.nongnu.org/archive/html/qemu-devel/2024-04/msg00816.html Yes, exactly the same, I just refine the comment > > > On Fri, 24 May 2024 at 12:02, Hyman H

Re: [PATCH RESEND 0/2] Fix crash of VMs configured with the CDROM device

2024-05-08 Thread Yong Huang
Ping2 Make sure that this patchset is not overlooked. Yong On Tue, Apr 23, 2024 at 9:45 AM Yong Huang wrote: > Ping. > I would appreciate comments on this series. Thanks, > > Yong > > On Mon, Apr 8, 2024 at 8:08 PM Hyman Huang wrote: > >> This patchset fixes the c

Re: [PATCH v2 1/1] migration/dirtyrate: Fix segmentation fault

2024-04-28 Thread Yong Huang
On Wed, Apr 24, 2024 at 3:17 PM Zhijian Li (Fujitsu) wrote: > > > On 24/04/2024 12:52, mii wrote: > > > > On 2024/04/24 10:28, Yong Huang wrote: > >> > >> > >> On Tue, Apr 23, 2024 at 9:35 PM Peter Xu wrote: > >> > >&g

Re: [PATCH v2 1/1] migration/dirtyrate: Fix segmentation fault

2024-04-23 Thread Yong Huang
On Tue, Apr 23, 2024 at 9:35 PM Peter Xu wrote: > On Tue, Apr 23, 2024 at 09:13:08AM +, Masato Imai wrote: > > When the KVM acceleration parameter is not set, executing calc_dirty_rate > > with the -r or -b option results in a segmentation fault due to accessing > > a null kvm_state pointer i

Re: [PATCH] migration/dirtyrate: Fix segmentation fault

2024-04-23 Thread Yong Huang
On Tue, Apr 23, 2024 at 2:19 PM Masato Imai wrote: > When the KVM acceleration parameter is not set, executing calc_dirty_rate > with the -r option results in a segmentation fault due to accessing a > null kvm_state pointer in the kvm_dirty_rate_enabled function. > s/kvm_dirty_rate_enabled/kvm_d

Re: [PATCH RESEND 0/2] Fix crash of VMs configured with the CDROM device

2024-04-22 Thread Yong Huang
Ping. I would appreciate comments on this series. Thanks, Yong On Mon, Apr 8, 2024 at 8:08 PM Hyman Huang wrote: > This patchset fixes the crash of VMs configured with the CDROM device > on the destination during live migration. See the commit message for > details. > > The previous patchset do

Re: [PATCH RESEND 2/2] scsi-disk: Fix crash of VMs configured with the CDROM device

2024-04-14 Thread Yong Huang
Ping On Mon, Apr 8, 2024 at 8:08 PM Hyman Huang wrote: > When configuring VMs with the CDROM device using the USB bus > in Libvirt, do as follows: > > > > > > > > > > > The destination Qemu process crashed, causing the VM migration > to fail; the backtrace reveals the following

Re: [PATCH 12/19] migration: fix -Werror=maybe-uninitialized false-positives

2024-03-28 Thread Yong Huang
On Thu, Mar 28, 2024 at 6:23 PM wrote: > From: Marc-André Lureau > > ../migration/dirtyrate.c:186:5: error: ‘records’ may be used uninitialized > [-Werror=maybe-uninitialized] > ../migration/dirtyrate.c:168:12: error: ‘gen_id’ may be used uninitialized > [-Werror=maybe-uninitialized] > ../migrat

Re: [PATCH for-9.1 v5 11/14] memory: Add Error** argument to the global_dirty_log routines

2024-03-21 Thread Yong Huang
On Wed, Mar 20, 2024 at 11:19 PM Peter Xu wrote: > On Wed, Mar 20, 2024 at 07:49:07AM +0100, Cédric Le Goater wrote: > > Now that the log_global*() handlers take an Error** parameter and > > return a bool, do the same for memory_global_dirty_log_start() and > > memory_global_dirty_log_stop(). The

Re: [PATCH v4 14/25] memory: Add Error** argument to the global_dirty_log routines

2024-03-15 Thread Yong Huang
On Wed, Mar 6, 2024 at 9:35 PM Cédric Le Goater wrote: > Now that the log_global*() handlers take an Error** parameter and > return a bool, do the same for memory_global_dirty_log_start() and > memory_global_dirty_log_stop(). The error is reported in the callers > for now and it will be propagate

Re: [PATCH v4 0/3] Adjust the output of x-query-virtio-status

2024-03-15 Thread Yong Huang
On Fri, Mar 15, 2024 at 3:51 PM Markus Armbruster wrote: > "Michael S. Tsirkin" writes: > > > On Wed, Mar 13, 2024 at 10:40:21AM +0100, Markus Armbruster wrote: > >> I could be awkward for the use case described in PATCH 1's commit > >> message: > >> > >> However, we sometimes want to compar

Re: [PATCH v3 15/26] memory: Add Error** argument to the global_dirty_log routines

2024-03-04 Thread Yong Huang
On Mon, Mar 4, 2024 at 8:29 PM Cédric Le Goater wrote: > Now that the log_global*() handlers take an Error** parameter and > return a bool, do the same for memory_global_dirty_log_start() and > memory_global_dirty_log_stop(). The error is reported in the callers > for now and it will be propagate

Re: [PATCH RFC 0/3] Support GM/T 0018-2012 cryptographic standard

2024-02-29 Thread Yong Huang
On Thu, Feb 29, 2024 at 5:04 PM Daniel P. Berrangé wrote: > On Sat, Feb 24, 2024 at 10:34:55PM +0800, Hyman Huang wrote: > > This patchset introduce GM/T 0018-2012 as a crypto backend driver, > > which is applied for block encryption. Currently, we support SM4 > > cipher algorithm only. > > > > G

Re: [PATCH] qapi: Craft the BlockdevCreateOptionsLUKS comment

2024-02-28 Thread Yong Huang
On Wed, Feb 28, 2024 at 7:58 PM Kevin Wolf wrote: > Am 28.02.2024 um 12:30 hat Daniel P. Berrangé geschrieben: > > On Wed, Feb 28, 2024 at 11:17:37AM +0100, Kevin Wolf wrote: > > > Until now, @size has been mandatory for creating images with every > > > driver. Maybe we should even have put it in

Re: [PATCH] qapi: Craft the BlockdevCreateOptionsLUKS comment

2024-02-21 Thread Yong Huang
On Wed, Feb 21, 2024 at 4:26 PM Markus Armbruster wrote: > Yong Huang writes: > > > On Wed, Feb 21, 2024 at 2:43 PM Markus Armbruster > wrote: > > > >> Hyman Huang writes: > >> > >> > Add comment in detail for commit 433957bb

Re: [PATCH] qapi: Craft the BlockdevCreateOptionsLUKS comment

2024-02-20 Thread Yong Huang
On Wed, Feb 21, 2024 at 2:43 PM Markus Armbruster wrote: > Hyman Huang writes: > > > Add comment in detail for commit 433957bb7f (qapi: > > Make parameter 'file' optional for > > BlockdevCreateOptionsLUKS). > > > > Signed-off-by: Hyman Huang > > --- > > qapi/block-core.json | 20 ++

Re: [PATCH v3 2/3] virtio: Declare the decoding functions to static

2024-02-20 Thread Yong Huang
On Mon, Feb 19, 2024 at 11:38 PM Markus Armbruster wrote: > Hyman Huang writes: > > > qmp_decode_protocols(), qmp_decode_status(), and qmp_decode_features() > > are now only used in virtio-hmp-cmds.c. So move them into there, > > redeclare them to static, and replace the qmp_ prefix with hmp_.

Re: [PATCH v4 2/7] qapi: Make parameter 'file' optional for BlockdevCreateOptionsLUKS

2024-02-20 Thread Yong Huang
On Tue, Feb 20, 2024 at 5:47 PM Markus Armbruster wrote: > Yong Huang writes: > > > On Tue, Feb 20, 2024 at 4:56 PM Markus Armbruster > wrote: > > > >> Yong Huang writes: > >> > >> > On Tue, Feb 20, 2024 at 2:31 PM Markus Armbruster >

Re: [PATCH v4 2/7] qapi: Make parameter 'file' optional for BlockdevCreateOptionsLUKS

2024-02-20 Thread Yong Huang
On Tue, Feb 20, 2024 at 4:56 PM Markus Armbruster wrote: > Yong Huang writes: > > > On Tue, Feb 20, 2024 at 2:31 PM Markus Armbruster > wrote: > > > >> yong.hu...@smartx.com writes: > >> > >> > From: Hyman Huang > >> > > >

Re: [PATCH v4 2/7] qapi: Make parameter 'file' optional for BlockdevCreateOptionsLUKS

2024-02-19 Thread Yong Huang
On Tue, Feb 20, 2024 at 2:31 PM Markus Armbruster wrote: > yong.hu...@smartx.com writes: > > > From: Hyman Huang > > > > To support detached LUKS header creation, make the existing 'file' > > field in BlockdevCreateOptionsLUKS optional. > > > > Signed-off-by: Hyman Huang > > Reviewed-by: Daniel

Re: [PATCH v3 1/3] qmp: Switch x-query-virtio-status back to numeric encoding

2024-02-19 Thread Yong Huang
On Mon, Feb 19, 2024 at 11:32 PM Markus Armbruster wrote: > Yong Huang writes: > > > On Tue, Feb 13, 2024 at 6:26 PM Michael S. Tsirkin > wrote: > > > >> On Fri, Feb 02, 2024 at 10:32:15PM +0800, Hyman Huang wrote: > >> > x-query-virtio-status

Re: [PATCH v3 1/3] qmp: Switch x-query-virtio-status back to numeric encoding

2024-02-15 Thread Yong Huang
On Tue, Feb 13, 2024 at 6:26 PM Michael S. Tsirkin wrote: > On Fri, Feb 02, 2024 at 10:32:15PM +0800, Hyman Huang wrote: > > x-query-virtio-status returns several sets of virtio feature and > > status flags. It goes back to v7.2.0. > > > > In the initial commit 90c066cd682 (qmp: add QMP command

[PATCH v4 3/7] crypto: Modify the qcrypto_block_create to support creation flags

2024-01-29 Thread yong . huang
From: Hyman Huang Expand the signature of qcrypto_block_create to enable the formation of LUKS volumes with detachable headers. To accomplish that, introduce QCryptoBlockCreateFlags to instruct the creation process to set the payload_offset_sector to 0. Signed-off-by: Hyman Huang --- block/cry

[PATCH v4 0/7] Support generic Luks encryption

2024-01-29 Thread yong . huang
From: Hyman Huang Sorry for the late post of version 4. The modifications are as follows: v4: - Rebase on master - squash [PATCH v3 02/10] to [PATCH v3 01/10] - refactor the logic of block_crypto_open_generic in [PATCH v3 02/10] as Daniel suggestted: a. drop the invalid parameter check for

[PATCH v4 6/7] crypto: Introduce 'detached-header' field in QCryptoBlockInfoLUKS

2024-01-29 Thread yong . huang
From: Hyman Huang When querying the LUKS disk with the qemu-img tool or other APIs, add information about whether the LUKS header is detached. Additionally, update the test case with the appropriate modification. Signed-off-by: Hyman Huang --- crypto/block-luks.c| 2 ++ qapi/crypto.js

[PATCH v4 1/7] crypto: Support LUKS volume with detached header

2024-01-29 Thread yong . huang
From: Hyman Huang By enhancing the LUKS driver, it is possible to implement the LUKS volume with a detached header. Normally a LUKS volume has a layout: disk: | header | key material | disk payload data | With a detached LUKS header, you need 2 disks so getting: disk1: | header | key mate

[PATCH v4 7/7] tests: Add case for LUKS volume with detached header

2024-01-29 Thread yong . huang
From: Hyman Huang Also, add a section to the MAINTAINERS file for detached LUKS header, it only has a test case in it currently. Signed-off-by: Hyman Huang --- MAINTAINERS | 5 + tests/qemu-iotests/tests/luks-detached-header | 218 ++ .../tes

[PATCH v4 4/7] block: Support detached LUKS header creation using blockdev-create

2024-01-29 Thread yong . huang
From: Hyman Huang Firstly, enable the ability to choose the block device containing a detachable LUKS header by adding the 'header' parameter to BlockdevCreateOptionsLUKS. Secondly, when formatting the LUKS volume with a detachable header, truncate the payload volume to length without a header s

[PATCH v4 2/7] qapi: Make parameter 'file' optional for BlockdevCreateOptionsLUKS

2024-01-29 Thread yong . huang
From: Hyman Huang To support detached LUKS header creation, make the existing 'file' field in BlockdevCreateOptionsLUKS optional. Signed-off-by: Hyman Huang Reviewed-by: Daniel P. Berrangé --- block/crypto.c | 21 ++--- qapi/block-core.json | 5 +++-- 2 files changed, 1

[PATCH v4 5/7] block: Support detached LUKS header creation using qemu-img

2024-01-29 Thread yong . huang
From: Hyman Huang Even though a LUKS header might be created with cryptsetup, qemu-img should be enhanced to accommodate it as well. Add the 'detached-header' option to specify the creation of a detached LUKS header. This is how it is used: $ qemu-img create --object secret,id=sec0,data=abc123 -

Re: [PATCH v2 0/2] Nitpick at the error message's output

2024-01-14 Thread Yong Huang
Ping On Sun, Jan 7, 2024 at 7:53 PM Hyman Huang wrote: > v2: > - rebase on master > - add a commit to sort the error message so that an explanation > error number can be returned on all failure paths > > Hyman Huang (2): > i386/sev: Sort the error message > i386/sev: Nitpick at the error m

Re: [PATCH v2 1/2] qapi/virtio: Keep feature and status bits in the QMP output

2024-01-12 Thread Yong Huang
On Fri, Jan 12, 2024 at 10:10 PM Markus Armbruster wrote: > Hyman Huang writes: > > > Maintain the feature and status bits in the x-query-virtio-status > > output and, as usual, add human-readable output only in HMP. > > > > Applications may find it useful to compare features and status > > info

Re: [PATCH v2 2/2] hmp: Drop unknown feature and status bits

2024-01-12 Thread Yong Huang
On Fri, Jan 12, 2024 at 10:04 PM Markus Armbruster wrote: > Hyman Huang writes: > > > The QMP command "x-query-virtio-status" outputs the full > > feature and status bit information, so there is no need > > to maintain it in the HMP output; drop it. > > > > Signed-off-by: Hyman Huang > > I'm no

Re: [PATCH RESEND v3 01/10] crypto: Introduce option and structure for detached LUKS header

2024-01-11 Thread Yong Huang
On Thu, Jan 11, 2024 at 10:58 PM Daniel P. Berrangé wrote: > On Thu, Jan 11, 2024 at 03:35:10PM +0100, Markus Armbruster wrote: > > Hyman Huang writes: > > > > > Add the "header" option for the LUKS format. This field would be > > > used to identify the blockdev's position where a detachable LUK

Re: [PATCH RESEND v3 06/10] block: Support detached LUKS header creation using blockdev-create

2024-01-11 Thread Yong Huang
On Thu, Jan 11, 2024 at 10:05 PM Markus Armbruster wrote: > Fails to compile for me: > > ../block/crypto.c: In function ‘block_crypto_co_create_luks’: > ../block/crypto.c:784:1: error: control reaches end of non-void function > [-Werror=return-type] > 784 | } > | ^ > > Ok, I'll check it o

Re: [PATCH RESEND v3 04/10] crypto: Introduce creation option and structure for detached LUKS header

2024-01-07 Thread Yong Huang
On Thu, Jan 4, 2024 at 10:51 PM Daniel P. Berrangé wrote: > On Mon, Dec 25, 2023 at 01:45:06PM +0800, Hyman Huang wrote: > > Introduce 'header' field in BlockdevCreateOptionsLUKS to support > > detached LUKS header creation. Meanwhile, introduce header-related > > field in QCryptoBlock. > > > > S

Re: [PATCH RESEND v3 02/10] crypto: Support generic LUKS encryption

2024-01-07 Thread Yong Huang
On Thu, Jan 4, 2024 at 10:40 PM Daniel P. Berrangé wrote: > On Mon, Dec 25, 2023 at 01:45:04PM +0800, Hyman Huang wrote: > > By enhancing the LUKS driver, it is possible to enable > > the detachable LUKS header and, as a result, achieve > > general encryption for any disk format that QEMU has > >

Re: [PATCH] i386/sev: Nitpick at the error message's output

2024-01-07 Thread Yong Huang
On Sat, Jan 6, 2024 at 12:43 AM Alex Bennée wrote: > Hyman Huang writes: > > > The incorrect error message was produced as a result of > > the return number being disregarded on the sev_kvm_init > > failure path. > > > > For instance, when a user's failure to launch a SEV guest > > is caused by

Re: [PATCH 1/2] qapi/virtio: Keep feature and status bits in the QMP output

2024-01-02 Thread Yong Huang
Thanks, I'll fix that in the next version. On Fri, Dec 29, 2023 at 5:31 PM Philippe Mathieu-Daudé wrote: > Hi, > > On 28/12/23 19:52, Hyman Huang wrote: > > Maintain the feature and status bits in the x-query-virtio-status > > output and, as usual, add human-readable output only in HMP. > > > >

Re: [v2 1/2] qapi/virtio: Add feature and status bits for x-query-virtio-status

2023-12-22 Thread Yong Huang
Sure. of course. I'll do that next week. While v3 would be a single patch that only contains the first commit. Thanks, Yong On Fri, Dec 22, 2023 at 5:54 PM Markus Armbruster wrote: > Hyman Huang writes: > > > This patch allows to display feature and status bits in > > virtio-status. > > > > Ap

Re: [v2 4/4] block: Support detached LUKS header creation for blockdev-create

2023-12-18 Thread Yong Huang
On Mon, Dec 18, 2023 at 7:19 PM Daniel P. Berrangé wrote: > On Thu, Dec 07, 2023 at 12:37:45AM +0800, Hyman Huang wrote: > > Provide the "detached-mode" option for detached LUKS header > > formatting. > > > > To format the LUKS header on the pre-creating disk, example > > as follows: > > > > 1. a

Re: [v2 3/4] crypto: Support generic LUKS encryption

2023-12-18 Thread Yong Huang
On Mon, Dec 18, 2023 at 7:16 PM Daniel P. Berrangé wrote: > On Thu, Dec 07, 2023 at 12:37:44AM +0800, Hyman Huang wrote: > > By enhancing the LUKS driver, it is possible to enable > > the detachable LUKS header and, as a result, achieve > > general encryption for any disk format that QEMU has > >

Re: [v2 0/4] Support generic Luks encryption

2023-12-18 Thread Yong Huang
On Mon, Dec 18, 2023 at 7:21 PM Daniel P. Berrangé wrote: > On Thu, Dec 07, 2023 at 12:37:41AM +0800, Hyman Huang wrote: > > v2: > > - Simplify the design by reusing the LUKS driver to implement > > the generic Luks encryption, thank Daniel for the insightful > > advice. > > - rebase on maste

Re: [RFC 1/2] qapi/virtio: introduce the "show-bits" argument for x-query-virtio-status

2023-12-08 Thread Yong Huang
On Fri, Dec 1, 2023 at 11:37 PM Laurent Vivier wrote: > On 12/1/23 16:21, Markus Armbruster wrote: > > Laurent Vivier writes: > > > >> On 11/21/23 08:58, Markus Armbruster wrote: > >>> Laurent, there's a question for you at the end. > >>>

Re: [PATCH v6] crypto: Introduce SM4 symmetric cipher algorithm

2023-12-07 Thread Yong Huang
I'll post the changelog between versions in the next version if needed. :) v6: - modify the comment about how to enable sm4 crypto feature v5: - drop the crypto-sm4 option in build system v4 - refactor the error report info, change "sm4"->"libsm4" On Fri, Dec 8, 2023 at 12:02 AM Philippe Mathieu-

Re: [PATCH v3] crypto: Introduce SM4 symmetric cipher algorithm

2023-12-06 Thread Yong Huang
On Thu, Dec 7, 2023 at 1:41 AM Daniel P. Berrangé wrote: > On Wed, Nov 29, 2023 at 11:17:49PM +0800, Hyman Huang wrote: > > Introduce the SM4 cipher algorithms (OSCCA GB/T 32907-2016). > > > > SM4 (GBT.32907-2016) is a cryptographic standard issued by the > > Organization of State Commercial Admi

Re: [RFC 0/8] Support generic Luks encryption

2023-12-04 Thread Yong Huang
On Tue, Dec 5, 2023 at 1:44 AM Daniel P. Berrangé wrote: > On Tue, Dec 05, 2023 at 01:32:51AM +0800, Yong Huang wrote: > > On Tue, Dec 5, 2023 at 12:51 AM Daniel P. Berrangé > > wrote: > > > > > On Tue, Dec 05, 2023 at 12:41:16AM +0800, Yong Huang wrote: > >

Re: [RFC 0/8] Support generic Luks encryption

2023-12-04 Thread Yong Huang
On Tue, Dec 5, 2023 at 12:51 AM Daniel P. Berrangé wrote: > On Tue, Dec 05, 2023 at 12:41:16AM +0800, Yong Huang wrote: > > On Tue, Dec 5, 2023 at 12:24 AM Daniel P. Berrangé > > wrote: > > > > > On Tue, Dec 05, 2023 at 12:06:17AM +0800, Hyman Huang wrote:

Re: [RFC 0/8] Support generic Luks encryption

2023-12-04 Thread Yong Huang
On Tue, Dec 5, 2023 at 12:24 AM Daniel P. Berrangé wrote: > On Tue, Dec 05, 2023 at 12:06:17AM +0800, Hyman Huang wrote: > > This functionality was motivated by the following to-do list seen > > in crypto documents: > > https://wiki.qemu.org/Features/Block/Crypto > > > > The last chapter says we

Re: [RFC 0/8] Support generic Luks encryption

2023-12-04 Thread Yong Huang
On Tue, Dec 5, 2023 at 12:24 AM Daniel P. Berrangé wrote: > On Tue, Dec 05, 2023 at 12:06:17AM +0800, Hyman Huang wrote: > > This functionality was motivated by the following to-do list seen > > in crypto documents: > > https://wiki.qemu.org/Features/Block/Crypto > > > > The last chapter says we

Re: [PATCH] migration/dirtyrate: Remove an extra parameter

2023-12-03 Thread Yong Huang
On Mon, Dec 4, 2023 at 9:23 AM Wafer wrote: > vcpu_dirty_stat_collect() has an unused parameter so remove it. > > Signed-off-by: Wafer > --- > migration/dirtyrate.c | 7 +++ > 1 file changed, 3 insertions(+), 4 deletions(-) > > diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c > in

Re: [PATCH v2] crypto: Introduce SM4 symmetric cipher algorithm

2023-11-28 Thread Yong Huang
On Tue, Nov 28, 2023 at 11:57 PM Philippe Mathieu-Daudé wrote: > Hi Hyman, > > On 28/11/23 16:24, Hyman Huang wrote: > > Introduce the SM4 cipher algorithms (OSCCA GB/T 32907-2016). > > > > SM4 (GBT.32907-2016) is a cryptographic standard issued by the > > Organization of State Commercial Adminis

Re: [PATCH v2] crypto: Introduce SM4 symmetric cipher algorithm

2023-11-28 Thread Yong Huang
I'll try to understand the comment, if i misunderstood, please point out. On Wed, Nov 29, 2023 at 12:20 AM Daniel P. Berrangé wrote: > On Tue, Nov 28, 2023 at 04:57:20PM +0100, Philippe Mathieu-Daudé wrote: > > Hi Hyman, > > > > On 28/11/23 16:24, Hyman Huang wrote: > > > Introduce the SM4 ciphe

Re: [PATCH] crypto: Introduce SM4 symmetric cipher algorithm

2023-11-27 Thread Yong Huang
On Tue, Nov 28, 2023 at 12:11 AM Daniel P. Berrangé wrote: > On Mon, Nov 27, 2023 at 11:55:34PM +0800, Hyman Huang wrote: > > Introduce the SM4 cipher algorithms (OSCCA GB/T 32907-2016). > > > > SM4 (GBT.32907-2016) is a cryptographic standard issued by the > > Organization of State Commercial Ad

Re: [RFC 1/2] qapi/virtio: introduce the "show-bits" argument for x-query-virtio-status

2023-11-21 Thread Yong Huang
On Tue, Nov 21, 2023 at 3:58 PM Markus Armbruster wrote: > Laurent, there's a question for you at the end. > > Yong Huang writes: > > > On Thu, Nov 16, 2023 at 10:44 PM Markus Armbruster > > wrote: > > > >> Hyman Huang writes: > >> > &

  1   2   >