[PATCH v3] i386/cpu: fixup number of addressable IDs for processor cores in the physical package

2024-06-10 Thread Chuang Xu
. This is safe because cores_per_pkg will not be 0 and will be at least 1. Fixes: d7caf13b5fcf ("x86: cpu: fixup number of addressable IDs for logical processors sharing cache") Signed-off-by: Guixiong Wei Signed-off-by: Yipeng Yin Signed-off-by: Chuang Xu --- target/i386/cpu.c | 6 ++--

[PATCH v2] i386/cpu: fixup number of addressable IDs for processor cores in the physical package

2024-06-03 Thread Chuang Xu
condition. Fixes: d7caf13b5fcf742e5680c1d3448ba070fc811644 ("x86: cpu: fixup number of addressable IDs for logical processors sharing cache") Signed-off-by: Guixiong Wei Signed-off-by: Yipeng Yin Signed-off-by: Chuang Xu --- target/i386/cpu.c | 6 ++ 1 file changed, 2 insert

Re: [PATCH] x86: cpu: fixup number of addressable IDs for processor cores in the physical package

2024-05-27 Thread Chuang Xu
Hi Zhao, On 2024/5/28 上午10:31, Zhao Liu wrote: Hi Chuang, On Mon, May 27, 2024 at 11:13:33AM +0800, Chuang Xu wrote: Date: Mon, 27 May 2024 11:13:33 +0800 From: Chuang Xu Subject: [PATCH] x86: cpu: fixup number of addressable IDs for processor cores in the physical package According

[PATCH] x86: cpu: fixup number of addressable IDs for processor cores in the physical package

2024-05-26 Thread Chuang Xu
d7caf13b5fcf742e5680c1d3448ba070fc811644. Fix it by changing the judgement condition to a >= 1. Signed-off-by: Chuang Xu Signed-off-by: Guixiong Wei Signed-off-by: Yipeng Yin --- target/i386/cpu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/target/i386/cpu.c b/target/i386/cpu.c index cd16cb8

Re: [PATCH v2 04/11] multifd: Count the number of bytes sent correctly

2023-06-16 Thread Chuang Xu
Hi,Juan, On 2023/1/30 下午4:09, Juan Quintela wrote: > Current code asumes that all pages are whole. That is not true for > example for compression already. Fix it for creating a new field > ->sent_bytes that includes it. > > All ram_counters are used only from the migration thread, so we have >

Re: [PATCH v8 0/6] migration: reduce time of loading non-iterable vmstate

2023-06-16 Thread Chuang Xu
Hi, Paolo, A few months ago, Juan told me that this series requires your or someone familiar with memory API's feedback. Could you please review it and provide some suggestions? On 2023/3/17 下午4:18, Chuang Xu wrote: In this version: - delete useless line change. - update comments

[PATCH v8 3/6] memory: Introduce memory_region_transaction_do_commit()

2023-03-17 Thread Chuang Xu
Split memory_region_transaction_do_commit() from memory_region_transaction_commit(). We'll call do_commit() in address_space_to_flatview() in the later patch. Signed-off-by: Chuang Xu --- softmmu/memory.c | 47 +++ 1 file changed, 27 insertions

[PATCH v8 0/6] migration: reduce time of loading non-iterable vmstate

2023-03-17 Thread Chuang Xu
In this version: - delete useless line change. - update comments and commit messages. The duration of loading non-iterable vmstate accounts for a significant portion of downtime (starting with the timestamp of source qemu stop and ending with the timestamp of target qemu start). Most of the

[PATCH v8 2/6] rcu: Introduce rcu_read_is_locked()

2023-03-17 Thread Chuang Xu
Add rcu_read_is_locked() to detect holding of rcu lock. Signed-off-by: Chuang Xu --- include/qemu/rcu.h | 7 +++ 1 file changed, 7 insertions(+) diff --git a/include/qemu/rcu.h b/include/qemu/rcu.h index 313fc414bc..7bf45602e1 100644 --- a/include/qemu/rcu.h +++ b/include/qemu/rcu.h

[PATCH v8 4/6] memory: Add do_commit() and sanity check in address_space_to_flatview

2023-03-17 Thread Chuang Xu
address_space_to_flatview will return the newest flatview and it should logically only be triggered in a few spots during vm_load. Other than that, sanity check whether BQL or rcu is held before using any flatview. Signed-off-by: Chuang Xu --- include/exec/memory.h | 23 +++ softmmu

[PATCH v8 1/6] memory: Reference as->current_map directly in memory commit

2023-03-17 Thread Chuang Xu
From: Peter Xu Calling RCU variants of address_space_get|to_flatview() during memory commit (flatview updates, triggering memory listeners, or updating ioeventfds, etc.) is not 100% accurate, because commit() requires BQL rather than RCU read lock, so the context exclusively owns current_map and

[PATCH v8 6/6] memory: Introduce address_space_to_flatview_rcu()

2023-03-17 Thread Chuang Xu
In last patch, we wrap vm_load with begin/commit, here we introduce address_space_to_flatview_rcu() to avoid unnecessary enforce commit during vm_load. Signed-off-by: Chuang Xu --- include/exec/memory.h | 17 + softmmu/memory.c | 2 +- 2 files changed, 18 insertions(+), 1

[PATCH v8 5/6] migration: Reduce time of loading non-iterable vmstate

2023-03-17 Thread Chuang Xu
and the number of devices have a great impact on the time of loading non-iterable vmstate. The growth of the number of devices and queues will lead to more mr commits, and the time consumption caused by the flatview reconstruction will also increase. Signed-off-by: Chuang Xu --- migration/savevm.c | 18

Re: [PATCH v7 6/6] memory: Introduce address_space_to_flatview_rcu()

2023-03-13 Thread Chuang Xu
Hi, Peter, On 2023/3/10 下午11:08, Peter Xu wrote: On Fri, Mar 10, 2023 at 10:24:25AM +0800, Chuang Xu wrote: In last patch, we wrap vm_load with begin/commit, here we introduce address_space_to_flatview_rcu() to avoid unnecessary enforce commit during vm_load. Signed-off-by: Chuang Xu

Re: [PATCH v7 3/6] memory: Introduce memory_region_transaction_do_commit()

2023-03-12 Thread Chuang Xu
Hi, Peter, On 2023/3/10 下午10:51, Peter Xu wrote: On Fri, Mar 10, 2023 at 10:24:22AM +0800, Chuang Xu wrote: Split memory_region_transaction_do_commit() from memory_region_transaction_commit(). We'll call do_commit() in address_space_to_flatview() in the later patch. Signed-off-by: Chuang Xu

[PATCH v7 6/6] memory: Introduce address_space_to_flatview_rcu()

2023-03-09 Thread Chuang Xu
In last patch, we wrap vm_load with begin/commit, here we introduce address_space_to_flatview_rcu() to avoid unnecessary enforce commit during vm_load. Signed-off-by: Chuang Xu --- include/exec/memory-internal.h | 2 +- include/exec/memory.h | 20 softmmu/memory.c

[PATCH v7 4/6] memory: Add sanity check in address_space_to_flatview

2023-03-09 Thread Chuang Xu
Before using any flatview, sanity check whether BQL or rcu is held. And if we're during a memory region transaction, try to immediately update mappings, or the map can be invalid. Signed-off-by: Chuang Xu --- include/exec/memory.h | 23 +++ softmmu/memory.c | 5

[PATCH v7 2/6] rcu: Introduce rcu_read_is_locked()

2023-03-09 Thread Chuang Xu
Add rcu_read_is_locked() to detect holding of rcu lock. Signed-off-by: Chuang Xu --- include/qemu/rcu.h | 7 +++ 1 file changed, 7 insertions(+) diff --git a/include/qemu/rcu.h b/include/qemu/rcu.h index 313fc414bc..7bf45602e1 100644 --- a/include/qemu/rcu.h +++ b/include/qemu/rcu.h

[PATCH v7 3/6] memory: Introduce memory_region_transaction_do_commit()

2023-03-09 Thread Chuang Xu
Split memory_region_transaction_do_commit() from memory_region_transaction_commit(). We'll call do_commit() in address_space_to_flatview() in the later patch. Signed-off-by: Chuang Xu --- softmmu/memory.c | 47 +++ 1 file changed, 27 insertions

[PATCH v7 1/6] memory: Reference as->current_map directly in memory commit

2023-03-09 Thread Chuang Xu
From: Peter Xu Calling RCU variance of address_space_get|to_flatview() during memory commit (flatview updates, triggering memory listeners, or updating ioeventfds, etc.) is not 100% accurate, because commit() requires BQL rather than RCU read lock, so the context exclusively owns current_map and

[PATCH v7 5/6] migration: Reduce time of loading non-iterable vmstate

2023-03-09 Thread Chuang Xu
and the number of devices have a great impact on the time of loading non-iterable vmstate. The growth of the number of devices and queues will lead to more mr commits, and the time consumption caused by the flatview reconstruction will also increase. Signed-off-by: Chuang Xu --- migration/savevm.c | 19

[PATCH v7 0/6] migration: reduce time of loading non-iterable vmstate

2023-03-09 Thread Chuang Xu
In this version: - introduce address_space_to_flatview_rcu() - squash peter's fix into patch 1 - rebase to latest upstream - update test results The duration of loading non-iterable vmstate accounts for a significant portion of downtime (starting with the timestamp of source qemu stop and

Re: [PATCH RESEND v6 0/5] migration: reduce time of loading non-iterable vmstate

2023-03-09 Thread Chuang Xu
Hi, Peter, On 2023/3/8 下午11:46, Peter Xu wrote: 1. squash fix into patch1 of yours. 2. introduce address_space_to_flatview_rcu() 3. add specific comment to define when to use which as_to_flat() This can be together with 2). We should suggest using address_space_to_flatview() by default in the

Re: [PATCH RESEND v6 0/5] migration: reduce time of loading non-iterable vmstate

2023-03-08 Thread Chuang Xu
Hi, Peter, On 2023/3/8 下午10:58, Peter Xu wrote: On Wed, Mar 08, 2023 at 06:03:45AM -0800, Chuang Xu wrote: IIUC, Do you mean that different ways to get flatview are tricky? Yes, and properly define when to use which. As you said, it's slightly beyond what this series does. Maybe it would

Re: [PATCH RESEND v6 0/5] migration: reduce time of loading non-iterable vmstate

2023-03-08 Thread Chuang Xu
Hi, Peter, On 2023/3/8 上午1:04, Peter Xu wrote: > On Tue, Mar 07, 2023 at 09:24:31PM +0800, Chuang Xu wrote: >>> Why do we need address_space_get_flatview_rcu()? I'm not sure whether you >> address_space_cahce_init() uses address_space_get_flatview() to acquire >> a ref

Re: [PATCH RESEND v6 0/5] migration: reduce time of loading non-iterable vmstate

2023-03-07 Thread Chuang Xu
Hi, Peter, On 2023/3/7 上午4:48, Peter Xu wrote: On Mon, Mar 06, 2023 at 08:48:05PM +0800, Chuang Xu wrote: Hi, Peter, On 2023/3/6 上午6:05, Peter Xu wrote: 1.virtio_load->virtio_init_region_cache 2.virtio_load->virtio_set_features_nocheck What is this one specifically? I failed to see q

Re: [PATCH RESEND v6 0/5] migration: reduce time of loading non-iterable vmstate

2023-03-06 Thread Chuang Xu
Hi, Peter, On 2023/3/6 上午6:05, Peter Xu wrote: Hi, Chuang, On Fri, Mar 03, 2023 at 06:56:50PM +0800, Chuang Xu wrote: Sorry to forget to update the test results in the last patch of v6. In this version: - add peter's patch. - split mr_do_commit() from mr_commit(). - adjust the sanity check

[PATCH RESEND v6 5/5] migration: Reduce time of loading non-iterable vmstate

2023-03-03 Thread Chuang Xu
to more mr commits, and the time consumption caused by the flatview reconstruction will also increase. Signed-off-by: Chuang Xu --- migration/savevm.c | 19 +++ 1 file changed, 19 insertions(+) diff --git a/migration/savevm.c b/migration/savevm.c index b5e6962bb6..3dd9daabd8 100644

[PATCH RESEND v6 3/5] memory: Introduce memory_region_transaction_do_commit()

2023-03-03 Thread Chuang Xu
Split memory_region_transaction_do_commit() from memory_region_transaction_commit(). We'll call do_commit() in address_space_to_flatview() in the later patch. Signed-off-by: Chuang Xu --- softmmu/memory.c | 47 +++ 1 file changed, 27 insertions

[PATCH RESEND v6 1/5] memory: Reference as->current_map directly in memory commit

2023-03-03 Thread Chuang Xu
From: Peter Xu Calling RCU variance of address_space_get|to_flatview() during memory commit (flatview updates, triggering memory listeners, or updating ioeventfds, etc.) is not 100% accurate, because commit() requires BQL rather than RCU read lock, so the context exclusively owns current_map and

[PATCH RESEND v6 0/5] migration: reduce time of loading non-iterable vmstate

2023-03-03 Thread Chuang Xu
Sorry to forget to update the test results in the last patch of v6. In this version: - add peter's patch. - split mr_do_commit() from mr_commit(). - adjust the sanity check in address_space_to_flatview(). - rebase to latest upstream. - replace 8260 with 8362 as testing host. - update the

[PATCH RESEND v6 2/5] rcu: Introduce rcu_read_is_locked()

2023-03-03 Thread Chuang Xu
Add rcu_read_is_locked() to detect holding of rcu lock. Signed-off-by: Chuang Xu --- include/qemu/rcu.h | 7 +++ 1 file changed, 7 insertions(+) diff --git a/include/qemu/rcu.h b/include/qemu/rcu.h index b063c6fde8..719916d9d3 100644 --- a/include/qemu/rcu.h +++ b/include/qemu/rcu.h

[PATCH RESEND v6 4/5] memory: Add sanity check in address_space_to_flatview

2023-03-03 Thread Chuang Xu
Before using any flatview, sanity check whether BQL or rcu is held. And if we're during a memory region transaction, try to immediately update mappings, or the map can be invalid. Signed-off-by: Chuang Xu --- include/exec/memory.h | 23 +++ softmmu/memory.c | 5

[PATCH v6 1/5] memory: Reference as->current_map directly in memory commit

2023-03-03 Thread Chuang Xu
From: Peter Xu Calling RCU variance of address_space_get|to_flatview() during memory commit (flatview updates, triggering memory listeners, or updating ioeventfds, etc.) is not 100% accurate, because commit() requires BQL rather than RCU read lock, so the context exclusively owns current_map and

[PATCH v6 2/5] rcu: Introduce rcu_read_is_locked()

2023-03-03 Thread Chuang Xu
Add rcu_read_is_locked() to detect holding of rcu lock. Signed-off-by: Chuang Xu --- include/qemu/rcu.h | 7 +++ 1 file changed, 7 insertions(+) diff --git a/include/qemu/rcu.h b/include/qemu/rcu.h index b063c6fde8..719916d9d3 100644 --- a/include/qemu/rcu.h +++ b/include/qemu/rcu.h

[PATCH v6 0/5] migration: reduce time of loading non-iterable vmstate

2023-03-03 Thread Chuang Xu
In this version: - add peter's patch. - split mr_do_commit() from mr_commit(). - adjust the sanity check in address_space_to_flatview(). - rebase to latest upstream. - replace 8260 with 8362 as testing host. - update the latest test results. Here I list some cases which will trigger

[PATCH v6 5/5] migration: Reduce time of loading non-iterable vmstate

2023-03-03 Thread Chuang Xu
commits, and the time consumption caused by the flatview reconstruction will also increase. Signed-off-by: Chuang Xu --- migration/savevm.c | 19 +++ 1 file changed, 19 insertions(+) diff --git a/migration/savevm.c b/migration/savevm.c index b5e6962bb6..3dd9daabd8 100644

[PATCH v6 4/5] memory: Add sanity check in address_space_to_flatview

2023-03-03 Thread Chuang Xu
Before using any flatview, sanity check whether BQL or rcu is held. And if we're during a memory region transaction, try to immediately update mappings, or the map can be invalid. Signed-off-by: Chuang Xu --- include/exec/memory.h | 23 +++ softmmu/memory.c | 5

[PATCH v6 3/5] memory: Introduce memory_region_transaction_do_commit()

2023-03-03 Thread Chuang Xu
Split memory_region_transaction_do_commit() from memory_region_transaction_commit(). We'll call do_commit() in address_space_to_flatview() in the later patch. Signed-off-by: Chuang Xu --- softmmu/memory.c | 47 +++ 1 file changed, 27 insertions

Re: [RFC v5 0/3] migration: reduce time of loading non-iterable vmstate

2023-02-27 Thread Chuang Xu
Hi, Peter On 2023/2/25 下午11:32, Peter Xu wrote: On Thu, Feb 23, 2023 at 11:28:46AM +0800, Chuang Xu wrote: Hi, Peter Hi, Chuang, On 2023/2/22 下午11:57, Peter Xu wrote: I don't see why it's wrong, and that's exactly what I wanted to guarantee, that if memory_region_update_pending==true when

Re: [RFC v5 0/3] migration: reduce time of loading non-iterable vmstate

2023-02-22 Thread Chuang Xu
Hi, Peter On 2023/2/22 下午11:57, Peter Xu wrote: On Wed, Feb 22, 2023 at 02:27:55PM +0800, Chuang Xu wrote: Hi, Peter Hi, Chuang, Note that as I mentioned in the comment, we temporarily replace this value to prevent commit() and address_space_to_flatview() call each other recursively

Re: [RFC v5 0/3] migration: reduce time of loading non-iterable vmstate

2023-02-21 Thread Chuang Xu
Hi, Peter On 2023/2/22 上午4:36, Peter Xu wrote: On 2023/2/21 上午11:38, Chuang Xu wrote: I think we need a memory_region_transaction_commit_force() to force commit some transactions when load vmstate. This function is designed like this: /*  * memory_region_transaction_commit_force

Re: [RFC v5 0/3] migration: reduce time of loading non-iterable vmstate

2023-02-21 Thread Chuang Xu
Hi, Peter This email is a supplement to the previous one. On 2023/2/21 上午11:38, Chuang Xu wrote: I think we need a memory_region_transaction_commit_force() to force commit some transactions when load vmstate. This function is designed like this: /*  * memory_region_transaction_commit_force

Re: [RFC v5 0/3] migration: reduce time of loading non-iterable vmstate

2023-02-20 Thread Chuang Xu
Hi, Peter It seems that there is a problem with the code format in my last email. I adjusted the format and resend this to you. Hope the format of this email won't be wrong again..  :) On 2023/2/17 下午11:52, Peter Xu wrote: Hello, Chuang, On Fri, Feb 17, 2023 at 04:11:19PM +0800, Chuang Xu

Re: [RFC v5 0/3] migration: reduce time of loading non-iterable vmstate

2023-02-20 Thread Chuang Xu
Hi, Peter On 2023/2/17 下午11:52, Peter Xu wrote: Hello, Chuang, On Fri, Feb 17, 2023 at 04:11:19PM +0800, Chuang Xu wrote: Error 1 was triggered by our sanity check. I try to add RCU_READ_LOCK_GUARD() in address_space_init() and it works. But I'm not sure if this code change is appropriate

Re: [RFC v5 0/3] migration: reduce time of loading non-iterable vmstate

2023-02-20 Thread Chuang Xu
Hi, Juan On 2023/2/16 上午3:10, Juan Quintela wrote: Chuang Xu wrote: In this version: Hi I had to drop this. It breaks migration of dbus-vmstate. .[K144/179 qemu:qtest+qtest-x86_64 / qtest-x86_64/virtio-net-failover ERROR 5.66s killed by signal 6 SIGABRT

Re: [RFC v5 0/3] migration: reduce time of loading non-iterable vmstate

2023-02-17 Thread Chuang Xu
Hi, Peter! In my last email to Juan, I mentioned two errors. Now I want to discuss them with you. On 2023/2/16 下午11:41, Chuang Xu wrote: I ran qtest with reference to your environment, and finally reported two errors. Error 1(the same as yours):  QTEST_QEMU_BINARY=./qemu-system-x86_64

Re: [RFC v5 0/3] migration: reduce time of loading non-iterable vmstate

2023-02-16 Thread Chuang Xu
Hi, Juan Thanks for your test results! On 2023/2/16 上午3:10, Juan Quintela wrote: > Chuang Xu wrote: >> In this version: > Hi > > I had to drop this. It breaks migration of dbus-vmstate. Previously, I only focused on the precopy migration test in the normal scenario, but did n

Re: [RFC v5 1/3] rcu: introduce rcu_read_is_locked()

2023-02-14 Thread Chuang Xu
Hi, Paolo! On 2023/2/2 下午6:59, Juan Quintela wrote: Chuang Xu wrote: add rcu_read_is_locked() to detect holding of rcu lock. Signed-off-by: Chuang Xu Reviewed-by: Juan Quintela Althought I think that petting a review from Paolo or anyone that knows more than RCU could be a good idea

[RFC v5 0/3] migration: reduce time of loading non-iterable vmstate

2023-01-17 Thread Chuang Xu
In this version: - rename rcu_read_locked() to rcu_read_is_locked(). - adjust the sanity check in address_space_to_flatview(). - improve some comments. The duration of loading non-iterable vmstate accounts for a significant portion of downtime (starting with the timestamp of source qemu stop and

[RFC v5 3/3] migration: reduce time of loading non-iterable vmstate

2023-01-17 Thread Chuang Xu
commits, and the time consumption caused by the flatview reconstruction will also increase. Signed-off-by: Chuang Xu --- migration/savevm.c | 18 ++ 1 file changed, 18 insertions(+) diff --git a/migration/savevm.c b/migration/savevm.c index a0cdb714f7..8ca6d396f4 100644

[RFC v5 1/3] rcu: introduce rcu_read_is_locked()

2023-01-17 Thread Chuang Xu
add rcu_read_is_locked() to detect holding of rcu lock. Signed-off-by: Chuang Xu --- include/qemu/rcu.h | 7 +++ 1 file changed, 7 insertions(+) diff --git a/include/qemu/rcu.h b/include/qemu/rcu.h index b063c6fde8..719916d9d3 100644 --- a/include/qemu/rcu.h +++ b/include/qemu/rcu.h

[RFC v5 2/3] memory: add depth assert in address_space_to_flatview

2023-01-17 Thread Chuang Xu
Before using any flatview, sanity check we're not during a memory region transaction or the map can be invalid. Signed-off-by: Chuang Xu --- include/exec/memory.h | 15 +++ softmmu/memory.c | 5 + 2 files changed, 20 insertions(+) diff --git a/include/exec/memory.h b

Re: [RFC v4 2/3] memory: add depth assert in address_space_to_flatview

2023-01-13 Thread Chuang Xu
Hi, Peter, On 2023/1/12 下午11:13, Peter Xu wrote: We wanted to capture outliers when you apply the follow up patch to vm load procedure. That will make depth>0 for the whole process of vm load during migration, and we wanted to make sure it's safe, hence this patch, right? In my perspective,

Re: [RFC v4 2/3] memory: add depth assert in address_space_to_flatview

2023-01-12 Thread Chuang Xu
Hi, Peter, Paolo, On 2023/1/10 下午10:45, Peter Xu wrote: On Tue, Jan 10, 2023 at 12:09:41AM -0800, Chuang Xu wrote: Hi, Peter and Paolo, Hi, Chuang, Paolo, I'm sorry I didn't reply to your email in time. I was infected with COVID-19 two weeks ago, so I couldn't think about the problems

Re: [RFC v4 2/3] memory: add depth assert in address_space_to_flatview

2023-01-10 Thread Chuang Xu
Hi, Peter and Paolo, I'm sorry I didn't reply to your email in time. I was infected with COVID-19 two weeks ago, so I couldn't think about the problems discussed in your email for a long time. On 2023/1/4 上午1:43, Peter Xu wrote: > Hi, Paolo, > > On Wed, Dec 28, 2022 at 09:27:50AM +0100, Paolo

Re: [RFC v4 1/3] rcu: introduce rcu_read_locked()

2023-01-05 Thread Chuang Xu
On 2023/1/4 下午10:20, Alex Bennée wrote: > Chuang Xu writes: > >> add rcu_read_locked() to detect holding of rcu lock. >> >> Signed-off-by: Chuang Xu >> --- >> include/qemu/rcu.h | 7 +++ >> 1 file changed, 7 insertions(+) >> >> diff --

Re: [External] Re: [RFC v4 2/3] memory: add depth assert in address_space_to_flatview

2023-01-03 Thread Chuang Xu
On 2022/12/28 下午6:50, Philippe Mathieu-Daudé wrote: On 23/12/22 15:23, Chuang Xu wrote: Before using any flatview, sanity check we're not during a memory region transaction or the map can be invalid. Signed-off-by: Chuang Xu --- include/exec/memory.h | 9 + softmmu/memory.c

Re: [RFC v4 3/3] migration: reduce time of loading non-iterable vmstate

2023-01-03 Thread Chuang Xu
On 2022/12/24 上午12:06, David Hildenbrand wrote: On 23.12.22 15:23, Chuang Xu wrote: The duration of loading non-iterable vmstate accounts for a significant portion of downtime (starting with the timestamp of source qemu stop and ending with the timestamp of target qemu start). Most of the time

Re: [RFC v4 0/3] migration: reduce time of loading non-iterable vmstate

2022-12-23 Thread Chuang Xu
On 2022/12/23 下午11:50, Peter Xu wrote: Chuang, On Fri, Dec 23, 2022 at 10:23:04PM +0800, Chuang Xu wrote: In this version: - attach more information in the cover letter. - remove changes on virtio_load(). - add rcu_read_locked() to detect holding of rcu lock. The duration of loading non

[RFC v4 2/3] memory: add depth assert in address_space_to_flatview

2022-12-23 Thread Chuang Xu
Before using any flatview, sanity check we're not during a memory region transaction or the map can be invalid. Signed-off-by: Chuang Xu --- include/exec/memory.h | 9 + softmmu/memory.c | 5 + 2 files changed, 14 insertions(+) diff --git a/include/exec/memory.h b/include/exec

[RFC v4 1/3] rcu: introduce rcu_read_locked()

2022-12-23 Thread Chuang Xu
add rcu_read_locked() to detect holding of rcu lock. Signed-off-by: Chuang Xu --- include/qemu/rcu.h | 7 +++ 1 file changed, 7 insertions(+) diff --git a/include/qemu/rcu.h b/include/qemu/rcu.h index b063c6fde8..42cbd0080f 100644 --- a/include/qemu/rcu.h +++ b/include/qemu/rcu.h @@ -119,6

[RFC v4 0/3] migration: reduce time of loading non-iterable vmstate

2022-12-23 Thread Chuang Xu
In this version: - attach more information in the cover letter. - remove changes on virtio_load(). - add rcu_read_locked() to detect holding of rcu lock. The duration of loading non-iterable vmstate accounts for a significant portion of downtime (starting with the timestamp of source qemu stop

[RFC v4 3/3] migration: reduce time of loading non-iterable vmstate

2022-12-23 Thread Chuang Xu
commits, and the time consumption caused by the flatview reconstruction will also increase. Signed-off-by: Chuang Xu --- migration/savevm.c | 7 +++ 1 file changed, 7 insertions(+) diff --git a/migration/savevm.c b/migration/savevm.c index a0cdb714f7..19785e5a54 100644 --- a/migration

Re: [RFC v3 1/3] memory: add depth assert in address_space_to_flatview

2022-12-20 Thread Chuang Xu
On 2022/12/16 上午12:04, Peter Xu wrote: On Wed, Dec 14, 2022 at 04:38:52PM -0500, Peter Xu wrote: On Wed, Dec 14, 2022 at 08:03:38AM -0800, Chuang Xu wrote: On 2022/12/13 下午9:35, Chuang Xu wrote: Before using any flatview, sanity check we're not during a memory region transaction or the map can

Re: [RFC v3 1/3] memory: add depth assert in address_space_to_flatview

2022-12-20 Thread Chuang Xu
On 2022/12/16 上午12:51, Peter Maydell wrote: On Tue, 13 Dec 2022 at 13:36, Chuang Xu wrote: Before using any flatview, sanity check we're not during a memory region transaction or the map can be invalid. Signed-off-by: Chuang Xu --- include/exec/memory.h | 9 + softmmu/memory.c

Re: [RFC v3 1/3] memory: add depth assert in address_space_to_flatview

2022-12-14 Thread Chuang Xu
On 2022/12/13 下午9:35, Chuang Xu wrote: Before using any flatview, sanity check we're not during a memory region transaction or the map can be invalid. Signed-off-by: Chuang Xu --- include/exec/memory.h | 9 + softmmu/memory.c | 1 - 2 files changed, 9 insertions(+), 1 deletion

Re: [RFC v3 2/3] virtio: support delay of checks in virtio_load()

2022-12-14 Thread Chuang Xu
On 2022/12/14 上午12:31, Peter Xu wrote: On Tue, Dec 13, 2022 at 09:35:09PM +0800, Chuang Xu wrote: Delay checks in virtio_load() to avoid possible address_space_to_flatview() call during memory region's begin/commit. I didn't notice virtio has the vm change handler already, looks good to reuse

[RFC v3 3/3] migration: reduce time of loading non-iterable vmstate

2022-12-13 Thread Chuang Xu
device. time of loading non-iterable vmstate before about 210 ms after about 40 ms Signed-off-by: Chuang Xu --- migration/savevm.c | 7 +++ 1 file changed, 7 insertions(+) diff --git a/migration/savevm.c b/migration/savevm.c index a0cdb714f7..19785e5a54 100644

[RFC v3 2/3] virtio: support delay of checks in virtio_load()

2022-12-13 Thread Chuang Xu
Delay checks in virtio_load() to avoid possible address_space_to_flatview() call during memory region's begin/commit. Signed-off-by: Chuang Xu --- hw/virtio/virtio.c | 37 +++-- include/hw/virtio/virtio.h | 2 ++ 2 files changed, 29 insertions(+), 10

[RFC v3 1/3] memory: add depth assert in address_space_to_flatview

2022-12-13 Thread Chuang Xu
Before using any flatview, sanity check we're not during a memory region transaction or the map can be invalid. Signed-off-by: Chuang Xu --- include/exec/memory.h | 9 + softmmu/memory.c | 1 - 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/include/exec/memory.h b

[RFC v3 0/3] migration: reduce time of loading non-iterable vmstate

2022-12-13 Thread Chuang Xu
Hi! In this version: - move virtio_load_check_delay() from virtio_memory_listener_commit() to virtio_vmstate_change(). - add delay_check flag to VirtIODevice to make sure virtio_load_check_delay() will be called when delay_check is true. Please review, Chuang. [v2] - rebase to latest

Re: [RFC v2 2/3] virtio: support delay of checks in virtio_load()

2022-12-13 Thread Chuang Xu
On 2022/12/13 上午4:18, Peter Xu wrote: On Tue, Dec 13, 2022 at 12:49:41AM +0800, Chuang Xu wrote: +bool migration_enable_load_check_delay; I'm just afraid this is still too hacky. One thing is because this variable itself to be only set at specific phase during migration to cover that commit

Re: [RFC v2 0/3] migration: reduce time of loading non-iterable vmstate

2022-12-13 Thread Chuang Xu
On 2022/12/13 上午4:23, Peter Xu wrote: On Tue, Dec 13, 2022 at 12:49:39AM +0800, Chuang Xu wrote: Hi! Chuang, In this version: - rebase to latest upstream. - add sanity check to address_space_to_flatview(). - postpone the init of the vring cache until migration's loading completes. Since

[RFC v2 0/3] migration: reduce time of loading non-iterable vmstate

2022-12-12 Thread Chuang Xu
Hi! In this version: - rebase to latest upstream. - add sanity check to address_space_to_flatview(). - postpone the init of the vring cache until migration's loading completes. Please review, Chuang. [v1] The duration of loading non-iterable vmstate accounts for a significant portion of

[RFC v2 3/3] migration: reduce time of loading non-iterable vmstate

2022-12-12 Thread Chuang Xu
device. time of loading non-iterable vmstate before about 210 ms after about 40 ms Signed-off-by: Chuang Xu --- migration/savevm.c | 13 + 1 file changed, 13 insertions(+) diff --git a/migration/savevm.c b/migration/savevm.c index a0cdb714f7..68a7a99b79

[RFC v2 1/3] memory: add depth assert in address_space_to_flatview

2022-12-12 Thread Chuang Xu
Before using any flatview, sanity check we're not during a memory region transaction or the map can be invalid. Signed-off-by: Chuang Xu --- include/exec/memory.h | 9 + softmmu/memory.c | 1 - 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/include/exec/memory.h b

[RFC v2 2/3] virtio: support delay of checks in virtio_load()

2022-12-12 Thread Chuang Xu
Delay checks in virtio_load() to avoid possible address_space_to_flatview() call during memory region's begin/commit. Signed-off-by: Chuang Xu --- hw/virtio/virtio.c | 33 ++--- include/sysemu/sysemu.h | 1 + softmmu/globals.c | 3 +++ 3 files changed

Re: [External] Re: [RFC PATCH] migration: reduce time of loading non-iterable vmstate

2022-12-09 Thread Chuang Xu
Hi, Peter! This email is a supplement to my previous email 7 hours ago. On 2022/12/9 上午12:00, Peter Xu wrote: If the assert will work that'll be even better. I'm actually worried this can trigger like what you mentioned in the virtio path, I didn't expect it comes that soon. So if there's a

Re: [RFC PATCH] migration: reduce time of loading non-iterable vmstate

2022-12-08 Thread Chuang Xu
On 2022/12/9 上午12:00, Peter Xu wrote: On Thu, Dec 08, 2022 at 10:39:11PM +0800, Chuang Xu wrote: On 2022/12/8 上午6:08, Peter Xu wrote: On Thu, Dec 08, 2022 at 12:07:03AM +0800, Chuang Xu wrote: On 2022/12/6 上午12:28, Peter Xu wrote: Chuang, No worry on the delay; you're faster than when I read

Re: [RFC PATCH] migration: reduce time of loading non-iterable vmstate

2022-12-08 Thread Chuang Xu
On 2022/12/8 上午6:08, Peter Xu wrote: On Thu, Dec 08, 2022 at 12:07:03AM +0800, Chuang Xu wrote: On 2022/12/6 上午12:28, Peter Xu wrote: Chuang, No worry on the delay; you're faster than when I read yours. :) On Mon, Dec 05, 2022 at 02:56:15PM +0800, Chuang Xu wrote: As a start, maybe you can

Re: [External] Re: [RFC PATCH] migration: reduce time of loading non-iterable vmstate

2022-12-07 Thread Chuang Xu
On 2022/12/6 上午12:28, Peter Xu wrote: Chuang, No worry on the delay; you're faster than when I read yours. :) On Mon, Dec 05, 2022 at 02:56:15PM +0800, Chuang Xu wrote: As a start, maybe you can try with poison address_space_to_flatview() (by e.g. checking the start_pack_mr_change flag

Re: [RFC PATCH] migration: reduce time of loading non-iterable vmstate

2022-12-04 Thread Chuang Xu
Peter, I'm sorry I didn't reply to your email in time, because I was busy with other work last week. Here is my latest progress. On 2022/11/29 上午1:41, Peter Xu wrote: On Mon, Nov 28, 2022 at 05:42:43PM +0800, Chuang Xu wrote: On 2022/11/25 上午12:40, Peter Xu wrote: On Fri, Nov 18, 2022 at 04

Re: [RFC PATCH] migration: reduce time of loading non-iterable vmstate

2022-11-28 Thread Chuang Xu
On 2022/11/25 上午12:40, Peter Xu wrote: On Fri, Nov 18, 2022 at 04:36:48PM +0800, Chuang Xu wrote: The duration of loading non-iterable vmstate accounts for a significant portion of downtime (starting with the timestamp of source qemu stop and ending with the timestamp of target qemu start

[RFC PATCH] migration: reduce time of loading non-iterable vmstate

2022-11-18 Thread Chuang Xu
device. time of loading non-iterable vmstate before about 210 ms after about 40 ms Signed-off-by: Chuang Xu --- migration/migration.c | 1 + migration/migration.h | 2 ++ migration/savevm.c| 8 3 files changed, 11 insertions(+) diff --git a/migration

Re: [PATCH v7 10/12] multifd: Support for zero pages transmission

2022-10-25 Thread chuang xu
On 2022/8/2 下午2:39, Juan Quintela wrote: This patch adds counters and similar. Logic will be added on the following patch. Signed-off-by: Juan Quintela --- Added counters for duplicated/non duplicated pages. Removed reviewed by from David. Add total_zero_pages --- migration/multifd.h

Re: [External] [PATCH v13 3/8] QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX

2022-06-15 Thread chuang xu
the errno is ENOBUFS and set optmem to the initial 20KB(echo 20480 > /proc/sys/net/core/optmem_max), now the multifd zero_copy migration goes well. Here are the changes I made to the code: Signed-off-by: chuang xu ---  io/channel-socket.c | 4 +---  1 file changed, 1 insertion(+), 3 deletions(-

Re: [External] [PATCH v13 3/8] QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX

2022-06-14 Thread chuang xu
the RFC says.Then I tested the multifd zero_copy migration repeatedly,and the error disappeared. So when sendmsg returns -1 with errno ENOBUFS, should we distinguish between error ''socket exceeds optmem limit" and error "user exceeds ulimit on locked pages"? Or is there any better way to avoid this problem? Best Regards, chuang xu

Re: [External] [PATCH v13 3/8] QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX

2022-06-08 Thread chuang xu
On 2022/6/8 下午1:24, Leonardo Bras Soares Passos wrote: I will send a fix shortly. Is that ok if I include a "Reported-by: 徐闯 " in the patch? okay. Best Regards, chuang xu