On 6/3/2020 4:53 PM, Andrew Jones wrote:
On Tue, Jun 02, 2020 at 03:47:22PM +0800, Ying Fang wrote:
On 2020/6/1 20:29, Andrew Jones wrote:
On Mon, Jun 01, 2020 at 08:07:31PM +0800, Ying Fang wrote:
On 2020/6/1 16:07, Andrew Jones wrote:
On Sat, May 30, 2020 at 04:56:26PM +0800, Ying Fang wrote:
About the kvm-no-adjvtime CPU property
Hi Andrew,
To adjust virutal time, a new kvm cpu property kvm-no-adjvtime
was introduced to 5.0 virt machine types. However the cpu
property was enabled only for host-passthrough and max cpu model.
As for other cpu model like cortex-a57, cortex-a53, cortex-a72,
this kvm-adjvtime is not enabled by default, which means the
virutal time can not be adjust for them.
Here, for example, if VM is configured with kvm enabled:
<cpu mode='custom' match='exact' check='partial'>
<model fallback='allow'>cortex-a72</model>
<topology sockets='2' dies='1' cores='2' threads='1'/>
<numa>
<cell id='0' cpus='0-1' memory='16777216' unit='KiB'/>
<cell id='1' cpus='2-3' memory='16777216' unit='KiB'/>
</numa>
</cpu>
We cannot adjust virtual time even if 5.0 virt machine is used.
So i'd like to add it to other cpu model, do you have any
suggestions here ?
Hi Fang,
The cpu feature only requires kvm. If a cpu model may be used with kvm,
then the feature can be allowed to be used with the model. What I find
interesting is that the cpu model is being used with kvm instead of 'host'
or 'max'. Can you explain the reasons for that? Currently, when using
yes,the cpu model is indeed used with kvm.
There is a situation where the host cpu model is Cortex-A72 and
a 'custom' cpu mode is used to keep insrtuction set compatible between
the source and destination host machine when doing live migration.
So the host physical machine cpu model is Cortex-A72 but
host-passthrough model is mode used here.
I mean host-passthrough model is 'not' used here. Sorry to make it
confusing.
I guessed as much.
Are the source and destinations hosts used in the migration identical?
If so, then the guest can use cpu 'host' and disable cpu features that
should not be exposed (e.g. -cpu host,pmu=off). If the source and
destination hosts are not identical, then I'm curious what those exact
differences are. With the way AArch64 KVM works today, even using the
Cortex-A72 cpu model should require identical hosts when migrating. Or,
at least both hosts must be compatible with Cortex-A72 and any difference
in ID registers must be somehow getting hidden from the guest.
Yes, you are right.
We have AAarch64 server with cpu based on Cortex-A72 and some extra
instruction set added. Source host with cpu based on V1 and destination host
with cpu based on V2 and they are compatible with Cortex-A72. We want to use
a 'custom' cpu mode here to make it possible to do live migration between
them. This is the scenario where the 'host' cpu model is not used since a
'custom' cpu model Cortex-A72 is used here .
What you've described here is indeed the reason to use CPU models. I.e.
enabling the migration from a host of one type to another by creating a
guest that only enables the features contained in both hosts (as well as
maintaining all state that describes the CPU type, e.g. MIDR).
Unfortunately, unless your KVM has patches that aren't upstream, then that
doesn't work on AArch64 KVM (more on that below). It may appear to be
working for you, because your guest kernel and userspace don't mind the
slight differences exposed to it between the hosts, or those differences
are limited to explicitly disabled features. If that's the case, then I
would guess that using '-cpu host' and disabling the same features would
"work" as well.
Yes, upstream KVM currently does not support it. We are planning to add
support for the aarch64 platform since we have the situation where it is
needed for our hardware.
@Marc Zyngier, is there anyone who doing on this?
Here's some more details on why the Cortex-A72 CPU model doesn't matter
with upstream KVM. First, upstream AArch64 KVM doesn't support CPU models,
and it doesn't even have a Cortex-A72 preferred target. For Cortex-A72
it will use "KVM_ARM_TARGET_GENERIC_V8", which is the same thing 'host'
would do when running on a Cortex-A72. Second, if V2 of the Cortex-A72-
based CPU you're using changed the revision of the MIDR, or any other
state that gets passed directly through to the guest like the MIDR, then
that state will change on migration. If a guest looks before migration and
again after migration, then it could get confused. A guest kernel may only
look once on boot and then not notice, but anything exposed to userspace
is extra risky, as userspace may check more frequently.
Yes, just as explained here.
In short, without KVM patches that aren't upstream, then it's risky to
migrate between machines with V1 and V2 of these CPUs. And, it doesn't
help to use the Cortex-A72 CPU model.
Thanks for your detailed introduction.
Thanks,
drew
However the kvm-adjvtime
feature is also need. So I think we should move kvm_arm_add_vcpu_properties
to arm_cpu_post_init, instead of limited to 'host' and 'max' cpu model[1].
1: https://lists.gnu.org/archive/html/qemu-devel/2020-06/msg00091.html
Thanks,
drew
.