On Mon, May 15, 2023 at 10:32:00AM +0200, Juan Quintela wrote: > Sometimes devices have different features depending of things outside > of qemu. For instance the kernel. Document how to handle that cases. > > Signed-off-by: Juan Quintela <quint...@redhat.com>
yes, e.g. vhost features are exactly like this. > --- > > If you have some example to put here, I am all ears. I guess that > virtio-* with some features that are on qemu but not on all kernel > would do the trick, but I am not a virtio guru myself. Patches > welcome. > --- > docs/devel/migration.rst | 93 ++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 93 insertions(+) > > diff --git a/docs/devel/migration.rst b/docs/devel/migration.rst > index b4c4f3ec35..95e797ee60 100644 > --- a/docs/devel/migration.rst > +++ b/docs/devel/migration.rst > @@ -357,6 +357,99 @@ machine types to have the right value: :: > ... > }; > > +A device with diferent features on both sides > +--------------------------------------------- > + > +Let's assume that we are using the same QEMU binary on both sides, > +just to make the things easier. But we have a device that has > +different features on both sides of the migration. That can be > +because the devices are different, because the kernel driver of both > +devices have different features, whatever. > + > +How can we get this to work with migration. The way to do that is > +"theoretically" easy. You have to get the features that the device > +has in the source of the migration. The features that the device has > +on the target of the migration, you get the intersection of the > +features of both sides, and that is the way that you should launch > +qemu. > + > +Notice that this is not completely related to qemu. The most > +important thing here is that this should be handle by the managing > +application that launches qemu. If qemu is configured correctly, the > +migration will suceeed. > + > +Once that we have defined that, doing this is complicated. Almost all > +devices are bad at being able to be launched with only some features > +enabled. With one big exception: cpus. > + > +You can read the documentation for QEMU x86 cpu models here: > + > +https://qemu-project.gitlab.io/qemu/system/qemu-cpu-models.html > + > +See when they talk about migration they recommend that one chooses the > +newest cpu model that is supported for all cpus. > + > +Let's say that we have: > + > +Host A: > + > +Device X has the feature Y > + > +Host B: > + > +Device X has not the feature Y > + > +If we try to migrate without any care from host A to host B, it will > +fail because when migration tries to load the feature Y on > +destination, it will find that the hardware is not there. > + > +Doing this would be the equivalent of doing with cpus: > + > +Host A: > + > +$ qemu-system-x86_64 -cpu host > + > +Host B: > + > +$ qemu-system-x86_64 -cpu host > + > +When both hosts have different cpu features this is waranteed to fail. > +Especially if Host B has less features than host A. If host A has > +less features than host B, sometimes it works. Important word of last > +sentence is "sometimes". > + > +So, forgetting about cpu models and continuing with the -cpu host > +example, let's see that the differences of the cpus is that Host A and > +B have the following features: > + > +Features: 'pcid' 'stibp' 'taa-no' > +Host A: X X > +Host B: X > + > +And we want to migrate between them, the way configure both qemu cpu > +will be: > + > +Host A: > + > +$ qemu-system-x86_64 -cpu host,pcid=off,stibp=off > + > +Host B: > + > +$ qemu-system-x86_64 -cpu host,taa-no=off > + > +And you would be able to migrate between them. It is responsability > +of the management application or of the user to make sure that the > +configuration is correct. QEMU don't know how to look at this kind of > +features in general. > + > +Other devices have worse control about individual features. If they > +want to be able to migrate between hosts that show different features, > +the device needs a way to configure which ones it is going to use. > + > +In this section we have considered that we are using the same QEMU > +binary in both sides of the migration. If we use different QEMU > +versions process, then we need to have into account all other > +differences and the examples become even more complicated. How do people know what to do? How about a tool that will help you get data from hosts and then tell you how to configure qemu to make them compatible? > VMState > ------- > -- > 2.40.1