RE: [PATCH v11 05/11] vfio/migration: Block multiple devices migration

[Shameerali Kolothum Thodi via]

> -----Original Message-----
> From: Jason Gunthorpe [mailto:j...@nvidia.com]
> Sent: 16 May 2023 13:00
> To: Shameerali Kolothum Thodi <shameerali.kolothum.th...@huawei.com>
> Cc: Avihai Horon <avih...@nvidia.com>; qemu-devel@nongnu.org; Alex
> Williamson <alex.william...@redhat.com>; Juan Quintela
> <quint...@redhat.com>; Dr. David Alan Gilbert <dgilb...@redhat.com>;
> Michael S. Tsirkin <m...@redhat.com>; Cornelia Huck <coh...@redhat.com>;
> Paolo Bonzini <pbonz...@redhat.com>; Vladimir Sementsov-Ogievskiy
> <vsement...@yandex-team.ru>; Cédric Le Goater <c...@redhat.com>; Yishai
> Hadas <yish...@nvidia.com>; Maor Gottlieb <ma...@nvidia.com>; Kirti
> Wankhede <kwankh...@nvidia.com>; Tarun Gupta <targu...@nvidia.com>;
> Joao Martins <joao.m.mart...@oracle.com>
> Subject: Re: [PATCH v11 05/11] vfio/migration: Block multiple devices
> migration
> 
> On Tue, May 16, 2023 at 10:03:54AM +0000, Shameerali Kolothum Thodi
> wrote:
> 
> > > Currently VFIO migration doesn't implement some kind of intermediate
> > > quiescent state in which P2P DMAs are quiesced before stopping or
> > > running the device. This can cause problems in multi-device migration
> > > where the devices are doing P2P DMAs, since the devices are not stopped
> > > together at the same time.
> > >
> > > Until such support is added, block migration of multiple devices.
> >
> > Missed this one. Currently this blocks even if the attached devices are not
> > capable of P2P DMAs. eg; HiSilicon ACC devices. These are integrated end
> point
> > devices without any P2P capability between them. Is it Ok to check for
> > VFIO_MIGRATION_P2P flag and allow if the devices are not supporting
> that?
> 
> Lacking VFIO_MIGRATION_P2P doesn't mean the device is incapable of
> P2P, it means the migration can't support P2P.
> 
> We'd need some kind of new flag to check and such devices should be
> blocked from creating P2P mappings. Basically we don't currently
> fully support devices that are incapable of P2P operations.

Ok. I will take a look.

> What happens on your platform if a guest tries to do P2P? Does the
> platform crash?

I am not sure. Since the devices are behind SMMU, I was under the assumption
that we do have the guarantee of isolation here(grouping). Or this is something
we are worried only during migration?

Thanks,
Shameer