Re: [PATCH v14 Kernel 1/7] vfio: KABI for migration interface for device state
On 3/23/2020 5:15 PM, Auger Eric wrote:
Hi Kirti,
On 3/18/20 8:41 PM, Kirti Wankhede wrote:
- Defined MIGRATION region type and sub-type.
- Defined vfio_device_migration_info structure which will be placed at the
0th offset of migration region to get/set VFIO device related
information. Defined members of structure and usage on read/write access.
- Defined device states and state transition details.
- Defined sequence to be followed while saving and resuming VFIO device.
Signed-off-by: Kirti Wankhede
Reviewed-by: Neo Jia
---
include/uapi/linux/vfio.h | 227 ++
1 file changed, 227 insertions(+)
diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
index 9e843a147ead..d0021467af53 100644
--- a/include/uapi/linux/vfio.h
+++ b/include/uapi/linux/vfio.h
@@ -305,6 +305,7 @@ struct vfio_region_info_cap_type {
#define VFIO_REGION_TYPE_PCI_VENDOR_MASK (0x)
#define VFIO_REGION_TYPE_GFX(1)
#define VFIO_REGION_TYPE_CCW (2)
+#define VFIO_REGION_TYPE_MIGRATION (3)
/* sub-types for VFIO_REGION_TYPE_PCI_* */
@@ -379,6 +380,232 @@ struct vfio_region_gfx_edid {
/* sub-types for VFIO_REGION_TYPE_CCW */
#define VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD (1)
+/* sub-types for VFIO_REGION_TYPE_MIGRATION */
+#define VFIO_REGION_SUBTYPE_MIGRATION (1)
+
+/*
+ * The structure vfio_device_migration_info is placed at the 0th offset of
+ * the VFIO_REGION_SUBTYPE_MIGRATION region to get and set VFIO device related
+ * migration information. Field accesses from this structure are only supported
+ * at their native width and alignment. Otherwise, the result is undefined and
+ * vendor drivers should return an error.
+ *
+ * device_state: (read/write)
+ * - The user application writes to this field to inform the vendor driver
+ *about the device state to be transitioned to.
+ * - The vendor driver should take the necessary actions to change the
+ *device state. After successful transition to a given state, the
+ *vendor driver should return success on write(device_state, state)
+ *system call. If the device state transition fails, the vendor driver
+ *should return an appropriate -errno for the fault condition.
+ * - On the user application side, if the device state transition fails,
+ * that is, if write(device_state, state) returns an error, read
+ * device_state again to determine the current state of the device from
+ * the vendor driver.
+ * - The vendor driver should return previous state of the device unless
+ *the vendor driver has encountered an internal error, in which case
+ *the vendor driver may report the device_state
VFIO_DEVICE_STATE_ERROR.
+ * - The user application must use the device reset ioctl to recover the
+ *device from VFIO_DEVICE_STATE_ERROR state. If the device is
+ *indicated to be in a valid device state by reading device_state, the
+ *user application may attempt to transition the device to any valid
+ *state reachable from the current state or terminate itself.
+ *
+ * device_state consists of 3 bits:
+ * - If bit 0 is set, it indicates the _RUNNING state. If bit 0 is clear,
+ *it indicates the _STOP state. When the device state is changed to
+ *_STOP, driver should stop the device before write() returns.
+ * - If bit 1 is set, it indicates the _SAVING state, which means that the
+ *driver should start gathering device state information that will be
+ *provided to the VFIO user application to save the device's state.
+ * - If bit 2 is set, it indicates the _RESUMING state, which means that
+ *the driver should prepare to resume the device. Data provided through
+ *the migration region should be used to resume the device.
+ * Bits 3 - 31 are reserved for future use. To preserve them, the user
+ * application should perform a read-modify-write operation on this
+ * field when modifying the specified bits.
+ *
+ * +--- _RESUMING
+ * |+-- _SAVING
+ * ||+- _RUNNING
+ * |||
+ * 000b => Device Stopped, not saving or resuming
+ * 001b => Device running, which is the default state
+ * 010b => Stop the device & save the device state, stop-and-copy state
+ * 011b => Device running and save the device state, pre-copy state
+ * 100b => Device stopped and the device state is resuming
+ * 101b => Invalid state
+ * 110b => Error state
+ * 111b => Invalid state
+ *
+ * State transitions:
+ *
+ * _RESUMING _RUNNINGPre-copyStop-and-copy _STOP
+ *(100b) (001b) (011b)(010b) (000b)
+ * 0. Running or default state
+ * |
+ *
+ * 1. Normal Shutdown (optional)
+ * |->|
+ *
+ * 2.
Re: [PATCH v14 Kernel 1/7] vfio: KABI for migration interface for device state
Hi,
On 3/19/20 2:09 PM, Alex Williamson wrote:
> On Thu, 19 Mar 2020 01:05:54 -0400
> Yan Zhao wrote:
>
>> On Thu, Mar 19, 2020 at 11:49:26AM +0800, Alex Williamson wrote:
>>> On Wed, 18 Mar 2020 21:17:03 -0400
>>> Yan Zhao wrote:
>>>
On Thu, Mar 19, 2020 at 03:41:08AM +0800, Kirti Wankhede wrote:
> - Defined MIGRATION region type and sub-type.
>
> - Defined vfio_device_migration_info structure which will be placed at the
> 0th offset of migration region to get/set VFIO device related
> information. Defined members of structure and usage on read/write
> access.
>
> - Defined device states and state transition details.
>
> - Defined sequence to be followed while saving and resuming VFIO device.
>
> Signed-off-by: Kirti Wankhede
> Reviewed-by: Neo Jia
> ---
> include/uapi/linux/vfio.h | 227
> ++
> 1 file changed, 227 insertions(+)
>
> diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
> index 9e843a147ead..d0021467af53 100644
> --- a/include/uapi/linux/vfio.h
> +++ b/include/uapi/linux/vfio.h
> @@ -305,6 +305,7 @@ struct vfio_region_info_cap_type {
> #define VFIO_REGION_TYPE_PCI_VENDOR_MASK (0x)
> #define VFIO_REGION_TYPE_GFX(1)
> #define VFIO_REGION_TYPE_CCW (2)
> +#define VFIO_REGION_TYPE_MIGRATION (3)
>
> /* sub-types for VFIO_REGION_TYPE_PCI_* */
>
> @@ -379,6 +380,232 @@ struct vfio_region_gfx_edid {
> /* sub-types for VFIO_REGION_TYPE_CCW */
> #define VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD(1)
>
> +/* sub-types for VFIO_REGION_TYPE_MIGRATION */
> +#define VFIO_REGION_SUBTYPE_MIGRATION (1)
> +
> +/*
> + * The structure vfio_device_migration_info is placed at the 0th offset
> of
> + * the VFIO_REGION_SUBTYPE_MIGRATION region to get and set VFIO device
> related
> + * migration information. Field accesses from this structure are only
> supported
> + * at their native width and alignment. Otherwise, the result is
> undefined and
> + * vendor drivers should return an error.
> + *
> + * device_state: (read/write)
> + * - The user application writes to this field to inform the vendor
> driver
> + *about the device state to be transitioned to.
> + * - The vendor driver should take the necessary actions to change
> the
> + *device state. After successful transition to a given state, the
> + *vendor driver should return success on write(device_state,
> state)
> + *system call. If the device state transition fails, the vendor
> driver
> + *should return an appropriate -errno for the fault condition.
> + * - On the user application side, if the device state transition
> fails,
> + * that is, if write(device_state, state) returns an error, read
> + * device_state again to determine the current state of the
> device from
> + * the vendor driver.
> + * - The vendor driver should return previous state of the device
> unless
> + *the vendor driver has encountered an internal error, in which
> case
> + *the vendor driver may report the device_state
> VFIO_DEVICE_STATE_ERROR.
> + * - The user application must use the device reset ioctl to
> recover the
> + *device from VFIO_DEVICE_STATE_ERROR state. If the device is
> + *indicated to be in a valid device state by reading
> device_state, the
> + *user application may attempt to transition the device to any
> valid
> + *state reachable from the current state or terminate itself.
> + *
> + * device_state consists of 3 bits:
> + * - If bit 0 is set, it indicates the _RUNNING state. If bit 0 is
> clear,
> + *it indicates the _STOP state. When the device state is changed
> to
> + *_STOP, driver should stop the device before write() returns.
> + * - If bit 1 is set, it indicates the _SAVING state, which means
> that the
> + *driver should start gathering device state information that
> will be
> + *provided to the VFIO user application to save the device's
> state.
> + * - If bit 2 is set, it indicates the _RESUMING state, which means
> that
> + *the driver should prepare to resume the device. Data provided
> through
> + *the migration region should be used to resume the device.
> + * Bits 3 - 31 are reserved for future use. To preserve them, the
> user
> + * application should perform a read-modify-write operation on this
> + * field when modifying the specified bits.
>
Re: [PATCH v14 Kernel 1/7] vfio: KABI for migration interface for device state
Hi Kirti,
On 3/18/20 8:41 PM, Kirti Wankhede wrote:
> - Defined MIGRATION region type and sub-type.
>
> - Defined vfio_device_migration_info structure which will be placed at the
> 0th offset of migration region to get/set VFIO device related
> information. Defined members of structure and usage on read/write access.
>
> - Defined device states and state transition details.
>
> - Defined sequence to be followed while saving and resuming VFIO device.
>
> Signed-off-by: Kirti Wankhede
> Reviewed-by: Neo Jia
> ---
> include/uapi/linux/vfio.h | 227
> ++
> 1 file changed, 227 insertions(+)
>
> diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
> index 9e843a147ead..d0021467af53 100644
> --- a/include/uapi/linux/vfio.h
> +++ b/include/uapi/linux/vfio.h
> @@ -305,6 +305,7 @@ struct vfio_region_info_cap_type {
> #define VFIO_REGION_TYPE_PCI_VENDOR_MASK (0x)
> #define VFIO_REGION_TYPE_GFX(1)
> #define VFIO_REGION_TYPE_CCW (2)
> +#define VFIO_REGION_TYPE_MIGRATION (3)
>
> /* sub-types for VFIO_REGION_TYPE_PCI_* */
>
> @@ -379,6 +380,232 @@ struct vfio_region_gfx_edid {
> /* sub-types for VFIO_REGION_TYPE_CCW */
> #define VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD(1)
>
> +/* sub-types for VFIO_REGION_TYPE_MIGRATION */
> +#define VFIO_REGION_SUBTYPE_MIGRATION (1)
> +
> +/*
> + * The structure vfio_device_migration_info is placed at the 0th offset of
> + * the VFIO_REGION_SUBTYPE_MIGRATION region to get and set VFIO device
> related
> + * migration information. Field accesses from this structure are only
> supported
> + * at their native width and alignment. Otherwise, the result is undefined
> and
> + * vendor drivers should return an error.
> + *
> + * device_state: (read/write)
> + * - The user application writes to this field to inform the vendor
> driver
> + *about the device state to be transitioned to.
> + * - The vendor driver should take the necessary actions to change the
> + *device state. After successful transition to a given state, the
> + *vendor driver should return success on write(device_state, state)
> + *system call. If the device state transition fails, the vendor
> driver
> + *should return an appropriate -errno for the fault condition.
> + * - On the user application side, if the device state transition fails,
> + * that is, if write(device_state, state) returns an error, read
> + * device_state again to determine the current state of the device from
> + * the vendor driver.
> + * - The vendor driver should return previous state of the device unless
> + *the vendor driver has encountered an internal error, in which case
> + *the vendor driver may report the device_state
> VFIO_DEVICE_STATE_ERROR.
> + * - The user application must use the device reset ioctl to recover the
> + *device from VFIO_DEVICE_STATE_ERROR state. If the device is
> + *indicated to be in a valid device state by reading device_state,
> the
> + *user application may attempt to transition the device to any valid
> + *state reachable from the current state or terminate itself.
> + *
> + * device_state consists of 3 bits:
> + * - If bit 0 is set, it indicates the _RUNNING state. If bit 0 is
> clear,
> + *it indicates the _STOP state. When the device state is changed to
> + *_STOP, driver should stop the device before write() returns.
> + * - If bit 1 is set, it indicates the _SAVING state, which means that
> the
> + *driver should start gathering device state information that will be
> + *provided to the VFIO user application to save the device's state.
> + * - If bit 2 is set, it indicates the _RESUMING state, which means that
> + *the driver should prepare to resume the device. Data provided
> through
> + *the migration region should be used to resume the device.
> + * Bits 3 - 31 are reserved for future use. To preserve them, the user
> + * application should perform a read-modify-write operation on this
> + * field when modifying the specified bits.
> + *
> + * +--- _RESUMING
> + * |+-- _SAVING
> + * ||+- _RUNNING
> + * |||
> + * 000b => Device Stopped, not saving or resuming
> + * 001b => Device running, which is the default state
> + * 010b => Stop the device & save the device state, stop-and-copy state
> + * 011b => Device running and save the device state, pre-copy state
> + * 100b => Device stopped and the device state is resuming
> + * 101b => Invalid state
> + * 110b => Error state
> + * 111b => Invalid state
> + *
> + * State transitions:
> + *
> + * _RESUMING _RUNNINGPre-copyStop-and-copy _STOP
> + *(100b) (001b) (011b)(010b) (000b)
> + * 0. Running or default state
> + *
Re: [PATCH v14 Kernel 1/7] vfio: KABI for migration interface for device state
On Fri, Mar 20, 2020 at 12:09:18PM +0800, Alex Williamson wrote:
> On Thu, 19 Mar 2020 23:06:56 -0400
> Yan Zhao wrote:
>
> > On Fri, Mar 20, 2020 at 10:34:40AM +0800, Alex Williamson wrote:
> > > On Thu, 19 Mar 2020 21:30:39 -0400
> > > Yan Zhao wrote:
> > >
> > > > On Thu, Mar 19, 2020 at 09:09:21PM +0800, Alex Williamson wrote:
> > > > > On Thu, 19 Mar 2020 01:05:54 -0400
> > > > > Yan Zhao wrote:
> > > > >
> > > > > > On Thu, Mar 19, 2020 at 11:49:26AM +0800, Alex Williamson wrote:
> > > > > > > On Wed, 18 Mar 2020 21:17:03 -0400
> > > > > > > Yan Zhao wrote:
> > > > > > >
> > > > > > > > On Thu, Mar 19, 2020 at 03:41:08AM +0800, Kirti Wankhede wrote:
> > > > > > > >
> > > > > > > > > - Defined MIGRATION region type and sub-type.
> > > > > > > > >
> > > > > > > > > - Defined vfio_device_migration_info structure which will be
> > > > > > > > > placed at the
> > > > > > > > > 0th offset of migration region to get/set VFIO device
> > > > > > > > > related
> > > > > > > > > information. Defined members of structure and usage on
> > > > > > > > > read/write access.
> > > > > > > > >
> > > > > > > > > - Defined device states and state transition details.
> > > > > > > > >
> > > > > > > > > - Defined sequence to be followed while saving and resuming
> > > > > > > > > VFIO device.
> > > > > > > > >
> > > > > > > > > Signed-off-by: Kirti Wankhede
> > > > > > > > > Reviewed-by: Neo Jia
> > > > > > > > > ---
> > > > > > > > > include/uapi/linux/vfio.h | 227
> > > > > > > > > ++
> > > > > > > > > 1 file changed, 227 insertions(+)
> > > > > > > > >
> > > > > > > > > diff --git a/include/uapi/linux/vfio.h
> > > > > > > > > b/include/uapi/linux/vfio.h
> > > > > > > > > index 9e843a147ead..d0021467af53 100644
> > > > > > > > > --- a/include/uapi/linux/vfio.h
> > > > > > > > > +++ b/include/uapi/linux/vfio.h
> > > > > > > > > @@ -305,6 +305,7 @@ struct vfio_region_info_cap_type {
> > > > > > > > > #define VFIO_REGION_TYPE_PCI_VENDOR_MASK (0x)
> > > > > > > > > #define VFIO_REGION_TYPE_GFX(1)
> > > > > > > > > #define VFIO_REGION_TYPE_CCW (2)
> > > > > > > > > +#define VFIO_REGION_TYPE_MIGRATION (3)
> > > > > > > > >
> > > > > > > > > /* sub-types for VFIO_REGION_TYPE_PCI_* */
> > > > > > > > >
> > > > > > > > > @@ -379,6 +380,232 @@ struct vfio_region_gfx_edid {
> > > > > > > > > /* sub-types for VFIO_REGION_TYPE_CCW */
> > > > > > > > > #define VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD(1)
> > > > > > > > >
> > > > > > > > > +/* sub-types for VFIO_REGION_TYPE_MIGRATION */
> > > > > > > > > +#define VFIO_REGION_SUBTYPE_MIGRATION (1)
> > > > > > > > > +
> > > > > > > > > +/*
> > > > > > > > > + * The structure vfio_device_migration_info is placed at the
> > > > > > > > > 0th offset of
> > > > > > > > > + * the VFIO_REGION_SUBTYPE_MIGRATION region to get and set
> > > > > > > > > VFIO device related
> > > > > > > > > + * migration information. Field accesses from this structure
> > > > > > > > > are only supported
> > > > > > > > > + * at their native width and alignment. Otherwise, the
> > > > > > > > > result is undefined and
> > > > > > > > > + * vendor drivers should return an error.
> > > > > > > > > + *
> > > > > > > > > + * device_state: (read/write)
> > > > > > > > > + * - The user application writes to this field to
> > > > > > > > > inform the vendor driver
> > > > > > > > > + *about the device state to be transitioned to.
> > > > > > > > > + * - The vendor driver should take the necessary
> > > > > > > > > actions to change the
> > > > > > > > > + *device state. After successful transition to a
> > > > > > > > > given state, the
> > > > > > > > > + *vendor driver should return success on
> > > > > > > > > write(device_state, state)
> > > > > > > > > + *system call. If the device state transition fails,
> > > > > > > > > the vendor driver
> > > > > > > > > + *should return an appropriate -errno for the fault
> > > > > > > > > condition.
> > > > > > > > > + * - On the user application side, if the device state
> > > > > > > > > transition fails,
> > > > > > > > > + * that is, if write(device_state, state) returns an
> > > > > > > > > error, read
> > > > > > > > > + * device_state again to determine the current state of
> > > > > > > > > the device from
> > > > > > > > > + * the vendor driver.
> > > > > > > > > + * - The vendor driver should return previous state of
> > > > > > > > > the device unless
> > > > > > > > > + *the vendor driver has encountered an internal
> > > > > > > > > error, in which case
> > > > > > > > > + *the vendor driver may report the device_state
> > > > > > > > > VFIO_DEVICE_STATE_ERROR.
> > > > > > > > > + * - The user application must use the device reset
> > > > > > > > > ioctl to recover the
> > > > > > > >
Re: [PATCH v14 Kernel 1/7] vfio: KABI for migration interface for device state
On Thu, 19 Mar 2020 23:06:56 -0400
Yan Zhao wrote:
> On Fri, Mar 20, 2020 at 10:34:40AM +0800, Alex Williamson wrote:
> > On Thu, 19 Mar 2020 21:30:39 -0400
> > Yan Zhao wrote:
> >
> > > On Thu, Mar 19, 2020 at 09:09:21PM +0800, Alex Williamson wrote:
> > > > On Thu, 19 Mar 2020 01:05:54 -0400
> > > > Yan Zhao wrote:
> > > >
> > > > > On Thu, Mar 19, 2020 at 11:49:26AM +0800, Alex Williamson wrote:
> > > > > > On Wed, 18 Mar 2020 21:17:03 -0400
> > > > > > Yan Zhao wrote:
> > > > > >
> > > > > > > On Thu, Mar 19, 2020 at 03:41:08AM +0800, Kirti Wankhede wrote:
> > > > > > >
> > > > > > > > - Defined MIGRATION region type and sub-type.
> > > > > > > >
> > > > > > > > - Defined vfio_device_migration_info structure which will be
> > > > > > > > placed at the
> > > > > > > > 0th offset of migration region to get/set VFIO device related
> > > > > > > > information. Defined members of structure and usage on
> > > > > > > > read/write access.
> > > > > > > >
> > > > > > > > - Defined device states and state transition details.
> > > > > > > >
> > > > > > > > - Defined sequence to be followed while saving and resuming
> > > > > > > > VFIO device.
> > > > > > > >
> > > > > > > > Signed-off-by: Kirti Wankhede
> > > > > > > > Reviewed-by: Neo Jia
> > > > > > > > ---
> > > > > > > > include/uapi/linux/vfio.h | 227
> > > > > > > > ++
> > > > > > > > 1 file changed, 227 insertions(+)
> > > > > > > >
> > > > > > > > diff --git a/include/uapi/linux/vfio.h
> > > > > > > > b/include/uapi/linux/vfio.h
> > > > > > > > index 9e843a147ead..d0021467af53 100644
> > > > > > > > --- a/include/uapi/linux/vfio.h
> > > > > > > > +++ b/include/uapi/linux/vfio.h
> > > > > > > > @@ -305,6 +305,7 @@ struct vfio_region_info_cap_type {
> > > > > > > > #define VFIO_REGION_TYPE_PCI_VENDOR_MASK (0x)
> > > > > > > > #define VFIO_REGION_TYPE_GFX(1)
> > > > > > > > #define VFIO_REGION_TYPE_CCW (2)
> > > > > > > > +#define VFIO_REGION_TYPE_MIGRATION (3)
> > > > > > > >
> > > > > > > > /* sub-types for VFIO_REGION_TYPE_PCI_* */
> > > > > > > >
> > > > > > > > @@ -379,6 +380,232 @@ struct vfio_region_gfx_edid {
> > > > > > > > /* sub-types for VFIO_REGION_TYPE_CCW */
> > > > > > > > #define VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD (1)
> > > > > > > >
> > > > > > > > +/* sub-types for VFIO_REGION_TYPE_MIGRATION */
> > > > > > > > +#define VFIO_REGION_SUBTYPE_MIGRATION (1)
> > > > > > > > +
> > > > > > > > +/*
> > > > > > > > + * The structure vfio_device_migration_info is placed at the
> > > > > > > > 0th offset of
> > > > > > > > + * the VFIO_REGION_SUBTYPE_MIGRATION region to get and set
> > > > > > > > VFIO device related
> > > > > > > > + * migration information. Field accesses from this structure
> > > > > > > > are only supported
> > > > > > > > + * at their native width and alignment. Otherwise, the result
> > > > > > > > is undefined and
> > > > > > > > + * vendor drivers should return an error.
> > > > > > > > + *
> > > > > > > > + * device_state: (read/write)
> > > > > > > > + * - The user application writes to this field to inform
> > > > > > > > the vendor driver
> > > > > > > > + *about the device state to be transitioned to.
> > > > > > > > + * - The vendor driver should take the necessary actions
> > > > > > > > to change the
> > > > > > > > + *device state. After successful transition to a given
> > > > > > > > state, the
> > > > > > > > + *vendor driver should return success on
> > > > > > > > write(device_state, state)
> > > > > > > > + *system call. If the device state transition fails,
> > > > > > > > the vendor driver
> > > > > > > > + *should return an appropriate -errno for the fault
> > > > > > > > condition.
> > > > > > > > + * - On the user application side, if the device state
> > > > > > > > transition fails,
> > > > > > > > + * that is, if write(device_state, state) returns an
> > > > > > > > error, read
> > > > > > > > + * device_state again to determine the current state of
> > > > > > > > the device from
> > > > > > > > + * the vendor driver.
> > > > > > > > + * - The vendor driver should return previous state of
> > > > > > > > the device unless
> > > > > > > > + *the vendor driver has encountered an internal error,
> > > > > > > > in which case
> > > > > > > > + *the vendor driver may report the device_state
> > > > > > > > VFIO_DEVICE_STATE_ERROR.
> > > > > > > > + * - The user application must use the device reset ioctl
> > > > > > > > to recover the
> > > > > > > > + *device from VFIO_DEVICE_STATE_ERROR state. If the
> > > > > > > > device is
> > > > > > > > + *indicated to be in a valid device state by reading
> > > > > > > > device_state, the
> > > > > > > > + *user application may attempt to
Re: [PATCH v14 Kernel 1/7] vfio: KABI for migration interface for device state
On Fri, Mar 20, 2020 at 10:34:40AM +0800, Alex Williamson wrote:
> On Thu, 19 Mar 2020 21:30:39 -0400
> Yan Zhao wrote:
>
> > On Thu, Mar 19, 2020 at 09:09:21PM +0800, Alex Williamson wrote:
> > > On Thu, 19 Mar 2020 01:05:54 -0400
> > > Yan Zhao wrote:
> > >
> > > > On Thu, Mar 19, 2020 at 11:49:26AM +0800, Alex Williamson wrote:
> > > > > On Wed, 18 Mar 2020 21:17:03 -0400
> > > > > Yan Zhao wrote:
> > > > >
> > > > > > On Thu, Mar 19, 2020 at 03:41:08AM +0800, Kirti Wankhede wrote:
> > > > > > > - Defined MIGRATION region type and sub-type.
> > > > > > >
> > > > > > > - Defined vfio_device_migration_info structure which will be
> > > > > > > placed at the
> > > > > > > 0th offset of migration region to get/set VFIO device related
> > > > > > > information. Defined members of structure and usage on
> > > > > > > read/write access.
> > > > > > >
> > > > > > > - Defined device states and state transition details.
> > > > > > >
> > > > > > > - Defined sequence to be followed while saving and resuming VFIO
> > > > > > > device.
> > > > > > >
> > > > > > > Signed-off-by: Kirti Wankhede
> > > > > > > Reviewed-by: Neo Jia
> > > > > > > ---
> > > > > > > include/uapi/linux/vfio.h | 227
> > > > > > > ++
> > > > > > > 1 file changed, 227 insertions(+)
> > > > > > >
> > > > > > > diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
> > > > > > > index 9e843a147ead..d0021467af53 100644
> > > > > > > --- a/include/uapi/linux/vfio.h
> > > > > > > +++ b/include/uapi/linux/vfio.h
> > > > > > > @@ -305,6 +305,7 @@ struct vfio_region_info_cap_type {
> > > > > > > #define VFIO_REGION_TYPE_PCI_VENDOR_MASK (0x)
> > > > > > > #define VFIO_REGION_TYPE_GFX(1)
> > > > > > > #define VFIO_REGION_TYPE_CCW (2)
> > > > > > > +#define VFIO_REGION_TYPE_MIGRATION (3)
> > > > > > >
> > > > > > > /* sub-types for VFIO_REGION_TYPE_PCI_* */
> > > > > > >
> > > > > > > @@ -379,6 +380,232 @@ struct vfio_region_gfx_edid {
> > > > > > > /* sub-types for VFIO_REGION_TYPE_CCW */
> > > > > > > #define VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD(1)
> > > > > > >
> > > > > > > +/* sub-types for VFIO_REGION_TYPE_MIGRATION */
> > > > > > > +#define VFIO_REGION_SUBTYPE_MIGRATION (1)
> > > > > > > +
> > > > > > > +/*
> > > > > > > + * The structure vfio_device_migration_info is placed at the 0th
> > > > > > > offset of
> > > > > > > + * the VFIO_REGION_SUBTYPE_MIGRATION region to get and set VFIO
> > > > > > > device related
> > > > > > > + * migration information. Field accesses from this structure are
> > > > > > > only supported
> > > > > > > + * at their native width and alignment. Otherwise, the result is
> > > > > > > undefined and
> > > > > > > + * vendor drivers should return an error.
> > > > > > > + *
> > > > > > > + * device_state: (read/write)
> > > > > > > + * - The user application writes to this field to inform
> > > > > > > the vendor driver
> > > > > > > + *about the device state to be transitioned to.
> > > > > > > + * - The vendor driver should take the necessary actions to
> > > > > > > change the
> > > > > > > + *device state. After successful transition to a given
> > > > > > > state, the
> > > > > > > + *vendor driver should return success on
> > > > > > > write(device_state, state)
> > > > > > > + *system call. If the device state transition fails, the
> > > > > > > vendor driver
> > > > > > > + *should return an appropriate -errno for the fault
> > > > > > > condition.
> > > > > > > + * - On the user application side, if the device state
> > > > > > > transition fails,
> > > > > > > + * that is, if write(device_state, state) returns an
> > > > > > > error, read
> > > > > > > + * device_state again to determine the current state of
> > > > > > > the device from
> > > > > > > + * the vendor driver.
> > > > > > > + * - The vendor driver should return previous state of the
> > > > > > > device unless
> > > > > > > + *the vendor driver has encountered an internal error,
> > > > > > > in which case
> > > > > > > + *the vendor driver may report the device_state
> > > > > > > VFIO_DEVICE_STATE_ERROR.
> > > > > > > + * - The user application must use the device reset ioctl
> > > > > > > to recover the
> > > > > > > + *device from VFIO_DEVICE_STATE_ERROR state. If the
> > > > > > > device is
> > > > > > > + *indicated to be in a valid device state by reading
> > > > > > > device_state, the
> > > > > > > + *user application may attempt to transition the device
> > > > > > > to any valid
> > > > > > > + *state reachable from the current state or terminate
> > > > > > > itself.
> > > > > > > + *
> > > > > > > + * device_state consists of 3 bits:
> > > > > > > + * - If bit 0 is set, it indicates the _RUNNING
Re: [PATCH v14 Kernel 1/7] vfio: KABI for migration interface for device state
On Thu, 19 Mar 2020 21:30:39 -0400
Yan Zhao wrote:
> On Thu, Mar 19, 2020 at 09:09:21PM +0800, Alex Williamson wrote:
> > On Thu, 19 Mar 2020 01:05:54 -0400
> > Yan Zhao wrote:
> >
> > > On Thu, Mar 19, 2020 at 11:49:26AM +0800, Alex Williamson wrote:
> > > > On Wed, 18 Mar 2020 21:17:03 -0400
> > > > Yan Zhao wrote:
> > > >
> > > > > On Thu, Mar 19, 2020 at 03:41:08AM +0800, Kirti Wankhede wrote:
> > > > > > - Defined MIGRATION region type and sub-type.
> > > > > >
> > > > > > - Defined vfio_device_migration_info structure which will be placed
> > > > > > at the
> > > > > > 0th offset of migration region to get/set VFIO device related
> > > > > > information. Defined members of structure and usage on read/write
> > > > > > access.
> > > > > >
> > > > > > - Defined device states and state transition details.
> > > > > >
> > > > > > - Defined sequence to be followed while saving and resuming VFIO
> > > > > > device.
> > > > > >
> > > > > > Signed-off-by: Kirti Wankhede
> > > > > > Reviewed-by: Neo Jia
> > > > > > ---
> > > > > > include/uapi/linux/vfio.h | 227
> > > > > > ++
> > > > > > 1 file changed, 227 insertions(+)
> > > > > >
> > > > > > diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
> > > > > > index 9e843a147ead..d0021467af53 100644
> > > > > > --- a/include/uapi/linux/vfio.h
> > > > > > +++ b/include/uapi/linux/vfio.h
> > > > > > @@ -305,6 +305,7 @@ struct vfio_region_info_cap_type {
> > > > > > #define VFIO_REGION_TYPE_PCI_VENDOR_MASK (0x)
> > > > > > #define VFIO_REGION_TYPE_GFX(1)
> > > > > > #define VFIO_REGION_TYPE_CCW (2)
> > > > > > +#define VFIO_REGION_TYPE_MIGRATION (3)
> > > > > >
> > > > > > /* sub-types for VFIO_REGION_TYPE_PCI_* */
> > > > > >
> > > > > > @@ -379,6 +380,232 @@ struct vfio_region_gfx_edid {
> > > > > > /* sub-types for VFIO_REGION_TYPE_CCW */
> > > > > > #define VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD (1)
> > > > > >
> > > > > > +/* sub-types for VFIO_REGION_TYPE_MIGRATION */
> > > > > > +#define VFIO_REGION_SUBTYPE_MIGRATION (1)
> > > > > > +
> > > > > > +/*
> > > > > > + * The structure vfio_device_migration_info is placed at the 0th
> > > > > > offset of
> > > > > > + * the VFIO_REGION_SUBTYPE_MIGRATION region to get and set VFIO
> > > > > > device related
> > > > > > + * migration information. Field accesses from this structure are
> > > > > > only supported
> > > > > > + * at their native width and alignment. Otherwise, the result is
> > > > > > undefined and
> > > > > > + * vendor drivers should return an error.
> > > > > > + *
> > > > > > + * device_state: (read/write)
> > > > > > + * - The user application writes to this field to inform the
> > > > > > vendor driver
> > > > > > + *about the device state to be transitioned to.
> > > > > > + * - The vendor driver should take the necessary actions to
> > > > > > change the
> > > > > > + *device state. After successful transition to a given
> > > > > > state, the
> > > > > > + *vendor driver should return success on
> > > > > > write(device_state, state)
> > > > > > + *system call. If the device state transition fails, the
> > > > > > vendor driver
> > > > > > + *should return an appropriate -errno for the fault
> > > > > > condition.
> > > > > > + * - On the user application side, if the device state
> > > > > > transition fails,
> > > > > > + * that is, if write(device_state, state) returns an error, read
> > > > > > + * device_state again to determine the current state of the
> > > > > > device from
> > > > > > + * the vendor driver.
> > > > > > + * - The vendor driver should return previous state of the
> > > > > > device unless
> > > > > > + *the vendor driver has encountered an internal error, in
> > > > > > which case
> > > > > > + *the vendor driver may report the device_state
> > > > > > VFIO_DEVICE_STATE_ERROR.
> > > > > > + * - The user application must use the device reset ioctl to
> > > > > > recover the
> > > > > > + *device from VFIO_DEVICE_STATE_ERROR state. If the device
> > > > > > is
> > > > > > + *indicated to be in a valid device state by reading
> > > > > > device_state, the
> > > > > > + *user application may attempt to transition the device to
> > > > > > any valid
> > > > > > + *state reachable from the current state or terminate
> > > > > > itself.
> > > > > > + *
> > > > > > + * device_state consists of 3 bits:
> > > > > > + * - If bit 0 is set, it indicates the _RUNNING state. If bit
> > > > > > 0 is clear,
> > > > > > + *it indicates the _STOP state. When the device state is
> > > > > > changed to
> > > > > > + *_STOP, driver should stop the device before write()
> > > > > > returns.
> > > > > > + * - If bit 1 is set, it indicates the _SAVING
Re: [PATCH v14 Kernel 1/7] vfio: KABI for migration interface for device state
On Thu, Mar 19, 2020 at 09:09:21PM +0800, Alex Williamson wrote:
> On Thu, 19 Mar 2020 01:05:54 -0400
> Yan Zhao wrote:
>
> > On Thu, Mar 19, 2020 at 11:49:26AM +0800, Alex Williamson wrote:
> > > On Wed, 18 Mar 2020 21:17:03 -0400
> > > Yan Zhao wrote:
> > >
> > > > On Thu, Mar 19, 2020 at 03:41:08AM +0800, Kirti Wankhede wrote:
> > > > > - Defined MIGRATION region type and sub-type.
> > > > >
> > > > > - Defined vfio_device_migration_info structure which will be placed
> > > > > at the
> > > > > 0th offset of migration region to get/set VFIO device related
> > > > > information. Defined members of structure and usage on read/write
> > > > > access.
> > > > >
> > > > > - Defined device states and state transition details.
> > > > >
> > > > > - Defined sequence to be followed while saving and resuming VFIO
> > > > > device.
> > > > >
> > > > > Signed-off-by: Kirti Wankhede
> > > > > Reviewed-by: Neo Jia
> > > > > ---
> > > > > include/uapi/linux/vfio.h | 227
> > > > > ++
> > > > > 1 file changed, 227 insertions(+)
> > > > >
> > > > > diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
> > > > > index 9e843a147ead..d0021467af53 100644
> > > > > --- a/include/uapi/linux/vfio.h
> > > > > +++ b/include/uapi/linux/vfio.h
> > > > > @@ -305,6 +305,7 @@ struct vfio_region_info_cap_type {
> > > > > #define VFIO_REGION_TYPE_PCI_VENDOR_MASK (0x)
> > > > > #define VFIO_REGION_TYPE_GFX(1)
> > > > > #define VFIO_REGION_TYPE_CCW (2)
> > > > > +#define VFIO_REGION_TYPE_MIGRATION (3)
> > > > >
> > > > > /* sub-types for VFIO_REGION_TYPE_PCI_* */
> > > > >
> > > > > @@ -379,6 +380,232 @@ struct vfio_region_gfx_edid {
> > > > > /* sub-types for VFIO_REGION_TYPE_CCW */
> > > > > #define VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD(1)
> > > > >
> > > > > +/* sub-types for VFIO_REGION_TYPE_MIGRATION */
> > > > > +#define VFIO_REGION_SUBTYPE_MIGRATION (1)
> > > > > +
> > > > > +/*
> > > > > + * The structure vfio_device_migration_info is placed at the 0th
> > > > > offset of
> > > > > + * the VFIO_REGION_SUBTYPE_MIGRATION region to get and set VFIO
> > > > > device related
> > > > > + * migration information. Field accesses from this structure are
> > > > > only supported
> > > > > + * at their native width and alignment. Otherwise, the result is
> > > > > undefined and
> > > > > + * vendor drivers should return an error.
> > > > > + *
> > > > > + * device_state: (read/write)
> > > > > + * - The user application writes to this field to inform the
> > > > > vendor driver
> > > > > + *about the device state to be transitioned to.
> > > > > + * - The vendor driver should take the necessary actions to
> > > > > change the
> > > > > + *device state. After successful transition to a given
> > > > > state, the
> > > > > + *vendor driver should return success on write(device_state,
> > > > > state)
> > > > > + *system call. If the device state transition fails, the
> > > > > vendor driver
> > > > > + *should return an appropriate -errno for the fault
> > > > > condition.
> > > > > + * - On the user application side, if the device state
> > > > > transition fails,
> > > > > + * that is, if write(device_state, state) returns an error, read
> > > > > + * device_state again to determine the current state of the
> > > > > device from
> > > > > + * the vendor driver.
> > > > > + * - The vendor driver should return previous state of the
> > > > > device unless
> > > > > + *the vendor driver has encountered an internal error, in
> > > > > which case
> > > > > + *the vendor driver may report the device_state
> > > > > VFIO_DEVICE_STATE_ERROR.
> > > > > + * - The user application must use the device reset ioctl to
> > > > > recover the
> > > > > + *device from VFIO_DEVICE_STATE_ERROR state. If the device is
> > > > > + *indicated to be in a valid device state by reading
> > > > > device_state, the
> > > > > + *user application may attempt to transition the device to
> > > > > any valid
> > > > > + *state reachable from the current state or terminate itself.
> > > > > + *
> > > > > + * device_state consists of 3 bits:
> > > > > + * - If bit 0 is set, it indicates the _RUNNING state. If bit 0
> > > > > is clear,
> > > > > + *it indicates the _STOP state. When the device state is
> > > > > changed to
> > > > > + *_STOP, driver should stop the device before write()
> > > > > returns.
> > > > > + * - If bit 1 is set, it indicates the _SAVING state, which
> > > > > means that the
> > > > > + *driver should start gathering device state information
> > > > > that will be
> > > > > + *provided to the VFIO user application to save the device's
> > > > > state.
> > > > > + * - If bit 2 is set, it indicates
Re: [PATCH v14 Kernel 1/7] vfio: KABI for migration interface for device state
On Thu, 19 Mar 2020 01:05:54 -0400
Yan Zhao wrote:
> On Thu, Mar 19, 2020 at 11:49:26AM +0800, Alex Williamson wrote:
> > On Wed, 18 Mar 2020 21:17:03 -0400
> > Yan Zhao wrote:
> >
> > > On Thu, Mar 19, 2020 at 03:41:08AM +0800, Kirti Wankhede wrote:
> > > > - Defined MIGRATION region type and sub-type.
> > > >
> > > > - Defined vfio_device_migration_info structure which will be placed at
> > > > the
> > > > 0th offset of migration region to get/set VFIO device related
> > > > information. Defined members of structure and usage on read/write
> > > > access.
> > > >
> > > > - Defined device states and state transition details.
> > > >
> > > > - Defined sequence to be followed while saving and resuming VFIO device.
> > > >
> > > > Signed-off-by: Kirti Wankhede
> > > > Reviewed-by: Neo Jia
> > > > ---
> > > > include/uapi/linux/vfio.h | 227
> > > > ++
> > > > 1 file changed, 227 insertions(+)
> > > >
> > > > diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
> > > > index 9e843a147ead..d0021467af53 100644
> > > > --- a/include/uapi/linux/vfio.h
> > > > +++ b/include/uapi/linux/vfio.h
> > > > @@ -305,6 +305,7 @@ struct vfio_region_info_cap_type {
> > > > #define VFIO_REGION_TYPE_PCI_VENDOR_MASK (0x)
> > > > #define VFIO_REGION_TYPE_GFX(1)
> > > > #define VFIO_REGION_TYPE_CCW (2)
> > > > +#define VFIO_REGION_TYPE_MIGRATION (3)
> > > >
> > > > /* sub-types for VFIO_REGION_TYPE_PCI_* */
> > > >
> > > > @@ -379,6 +380,232 @@ struct vfio_region_gfx_edid {
> > > > /* sub-types for VFIO_REGION_TYPE_CCW */
> > > > #define VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD (1)
> > > >
> > > > +/* sub-types for VFIO_REGION_TYPE_MIGRATION */
> > > > +#define VFIO_REGION_SUBTYPE_MIGRATION (1)
> > > > +
> > > > +/*
> > > > + * The structure vfio_device_migration_info is placed at the 0th
> > > > offset of
> > > > + * the VFIO_REGION_SUBTYPE_MIGRATION region to get and set VFIO device
> > > > related
> > > > + * migration information. Field accesses from this structure are only
> > > > supported
> > > > + * at their native width and alignment. Otherwise, the result is
> > > > undefined and
> > > > + * vendor drivers should return an error.
> > > > + *
> > > > + * device_state: (read/write)
> > > > + * - The user application writes to this field to inform the
> > > > vendor driver
> > > > + *about the device state to be transitioned to.
> > > > + * - The vendor driver should take the necessary actions to
> > > > change the
> > > > + *device state. After successful transition to a given state,
> > > > the
> > > > + *vendor driver should return success on write(device_state,
> > > > state)
> > > > + *system call. If the device state transition fails, the
> > > > vendor driver
> > > > + *should return an appropriate -errno for the fault condition.
> > > > + * - On the user application side, if the device state transition
> > > > fails,
> > > > + * that is, if write(device_state, state) returns an error, read
> > > > + * device_state again to determine the current state of the
> > > > device from
> > > > + * the vendor driver.
> > > > + * - The vendor driver should return previous state of the device
> > > > unless
> > > > + *the vendor driver has encountered an internal error, in
> > > > which case
> > > > + *the vendor driver may report the device_state
> > > > VFIO_DEVICE_STATE_ERROR.
> > > > + * - The user application must use the device reset ioctl to
> > > > recover the
> > > > + *device from VFIO_DEVICE_STATE_ERROR state. If the device is
> > > > + *indicated to be in a valid device state by reading
> > > > device_state, the
> > > > + *user application may attempt to transition the device to any
> > > > valid
> > > > + *state reachable from the current state or terminate itself.
> > > > + *
> > > > + * device_state consists of 3 bits:
> > > > + * - If bit 0 is set, it indicates the _RUNNING state. If bit 0
> > > > is clear,
> > > > + *it indicates the _STOP state. When the device state is
> > > > changed to
> > > > + *_STOP, driver should stop the device before write() returns.
> > > > + * - If bit 1 is set, it indicates the _SAVING state, which means
> > > > that the
> > > > + *driver should start gathering device state information that
> > > > will be
> > > > + *provided to the VFIO user application to save the device's
> > > > state.
> > > > + * - If bit 2 is set, it indicates the _RESUMING state, which
> > > > means that
> > > > + *the driver should prepare to resume the device. Data
> > > > provided through
> > > > + *the migration region should be used to resume the device.
> > > > + * Bits 3 - 31 are reserved for future use. To preserve
Re: [PATCH v14 Kernel 1/7] vfio: KABI for migration interface for device state
On Thu, Mar 19, 2020 at 11:49:26AM +0800, Alex Williamson wrote:
> On Wed, 18 Mar 2020 21:17:03 -0400
> Yan Zhao wrote:
>
> > On Thu, Mar 19, 2020 at 03:41:08AM +0800, Kirti Wankhede wrote:
> > > - Defined MIGRATION region type and sub-type.
> > >
> > > - Defined vfio_device_migration_info structure which will be placed at the
> > > 0th offset of migration region to get/set VFIO device related
> > > information. Defined members of structure and usage on read/write
> > > access.
> > >
> > > - Defined device states and state transition details.
> > >
> > > - Defined sequence to be followed while saving and resuming VFIO device.
> > >
> > > Signed-off-by: Kirti Wankhede
> > > Reviewed-by: Neo Jia
> > > ---
> > > include/uapi/linux/vfio.h | 227
> > > ++
> > > 1 file changed, 227 insertions(+)
> > >
> > > diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
> > > index 9e843a147ead..d0021467af53 100644
> > > --- a/include/uapi/linux/vfio.h
> > > +++ b/include/uapi/linux/vfio.h
> > > @@ -305,6 +305,7 @@ struct vfio_region_info_cap_type {
> > > #define VFIO_REGION_TYPE_PCI_VENDOR_MASK (0x)
> > > #define VFIO_REGION_TYPE_GFX(1)
> > > #define VFIO_REGION_TYPE_CCW (2)
> > > +#define VFIO_REGION_TYPE_MIGRATION (3)
> > >
> > > /* sub-types for VFIO_REGION_TYPE_PCI_* */
> > >
> > > @@ -379,6 +380,232 @@ struct vfio_region_gfx_edid {
> > > /* sub-types for VFIO_REGION_TYPE_CCW */
> > > #define VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD(1)
> > >
> > > +/* sub-types for VFIO_REGION_TYPE_MIGRATION */
> > > +#define VFIO_REGION_SUBTYPE_MIGRATION (1)
> > > +
> > > +/*
> > > + * The structure vfio_device_migration_info is placed at the 0th offset
> > > of
> > > + * the VFIO_REGION_SUBTYPE_MIGRATION region to get and set VFIO device
> > > related
> > > + * migration information. Field accesses from this structure are only
> > > supported
> > > + * at their native width and alignment. Otherwise, the result is
> > > undefined and
> > > + * vendor drivers should return an error.
> > > + *
> > > + * device_state: (read/write)
> > > + * - The user application writes to this field to inform the vendor
> > > driver
> > > + *about the device state to be transitioned to.
> > > + * - The vendor driver should take the necessary actions to change
> > > the
> > > + *device state. After successful transition to a given state, the
> > > + *vendor driver should return success on write(device_state,
> > > state)
> > > + *system call. If the device state transition fails, the vendor
> > > driver
> > > + *should return an appropriate -errno for the fault condition.
> > > + * - On the user application side, if the device state transition
> > > fails,
> > > + * that is, if write(device_state, state) returns an error, read
> > > + * device_state again to determine the current state of the
> > > device from
> > > + * the vendor driver.
> > > + * - The vendor driver should return previous state of the device
> > > unless
> > > + *the vendor driver has encountered an internal error, in which
> > > case
> > > + *the vendor driver may report the device_state
> > > VFIO_DEVICE_STATE_ERROR.
> > > + * - The user application must use the device reset ioctl to
> > > recover the
> > > + *device from VFIO_DEVICE_STATE_ERROR state. If the device is
> > > + *indicated to be in a valid device state by reading
> > > device_state, the
> > > + *user application may attempt to transition the device to any
> > > valid
> > > + *state reachable from the current state or terminate itself.
> > > + *
> > > + * device_state consists of 3 bits:
> > > + * - If bit 0 is set, it indicates the _RUNNING state. If bit 0 is
> > > clear,
> > > + *it indicates the _STOP state. When the device state is changed
> > > to
> > > + *_STOP, driver should stop the device before write() returns.
> > > + * - If bit 1 is set, it indicates the _SAVING state, which means
> > > that the
> > > + *driver should start gathering device state information that
> > > will be
> > > + *provided to the VFIO user application to save the device's
> > > state.
> > > + * - If bit 2 is set, it indicates the _RESUMING state, which means
> > > that
> > > + *the driver should prepare to resume the device. Data provided
> > > through
> > > + *the migration region should be used to resume the device.
> > > + * Bits 3 - 31 are reserved for future use. To preserve them, the
> > > user
> > > + * application should perform a read-modify-write operation on this
> > > + * field when modifying the specified bits.
> > > + *
> > > + * +--- _RESUMING
> > > + * |+-- _SAVING
> > > + * ||+- _RUNNING
> > > + * |||
> > > + *
Re: [PATCH v14 Kernel 1/7] vfio: KABI for migration interface for device state
On Wed, 18 Mar 2020 21:17:03 -0400
Yan Zhao wrote:
> On Thu, Mar 19, 2020 at 03:41:08AM +0800, Kirti Wankhede wrote:
> > - Defined MIGRATION region type and sub-type.
> >
> > - Defined vfio_device_migration_info structure which will be placed at the
> > 0th offset of migration region to get/set VFIO device related
> > information. Defined members of structure and usage on read/write access.
> >
> > - Defined device states and state transition details.
> >
> > - Defined sequence to be followed while saving and resuming VFIO device.
> >
> > Signed-off-by: Kirti Wankhede
> > Reviewed-by: Neo Jia
> > ---
> > include/uapi/linux/vfio.h | 227
> > ++
> > 1 file changed, 227 insertions(+)
> >
> > diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
> > index 9e843a147ead..d0021467af53 100644
> > --- a/include/uapi/linux/vfio.h
> > +++ b/include/uapi/linux/vfio.h
> > @@ -305,6 +305,7 @@ struct vfio_region_info_cap_type {
> > #define VFIO_REGION_TYPE_PCI_VENDOR_MASK (0x)
> > #define VFIO_REGION_TYPE_GFX(1)
> > #define VFIO_REGION_TYPE_CCW (2)
> > +#define VFIO_REGION_TYPE_MIGRATION (3)
> >
> > /* sub-types for VFIO_REGION_TYPE_PCI_* */
> >
> > @@ -379,6 +380,232 @@ struct vfio_region_gfx_edid {
> > /* sub-types for VFIO_REGION_TYPE_CCW */
> > #define VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD (1)
> >
> > +/* sub-types for VFIO_REGION_TYPE_MIGRATION */
> > +#define VFIO_REGION_SUBTYPE_MIGRATION (1)
> > +
> > +/*
> > + * The structure vfio_device_migration_info is placed at the 0th offset of
> > + * the VFIO_REGION_SUBTYPE_MIGRATION region to get and set VFIO device
> > related
> > + * migration information. Field accesses from this structure are only
> > supported
> > + * at their native width and alignment. Otherwise, the result is undefined
> > and
> > + * vendor drivers should return an error.
> > + *
> > + * device_state: (read/write)
> > + * - The user application writes to this field to inform the vendor
> > driver
> > + *about the device state to be transitioned to.
> > + * - The vendor driver should take the necessary actions to change the
> > + *device state. After successful transition to a given state, the
> > + *vendor driver should return success on write(device_state, state)
> > + *system call. If the device state transition fails, the vendor
> > driver
> > + *should return an appropriate -errno for the fault condition.
> > + * - On the user application side, if the device state transition
> > fails,
> > + * that is, if write(device_state, state) returns an error, read
> > + * device_state again to determine the current state of the device from
> > + * the vendor driver.
> > + * - The vendor driver should return previous state of the device
> > unless
> > + *the vendor driver has encountered an internal error, in which
> > case
> > + *the vendor driver may report the device_state
> > VFIO_DEVICE_STATE_ERROR.
> > + * - The user application must use the device reset ioctl to recover
> > the
> > + *device from VFIO_DEVICE_STATE_ERROR state. If the device is
> > + *indicated to be in a valid device state by reading device_state,
> > the
> > + *user application may attempt to transition the device to any
> > valid
> > + *state reachable from the current state or terminate itself.
> > + *
> > + * device_state consists of 3 bits:
> > + * - If bit 0 is set, it indicates the _RUNNING state. If bit 0 is
> > clear,
> > + *it indicates the _STOP state. When the device state is changed to
> > + *_STOP, driver should stop the device before write() returns.
> > + * - If bit 1 is set, it indicates the _SAVING state, which means
> > that the
> > + *driver should start gathering device state information that will
> > be
> > + *provided to the VFIO user application to save the device's state.
> > + * - If bit 2 is set, it indicates the _RESUMING state, which means
> > that
> > + *the driver should prepare to resume the device. Data provided
> > through
> > + *the migration region should be used to resume the device.
> > + * Bits 3 - 31 are reserved for future use. To preserve them, the user
> > + * application should perform a read-modify-write operation on this
> > + * field when modifying the specified bits.
> > + *
> > + * +--- _RESUMING
> > + * |+-- _SAVING
> > + * ||+- _RUNNING
> > + * |||
> > + * 000b => Device Stopped, not saving or resuming
> > + * 001b => Device running, which is the default state
> > + * 010b => Stop the device & save the device state, stop-and-copy state
> > + * 011b => Device running and save the device state, pre-copy state
> > + * 100b => Device stopped and the device state is resuming
> > + * 101b => Invalid
Re: [PATCH v14 Kernel 1/7] vfio: KABI for migration interface for device state
On Thu, Mar 19, 2020 at 03:41:08AM +0800, Kirti Wankhede wrote:
> - Defined MIGRATION region type and sub-type.
>
> - Defined vfio_device_migration_info structure which will be placed at the
> 0th offset of migration region to get/set VFIO device related
> information. Defined members of structure and usage on read/write access.
>
> - Defined device states and state transition details.
>
> - Defined sequence to be followed while saving and resuming VFIO device.
>
> Signed-off-by: Kirti Wankhede
> Reviewed-by: Neo Jia
> ---
> include/uapi/linux/vfio.h | 227
> ++
> 1 file changed, 227 insertions(+)
>
> diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
> index 9e843a147ead..d0021467af53 100644
> --- a/include/uapi/linux/vfio.h
> +++ b/include/uapi/linux/vfio.h
> @@ -305,6 +305,7 @@ struct vfio_region_info_cap_type {
> #define VFIO_REGION_TYPE_PCI_VENDOR_MASK (0x)
> #define VFIO_REGION_TYPE_GFX(1)
> #define VFIO_REGION_TYPE_CCW (2)
> +#define VFIO_REGION_TYPE_MIGRATION (3)
>
> /* sub-types for VFIO_REGION_TYPE_PCI_* */
>
> @@ -379,6 +380,232 @@ struct vfio_region_gfx_edid {
> /* sub-types for VFIO_REGION_TYPE_CCW */
> #define VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD(1)
>
> +/* sub-types for VFIO_REGION_TYPE_MIGRATION */
> +#define VFIO_REGION_SUBTYPE_MIGRATION (1)
> +
> +/*
> + * The structure vfio_device_migration_info is placed at the 0th offset of
> + * the VFIO_REGION_SUBTYPE_MIGRATION region to get and set VFIO device
> related
> + * migration information. Field accesses from this structure are only
> supported
> + * at their native width and alignment. Otherwise, the result is undefined
> and
> + * vendor drivers should return an error.
> + *
> + * device_state: (read/write)
> + * - The user application writes to this field to inform the vendor
> driver
> + *about the device state to be transitioned to.
> + * - The vendor driver should take the necessary actions to change the
> + *device state. After successful transition to a given state, the
> + *vendor driver should return success on write(device_state, state)
> + *system call. If the device state transition fails, the vendor
> driver
> + *should return an appropriate -errno for the fault condition.
> + * - On the user application side, if the device state transition fails,
> + * that is, if write(device_state, state) returns an error, read
> + * device_state again to determine the current state of the device from
> + * the vendor driver.
> + * - The vendor driver should return previous state of the device unless
> + *the vendor driver has encountered an internal error, in which case
> + *the vendor driver may report the device_state
> VFIO_DEVICE_STATE_ERROR.
> + * - The user application must use the device reset ioctl to recover the
> + *device from VFIO_DEVICE_STATE_ERROR state. If the device is
> + *indicated to be in a valid device state by reading device_state,
> the
> + *user application may attempt to transition the device to any valid
> + *state reachable from the current state or terminate itself.
> + *
> + * device_state consists of 3 bits:
> + * - If bit 0 is set, it indicates the _RUNNING state. If bit 0 is
> clear,
> + *it indicates the _STOP state. When the device state is changed to
> + *_STOP, driver should stop the device before write() returns.
> + * - If bit 1 is set, it indicates the _SAVING state, which means that
> the
> + *driver should start gathering device state information that will be
> + *provided to the VFIO user application to save the device's state.
> + * - If bit 2 is set, it indicates the _RESUMING state, which means that
> + *the driver should prepare to resume the device. Data provided
> through
> + *the migration region should be used to resume the device.
> + * Bits 3 - 31 are reserved for future use. To preserve them, the user
> + * application should perform a read-modify-write operation on this
> + * field when modifying the specified bits.
> + *
> + * +--- _RESUMING
> + * |+-- _SAVING
> + * ||+- _RUNNING
> + * |||
> + * 000b => Device Stopped, not saving or resuming
> + * 001b => Device running, which is the default state
> + * 010b => Stop the device & save the device state, stop-and-copy state
> + * 011b => Device running and save the device state, pre-copy state
> + * 100b => Device stopped and the device state is resuming
> + * 101b => Invalid state
> + * 110b => Error state
> + * 111b => Invalid state
> + *
> + * State transitions:
> + *
> + * _RESUMING _RUNNINGPre-copyStop-and-copy _STOP
> + *(100b) (001b) (011b)(010b) (000b)
> + * 0. Running or default
[PATCH v14 Kernel 1/7] vfio: KABI for migration interface for device state
- Defined MIGRATION region type and sub-type.
- Defined vfio_device_migration_info structure which will be placed at the
0th offset of migration region to get/set VFIO device related
information. Defined members of structure and usage on read/write access.
- Defined device states and state transition details.
- Defined sequence to be followed while saving and resuming VFIO device.
Signed-off-by: Kirti Wankhede
Reviewed-by: Neo Jia
---
include/uapi/linux/vfio.h | 227 ++
1 file changed, 227 insertions(+)
diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
index 9e843a147ead..d0021467af53 100644
--- a/include/uapi/linux/vfio.h
+++ b/include/uapi/linux/vfio.h
@@ -305,6 +305,7 @@ struct vfio_region_info_cap_type {
#define VFIO_REGION_TYPE_PCI_VENDOR_MASK (0x)
#define VFIO_REGION_TYPE_GFX(1)
#define VFIO_REGION_TYPE_CCW (2)
+#define VFIO_REGION_TYPE_MIGRATION (3)
/* sub-types for VFIO_REGION_TYPE_PCI_* */
@@ -379,6 +380,232 @@ struct vfio_region_gfx_edid {
/* sub-types for VFIO_REGION_TYPE_CCW */
#define VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD (1)
+/* sub-types for VFIO_REGION_TYPE_MIGRATION */
+#define VFIO_REGION_SUBTYPE_MIGRATION (1)
+
+/*
+ * The structure vfio_device_migration_info is placed at the 0th offset of
+ * the VFIO_REGION_SUBTYPE_MIGRATION region to get and set VFIO device related
+ * migration information. Field accesses from this structure are only supported
+ * at their native width and alignment. Otherwise, the result is undefined and
+ * vendor drivers should return an error.
+ *
+ * device_state: (read/write)
+ * - The user application writes to this field to inform the vendor driver
+ *about the device state to be transitioned to.
+ * - The vendor driver should take the necessary actions to change the
+ *device state. After successful transition to a given state, the
+ *vendor driver should return success on write(device_state, state)
+ *system call. If the device state transition fails, the vendor driver
+ *should return an appropriate -errno for the fault condition.
+ * - On the user application side, if the device state transition fails,
+ * that is, if write(device_state, state) returns an error, read
+ * device_state again to determine the current state of the device from
+ * the vendor driver.
+ * - The vendor driver should return previous state of the device unless
+ *the vendor driver has encountered an internal error, in which case
+ *the vendor driver may report the device_state
VFIO_DEVICE_STATE_ERROR.
+ * - The user application must use the device reset ioctl to recover the
+ *device from VFIO_DEVICE_STATE_ERROR state. If the device is
+ *indicated to be in a valid device state by reading device_state, the
+ *user application may attempt to transition the device to any valid
+ *state reachable from the current state or terminate itself.
+ *
+ * device_state consists of 3 bits:
+ * - If bit 0 is set, it indicates the _RUNNING state. If bit 0 is clear,
+ *it indicates the _STOP state. When the device state is changed to
+ *_STOP, driver should stop the device before write() returns.
+ * - If bit 1 is set, it indicates the _SAVING state, which means that the
+ *driver should start gathering device state information that will be
+ *provided to the VFIO user application to save the device's state.
+ * - If bit 2 is set, it indicates the _RESUMING state, which means that
+ *the driver should prepare to resume the device. Data provided through
+ *the migration region should be used to resume the device.
+ * Bits 3 - 31 are reserved for future use. To preserve them, the user
+ * application should perform a read-modify-write operation on this
+ * field when modifying the specified bits.
+ *
+ * +--- _RESUMING
+ * |+-- _SAVING
+ * ||+- _RUNNING
+ * |||
+ * 000b => Device Stopped, not saving or resuming
+ * 001b => Device running, which is the default state
+ * 010b => Stop the device & save the device state, stop-and-copy state
+ * 011b => Device running and save the device state, pre-copy state
+ * 100b => Device stopped and the device state is resuming
+ * 101b => Invalid state
+ * 110b => Error state
+ * 111b => Invalid state
+ *
+ * State transitions:
+ *
+ * _RESUMING _RUNNINGPre-copyStop-and-copy _STOP
+ *(100b) (001b) (011b)(010b) (000b)
+ * 0. Running or default state
+ * |
+ *
+ * 1. Normal Shutdown (optional)
+ * |->|
+ *
+ * 2. Save the state or suspend
+ * |->|-->|
+ *
+ * 3.
