On 9/14/2023 7:34 PM, Michael S. Tsirkin wrote:
On Wed, Sep 06, 2023 at 04:16:34PM +0800, Zhu Lingshan wrote:
This patch introduces a new status bit in the device status: SUSPEND.
This SUSPEND bit can be used by the driver to suspend a device,
in order to stabilize the device states and virtqueue states.
Its main use case is live migration.
Signed-off-by: Zhu Lingshan <lingshan....@intel.com>
Signed-off-by: Jason Wang <jasow...@redhat.com>
Signed-off-by: Eugenio Pérez <epere...@redhat.com>
---
content.tex | 31 +++++++++++++++++++++++++++++++
1 file changed, 31 insertions(+)
diff --git a/content.tex b/content.tex
index 0e492cd..0fab537 100644
--- a/content.tex
+++ b/content.tex
@@ -47,6 +47,9 @@ \section{\field{Device Status} Field}\label{sec:Basic
Facilities of a Virtio Dev
\item[DRIVER_OK (4)] Indicates that the driver is set up and ready to
drive the device.
+\item[SUSPEND (16)] When VIRTIO_F_SUSPEND is negotiated, indicates that the
+ device has been suspended by the driver.
+
\item[DEVICE_NEEDS_RESET (64)] Indicates that the device has experienced
an error from which it can't recover.
\end{description}
@@ -73,6 +76,10 @@ \section{\field{Device Status} Field}\label{sec:Basic
Facilities of a Virtio Dev
recover by issuing a reset.
\end{note}
+The driver SHOULD NOT set SUSPEND if FEATURES_OK is not set.
+
+When setting SUSPEND, the driver MUST re-read \field{device status} to ensure
the SUSPEND bit is set.
+
\devicenormative{\subsection}{Device Status Field}{Basic Facilities of a
Virtio Device / Device Status Field}
The device MUST NOT consume buffers or send any used buffer
@@ -82,6 +89,26 @@ \section{\field{Device Status} Field}\label{sec:Basic
Facilities of a Virtio Dev
that a reset is needed. If DRIVER_OK is set, after it sets
DEVICE_NEEDS_RESET, the device
MUST send a device configuration change notification to the driver.
+The device MUST ignore SUSPEND if FEATURES_OK is not set.
+
+The device MUST ignore SUSPEND if VIRTIO_F_SUSPEND is not negotiated.
why? let's just forbid driver from setting it.
OK
+
+The device SHOULD allow settings to \field{device status} even when SUSPEND is
set.
+
+If VIRTIO_F_SUSPEND is negotiated and SUSPEND is set, the device SHOULD clear
SUSPEND
+and resumes operation upon DRIVER_OK.
+
sorry what?
In case of a failed or cancelled Live Migration, the device needs to
resume operation.
However the spec forbids the driver to clear a device status bit, so
re-writing
DRIVER_OK is expected to clear SUSPEND and the device resume operation.
+If VIRTIO_F_SUSPEND is negotiated, when the driver sets SUSPEND,
+the device SHOULD perform the following actions before presenting SUSPEND bit
in the \field{device status}:
+
+\begin{itemize}
+\item Stop consuming buffers of any virtqueues and mark all finished
descritors as used.
+\item Wait until all descriptors that being processed to finish and mark them
as used.
+\item Flush all used buffer and send used buffer notifications to the driver.
flush how?
This is device-type-specific, and we will include tracking inflight
descriptors(buffers) in V2.
+\item Record Virtqueue State of each enabled virtqueue, see section
\ref{sec:Virtqueues / Virtqueue State}
record where?
This is transport specific, for PCI, patch 5 introduces two new fields
for avail and used state
+\item Pause its operation except \field{device status} and preserve
configurations in its Device Configuration Space, see \ref{sec:Basic Facilities
of a Virtio Device / Device Configuration Space}
pause in what sense? completely? this does not seem realistic.
e.g. pci express link has to stay active or device will die.
only pause virtio, I will rephrase the sentence as "pause its virtio
operation".
Others like PCI link in the example is out of the spec and we don't need
to migrate them.
also, presumably here it is except a bunch of other fields.
e.g. what about queue select and all related queue fields?
For now they are forbidden.
As SiWei suggested, we will introduce a new feature bit to control whether
allowing resetting a VQ after SUSPEND. We can use more feature bits if
there are requirements to perform anything after SUSPEND. But for now
they are forbidden.
+\end{itemize}
+
\section{Feature Bits}\label{sec:Basic Facilities of a Virtio Device /
Feature Bits}
Each virtio device offers all the features it understands. During
@@ -937,6 +964,10 @@ \chapter{Reserved Feature Bits}\label{sec:Reserved Feature
Bits}
\ref{devicenormative:Basic Facilities of a Virtio Device / Feature
Bits} for
handling features reserved for future use.
+ \item[VIRTIO_F_SUSPEND(42)] This feature indicates that the driver can
+ SUSPEND the device.
+ See \ref{sec:Basic Facilities of a Virtio Device / Device Status Field}.
+
\end{description}
\drivernormative{\section}{Reserved Feature Bits}{Reserved Feature Bits}
--
2.35.3
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org
