Re: [PATCH-for-8.0] block/dmg: Ignore C99 prototype declaration mismatch from
Am 27.03.23 um 23:09 schrieb Paolo Bonzini: Il lun 27 mar 2023, 20:58 Philippe Mathieu-Daudé ha scritto: > The warning can also be suppressed if the build uses `-isystem > /opt/homebrew/include` instead of `-I/opt/homebrew/include` as I just > have tested. Is that option added by QEMU's configure or meson.build? Or is it added by homebrew? The fact that /opt/homebrew/include it isn't considered a system seems to be a homebrew decision. IIUC by design meson only allows including *relative* directories, and manage the system ones: https://mesonbuild.com/Include-directories.html That's for includes that are part of QEMU. Meson has as_system for dependency objects (https://mesonbuild.com/Reference-manual_returned_dep.html) but lzfse doesn't have a .pc file, its detection has to be done by hand. Paolo > If we can find a solution how to implement that I thing it would look > nicer. Technically the patch looks good. > > Reviewed-by: Stefan Weil Thanks! Typically I configure the build on macOS with `./configure --extra-cflags=-I/opt/homebrew/include --extra-ldflags=-L/opt/homebrew/lib --disable-werror`. With that configuration I get the two warnings for lzfse.h. When I use `./configure '--extra-cflags=-isystem /opt/homebrew/include' --extra-ldflags=-L/opt/homebrew/lib --disable-werror` instead, I get no compiler warnings (and `--disable-werror` could be ommitted). So at least for macOS with Homebrew in /opt/homebrew (M1 / M2 Macs) the patch is not needed when the right configure options (`--extra-cflags`) were used. Stefan
Re: [PATCH for-8.0] nbd/server: Request TCP_NODELAY
On Tue, Mar 28, 2023 at 12:42:59AM +0200, Florian Westphal wrote: > Eric Blake wrote: > > Nagle's algorithm adds latency in order to reduce network packet > > overhead on small packets. But when we are already using corking to > > merge smaller packets into transactional requests, the extra delay > > from TCP defaults just gets in the way. > > > > For reference, qemu as an NBD client already requests TCP_NODELAY (see > > nbd_connect() in nbd/client-connection.c); as does libnbd as a client > > [1], and nbdkit as a server [2]. > > > > [1] > > https://gitlab.com/nbdkit/libnbd/-/blob/a48a1142/generator/states-connect.c#L39 > > [2] https://gitlab.com/nbdkit/nbdkit/-/blob/45b72f5b/server/sockets.c#L430 > > > > CC: Florian Westphal > > Signed-off-by: Eric Blake > > --- > > nbd/server.c | 1 + > > 1 file changed, 1 insertion(+) > > > > diff --git a/nbd/server.c b/nbd/server.c > > index a4750e41880..976223860bf 100644 > > --- a/nbd/server.c > > +++ b/nbd/server.c > > @@ -2755,6 +2755,7 @@ void nbd_client_new(QIOChannelSocket *sioc, > > } > > client->tlsauthz = g_strdup(tlsauthz); > > client->sioc = sioc; > > +qio_channel_set_delay(QIO_CHANNEL(cioc), false); > > ../nbd/server.c: In function 'nbd_client_new': > ../nbd/server.c:2763:39: error: 'cioc' undeclared (first use in this > function); did you mean 'sioc'? > > Other than that this looks good to me. Arrgh. Bitten by hitting send before saving the edits in my buffer. Yes, the obvious fix is needed and intended. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Re: [PATCH for-8.0] nbd/server: Request TCP_NODELAY
Eric Blake wrote: > Nagle's algorithm adds latency in order to reduce network packet > overhead on small packets. But when we are already using corking to > merge smaller packets into transactional requests, the extra delay > from TCP defaults just gets in the way. > > For reference, qemu as an NBD client already requests TCP_NODELAY (see > nbd_connect() in nbd/client-connection.c); as does libnbd as a client > [1], and nbdkit as a server [2]. > > [1] > https://gitlab.com/nbdkit/libnbd/-/blob/a48a1142/generator/states-connect.c#L39 > [2] https://gitlab.com/nbdkit/nbdkit/-/blob/45b72f5b/server/sockets.c#L430 > > CC: Florian Westphal > Signed-off-by: Eric Blake > --- > nbd/server.c | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/nbd/server.c b/nbd/server.c > index a4750e41880..976223860bf 100644 > --- a/nbd/server.c > +++ b/nbd/server.c > @@ -2755,6 +2755,7 @@ void nbd_client_new(QIOChannelSocket *sioc, > } > client->tlsauthz = g_strdup(tlsauthz); > client->sioc = sioc; > +qio_channel_set_delay(QIO_CHANNEL(cioc), false); ../nbd/server.c: In function 'nbd_client_new': ../nbd/server.c:2763:39: error: 'cioc' undeclared (first use in this function); did you mean 'sioc'? Other than that this looks good to me.
[PATCH for-8.0] block/export: fix assume_graph_lock() assertion failure
When I/O request parameters are validated for virtio-blk exports like
vhost-user-blk and vduse-blk, we call blk_get_geometry() from a
coroutine. This hits an assume_graph_lock() assertion failure.
Use blk_co_nb_sectors() instead and mark virtio_blk_sect_range_ok() with
coroutine_fn.
This assertion failure is triggered by any I/O to a vhost-user-blk
export.
Fixes: 8ab8140a04cf ("block: Mark bdrv_co_refresh_total_sectors() and callers
GRAPH_RDLOCK")
Cc: Kevin Wolf
Signed-off-by: Stefan Hajnoczi
---
block/export/virtio-blk-handler.c | 7 ---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/block/export/virtio-blk-handler.c
b/block/export/virtio-blk-handler.c
index 313666e8ab..2f729a9ce2 100644
--- a/block/export/virtio-blk-handler.c
+++ b/block/export/virtio-blk-handler.c
@@ -22,8 +22,9 @@ struct virtio_blk_inhdr {
unsigned char status;
};
-static bool virtio_blk_sect_range_ok(BlockBackend *blk, uint32_t block_size,
- uint64_t sector, size_t size)
+static bool coroutine_fn
+virtio_blk_sect_range_ok(BlockBackend *blk, uint32_t block_size,
+ uint64_t sector, size_t size)
{
uint64_t nb_sectors;
uint64_t total_sectors;
@@ -41,7 +42,7 @@ static bool virtio_blk_sect_range_ok(BlockBackend *blk,
uint32_t block_size,
if ((sector << VIRTIO_BLK_SECTOR_BITS) % block_size) {
return false;
}
-blk_get_geometry(blk, &total_sectors);
+total_sectors = blk_co_nb_sectors(blk);
if (sector > total_sectors || nb_sectors > total_sectors - sector) {
return false;
}
--
2.39.2
Re: [PATCH-for-8.0] block/dmg: Ignore C99 prototype declaration mismatch from
Il lun 27 mar 2023, 20:58 Philippe Mathieu-Daudé ha scritto: > > The warning can also be suppressed if the build uses `-isystem > > /opt/homebrew/include` instead of `-I/opt/homebrew/include` as I just > > have tested. > Is that option added by QEMU's configure or meson.build? Or is it added by homebrew? The fact that /opt/homebrew/include it isn't considered a system seems to be a homebrew decision. IIUC by design meson only allows including *relative* directories, > and manage the system ones: > https://mesonbuild.com/Include-directories.html That's for includes that are part of QEMU. Meson has as_system for dependency objects ( https://mesonbuild.com/Reference-manual_returned_dep.html) but lzfse doesn't have a .pc file, its detection has to be done by hand. Paolo > > If we can find a solution how to implement that I thing it would look > > nicer. Technically the patch looks good. > > > > Reviewed-by: Stefan Weil > > Thanks! > >
[PATCH for-8.0] nbd/server: Request TCP_NODELAY
Nagle's algorithm adds latency in order to reduce network packet overhead on small packets. But when we are already using corking to merge smaller packets into transactional requests, the extra delay from TCP defaults just gets in the way. For reference, qemu as an NBD client already requests TCP_NODELAY (see nbd_connect() in nbd/client-connection.c); as does libnbd as a client [1], and nbdkit as a server [2]. [1] https://gitlab.com/nbdkit/libnbd/-/blob/a48a1142/generator/states-connect.c#L39 [2] https://gitlab.com/nbdkit/nbdkit/-/blob/45b72f5b/server/sockets.c#L430 CC: Florian Westphal Signed-off-by: Eric Blake --- nbd/server.c | 1 + 1 file changed, 1 insertion(+) diff --git a/nbd/server.c b/nbd/server.c index a4750e41880..976223860bf 100644 --- a/nbd/server.c +++ b/nbd/server.c @@ -2755,6 +2755,7 @@ void nbd_client_new(QIOChannelSocket *sioc, } client->tlsauthz = g_strdup(tlsauthz); client->sioc = sioc; +qio_channel_set_delay(QIO_CHANNEL(cioc), false); object_ref(OBJECT(client->sioc)); client->ioc = QIO_CHANNEL(sioc); object_ref(OBJECT(client->ioc)); base-commit: e3debd5e7d0ce031356024878a0a18b9d109354a -- 2.39.2
Re: [PATCH-for-8.0] block/dmg: Ignore C99 prototype declaration mismatch from
+Marc-André & Paolo
On 27/3/23 19:08, Stefan Weil wrote:
Am 27.03.23 um 17:13 schrieb Philippe Mathieu-Daudé:
When liblzfe (Apple LZFSE compression library) is present
(for example installed via 'brew') on Darwin, QEMU build
fails as:
Has header "lzfse.h" : YES
Library lzfse found: YES
Dependencies
lzo support : NO
snappy support : NO
bzip2 support : YES
lzfse support : YES
zstd support : YES 1.5.2
User defined options
dmg : enabled
lzfse : enabled
[221/903] Compiling C object libblock.fa.p/block_dmg-lzfse.c.o
FAILED: libblock.fa.p/block_dmg-lzfse.c.o
/opt/homebrew/Cellar/lzfse/1.0/include/lzfse.h:56:43: error: this
function declaration is not a prototype [-Werror,-Wstrict-prototypes]
LZFSE_API size_t lzfse_encode_scratch_size();
^
void
/opt/homebrew/Cellar/lzfse/1.0/include/lzfse.h:94:43: error: this
function declaration is not a prototype [-Werror,-Wstrict-prototypes]
LZFSE_API size_t lzfse_decode_scratch_size();
^
void
2 errors generated.
ninja: build stopped: subcommand failed.
This issue has been reported in the lzfse project in 2016:
https://github.com/lzfse/lzfse/issues/3#issuecomment-226574719
Since the project seems unmaintained, simply ignore the
strict-prototypes warning check for the header,
similarly to how we deal with the GtkItemFactoryCallback
prototype from , indirectly included
by .
Cc: Julio Faracco
Signed-off-by: Philippe Mathieu-Daudé
---
block/dmg-lzfse.c | 5 +
1 file changed, 5 insertions(+)
diff --git a/block/dmg-lzfse.c b/block/dmg-lzfse.c
index 6798cf4fbf..0abc970bf6 100644
--- a/block/dmg-lzfse.c
+++ b/block/dmg-lzfse.c
@@ -23,7 +23,12 @@
*/
#include "qemu/osdep.h"
#include "dmg.h"
+
+/* Work around an -Wstrict-prototypes warning in LZFSE headers */
"Work around a -Wstrict-prototypes" ("a" instead of "an")?
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wstrict-prototypes"
#include
+#pragma GCC diagnostic pop
static int dmg_uncompress_lzfse_do(char *next_in, unsigned int
avail_in,
char *next_out, unsigned int
avail_out)
The warning can also be suppressed if the build uses `-isystem
/opt/homebrew/include` instead of `-I/opt/homebrew/include` as I just
have tested.
IIUC by design meson only allows including *relative* directories,
and manage the system ones:
https://mesonbuild.com/Include-directories.html
If we can find a solution how to implement that I thing it would look
nicer. Technically the patch looks good.
Reviewed-by: Stefan Weil
Thanks!
[PULL 0/2] hw/nvme fixes
From: Klaus Jensen Hi Peter, The following changes since commit e3debd5e7d0ce031356024878a0a18b9d109354a: Merge tag 'pull-request-2023-03-24' of https://gitlab.com/thuth/qemu into staging (2023-03-24 16:08:46 +) are available in the Git repository at: https://gitlab.com/birkelund/qemu.git tags/nvme-next-pull-request for you to fetch changes up to ca2a091802872b265bc6007a2d36276d51d8e4b3: hw/nvme: fix missing DNR on compare failure (2023-03-27 19:05:23 +0200) hw/nvme fixes Klaus Jensen (1): hw/nvme: fix missing DNR on compare failure Mateusz Kozlowski (1): hw/nvme: Change alignment in dma functions for nvme_blk_* hw/nvme/ctrl.c | 26 +- 1 file changed, 13 insertions(+), 13 deletions(-) -- 2.39.2
[PULL 2/2] hw/nvme: fix missing DNR on compare failure
From: Klaus Jensen
Even if the host is somehow using compare to do compare-and-write, the
host should be notified immediately about the compare failure and not
have to wait for the driver to potentially retry the command.
Fixes: 0a384f923f51 ("hw/block/nvme: add compare command")
Reported-by: Jim Harris
Signed-off-by: Klaus Jensen
---
hw/nvme/ctrl.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c
index 291009545f03..8b7be1420912 100644
--- a/hw/nvme/ctrl.c
+++ b/hw/nvme/ctrl.c
@@ -2378,7 +2378,7 @@ static void nvme_compare_mdata_cb(void *opaque, int ret)
for (bufp = buf; mbufp < end; bufp += ns->lbaf.ms, mbufp +=
ns->lbaf.ms) {
if (memcmp(bufp + pil, mbufp + pil, ns->lbaf.ms - pil)) {
-req->status = NVME_CMP_FAILURE;
+req->status = NVME_CMP_FAILURE | NVME_DNR;
goto out;
}
}
@@ -2387,7 +2387,7 @@ static void nvme_compare_mdata_cb(void *opaque, int ret)
}
if (memcmp(buf, ctx->mdata.bounce, ctx->mdata.iov.size)) {
-req->status = NVME_CMP_FAILURE;
+req->status = NVME_CMP_FAILURE | NVME_DNR;
goto out;
}
@@ -2436,7 +2436,7 @@ static void nvme_compare_data_cb(void *opaque, int ret)
}
if (memcmp(buf, ctx->data.bounce, ctx->data.iov.size)) {
-req->status = NVME_CMP_FAILURE;
+req->status = NVME_CMP_FAILURE | NVME_DNR;
goto out;
}
--
2.39.2
[PULL 1/2] hw/nvme: Change alignment in dma functions for nvme_blk_*
From: Mateusz Kozlowski
Since the nvme_blk_read/write are used by both the data and metadata
portions of the IO, it can't have the 512B alignment requirement.
Without this change any metadata transfer, which length isn't a multiple
of 512B and which is bigger than 512B, will result in only a partial
transfer.
Signed-off-by: Mateusz Kozlowski
Reviewed-by: Klaus Jensen
Signed-off-by: Klaus Jensen
---
hw/nvme/ctrl.c | 20 ++--
1 file changed, 10 insertions(+), 10 deletions(-)
diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c
index 49c1210fce2b..291009545f03 100644
--- a/hw/nvme/ctrl.c
+++ b/hw/nvme/ctrl.c
@@ -1434,26 +1434,26 @@ uint16_t nvme_bounce_mdata(NvmeCtrl *n, void *ptr,
uint32_t len,
}
static inline void nvme_blk_read(BlockBackend *blk, int64_t offset,
- BlockCompletionFunc *cb, NvmeRequest *req)
+ uint32_t align, BlockCompletionFunc *cb,
+ NvmeRequest *req)
{
assert(req->sg.flags & NVME_SG_ALLOC);
if (req->sg.flags & NVME_SG_DMA) {
-req->aiocb = dma_blk_read(blk, &req->sg.qsg, offset, BDRV_SECTOR_SIZE,
- cb, req);
+req->aiocb = dma_blk_read(blk, &req->sg.qsg, offset, align, cb, req);
} else {
req->aiocb = blk_aio_preadv(blk, offset, &req->sg.iov, 0, cb, req);
}
}
static inline void nvme_blk_write(BlockBackend *blk, int64_t offset,
- BlockCompletionFunc *cb, NvmeRequest *req)
+ uint32_t align, BlockCompletionFunc *cb,
+ NvmeRequest *req)
{
assert(req->sg.flags & NVME_SG_ALLOC);
if (req->sg.flags & NVME_SG_DMA) {
-req->aiocb = dma_blk_write(blk, &req->sg.qsg, offset, BDRV_SECTOR_SIZE,
- cb, req);
+req->aiocb = dma_blk_write(blk, &req->sg.qsg, offset, align, cb, req);
} else {
req->aiocb = blk_aio_pwritev(blk, offset, &req->sg.iov, 0, cb, req);
}
@@ -2207,10 +2207,10 @@ static void nvme_rw_cb(void *opaque, int ret)
}
if (req->cmd.opcode == NVME_CMD_READ) {
-return nvme_blk_read(blk, offset, nvme_rw_complete_cb, req);
+return nvme_blk_read(blk, offset, 1, nvme_rw_complete_cb, req);
}
-return nvme_blk_write(blk, offset, nvme_rw_complete_cb, req);
+return nvme_blk_write(blk, offset, 1, nvme_rw_complete_cb, req);
}
}
@@ -3437,7 +3437,7 @@ static uint16_t nvme_read(NvmeCtrl *n, NvmeRequest *req)
block_acct_start(blk_get_stats(blk), &req->acct, data_size,
BLOCK_ACCT_READ);
-nvme_blk_read(blk, data_offset, nvme_rw_cb, req);
+nvme_blk_read(blk, data_offset, BDRV_SECTOR_SIZE, nvme_rw_cb, req);
return NVME_NO_COMPLETE;
invalid:
@@ -3607,7 +3607,7 @@ static uint16_t nvme_do_write(NvmeCtrl *n, NvmeRequest
*req, bool append,
block_acct_start(blk_get_stats(blk), &req->acct, data_size,
BLOCK_ACCT_WRITE);
-nvme_blk_write(blk, data_offset, nvme_rw_cb, req);
+nvme_blk_write(blk, data_offset, BDRV_SECTOR_SIZE, nvme_rw_cb, req);
} else {
req->aiocb = blk_aio_pwrite_zeroes(blk, data_offset, data_size,
BDRV_REQ_MAY_UNMAP, nvme_rw_cb,
--
2.39.2
Re: [PATCH-for-8.0] block/dmg: Ignore C99 prototype declaration mismatch from
Am 27.03.23 um 17:13 schrieb Philippe Mathieu-Daudé:
When liblzfe (Apple LZFSE compression library) is present
(for example installed via 'brew') on Darwin, QEMU build
fails as:
Has header "lzfse.h" : YES
Library lzfse found: YES
Dependencies
lzo support : NO
snappy support : NO
bzip2 support: YES
lzfse support: YES
zstd support : YES 1.5.2
User defined options
dmg : enabled
lzfse: enabled
[221/903] Compiling C object libblock.fa.p/block_dmg-lzfse.c.o
FAILED: libblock.fa.p/block_dmg-lzfse.c.o
/opt/homebrew/Cellar/lzfse/1.0/include/lzfse.h:56:43: error: this function
declaration is not a prototype [-Werror,-Wstrict-prototypes]
LZFSE_API size_t lzfse_encode_scratch_size();
^
void
/opt/homebrew/Cellar/lzfse/1.0/include/lzfse.h:94:43: error: this function
declaration is not a prototype [-Werror,-Wstrict-prototypes]
LZFSE_API size_t lzfse_decode_scratch_size();
^
void
2 errors generated.
ninja: build stopped: subcommand failed.
This issue has been reported in the lzfse project in 2016:
https://github.com/lzfse/lzfse/issues/3#issuecomment-226574719
Since the project seems unmaintained, simply ignore the
strict-prototypes warning check for the header,
similarly to how we deal with the GtkItemFactoryCallback
prototype from , indirectly included
by .
Cc: Julio Faracco
Signed-off-by: Philippe Mathieu-Daudé
---
block/dmg-lzfse.c | 5 +
1 file changed, 5 insertions(+)
diff --git a/block/dmg-lzfse.c b/block/dmg-lzfse.c
index 6798cf4fbf..0abc970bf6 100644
--- a/block/dmg-lzfse.c
+++ b/block/dmg-lzfse.c
@@ -23,7 +23,12 @@
*/
#include "qemu/osdep.h"
#include "dmg.h"
+
+/* Work around an -Wstrict-prototypes warning in LZFSE headers */
"Work around a -Wstrict-prototypes" ("a" instead of "an")?
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wstrict-prototypes"
#include
+#pragma GCC diagnostic pop
static int dmg_uncompress_lzfse_do(char *next_in, unsigned int avail_in,
char *next_out, unsigned int avail_out)
The warning can also be suppressed if the build uses `-isystem
/opt/homebrew/include` instead of `-I/opt/homebrew/include` as I just
have tested.
If we can find a solution how to implement that I thing it would look
nicer. Technically the patch looks good.
Reviewed-by: Stefan Weil
Thanks,
Stefan
[PATCH-for-8.0] block/dmg: Ignore C99 prototype declaration mismatch from
When liblzfe (Apple LZFSE compression library) is present (for example installed via 'brew') on Darwin, QEMU build fails as: Has header "lzfse.h" : YES Library lzfse found: YES Dependencies lzo support : NO snappy support : NO bzip2 support: YES lzfse support: YES zstd support : YES 1.5.2 User defined options dmg : enabled lzfse: enabled [221/903] Compiling C object libblock.fa.p/block_dmg-lzfse.c.o FAILED: libblock.fa.p/block_dmg-lzfse.c.o /opt/homebrew/Cellar/lzfse/1.0/include/lzfse.h:56:43: error: this function declaration is not a prototype [-Werror,-Wstrict-prototypes] LZFSE_API size_t lzfse_encode_scratch_size(); ^ void /opt/homebrew/Cellar/lzfse/1.0/include/lzfse.h:94:43: error: this function declaration is not a prototype [-Werror,-Wstrict-prototypes] LZFSE_API size_t lzfse_decode_scratch_size(); ^ void 2 errors generated. ninja: build stopped: subcommand failed. This issue has been reported in the lzfse project in 2016: https://github.com/lzfse/lzfse/issues/3#issuecomment-226574719 Since the project seems unmaintained, simply ignore the strict-prototypes warning check for the header, similarly to how we deal with the GtkItemFactoryCallback prototype from , indirectly included by . Cc: Julio Faracco Signed-off-by: Philippe Mathieu-Daudé --- block/dmg-lzfse.c | 5 + 1 file changed, 5 insertions(+) diff --git a/block/dmg-lzfse.c b/block/dmg-lzfse.c index 6798cf4fbf..0abc970bf6 100644 --- a/block/dmg-lzfse.c +++ b/block/dmg-lzfse.c @@ -23,7 +23,12 @@ */ #include "qemu/osdep.h" #include "dmg.h" + +/* Work around an -Wstrict-prototypes warning in LZFSE headers */ +#pragma GCC diagnostic push +#pragma GCC diagnostic ignored "-Wstrict-prototypes" #include +#pragma GCC diagnostic pop static int dmg_uncompress_lzfse_do(char *next_in, unsigned int avail_in, char *next_out, unsigned int avail_out) -- 2.38.1
[PATCH v9 3/5] block: add accounting for zone append operation
Taking account of the new zone append write operation for zoned devices,
BLOCK_ACCT_ZONE_APPEND enum is introduced as other I/O request type (read,
write, flush).
Signed-off-by: Sam Li
---
block/qapi-sysemu.c| 11 ++
block/qapi.c | 18 ++
hw/block/virtio-blk.c | 4 +++
include/block/accounting.h | 1 +
qapi/block-core.json | 68 --
qapi/block.json| 4 +++
6 files changed, 95 insertions(+), 11 deletions(-)
diff --git a/block/qapi-sysemu.c b/block/qapi-sysemu.c
index 7bd7554150..cec3c1afb4 100644
--- a/block/qapi-sysemu.c
+++ b/block/qapi-sysemu.c
@@ -517,6 +517,7 @@ void qmp_block_latency_histogram_set(
bool has_boundaries, uint64List *boundaries,
bool has_boundaries_read, uint64List *boundaries_read,
bool has_boundaries_write, uint64List *boundaries_write,
+bool has_boundaries_append, uint64List *boundaries_append,
bool has_boundaries_flush, uint64List *boundaries_flush,
Error **errp)
{
@@ -557,6 +558,16 @@ void qmp_block_latency_histogram_set(
}
}
+if (has_boundaries || has_boundaries_append) {
+ret = block_latency_histogram_set(
+stats, BLOCK_ACCT_ZONE_APPEND,
+has_boundaries_append ? boundaries_append : boundaries);
+if (ret) {
+error_setg(errp, "Device '%s' set append write boundaries fail",
id);
+return;
+}
+}
+
if (has_boundaries || has_boundaries_flush) {
ret = block_latency_histogram_set(
stats, BLOCK_ACCT_FLUSH,
diff --git a/block/qapi.c b/block/qapi.c
index c84147849d..2684484e9d 100644
--- a/block/qapi.c
+++ b/block/qapi.c
@@ -533,27 +533,36 @@ static void bdrv_query_blk_stats(BlockDeviceStats *ds,
BlockBackend *blk)
ds->rd_bytes = stats->nr_bytes[BLOCK_ACCT_READ];
ds->wr_bytes = stats->nr_bytes[BLOCK_ACCT_WRITE];
+ds->zone_append_bytes = stats->nr_bytes[BLOCK_ACCT_ZONE_APPEND];
ds->unmap_bytes = stats->nr_bytes[BLOCK_ACCT_UNMAP];
ds->rd_operations = stats->nr_ops[BLOCK_ACCT_READ];
ds->wr_operations = stats->nr_ops[BLOCK_ACCT_WRITE];
+ds->zone_append_operations = stats->nr_ops[BLOCK_ACCT_ZONE_APPEND];
ds->unmap_operations = stats->nr_ops[BLOCK_ACCT_UNMAP];
ds->failed_rd_operations = stats->failed_ops[BLOCK_ACCT_READ];
ds->failed_wr_operations = stats->failed_ops[BLOCK_ACCT_WRITE];
+ds->failed_zone_append_operations =
+stats->failed_ops[BLOCK_ACCT_ZONE_APPEND];
ds->failed_flush_operations = stats->failed_ops[BLOCK_ACCT_FLUSH];
ds->failed_unmap_operations = stats->failed_ops[BLOCK_ACCT_UNMAP];
ds->invalid_rd_operations = stats->invalid_ops[BLOCK_ACCT_READ];
ds->invalid_wr_operations = stats->invalid_ops[BLOCK_ACCT_WRITE];
+ds->invalid_zone_append_operations =
+stats->invalid_ops[BLOCK_ACCT_ZONE_APPEND];
ds->invalid_flush_operations =
stats->invalid_ops[BLOCK_ACCT_FLUSH];
ds->invalid_unmap_operations = stats->invalid_ops[BLOCK_ACCT_UNMAP];
ds->rd_merged = stats->merged[BLOCK_ACCT_READ];
ds->wr_merged = stats->merged[BLOCK_ACCT_WRITE];
+ds->zone_append_merged = stats->merged[BLOCK_ACCT_ZONE_APPEND];
ds->unmap_merged = stats->merged[BLOCK_ACCT_UNMAP];
ds->flush_operations = stats->nr_ops[BLOCK_ACCT_FLUSH];
ds->wr_total_time_ns = stats->total_time_ns[BLOCK_ACCT_WRITE];
+ds->zone_append_total_time_ns =
+stats->total_time_ns[BLOCK_ACCT_ZONE_APPEND];
ds->rd_total_time_ns = stats->total_time_ns[BLOCK_ACCT_READ];
ds->flush_total_time_ns = stats->total_time_ns[BLOCK_ACCT_FLUSH];
ds->unmap_total_time_ns = stats->total_time_ns[BLOCK_ACCT_UNMAP];
@@ -571,6 +580,7 @@ static void bdrv_query_blk_stats(BlockDeviceStats *ds,
BlockBackend *blk)
TimedAverage *rd = &ts->latency[BLOCK_ACCT_READ];
TimedAverage *wr = &ts->latency[BLOCK_ACCT_WRITE];
+TimedAverage *zap = &ts->latency[BLOCK_ACCT_ZONE_APPEND];
TimedAverage *fl = &ts->latency[BLOCK_ACCT_FLUSH];
dev_stats->interval_length = ts->interval_length;
@@ -583,6 +593,10 @@ static void bdrv_query_blk_stats(BlockDeviceStats *ds,
BlockBackend *blk)
dev_stats->max_wr_latency_ns = timed_average_max(wr);
dev_stats->avg_wr_latency_ns = timed_average_avg(wr);
+dev_stats->min_zone_append_latency_ns = timed_average_min(zap);
+dev_stats->max_zone_append_latency_ns = timed_average_max(zap);
+dev_stats->avg_zone_append_latency_ns = timed_average_avg(zap);
+
dev_stats->min_flush_latency_ns = timed_average_min(fl);
dev_stats->max_flush_latency_ns = timed_average_max(fl);
dev_stats->avg_flush_latency_ns = timed_average_avg(fl);
@@ -591,6 +605,8 @@ static void bdrv_query_blk_stats(BlockDeviceStats *ds,
BlockBackend *blk)
block_acct_queue_depth(ts, BLOCK_ACCT_READ);
dev_stats->avg_wr_queue_depth =
bloc
[PATCH v9 5/5] docs/zoned-storage:add zoned emulation use case
Add the documentation about the example of using virtio-blk driver to pass the zoned block devices through to the guest. Signed-off-by: Sam Li --- docs/devel/zoned-storage.rst | 17 + 1 file changed, 17 insertions(+) diff --git a/docs/devel/zoned-storage.rst b/docs/devel/zoned-storage.rst index 6a36133e51..05ecf3729c 100644 --- a/docs/devel/zoned-storage.rst +++ b/docs/devel/zoned-storage.rst @@ -41,3 +41,20 @@ APIs for zoned storage emulation or testing. For example, to test zone_report on a null_blk device using qemu-io is: $ path/to/qemu-io --image-opts -n driver=host_device,filename=/dev/nullb0 -c "zrp offset nr_zones" + +To expose the host's zoned block device through virtio-blk, the command line +can be (includes the -device parameter): +-blockdev node-name=drive0,driver=host_device,filename=/dev/nullb0, +cache.direct=on \ +-device virtio-blk-pci,drive=drive0 +Or only use the -drive parameter: +-driver driver=host_device,file=/dev/nullb0,if=virtio,cache.direct=on + +Additionally, QEMU has several ways of supporting zoned storage, including: +(1) Using virtio-scsi: --device scsi-block allows for the passing through of +SCSI ZBC devices, enabling the attachment of ZBC or ZAC HDDs to QEMU. +(2) PCI device pass-through: While NVMe ZNS emulation is available for testing +purposes, it cannot yet pass through a zoned device from the host. To pass on +the NVMe ZNS device to the guest, use VFIO PCI pass the entire NVMe PCI adapter +through to the guest. Likewise, an HDD HBA can be passed on to QEMU all HDDs +attached to the HBA. -- 2.39.2
[PATCH v9 2/5] virtio-blk: add zoned storage emulation for zoned devices
This patch extends virtio-blk emulation to handle zoned device commands
by calling the new block layer APIs to perform zoned device I/O on
behalf of the guest. It supports Report Zone, four zone oparations (open,
close, finish, reset), and Append Zone.
The VIRTIO_BLK_F_ZONED feature bit will only be set if the host does
support zoned block devices. Regular block devices(conventional zones)
will not be set.
The guest os can use blktests, fio to test those commands on zoned devices.
Furthermore, using zonefs to test zone append write is also supported.
Signed-off-by: Sam Li
---
hw/block/virtio-blk-common.c | 2 +
hw/block/virtio-blk.c| 389 +++
hw/virtio/virtio-qmp.c | 2 +
3 files changed, 393 insertions(+)
diff --git a/hw/block/virtio-blk-common.c b/hw/block/virtio-blk-common.c
index ac52d7c176..e2f8e2f6da 100644
--- a/hw/block/virtio-blk-common.c
+++ b/hw/block/virtio-blk-common.c
@@ -29,6 +29,8 @@ static const VirtIOFeature feature_sizes[] = {
.end = endof(struct virtio_blk_config, discard_sector_alignment)},
{.flags = 1ULL << VIRTIO_BLK_F_WRITE_ZEROES,
.end = endof(struct virtio_blk_config, write_zeroes_may_unmap)},
+{.flags = 1ULL << VIRTIO_BLK_F_ZONED,
+ .end = endof(struct virtio_blk_config, zoned)},
{}
};
diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
index cefca93b31..66c2bc4b16 100644
--- a/hw/block/virtio-blk.c
+++ b/hw/block/virtio-blk.c
@@ -17,6 +17,7 @@
#include "qemu/module.h"
#include "qemu/error-report.h"
#include "qemu/main-loop.h"
+#include "block/block_int.h"
#include "trace.h"
#include "hw/block/block.h"
#include "hw/qdev-properties.h"
@@ -601,6 +602,335 @@ err:
return err_status;
}
+typedef struct ZoneCmdData {
+VirtIOBlockReq *req;
+struct iovec *in_iov;
+unsigned in_num;
+union {
+struct {
+unsigned int nr_zones;
+BlockZoneDescriptor *zones;
+} zone_report_data;
+struct {
+int64_t offset;
+} zone_append_data;
+};
+} ZoneCmdData;
+
+/*
+ * check zoned_request: error checking before issuing requests. If all checks
+ * passed, return true.
+ * append: true if only zone append requests issued.
+ */
+static bool check_zoned_request(VirtIOBlock *s, int64_t offset, int64_t len,
+ bool append, uint8_t *status) {
+BlockDriverState *bs = blk_bs(s->blk);
+int index;
+
+if (!virtio_has_feature(s->host_features, VIRTIO_BLK_F_ZONED)) {
+*status = VIRTIO_BLK_S_UNSUPP;
+return false;
+}
+
+if (offset < 0 || len < 0 || len > (bs->total_sectors << BDRV_SECTOR_BITS)
+|| offset > (bs->total_sectors << BDRV_SECTOR_BITS) - len) {
+*status = VIRTIO_BLK_S_ZONE_INVALID_CMD;
+return false;
+}
+
+if (append) {
+if (bs->bl.write_granularity) {
+if ((offset % bs->bl.write_granularity) != 0) {
+*status = VIRTIO_BLK_S_ZONE_UNALIGNED_WP;
+return false;
+}
+}
+
+index = offset / bs->bl.zone_size;
+if (BDRV_ZT_IS_CONV(bs->bl.wps->wp[index])) {
+*status = VIRTIO_BLK_S_ZONE_INVALID_CMD;
+return false;
+}
+
+if (len / 512 > bs->bl.max_append_sectors) {
+if (bs->bl.max_append_sectors == 0) {
+*status = VIRTIO_BLK_S_UNSUPP;
+} else {
+*status = VIRTIO_BLK_S_ZONE_INVALID_CMD;
+}
+return false;
+}
+}
+return true;
+}
+
+static void virtio_blk_zone_report_complete(void *opaque, int ret)
+{
+ZoneCmdData *data = opaque;
+VirtIOBlockReq *req = data->req;
+VirtIOBlock *s = req->dev;
+VirtIODevice *vdev = VIRTIO_DEVICE(req->dev);
+struct iovec *in_iov = data->in_iov;
+unsigned in_num = data->in_num;
+int64_t zrp_size, n, j = 0;
+int64_t nz = data->zone_report_data.nr_zones;
+int8_t err_status = VIRTIO_BLK_S_OK;
+
+if (ret) {
+err_status = VIRTIO_BLK_S_ZONE_INVALID_CMD;
+goto out;
+}
+
+struct virtio_blk_zone_report zrp_hdr = (struct virtio_blk_zone_report) {
+.nr_zones = cpu_to_le64(nz),
+};
+zrp_size = sizeof(struct virtio_blk_zone_report)
+ + sizeof(struct virtio_blk_zone_descriptor) * nz;
+n = iov_from_buf(in_iov, in_num, 0, &zrp_hdr, sizeof(zrp_hdr));
+if (n != sizeof(zrp_hdr)) {
+virtio_error(vdev, "Driver provided input buffer that is too small!");
+err_status = VIRTIO_BLK_S_ZONE_INVALID_CMD;
+goto out;
+}
+
+for (size_t i = sizeof(zrp_hdr); i < zrp_size;
+i += sizeof(struct virtio_blk_zone_descriptor), ++j) {
+struct virtio_blk_zone_descriptor desc =
+(struct virtio_blk_zone_descriptor) {
+.z_start = cpu_to_le64(data->zone_report_data.zones[j].start
+>> BDRV_SECTOR_BITS),
+.z_cap = cpu_to_le64(data->
[PATCH v9 4/5] virtio-blk: add some trace events for zoned emulation
Signed-off-by: Sam Li
Reviewed-by: Stefan Hajnoczi
---
hw/block/trace-events | 7 +++
hw/block/virtio-blk.c | 12
2 files changed, 19 insertions(+)
diff --git a/hw/block/trace-events b/hw/block/trace-events
index 2c45a62bd5..34be8b9135 100644
--- a/hw/block/trace-events
+++ b/hw/block/trace-events
@@ -44,9 +44,16 @@ pflash_write_unknown(const char *name, uint8_t cmd) "%s:
unknown command 0x%02x"
# virtio-blk.c
virtio_blk_req_complete(void *vdev, void *req, int status) "vdev %p req %p
status %d"
virtio_blk_rw_complete(void *vdev, void *req, int ret) "vdev %p req %p ret %d"
+virtio_blk_zone_report_complete(void *vdev, void *req, unsigned int nr_zones,
int ret) "vdev %p req %p nr_zones %u ret %d"
+virtio_blk_zone_mgmt_complete(void *vdev, void *req, int ret) "vdev %p req %p
ret %d"
+virtio_blk_zone_append_complete(void *vdev, void *req, int64_t sector, int
ret) "vdev %p req %p, append sector 0x%" PRIx64 " ret %d"
virtio_blk_handle_write(void *vdev, void *req, uint64_t sector, size_t
nsectors) "vdev %p req %p sector %"PRIu64" nsectors %zu"
virtio_blk_handle_read(void *vdev, void *req, uint64_t sector, size_t
nsectors) "vdev %p req %p sector %"PRIu64" nsectors %zu"
virtio_blk_submit_multireq(void *vdev, void *mrb, int start, int num_reqs,
uint64_t offset, size_t size, bool is_write) "vdev %p mrb %p start %d num_reqs
%d offset %"PRIu64" size %zu is_write %d"
+virtio_blk_handle_zone_report(void *vdev, void *req, int64_t sector, unsigned
int nr_zones) "vdev %p req %p sector 0x%" PRIx64 " nr_zones %u"
+virtio_blk_handle_zone_mgmt(void *vdev, void *req, uint8_t op, int64_t sector,
int64_t len) "vdev %p req %p op 0x%x sector 0x%" PRIx64 " len 0x%" PRIx64 ""
+virtio_blk_handle_zone_reset_all(void *vdev, void *req, int64_t sector,
int64_t len) "vdev %p req %p sector 0x%" PRIx64 " cap 0x%" PRIx64 ""
+virtio_blk_handle_zone_append(void *vdev, void *req, int64_t sector) "vdev %p
req %p, append sector 0x%" PRIx64 ""
# hd-geometry.c
hd_geometry_lchs_guess(void *blk, int cyls, int heads, int secs) "blk %p LCHS
%d %d %d"
diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
index 0d85c2c9b0..2afd5cf96c 100644
--- a/hw/block/virtio-blk.c
+++ b/hw/block/virtio-blk.c
@@ -676,6 +676,7 @@ static void virtio_blk_zone_report_complete(void *opaque,
int ret)
int64_t nz = data->zone_report_data.nr_zones;
int8_t err_status = VIRTIO_BLK_S_OK;
+trace_virtio_blk_zone_report_complete(vdev, req, nz, ret);
if (ret) {
err_status = VIRTIO_BLK_S_ZONE_INVALID_CMD;
goto out;
@@ -792,6 +793,8 @@ static void virtio_blk_handle_zone_report(VirtIOBlockReq
*req,
nr_zones = (req->in_len - sizeof(struct virtio_blk_inhdr) -
sizeof(struct virtio_blk_zone_report)) /
sizeof(struct virtio_blk_zone_descriptor);
+trace_virtio_blk_handle_zone_report(vdev, req,
+offset >> BDRV_SECTOR_BITS, nr_zones);
zone_size = sizeof(BlockZoneDescriptor) * nr_zones;
data = g_malloc(sizeof(ZoneCmdData));
@@ -814,7 +817,9 @@ static void virtio_blk_zone_mgmt_complete(void *opaque, int
ret)
{
VirtIOBlockReq *req = opaque;
VirtIOBlock *s = req->dev;
+VirtIODevice *vdev = VIRTIO_DEVICE(s);
int8_t err_status = VIRTIO_BLK_S_OK;
+trace_virtio_blk_zone_mgmt_complete(vdev, req,ret);
if (ret) {
err_status = VIRTIO_BLK_S_ZONE_INVALID_CMD;
@@ -841,6 +846,8 @@ static int virtio_blk_handle_zone_mgmt(VirtIOBlockReq *req,
BlockZoneOp op)
/* Entire drive capacity */
offset = 0;
len = capacity;
+trace_virtio_blk_handle_zone_reset_all(vdev, req, 0,
+ bs->total_sectors);
} else {
if (bs->bl.zone_size > capacity - offset) {
/* The zoned device allows the last smaller zone. */
@@ -848,6 +855,9 @@ static int virtio_blk_handle_zone_mgmt(VirtIOBlockReq *req,
BlockZoneOp op)
} else {
len = bs->bl.zone_size;
}
+trace_virtio_blk_handle_zone_mgmt(vdev, req, op,
+ offset >> BDRV_SECTOR_BITS,
+ len >> BDRV_SECTOR_BITS);
}
if (!check_zoned_request(s, offset, len, false, &err_status)) {
@@ -888,6 +898,7 @@ static void virtio_blk_zone_append_complete(void *opaque,
int ret)
err_status = VIRTIO_BLK_S_ZONE_INVALID_CMD;
goto out;
}
+trace_virtio_blk_zone_append_complete(vdev, req, append_sector, ret);
out:
aio_context_acquire(blk_get_aio_context(s->conf.conf.blk));
@@ -909,6 +920,7 @@ static int virtio_blk_handle_zone_append(VirtIOBlockReq
*req,
int64_t offset = virtio_ldq_p(vdev, &req->out.sector) << BDRV_SECTOR_BITS;
int64_t len = iov_size(out_iov, out_num);
+trace_virtio_blk_handle_zone_append(vdev, req, offset >> BDRV_SECTOR_BITS);
if (!check_zoned_request(s, offset, len, true, &err_status)) {
[PATCH v9 1/5] include: update virtio_blk headers to v6.3-rc1
Use scripts/update-linux-headers.sh to update headers to 6.3-rc1.
Signed-off-by: Sam Li
Reviewed-by: Stefan Hajnoczi
Reviewed-by: Dmitry Fomichev
---
include/standard-headers/drm/drm_fourcc.h| 12 +++
include/standard-headers/linux/ethtool.h | 48 -
include/standard-headers/linux/fuse.h| 45 +++-
include/standard-headers/linux/pci_regs.h| 1 +
include/standard-headers/linux/vhost_types.h | 2 +
include/standard-headers/linux/virtio_blk.h | 105 +++
linux-headers/asm-arm64/kvm.h| 1 +
linux-headers/asm-x86/kvm.h | 34 +-
linux-headers/linux/kvm.h| 9 ++
linux-headers/linux/vfio.h | 15 +--
linux-headers/linux/vhost.h | 8 ++
11 files changed, 270 insertions(+), 10 deletions(-)
diff --git a/include/standard-headers/drm/drm_fourcc.h
b/include/standard-headers/drm/drm_fourcc.h
index 69cab17b38..dc3e6112c1 100644
--- a/include/standard-headers/drm/drm_fourcc.h
+++ b/include/standard-headers/drm/drm_fourcc.h
@@ -87,6 +87,18 @@ extern "C" {
*
* The authoritative list of format modifier codes is found in
* `include/uapi/drm/drm_fourcc.h`
+ *
+ * Open Source User Waiver
+ * ---
+ *
+ * Because this is the authoritative source for pixel formats and modifiers
+ * referenced by GL, Vulkan extensions and other standards and hence used both
+ * by open source and closed source driver stacks, the usual requirement for an
+ * upstream in-kernel or open source userspace user does not apply.
+ *
+ * To ensure, as much as feasible, compatibility across stacks and avoid
+ * confusion with incompatible enumerations stakeholders for all relevant
driver
+ * stacks should approve additions.
*/
#define fourcc_code(a, b, c, d) ((uint32_t)(a) | ((uint32_t)(b) << 8) | \
diff --git a/include/standard-headers/linux/ethtool.h
b/include/standard-headers/linux/ethtool.h
index 87176ab075..99fcddf04f 100644
--- a/include/standard-headers/linux/ethtool.h
+++ b/include/standard-headers/linux/ethtool.h
@@ -711,6 +711,24 @@ enum ethtool_stringset {
ETH_SS_COUNT
};
+/**
+ * enum ethtool_mac_stats_src - source of ethtool MAC statistics
+ * @ETHTOOL_MAC_STATS_SRC_AGGREGATE:
+ * if device supports a MAC merge layer, this retrieves the aggregate
+ * statistics of the eMAC and pMAC. Otherwise, it retrieves just the
+ * statistics of the single (express) MAC.
+ * @ETHTOOL_MAC_STATS_SRC_EMAC:
+ * if device supports a MM layer, this retrieves the eMAC statistics.
+ * Otherwise, it retrieves the statistics of the single (express) MAC.
+ * @ETHTOOL_MAC_STATS_SRC_PMAC:
+ * if device supports a MM layer, this retrieves the pMAC statistics.
+ */
+enum ethtool_mac_stats_src {
+ ETHTOOL_MAC_STATS_SRC_AGGREGATE,
+ ETHTOOL_MAC_STATS_SRC_EMAC,
+ ETHTOOL_MAC_STATS_SRC_PMAC,
+};
+
/**
* enum ethtool_module_power_mode_policy - plug-in module power mode policy
* @ETHTOOL_MODULE_POWER_MODE_POLICY_HIGH: Module is always in high power mode.
@@ -779,6 +797,31 @@ enum ethtool_podl_pse_pw_d_status {
ETHTOOL_PODL_PSE_PW_D_STATUS_ERROR,
};
+/**
+ * enum ethtool_mm_verify_status - status of MAC Merge Verify function
+ * @ETHTOOL_MM_VERIFY_STATUS_UNKNOWN:
+ * verification status is unknown
+ * @ETHTOOL_MM_VERIFY_STATUS_INITIAL:
+ * the 802.3 Verify State diagram is in the state INIT_VERIFICATION
+ * @ETHTOOL_MM_VERIFY_STATUS_VERIFYING:
+ * the Verify State diagram is in the state VERIFICATION_IDLE,
+ * SEND_VERIFY or WAIT_FOR_RESPONSE
+ * @ETHTOOL_MM_VERIFY_STATUS_SUCCEEDED:
+ * indicates that the Verify State diagram is in the state VERIFIED
+ * @ETHTOOL_MM_VERIFY_STATUS_FAILED:
+ * the Verify State diagram is in the state VERIFY_FAIL
+ * @ETHTOOL_MM_VERIFY_STATUS_DISABLED:
+ * verification of preemption operation is disabled
+ */
+enum ethtool_mm_verify_status {
+ ETHTOOL_MM_VERIFY_STATUS_UNKNOWN,
+ ETHTOOL_MM_VERIFY_STATUS_INITIAL,
+ ETHTOOL_MM_VERIFY_STATUS_VERIFYING,
+ ETHTOOL_MM_VERIFY_STATUS_SUCCEEDED,
+ ETHTOOL_MM_VERIFY_STATUS_FAILED,
+ ETHTOOL_MM_VERIFY_STATUS_DISABLED,
+};
+
/**
* struct ethtool_gstrings - string set for data tagging
* @cmd: Command number = %ETHTOOL_GSTRINGS
@@ -1183,7 +1226,7 @@ struct ethtool_rxnfc {
uint32_trule_cnt;
uint32_trss_context;
};
- uint32_trule_locs[0];
+ uint32_trule_locs[];
};
@@ -1741,6 +1784,9 @@ enum ethtool_link_mode_bit_indices {
ETHTOOL_LINK_MODE_80baseDR8_2_Full_BIT = 96,
ETHTOOL_LINK_MODE_80baseSR8_Full_BIT = 97,
ETHTOOL_LINK_MODE_80baseVR8_Full_BIT = 98,
+ ETHTOOL_LINK_MODE_10baseT1S_Full_BIT = 99,
+ ETHTOOL_LINK_MODE_10baseT1S_Half_BIT =
[PATCH v9 0/5] Add zoned storage emulation to virtio-blk driver
This patch adds zoned storage emulation to the virtio-blk driver. It implements the virtio-blk ZBD support standardization that is recently accepted by virtio-spec. The link to related commit is at https://github.com/oasis-tcs/virtio-spec/commit/b4e8efa0fa6c8d844328090ad15db65af8d7d981 The Linux zoned device code that implemented by Dmitry Fomichev has been released at the latest Linux version v6.3-rc1. Aside: adding zoned=on alike options to virtio-blk device will be considered in following-up plan. Note: Sorry to send it again because of the previous incoherent patches caused by network error. v9: - address review comments * add docs for zoned emulation use case [Matias] * add the zoned feature bit to qmp monitor [Matias] * add the version number for newly added configs of accounting [Markus] v8: - address Stefan's review comments * rm aio_context_acquire/release in handle_req * rename function return type * rename BLOCK_ACCT_APPEND to BLOCK_ACCT_ZONE_APPEND for clarity v7: - update headers to v6.3-rc1 v6: - address Stefan's review comments * add accounting for zone append operation * fix in_iov usage in handle_request, error handling and typos v5: - address Stefan's review comments * restore the way writing zone append result to buffer * fix error checking case and other errands v4: - change the way writing zone append request result to buffer - change zone state, zone type value of virtio_blk_zone_descriptor - add trace events for new zone APIs v3: - use qemuio_from_buffer to write status bit [Stefan] - avoid using req->elem directly [Stefan] - fix error checkings and memory leak [Stefan] v2: - change units of emulated zone op coresponding to block layer APIs - modify error checking cases [Stefan, Damien] v1: - add zoned storage emulation Sam Li (5): include: update virtio_blk headers to v6.3-rc1 virtio-blk: add zoned storage emulation for zoned devices block: add accounting for zone append operation virtio-blk: add some trace events for zoned emulation docs/zoned-storage:add zoned emulation use case block/qapi-sysemu.c | 11 + block/qapi.c | 18 + docs/devel/zoned-storage.rst | 17 + hw/block/trace-events| 7 + hw/block/virtio-blk-common.c | 2 + hw/block/virtio-blk.c| 405 +++ hw/virtio/virtio-qmp.c | 2 + include/block/accounting.h | 1 + include/standard-headers/drm/drm_fourcc.h| 12 + include/standard-headers/linux/ethtool.h | 48 ++- include/standard-headers/linux/fuse.h| 45 ++- include/standard-headers/linux/pci_regs.h| 1 + include/standard-headers/linux/vhost_types.h | 2 + include/standard-headers/linux/virtio_blk.h | 105 + linux-headers/asm-arm64/kvm.h| 1 + linux-headers/asm-x86/kvm.h | 34 +- linux-headers/linux/kvm.h| 9 + linux-headers/linux/vfio.h | 15 +- linux-headers/linux/vhost.h | 8 + qapi/block-core.json | 68 +++- qapi/block.json | 4 + 21 files changed, 794 insertions(+), 21 deletions(-) -- 2.39.2
Re: [PATCH for-8.0] block/export: Fix graph locking in blk_get_geometry() call
Am 27.03.2023 um 13:39 hat Kevin Wolf geschrieben: > blk_get_geometry() eventually calls bdrv_nb_sectors(), which is a > co_wrapper_mixed_bdrv_rdlock. This means that when it is called from > coroutine context, it already assume to have the graph locked. > > However, virtio_blk_sect_range_ok() in block/export/virtio-blk-handler.c > (used by vhost-user-blk and VDUSE exports) runs in a coroutine, but > doesn't take the graph lock - blk_*() functions are generally expected > to do that internally. This causes an assertion failure when accessing > an export for the first time if it runs in an iothread. > > This is an example of the crash: > > $ ./storage-daemon/qemu-storage-daemon --object iothread,id=th0 --blockdev > file,filename=/home/kwolf/images/hd.img,node-name=disk --export > vhost-user-blk,addr.type=unix,addr.path=/tmp/vhost.sock,node-name=disk,id=exp0,iothread=th0 > qemu-storage-daemon: ../block/graph-lock.c:268: void > assert_bdrv_graph_readable(void): Assertion `qemu_in_main_thread() || > reader_count()' failed. > > (gdb) bt Oops, git helpfully removed the "comments"... (gdb) bt #0 0x76eafe5c in __pthread_kill_implementation () from /lib64/libc.so.6 #1 0x76e5fa76 in raise () from /lib64/libc.so.6 #2 0x76e497fc in abort () from /lib64/libc.so.6 #3 0x76e4971b in __assert_fail_base.cold () from /lib64/libc.so.6 #4 0x76e58656 in __assert_fail () from /lib64/libc.so.6 #5 0x556337a3 in assert_bdrv_graph_readable () at ../block/graph-lock.c:268 #6 0x555fd5a2 in bdrv_co_nb_sectors (bs=0x564c5ef0) at ../block.c:5847 #7 0x555ee949 in bdrv_nb_sectors (bs=0x564c5ef0) at block/block-gen.c:256 #8 0x555fd6b9 in bdrv_get_geometry (bs=0x564c5ef0, nb_sectors_ptr=0x7fffef7fedd0) at ../block.c:5884 #9 0x5562ad6d in blk_get_geometry (blk=0x564cb200, nb_sectors_ptr=0x7fffef7fedd0) at ../block/block-backend.c:1624 #10 0x555ddb74 in virtio_blk_sect_range_ok (blk=0x564cb200, block_size=512, sector=0, size=512) at ../block/export/virtio-blk-handler.c:44 #11 0x555dd80d in virtio_blk_process_req (handler=0x564cbb98, in_iov=0x7fffe8003830, out_iov=0x7fffe8003860, in_num=1, out_num=0) at ../block/export/virtio-blk-handler.c:189 #12 0x555dd546 in vu_blk_virtio_process_req (opaque=0x7fffe8003800) at ../block/export/vhost-user-blk-server.c:66 #13 0x557bf4a1 in coroutine_trampoline (i0=-402635264, i1=32767) at ../util/coroutine-ucontext.c:177 #14 0x76e75c20 in ?? () from /lib64/libc.so.6 #15 0x7fffefffa870 in ?? () #16 0x in ?? () I'm adding this back while applying (but with indentation this time so that git doesn't interpret it as comments). Kevin > Fix this by creating a new blk_co_get_geometry() that takes the lock, > and changing blk_get_geometry() to be a co_wrapper_mixed around it. > > To make the resulting code cleaner, virtio-blk-handler.c can directly > call the coroutine version now (though that wouldn't be necessary for > fixing the bug, taking the lock in blk_co_get_geometry() is what fixes > it). > > Fixes: 8ab8140a04cf771d63e9754d6ba6c1e676bfe507 > Reported-by: Lukáš Doktor > Signed-off-by: Kevin Wolf
Re: [PATCH for-8.0 v2] aio-posix: fix race between epoll upgrade and aio_set_fd_handler()
Am 23.03.2023 um 15:48 hat Stefan Hajnoczi geschrieben: > If another thread calls aio_set_fd_handler() while the IOThread event > loop is upgrading from ppoll(2) to epoll(7) then we might miss new > AioHandlers. The epollfd will not monitor the new AioHandler's fd, > resulting in hangs. > > Take the AioHandler list lock while upgrading to epoll. This prevents > AioHandlers from changing while epoll is being set up. If we cannot lock > because we're in a nested event loop, then don't upgrade to epoll (it > will happen next time we're not in a nested call). > > The downside to taking the lock is that the aio_set_fd_handler() thread > has to wait until the epoll upgrade is finished, which involves many > epoll_ctl(2) system calls. However, this scenario is rare and I couldn't > think of another solution that is still simple. > > Reported-by: Qing Wang > Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=2090998 > Cc: Paolo Bonzini > Cc: Fam Zheng > Signed-off-by: Stefan Hajnoczi > --- > v2: > - Use qemu_lockcnt_inc_and_unlock() instead of qemu_lockcnt_unlock() [Paolo] Thanks, applied to the block branch. Kevin
Re: [PATCH for-8.0] block/export: Fix graph locking in blk_get_geometry() call
Am 27/03/2023 um 13:39 schrieb Kevin Wolf:
> blk_get_geometry() eventually calls bdrv_nb_sectors(), which is a
> co_wrapper_mixed_bdrv_rdlock. This means that when it is called from
> coroutine context, it already assume to have the graph locked.
>
> However, virtio_blk_sect_range_ok() in block/export/virtio-blk-handler.c
> (used by vhost-user-blk and VDUSE exports) runs in a coroutine, but
> doesn't take the graph lock - blk_*() functions are generally expected
> to do that internally. This causes an assertion failure when accessing
> an export for the first time if it runs in an iothread.
>
> This is an example of the crash:
>
> $ ./storage-daemon/qemu-storage-daemon --object iothread,id=th0 --blockdev
> file,filename=/home/kwolf/images/hd.img,node-name=disk --export
> vhost-user-blk,addr.type=unix,addr.path=/tmp/vhost.sock,node-name=disk,id=exp0,iothread=th0
> qemu-storage-daemon: ../block/graph-lock.c:268: void
> assert_bdrv_graph_readable(void): Assertion `qemu_in_main_thread() ||
> reader_count()' failed.
>
> (gdb) bt
>
> Fix this by creating a new blk_co_get_geometry() that takes the lock,
> and changing blk_get_geometry() to be a co_wrapper_mixed around it.
>
> To make the resulting code cleaner, virtio-blk-handler.c can directly
> call the coroutine version now (though that wouldn't be necessary for
> fixing the bug, taking the lock in blk_co_get_geometry() is what fixes
> it).
>
> Fixes: 8ab8140a04cf771d63e9754d6ba6c1e676bfe507
> Reported-by: Lukáš Doktor
> Signed-off-by: Kevin Wolf
Reviewed-by: Emanuele Giuseppe Esposito
> ---
> include/block/block-io.h | 4 +++-
> include/sysemu/block-backend-io.h | 5 -
> block.c | 5 +++--
> block/block-backend.c | 7 +--
> block/export/virtio-blk-handler.c | 7 ---
> 5 files changed, 19 insertions(+), 9 deletions(-)
>
> diff --git a/include/block/block-io.h b/include/block/block-io.h
> index 5da99d4d60..dbc034b728 100644
> --- a/include/block/block-io.h
> +++ b/include/block/block-io.h
> @@ -89,7 +89,9 @@ int64_t co_wrapper
> bdrv_get_allocated_file_size(BlockDriverState *bs);
>
> BlockMeasureInfo *bdrv_measure(BlockDriver *drv, QemuOpts *opts,
> BlockDriverState *in_bs, Error **errp);
> -void bdrv_get_geometry(BlockDriverState *bs, uint64_t *nb_sectors_ptr);
> +
> +void coroutine_fn GRAPH_RDLOCK
> +bdrv_co_get_geometry(BlockDriverState *bs, uint64_t *nb_sectors_ptr);
>
> int coroutine_fn GRAPH_RDLOCK
> bdrv_co_delete_file(BlockDriverState *bs, Error **errp);
> diff --git a/include/sysemu/block-backend-io.h
> b/include/sysemu/block-backend-io.h
> index 40ab178719..c672b77247 100644
> --- a/include/sysemu/block-backend-io.h
> +++ b/include/sysemu/block-backend-io.h
> @@ -70,7 +70,10 @@ void co_wrapper blk_eject(BlockBackend *blk, bool
> eject_flag);
> int64_t coroutine_fn blk_co_getlength(BlockBackend *blk);
> int64_t co_wrapper_mixed blk_getlength(BlockBackend *blk);
>
> -void blk_get_geometry(BlockBackend *blk, uint64_t *nb_sectors_ptr);
> +void coroutine_fn blk_co_get_geometry(BlockBackend *blk,
> + uint64_t *nb_sectors_ptr);
> +void co_wrapper_mixed blk_get_geometry(BlockBackend *blk,
> + uint64_t *nb_sectors_ptr);
>
> int64_t coroutine_fn blk_co_nb_sectors(BlockBackend *blk);
> int64_t co_wrapper_mixed blk_nb_sectors(BlockBackend *blk);
> diff --git a/block.c b/block.c
> index 0dd604d0f6..e0c6c648b1 100644
> --- a/block.c
> +++ b/block.c
> @@ -5879,9 +5879,10 @@ int64_t coroutine_fn
> bdrv_co_getlength(BlockDriverState *bs)
> }
>
> /* return 0 as number of sectors if no device present or error */
> -void bdrv_get_geometry(BlockDriverState *bs, uint64_t *nb_sectors_ptr)
> +void coroutine_fn bdrv_co_get_geometry(BlockDriverState *bs,
> + uint64_t *nb_sectors_ptr)
> {
> -int64_t nb_sectors = bdrv_nb_sectors(bs);
> +int64_t nb_sectors = bdrv_co_nb_sectors(bs);
> IO_CODE();
>
> *nb_sectors_ptr = nb_sectors < 0 ? 0 : nb_sectors;
> diff --git a/block/block-backend.c b/block/block-backend.c
> index 278b04ce69..2ee39229e4 100644
> --- a/block/block-backend.c
> +++ b/block/block-backend.c
> @@ -1615,13 +1615,16 @@ int64_t coroutine_fn blk_co_getlength(BlockBackend
> *blk)
> return bdrv_co_getlength(blk_bs(blk));
> }
>
> -void blk_get_geometry(BlockBackend *blk, uint64_t *nb_sectors_ptr)
> +void coroutine_fn blk_co_get_geometry(BlockBackend *blk,
> + uint64_t *nb_sectors_ptr)
> {
> IO_CODE();
> +GRAPH_RDLOCK_GUARD();
> +
> if (!blk_bs(blk)) {
> *nb_sectors_ptr = 0;
> } else {
> -bdrv_get_geometry(blk_bs(blk), nb_sectors_ptr);
> +bdrv_co_get_geometry(blk_bs(blk), nb_sectors_ptr);
> }
> }
>
> diff --git a/block/export/virtio-blk-handler.c
> b/block/export/virtio-blk-handler.c
> index 313666e8ab..bc1cec675
Re: [PATCH] block/export: only acquire AioContext once for vhost_user_server_stop()
Am 23.03.2023 um 15:58 hat Stefan Hajnoczi geschrieben: > vhost_user_server_stop() uses AIO_WAIT_WHILE(). AIO_WAIT_WHILE() > requires that AioContext is only acquired once. > > Since blk_exp_request_shutdown() already acquires the AioContext it > shouldn't be acquired again in vhost_user_server_stop(). > > Signed-off-by: Stefan Hajnoczi Thanks, applied to the block branch. Kevin
Re: [PATCH 1/1] nbd/server: push pending frames after sending reply
Am 24.03.2023 um 11:47 hat Florian Westphal geschrieben: > qemu-nbd doesn't set TCP_NODELAY on the tcp socket. > > Kernel waits for more data and avoids transmission of small packets. > Without TLS this is barely noticeable, but with TLS this really shows. > > Booting a VM via qemu-nbd on localhost (with tls) takes more than > 2 minutes on my system. tcpdump shows frequent wait periods, where no > packets get sent for a 40ms period. > > Add explicit (un)corking when processing (and responding to) requests. > "TCP_CORK, &zero" after earlier "CORK, &one" will flush pending data. > > VM Boot time: > main:no tls: 23s, with tls: 2m45s > patched: no tls: 14s, with tls: 15s > > VM Boot time, qemu-nbd via network (same lan): > main:no tls: 18s, with tls: 1m50s > patched: no tls: 17s, with tls: 18s > > Future optimization: if we could detect if there is another pending > request we could defer the uncork operation because more data would be > appended. > > Signed-off-by: Florian Westphal Thanks, applied to the block branch. Kevin
[PATCH for-8.0] block/export: Fix graph locking in blk_get_geometry() call
blk_get_geometry() eventually calls bdrv_nb_sectors(), which is a
co_wrapper_mixed_bdrv_rdlock. This means that when it is called from
coroutine context, it already assume to have the graph locked.
However, virtio_blk_sect_range_ok() in block/export/virtio-blk-handler.c
(used by vhost-user-blk and VDUSE exports) runs in a coroutine, but
doesn't take the graph lock - blk_*() functions are generally expected
to do that internally. This causes an assertion failure when accessing
an export for the first time if it runs in an iothread.
This is an example of the crash:
$ ./storage-daemon/qemu-storage-daemon --object iothread,id=th0 --blockdev
file,filename=/home/kwolf/images/hd.img,node-name=disk --export
vhost-user-blk,addr.type=unix,addr.path=/tmp/vhost.sock,node-name=disk,id=exp0,iothread=th0
qemu-storage-daemon: ../block/graph-lock.c:268: void
assert_bdrv_graph_readable(void): Assertion `qemu_in_main_thread() ||
reader_count()' failed.
(gdb) bt
Fix this by creating a new blk_co_get_geometry() that takes the lock,
and changing blk_get_geometry() to be a co_wrapper_mixed around it.
To make the resulting code cleaner, virtio-blk-handler.c can directly
call the coroutine version now (though that wouldn't be necessary for
fixing the bug, taking the lock in blk_co_get_geometry() is what fixes
it).
Fixes: 8ab8140a04cf771d63e9754d6ba6c1e676bfe507
Reported-by: Lukáš Doktor
Signed-off-by: Kevin Wolf
---
include/block/block-io.h | 4 +++-
include/sysemu/block-backend-io.h | 5 -
block.c | 5 +++--
block/block-backend.c | 7 +--
block/export/virtio-blk-handler.c | 7 ---
5 files changed, 19 insertions(+), 9 deletions(-)
diff --git a/include/block/block-io.h b/include/block/block-io.h
index 5da99d4d60..dbc034b728 100644
--- a/include/block/block-io.h
+++ b/include/block/block-io.h
@@ -89,7 +89,9 @@ int64_t co_wrapper
bdrv_get_allocated_file_size(BlockDriverState *bs);
BlockMeasureInfo *bdrv_measure(BlockDriver *drv, QemuOpts *opts,
BlockDriverState *in_bs, Error **errp);
-void bdrv_get_geometry(BlockDriverState *bs, uint64_t *nb_sectors_ptr);
+
+void coroutine_fn GRAPH_RDLOCK
+bdrv_co_get_geometry(BlockDriverState *bs, uint64_t *nb_sectors_ptr);
int coroutine_fn GRAPH_RDLOCK
bdrv_co_delete_file(BlockDriverState *bs, Error **errp);
diff --git a/include/sysemu/block-backend-io.h
b/include/sysemu/block-backend-io.h
index 40ab178719..c672b77247 100644
--- a/include/sysemu/block-backend-io.h
+++ b/include/sysemu/block-backend-io.h
@@ -70,7 +70,10 @@ void co_wrapper blk_eject(BlockBackend *blk, bool
eject_flag);
int64_t coroutine_fn blk_co_getlength(BlockBackend *blk);
int64_t co_wrapper_mixed blk_getlength(BlockBackend *blk);
-void blk_get_geometry(BlockBackend *blk, uint64_t *nb_sectors_ptr);
+void coroutine_fn blk_co_get_geometry(BlockBackend *blk,
+ uint64_t *nb_sectors_ptr);
+void co_wrapper_mixed blk_get_geometry(BlockBackend *blk,
+ uint64_t *nb_sectors_ptr);
int64_t coroutine_fn blk_co_nb_sectors(BlockBackend *blk);
int64_t co_wrapper_mixed blk_nb_sectors(BlockBackend *blk);
diff --git a/block.c b/block.c
index 0dd604d0f6..e0c6c648b1 100644
--- a/block.c
+++ b/block.c
@@ -5879,9 +5879,10 @@ int64_t coroutine_fn bdrv_co_getlength(BlockDriverState
*bs)
}
/* return 0 as number of sectors if no device present or error */
-void bdrv_get_geometry(BlockDriverState *bs, uint64_t *nb_sectors_ptr)
+void coroutine_fn bdrv_co_get_geometry(BlockDriverState *bs,
+ uint64_t *nb_sectors_ptr)
{
-int64_t nb_sectors = bdrv_nb_sectors(bs);
+int64_t nb_sectors = bdrv_co_nb_sectors(bs);
IO_CODE();
*nb_sectors_ptr = nb_sectors < 0 ? 0 : nb_sectors;
diff --git a/block/block-backend.c b/block/block-backend.c
index 278b04ce69..2ee39229e4 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -1615,13 +1615,16 @@ int64_t coroutine_fn blk_co_getlength(BlockBackend *blk)
return bdrv_co_getlength(blk_bs(blk));
}
-void blk_get_geometry(BlockBackend *blk, uint64_t *nb_sectors_ptr)
+void coroutine_fn blk_co_get_geometry(BlockBackend *blk,
+ uint64_t *nb_sectors_ptr)
{
IO_CODE();
+GRAPH_RDLOCK_GUARD();
+
if (!blk_bs(blk)) {
*nb_sectors_ptr = 0;
} else {
-bdrv_get_geometry(blk_bs(blk), nb_sectors_ptr);
+bdrv_co_get_geometry(blk_bs(blk), nb_sectors_ptr);
}
}
diff --git a/block/export/virtio-blk-handler.c
b/block/export/virtio-blk-handler.c
index 313666e8ab..bc1cec6757 100644
--- a/block/export/virtio-blk-handler.c
+++ b/block/export/virtio-blk-handler.c
@@ -22,8 +22,9 @@ struct virtio_blk_inhdr {
unsigned char status;
};
-static bool virtio_blk_sect_range_ok(BlockBackend *blk, uint32_t block_size,
- uint64_t sector, size_t size)
+static bool cor
