[Qemu-commits] [qemu/qemu] 0421b5: coroutine: add flag to re-queue at front of CoQueue

2022-10-27 Thread Paolo Bonzini via Qemu-commits
  Branch: refs/heads/staging
  Home:   https://github.com/qemu/qemu
  Commit: 0421b563ab4d947a388078331c057daa9b979f41
  
https://github.com/qemu/qemu/commit/0421b563ab4d947a388078331c057daa9b979f41
  Author: Stefan Hajnoczi 
  Date:   2022-10-26 (Wed, 26 Oct 2022)

  Changed paths:
M include/qemu/coroutine.h
M util/qemu-coroutine-lock.c

  Log Message:
  ---
  coroutine: add flag to re-queue at front of CoQueue

When a coroutine wakes up it may determine that it must re-queue.
Normally coroutines are pushed onto the back of the CoQueue, but for
fairness it may be necessary to push it onto the front of the CoQueue.

Add a flag to specify that the coroutine should be pushed onto the front
of the CoQueue. A later patch will use this to ensure fairness in the
bounce buffer CoQueue used by the blkio BlockDriver.

Signed-off-by: Stefan Hajnoczi 
Message-id: 20221013185908.1297568-2-stefa...@redhat.com
Signed-off-by: Stefan Hajnoczi 


  Commit: fd66dbd424f5c90fcff3d27afed2c6c59d8be3ac
  
https://github.com/qemu/qemu/commit/fd66dbd424f5c90fcff3d27afed2c6c59d8be3ac
  Author: Stefan Hajnoczi 
  Date:   2022-10-26 (Wed, 26 Oct 2022)

  Changed paths:
M MAINTAINERS
A block/blkio.c
M block/meson.build
M meson.build
M meson_options.txt
M qapi/block-core.json
M scripts/meson-buildoptions.sh
M tests/qtest/modules-test.c

  Log Message:
  ---
  blkio: add libblkio block driver

libblkio (https://gitlab.com/libblkio/libblkio/) is a library for
high-performance disk I/O. It currently supports io_uring,
virtio-blk-vhost-user, and virtio-blk-vhost-vdpa with additional drivers
under development.

One of the reasons for developing libblkio is that other applications
besides QEMU can use it. This will be particularly useful for
virtio-blk-vhost-user which applications may wish to use for connecting
to qemu-storage-daemon.

libblkio also gives us an opportunity to develop in Rust behind a C API
that is easy to consume from QEMU.

This commit adds io_uring, nvme-io_uring, virtio-blk-vhost-user, and
virtio-blk-vhost-vdpa BlockDrivers to QEMU using libblkio. It will be
easy to add other libblkio drivers since they will share the majority of
code.

For now I/O buffers are copied through bounce buffers if the libblkio
driver requires it. Later commits add an optimization for
pre-registering guest RAM to avoid bounce buffers.

The syntax is:

  --blockdev 
io_uring,node-name=drive0,filename=test.img,readonly=on|off,cache.direct=on|off

  --blockdev 
nvme-io_uring,node-name=drive0,filename=/dev/ng0n1,readonly=on|off,cache.direct=on

  --blockdev 
virtio-blk-vhost-vdpa,node-name=drive0,path=/dev/vdpa...,readonly=on|off,cache.direct=on

  --blockdev 
virtio-blk-vhost-user,node-name=drive0,path=vhost-user-blk.sock,readonly=on|off,cache.direct=on

Signed-off-by: Stefan Hajnoczi 
Acked-by: Markus Armbruster 
Reviewed-by: Stefano Garzarella 
Message-id: 20221013185908.1297568-3-stefa...@redhat.com
Signed-off-by: Stefan Hajnoczi 


  Commit: 1f0fea38f46a786dd87407997e8bfbccca5e458f
  
https://github.com/qemu/qemu/commit/1f0fea38f46a786dd87407997e8bfbccca5e458f
  Author: Stefan Hajnoczi 
  Date:   2022-10-26 (Wed, 26 Oct 2022)

  Changed paths:
M hw/core/numa.c
M util/vfio-helpers.c

  Log Message:
  ---
  numa: call ->ram_block_removed() in ram_block_notifer_remove()

When a RAMBlockNotifier is added, ->ram_block_added() is called with all
existing RAMBlocks. There is no equivalent ->ram_block_removed() call
when a RAMBlockNotifier is removed.

The util/vfio-helpers.c code (the sole user of RAMBlockNotifier) is fine
with this asymmetry because it does not rely on RAMBlockNotifier for
cleanup. It walks its internal list of DMA mappings and unmaps them by
itself.

Future users of RAMBlockNotifier may not have an internal data structure
that records added RAMBlocks so they will need ->ram_block_removed()
callbacks.

This patch makes ram_block_notifier_remove() symmetric with respect to
callbacks. Now util/vfio-helpers.c needs to unmap remaining DMA mappings
after ram_block_notifier_remove() has been called. This is necessary
since users like block/nvme.c may create additional DMA mappings that do
not originate from the RAMBlockNotifier.

Reviewed-by: David Hildenbrand 
Signed-off-by: Stefan Hajnoczi 
Message-id: 20221013185908.1297568-4-stefa...@redhat.com
Signed-off-by: Stefan Hajnoczi 


  Commit: 4f384011c5a37f80dc6cadefffac61ffb1c3aa1e
  
https://github.com/qemu/qemu/commit/4f384011c5a37f80dc6cadefffac61ffb1c3aa1e
  Author: Stefan Hajnoczi 
  Date:   2022-10-26 (Wed, 26 Oct 2022)

  Changed paths:
M block/block-backend.c
M block/io.c
M block/nvme.c
M include/block/block-global-state.h
M include/block/block_int-common.h
M include/sysemu/block-backend-global-state.h
M qemu-img.c

  Log Message:
  ---
  block: pass size to bdrv_unregister_buf()

The only implementor of bdrv_register_buf() is block/nvme.c, where the
size 

[Qemu-commits] [qemu/qemu] 66a1ca: Merge tag 'qga-pull-2022-10-26' of https://github....

2022-10-27 Thread Paolo Bonzini via Qemu-commits
  Branch: refs/heads/staging
  Home:   https://github.com/qemu/qemu
  Commit: 66a1ca5d77f326829afc4d780edb327acea55a32
  
https://github.com/qemu/qemu/commit/66a1ca5d77f326829afc4d780edb327acea55a32
  Author: Stefan Hajnoczi 
  Date:   2022-10-27 (Thu, 27 Oct 2022)

  Changed paths:
M meson.build
M qga/channel-posix.c
A qga/commands-bsd.c
M qga/commands-common.h
A qga/commands-linux.c
M qga/commands-posix.c
M qga/main.c
M qga/meson.build

  Log Message:
  ---
  Merge tag 'qga-pull-2022-10-26' of https://github.com/kostyanf14/qemu into 
staging

Conflicts:
- "qga: Move HW address getting to a separate function" still uses
  has_hardware_address but it was just removed by "qapi qga: Elide
  redundant has_FOO in generated C". Drop has_hardware_address.

* tag 'qga-pull-2022-10-26' of https://github.com/kostyanf14/qemu:
  qga: add channel path to error messages
  qga: Add HW address getting for FreeBSD
  qga: Move HW address getting to a separate function
  qga: Add support for user password setting in FreeBSD
  qga: Add shutdown/halt/reboot support for FreeBSD
  qga: Add UFS freeze/thaw support for FreeBSD
  qga: Move Linux-specific FS freeze/thaw code to a separate file
  qga: Add initial FreeBSD support

Signed-off-by: Stefan Hajnoczi 





[Qemu-commits] [qemu/qemu] fad992: docs/devel/qapi-code-gen: Update example to match ...

2022-10-27 Thread Paolo Bonzini via Qemu-commits
  Branch: refs/heads/staging
  Home:   https://github.com/qemu/qemu
  Commit: fad992334d4a88f62a6f9d20c9b8f05446bdd904
  
https://github.com/qemu/qemu/commit/fad992334d4a88f62a6f9d20c9b8f05446bdd904
  Author: Markus Armbruster 
  Date:   2022-10-25 (Tue, 25 Oct 2022)

  Changed paths:
M docs/devel/qapi-code-gen.rst

  Log Message:
  ---
  docs/devel/qapi-code-gen: Update example to match current code

Signed-off-by: Markus Armbruster 
Reviewed-by: Daniel P. Berrangé 
Message-Id: <20221018062849.3420573-2-arm...@redhat.com>


  Commit: 8a4e771d26490aaae8dadfea00ce2513aed5acfe
  
https://github.com/qemu/qemu/commit/8a4e771d26490aaae8dadfea00ce2513aed5acfe
  Author: Markus Armbruster 
  Date:   2022-10-25 (Tue, 25 Oct 2022)

  Changed paths:
M docs/devel/qapi-code-gen.rst
M scripts/qapi/commands.py
M scripts/qapi/events.py

  Log Message:
  ---
  qapi: Tidy up whitespace in generated code

Signed-off-by: Markus Armbruster 
Reviewed-by: Daniel P. Berrangé 
Message-Id: <20221018062849.3420573-3-arm...@redhat.com>


  Commit: e8dce69d7d54990f63e311a8f0b8855f86d52d00
  
https://github.com/qemu/qemu/commit/e8dce69d7d54990f63e311a8f0b8855f86d52d00
  Author: Markus Armbruster 
  Date:   2022-10-25 (Tue, 25 Oct 2022)

  Changed paths:
M docs/devel/qapi-code-gen.rst

  Log Message:
  ---
  docs/devel/qapi-code-gen: Extend example for next commit's change

The next commit will change the code generated for some optional
members.  The example schema contains an optional member affected by
the change.  Add one that is not affected.

Signed-off-by: Markus Armbruster 
Reviewed-by: Daniel P. Berrangé 
Message-Id: <20221018062849.3420573-4-arm...@redhat.com>


  Commit: 987d542069acc73e7d7843f19ea145aafeefbbb6
  
https://github.com/qemu/qemu/commit/987d542069acc73e7d7843f19ea145aafeefbbb6
  Author: Markus Armbruster 
  Date:   2022-10-25 (Tue, 25 Oct 2022)

  Changed paths:
M docs/devel/qapi-code-gen.rst
M docs/devel/writing-monitor-commands.rst
M scripts/qapi/commands.py
M scripts/qapi/events.py
M scripts/qapi/gen.py
M scripts/qapi/schema.py
M scripts/qapi/types.py
M scripts/qapi/visit.py

  Log Message:
  ---
  qapi: Start to elide redundant has_FOO in generated C

In QAPI, absent optional members are distinct from any present value.
We thus represent an optional schema member FOO as two C members: a
FOO with the member's type, and a bool has_FOO.  Likewise for function
arguments.

However, has_FOO is actually redundant for a pointer-valued FOO, which
can be null only when has_FOO is false, i.e. has_FOO == !!FOO.  Except
for arrays, where we a null FOO can also be a present empty array.

The redundant has_FOO are a nuisance to work with.  Improve the
generator to elide them.  Uses of has_FOO need to be replaced as
follows.

Tests of has_FOO become the equivalent comparison of FOO with null.
For brevity, this is commonly done by implicit conversion to bool.

Assignments to has_FOO get dropped.

Likewise for arguments to has_FOO parameters.

Beware: code may violate the invariant has_FOO == !!FOO before the
transformation, and get away with it.  The above transformation can
then break things.  Two cases:

* Absent: if code ignores FOO entirely when !has_FOO (except for
  freeing it if necessary), even non-null / uninitialized FOO works.
  Such code is known to exist.

* Present: if code ignores FOO entirely when has_FOO, even null FOO
  works.  Such code should not exist.

In both cases, replacing tests of has_FOO by FOO reverts their sense.
We have to fix the value of FOO then.

To facilitate review of the necessary updates to handwritten code, add
means to opt out of this change, and opt out for all QAPI schema
modules where the change requires updates to handwritten code.  The
next few commits will remove these opt-outs in reviewable chunks, then
drop the means to opt out.

Signed-off-by: Markus Armbruster 
Reviewed-by: Daniel P. Berrangé 
Message-Id: <20221018062849.3420573-5-arm...@redhat.com>


  Commit: 0673a9bb702d3fe7c4145b1ec2ee9aca9d9f2965
  
https://github.com/qemu/qemu/commit/0673a9bb702d3fe7c4145b1ec2ee9aca9d9f2965
  Author: Markus Armbruster 
  Date:   2022-10-25 (Tue, 25 Oct 2022)

  Changed paths:
M scripts/qapi/schema.py
M tests/qtest/qmp-cmd-test.c
M tests/unit/test-qmp-cmds.c
M tests/unit/test-qmp-event.c
M tests/unit/test-qobject-input-visitor.c
M tests/unit/test-qobject-output-visitor.c
M tests/unit/test-visitor-serialization.c

  Log Message:
  ---
  qapi tests: Elide redundant has_FOO in generated C

The has_FOO for pointer-valued FOO are redundant, except for arrays.
They are also a nuisance to work with.  Recent commit "qapi: Start to
elide redundant has_FOO in generated C" provided the means to elide
them step by step.  This is the step for
tests/qapi-schema/qapi-schema-test.json.

Said commit explains the transformation in more detail.  The invariant