Re: [PATCH 2/4] export/fuse: process FUSE-over-io_uring requests

2025-09-20 Thread Brian Song
On 9/8/25 3:45 PM, Bernd Schubert wrote: On 9/8/25 21:09, Brian Song wrote: On 9/3/25 7:51 AM, Stefan Hajnoczi wrote: On Fri, Aug 29, 2025 at 10:50:23PM -0400, Brian Song wrote: https://docs.kernel.org/filesystems/fuse-io-uring.html As described in the kernel documentation, after FUSE

Re: [PATCH 3/4] export/fuse: Safe termination for FUSE-uring

2025-09-17 Thread Brian Song
On 9/17/25 9:01 AM, Hanna Czenczek wrote: On 15.09.25 07:43, Brian Song wrote: Hi Hanna, Hi Brian! (Thanks for your heads-up!) Stefan raised the above issue and proposed a preliminary solution: keep closing the file descriptor in the delete section, but perform umount separately for

Re: [PATCH 1/4] export/fuse: add opt to enable FUSE-over-io_uring

2025-09-17 Thread Brian Song
On 9/16/25 3:08 PM, Kevin Wolf wrote: Am 30.08.2025 um 04:50 hat Brian Song geschrieben: This patch adds a new export option for storage-export-daemon to enable FUSE-over-io_uring via the switch io-uring=on|off (disableby default). It also implements the protocol handshake with the Linux

Re: [PATCH 3/4] export/fuse: Safe termination for FUSE-uring

2025-09-14 Thread Brian Song
? Additionally, what is the correct ordering between close(fd) and umount, does one need to precede the other? Thanks, Brian On 9/9/25 3:33 PM, Stefan Hajnoczi wrote: > On Fri, Aug 29, 2025 at 10:50:24PM -0400, Brian Song wrote: >> @@ -901,24 +941,15 @@ static void fuse_export

Re: [PATCH 4/4] iotests: add tests for FUSE-over-io_uring

2025-09-11 Thread Brian Song
On 9/10/25 9:14 AM, Stefan Hajnoczi wrote: On Tue, Sep 09, 2025 at 04:51:12PM -0400, Brian Song wrote: On 9/9/25 3:38 PM, Stefan Hajnoczi wrote: On Fri, Aug 29, 2025 at 10:50:25PM -0400, Brian Song wrote: To test FUSE-over-io_uring, set the environment variable FUSE_OVER_IO_URING=1. This

Re: [PATCH 3/4] export/fuse: Safe termination for FUSE-uring

2025-09-09 Thread Brian Song
On 9/9/25 3:33 PM, Stefan Hajnoczi wrote: On Fri, Aug 29, 2025 at 10:50:24PM -0400, Brian Song wrote: @@ -901,24 +941,15 @@ static void fuse_export_shutdown(BlockExport *blk_exp) */ g_hash_table_remove(exports, exp->mountpoint); } -} - -static v

Re: [PATCH 4/4] iotests: add tests for FUSE-over-io_uring

2025-09-09 Thread Brian Song
On 9/9/25 3:38 PM, Stefan Hajnoczi wrote: On Fri, Aug 29, 2025 at 10:50:25PM -0400, Brian Song wrote: To test FUSE-over-io_uring, set the environment variable FUSE_OVER_IO_URING=1. This applies only when using the 'fuse' protocol. $ FUSE_OVER_IO_URING=1 ./check -fuse Suggested

Re: [PATCH 1/4] export/fuse: add opt to enable FUSE-over-io_uring

2025-09-09 Thread Brian Song
On 9/9/25 10:48 AM, Stefan Hajnoczi wrote: On Wed, Sep 03, 2025 at 02:00:55PM -0400, Brian Song wrote: On 9/3/25 6:53 AM, Stefan Hajnoczi wrote: On Fri, Aug 29, 2025 at 10:50:22PM -0400, Brian Song wrote: This patch adds a new export option for storage-export-daemon to enable FUSE-over

Re: [PATCH 2/4] export/fuse: process FUSE-over-io_uring requests

2025-09-08 Thread Brian Song
On 9/3/25 7:51 AM, Stefan Hajnoczi wrote: On Fri, Aug 29, 2025 at 10:50:23PM -0400, Brian Song wrote: https://docs.kernel.org/filesystems/fuse-io-uring.html As described in the kernel documentation, after FUSE-over-io_uring initialization and handshake, FUSE interacts with the kernel using

Re: [PATCH 0/4] export/fuse: Add FUSE-over-io_uring for Storage Exports

2025-09-03 Thread Brian Song
On 9/3/25 5:49 AM, Stefan Hajnoczi wrote: On Sat, Aug 30, 2025 at 08:00:00AM -0400, Brian Song wrote: We used fio to test a 1 GB file under both traditional FUSE and FUSE-over-io_uring modes. The experiments were conducted with the following iodepth and numjobs configurations: 1-1, 64-1, 1-4

Re: [PATCH 1/4] export/fuse: add opt to enable FUSE-over-io_uring

2025-09-03 Thread Brian Song
On 9/3/25 6:53 AM, Stefan Hajnoczi wrote: On Fri, Aug 29, 2025 at 10:50:22PM -0400, Brian Song wrote: This patch adds a new export option for storage-export-daemon to enable FUSE-over-io_uring via the switch io-uring=on|off (disableby default). It also implements the protocol handshake with

Re: [PATCH 0/4] export/fuse: Add FUSE-over-io_uring for Storage Exports

2025-08-30 Thread Brian Song
and throughput. Test results: https://gist.github.com/hibriansong/a4849903387b297516603e83b53bbde4 On 8/29/25 10:50 PM, Brian Song wrote: > Hi all, > > This is a GSoC project. More details are available here: > https://wiki.qemu.org/Google_Summer_of_Code_2025#FUSE-over-io_uring_expo

[PATCH 1/4] export/fuse: add opt to enable FUSE-over-io_uring

2025-08-30 Thread Brian Song
, to support multiple in-flight requests, we configure each Ring Queue with FUSE_DEFAULT_RING_QUEUE_DEPTH entries/requests. Suggested-by: Kevin Wolf Suggested-by: Stefan Hajnoczi Signed-off-by: Brian Song --- block/export/fuse.c | 310 +-- docs/tools/qemu

[PATCH 3/4] export/fuse: Safe termination for FUSE-uring

2025-08-30 Thread Brian Song
). Suggested-by: Kevin Wolf Suggested-by: Stefan Hajnoczi Signed-off-by: Brian Song --- block/export/fuse.c | 75 +++-- 1 file changed, 65 insertions(+), 10 deletions(-) diff --git a/block/export/fuse.c b/block/export/fuse.c index 07f74fc8ec..ab2eb895ad 100644

[PATCH 0/4] export/fuse: Add FUSE-over-io_uring for Storage Exports

2025-08-30 Thread Brian Song
requests (multiple ring entries) - Add tests for FUSE-over-io_uring More detail in the v2 cover letter: https://lists.nongnu.org/archive/html/qemu-block/2025-08/msg00140.html And in the v1 cover letter: https://lists.nongnu.org/archive/html/qemu-block/2025-07/msg00280.html Brian Song (4): export/fuse

[PATCH 2/4] export/fuse: process FUSE-over-io_uring requests

2025-08-30 Thread Brian Song
h the FUSE-over-io_uring handling functions. Suggested-by: Kevin Wolf Suggested-by: Stefan Hajnoczi Signed-off-by: Brian Song --- block/export/fuse.c | 457 ++-- 1 file changed, 309 insertions(+), 148 deletions(-) diff --git a/block/export/fuse.c b/block/exp

[PATCH 4/4] iotests: add tests for FUSE-over-io_uring

2025-08-30 Thread Brian Song
To test FUSE-over-io_uring, set the environment variable FUSE_OVER_IO_URING=1. This applies only when using the 'fuse' protocol. $ FUSE_OVER_IO_URING=1 ./check -fuse Suggested-by: Kevin Wolf Suggested-by: Stefan Hajnoczi Signed-off-by: Brian Song --- tests/qemu-iotests/chec

Re: [RFC PATCH 0/3] block/export: Add FUSE-over-io_uring for Storage Exports

2025-08-20 Thread Brian Song
On 8/17/25 9:45 AM, Stefan Hajnoczi wrote: > On Thu, Aug 14, 2025 at 11:46:16PM -0400, Zhi Song wrote: >> Due to kernel limitations, when the FUSE-over-io_uring option is >> enabled, >> you must create and assign nr_cpu IOThreads. For example: > > While it would be nice for the kernel to support a

Re: [PATCH 1/3] fuse: add FUSE-over-io_uring enable opt and init

2025-08-19 Thread Brian Song
On 8/19/25 7:23 PM, Brian Song wrote: > > > On 8/19/25 6:26 PM, Bernd Schubert wrote: >> >> >> On 8/19/25 03:15, Brian Song wrote: >>> >>> >>> On 8/18/25 7:04 PM, Bernd Schubert wrote: >>>> >>>> >>>> O

Re: [PATCH 1/3] fuse: add FUSE-over-io_uring enable opt and init

2025-08-19 Thread Brian Song
On 8/19/25 6:26 PM, Bernd Schubert wrote: On 8/19/25 03:15, Brian Song wrote: On 8/18/25 7:04 PM, Bernd Schubert wrote: On 8/17/25 01:13, Brian Song wrote: On 8/14/25 11:46 PM, Brian Song wrote: From: Brian Song This patch adds a new export option for storage-export-daemon to

Re: [PATCH 1/3] fuse: add FUSE-over-io_uring enable opt and init

2025-08-18 Thread Brian Song
On 8/18/25 7:04 PM, Bernd Schubert wrote: On 8/17/25 01:13, Brian Song wrote: On 8/14/25 11:46 PM, Brian Song wrote: From: Brian Song This patch adds a new export option for storage-export-daemon to enable or disable FUSE-over-io_uring via the switch io-uring=on|off (disable by

Re: [PATCH 1/3] fuse: add FUSE-over-io_uring enable opt and init

2025-08-16 Thread Brian Song
On 8/14/25 11:46 PM, Brian Song wrote: From: Brian Song This patch adds a new export option for storage-export-daemon to enable or disable FUSE-over-io_uring via the switch io-uring=on|off (disable by default). It also implements the protocol handshake with the Linux kernel during the FUSE

Re: [QEMU/FUSE] Discussion on Proper Termination and Async Cancellation in fuse-over-io_uring

2025-08-04 Thread Brian Song
On 2025-08-04 7:33 a.m., Bernd Schubert wrote: Hi Brian, sorry for my late reply, just back from vacation and fighting through my mails. On 8/4/25 01:33, Brian Song wrote: On 2025-08-01 12:09 p.m., Brian Song wrote: Hi Bernd, We are currently working on implementing termination support

Re: [QEMU/FUSE] Discussion on Proper Termination and Async Cancellation in fuse-over-io_uring

2025-08-03 Thread Brian Song
On 2025-08-01 12:09 p.m., Brian Song wrote: Hi Bernd, We are currently working on implementing termination support for fuse- over-io_uring in QEMU, and right now we are focusing on how to clean up in-flight SQEs properly. Our main question is about how well the kernel supports robust

[QEMU/FUSE] Discussion on Proper Termination and Async Cancellation in fuse-over-io_uring

2025-08-01 Thread Brian Song
nel's async cancel mechanism. We also realized libfuse manages shutdowns differently, typically by signaling a thread via eventfd rather than relying on async cancel. Would love to hear your thoughts or suggestions on this! Thanks, Brian

Re: [PATCH RFC 1/1] block/export: FUSE-over-io_uring Support for QEMU FUSE Exports

2025-07-22 Thread Brian Song
On 7/17/25 2:03 AM, Markus Armbruster wrote: > Brian Song writes: > >> This work provides an initial implementation of fuse-over-io_uring >> support for QEMU export. According to the fuse-over-io_uring protocol >> specification, the userspace side must create the same nu

Re: [PATCH RFC 0/1] block/export: FUSE-over-io_uring Support for QEMU FUSE Exports

2025-07-22 Thread Brian Song
On 7/20/25 12:13 PM, Stefan Hajnoczi wrote: > On Wed, Jul 16, 2025 at 02:38:23PM -0400, Brian Song wrote: >> This RFC patch represents an initial implementation of the FUSE-over- >> io_uring Exports idea proposed for Google Summer of Code (2025) under >> the QEM

Re: [PATCH RFC 1/1] block/export: FUSE-over-io_uring Support for QEMU FUSE Exports

2025-07-22 Thread Brian Song
On 7/21/25 9:51 AM, Bernd Schubert wrote: > Hi Brian, > > nice work! Just some minor comments from my side below. > Really appreciate you taking the time to review my code and many thanks for the helpful suggestions! > On 7/16/25 20:38, Brian Song wrote: >> [You don&#

Re: [PATCH RFC 1/1] block/export: FUSE-over-io_uring Support for QEMU FUSE Exports

2025-07-22 Thread Brian Song
On 7/20/25 8:53 PM, Stefan Hajnoczi wrote: > On Wed, Jul 16, 2025 at 02:38:24PM -0400, Brian Song wrote: >> This work provides an initial implementation of fuse-over-io_uring >> support for QEMU export. According to the fuse-over-io_uring protocol >> specification, the users

[PATCH RFC 0/1] block/export: FUSE-over-io_uring Support for QEMU FUSE Exports

2025-07-16 Thread Brian Song
CPUs. Therefore, this initial version only works on single-core VMs (i.e., QEMU started with -smp 1) or single core machine. Brian Song (1): block/export: FUSE-over-io_uring Support for QEMU FUSE Exports block/export/fuse.c | 423 +-- docs/tools/qemu

[PATCH RFC 1/1] block/export: FUSE-over-io_uring Support for QEMU FUSE Exports

2025-07-16 Thread Brian Song
support is still incomplete. In addition, handling connection termination and managing the "drained" state of a FUSE block export in QEMU remain as pending work. Suggested-by: Kevin Wolf Suggested-by: Stefan Hajnoczi Signed-off-by: Brian Song --- block/export/fuse.c

Re: [PATCH v2 18/21] fuse: Implement multi-threading

2025-06-26 Thread Brian
On 6/4/25 9:28 AM, Hanna Czenczek wrote: FUSE allows creating multiple request queues by "cloning" /dev/fuse FDs (via open("/dev/fuse") + ioctl(FUSE_DEV_IOC_CLONE)). We can use this to implement multi-threading. For configuration, we don't need any more information beyond the simple array pr

Re: [PATCH v2 21/21] fuse: Increase MAX_WRITE_SIZE with a second buffer

2025-06-10 Thread Brian
r_buf size, which is FUSE_MAX_WRITE_BYTES (64 * 4k) minus FUSE_IN_PLACE_WRITE_BYTES (4 * 4k)) is smaller than ring->max_payload_sz (which is 32 pages * 4k). Do we need to increase the spillover_buf size, or is there any other workaround? Brian On 6/4/25 9:28 AM, Hanna Czenczek wrote: We proba

Re: [RFC 08/11] aio-posix: gracefully handle io_uring_queue_init() failure

2025-06-02 Thread Brian
On 6/2/25 4:20 PM, Stefan Hajnoczi wrote: On Mon, Jun 02, 2025 at 08:26:39AM -0400, Brian wrote: On 5/28/25 3:09 PM, Stefan Hajnoczi wrote: io_uring may not be available at runtime due to system policies (e.g. the io_uring_disabled sysctl) or creation could fail due to file descriptor

Re: [RFC 08/11] aio-posix: gracefully handle io_uring_queue_init() failure

2025-06-02 Thread Brian
On 5/28/25 3:09 PM, Stefan Hajnoczi wrote: io_uring may not be available at runtime due to system policies (e.g. the io_uring_disabled sysctl) or creation could fail due to file descriptor resource limits. Handle failure scenarios as follows: If another AioContext already has io_uring, then fai

Re: [PATCH 0/1] qcow2: Skip copy-on-write when allocating a zero cluster

2020-08-27 Thread Brian Foster
On Wed, Aug 26, 2020 at 08:34:32PM +0200, Alberto Garcia wrote: > On Tue 25 Aug 2020 09:47:24 PM CEST, Brian Foster wrote: > > My fio fallocates the entire file by default with this command. Is that > > the intent of this particular test? I added --fallocate=none to my t

Re: [PATCH 0/1] qcow2: Skip copy-on-write when allocating a zero cluster

2020-08-25 Thread Brian Foster
On Tue, Aug 25, 2020 at 07:18:19PM +0200, Alberto Garcia wrote: > On Tue 25 Aug 2020 06:54:15 PM CEST, Brian Foster wrote: > > If I compare this 5m fio test between XFS and ext4 on a couple of my > > systems (with either no prealloc or full file prealloc), I end up seeing > &

Re: [PATCH 0/1] qcow2: Skip copy-on-write when allocating a zero cluster

2020-08-25 Thread Brian Foster
On Tue, Aug 25, 2020 at 02:24:58PM +0200, Alberto Garcia wrote: > On Fri 21 Aug 2020 07:02:32 PM CEST, Brian Foster wrote: > >> I was running fio with --ramp_time=5 which ignores the first 5 seconds > >> of data in order to let performance settle, but if I remove that I ca

Re: [PATCH 0/1] qcow2: Skip copy-on-write when allocating a zero cluster

2020-08-21 Thread Brian Foster
On Fri, Aug 21, 2020 at 02:12:32PM +0200, Alberto Garcia wrote: > On Fri 21 Aug 2020 01:42:52 PM CEST, Alberto Garcia wrote: > > On Fri 21 Aug 2020 01:05:06 PM CEST, Brian Foster > > wrote: > >>> > 1) off: for every write request QEMU initializes the cluster (64KB)

Re: [PATCH 0/1] qcow2: Skip copy-on-write when allocating a zero cluster

2020-08-21 Thread Brian Foster
On Fri, Aug 21, 2020 at 01:42:52PM +0200, Alberto Garcia wrote: > On Fri 21 Aug 2020 01:05:06 PM CEST, Brian Foster wrote: > >> > 1) off: for every write request QEMU initializes the cluster (64KB) > >> > with fallocate(ZERO_RANGE) and then writes the 4KB of da

Re: [PATCH 0/1] qcow2: Skip copy-on-write when allocating a zero cluster

2020-08-21 Thread Brian Foster
On Fri, Aug 21, 2020 at 07:58:11AM +1000, Dave Chinner wrote: > On Thu, Aug 20, 2020 at 10:03:10PM +0200, Alberto Garcia wrote: > > Cc: linux-xfs > > > > On Wed 19 Aug 2020 07:53:00 PM CEST, Brian Foster wrote: > > > In any event, if you're seeing unclear or

Re: [PATCH 0/1] qcow2: Skip copy-on-write when allocating a zero cluster

2020-08-19 Thread Brian Foster
t; > anything. > > > > What I got with my tests (using fio): > > > > - Using extent_size_hint didn't make any difference in my test case (I > > do see a clear difference however with the test case described in > > commit ffa244c84a). > > Hm, interestin

Re: [Qemu-block] [Qemu-devel] [PATCH 1/1] block: add block device shared field

2017-09-05 Thread Brian Steffens
Thanks for taking a look at the patch and fixing the missing CC addresses! > Have you considered extending the 'migrate' command with a list of > drives instead? That was my original plan but I thought having information on whether a device is shared or not could potentially be useful to other sy