This series add adaptive queue size feature, which gives great
performance improvmemnt on my laptop, but less exciting results on a
real server. When qemu-nbd will support MULTI-CON for writes, this
should become more interesting.
To implement this I added a worker struct for keeping worker state,
I want to keep more info per worker, and using a worker struct is the
natural way to do this. This also allows cleaning up the ops-* interface
which accepted uintptr_t index while the index is never a pointer. I
think the pointer is a result of passing the index to the thread using
the void* pointe
In the past nbdcopy was looking for hole extents instead of zero
extents. When we fixed this, we forgot to update some comments and
variable names referencing hole instead of zero.
Signed-off-by: Nir Soffer
---
copy/multi-thread-copying.c | 34 +-
1 file changed,
Creating a new command requires lot of boilerplate that makes it harder
to focus on the interesting code. Extract a helpers to create a command,
and the command slice buffer.
create_buffer is called only once, but the compiler is smart enough to
inline it, and adding it makes the code much simpler
copy_subcommand creates a new command without copying the original
command. Rename the function to make this more clear.
Signed-off-by: Nir Soffer
---
copy/multi-thread-copying.c | 29 ++---
1 file changed, 14 insertions(+), 15 deletions(-)
diff --git a/copy/multi-thread
Replace the command index with a worker pointer. The nbd-ops access the
index via the worker pointer. This allows commands to modify worker
state during processing.
Signed-off-by: Nir Soffer
---
copy/multi-thread-copying.c | 12 ++--
copy/nbd-ops.c | 6 +++---
copy/nbdcopy.
free_command() was abused as a completion callback. Introduce
finish_command() completion callback, so code that want to free a
command does not have to add dummy errors.
This will make it easier to manage worker state when a command
completes.
Signed-off-by: Nir Soffer
---
copy/multi-thread-co
Tracking the number of queued bytes per worker will allow optimizing the
number of in flight requests based on the actual requests size.
The goal is to allow large number of small requests, required to get
good performance, and in the same time limit the number of large
requests, that can be faste
Limit the size of the copy queue also by the number of queued bytes.
This allows using many concurrent small requests, required to get good
performance, but limiting the number of large requests that are actually
faster with lower concurrency.
New --queue-size option added to control the maximum q
On Sun, Feb 20, 2022 at 02:13:59PM +0200, Nir Soffer wrote:
> free_command() was abused as a completion callback. Introduce
> finish_command() completion callback, so code that want to free a
In the code it's called "finished_command".
ACK patches 1, 2, 3 & 4 (with this fix to the commit message)
ACK patch 5
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine. Supports Linux and W
On Sun, Feb 20, 2022 at 02:14:03PM +0200, Nir Soffer wrote:
> Limit the size of the copy queue also by the number of queued bytes.
> This allows using many concurrent small requests, required to get good
> performance, but limiting the number of large requests that are actually
> faster with lower
On Sun, Feb 20, 2022 at 02:14:02PM +0200, Nir Soffer wrote:
> +static inline void
> +increase_queue_size(struct worker *worker, size_t len)
^ space
and the same in the next function:
> +{
> + worker->queue_size += len;
> +}
> +
> +static inline void
> +decrease_queue_size(
That was quick, thank you Rich. Temporarily I used the "cp" approach, but
escaping possibility would be IMO useful, especially for automated setups where
"--upload" args are generated automatically (my case).
Regards,
Lukáš
Dne 18. 02. 22 v 20:00 Richard W.M. Jones napsal(a):
> On Fri, Feb 18,
On Fri, Feb 18, 2022 at 12:25:56PM +, Richard W.M. Jones wrote:
> On Thu, Feb 17, 2022 at 04:44:56PM -0600, Eric Blake wrote:
> > On Thu, Feb 17, 2022 at 10:00:52PM +, Richard W.M. Jones wrote:
> > > On Thu, Feb 17, 2022 at 03:58:07PM -0600, Eric Blake wrote:
> > > > The NBD protocol spec r
On Thu, Feb 17, 2022 at 05:04:13PM -0600, Eric Blake wrote:
> WIP: I need to finish writing the actual interop-qemu-block-size.sh to
> demonstrate scenarios where qemu-nbd advertises a block size > 1 to
> clients that request it, but sticks to 1 otherwise (see
> https://gitlab.com/qemu-project/qemu
I don't think I announced it, but back in January I thought I would be
doing a new stable release of libnbd (1.12) and nbdkit (1.30). That
didn't happen obviously because once I'd written up the release notes
they seemed a bit thin:
https://libguestfs.org/libnbd-release-notes-1.12.1.html
https://
Because these filters perform a read-modify-write cycle for requests
which are smaller than the block size of the filter, we can adjust or
set the preferred block size to the block size of the filter or the
preferred block size of the underlying plugin, whichever is larger.
We're careful not to se
This filter is a little unusual because it allows clients to send a
wider range of request sizes than the underlying plugin allows.
Therefore we advertise the widest possible minimum and maximum block
size to clients.
We still need to pick a suitable preferred block size assuming the
plugin itself
On Sun, Feb 20, 2022 at 8:53 PM Richard W.M. Jones wrote:
>
> On Sun, Feb 20, 2022 at 02:14:02PM +0200, Nir Soffer wrote:
> > +static inline void
> > +increase_queue_size(struct worker *worker, size_t len)
>
> ^ space
>
> and the same in the next function:
Sure will fix befo
On 02/17/22 17:56, Eric Blake wrote:
> On Thu, Feb 17, 2022 at 04:41:31PM +, Richard W.M. Jones wrote:
>> On Thu, Feb 17, 2022 at 10:30:13AM -0600, Eric Blake wrote:
>>> On Thu, Feb 17, 2022 at 02:36:43PM +, Richard W.M. Jones wrote:
This filter doesn't call the next_open function in t
21 matches
Mail list logo
