Make 'qemu-img commit' work on Windows.
Command 'commit' requires reopening backing file in RW mode. So,
add reopen prepare/commit/abort handlers and change dwShareMode
for CreateFile call in order to allow further read/write reopening.
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/418
On Mon, Aug 23, 2021 at 11:38 AM Peter Lieven wrote:
>
> Am 22.08.21 um 23:02 schrieb Ilya Dryomov:
> > On Tue, Aug 10, 2021 at 3:41 PM Peter Lieven wrote:
> >> the qemu rbd driver currently lacks support for bdrv_co_block_status.
> >> This results mainly in incorrect progress during block operat
24.08.2021 18:51, Hanna Reitz wrote:
On 24.08.21 10:38, Vladimir Sementsov-Ogievskiy wrote:
Hi all!
v8:
06: add Hanna's r-b
07: keep is_fleecing detection in _new() function
08,17,18: add Hanna's r-b
19: wording, s/6.1/6.2/, add Markus's a-b
25: new
29: add John's r-b
34: new
Patches without
On 24.08.21 10:38, Vladimir Sementsov-Ogievskiy wrote:
Hi all!
v8:
06: add Hanna's r-b
07: keep is_fleecing detection in _new() function
08,17,18: add Hanna's r-b
19: wording, s/6.1/6.2/, add Markus's a-b
25: new
29: add John's r-b
34: new
Patches without r-b: 07, 25, 34
Thanks! I’ve applie
On 8/24/21 5:35 PM, Hanna Reitz wrote:
> As of recently, pylint complains when `open()` calls are missing an
> `encoding=` specified. Everything we have should be UTF-8 (and in fact,
> everything should be UTF-8, period (exceptions apply)), so use that.
>
> Signed-off-by: Hanna Reitz
> ---
> te
[ Cc: qemu-block ]
Am 11.08.2021 um 13:36 hat Christopher Pereira geschrieben:
> Hi,
>
> I'm reading a directory with 5.000.000 files (2,4 GB) inside a guest using
> "find | grep -c".
>
> On the host I saw high write IO (40 MB/s !) during over 1 hour using
> virt-top.
>
> I later repeated the r
pylint proposes using `[]` instead of `list()` and `{}` instead of
`dict()`, because it is faster. That seems simple enough, so heed its
advice.
Signed-off-by: Hanna Reitz
---
tests/qemu-iotests/iotests.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/tests/qemu-iotest
Hi,
I’ve updated my pylint to 2.10.2 and was greeted with some new warnings.
Some are fixed by John’s “Run iotest linters during Python CI” series
(https://lists.nongnu.org/archive/html/qemu-block/2021-07/msg00611.html),
but some are not (namely unspecified-encoding, use-list-literal, and
use-dict
As of recently, pylint complains when `open()` calls are missing an
`encoding=` specified. Everything we have should be UTF-8 (and in fact,
everything should be UTF-8, period (exceptions apply)), so use that.
Signed-off-by: Hanna Reitz
---
tests/qemu-iotests/297| 2 +-
tests/qemu-iotest
On 20.07.21 19:33, John Snow wrote:
GitLab: https://gitlab.com/jsnow/qemu/-/commits/python-package-iotest
CI: https://gitlab.com/jsnow/qemu/-/pipelines/340144191
I’ll take the liberty of applying patches 1 and 2 to my block-next
branch, because, well, they fix some of the 297 errors I’m seeing
qemu_vfio_add_mapping() returns a pointer to an indexed entry
in pre-allocated QEMUVFIOState::mappings[], thus can not be NULL.
Remove the pointless check.
Signed-off-by: Philippe Mathieu-Daudé
---
util/vfio-helpers.c | 4
1 file changed, 4 deletions(-)
diff --git a/util/vfio-helpers.c b/u
Pass qemu_vfio_do_mapping() an Error* argument so it can propagate
any error to callers. Replace error_report() which only report
to the monitor by the more generic error_setg_errno().
Reviewed-by: Fam Zheng
Reviewed-by: Stefan Hajnoczi
Signed-off-by: Philippe Mathieu-Daudé
---
util/vfio-helpe
Simplify qemu_vfio_dma_[un]map() handlers by replacing a pair of
qemu_mutex_lock/qemu_mutex_unlock calls by the WITH_QEMU_LOCK_GUARD
macro.
Signed-off-by: Philippe Mathieu-Daudé
---
util/vfio-helpers.c | 9 +++--
1 file changed, 3 insertions(+), 6 deletions(-)
diff --git a/util/vfio-helpers
We expect the first qemu_vfio_dma_map() to fail (indicating
DMA mappings exhaustion, see commit 15a730e7a3a). Do not
report the first failure as error, since we are going to
flush the mappings and retry.
This removes spurious error message displayed on the monitor:
(qemu) c
(qemu) qemu-kvm: V
Now that all qemu_vfio_dma_map() callers provide an Error* argument,
fill it with relevant / more descriptive message. Reduce 'ret'
(returned value) scope by returning errno directly to simplify
(removing the goto / 'out' label).
Signed-off-by: Philippe Mathieu-Daudé
---
block/nvme.c| 1
Currently qemu_vfio_dma_map() displays errors on stderr.
When using management interface, this information is simply
lost. Pass qemu_vfio_dma_map() an Error** handle so it can
propagate the error to callers.
Reviewed-by: Fam Zheng
Reviewed-by: Stefan Hajnoczi
Signed-off-by: Philippe Mathieu-Daud
Instead of displaying the error on stderr, use error_report()
which also report to the monitor.
Reviewed-by: Fam Zheng
Reviewed-by: Stefan Hajnoczi
Signed-off-by: Philippe Mathieu-Daudé
---
util/vfio-helpers.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/util/vfio-he
nvme_create_queue_pair() does not return a boolean value (indicating
eventual error) but a pointer, and is inconsistent in how it fills the
error handler. To fulfill callers expectations, always set an error
message on failure.
Reported-by: Auger Eric
Signed-off-by: Philippe Mathieu-Daudé
---
b
Fix when building with -Wshorten-64-to-32:
warning: implicit conversion loses integer precision: 'unsigned long' to
'int' [-Wshorten-64-to-32]
Signed-off-by: Philippe Mathieu-Daudé
---
block/trace-events | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/block/trace-events b
Hi,
This series contains various patches sent last year with
review comments addressed, few more cleanups, and a new
patch which remove the spurious "VFIO_MAP_DMA failed: No
space left on device" now poping up since commit 15a730e7a.
(it is the expected behavior, which is why we retry the
same cal
On 24.08.21 10:38, Vladimir Sementsov-Ogievskiy wrote:
The only caller pass copy_range and compress both false. Let's just
drop these arguments.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
include/block/block-copy.h | 1 -
block/block-copy.c | 5 ++---
block/copy-before-write.c
On 24.08.21 10:38, Vladimir Sementsov-Ogievskiy wrote:
mypy thinks that return value of these methods in subclusses is
QEMUMachine, which is wrong. So, make typing smarter.
Suggested-by: John Snow
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
python/qemu/machine/machine.py | 10 +++---
On 24.08.21 10:38, Vladimir Sementsov-Ogievskiy wrote:
We'll need a possibility to set compress and use_copy_range options
after initialization of the state. So make corresponding part of
block_copy_state_new() separate and public.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
include/block
The LBA28 capacity (at offsets 60/61 of identification) is supposed to
express the maximum size supported by LBA28 commands. If the device is
larger than this, we have to cap it to 2^28-1.
At least NetBSD happens to be using this value to determine whether to use
LBA28 or LBA48 for its commands, u
There is no conflict and no dependency if we have parallel writes to
different subclusters of one cluster when the cluster itself is already
allocated. So, relax extra dependency.
Measure performance:
First, prepare build/qemu-img-old and build/qemu-img-new images.
cd scripts/simplebench
./img_be
No logic change, just prepare for the following commit. While being
here do also small grammar fix in a comment.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Eric Blake
Reviewed-by: Hanna Reitz
---
block/qcow2-cluster.c | 49 ---
1 file chang
Add simple grammar-parsing template benchmark. New tool consume test
template written in bash with some special grammar injections and
produces multiple tests, run them and finally print a performance
comparison table of different tests produced from one template.
Signed-off-by: Vladimir Sementsov
Hi all!
v2:
01: improve documentation
02: add Hanna's and Eric's r-bs, add tiny grammar fix
03: fix test by filtering instead of reducing number of writes
Parallel small writes to unallocated cluster works bad when subclusters
enabled.
Look, without subclusters, one of write requests will alloca
24.08.2021 11:59, Hanna Reitz wrote:
On 24.08.21 10:53, Vladimir Sementsov-Ogievskiy wrote:
19.08.2021 19:37, Hanna Reitz wrote:
On 24.07.21 15:38, Vladimir Sementsov-Ogievskiy wrote:
[...]
+import itertools
+from lark import Lark
+
+grammar = """
+start: ( text | column_switch | row_switch
19.08.2021 19:37, Hanna Reitz wrote:
On 24.07.21 15:38, Vladimir Sementsov-Ogievskiy wrote:
Add simple grammar-parsing template benchmark.
This doesn’t really say much, and FWIW, for like ten minutes I thought this
would do something completely different than it did (while I was trying to
pa
The only caller pass copy_range and compress both false. Let's just
drop these arguments.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
include/block/block-copy.h | 1 -
block/block-copy.c | 5 ++---
block/copy-before-write.c | 2 +-
3 files changed, 3 insertions(+), 5 deletions(-)
d
New fleecing method becomes available: copy-before-write filter.
Actually we don't need backup job to setup image fleecing. Add test
for new recommended way of image fleecing.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
---
tests/qemu-iotests/tests/image-fleecing | 5
Define scsi device to operate with it by qom-set in further patch.
Give a new node-name to source block node, to not look like device
name.
Job now don't want to work without giving explicit id, so, let's call
it "fleecing".
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
--
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
Reviewed-by: John Snow
---
tests/qemu-iotests/iotests.py | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 89663dac06..77efcb0927 100644
Here:
- long line
- move to new interface of vm.qmp() (direct passing dict), to avoid
mypy false-positive, as it thinks that unpacked dict is a positional
argument.
- extra parenthesis
- handle event_wait possible None value
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max R
On 24.08.21 10:53, Vladimir Sementsov-Ogievskiy wrote:
19.08.2021 19:37, Hanna Reitz wrote:
On 24.07.21 15:38, Vladimir Sementsov-Ogievskiy wrote:
[...]
+import itertools
+from lark import Lark
+
+grammar = """
+start: ( text | column_switch | row_switch )+
+
+column_switch: "{" text ["|" te
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
Acked-by: Markus Armbruster
---
qapi/block-core.json | 25 +++--
1 file changed, 23 insertions(+), 2 deletions(-)
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 675d8265eb..6764d8b84f 100644
mypy thinks that return value of these methods in subclusses is
QEMUMachine, which is wrong. So, make typing smarter.
Suggested-by: John Snow
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
python/qemu/machine/machine.py | 10 +++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --g
We are going to publish copy-before-write filter to be used in separate
of backup. Future step would support bitmap for the filter. But let's
start from full set bitmap.
We have to modify backup, as bitmap is first initialized by
copy-before-write filter, and then backup modifies it.
Signed-off-b
- use shorter construction
- don't create new dict if not needed
- drop extra unpacking key-val arguments
- drop extra default values
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
Reviewed-by: John Snow
---
python/qemu/machine/machine.py | 18 --
1 file
Give a good name to test file.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
---
tests/qemu-iotests/{222 => tests/image-fleecing} | 0
tests/qemu-iotests/{222.out => tests/image-fleecing.out} | 0
2 files changed, 0 insertions(+), 0 deletions(-)
rename tests/qemu-i
One more step closer to .bdrv_open(): use options instead of plain
arguments. Move to bdrv_open_child() calls, native for drive open
handlers.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Hanna Reitz
---
block/copy-before-write.c | 29 +++--
1 file changed, 1
Finally, copy-before-write gets own .bdrv_open and .bdrv_close
handlers, block_init() call and becomes available through bdrv_open().
To achieve this:
- cbw_init gets unused flags argument and becomes cbw_open
- block_copy_state_free() call moved to new cbw_close()
- in bdrv_cbw_append:
- o
In the next commit we'll get rid of source argument of cbw_init().
Prepare to it now, to make next commit simpler: move the code block
that uses source below attaching the child and use bs->file->bs instead
of source variable.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
--
We often call qmp() with unpacking dict, like qmp('foo', **{...}).
mypy don't really like it, it thinks that passed unpacked dict is a
positional argument and complains that it type should be bool (because
second argument of qmp() is conv_keys: bool).
Allow passing dict directly, simplifying inter
We are going to add a test-case with some behavior modifications. So,
let's prepare a function to be reused.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
---
tests/qemu-iotests/tests/image-fleecing | 19 +--
1 file changed, 13 insertions(+), 6 deletions(-)
One more step closer to real .bdrv_open() handler: use more usual names
for bs being initialized and its state.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
---
block/copy-before-write.c | 29 ++---
1 file changed, 14 insertions(+), 15 deletions(-)
Actually target of backup(sync=None) is not a final backup target:
image fleecing is intended to be used with external tool, which will
copy data from fleecing node to some real backup target.
Also, we are going to add a test case for "push backup with fleecing",
where instead of exporting fleecin
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Hanna Reitz
---
block/copy-before-write.h | 1 -
block/backup.c| 2 +-
block/copy-before-write.c | 7 +++
3 files changed, 4 insertions(+), 6 deletions(-)
diff --git a/block/copy-before-write.h b/block/copy-before-write.h
Move part of bdrv_cbw_append() to new function cbw_open(). It's an
intermediate step for adding normal .bdrv_open() handler to the
filter. With this commit no logic is changed, but we have a function
which will be turned into .bdrv_open() handler in future commit.
Signed-off-by: Vladimir Sementsov
We are going to publish copy-before-write filter. So, user should be
able to create it with blockdev-add first, specifying both filtered and
target children. And then do blockdev-reopen, to actually insert the
filter where needed.
Currently, filter unshares write permission unconditionally on sour
The file use both single and double quotes for strings. Let's be
consistent.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
---
tests/qemu-iotests/222 | 68 +-
1 file changed, 34 insertions(+), 34 deletions(-)
diff --git a/tests/qemu-
We need an ability to insert filters above top block node, attached to
block device. It can't be achieved with blockdev-reopen command. So, we
want do it with help of qom-set.
Intended usage:
Assume there is a node A that is attached to some guest device.
1. blockdev-add to create a filter node
We are going to publish copy-before-write filter, so it would be
initialized through options. Still we don't want to publish compress
and copy-range options, as
1. Modern way to enable compression is to use compress filter.
2. For copy-range it's unclean how to make proper interface:
- it's has
We are going to publish copy-before-write filter, and there no public
backing-child-based filter in Qemu. No reason to create a precedent, so
let's refactor copy-before-write filter instead.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
---
block/copy-before-write.c | 39 ++
Add field, so property can declare support for setting the property
when device is realized. To be used in the following commit.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
---
include/hw/qdev-properties.h | 1 +
hw/core/qdev-properties.c| 6 +++---
2 files changed, 4
Now block-copy will crash if user don't set progress meter by
block_copy_set_progress_meter(). copy-before-write filter will be used
in separate of backup job, and it doesn't want any progress meter (for
now). So, allow not setting it.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max
bdrv_attach_child() do bdrv_unref() on failure, so we shouldn't do it
by hand here.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
---
block/copy-before-write.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/block/copy-before-write.c b/block/copy-before-write.c
index 2a5
Refactor the function to replace child at last. Thus we don't need to
revert it and code is simplified.
block-copy state initialization being done before replacing the child
doesn't need any drained section.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
---
block/copy-befo
We are going to convert backup_top to full featured public filter,
which can be used in separate of backup job. Start from renaming from
"how it used" to "what it does".
While updating comments in 283 iotest, drop and rephrase also things
about ".active", as this field is now dropped, and filter d
The main consumer of cluster-size is block-copy. Let's calculate it
here instead of passing through backup-top.
We are going to publish copy-before-write filter soon, so it will be
created through options. But we don't want for now to make explicit
option for cluster-size, let's continue to calcul
Add function to change bs inside blk.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
---
include/sysemu/block-backend.h | 1 +
block/block-backend.c | 8
2 files changed, 9 insertions(+)
diff --git a/include/sysemu/block-backend.h b/include/sysemu/block-ba
We'll need a possibility to set compress and use_copy_range options
after initialization of the state. So make corresponding part of
block_copy_state_new() separate and public.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
include/block/block-copy.h | 3 +++
block/block-copy.c | 49 ++
Add function to transactionally replace bs inside BdrvChild.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
---
include/block/block.h | 2 ++
block.c | 31 +++
2 files changed, 33 insertions(+)
diff --git a/include/block/block.h b/
We want to simplify initialization interface of copy-before-write
filter as we are going to make it public. So, let's detect fleecing
scheme exactly in block-copy code, to not pass this information through
extra levels.
Why not just set BDRV_REQ_SERIALISING unconditionally: because we are
going to
Hi all!
v8:
06: add Hanna's r-b
07: keep is_fleecing detection in _new() function
08,17,18: add Hanna's r-b
19: wording, s/6.1/6.2/, add Markus's a-b
25: new
29: add John's r-b
34: new
Patches without r-b: 07, 25, 34
Vladimir Sementsov-Ogievskiy (34):
block: introduce bdrv_replace_child_bs()
66 matches
Mail list logo