Re: [PULL 00/34] Block patches

2020-08-25 Thread Max Reitz
On 25.08.20 16:22, Alberto Garcia wrote:
> On Tue 25 Aug 2020 04:01:14 PM CEST, Peter Maydell  
> wrote:
>> On Tue, 25 Aug 2020 at 09:33, Max Reitz  wrote:
>>>
>>> The following changes since commit 30aa19446d82358a30eac3b556b4d6641e00b7c1:
>>>
>>>   Merge remote-tracking branch 'remotes/cschoenebeck/tags/pull-9p-20200812' 
>>> into staging (2020-08-24 16:39:53 +0100)
>>>
>>> are available in the Git repository at:
>>>
>>>   https://github.com/XanClic/qemu.git tags/pull-block-2020-08-25
>>>
>>> for you to fetch changes up to c576fd97d4ca77b5a1a27728df11a61083dbfa98:
>>>
>>>   iotests: Add tests for qcow2 images with extended L2 entries (2020-08-25 
>>> 10:20:18 +0200)
>>>
>>> 
>>> Block patches:
>>> - qcow2 subclusters (extended L2 entries)
>>
>> This fails 'make check' on the BSDs because it assumes it has a bash
>> in /bin/bash, which isn't necessarily true:

:(

> I guess it needs to be replaced by '#!/usr/bin/env bash', shall I send
> the patches again, or can you fix it Max?

Sure, I’ll fix it and send a v2 tomorrow.

Max



signature.asc
Description: OpenPGP digital signature


Re: [PULL 00/34] Block patches

2020-08-25 Thread Alberto Garcia
On Tue 25 Aug 2020 04:01:14 PM CEST, Peter Maydell  
wrote:
> On Tue, 25 Aug 2020 at 09:33, Max Reitz  wrote:
>>
>> The following changes since commit 30aa19446d82358a30eac3b556b4d6641e00b7c1:
>>
>>   Merge remote-tracking branch 'remotes/cschoenebeck/tags/pull-9p-20200812' 
>> into staging (2020-08-24 16:39:53 +0100)
>>
>> are available in the Git repository at:
>>
>>   https://github.com/XanClic/qemu.git tags/pull-block-2020-08-25
>>
>> for you to fetch changes up to c576fd97d4ca77b5a1a27728df11a61083dbfa98:
>>
>>   iotests: Add tests for qcow2 images with extended L2 entries (2020-08-25 
>> 10:20:18 +0200)
>>
>> 
>> Block patches:
>> - qcow2 subclusters (extended L2 entries)
>
> This fails 'make check' on the BSDs because it assumes it has a bash
> in /bin/bash, which isn't necessarily true:

I guess it needs to be replaced by '#!/usr/bin/env bash', shall I send
the patches again, or can you fix it Max?

Berto



Re: [PULL 00/34] Block patches

2020-08-25 Thread Peter Maydell
On Tue, 25 Aug 2020 at 09:33, Max Reitz  wrote:
>
> The following changes since commit 30aa19446d82358a30eac3b556b4d6641e00b7c1:
>
>   Merge remote-tracking branch 'remotes/cschoenebeck/tags/pull-9p-20200812' 
> into staging (2020-08-24 16:39:53 +0100)
>
> are available in the Git repository at:
>
>   https://github.com/XanClic/qemu.git tags/pull-block-2020-08-25
>
> for you to fetch changes up to c576fd97d4ca77b5a1a27728df11a61083dbfa98:
>
>   iotests: Add tests for qcow2 images with extended L2 entries (2020-08-25 
> 10:20:18 +0200)
>
> 
> Block patches:
> - qcow2 subclusters (extended L2 entries)

This fails 'make check' on the BSDs because it assumes it has a bash
in /bin/bash, which isn't necessarily true:


  TESTiotest-qcow2: 271 [fail]
QEMU  --
"/home/qemu/qemu-test.dvSnX6/build/tests/qemu-iotests/../../qemu-system-aarch64"
-nodefaults -display none -accel qtest -machine virt
QEMU_IMG  --
"/home/qemu/qemu-test.dvSnX6/build/tests/qemu-iotests/../../qemu-img"
QEMU_IO   --
"/home/qemu/qemu-test.dvSnX6/build/tests/qemu-iotests/../../qemu-io"
--cache writeback --aio threads -f qcow2
QEMU_NBD  --
"/home/qemu/qemu-test.dvSnX6/build/tests/qemu-iotests/../../qemu-nbd"
IMGFMT-- qcow2 (compat=1.1)
IMGPROTO  -- file
PLATFORM  -- NetBSD/amd64 localhost 9.0
TEST_DIR  -- /home/qemu/qemu-test.dvSnX6/build/tests/qemu-iotests/scratch
SOCK_DIR  -- /tmp/mktemp.fPDlHdxw
SOCKET_SCM_HELPER --

--- /home/qemu/qemu-test.dvSnX6/src/tests/qemu-iotests/271.out
2020-08-25 12:59:52.0 +
+++ /home/qemu/qemu-test.dvSnX6/build/tests/qemu-iotests/271.out.bad
 2020-08-25 13:11:25.483774595 +
@@ -1,726 +1 @@
-QA output created by 271
-
-### Standard write tests (backing file: yes) ###
-
-Formatting 'TEST_DIR/t.IMGFMT.raw', fmt=raw size=1048576
-Formatting 'TEST_DIR/t.IMGFMT.base', fmt=raw size=1048576
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1048576
backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=raw
-write -q -P PATTERN 0 1k
-L2 entry #0: 0x8005 0001
-write -q -P PATTERN 3k 512
-L2 entry #0: 0x8005 0003
-write -q -P PATTERN 5k 1k
[skip rest of expected output]
-wrote 2048/2048 bytes at offset 40960
-2 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-*** done
+./check: ./271: /bin/bash: bad interpreter: No such file or directory
  TESTiotest-qcow2: 283


thanks
-- PMM



[PULL 00/34] Block patches

2020-08-25 Thread Max Reitz
The following changes since commit 30aa19446d82358a30eac3b556b4d6641e00b7c1:

  Merge remote-tracking branch 'remotes/cschoenebeck/tags/pull-9p-20200812' 
into staging (2020-08-24 16:39:53 +0100)

are available in the Git repository at:

  https://github.com/XanClic/qemu.git tags/pull-block-2020-08-25

for you to fetch changes up to c576fd97d4ca77b5a1a27728df11a61083dbfa98:

  iotests: Add tests for qcow2 images with extended L2 entries (2020-08-25 
10:20:18 +0200)


Block patches:
- qcow2 subclusters (extended L2 entries)


Alberto Garcia (34):
  qcow2: Make Qcow2AioTask store the full host offset
  qcow2: Convert qcow2_get_cluster_offset() into qcow2_get_host_offset()
  qcow2: Add calculate_l2_meta()
  qcow2: Split cluster_needs_cow() out of count_cow_clusters()
  qcow2: Process QCOW2_CLUSTER_ZERO_ALLOC clusters in handle_copied()
  qcow2: Add get_l2_entry() and set_l2_entry()
  qcow2: Document the Extended L2 Entries feature
  qcow2: Add dummy has_subclusters() function
  qcow2: Add subcluster-related fields to BDRVQcow2State
  qcow2: Add offset_to_sc_index()
  qcow2: Add offset_into_subcluster() and size_to_subclusters()
  qcow2: Add l2_entry_size()
  qcow2: Update get/set_l2_entry() and add get/set_l2_bitmap()
  qcow2: Add QCow2SubclusterType and qcow2_get_subcluster_type()
  qcow2: Add qcow2_get_subcluster_range_type()
  qcow2: Add qcow2_cluster_is_allocated()
  qcow2: Add cluster type parameter to qcow2_get_host_offset()
  qcow2: Replace QCOW2_CLUSTER_* with QCOW2_SUBCLUSTER_*
  qcow2: Handle QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC
  qcow2: Add subcluster support to calculate_l2_meta()
  qcow2: Add subcluster support to qcow2_get_host_offset()
  qcow2: Add subcluster support to zero_in_l2_slice()
  qcow2: Add subcluster support to discard_in_l2_slice()
  qcow2: Add subcluster support to check_refcounts_l2()
  qcow2: Update L2 bitmap in qcow2_alloc_cluster_link_l2()
  qcow2: Clear the L2 bitmap when allocating a compressed cluster
  qcow2: Add subcluster support to handle_alloc_space()
  qcow2: Add subcluster support to qcow2_co_pwrite_zeroes()
  qcow2: Add subcluster support to qcow2_measure()
  qcow2: Add prealloc field to QCowL2Meta
  qcow2: Add the 'extended_l2' option and the QCOW2_INCOMPAT_EXTL2 bit
  qcow2: Allow preallocation and backing files if extended_l2 is set
  qcow2: Assert that expand_zero_clusters_in_l1() does not support
subclusters
  iotests: Add tests for qcow2 images with extended L2 entries

 docs/interop/qcow2.txt   |  68 ++-
 docs/qcow2-cache.txt |  19 +-
 qapi/block-core.json |   7 +
 block/qcow2.h| 211 ++-
 include/block/block_int.h|   1 +
 block/qcow2-cluster.c| 906 +--
 block/qcow2-refcount.c   |  47 +-
 block/qcow2.c| 302 +++
 block/trace-events   |   2 +-
 tests/qemu-iotests/031.out   |   8 +-
 tests/qemu-iotests/036.out   |   4 +-
 tests/qemu-iotests/049.out   | 102 ++--
 tests/qemu-iotests/060.out   |   3 +-
 tests/qemu-iotests/061   |   6 +
 tests/qemu-iotests/061.out   |  25 +-
 tests/qemu-iotests/065   |  12 +-
 tests/qemu-iotests/082.out   |  39 +-
 tests/qemu-iotests/085.out   |  38 +-
 tests/qemu-iotests/144.out   |   4 +-
 tests/qemu-iotests/182.out   |   2 +-
 tests/qemu-iotests/185.out   |   8 +-
 tests/qemu-iotests/198   |   2 +
 tests/qemu-iotests/206.out   |   6 +-
 tests/qemu-iotests/242.out   |   5 +
 tests/qemu-iotests/255.out   |   8 +-
 tests/qemu-iotests/271   | 901 ++
 tests/qemu-iotests/271.out   | 726 +
 tests/qemu-iotests/274.out   |  49 +-
 tests/qemu-iotests/280.out   |   2 +-
 tests/qemu-iotests/291.out   |   2 +
 tests/qemu-iotests/302.out   |   1 +
 tests/qemu-iotests/303.out   |   4 +-
 tests/qemu-iotests/common.filter |   1 +
 tests/qemu-iotests/group |   1 +
 34 files changed, 2952 insertions(+), 570 deletions(-)
 create mode 100755 tests/qemu-iotests/271
 create mode 100644 tests/qemu-iotests/271.out

-- 
2.26.2




Re: [PULL 00/34] Block patches

2020-01-06 Thread Peter Maydell
On Mon, 6 Jan 2020 at 14:42, Max Reitz  wrote:
>
> The following changes since commit f17783e706ab9c7b3a2b69cf48e4f0ba40664f54:
>
>   Merge remote-tracking branch 
> 'remotes/pmaydell/tags/pull-target-arm-20191220' into staging (2020-01-03 
> 18:50:33 +)
>
> are available in the Git repository at:
>
>   https://github.com/XanClic/qemu.git tags/pull-block-2020-01-06
>
> for you to fetch changes up to 503ca1262bab2c11c533a4816d1ff4297d4f58a6:
>
>   backup-top: Begin drain earlier (2020-01-06 14:26:23 +0100)
>
> 
> Block patches:
> - Minor fixes and tests from the freeze period (too minor to be included
>   in 4.2)
> - Allow many bash iotests to test qcow2's external data file feature
> - Add compress filter driver
> - Fix Python iotests after 6f6e1698a6
> - Fix for the backup job
>


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/5.0
for any user-visible changes.

-- PMM



[PULL 00/34] Block patches

2020-01-06 Thread Max Reitz
The following changes since commit f17783e706ab9c7b3a2b69cf48e4f0ba40664f54:

  Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20191220' 
into staging (2020-01-03 18:50:33 +)

are available in the Git repository at:

  https://github.com/XanClic/qemu.git tags/pull-block-2020-01-06

for you to fetch changes up to 503ca1262bab2c11c533a4816d1ff4297d4f58a6:

  backup-top: Begin drain earlier (2020-01-06 14:26:23 +0100)


Block patches:
- Minor fixes and tests from the freeze period (too minor to be included
  in 4.2)
- Allow many bash iotests to test qcow2's external data file feature
- Add compress filter driver
- Fix Python iotests after 6f6e1698a6
- Fix for the backup job


Andrey Shinkevich (3):
  block: introduce compress filter driver
  qcow2: Allow writing compressed data of multiple clusters
  tests/qemu-iotests: add case to write compressed data of multiple
clusters

Max Reitz (28):
  block: Add bdrv_qapi_perm_to_blk_perm()
  block: Use bdrv_qapi_perm_to_blk_perm()
  blkdebug: Allow taking/unsharing permissions
  iotests: Add @error to wait_until_completed
  iotests: Add test for failing mirror complete
  iotests: s/qocw2/qcow2/
  iotests/qcow2.py: Add dump-header-exts
  iotests/qcow2.py: Split feature fields into bits
  iotests: Add _filter_json_filename
  iotests: Filter refcount_order in 036
  iotests: Replace IMGOPTS by _unsupported_imgopts
  iotests: Drop compat=1.1 in 050
  iotests: Let _make_test_img parse its parameters
  iotests: Add -o and --no-opts to _make_test_img
  iotests: Inject space into -ocompat=0.10 in 051
  iotests: Replace IMGOPTS= by -o
  iotests: Replace IMGOPTS='' by --no-opts
  iotests: Drop IMGOPTS use in 267
  iotests: Avoid qemu-img create
  iotests: Use _rm_test_img for deleting test images
  iotests: Avoid cp/mv of test images
  iotests: Make 091 work with data_file
  iotests: Make 110 work with data_file
  iotests: Make 137 work with data_file
  iotests: Make 198 work with data_file
  iotests: Disable data_file where it cannot be used
  iotests: Allow check -o data_file
  backup-top: Begin drain earlier

PanNengyuan (1):
  throttle-groups: fix memory leak in throttle_group_set_limit:

Philippe Mathieu-Daudé (1):
  tests/qemu-iotests: Update tests to recent desugarized -accel option

Vladimir Sementsov-Ogievskiy (1):
  qcow2-bitmaps: fix qcow2_can_store_new_dirty_bitmap

 block.c  |  47 +
 block/Makefile.objs  |   1 +
 block/backup-top.c   |   4 +-
 block/blkdebug.c |  93 -
 block/filter-compress.c  | 168 +++
 block/qcow2-bitmap.c |  41 
 block/qcow2.c| 102 ++-
 block/throttle-groups.c  |   4 +-
 include/block/block.h|   1 +
 qapi/block-core.json |  24 -
 tests/qemu-iotests/007   |   5 +-
 tests/qemu-iotests/014   |   2 +
 tests/qemu-iotests/015   |   5 +-
 tests/qemu-iotests/019   |   6 +-
 tests/qemu-iotests/020   |   6 +-
 tests/qemu-iotests/024   |  10 +-
 tests/qemu-iotests/026   |   5 +-
 tests/qemu-iotests/028   |   2 +-
 tests/qemu-iotests/029   |   7 +-
 tests/qemu-iotests/031   |   9 +-
 tests/qemu-iotests/031.out   |  36 +++
 tests/qemu-iotests/036   |  15 +--
 tests/qemu-iotests/036.out   |  66 ++--
 tests/qemu-iotests/039   |  27 ++---
 tests/qemu-iotests/039.out   |  22 ++--
 tests/qemu-iotests/041   |  44 
 tests/qemu-iotests/041.out   |   4 +-
 tests/qemu-iotests/043   |   4 +-
 tests/qemu-iotests/046   |   2 +
 tests/qemu-iotests/048   |   4 +-
 tests/qemu-iotests/050   |   8 +-
 tests/qemu-iotests/051   |   7 +-
 tests/qemu-iotests/053   |   4 +-
 tests/qemu-iotests/058   |   7 +-
 tests/qemu-iotests/059   |  20 ++--
 tests/qemu-iotests/060   |  14 +--
 tests/qemu-iotests/060.out   |  20 ++--
 tests/qemu-iotests/061   |  63 ++--
 tests/qemu-iotests/061.out   |  72 ++---
 tests/qemu-iotests/062   |   5 +-
 tests/qemu-iotests/063   |  18 ++--
 tests/qemu-iotests/063.out   |   3 +-
 tests/qemu-iotests/066   |   7 +-
 tests/qemu-iotests/067   |   6 +-
 tests/qemu-iotests/068   |   6 +-
 tests/qemu-iotests/069   |   2 +-
 tests/qemu-iotests/071   |   7 +-
 tests/qemu-iotests/073   |   4 +
 tests/qemu-iotests/074   |   4 +-
 tests/qemu-iotests/079   |   3 +-
 tests/qemu-iotests/080   |   7 +-
 tests/qemu-iotests/081   |   6 +-
 tests/qemu-iotests/085   |  18 ++--
 tests/qemu-iotests/085.out   |   8 +-
 tests/qemu-iotests/088   |   2 +-