Re: [Qemu-block] [Qemu-devel] [PATCH RFC 2/3] pxtool: Add new qemu-img command info generation tool

2019-04-09 Thread Markus Armbruster
John Snow writes: > Presently we use hxtool and a .hx format to generate a few things like > the qemu_img subcommand dispatch table, the qemu_img help() display output, > and a help output in qemu-img.texi. > > Unfortunately, this means that this information is duplicated in at least > three

Re: [Qemu-block] [Qemu-devel] [PATCH] qemu-img: fix .hx and .texi disparity

2019-04-09 Thread Markus Armbruster
John Snow writes: > It turns out that having options listed in three places continues to be > a bad idea. I'm still toying with the idea of an improved infrastructure > here, but in the meantime, another bandaid. > > There are three locations: > (1) .hx file, formatted as texi > (2) .hx file,

[Qemu-block] [PATCH RFC 3/3] qemu-img.texi: use macros for command summaries

2019-04-09 Thread John Snow
Use a macro to recall this text instead of re-entering it manually. Modify the pxtool script to create macros for each subcommand, then alter the qemu-img.texi file to recall these macros. Signed-off-by: John Snow --- qemu-img.texi | 28 ++-- scripts/pxtool.py | 4

[Qemu-block] [PATCH RFC 1/3] qemu-img: fix .hx and .texi disparity

2019-04-09 Thread John Snow
It turns out that having options listed in three places continues to be a bad idea. I'm still toying with the idea of an improved infrastructure here, but in the meantime, another bandaid. There are three locations: (1) .hx file, formatted as texi (2) .hx file, formatted as human readable. (3)

[Qemu-block] [PATCH RFC 0/3] qemu-img: remove command documentation duplication

2019-04-09 Thread John Snow
This might hopefully cut down on the doc duplication/mismatching until I can devise something more comprehensive. Ideally, I'd like to redo all of the documentation for qemu-img nearly from scratch; with a parser generator that helps generate the documentation as well so they'll never get out of

[Qemu-block] [PATCH RFC 2/3] pxtool: Add new qemu-img command info generation tool

2019-04-09 Thread John Snow
Presently we use hxtool and a .hx format to generate a few things like the qemu_img subcommand dispatch table, the qemu_img help() display output, and a help output in qemu-img.texi. Unfortunately, this means that this information is duplicated in at least three places: (1) in qemu-img-cmds.hx

[Qemu-block] [PATCH] qemu-img: fix .hx and .texi disparity

2019-04-09 Thread John Snow
It turns out that having options listed in three places continues to be a bad idea. I'm still toying with the idea of an improved infrastructure here, but in the meantime, another bandaid. There are three locations: (1) .hx file, formatted as texi (2) .hx file, formatted as human readable. (3)

Re: [Qemu-block] [Qemu-devel] [PATCH 00/10] block/pflash_cfi02: Implement missing AMD pflash functionality

2019-04-09 Thread Stephen Checkoway
On Apr 9, 2019, at 12:15, Philippe Mathieu-Daudé wrote: > Since you did changes in the CFI table, I think we should add a tests > verifying the table is correctly generated for all you FlashConfig entries. That's a good idea. Some of the values are essentially arbitrary (e.g., how long an

[Qemu-block] [PATCH] xen-block: support feature-large-sector-size

2019-04-09 Thread Paul Durrant
A recent Xen commit [1] clarified the semantics of sector based quantities used in the blkif protocol such that it is now safe to create a xen-block device with a logical_block_size != 512, as long as the device only connects to a frontend advertizing 'feature-large-block-size'. This patch

Re: [Qemu-block] [Qemu-devel] [PATCH 00/10] block/pflash_cfi02: Implement missing AMD pflash functionality

2019-04-09 Thread Philippe Mathieu-Daudé
On 4/9/19 5:55 PM, Stephen Checkoway wrote: > Hi Phil, > >> On Apr 9, 2019, at 06:34, Philippe Mathieu-Daudé wrote: >> >> Hi Stephen, >> >> [Cc'ing Markus and Laszlo, we have similar interest in pflash01 testing] >> >> On 4/8/19 10:55 PM, Stephen Checkoway wrote: >>> The goal of this patch

Re: [Qemu-block] [Qemu-devel] [PATCH 00/10] block/pflash_cfi02: Implement missing AMD pflash functionality

2019-04-09 Thread Stephen Checkoway
Hi Phil, > On Apr 9, 2019, at 06:34, Philippe Mathieu-Daudé wrote: > > Hi Stephen, > > [Cc'ing Markus and Laszlo, we have similar interest in pflash01 testing] > > On 4/8/19 10:55 PM, Stephen Checkoway wrote: >> The goal of this patch series implement the following AMD command-set >>

Re: [Qemu-block] [Qemu-devel] [PATCH 01/10] block/pflash_cfi02: Add test for supported commands

2019-04-09 Thread Stephen Checkoway
On Apr 9, 2019, at 02:13, Thomas Huth wrote: > We'd like to get rid of global_qtest in the long run (since it is > causing trouble for tests that run multiple instances of QEMU in > parallel, e.g. migration tests)... so if it is feasible, please don't > use it in new code anymore. Try to use

[Qemu-block] [PATCH 2/2] iotests: new test 253 check qemu-img convert force read

2019-04-09 Thread Andrey Shinkevich
A new test for the patch 'qemu-img convert: ignore read errors' Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/253 | 69 ++ tests/qemu-iotests/253.out | 4 +++ tests/qemu-iotests/group | 1 + 3 files changed, 74 insertions(+) create

[Qemu-block] [PATCH 1/2] qemu-img convert: ignore read errors

2019-04-09 Thread Andrey Shinkevich
The 'qemu-img convert' new command option 'force read' with the key '-R' allows converting a damaged image to get all the available information in case of the read errors. The program reports read errors and continue the image conversion. The users should keep in their minds that the resulting

[Qemu-block] [PATCH 0/2] qemu-img convert: ignore read errors

2019-04-09 Thread Andrey Shinkevich
The 'qemu-img convert' new command option 'force read' with the key '-R' allows converting a damaged image to get all the available information in case of the read errors. The program reports read errors and continue the image conversion. The users should keep in their minds that the resulting

Re: [Qemu-block] [PATCH v4 1/3] block: include base when checking image chain for block allocation

2019-04-09 Thread Alberto Garcia
On Tue 09 Apr 2019 04:43:12 PM CEST, Vladimir Sementsov-Ogievskiy wrote: >>> -while (intermediate && intermediate != base) { >>> +while (include_base || intermediate != base) { >>> int64_t pnum_inter; >>> int64_t size_inter; >>> >>> @@ -2360,6 +2364,10 @@ int

Re: [Qemu-block] [PATCH v4 1/3] block: include base when checking image chain for block allocation

2019-04-09 Thread Vladimir Sementsov-Ogievskiy
09.04.2019 17:18, Alberto Garcia wrote: > On Mon 08 Apr 2019 08:22:19 PM CEST, Andrey Shinkevich wrote: >>* Return true if (a prefix of) the given range is allocated in any image >> - * between BASE and TOP (inclusive). BASE can be NULL to check if the given >> + * between BASE and TOP (TOP

Re: [Qemu-block] [PATCH v4 1/3] block: include base when checking image chain for block allocation

2019-04-09 Thread Alberto Garcia
On Mon 08 Apr 2019 08:22:19 PM CEST, Andrey Shinkevich wrote: > * Return true if (a prefix of) the given range is allocated in any image > - * between BASE and TOP (inclusive). BASE can be NULL to check if the given > + * between BASE and TOP (TOP included). To check the BASE image, set the > +

Re: [Qemu-block] [PATCH] nvme: add Get/Set Feature Timestamp support

2019-04-09 Thread Stefan Hajnoczi
On Fri, Apr 05, 2019 at 03:41:17PM -0600, Kenneth Heitke wrote: > Signed-off-by: Kenneth Heitke > --- > hw/block/nvme.c | 120 +- > hw/block/nvme.h | 3 ++ > hw/block/trace-events | 2 + > include/block/nvme.h | 2 + > 4 files changed,

Re: [Qemu-block] [PATCH v2] qcow2: discard bitmap when removed

2019-04-09 Thread Andrey Shinkevich
On 08/04/2019 22:03, Max Reitz wrote: > On 28.02.19 10:26, Andrey Shinkevich wrote: >> When a bitmap is removed, we can clean some space on the disk. The size >> of a cluster may be larger, so is the size of the bitmap that includes >> many clusters. Some bitmaps can be as large as tens of

Re: [Qemu-block] [PATCH v4 3/3] block/stream: introduce a bottom node

2019-04-09 Thread Alberto Garcia
On Mon 08 Apr 2019 08:22:21 PM CEST, Andrey Shinkevich wrote: > The bottom node is the intermediate block device that has the base as its > backing image. It is used instead of the base node while a block stream > job is running to avoid dependency on the base that may change due to the > parallel

Re: [Qemu-block] [PATCH v3 3/3] block/stream: introduce a bottom node

2019-04-09 Thread Alberto Garcia
On Mon 08 Apr 2019 08:17:37 PM CEST, Andrey Shinkevich wrote: >>> +for (iter = bs; iter != bottom; iter = backing_bs(iter)) { >>> +block_job_add_bdrv(>common, "intermediate node", >>> backing_bs(iter), >>> + 0, basic_flags, _abort); >>> } >> >> This

[Qemu-block] [PATCH v2 for-4.0?] aio-posix: ensure poll mode is left when aio_notify is called

2019-04-09 Thread Paolo Bonzini
With aio=thread, adaptive polling makes latency worse rather than better, because it delays the execution of the ThreadPool's completion bottom half. event_notifier_poll() does run while polling, detecting that a bottom half was scheduled by a worker thread, but because ctx->notifier is

Re: [Qemu-block] [RFC PATCH] aio: Add a knob to always poll if there are in-flight requests

2019-04-09 Thread Stefan Hajnoczi
On Tue, Apr 02, 2019 at 02:19:08PM +0200, Sergio Lopez wrote: > The polling mode in aio_poll is able to trim down ~20us on the average > request latency, but it needs manual fine tuning to adjust it to the > characteristics of the storage. > > Here we add a new knob to the IOThread object,

Re: [Qemu-block] [Qemu-devel] [PATCH 00/10] block/pflash_cfi02: Implement missing AMD pflash functionality

2019-04-09 Thread Philippe Mathieu-Daudé
Hi Stephen, [Cc'ing Markus and Laszlo, we have similar interest in pflash01 testing] On 4/8/19 10:55 PM, Stephen Checkoway wrote: > The goal of this patch series implement the following AMD command-set parallel > flash functionality: > - flash interleaving; > - nonuniform sector sizes; > - erase

Re: [Qemu-block] [PATCH] blk: postpone request execution on a context protected with "drained section"

2019-04-09 Thread Kevin Wolf
Am 02.04.2019 um 10:35 hat Denis Plotnikov geschrieben: > > > On 13.03.2019 19:04, Kevin Wolf wrote: > > Am 14.12.2018 um 12:54 hat Denis Plotnikov geschrieben: > >> On 13.12.2018 15:20, Kevin Wolf wrote: > >>> Am 13.12.2018 um 12:07 hat Denis Plotnikov geschrieben: > Sounds it should be

Re: [Qemu-block] [PATCH for-4.1 4/5] tests/test-hmp: Use qtest_init() instead of qtest_start()

2019-04-09 Thread Dr. David Alan Gilbert
* Thomas Huth (th...@redhat.com) wrote: > qtest_start() + qtest_end() should be avoided, since they use the > global_qtest variable that we want to get rid of in the long run > Use qtest_init() and qtest_quit() instead. > > Signed-off-by: Thomas Huth Reviewed-by: Dr. David Alan Gilbert > ---

Re: [Qemu-block] [PATCH v4 3/3] block/stream: introduce a bottom node

2019-04-09 Thread Vladimir Sementsov-Ogievskiy
08.04.2019 21:22, Andrey Shinkevich wrote: > The bottom node is the intermediate block device that has the base as its > backing image. It is used instead of the base node while a block stream > job is running to avoid dependency on the base that may change due to the > parallel jobs. The change

Re: [Qemu-block] [PATCH v4 1/3] block: include base when checking image chain for block allocation

2019-04-09 Thread Vladimir Sementsov-Ogievskiy
08.04.2019 21:22, Andrey Shinkevich wrote: > This patch is used in the 'block/stream: introduce a bottom node' > that is following. Instead of the base node, the caller may pass > the node that has the base as its backing image to the function > bdrv_is_allocated_above() with a new parameter

[Qemu-block] [PATCH for-4.1 3/5] tests/qmp-cmd-test: Use qtest_init() instead of qtest_start()

2019-04-09 Thread Thomas Huth
qtest_start() + qtest_end() should be avoided, since they use the global_qtest variable that we want to get rid of in the long run (since global_qtest can not be used in tests that have to track multiple QEMU states, like migration tests). Use qtest_init() and qtest_quit() instead. Signed-off-by:

[Qemu-block] [PATCH for-4.1 1/5] tests/tco: Make test independent of global_qtest

2019-04-09 Thread Thomas Huth
Pass around the QTestState in the TestData, so we do not need the global_qtest variable here anymore. Signed-off-by: Thomas Huth --- tests/tco-test.c | 63 1 file changed, 32 insertions(+), 31 deletions(-) diff --git a/tests/tco-test.c

[Qemu-block] [PATCH for-4.1 4/5] tests/test-hmp: Use qtest_init() instead of qtest_start()

2019-04-09 Thread Thomas Huth
qtest_start() + qtest_end() should be avoided, since they use the global_qtest variable that we want to get rid of in the long run Use qtest_init() and qtest_quit() instead. Signed-off-by: Thomas Huth --- tests/test-hmp.c | 19 ++- 1 file changed, 10 insertions(+), 9

[Qemu-block] [PATCH for-4.1 5/5] tests/ide-test: Make test independent of global_qtest

2019-04-09 Thread Thomas Huth
Pass around the QTestState, so we do not need the problematic global_qtest variable (which causes trouble for tests that have multiple test states) here anymore. Signed-off-by: Thomas Huth --- tests/ide-test.c | 284 ++- 1 file changed, 159

[Qemu-block] [PATCH for-4.1 0/5] First batch of global_qtest-removement patches for QEMU 4.1

2019-04-09 Thread Thomas Huth
While global_qtest and its wrapper functions work fine for tests that only run one instance of QEMU, using the global_qtest variable in our qtests is very problematic for tests that use multiple test states (e.g. migration tests). Thus tests that still use global_qtest make it hard to share code

[Qemu-block] [PATCH for-4.1 2/5] tests/megasas: Make test independent of global_qtest

2019-04-09 Thread Thomas Huth
The test uses memwrite() and thus relies on global_qtest. Let's replace it with qtest_memwrite(), so that we are independent from global_qtest here. Signed-off-by: Thomas Huth --- tests/megasas-test.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/megasas-test.c

Re: [Qemu-block] [Qemu-devel] [PATCH 01/10] block/pflash_cfi02: Add test for supported commands

2019-04-09 Thread Markus Armbruster
Thomas Huth writes: > On 09/04/2019 10.35, Markus Armbruster wrote: >> Thomas Huth writes: >> >>> On 09/04/2019 09.45, Markus Armbruster wrote: Thomas Huth writes: > We'd like to get rid of global_qtest in the long run (since it is > causing trouble for tests that run

Re: [Qemu-block] [Qemu-devel] [PATCH 01/10] block/pflash_cfi02: Add test for supported commands

2019-04-09 Thread Thomas Huth
On 09/04/2019 10.35, Markus Armbruster wrote: > Thomas Huth writes: > >> On 09/04/2019 09.45, Markus Armbruster wrote: >>> Thomas Huth writes: >>> We'd like to get rid of global_qtest in the long run (since it is causing trouble for tests that run multiple instances of QEMU in

Re: [Qemu-block] [Qemu-devel] [PATCH 01/10] block/pflash_cfi02: Add test for supported commands

2019-04-09 Thread Markus Armbruster
Thomas Huth writes: > On 09/04/2019 09.45, Markus Armbruster wrote: >> Thomas Huth writes: >> >>> We'd like to get rid of global_qtest in the long run (since it is >>> causing trouble for tests that run multiple instances of QEMU in >>> parallel, e.g. migration tests)... so if it is feasible,

Re: [Qemu-block] [Qemu-devel] [RFC PATCH] hw/arm/virt: use variable size of flash device to save memory

2019-04-09 Thread Kevin Wolf
Am 09.04.2019 um 08:01 hat Markus Armbruster geschrieben: > László's last sentence below is "This really needs the attention of the > block people." Cc'ing some. > > Laszlo Ersek writes: > > > On 04/08/19 15:43, Xiang Zheng wrote: > >> > >> On 2019/4/3 23:35, Laszlo Ersek wrote: > I

Re: [Qemu-block] [Qemu-devel] [PATCH 01/10] block/pflash_cfi02: Add test for supported commands

2019-04-09 Thread Thomas Huth
On 09/04/2019 09.45, Markus Armbruster wrote: > Thomas Huth writes: > >> We'd like to get rid of global_qtest in the long run (since it is >> causing trouble for tests that run multiple instances of QEMU in >> parallel, e.g. migration tests)... so if it is feasible, please don't >> use it in new

Re: [Qemu-block] [Qemu-devel] [PATCH 01/10] block/pflash_cfi02: Add test for supported commands

2019-04-09 Thread Markus Armbruster
Thomas Huth writes: > We'd like to get rid of global_qtest in the long run (since it is > causing trouble for tests that run multiple instances of QEMU in > parallel, e.g. migration tests)... so if it is feasible, please don't > use it in new code anymore. Try to use a local variable in the

Re: [Qemu-block] [Qemu-devel] [PATCH 01/10] block/pflash_cfi02: Add test for supported commands

2019-04-09 Thread Thomas Huth
On 08/04/2019 22.55, Stephen Checkoway wrote: > From: Stephen Checkoway > > Test the AMD command set for parallel flash chips. This test uses an > ARM musicpal board with a pflash drive to test the following list of > currently-supported commands. > - Autoselect > - CFI > - Sector erase > - Chip

Re: [Qemu-block] [Qemu-devel] [PATCH 02/15] block/ssh: Do not report read/write/flush errors to the user

2019-04-09 Thread Markus Armbruster
Eric Blake writes: > On 4/8/19 3:36 AM, Markus Armbruster wrote: >> Callbacks ssh_co_readv(), ssh_co_writev(), ssh_co_flush() report >> errors to the user with error_printf(). They shouldn't, it's their >> caller's job. Replace by a suitable trace point. >> >> Perhaps we should convert this

Re: [Qemu-block] [Qemu-devel] Whither qemu's ssh driver?

2019-04-09 Thread Markus Armbruster
Max Reitz writes: > On 08.04.19 19:22, Richard W.M. Jones wrote: >> I don't know much about this patch which looks like internal qemu >> rearrangements so I guess fine. However I do have a few things to say >> about the ssh driver ... >> >> As you know I wrote this a few years ago, and it uses

Re: [Qemu-block] [Qemu-devel] [RFC PATCH] hw/arm/virt: use variable size of flash device to save memory

2019-04-09 Thread Markus Armbruster
László's last sentence below is "This really needs the attention of the block people." Cc'ing some. Laszlo Ersek writes: > On 04/08/19 15:43, Xiang Zheng wrote: >> >> On 2019/4/3 23:35, Laszlo Ersek wrote: I thought about your comments and wrote the following patch (just for test)