Re: [libvirt PATCH] tools: add virt-qmp-proxy for proxying QMP clients to libvirt QEMU guests
On Fri, May 27, 2022, 7:32 AM Daniel P. Berrangé
wrote:
> On Fri, May 27, 2022 at 12:20:39PM +0200, Peter Krempa wrote:
> > On Fri, May 27, 2022 at 10:47:58 +0100, Daniel P. Berrangé wrote:
> > > Libvirt provides QMP passthrough APIs for the QEMU driver and these are
> > > exposed in virsh. It is not especially pleasant, however, using the raw
> > > QMP JSON syntax. QEMU has a tool 'qmp-shell' which can speak QMP and
> > > exposes a human friendly interactive shell. It is not possible to use
> > > this with libvirt managed guest, however, since only one client can
> > > attach to he QMP socket at any point in time.
> > >
> > > The virt-qmp-proxy tool aims to solve this problem. It opens a UNIX
> > > socket and listens for incoming client connections, speaking QMP on
> > > the connected socket. It will forward any QMP commands received onto
> > > the running libvirt QEMU guest, and forward any replies back to the
> > > QMP client.
> > >
> > > $ virsh start demo
> > > $ virt-qmp-proxy demo demo.qmp &
> > > $ qmp-shell demo.qmp
> > > Welcome to the QMP low-level shell!
> > > Connected to QEMU 6.2.0
> > >
> > > (QEMU) query-kvm
> > > {
> > > "return": {
> > > "enabled": true,
> > > "present": true
> > > }
> > > }
> > >
> > > Note this tool of course has the same risks as the raw libvirt
> > > QMP passthrough. It is safe to run query commands to fetch information
> > > but commands which change the QEMU state risk disrupting libvirt's
> > > management of QEMU, potentially resulting in data loss/corruption in
> > > the worst case.
> > >
> > > Signed-off-by: Daniel P. Berrangé
> > > ---
> > >
> > > CC'ing QEMU since this is likely of interest to maintainers and users
> > > who work with QEMU and libvirt
> > >
> > > Note this impl is fairly crude in that it assumes it is receiving
> > > the QMP commands linewise one at a time. None the less it is good
> > > enough to work with qmp-shell already, so I figured it was worth
> > > exposing to the world. It also lacks support for forwarding events
> > > back to the QMP client.
> >
> > I originally wanted to teach the qemu tools to work with libvirt
> > directly similarly how 'scripts/render_block_graph.py' from the qemu
> > tree already does but I guess this is also an option.
>
> Yes, I do wonder about whether with John's new QMP python APIs,
> it would be possible to plug in a livirt transport instead of
> the socket transport. I've not spent enough time looking at the
> Python QMP code to know if that's viable or not though.
>
I can look into it. It looks like render_block_graph works by actually
executing a subprocess. Is there a chance of getting anything socket-like
or stream-like out of libvirt to work with instead?
As long as I can get some kind of stream going it should be easily possible
to just replace the fd(s) the qmp lib uses and talk to libvirt instead.
(possibly with changing some details about the handshake, yadda yadda.)
> > This is an option too albeit a bit more complex to set up, but on the
> > other hand a bit more universal.
>
> The two approaches aren't mutually exclusive either. There's no
> reason we can't have both options.
I've in the past thought about implementing MITM directly in the qmp
library. I wrote a QMPServer() class that I did not check in upstream
because it is only prototype quality, but I used it for testing. I still
have plans to upstream it because it is useful for the test suite.
It has the same limitation as a lot of our Python tools have, though; it
requires newlines because it doesn't have a streaming parser ...
It's been something that I want to improve upon, but since libvirt is
well-behaved and sends newlines, I haven't been properly motivated to.
Re: [PATCH v4 5/5] block: Deprecate transaction type drive-backup
On Mon, Oct 25, 2021 at 12:24 AM Markus Armbruster
wrote:
> Several moons ago, Vladimir posted
>
> Subject: [PATCH v2 3/3] qapi: deprecate drive-backup
> Date: Wed, 5 May 2021 16:58:03 +0300
> Message-Id: <20210505135803.67896-4-vsement...@virtuozzo.com>
> https://lists.gnu.org/archive/html/qemu-devel/2021-05/msg01394.html
>
> with this
>
> TODO: We also need to deprecate drive-backup transaction action..
> But union members in QAPI doesn't support 'deprecated' feature. I tried
> to dig a bit, but failed :/ Markus, could you please help with it? At
> least by advice?
>
> This is one way to resolve it. Sorry it took so long.
>
>
I'll share the blame for not pushing back on the other series, but ...
> John explored another way, namely adding feature flags to union
> branches. Could also be useful, say to add different features to
> branches in multiple unions sharing the same tag enum.
>
>
... this way seems simpler for now, and I trust your intuition on what's
easier to support as I don't have a solid grasp of the C interfaces at play
for actually parsing the input. We can always revisit the other thing later
if/when we need it.
> Signed-off-by: Markus Armbruster
> ---
> qapi/transaction.json | 6 +-
> 1 file changed, 5 insertions(+), 1 deletion(-)
>
> diff --git a/qapi/transaction.json b/qapi/transaction.json
> index d175b5f863..381a2df782 100644
> --- a/qapi/transaction.json
> +++ b/qapi/transaction.json
> @@ -54,6 +54,10 @@
> # @blockdev-snapshot-sync: since 1.1
> # @drive-backup: Since 1.6
> #
> +# Features:
> +# @deprecated: Member @drive-backup is deprecated. Use member
> +# @blockdev-backup instead.
> +#
> # Since: 1.1
> ##
> { 'enum': 'TransactionActionKind',
> @@ -62,7 +66,7 @@
> 'block-dirty-bitmap-disable', 'block-dirty-bitmap-merge',
> 'blockdev-backup', 'blockdev-snapshot',
> 'blockdev-snapshot-internal-sync', 'blockdev-snapshot-sync',
> -'drive-backup' ] }
> +{ 'name': 'drive-backup', 'features': [ 'deprecated' ] } ] }
>
> ##
> # @AbortWrapper:
> --
> 2.31.1
>
>
Seems pretty clean to me overall. What's the reason for wanting it to be
RFC?
Re: [PATCH v4 3/5] qapi: Move compat policy from QObject to generic visitor
On Mon, Oct 25, 2021 at 12:24 AM Markus Armbruster wrote: > The next commit needs to access compat policy from the generic visitor > core. Move it there from qobject input and output visitor. > > Signed-off-by: Markus Armbruster > Reviewed-by: Eric Blake > "LGTM".
Re: [PATCH v4 2/5] qapi: Add feature flags to enum members
On Mon, Oct 25, 2021 at 12:24 AM Markus Armbruster
wrote:
> This is quite similar to commit 84ab008687 "qapi: Add feature flags to
> struct members", only for enums instead of structs.
>
> Special feature flag 'deprecated' is silently ignored there. This is
> okay only because it will be implemented shortly.
>
> Signed-off-by: Markus Armbruster
> Reviewed-by: Eric Blake
>
Reviewed-by: John Snow
> ---
> docs/devel/qapi-code-gen.rst | 16 +-
> qapi/compat.json | 2 ++
> qapi/introspect.json | 5 -
> scripts/qapi/expr.py | 3 ++-
> scripts/qapi/introspect.py| 5 +++--
> scripts/qapi/schema.py| 22 +--
> tests/qapi-schema/doc-good.json | 5 -
> tests/qapi-schema/doc-good.out| 3 +++
> tests/qapi-schema/doc-good.txt| 3 +++
> .../qapi-schema/enum-dict-member-unknown.err | 2 +-
> tests/qapi-schema/qapi-schema-test.json | 3 ++-
> tests/qapi-schema/qapi-schema-test.out| 1 +
> tests/qapi-schema/test-qapi.py| 1 +
> 13 files changed, 57 insertions(+), 14 deletions(-)
>
> diff --git a/docs/devel/qapi-code-gen.rst b/docs/devel/qapi-code-gen.rst
> index d267889d2c..4071c9074a 100644
> --- a/docs/devel/qapi-code-gen.rst
> +++ b/docs/devel/qapi-code-gen.rst
> @@ -200,7 +200,9 @@ Syntax::
> '*if': COND,
> '*features': FEATURES }
> ENUM-VALUE = STRING
> - | { 'name': STRING, '*if': COND }
> + | { 'name': STRING,
> + '*if': COND,
> + '*features': FEATURES }
>
> Member 'enum' names the enum type.
>
> @@ -706,8 +708,10 @@ QEMU shows a certain behaviour.
> Special features
>
>
> -Feature "deprecated" marks a command, event, or struct member as
> -deprecated. It is not supported elsewhere so far.
> +Feature "deprecated" marks a command, event, enum value, or struct
> +member as deprecated. It is not supported elsewhere so far.
> +Interfaces so marked may be withdrawn in future releases in accordance
> +with QEMU's deprecation policy.
>
>
> Naming rules and reserved names
> @@ -1157,7 +1161,8 @@ and "variants".
>
> "members" is a JSON array describing the object's common members, if
> any. Each element is a JSON object with members "name" (the member's
> -name), "type" (the name of its type), and optionally "default". The
> +name), "type" (the name of its type), "features" (a JSON array of
> +feature strings), and "default". The latter two are optional. The
> member is optional if "default" is present. Currently, "default" can
> only have value null. Other values are reserved for future
> extensions. The "members" array is in no particular order; clients
> @@ -1234,7 +1239,8 @@ The SchemaInfo for an enumeration type has meta-type
> "enum" and
> variant member "members".
>
> "members" is a JSON array describing the enumeration values. Each
> -element is a JSON object with member "name" (the member's name). The
> +element is a JSON object with member "name" (the member's name), and
> +optionally "features" (a JSON array of feature strings). The
> "members" array is in no particular order; clients must search the
> entire array when learning whether a particular value is supported.
>
> diff --git a/qapi/compat.json b/qapi/compat.json
> index ae3afc22df..1d2b76f00c 100644
> --- a/qapi/compat.json
> +++ b/qapi/compat.json
> @@ -42,6 +42,8 @@
> # with feature 'deprecated'. We may want to extend it to cover
> # semantic aspects, CLI, and experimental features.
> #
> +# Limitation: not implemented for deprecated enumeration values.
> +#
> # @deprecated-input: how to handle deprecated input (default 'accept')
> # @deprecated-output: how to handle deprecated output (default 'accept')
> #
> diff --git a/qapi/introspect.json b/qapi/introspect.json
> index 9683e884f8..183148b2e9 100644
> --- a/qapi/introspect.json
> +++ b/qapi/introspect.json
> @@ -167,10 +167,13 @@
> #
> # @name: the member's name, as defined in the QAPI schema.
> #
> +# @features: names of features associated with the member, in no
> +#particular order.
> +#
> # Since: 6.2
> ##
> {
Re: [PATCH v4 1/5] qapi: Enable enum member introspection to show more than name
On Mon, Oct 25, 2021 at 12:24 AM Markus Armbruster wrote: > The next commit will add feature flags to enum members. There's a > problem, though: query-qmp-schema shows an enum type's members as an > array of member names (SchemaInfoEnum member @values). If it showed > an array of objects with a name member, we could simply add more > members to these objects. Since it's just strings, we can't. > > I can see three ways to correct this design mistake: > > 1. Do it the way we should have done it, plus compatibility goo. > >We want a ['SchemaInfoEnumMember'] member in SchemaInfoEnum. Since >changing @values would be a compatibility break, add a new member >@members instead. > >@values is now redundant. In my testing, output of >qemu-system-x86_64's query-qmp-schema grows by 11% (18.5KiB). > >We can deprecate @values now and drop it later. This will break >outmoded clients. Well-behaved clients such as libvirt are >expected to break cleanly. > > 2. Like 1, but omit "boring" elements of @member, and empty @member. > >@values does not become redundant. @members augments it. Somewhat >cumbersome, but output of query-qmp-schema grows only as we make >enum members non-boring. > >There is nothing to deprecate here. > > 3. Versioned query-qmp-schema. > >query-qmp-schema provides either @values or @members. The QMP >client can select which version it wants. There is no redundant >output. > >We can deprecate old versions and eventually drop them. This will >break outmoded clients. Breaking cleanly is easier than for 1. > >While 1 and 2 operate within the common rules for compatible >evolution apply (section "Compatibility considerations" in >docs/devel/qapi-code-gen.rst), 3 bypasses them. Attractive when >operating within the rules is just too awkward. Not the case here. > > This commit implements 1. Libvirt developers prefer it. > > Deprecate @values in favour of @members. Since query-qmp-schema > compatibility is pretty fundamental for management applications, an > extended grace period is advised. > > Signed-off-by: Markus Armbruster > Reviewed-by: Eric Blake > Tested-by: Peter Krempa > Acked-by: Peter Krempa > Reviewed-by: John Snow
Re: [PATCH 2/9] qapi: Mark unstable QMP parts with feature 'unstable'
On Tue, Oct 26, 2021 at 3:56 AM Markus Armbruster wrote: > John Snow writes: > > > On Mon, Oct 25, 2021 at 1:25 AM Markus Armbruster > wrote: > > > >> Add special feature 'unstable' everywhere the name starts with 'x-', > >> except for InputBarrierProperties member x-origin and > >> MemoryBackendProperties member x-use-canonical-path-for-ramblock-id, > >> because these two are actually stable. > >> > >> Signed-off-by: Markus Armbruster > >> --- > >> qapi/block-core.json | 123 +++ > >> qapi/migration.json | 35 +--- > >> qapi/misc.json | 6 ++- > >> qapi/qom.json| 11 ++-- > >> 4 files changed, 130 insertions(+), 45 deletions(-) > >> > >> diff --git a/qapi/block-core.json b/qapi/block-core.json > >> index 6d3217abb6..ce2c1352cb 100644 > >> --- a/qapi/block-core.json > >> +++ b/qapi/block-core.json > >> @@ -1438,6 +1438,9 @@ > >> # > >> # @x-perf: Performance options. (Since 6.0) > >> # > >> +# Features: > >> +# @unstable: Member @x-perf is experimental. > >> +# > >> > > > > It'd be a lot cooler if we could annotate the unstable member directly > > instead of confusing it with the syntax that might describe the entire > > struct/union/command/etc, but ... eh, it's just a doc field, so I'm not > > gonna press on this. I don't have the energy to get into a doc formatting > > standard discussion right now, so: sure, why not? > > By design, we have a single doc comment block for the entire definition. > > When Kevin invented feature flags (merge commit 4747524f9f2), he added > them just to struct types. He added "feature sections" for documenting > features. It mirrors the "argument sections" for documenting members. > Makes sense. > > Aside: he neglected to update qapi-code-gen.rst section "Definition > documentation", and I failed to catch it. I'll make up for it. > > Peter and I then added feature flags to the remaining definitions > (commit 23394b4c39 and 013b4efc9b). Feature sections make sense there, > too. > > I then added them to struct members (commit 84ab008687). Instead of > doing something fancy for documenting feature flags of members, I simply > used the existing feature sections. This conflates member features with > struct features. > > Yeah, that's the part I don't like. If this isn't the first instance of breaking the seal, though, this is the wrong time for me to comment on it. Don't worry about it for this series. > What could "something fancy" be? All we have for members is "argument > sections", which are basically a name plus descriptive text. We'd have > to add structure to that, say nest feature sections into argument > sections. I have no appetite for that right now. > > (Tangent below, unrelated to acceptance of this series) Yeah, I don't have an appetite for it at the moment either. I'll have to read Marc-Andre's recent sphinx patches and see if I want to do more work -- I had a bigger prototype a few months back I didn't bring all the way home, but I am still thinking about reworking our QAPIDoc format. It's ... well. I don't really want to, but I am not sure how else to bring some of the features I want home, and I think I need some more time to think carefully through exactly what I want to do and why. I wouldn't mind chatting about it with you sometime soon -- there's a few things to balance here, such as: (1) Reworking the doc format would be an obnoxious amount of churn, ... (2) ...but the Sphinx internals are really not meant to be used the way we're using them right now, ... (3) ...but I also don't want to write our QAPI docstrings in a way that *targets* Sphinx. Not that I think we'll be dropping it any time soon, but it still feels like the wrong idea to tie them so closely together. I'm hoping there's an opportunity to make the parsing nicer with minimal changes to the parsing and format, though. It just might require enforcing a *pinch* more structure. I could see how I feel about per-field annotations at that point. I consider them interesting for things like the Python SDK where I may want to enable/disable warnings for using deprecated stuff at the client-level. (e.g., let's say you're using Python SDK 6.2 to talk to a 6.1 client. Nothing stops you from doing this, but some commands will simply be rejected by QEMU as it won't know what you're talking about. Using deprecated fields or commands to talk to an older client wi
Re: [PATCH 7/9] qapi: Generalize enum member policy checking
On Tue, Oct 26, 2021 at 5:43 AM Markus Armbruster wrote:
> John Snow writes:
>
> > On Mon, Oct 25, 2021 at 1:26 AM Markus Armbruster
> wrote:
> >
> >> The code to check enumeration value policy can see special feature
> >> flag 'deprecated' in QEnumLookup member flags[value]. I want to make
> >> feature flag 'unstable' visible there as well, so I can add policy for
> >> it.
> >>
> >> Instead of extending flags[], replace it by @special_features (a
> >> bitset of QapiSpecialFeature), because that's how special features get
> >> passed around elsewhere.
> >>
> >> Signed-off-by: Markus Armbruster
> >> ---
> >> include/qapi/util.h| 5 +
> >> qapi/qapi-visit-core.c | 3 ++-
> >> scripts/qapi/types.py | 22 --
> >> 3 files changed, 15 insertions(+), 15 deletions(-)
> >>
> >> diff --git a/include/qapi/util.h b/include/qapi/util.h
> >> index 7a8d5c7d72..0cc98db9f9 100644
> >> --- a/include/qapi/util.h
> >> +++ b/include/qapi/util.h
> >> @@ -15,12 +15,9 @@ typedef enum {
> >> QAPI_DEPRECATED,
> >> } QapiSpecialFeature;
> >>
> >> -/* QEnumLookup flags */
> >> -#define QAPI_ENUM_DEPRECATED 1
> >> -
> >> typedef struct QEnumLookup {
> >> const char *const *array;
> >> -const unsigned char *const flags;
> >> +const unsigned char *const special_features;
> >> const int size;
> >> } QEnumLookup;
> >>
> >> diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
> >> index b4a81f1757..5572d90efb 100644
> >> --- a/qapi/qapi-visit-core.c
> >> +++ b/qapi/qapi-visit-core.c
> >> @@ -407,7 +407,8 @@ static bool input_type_enum(Visitor *v, const char
> >> *name, int *obj,
> >> return false;
> >> }
> >>
> >> -if (lookup->flags && (lookup->flags[value] &
> QAPI_ENUM_DEPRECATED)) {
> >> +if (lookup->special_features
> >> +&& (lookup->special_features[value] & QAPI_DEPRECATED)) {
> >> switch (v->compat_policy.deprecated_input) {
> >> case COMPAT_POLICY_INPUT_ACCEPT:
> >> break;
> >> diff --git a/scripts/qapi/types.py b/scripts/qapi/types.py
> >> index ab2441adc9..3013329c24 100644
> >> --- a/scripts/qapi/types.py
> >> +++ b/scripts/qapi/types.py
> >> @@ -16,7 +16,7 @@
> >> from typing import List, Optional
> >>
> >> from .common import c_enum_const, c_name, mcgen
> >> -from .gen import QAPISchemaModularCVisitor, ifcontext
> >> +from .gen import QAPISchemaModularCVisitor, gen_special_features,
> >> ifcontext
> >> from .schema import (
> >> QAPISchema,
> >> QAPISchemaEnumMember,
> >> @@ -39,7 +39,7 @@ def gen_enum_lookup(name: str,
> >> members: List[QAPISchemaEnumMember],
> >> prefix: Optional[str] = None) -> str:
> >> max_index = c_enum_const(name, '_MAX', prefix)
> >> -flags = ''
> >> +feats = ''
> >> ret = mcgen('''
> >>
> >> const QEnumLookup %(c_name)s_lookup = {
> >> @@ -54,19 +54,21 @@ def gen_enum_lookup(name: str,
> >> ''',
> >> index=index, name=memb.name)
> >> ret += memb.ifcond.gen_endif()
> >> -if 'deprecated' in (f.name for f in memb.features):
> >> -flags += mcgen('''
> >> -[%(index)s] = QAPI_ENUM_DEPRECATED,
> >> -''',
> >> - index=index)
> >>
> >> -if flags:
> >> +special_features = gen_special_features(memb.features)
> >> +if special_features != '0':
> >>
> >
> > Though, I have to admit the common reoccurrence of this pattern suggests
> to
> > me that gen_special_features really ought to be returning something
> false-y
> > in these cases. Something about testing for the empty case with something
> > that represents, but isn't empty, gives me a brief pause.
> >
> > Not willing to wage war over it.
>
> I habitually start stupid, and when stupid confuses or annoys me later,
> I smarten it up some.
>
> Let's see how this instance of "stupid" ages, okay?
>
>
"I see what you're saying, but meh" is a relatable feeling ;)
>
> >> +feats += mcgen('''
> >> +[%(index)s] = %(special_features)s,
> >> +''',
> >> + index=index,
> special_features=special_features)
> >> +
> >> +if feats:
> >> ret += mcgen('''
> >> },
> >> -.flags = (const unsigned char[%(max_index)s]) {
> >> +.special_features = (const unsigned char[%(max_index)s]) {
> >> ''',
> >> max_index=max_index)
> >> -ret += flags
> >> +ret += feats
> >>
> >> ret += mcgen('''
> >> },
> >> --
> >> 2.31.1
> >>
> >>
> > Python bits: Acked-by: John Snow
>
> Thanks!
>
>
Re: [PATCH 8/9] qapi: Factor out compat_policy_input_ok()
On Mon, Oct 25, 2021 at 1:25 AM Markus Armbruster wrote: > The code to check policy for handling deprecated input is triplicated. > Factor it out into compat_policy_input_ok() before I mess with it in > the next commit. > > Signed-off-by: Markus Armbruster > (Skipping C-only patches for quick review. I'll trust you on these.) --js
Re: [PATCH 9/9] qapi: Extend -compat to set policy for unstable interfaces
Policy for handling deprecated management
> interfaces\n",
> +"Policy for handling deprecated management
> interfaces\n"
> +"-compat
> [unstable-input=accept|reject|crash][,unstable-output=accept|hide]\n"
> +"Policy for handling unstable management
> interfaces\n",
> QEMU_ARCH_ALL)
> SRST
> ``-compat
> [deprecated-input=@var{input-policy}][,deprecated-output=@var{output-policy}]``
> @@ -3659,6 +3661,22 @@ SRST
> Suppress deprecated command results and events
>
> Limitation: covers only syntactic aspects of QMP.
> +
> +``-compat
> [unstable-input=@var{input-policy}][,unstable-output=@var{output-policy}]``
> +Set policy for handling unstable management interfaces (experimental):
> +
> +``unstable-input=accept`` (default)
> +Accept unstable commands and arguments
> +``unstable-input=reject``
> +Reject unstable commands and arguments
> +``unstable-input=crash``
> +Crash on unstable commands and arguments
> +``unstable-output=accept`` (default)
> +Emit unstable command results and events
> +``unstable-output=hide``
> +Suppress unstable command results and events
> +
> +Limitation: covers only syntactic aspects of QMP.
> ERST
>
> DEF("fw_cfg", HAS_ARG, QEMU_OPTION_fwcfg,
> diff --git a/scripts/qapi/events.py b/scripts/qapi/events.py
> index 82475e84ec..27b44c49f5 100644
> --- a/scripts/qapi/events.py
> +++ b/scripts/qapi/events.py
> @@ -109,13 +109,15 @@ def gen_event_send(name: str,
> if not boxed:
> ret += gen_param_var(arg_type)
>
> -if 'deprecated' in [f.name for f in features]:
> -ret += mcgen('''
> +for f in features:
> +if f.is_special():
> +ret += mcgen('''
>
> -if (compat_policy.deprecated_output == COMPAT_POLICY_OUTPUT_HIDE) {
> +if (compat_policy.%(feat)s_output == COMPAT_POLICY_OUTPUT_HIDE) {
> return;
> }
> -''')
> +''',
> + feat=f.name)
>
> ret += mcgen('''
>
> diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py
> index 55f82d7389..b7b3fc0ce4 100644
> --- a/scripts/qapi/schema.py
> +++ b/scripts/qapi/schema.py
> @@ -254,9 +254,11 @@ def doc_type(self):
>
> def check(self, schema):
> QAPISchemaEntity.check(self, schema)
> -if 'deprecated' in [f.name for f in self.features]:
> -raise QAPISemError(
> -self.info, "feature 'deprecated' is not supported for
> types")
> +for feat in self.features:
> +if feat.is_special():
> +raise QAPISemError(
> +self.info,
> +f"feature '{feat.name}' is not supported for types")
>
> def describe(self):
> assert self.meta
> @@ -726,7 +728,7 @@ class QAPISchemaFeature(QAPISchemaMember):
> role = 'feature'
>
> def is_special(self):
> -return self.name in ('deprecated')
> +return self.name in ('deprecated', 'unstable')
>
>
> class QAPISchemaObjectTypeMember(QAPISchemaMember):
> --
> 2.31.1
>
>
Python bits: Acked-by: John Snow
Looks good overall from what I can see, minor style quibbles that I'd
probably fold on if you frowned at me.
--js
Re: [PATCH 7/9] qapi: Generalize enum member policy checking
On Mon, Oct 25, 2021 at 1:26 AM Markus Armbruster wrote:
> The code to check enumeration value policy can see special feature
> flag 'deprecated' in QEnumLookup member flags[value]. I want to make
> feature flag 'unstable' visible there as well, so I can add policy for
> it.
>
> Instead of extending flags[], replace it by @special_features (a
> bitset of QapiSpecialFeature), because that's how special features get
> passed around elsewhere.
>
> Signed-off-by: Markus Armbruster
> ---
> include/qapi/util.h| 5 +
> qapi/qapi-visit-core.c | 3 ++-
> scripts/qapi/types.py | 22 --
> 3 files changed, 15 insertions(+), 15 deletions(-)
>
> diff --git a/include/qapi/util.h b/include/qapi/util.h
> index 7a8d5c7d72..0cc98db9f9 100644
> --- a/include/qapi/util.h
> +++ b/include/qapi/util.h
> @@ -15,12 +15,9 @@ typedef enum {
> QAPI_DEPRECATED,
> } QapiSpecialFeature;
>
> -/* QEnumLookup flags */
> -#define QAPI_ENUM_DEPRECATED 1
> -
> typedef struct QEnumLookup {
> const char *const *array;
> -const unsigned char *const flags;
> +const unsigned char *const special_features;
> const int size;
> } QEnumLookup;
>
> diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
> index b4a81f1757..5572d90efb 100644
> --- a/qapi/qapi-visit-core.c
> +++ b/qapi/qapi-visit-core.c
> @@ -407,7 +407,8 @@ static bool input_type_enum(Visitor *v, const char
> *name, int *obj,
> return false;
> }
>
> -if (lookup->flags && (lookup->flags[value] & QAPI_ENUM_DEPRECATED)) {
> +if (lookup->special_features
> +&& (lookup->special_features[value] & QAPI_DEPRECATED)) {
> switch (v->compat_policy.deprecated_input) {
> case COMPAT_POLICY_INPUT_ACCEPT:
> break;
> diff --git a/scripts/qapi/types.py b/scripts/qapi/types.py
> index ab2441adc9..3013329c24 100644
> --- a/scripts/qapi/types.py
> +++ b/scripts/qapi/types.py
> @@ -16,7 +16,7 @@
> from typing import List, Optional
>
> from .common import c_enum_const, c_name, mcgen
> -from .gen import QAPISchemaModularCVisitor, ifcontext
> +from .gen import QAPISchemaModularCVisitor, gen_special_features,
> ifcontext
> from .schema import (
> QAPISchema,
> QAPISchemaEnumMember,
> @@ -39,7 +39,7 @@ def gen_enum_lookup(name: str,
> members: List[QAPISchemaEnumMember],
> prefix: Optional[str] = None) -> str:
> max_index = c_enum_const(name, '_MAX', prefix)
> -flags = ''
> +feats = ''
> ret = mcgen('''
>
> const QEnumLookup %(c_name)s_lookup = {
> @@ -54,19 +54,21 @@ def gen_enum_lookup(name: str,
> ''',
> index=index, name=memb.name)
> ret += memb.ifcond.gen_endif()
> -if 'deprecated' in (f.name for f in memb.features):
> -flags += mcgen('''
> -[%(index)s] = QAPI_ENUM_DEPRECATED,
> -''',
> - index=index)
>
> -if flags:
> +special_features = gen_special_features(memb.features)
> +if special_features != '0':
>
Though, I have to admit the common reoccurrence of this pattern suggests to
me that gen_special_features really ought to be returning something false-y
in these cases. Something about testing for the empty case with something
that represents, but isn't empty, gives me a brief pause.
Not willing to wage war over it.
> +feats += mcgen('''
> +[%(index)s] = %(special_features)s,
> +''',
> + index=index, special_features=special_features)
> +
> +if feats:
> ret += mcgen('''
> },
> -.flags = (const unsigned char[%(max_index)s]) {
> +.special_features = (const unsigned char[%(max_index)s]) {
> ''',
> max_index=max_index)
> -ret += flags
> +ret += feats
>
> ret += mcgen('''
> },
> --
> 2.31.1
>
>
Python bits: Acked-by: John Snow
Re: [PATCH 6/9] qapi: Generalize command policy checking
ode);
> }
>
> diff --git a/storage-daemon/qemu-storage-daemon.c
> b/storage-daemon/qemu-storage-daemon.c
> index 10a1a33761..52cf17e8ac 100644
> --- a/storage-daemon/qemu-storage-daemon.c
> +++ b/storage-daemon/qemu-storage-daemon.c
> @@ -146,7 +146,8 @@ static void init_qmp_commands(void)
>
> QTAILQ_INIT(&qmp_cap_negotiation_commands);
> qmp_register_command(&qmp_cap_negotiation_commands,
> "qmp_capabilities",
> - qmp_marshal_qmp_capabilities,
> QCO_ALLOW_PRECONFIG);
> + qmp_marshal_qmp_capabilities,
> + QCO_ALLOW_PRECONFIG, 0);
> }
>
> static int getopt_set_loc(int argc, char **argv, const char *optstring,
> diff --git a/scripts/qapi/commands.py b/scripts/qapi/commands.py
> index c8a975528f..21001bbd6b 100644
> --- a/scripts/qapi/commands.py
> +++ b/scripts/qapi/commands.py
> @@ -26,6 +26,7 @@
> QAPISchemaModularCVisitor,
> build_params,
> ifcontext,
> +gen_special_features,
> )
> from .schema import (
> QAPISchema,
> @@ -217,9 +218,6 @@ def gen_register_command(name: str,
> coroutine: bool) -> str:
> options = []
>
> -if 'deprecated' in [f.name for f in features]:
> -options += ['QCO_DEPRECATED']
> -
> if not success_response:
> options += ['QCO_NO_SUCCESS_RESP']
> if allow_oob:
> @@ -231,10 +229,11 @@ def gen_register_command(name: str,
>
> ret = mcgen('''
> qmp_register_command(cmds, "%(name)s",
> - qmp_marshal_%(c_name)s, %(opts)s);
> + qmp_marshal_%(c_name)s, %(opts)s, %(feats)s);
> ''',
> name=name, c_name=c_name(name),
> -opts=' | '.join(options) or 0)
> +opts=' | '.join(options) or 0,
> +feats=gen_special_features(features))
>
Ah, you use the '0' return here. Alright then.
> return ret
>
>
> --
> 2.31.1
>
>
Python bits: Acked-by: John Snow
C bits: "I believe in my heart that they probably work."
(for this and previous patch.)
Re: [PATCH 5/9] qapi: Generalize struct member policy checking
On Mon, Oct 25, 2021 at 1:25 AM Markus Armbruster wrote:
> The generated visitor functions call visit_deprecated_accept() and
> visit_deprecated() when visiting a struct member with special feature
> flag 'deprecated'. This makes the feature flag visible to the actual
> visitors. I want to make feature flag 'unstable' visible there as
> well, so I can add policy for it.
>
> To let me make it visible, replace these functions by
> visit_policy_reject() and visit_policy_skip(), which take the member's
> special features as an argument. Note that the new functions have the
> opposite sense, i.e. the return value flips.
>
> Signed-off-by: Markus Armbruster
> ---
> include/qapi/visitor-impl.h | 6 --
> include/qapi/visitor.h| 17 +
> qapi/qapi-forward-visitor.c | 16 +---
> qapi/qapi-visit-core.c| 22 --
> qapi/qobject-input-visitor.c | 15 ++-
> qapi/qobject-output-visitor.c | 9 ++---
> qapi/trace-events | 4 ++--
> scripts/qapi/visit.py | 14 +++---
> 8 files changed, 63 insertions(+), 40 deletions(-)
>
> diff --git a/include/qapi/visitor-impl.h b/include/qapi/visitor-impl.h
> index 72b6537bef..2badec5ba4 100644
> --- a/include/qapi/visitor-impl.h
> +++ b/include/qapi/visitor-impl.h
> @@ -114,10 +114,12 @@ struct Visitor
> void (*optional)(Visitor *v, const char *name, bool *present);
>
> /* Optional */
> -bool (*deprecated_accept)(Visitor *v, const char *name, Error **errp);
> +bool (*policy_reject)(Visitor *v, const char *name,
> + unsigned special_features, Error **errp);
>
> /* Optional */
> -bool (*deprecated)(Visitor *v, const char *name);
> +bool (*policy_skip)(Visitor *v, const char *name,
> +unsigned special_features);
>
> /* Must be set */
> VisitorType type;
> diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
> index dcb96018a9..d53a84c9ba 100644
> --- a/include/qapi/visitor.h
> +++ b/include/qapi/visitor.h
> @@ -461,22 +461,31 @@ void visit_end_alternate(Visitor *v, void **obj);
> bool visit_optional(Visitor *v, const char *name, bool *present);
>
> /*
> - * Should we reject deprecated member @name?
> + * Should we reject member @name due to policy?
> + *
> + * @special_features is the member's special features encoded as a
> + * bitset of QapiSpecialFeature.
> *
> * @name must not be NULL. This function is only useful between
> * visit_start_struct() and visit_end_struct(), since only objects
> * have deprecated members.
> */
> -bool visit_deprecated_accept(Visitor *v, const char *name, Error **errp);
> +bool visit_policy_reject(Visitor *v, const char *name,
> + unsigned special_features, Error **errp);
>
> /*
> - * Should we visit deprecated member @name?
> + *
> + * Should we skip member @name due to policy?
> + *
> + * @special_features is the member's special features encoded as a
> + * bitset of QapiSpecialFeature.
> *
> * @name must not be NULL. This function is only useful between
> * visit_start_struct() and visit_end_struct(), since only objects
> * have deprecated members.
> */
> -bool visit_deprecated(Visitor *v, const char *name);
> +bool visit_policy_skip(Visitor *v, const char *name,
> + unsigned special_features);
>
> /*
> * Set policy for handling deprecated management interfaces.
> diff --git a/qapi/qapi-forward-visitor.c b/qapi/qapi-forward-visitor.c
> index a4b111e22a..25d098aa8a 100644
> --- a/qapi/qapi-forward-visitor.c
> +++ b/qapi/qapi-forward-visitor.c
> @@ -246,25 +246,27 @@ static void forward_field_optional(Visitor *v, const
> char *name, bool *present)
> visit_optional(ffv->target, name, present);
> }
>
> -static bool forward_field_deprecated_accept(Visitor *v, const char *name,
> -Error **errp)
> +static bool forward_field_policy_reject(Visitor *v, const char *name,
> +unsigned special_features,
> +Error **errp)
> {
> ForwardFieldVisitor *ffv = to_ffv(v);
>
> if (!forward_field_translate_name(ffv, &name, errp)) {
> return false;
> }
> -return visit_deprecated_accept(ffv->target, name, errp);
> +return visit_policy_reject(ffv->target, name, special_features, errp);
> }
>
> -static bool forward_field_deprecated(Visitor *v, const char *name)
> +static bool forward_field_policy_skip(Visitor *v, const char *name,
> + unsigned special_features)
> {
> ForwardFieldVisitor *ffv = to_ffv(v);
>
> if (!forward_field_translate_name(ffv, &name, NULL)) {
> return false;
> }
> -return visit_deprecated(ffv->target, name);
> +return visit_policy_skip(ffv->target, name, special_features);
> }
>
> static void forward_field_complete(Visitor *v, void *opaque)
>
Re: [PATCH 4/9] qapi: Tools for sets of special feature flags in generated code
On Mon, Oct 25, 2021 at 1:25 AM Markus Armbruster wrote:
> New enum QapiSpecialFeature enumerates the special feature flags.
>
> New helper gen_special_features() returns code to represent a
> collection of special feature flags as a bitset.
>
> The next few commits will put them to use.
>
> Signed-off-by: Markus Armbruster
> ---
> include/qapi/util.h| 4
> scripts/qapi/gen.py| 13 +
> scripts/qapi/schema.py | 3 +++
> 3 files changed, 20 insertions(+)
>
> diff --git a/include/qapi/util.h b/include/qapi/util.h
> index 257c600f99..7a8d5c7d72 100644
> --- a/include/qapi/util.h
> +++ b/include/qapi/util.h
> @@ -11,6 +11,10 @@
> #ifndef QAPI_UTIL_H
> #define QAPI_UTIL_H
>
> +typedef enum {
> +QAPI_DEPRECATED,
> +} QapiSpecialFeature;
> +
> /* QEnumLookup flags */
> #define QAPI_ENUM_DEPRECATED 1
>
> diff --git a/scripts/qapi/gen.py b/scripts/qapi/gen.py
> index 2ec1e7b3b6..9d07b88cf6 100644
> --- a/scripts/qapi/gen.py
> +++ b/scripts/qapi/gen.py
> @@ -29,6 +29,7 @@
> mcgen,
> )
> from .schema import (
> +QAPISchemaFeature,
> QAPISchemaIfCond,
> QAPISchemaModule,
> QAPISchemaObjectType,
> @@ -37,6 +38,18 @@
> from .source import QAPISourceInfo
>
>
> +def gen_special_features(features: QAPISchemaFeature):
> +ret = ''
> +sep = ''
> +
> +for feat in features:
> +if feat.is_special():
> +ret += ('%s1u << QAPI_%s' % (sep, feat.name.upper()))
>
Building the constant name here "feels" fragile, but I'll trust that the
test suite and/or the compiler will catch us if we accidentally goof up
this mapping. In the interest of simplicity, then, "sure, why not."
> +sep = ' | '
> +
>
+return ret or '0'
> +
>
Subjectively more pythonic:
special_features = [f"1u << QAPI_{feat.name.upper()}" for feat in features
if feat.is_special()]
ret = ' | '.join(special_features)
return ret or '0'
A bit more dense, but more functional. Up to you, but I find join() easier
to read and reason about for the presence of separators.
> +
> class QAPIGen:
> def __init__(self, fname: str):
> self.fname = fname
> diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py
> index 6d5f46509a..55f82d7389 100644
> --- a/scripts/qapi/schema.py
> +++ b/scripts/qapi/schema.py
> @@ -725,6 +725,9 @@ def connect_doc(self, doc):
> class QAPISchemaFeature(QAPISchemaMember):
> role = 'feature'
>
> +def is_special(self):
> +return self.name in ('deprecated')
> +
>
alrighty.
(Briefly wondered: is it worth naming special features as a property of the
class, but with only two names, it's probably fine enough to leave it
embedded in the method logic. Only a style thing and doesn't have any
actual impact that I can imagine, so ... nevermind.)
>
> class QAPISchemaObjectTypeMember(QAPISchemaMember):
> def __init__(self, name, info, typ, optional, ifcond=None,
> features=None):
> --
> 2.31.1
>
>
Well, either way:
Reviewed-by: John Snow
Re: [PATCH 3/9] qapi: Eliminate QCO_NO_OPTIONS for a slight simplification
On Mon, Oct 25, 2021 at 1:25 AM Markus Armbruster wrote:
> Signed-off-by: Markus Armbruster
> ---
> include/qapi/qmp/dispatch.h | 1 -
> monitor/misc.c | 3 +--
> scripts/qapi/commands.py| 5 +
> 3 files changed, 2 insertions(+), 7 deletions(-)
>
> diff --git a/include/qapi/qmp/dispatch.h b/include/qapi/qmp/dispatch.h
> index 075203dc67..0ce88200b9 100644
> --- a/include/qapi/qmp/dispatch.h
> +++ b/include/qapi/qmp/dispatch.h
> @@ -21,7 +21,6 @@ typedef void (QmpCommandFunc)(QDict *, QObject **, Error
> **);
>
> typedef enum QmpCommandOptions
> {
> -QCO_NO_OPTIONS= 0x0,
> QCO_NO_SUCCESS_RESP = (1U << 0),
> QCO_ALLOW_OOB = (1U << 1),
> QCO_ALLOW_PRECONFIG = (1U << 2),
> diff --git a/monitor/misc.c b/monitor/misc.c
> index ffe7966870..3556b177f6 100644
> --- a/monitor/misc.c
> +++ b/monitor/misc.c
> @@ -230,8 +230,7 @@ static void monitor_init_qmp_commands(void)
>
> qmp_init_marshal(&qmp_commands);
>
> -qmp_register_command(&qmp_commands, "device_add", qmp_device_add,
> - QCO_NO_OPTIONS);
> +qmp_register_command(&qmp_commands, "device_add", qmp_device_add, 0);
>
> QTAILQ_INIT(&qmp_cap_negotiation_commands);
> qmp_register_command(&qmp_cap_negotiation_commands,
> "qmp_capabilities",
> diff --git a/scripts/qapi/commands.py b/scripts/qapi/commands.py
> index 3654825968..c8a975528f 100644
> --- a/scripts/qapi/commands.py
> +++ b/scripts/qapi/commands.py
> @@ -229,15 +229,12 @@ def gen_register_command(name: str,
> if coroutine:
> options += ['QCO_COROUTINE']
>
> -if not options:
> -options = ['QCO_NO_OPTIONS']
> -
> ret = mcgen('''
> qmp_register_command(cmds, "%(name)s",
> qmp_marshal_%(c_name)s, %(opts)s);
> ''',
> name=name, c_name=c_name(name),
> - opts=" | ".join(options))
> +opts=' | '.join(options) or 0)
> return ret
>
>
>
I'm not a big fan of naked constants on the C side, but the generator
simplification is nice. I suppose it's worth the trade-off if you like it
better this way.
"eh".
Reviewed-by: John Snow
Re: [PATCH 2/9] qapi: Mark unstable QMP parts with feature 'unstable'
'xbzrle', 'rdma-pin-all', 'auto-converge', 'zero-blocks',
> - 'compress', 'events', 'postcopy-ram', 'x-colo', 'release-ram',
> + 'compress', 'events', 'postcopy-ram',
> + { 'name': 'x-colo', 'features': [ 'unstable' ] },
> + 'release-ram',
> 'block', 'return-path', 'pause-before-switchover', 'multifd',
> 'dirty-bitmaps', 'postcopy-blocktime', 'late-block-activate',
> - 'x-ignore-shared', 'validate-uuid', 'background-snapshot'] }
> + { 'name': 'x-ignore-shared', 'features': [ 'unstable' ] },
> + 'validate-uuid', 'background-snapshot'] }
>
> ##
> # @MigrationCapabilityStatus:
> @@ -743,6 +749,9 @@
> #block device name if there is one, and to their
> node name
> #otherwise. (Since 5.2)
> #
> +# Features:
> +# @unstable: Member @x-checkpoint-delay is experimental.
> +#
> # Since: 2.4
> ##
> { 'enum': 'MigrationParameter',
> @@ -753,7 +762,9 @@
> 'cpu-throttle-initial', 'cpu-throttle-increment',
> 'cpu-throttle-tailslow',
> 'tls-creds', 'tls-hostname', 'tls-authz', 'max-bandwidth',
> - 'downtime-limit', 'x-checkpoint-delay', 'block-incremental',
> + 'downtime-limit',
> + { 'name': 'x-checkpoint-delay', 'features': [ 'unstable' ] },
> + 'block-incremental',
> 'multifd-channels',
> 'xbzrle-cache-size', 'max-postcopy-bandwidth',
> 'max-cpu-throttle', 'multifd-compression',
> @@ -903,6 +914,9 @@
> #block device name if there is one, and to their
> node name
> #otherwise. (Since 5.2)
> #
> +# Features:
> +# @unstable: Member @x-checkpoint-delay is experimental.
> +#
> # Since: 2.4
> ##
> # TODO either fuse back into MigrationParameters, or make
> @@ -925,7 +939,8 @@
> '*tls-authz': 'StrOrNull',
> '*max-bandwidth': 'size',
> '*downtime-limit': 'uint64',
> -'*x-checkpoint-delay': 'uint32',
> +'*x-checkpoint-delay': { 'type': 'uint32',
> + 'features': [ 'unstable' ] },
> '*block-incremental': 'bool',
> '*multifd-channels': 'uint8',
> '*xbzrle-cache-size': 'size',
> @@ -1099,6 +1114,9 @@
> #block device name if there is one, and to their
> node name
> #otherwise. (Since 5.2)
> #
> +# Features:
> +# @unstable: Member @x-checkpoint-delay is experimental.
> +#
> # Since: 2.4
> ##
> { 'struct': 'MigrationParameters',
> @@ -1119,7 +1137,8 @@
> '*tls-authz': 'str',
> '*max-bandwidth': 'size',
> '*downtime-limit': 'uint64',
> -'*x-checkpoint-delay': 'uint32',
> +'*x-checkpoint-delay': { 'type': 'uint32',
> + 'features': [ 'unstable' ] },
> '*block-incremental': 'bool',
> '*multifd-channels': 'uint8',
> '*xbzrle-cache-size': 'size',
> @@ -1351,6 +1370,9 @@
> # If sent to the Secondary, the Secondary side will run failover work,
> # then takes over server operation to become the service VM.
> #
> +# Features:
> +# @unstable: This command is experimental.
> +#
> # Since: 2.8
> #
> # Example:
> @@ -1359,7 +1381,8 @@
> # <- { "return": {} }
> #
> ##
> -{ 'command': 'x-colo-lost-heartbeat' }
> +{ 'command': 'x-colo-lost-heartbeat',
> + 'features': [ 'unstable' ] }
>
> ##
> # @migrate_cancel:
> diff --git a/qapi/misc.json b/qapi/misc.json
> index 5c2ca3b556..358548abe1 100644
> --- a/qapi/misc.json
> +++ b/qapi/misc.json
> @@ -185,6 +185,9 @@
> # available during the preconfig state (i.e. when the --preconfig command
> # line option was in use).
> #
> +# Features:
> +# @unstable: This command is experimental.
> +#
> # Since 3.0
> #
> # Returns: nothing
> @@ -195,7 +198,8 @@
> # <- { "return": {} }
> #
> ##
> -{ 'command': 'x-exit-preconfig', 'allow-preconfig': true }
> +{ 'command': 'x-exit-preconfig', 'allow-preconfig': true,
> + 'features': [ 'unstable' ] }
>
> ##
> # @human-monitor-command:
> diff --git a/qapi/qom.json b/qapi/qom.json
> index 7231ac3f34..ccd1167808 100644
> --- a/qapi/qom.json
> +++ b/qapi/qom.json
> @@ -559,10 +559,8 @@
> #for ramblock-id. Disable this
> for 4.0
> #machine types or older to allow
> #migration with newer QEMU
> versions.
> -#This option is considered stable
> -#despite the x- prefix. (default:
> -#false generally, but true for
> machine
> -#types <= 4.0)
> +#(default: false generally,
> +#but true for machine types <=
> 4.0)
> #
> # Note: prealloc=true and reserve=false cannot be set at the same time.
> With
> # reserve=true, the behavior depends on the operating system: for
> example,
> @@ -785,6 +783,9 @@
> ##
> # @ObjectType:
> #
> +# Features:
> +# @unstable: Member @x-remote-object is experimental.
> +#
> # Since: 6.0
> ##
> { 'enum': 'ObjectType',
> @@ -836,7 +837,7 @@
> 'tls-creds-psk',
> 'tls-creds-x509',
> 'tls-cipher-suites',
> -'x-remote-object'
> +{ 'name': 'x-remote-object', 'features': [ 'unstable' ] }
>] }
>
> ##
> --
> 2.31.1
>
>
Seems OK, but I didn't audit for false positives/negatives. Trusting your
judgment here. (It looks like Phil started to audit this in his reply to
your previous commit, so I'll trust that.)
Acked-by: John Snow
Re: [PATCH 1/9] qapi: New special feature flag "unstable"
s': [ 'unstable' ] } > diff --git a/tests/qapi-schema/qapi-schema-test.out > b/tests/qapi-schema/qapi-schema-test.out > index 16846dbeb8..1f9585fa9b 100644 > --- a/tests/qapi-schema/qapi-schema-test.out > +++ b/tests/qapi-schema/qapi-schema-test.out > @@ -308,6 +308,7 @@ object FeatureStruct1 > feature feature1 > object FeatureStruct2 > member foo: int optional=False > +feature unstable > feature feature1 > object FeatureStruct3 > member foo: int optional=False > @@ -373,6 +374,7 @@ command test-command-features1 None -> None > feature deprecated > command test-command-features3 None -> None > gen=True success_response=True boxed=False oob=False preconfig=False > +feature unstable > feature feature1 > feature feature2 > command test-command-cond-features1 None -> None > @@ -394,6 +396,9 @@ event TEST_EVENT_FEATURES0 FeatureStruct1 > event TEST_EVENT_FEATURES1 None > boxed=False > feature deprecated > +event TEST_EVENT_FEATURES2 None > +boxed=False > +feature unstable > module include/sub-module.json > include sub-sub-module.json > object SecondArrayRef > -- > 2.31.1 > > Feels odd to combine the doc update *and* test prep, but eh, whatever. Reviewed-by: John Snow
Re: [PATCH v2 3/3] qapi: deprecate drive-backup
On 5/24/21 10:06 AM, Vladimir Sementsov-Ogievskiy wrote: 15.05.2021 01:38, John Snow wrote: On 5/6/21 5:57 AM, Kashyap Chamarthy wrote: TODO: We also need to deprecate drive-backup transaction action.. But union members in QAPI doesn't support 'deprecated' feature. I tried to dig a bit, but failed :/ Markus, could you please help with it? At least by advice? Oho, I see. OK, I'm not Markus, but I've been getting into lots of trouble in the QAPI generator lately, so let me see if I can help get you started... https://gitlab.com/jsnow/qemu/-/commits/hack-deprecate-union-branches/ Here's a quick hack that might expose that feature. I suppose we can discuss this with Markus and turn these into real patches if that's the direction we wanna head. Hi! Markus is silent.. Maybe, you'll send patches ? ) He just went on PTO for 2 weeks :') It's going to have to wait, I'm afraid ...
Re: [PATCH v2 3/3] qapi: deprecate drive-backup
On 5/6/21 5:57 AM, Kashyap Chamarthy wrote: TODO: We also need to deprecate drive-backup transaction action.. But union members in QAPI doesn't support 'deprecated' feature. I tried to dig a bit, but failed :/ Markus, could you please help with it? At least by advice? Oho, I see. OK, I'm not Markus, but I've been getting into lots of trouble in the QAPI generator lately, so let me see if I can help get you started... https://gitlab.com/jsnow/qemu/-/commits/hack-deprecate-union-branches/ Here's a quick hack that might expose that feature. I suppose we can discuss this with Markus and turn these into real patches if that's the direction we wanna head. --js
Re: [PATCH v2 2/3] docs/interop/bitmaps: use blockdev-backup
On 5/5/21 9:58 AM, Vladimir Sementsov-Ogievskiy wrote: We are going to deprecate drive-backup, so use modern interface here. In examples where target image creation is shown, show blockdev-add as well. If target creation omitted, omit blockdev-add as well. Signed-off-by: Vladimir Sementsov-Ogievskiy Seems good, thanks! (aside: I really need to push forward with the QMP cross-references work ...)
Re: [PATCH v2 1/3] docs/block-replication: use blockdev-backup
On 5/5/21 9:58 AM, Vladimir Sementsov-Ogievskiy wrote: We are going to deprecate drive-backup, so don't mention it here. Moreover, blockdev-backup seems more correct in the context. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow --- docs/block-replication.txt | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/block-replication.txt b/docs/block-replication.txt index 108e9166a8..59eb2b33b3 100644 --- a/docs/block-replication.txt +++ b/docs/block-replication.txt @@ -79,7 +79,7 @@ Primary | || Secondary disk <- hidden-disk 5 <- ||| | ||| | ||'-' - || drive-backup sync=none 6 + || blockdev-backup sync=none 6 1) The disk on the primary is represented by a block device with two children, providing replication between a primary disk and the host that @@ -101,7 +101,7 @@ should support bdrv_make_empty() and backing file. that is modified by the primary VM. It should also start as an empty disk, and the driver supports bdrv_make_empty() and backing file. -6) The drive-backup job (sync=none) is run to allow hidden-disk to buffer +6) The blockdev-backup job (sync=none) is run to allow hidden-disk to buffer any state that would otherwise be lost by the speculative write-through of the NBD server into the secondary disk. So before block replication, the primary disk and secondary disk should contain the same data.
Re: [PATCH] qapi: deprecate drive-backup
On 4/26/21 2:41 PM, Vladimir Sementsov-Ogievskiy wrote: 26.04.2021 21:30, John Snow wrote: On 4/26/21 2:05 PM, Daniel P. Berrangé wrote: On Mon, Apr 26, 2021 at 09:00:36PM +0300, Vladimir Sementsov-Ogievskiy wrote: 26.04.2021 20:34, John Snow wrote: On 4/23/21 8:59 AM, Vladimir Sementsov-Ogievskiy wrote: Modern way is using blockdev-add + blockdev-backup, which provides a lot more control on how target is opened. As example of drive-backup problems consider the following: User of drive-backup expects that target will be opened in the same cache and aio mode as source. Corresponding logic is in drive_backup_prepare(), where we take bs->open_flags of source. It works rather bad if source was added by blockdev-add. Assume source is qcow2 image. On blockdev-add we should specify aio and cache options for file child of qcow2 node. What happens next: drive_backup_prepare() looks at bs->open_flags of qcow2 source node. But there no BDRV_O_NOCAHE neither BDRV_O_NATIVE_AIO: BDRV_O_NOCAHE is places in bs->file->bs->open_flags, and BDRV_O_NATIVE_AIO is nowhere, as file-posix parse options and simply set s->use_linux_aio. No complaints from me, especially if Virtuozzo is on board. I would like to see some documentation changes alongside this deprecation, though. Signed-off-by: Vladimir Sementsov-Ogievskiy --- Hi all! I remember, I suggested to deprecate drive-backup some time ago, and nobody complain.. But that old patch was inside the series with other more questionable deprecations and it did not landed. Let's finally deprecate what should be deprecated long ago. We now faced a problem in our downstream, described in commit message. In downstream I've fixed it by simply enabling O_DIRECT and linux_aio unconditionally for drive_backup target. But actually this just shows that using drive-backup in blockdev era is a bad idea. So let's motivate everyone (including Virtuozzo of course) to move to new interfaces and avoid problems with all that outdated option inheritance.   docs/system/deprecated.rst | 5 +   qapi/block-core.json  | 5 -   2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/docs/system/deprecated.rst b/docs/system/deprecated.rst index 80cae86252..b6f5766e17 100644 --- a/docs/system/deprecated.rst +++ b/docs/system/deprecated.rst @@ -186,6 +186,11 @@ Use the more generic commands ``block-export-add`` and ``block-export-del``   instead. As part of this deprecation, where ``nbd-server-add`` used a   single ``bitmap``, the new ``block-export-add`` uses a list of ``bitmaps``. +``drive-backup`` (since 6.0) +'''''''''''''''''''''''''''' + +Use ``blockdev-backup`` in pair with ``blockdev-add`` instead. + 1) Let's add a sphinx reference to https://qemu-project.gitlab.io/qemu/interop/live-block-operations.html#live-disk-backup-drive-backup-and-blockdev-backup 2) Just a thought, not a request: We also may wish to update https://qemu-project.gitlab.io/qemu/interop/bitmaps.html to use the new, preferred method. However, this doc is a bit old and is in need of an overhaul anyway (Especially to add the NBD pull workflow.) Since the doc is in need of an overhaul anyway, can we ask Kashyap to help us here, if he has time? 3) Let's add a small explanation here that outlines the differences in using these two commands. Here's a suggestion: This change primarily separates the creation/opening process of the backup target with explicit, separate steps. BlockdevBackup uses mostly the same arguments as DriveBackup, except the "format" and "mode" options are removed in favor of using explicit "blockdev-create" and "blockdev-add" calls. (Here, I accidentally used the names of the argument objects instead of the names of the commands. It's likely better to spell out the names of the commands instead.) The "target" argument changes semantics. It no longer accepts filenames, and will now additionally accept arbitrary node names in addition to device names. 4) Also not a request: If we want to go above and beyond, it might be nice to spell out the exact steps required to transition from the old interface to the new one. Here's a (hasty) suggestion for how that might look: - The MODE argument is deprecated.   - "existing" is replaced by using "blockdev-add" commands.   - "absolute-paths" is replaced by using "blockdev-add" and    "blockdev-create" commands. - The FORMAT argument is deprecated.   - Format information is given to "blockdev-add"/"blockdev-create". - The TARGET argument has new semantics:   - Filenames are no longer supported, use blockdev-add/blockdev-create    as necessary
Re: [PATCH] qapi: deprecate drive-backup
On 4/26/21 2:05 PM, Daniel P. Berrangé wrote:
On Mon, Apr 26, 2021 at 09:00:36PM +0300, Vladimir Sementsov-Ogievskiy wrote:
26.04.2021 20:34, John Snow wrote:
On 4/23/21 8:59 AM, Vladimir Sementsov-Ogievskiy wrote:
Modern way is using blockdev-add + blockdev-backup, which provides a
lot more control on how target is opened.
As example of drive-backup problems consider the following:
User of drive-backup expects that target will be opened in the same
cache and aio mode as source. Corresponding logic is in
drive_backup_prepare(), where we take bs->open_flags of source.
It works rather bad if source was added by blockdev-add. Assume source
is qcow2 image. On blockdev-add we should specify aio and cache options
for file child of qcow2 node. What happens next:
drive_backup_prepare() looks at bs->open_flags of qcow2 source node.
But there no BDRV_O_NOCAHE neither BDRV_O_NATIVE_AIO: BDRV_O_NOCAHE is
places in bs->file->bs->open_flags, and BDRV_O_NATIVE_AIO is nowhere,
as file-posix parse options and simply set s->use_linux_aio.
No complaints from me, especially if Virtuozzo is on board. I would like to see
some documentation changes alongside this deprecation, though.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
Hi all! I remember, I suggested to deprecate drive-backup some time ago,
and nobody complain.. But that old patch was inside the series with
other more questionable deprecations and it did not landed.
Let's finally deprecate what should be deprecated long ago.
We now faced a problem in our downstream, described in commit message.
In downstream I've fixed it by simply enabling O_DIRECT and linux_aio
unconditionally for drive_backup target. But actually this just shows
that using drive-backup in blockdev era is a bad idea. So let's motivate
everyone (including Virtuozzo of course) to move to new interfaces and
avoid problems with all that outdated option inheritance.
 docs/system/deprecated.rst | 5 +
 qapi/block-core.json  | 5 -
 2 files changed, 9 insertions(+), 1 deletion(-)
diff --git a/docs/system/deprecated.rst b/docs/system/deprecated.rst
index 80cae86252..b6f5766e17 100644
--- a/docs/system/deprecated.rst
+++ b/docs/system/deprecated.rst
@@ -186,6 +186,11 @@ Use the more generic commands ``block-export-add`` and
``block-export-del``
 instead. As part of this deprecation, where ``nbd-server-add`` used a
 single ``bitmap``, the new ``block-export-add`` uses a list of ``bitmaps``.
+``drive-backup`` (since 6.0)
+''''''''''''''''''''''''''''
+
+Use ``blockdev-backup`` in pair with ``blockdev-add`` instead.
+
1) Let's add a sphinx reference to
https://qemu-project.gitlab.io/qemu/interop/live-block-operations.html#live-disk-backup-drive-backup-and-blockdev-backup
2) Just a thought, not a request: We also may wish to update
https://qemu-project.gitlab.io/qemu/interop/bitmaps.html to use the new,
preferred method. However, this doc is a bit old and is in need of an overhaul
anyway (Especially to add the NBD pull workflow.) Since the doc is in need of
an overhaul anyway, can we ask Kashyap to help us here, if he has time?
3) Let's add a small explanation here that outlines the differences in using
these two commands. Here's a suggestion:
This change primarily separates the creation/opening process of the backup target with explicit, separate steps.
BlockdevBackup uses mostly the same arguments as DriveBackup, except the "format" and "mode"
options are removed in favor of using explicit "blockdev-create" and "blockdev-add" calls.
The "target" argument changes semantics. It no longer accepts filenames, and
will now additionally accept arbitrary node names in addition to device names.
4) Also not a request: If we want to go above and beyond, it might be nice to
spell out the exact steps required to transition from the old interface to the
new one. Here's a (hasty) suggestion for how that might look:
- The MODE argument is deprecated.
 - "existing" is replaced by using "blockdev-add" commands.
 - "absolute-paths" is replaced by using "blockdev-add" and
  "blockdev-create" commands.
- The FORMAT argument is deprecated.
 - Format information is given to "blockdev-add"/"blockdev-create".
- The TARGET argument has new semantics:
 - Filenames are no longer supported, use blockdev-add/blockdev-create
  as necessary instead.
 - Device targets remain supported.
Example:
drive-backup $ARGS format=$FORMAT mode=$MODE target=$FILENAME becomes:
(taking some liberties with syntax to just illustrate the idea ...)
blockdev-create options={
  "driver": "file",
  "filename": $FILENAME
Re: [PATCH] qapi: deprecate drive-backup
On 4/23/21 8:59 AM, Vladimir Sementsov-Ogievskiy wrote:
Modern way is using blockdev-add + blockdev-backup, which provides a
lot more control on how target is opened.
As example of drive-backup problems consider the following:
User of drive-backup expects that target will be opened in the same
cache and aio mode as source. Corresponding logic is in
drive_backup_prepare(), where we take bs->open_flags of source.
It works rather bad if source was added by blockdev-add. Assume source
is qcow2 image. On blockdev-add we should specify aio and cache options
for file child of qcow2 node. What happens next:
drive_backup_prepare() looks at bs->open_flags of qcow2 source node.
But there no BDRV_O_NOCAHE neither BDRV_O_NATIVE_AIO: BDRV_O_NOCAHE is
places in bs->file->bs->open_flags, and BDRV_O_NATIVE_AIO is nowhere,
as file-posix parse options and simply set s->use_linux_aio.
No complaints from me, especially if Virtuozzo is on board. I would like
to see some documentation changes alongside this deprecation, though.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
Hi all! I remember, I suggested to deprecate drive-backup some time ago,
and nobody complain.. But that old patch was inside the series with
other more questionable deprecations and it did not landed.
Let's finally deprecate what should be deprecated long ago.
We now faced a problem in our downstream, described in commit message.
In downstream I've fixed it by simply enabling O_DIRECT and linux_aio
unconditionally for drive_backup target. But actually this just shows
that using drive-backup in blockdev era is a bad idea. So let's motivate
everyone (including Virtuozzo of course) to move to new interfaces and
avoid problems with all that outdated option inheritance.
docs/system/deprecated.rst | 5 +
qapi/block-core.json | 5 -
2 files changed, 9 insertions(+), 1 deletion(-)
diff --git a/docs/system/deprecated.rst b/docs/system/deprecated.rst
index 80cae86252..b6f5766e17 100644
--- a/docs/system/deprecated.rst
+++ b/docs/system/deprecated.rst
@@ -186,6 +186,11 @@ Use the more generic commands ``block-export-add`` and
``block-export-del``
instead. As part of this deprecation, where ``nbd-server-add`` used a
single ``bitmap``, the new ``block-export-add`` uses a list of ``bitmaps``.
+``drive-backup`` (since 6.0)
+
+
+Use ``blockdev-backup`` in pair with ``blockdev-add`` instead.
+
1) Let's add a sphinx reference to
https://qemu-project.gitlab.io/qemu/interop/live-block-operations.html#live-disk-backup-drive-backup-and-blockdev-backup
2) Just a thought, not a request: We also may wish to update
https://qemu-project.gitlab.io/qemu/interop/bitmaps.html to use the new,
preferred method. However, this doc is a bit old and is in need of an
overhaul anyway (Especially to add the NBD pull workflow.) Since the doc
is in need of an overhaul anyway, can we ask Kashyap to help us here, if
he has time?
3) Let's add a small explanation here that outlines the differences in
using these two commands. Here's a suggestion:
This change primarily separates the creation/opening process of the
backup target with explicit, separate steps. BlockdevBackup uses mostly
the same arguments as DriveBackup, except the "format" and "mode"
options are removed in favor of using explicit "blockdev-create" and
"blockdev-add" calls.
The "target" argument changes semantics. It no longer accepts filenames,
and will now additionally accept arbitrary node names in addition to
device names.
4) Also not a request: If we want to go above and beyond, it might be
nice to spell out the exact steps required to transition from the old
interface to the new one. Here's a (hasty) suggestion for how that might
look:
- The MODE argument is deprecated.
- "existing" is replaced by using "blockdev-add" commands.
- "absolute-paths" is replaced by using "blockdev-add" and
"blockdev-create" commands.
- The FORMAT argument is deprecated.
- Format information is given to "blockdev-add"/"blockdev-create".
- The TARGET argument has new semantics:
- Filenames are no longer supported, use blockdev-add/blockdev-create
as necessary instead.
- Device targets remain supported.
Example:
drive-backup $ARGS format=$FORMAT mode=$MODE target=$FILENAME becomes:
(taking some liberties with syntax to just illustrate the idea ...)
blockdev-create options={
"driver": "file",
"filename": $FILENAME,
"size": 0,
}
blockdev-add arguments={
"driver": "file",
"filename": $FILENAME,
"node-name": "Example_Filenode0"
}
blockdev-create options={
"driver": $FORMAT,
"file": "Example_Filenode0",
"size": $SIZE,
}
blockdev-add arguments={
"driver": $FORMAT,
"file": "Example_Filenode0",
"node-name": "Example_Formatnode0",
}
blockdev-backup arguments={
$ARGS ...,
"target": "Example_Formatnode0",
}
System accelerators
---
diff --git a/qapi/bloc
Re: [PATCH v4 0/4] Drop deprecated floppy config & bogus -drive if=T
On 3/11/21 2:52 AM, Markus Armbruster wrote: v4: * PATCH 3: Move a declaration into a loop [Richard] * PATCH 4: Drop a superfluous call to drive_check_orphaned() [Daniel], fix comments [John] v3: * PATCH 1: New [Daniel] v2: * Rebased, straightforward conflict with commit f5d33dd51f "hw/block/fdc: Remove the check_media_rate property" resolved * PATCH 2: Commit message fixed [Kevin] Markus Armbruster (4): docs/system/deprecated: Fix note on fdc drive properties fdc: Drop deprecated floppy configuration fdc: Inline fdctrl_connect_drives() into fdctrl_realize_common() blockdev: Drop deprecated bogus -drive interface type docs/system/deprecated.rst | 33 -- docs/system/removed-features.rst | 56 +++ include/sysemu/blockdev.h| 1 - blockdev.c | 37 +- hw/block/fdc.c | 73 +--- softmmu/vl.c | 11 +- tests/qemu-iotests/172 | 31 +- tests/qemu-iotests/172.out | 562 +-- 8 files changed, 82 insertions(+), 722 deletions(-) Reviewed-by: John Snow Tentatively staged, pending CI: https://gitlab.com/jsnow/qemu/-/pipelines/269277138 --js
Re: [PATCH v3 4/4] blockdev: Drop deprecated bogus -drive interface type
On 3/9/21 11:12 AM, Markus Armbruster wrote:
Drop the crap deprecated in commit a1b40bda08 "blockdev: Deprecate
-drive with bogus interface type" (v5.1.0).
Signed-off-by: Markus Armbruster
---
docs/system/deprecated.rst | 7 --
docs/system/removed-features.rst | 7 ++
include/sysemu/blockdev.h| 1 -
blockdev.c | 37 +---
softmmu/vl.c | 8 +--
5 files changed, 23 insertions(+), 37 deletions(-)
diff --git a/docs/system/deprecated.rst b/docs/system/deprecated.rst
index 601e9647a5..664ed60e9f 100644
--- a/docs/system/deprecated.rst
+++ b/docs/system/deprecated.rst
@@ -94,13 +94,6 @@ QEMU 5.1 has three options:
to the user to load all the images they need.
3. ``-bios `` - Tells QEMU to load the specified file as the firmwrae.
-``-drive`` with bogus interface type (since 5.1)
-
-
-Drives with interface types other than ``if=none`` are for onboard
-devices. It is possible to use drives the board doesn't pick up with
--device. This usage is now deprecated. Use ``if=none`` instead.
-
Short-form boolean options (since 6.0)
''
diff --git a/docs/system/removed-features.rst b/docs/system/removed-features.rst
index 77e7ba1339..e6d2fbe798 100644
--- a/docs/system/removed-features.rst
+++ b/docs/system/removed-features.rst
@@ -87,6 +87,13 @@ becomes
-device isa-fdc,...
-device floppy,unit=1,drive=...
+``-drive`` with bogus interface type (removed in 6.0)
+'
+
+Drives with interface types other than ``if=none`` are for onboard
+devices. Drives the board doesn't pick up can no longer be used with
+-device. Use ``if=none`` instead.
+
QEMU Machine Protocol (QMP) commands
diff --git a/include/sysemu/blockdev.h b/include/sysemu/blockdev.h
index 3b5fcda08d..32c2d6023c 100644
--- a/include/sysemu/blockdev.h
+++ b/include/sysemu/blockdev.h
@@ -35,7 +35,6 @@ struct DriveInfo {
bool is_default;/* Added by default_drive() ? */
int media_cd;
QemuOpts *opts;
-bool claimed_by_board;
QTAILQ_ENTRY(DriveInfo) next;
};
diff --git a/blockdev.c b/blockdev.c
index cd438e60e3..2e01889cff 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -240,19 +240,10 @@ DriveInfo *drive_get(BlockInterfaceType type, int bus,
int unit)
return NULL;
}
-void drive_mark_claimed_by_board(void)
-{
-BlockBackend *blk;
-DriveInfo *dinfo;
-
-for (blk = blk_next(NULL); blk; blk = blk_next(blk)) {
-dinfo = blk_legacy_dinfo(blk);
-if (dinfo && blk_get_attached_dev(blk)) {
-dinfo->claimed_by_board = true;
-}
-}
-}
-
+/*
+ * Check board claimed all -drive that are meant to be claimed.
+ * Fatal error if any remain unclaimed.
+ */
void drive_check_orphaned(void)
{
BlockBackend *blk;
@@ -262,7 +253,17 @@ void drive_check_orphaned(void)
for (blk = blk_next(NULL); blk; blk = blk_next(blk)) {
dinfo = blk_legacy_dinfo(blk);
-if (dinfo->is_default || dinfo->type == IF_NONE) {
+/*
+ * Ignore default drives, because we create certain default
+ * drives unconditionally, then leave them unclaimed. Not the
+ * users fault.
"user's" ?
+ * Ignore IF_VIRTIO, because it gets desugared into -device,
+ * so we can leave failing to -device.
+ * Ignore IF_NONE, because leaving unclaimed IF_NONE remains
+ * available for device_add is a feature.
Do you mean "as a feature" ?
+ */
+if (dinfo->is_default || dinfo->type == IF_VIRTIO
+|| dinfo->type == IF_NONE) {
continue;
}
if (!blk_get_attached_dev(blk)) {
@@ -273,14 +274,6 @@ void drive_check_orphaned(void)
if_name[dinfo->type], dinfo->bus, dinfo->unit);
loc_pop(&loc);
orphans = true;
-continue;
-}
-if (!dinfo->claimed_by_board && dinfo->type != IF_VIRTIO) {
-loc_push_none(&loc);
-qemu_opts_loc_restore(dinfo->opts);
-warn_report("bogus if=%s is deprecated, use if=none",
-if_name[dinfo->type]);
-loc_pop(&loc);
}
}
diff --git a/softmmu/vl.c b/softmmu/vl.c
index ff488ea3e7..7453611152 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -2460,13 +2460,7 @@ static void qemu_init_board(void)
/* From here on we enter MACHINE_PHASE_INITIALIZED. */
machine_run_board_init(current_machine);
-/*
- * TODO To drop support for deprecated bogus if=..., move
- * drive_check_orphaned() here, replacing this call. Also drop
- * its deprecation warning, along with DriveInfo member
- * @claimed_by_board.
- */
-drive_mark_claimed_by_board();
+drive
Re: [PATCH 2/4] hw/block/fdc: Remove the check_media_rate property
On 2/5/21 1:37 AM, Thomas Huth wrote:
On 05/02/2021 01.40, John Snow wrote:
On 2/3/21 12:18 PM, Thomas Huth wrote:
This was only required for the pc-1.0 and earlier machine types.
Now that these have been removed, we can also drop the corresponding
code from the FDC device.
Signed-off-by: Thomas Huth
---
 hw/block/fdc.c | 17 ++---
 tests/qemu-iotests/172.out | 35 ---
 2 files changed, 2 insertions(+), 50 deletions(-)
diff --git a/hw/block/fdc.c b/hw/block/fdc.c
index 292ea87805..198940e737 100644
--- a/hw/block/fdc.c
+++ b/hw/block/fdc.c
@@ -874,7 +874,6 @@ struct FDCtrl {
 FloppyDriveType type;
 } qdev_for_drives[MAX_FD];
 int reset_sensei;
-Â Â Â uint32_t check_media_rate;
I am a bit of a dunce when it comes to the compatibility properties...
does this mess with the migration format?
I guess it doesn't, since it's not in the VMSTATE declaration.
H, alright.
I think that should be fine, yes.
 FloppyDriveType fallback; /* type=auto failure fallback */
 /* Timers state */
 uint8_t timer0;
@@ -1021,18 +1020,10 @@ static const VMStateDescription
vmstate_fdrive_media_changed = {
 }
 };
-static bool fdrive_media_rate_needed(void *opaque)
-{
-Â Â Â FDrive *drive = opaque;
-
-Â Â Â return drive->fdctrl->check_media_rate;
-}
-
 static const VMStateDescription vmstate_fdrive_media_rate = {
 .name = "fdrive/media_rate",
 .version_id = 1,
 .minimum_version_id = 1,
-Â Â Â .needed = fdrive_media_rate_needed,
 .fields = (VMStateField[]) {
 VMSTATE_UINT8(media_rate, FDrive),
 VMSTATE_END_OF_LIST()
@@ -1689,8 +1680,7 @@ static void fdctrl_start_transfer(FDCtrl
*fdctrl, int direction)
 /* Check the data rate. If the programmed data rate does not match
  * the currently inserted medium, the operation has to fail. */
-Â Â Â if (fdctrl->check_media_rate &&
-Â Â Â (fdctrl->dsr & FD_DSR_DRATEMASK) != cur_drv->media_rate) {
+Â Â Â if ((fdctrl->dsr & FD_DSR_DRATEMASK) != cur_drv->media_rate) {
 FLOPPY_DPRINTF("data rate mismatch (fdc=%d, media=%d)\n",
fdctrl->dsr & FD_DSR_DRATEMASK,
cur_drv->media_rate);
 fdctrl_stop_transfer(fdctrl, FD_SR0_ABNTERM, FD_SR1_MA, 0x00);
@@ -2489,8 +2479,7 @@ static void fdctrl_result_timer(void *opaque)
 cur_drv->sect = (cur_drv->sect % cur_drv->last_sect) + 1;
 }
 /* READ_ID can't automatically succeed! */
-Â Â Â if (fdctrl->check_media_rate &&
-Â Â Â (fdctrl->dsr & FD_DSR_DRATEMASK) != cur_drv->media_rate) {
+Â Â Â if ((fdctrl->dsr & FD_DSR_DRATEMASK) != cur_drv->media_rate) {
 FLOPPY_DPRINTF("read id rate mismatch (fdc=%d, media=%d)\n",
fdctrl->dsr & FD_DSR_DRATEMASK,
cur_drv->media_rate);
 fdctrl_stop_transfer(fdctrl, FD_SR0_ABNTERM, FD_SR1_MA, 0x00);
@@ -2895,8 +2884,6 @@ static Property isa_fdc_properties[] = {
 DEFINE_PROP_UINT32("dma", FDCtrlISABus, dma, 2),
 DEFINE_PROP_DRIVE("driveA", FDCtrlISABus,
state.qdev_for_drives[0].blk),
 DEFINE_PROP_DRIVE("driveB", FDCtrlISABus,
state.qdev_for_drives[1].blk),
-Â Â Â DEFINE_PROP_BIT("check_media_rate", FDCtrlISABus,
state.check_media_rate,
-Â Â Â 0, true),
Could you theoretically set this via QOM commands in QMP, and claim
that this is a break in behavior?
Though, it's ENTIRELY undocumented, so ... it's probably fine, I
think. Probably. (Please soothe my troubled mind.)
A user actually could mess with this property even on the command line,
e.g. by using:
 qemu-system-x86_64 -global isa-fdc.check_media_rate=false
... but, as you said, it's completely undocumented, the property is
really just there for the internal use of machine type compatibility.
We've done such clean-ups in the past already, see e.g.
c6026998eef382d7ad76 or 2a4dbaf1c0db2453ab78f, so I think this should be
fine. But if you disagree, I could replace this by a patch that adds
this property to the list of deprecated features instead, so we could at
least remove it after it has been deprecated for two releases?
I don't think it's necessary, personally -- just wanted to make sure I
knew the exact stakes here.
Reviewed-by: John Snow
Acked-by: John Snow
Re: [PATCH 2/4] hw/block/fdc: Remove the check_media_rate property
On 2/3/21 12:18 PM, Thomas Huth wrote:
This was only required for the pc-1.0 and earlier machine types.
Now that these have been removed, we can also drop the corresponding
code from the FDC device.
Signed-off-by: Thomas Huth
---
hw/block/fdc.c | 17 ++---
tests/qemu-iotests/172.out | 35 ---
2 files changed, 2 insertions(+), 50 deletions(-)
diff --git a/hw/block/fdc.c b/hw/block/fdc.c
index 292ea87805..198940e737 100644
--- a/hw/block/fdc.c
+++ b/hw/block/fdc.c
@@ -874,7 +874,6 @@ struct FDCtrl {
FloppyDriveType type;
} qdev_for_drives[MAX_FD];
int reset_sensei;
-uint32_t check_media_rate;
I am a bit of a dunce when it comes to the compatibility properties...
does this mess with the migration format?
I guess it doesn't, since it's not in the VMSTATE declaration.
H, alright.
FloppyDriveType fallback; /* type=auto failure fallback */
/* Timers state */
uint8_t timer0;
@@ -1021,18 +1020,10 @@ static const VMStateDescription
vmstate_fdrive_media_changed = {
}
};
-static bool fdrive_media_rate_needed(void *opaque)
-{
-FDrive *drive = opaque;
-
-return drive->fdctrl->check_media_rate;
-}
-
static const VMStateDescription vmstate_fdrive_media_rate = {
.name = "fdrive/media_rate",
.version_id = 1,
.minimum_version_id = 1,
-.needed = fdrive_media_rate_needed,
.fields = (VMStateField[]) {
VMSTATE_UINT8(media_rate, FDrive),
VMSTATE_END_OF_LIST()
@@ -1689,8 +1680,7 @@ static void fdctrl_start_transfer(FDCtrl *fdctrl, int
direction)
/* Check the data rate. If the programmed data rate does not match
* the currently inserted medium, the operation has to fail. */
-if (fdctrl->check_media_rate &&
-(fdctrl->dsr & FD_DSR_DRATEMASK) != cur_drv->media_rate) {
+if ((fdctrl->dsr & FD_DSR_DRATEMASK) != cur_drv->media_rate) {
FLOPPY_DPRINTF("data rate mismatch (fdc=%d, media=%d)\n",
fdctrl->dsr & FD_DSR_DRATEMASK, cur_drv->media_rate);
fdctrl_stop_transfer(fdctrl, FD_SR0_ABNTERM, FD_SR1_MA, 0x00);
@@ -2489,8 +2479,7 @@ static void fdctrl_result_timer(void *opaque)
cur_drv->sect = (cur_drv->sect % cur_drv->last_sect) + 1;
}
/* READ_ID can't automatically succeed! */
-if (fdctrl->check_media_rate &&
-(fdctrl->dsr & FD_DSR_DRATEMASK) != cur_drv->media_rate) {
+if ((fdctrl->dsr & FD_DSR_DRATEMASK) != cur_drv->media_rate) {
FLOPPY_DPRINTF("read id rate mismatch (fdc=%d, media=%d)\n",
fdctrl->dsr & FD_DSR_DRATEMASK, cur_drv->media_rate);
fdctrl_stop_transfer(fdctrl, FD_SR0_ABNTERM, FD_SR1_MA, 0x00);
@@ -2895,8 +2884,6 @@ static Property isa_fdc_properties[] = {
DEFINE_PROP_UINT32("dma", FDCtrlISABus, dma, 2),
DEFINE_PROP_DRIVE("driveA", FDCtrlISABus, state.qdev_for_drives[0].blk),
DEFINE_PROP_DRIVE("driveB", FDCtrlISABus, state.qdev_for_drives[1].blk),
-DEFINE_PROP_BIT("check_media_rate", FDCtrlISABus, state.check_media_rate,
-0, true),
Could you theoretically set this via QOM commands in QMP, and claim that
this is a break in behavior?
Though, it's ENTIRELY undocumented, so ... it's probably fine, I think.
Probably. (Please soothe my troubled mind.)
--js
Re: [PATCH] docs/interop/qmp-spec: Document the request queue limit
On 1/27/21 9:47 AM, Markus Armbruster wrote: Signed-off-by: Markus Armbruster --- docs/interop/qmp-spec.txt | 8 +--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/docs/interop/qmp-spec.txt b/docs/interop/qmp-spec.txt index cdf5842555..b0e8351d5b 100644 --- a/docs/interop/qmp-spec.txt +++ b/docs/interop/qmp-spec.txt @@ -133,9 +133,11 @@ to pass "id" with out-of-band commands. Passing it with all commands is recommended for clients that accept capability "oob". If the client sends in-band commands faster than the server can -execute them, the server will stop reading the requests from the QMP -channel until the request queue length is reduced to an acceptable -range. +execute them, the server will stop reading requests until the request +queue length is reduced to an acceptable range. + +To ensure commands to be executed out-of-band get read and executed, +the client should have at most eight in-band commands in flight. Only a few commands support out-of-band execution. The ones that do have "allow-oob": true in output of query-qmp-schema. Great to know, thank you! Reviewed-by: John Snow
Re: qmp-shell TUI (was: Re: Call for Google Summer of Code 2021 project ideas)
On 1/14/21 8:52 AM, Stefan Hajnoczi wrote:
On Wed, Jan 13, 2021 at 01:59:43PM -0500, John Snow wrote:
On 1/13/21 3:53 AM, Stefan Hajnoczi wrote:
On Tue, Jan 12, 2021 at 9:10 PM John Snow wrote:
2. Ability to watch QMP activity on a running QEMU process, e.g. even
when libvirt is directly connected to the monitor.
That *WOULD* be extremely cool, and moves a lot closer to how mitmproxy
works.
(Actually, mitmproxy could theoretically be taught how to read and
understand QMP traffic, but that's not something I know how to do or would
be prepared to mentor.)
Is this possible to do in a post-hoc fashion? Let's say you are using
production environment QEMU, how do we attach the QMP listener to it? Or
does this idea require that we start QEMU in a specific fashion with a
second debug socket that qmp-shell can connect to in order to listen?
... Or do we engineer qmp-shell to open its own socket that libvirt connects
to ...?
Here is the QEMU command-line that libvirt uses on my F33 system:
-chardev socket,id=charmonitor,fd=36,server,nowait
-mon chardev=charmonitor,id=monitor,mode=control
Goals for this feature:
1. No manual steps required for setup.
2. Ability to start/stop monitoring traffic at runtime without
restarting QEMU.
3. Available to unprivileged users.
Excellent goals, and I agree completely.
I think the easiest way to achieve this is through a new QEMU monitor
command. Approaches that come to mind:
1. Add a -mon debug-chardev property and a QMP command to set it at
runtime. The debug-chardev receives both monitor input (commands) and
output (responses and events). This does not allow MITM, rather it
mirrors traffic.
So you have a socket that relays I/O. I wonder if it needs to modify the
stream format to some extent to annotate directionality?
For now, directionality can be inferred, but maybe that's brittle.
(greeting messages, events and return statements are from the server;
negotiation and execute statements are from the client.)
Maybe if we used a hypothetical qmp-shell log format, we could add
timestamps here instead of relying on the client to produce them. This
might be interesting for analyzing race conditions and measuring
response delays as experienced by the server.
{"message": original_json_message_here, "direction": "in", "timestamp":
1610627721}
(Downside: JSON is still not a streaming message format, but I guess
it's one we already use all over the place anyway.)
2. Add a chardev-get-fd command that fetches the fd from a chardev and
then use the existing chardev-change command to replace the monitor
chardev with a chardev connected to qmp-shell. This inserts qmp-shell
as a proxy between the QMP client and server. qmp-shell can remove
itself again with another chardev-change command. This approach
allows MITM. The downside is it assumes the QMP chardev is a file
descriptor, so it won't work with all types of chardev.
It seems a little more prone to failure if the insertion/removal fails,
and has some downsides about which configurations it can inject into.
3. Add a new chardev-proxy type that aggregates 3 chardevs: 1. an origin
source chardev, 2. a monitoring sink chardev, and 3. a monitoring
source chardev. The data flow is origin <-> monitoring sink <->
monitoring source <-> QMP monitor. qmp-shell creates the monitoring
sink (for receiving incoming QMP commands) and monitoring source
chardev (for forwarding QMP commands or MITM commands), and then it
uses change-chardev to instantiate a chardev-proxy that directs the
original libvirt chardev through the monitoring sink and source.
I'm not sure I understand the topology here, exactly. I could stand to
be a little more familiar with how chardevs are modeled in QEMU ...
This is the most complex but also completely contained within the
QEMU chardev layer.
In all these approaches qmp-shell uses virsh qemu-monitor-command or an
equivalent API to start/stop monitoring a running VM without manual
setup steps.
Gotcha. I think I am leaning towards the first suggestion, but maybe the
third one that I don't quite grasp yet is good too.
Stefan
Re: [PATCH] os: deprecate the -enable-fips option and QEMU's FIPS enforcement
On 10/21/20 6:17 AM, Daniel P. Berrangé wrote: Claiming QEMU is FIPS compliant without using libgcrypt is a bit of joke since we don't do any self-tests of ciphers, hence this deprecation notice is warning people that libgcrypt is going to be mandatory if you care about FIPS. FWIW this is my main problem with this flag: we read the value in procfs and then use this to change precisely one behavior for one of our components. It doesn't really ... do what the name might imply it does. Leaving that business to the crypto libraries is indeed the correct thing to do. So: Reviewed-by: John Snow
Re: [PATCH v2] Remove deprecated -no-kvm option
On 10/20/20 12:05 PM, Thomas Huth wrote:
The option has never been mentioned in our documentation, it's been
deprecated since years, it's marked with QEMU_ARCH_I386 (which does
not make sense anymore since KVM is available on other architectures,
too), it does not do anything by default in upstream QEMU (since TCG
is the default here anyway), and we're spending too much precious time
each year discussing whether it makes sense to keep this option as a
nice suger or not... let's finally put an end on this and remove it.
Signed-off-by: Thomas Huth
---
v2: Mention the parameter in the "removed feature" section (thanks Paolo!)
docs/system/deprecated.rst | 11 ++-
qemu-options.hx| 3 ---
softmmu/vl.c | 4
3 files changed, 6 insertions(+), 12 deletions(-)
diff --git a/docs/system/deprecated.rst b/docs/system/deprecated.rst
index 905628f3a0..9bae13bc01 100644
--- a/docs/system/deprecated.rst
+++ b/docs/system/deprecated.rst
@@ -27,11 +27,6 @@ System emulator command line arguments
The ``enforce-config-section`` parameter is replaced by the
``-global migration.send-configuration={on|off}`` option.
-``-no-kvm`` (since 1.3.0)
-'''''''''''''''''''''''''
-
-The ``-no-kvm`` argument is now a synonym for setting ``-accel tcg``.
-
``-usbdevice`` (since 2.10.0)
'''''''''''''''''''''''''''''
@@ -504,6 +499,12 @@ System emulator command line arguments
The ``name`` parameter of the ``-net`` option was a synonym
for the ``id`` parameter, which should now be used instead.
+``-no-kvm`` (removed in 5.2)
+''''''''''''''''''''''''''''
+
+The ``-no-kvm`` argument was a synonym for setting ``-accel tcg``.
+
+
QEMU Machine Protocol (QMP) commands
diff --git a/qemu-options.hx b/qemu-options.hx
index 1da52a269c..9e1ace04f7 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -4351,9 +4351,6 @@ SRST
Enable FIPS 140-2 compliance mode.
ERST
-HXCOMM Deprecated by -accel tcg
-DEF("no-kvm", 0, QEMU_OPTION_no_kvm, "", QEMU_ARCH_I386)
-
DEF("msg", HAS_ARG, QEMU_OPTION_msg,
"-msg [timestamp[=on|off]][,guest-name=[on|off]]\n"
"control error message format\n"
diff --git a/softmmu/vl.c b/softmmu/vl.c
index cb476aa70b..6f5b000f07 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -3502,10 +3502,6 @@ void qemu_init(int argc, char **argv, char **envp)
exit(1);
}
break;
- case QEMU_OPTION_no_kvm:
-olist = qemu_find_opts("machine");
-qemu_opts_parse_noisily(olist, "accel=tcg", false);
-break;
case QEMU_OPTION_accel:
accel_opts = qemu_opts_parse_noisily(qemu_find_opts("accel"),
optarg, true);
One down, 130 to go? *g*
Reviewed-by: John Snow
Re: [PATCH 1/1] configure: prefer python's sphinx module
On 6/18/20 5:56 AM, Peter Maydell wrote:
> On Tue, 16 Jun 2020 at 20:09, John Snow wrote:
>> Using an explicit entry path script for sphinx can lead to confusing
>> results: If the python binary belongs to a virtual environment, our
>> configure script may still select a sphinx script that belongs to the
>> system distribution packages.
>>
>> It is likely best to use python itself (whichever one the user provides)
>> to resolve the sphinx script.
>
> I'm not convinced. How do I find out which sphinx-build this
> is actually going to use ? ("python3 -m sphinx" doesn't list a
> path to anything.)
>
> python3 -c 'import sphinx; print(sphinx.__file__)'
/usr/lib/python3.7/site-packages/sphinx/__init__.py
> How do I use the system python but a venv sphinx-build? At the
> python3 -m venv myvenv
> cd myvenv/bin
> ls -l python*
lrwxrwxrwx. 1 jsnow jsnow 7 Jun 19 13:23 python -> python3*
lrwxrwxrwx. 1 jsnow jsnow 16 Jun 19 13:23 python3 -> /usr/bin/python3*
The venv uses symlinks, so it will continue to use your system version,
but you can install sphinx here.
I'm proposing you do either one of:
A) ./configure --python=/home/petmay01/python-env/bin/python3
B) source ~/python-env/bin/activate
./configure
> moment I can easily do that with
> --sphinx-build=/home/petmay01/python-env/bin/sphinx-build
> because scripts inside a venv have #! lines that make them
> work without having to manually activate the venv. I don't
> want to have to use some random non-system Python just
> because I have a newer Sphinx.
>
I was under the impression that it would be best if sphinx was executed
using the user's specified python binary instead of allowing
scripts/qapi to run under the user's python but sphinx to run under a
different python.
One of the reasons I came to this belief was to ensure that when
operating inside of a venv that QEMU was always using that venv's python
and sphinx instead of "leaking" out to the system's installation. It
felt more explicit.
A problem with looking for 'sphinx-build-3' and 'sphinx-build' entry
scripts is that the /usr/bin/xxx namespace is shared between python2 and
python3 packages and we may wind up selecting a sphinx for the wrong
python version entirely -- and from what I could tell, there wasn't a
way to interrogate sphinx to get it to tell us what python it was using,
or any other way to force this kind of scripted entrypoint to use *my*
python.
Fedora gets into trouble here because we want 'sphinx-build-3', but this
ignores our venv version because the script entrypoint in a venv is
'sphinx-build' -- which might be the system's python2 version.
Using `python -m sphinx` seemed nice because:
1. Works the same inside and outside of venv
2. Don't need to interrogate python version, we already know it
3. We are confident it uses the venv.
Or, put another way, I think it's quite odd to:
1. Activate a venv
2. Specify an explicit python version to ./configure
3. Have sphinx use a python/sphinx that is not necessarily correlated
with either #1 or #2.
> Put another way, I don't think the fact that sphinx-build
> happens to be implemented in Python means that we should
> let the user's decision about which Python they want us to
> use control which version of sphinx-build we should use.
>
I see your point: Why not treat sphinx-build as a black box executable
instead of treating it like a python plugin?
Mostly,
(1) The aforementioned problems locating and interrogating the correct
script entrypoint to use based on the user's environment and configure
options
(2) By treating it as a Python dependency instead, I can ensure that we
have a correctly modern sphinx in a venv as a repeatable build step for
platforms that do not ship a modern-enough sphinx.
--js
[PATCH 1/1] configure: prefer python's sphinx module
Using an explicit entry path script for sphinx can lead to confusing
results: If the python binary belongs to a virtual environment, our
configure script may still select a sphinx script that belongs to the
system distribution packages.
It is likely best to use python itself (whichever one the user provides)
to resolve the sphinx script. This has a few benefits:
1. It will always be Python 3. Script entry points like 'sphinx-build'
might attempt to invoke python2 -- or indeed, any version of python that
the user did not specify with --python.
2. When using a user installation of sphinx, it will naturally prefer
that version.
3. When using a virtual environment, it will naturally prefer that
version.
4. Removing explicit python script entry points gives us greater control
within the QEMU build ecosystem; we can be confident we are respecting
the user's --python configuration, if any.
It's my hope that this will make it simpler to integrate some optional
pipenv functionality into the build for adding regression testing to our
python module -- by offloading more of the python environment
configuration and testing directly to the python tooling ecosystem.
This would, ideally, free us up to use more modern python packages not
always offered in older system distributions. (For instance, we already
violate this requirement with sphinx, which does not have the necessary
versions in our oldest build platforms.)
Signed-off-by: John Snow
---
docs/system/deprecated.rst | 19 +++
configure | 20 +---
2 files changed, 28 insertions(+), 11 deletions(-)
diff --git a/docs/system/deprecated.rst b/docs/system/deprecated.rst
index 544ece0a45e..a762a8c8043 100644
--- a/docs/system/deprecated.rst
+++ b/docs/system/deprecated.rst
@@ -17,6 +17,25 @@ they were first deprecated in the 2.10.0 release.
What follows is a list of all features currently marked as
deprecated.
+Configure script options
+
+
+``--sphinx-build=`` (since 5.1)
+'''''''''''''''''''''''''''''''
+
+The ``--sphinx-build`` option to select a specific sphinx-build entry
+point is replaced by configuring Python accordingly. QEMU will now
+default to using ``$python -m sphinx`` to use Python's preferred version
+of sphinx.
+
+Python will generally default to preferring user packages installed with
+``pip install --user`` first, and system distribution packages
+second. By specifying a custom Python binary or activating a virtual
+environment, Python will alter its module search behavior. As a last
+resort, the PYTHONPATH environment variable can be modified to specify
+an explicit directory.
+
+
System emulator command line arguments
--
diff --git a/configure b/configure
index b01b5e3bed0..e04b9d836ee 100755
--- a/configure
+++ b/configure
@@ -932,16 +932,6 @@ do
fi
done
-sphinx_build=
-for binary in sphinx-build-3 sphinx-build
-do
-if has "$binary"
-then
-sphinx_build=$(command -v "$binary")
-break
-fi
-done
-
# Check for ancillary tools used in testing
genisoimage=
for binary in genisoimage mkisofs
@@ -1014,7 +1004,9 @@ for opt do
;;
--python=*) python="$optarg"
;;
- --sphinx-build=*) sphinx_build="$optarg"
+ --sphinx-build=*)
+ sphinx_build="$optarg"
+ echo "--sphinx-build is deprecated. QEMU uses the specified python's
sphinx module."
;;
--gcov=*) gcov_tool="$optarg"
;;
@@ -4996,6 +4988,12 @@ has_sphinx_build() {
"$TMPDIR1/sphinx/out" >> config.log 2>&1
}
+# While we support --sphinx-build, defer to that.
+# Otherwise, use the module execution feature of Python.
+if ! has $sphinx_build; then
+sphinx_build="$python -m sphinx"
+fi
+
# Check if tools are available to build documentation.
if test "$docs" != "no" ; then
if has_sphinx_build; then
--
2.21.3
[PATCH 0/1] configure: prefer python's sphinx module
Hi, this is a followup to my patch which attempted to prefer 'sphinx-build' to 'sphinx-build-3' which intended to prefer a VENV installation of sphinx over Fedora's system installation. That got confusing, though, so here's something that's simpler and better: use python to invoke sphinx so we don't have to worry about python/sphinx version mismatches. This should essentially always DTRT. In cases where a specific sphinx is preferred, you would use a virtual environment with sphinx installed and configure QEMU with e.g. --python /path/to/my/venv/python35 John Snow (1): configure: prefer python's sphinx module docs/system/deprecated.rst | 19 +++ configure | 20 +--- 2 files changed, 28 insertions(+), 11 deletions(-) -- 2.21.3
Re: [PULL v2 00/11] Bitmaps patches
On 3/19/20 1:57 PM, Peter Maydell wrote: > On Wed, 18 Mar 2020 at 20:24, John Snow wrote: >> >> The following changes since commit d649689a8ecb2e276cc20d3af6d416e3c299cb17: >> >> Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into >> staging (2020-03-17 18:33:05 +) >> >> are available in the Git repository at: >> >> https://github.com/jnsnow/qemu.git tags/bitmaps-pull-request >> >> for you to fetch changes up to 2d00cbd8e222a4adc08f415c399e84590ee8ff9a: >> >> block/qcow2-bitmap: use bdrv_dirty_bitmap_next_dirty (2020-03-18 14:03:46 >> -0400) >> >> >> Pull request >> >> > > > Applied, thanks. > Wonderful, thanks! > Please update the changelog at https://wiki.qemu.org/ChangeLog/5.0 > for any user-visible changes. > > -- PMM Will do.
[PULL v2 10/11] nbd/server: use bdrv_dirty_bitmap_next_dirty_area
From: Vladimir Sementsov-Ogievskiy
Use bdrv_dirty_bitmap_next_dirty_area for bitmap_to_extents. Since
bdrv_dirty_bitmap_next_dirty_area is very accurate in its interface,
we'll never exceed requested region with last chunk. So, we don't need
dont_fragment, and bitmap_to_extents() interface becomes clean enough
to not require any comment.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Eric Blake
Message-id: 20200205112041.6003-10-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
nbd/server.c | 59 +---
1 file changed, 19 insertions(+), 40 deletions(-)
diff --git a/nbd/server.c b/nbd/server.c
index f90bb33a75..02b1ed0801 100644
--- a/nbd/server.c
+++ b/nbd/server.c
@@ -2068,57 +2068,36 @@ static int nbd_co_send_block_status(NBDClient *client,
uint64_t handle,
return nbd_co_send_extents(client, handle, ea, last, context_id, errp);
}
-/*
- * Populate @ea from a dirty bitmap. Unless @dont_fragment, the
- * final extent may exceed the original @length.
- */
+/* Populate @ea from a dirty bitmap. */
static void bitmap_to_extents(BdrvDirtyBitmap *bitmap,
uint64_t offset, uint64_t length,
- NBDExtentArray *ea, bool dont_fragment)
+ NBDExtentArray *es)
{
-uint64_t begin = offset, end = offset;
-uint64_t overall_end = offset + length;
-BdrvDirtyBitmapIter *it;
-bool dirty;
+int64_t start, dirty_start, dirty_count;
+int64_t end = offset + length;
+bool full = false;
bdrv_dirty_bitmap_lock(bitmap);
-it = bdrv_dirty_iter_new(bitmap);
-dirty = bdrv_dirty_bitmap_get_locked(bitmap, offset);
-
-while (begin < overall_end) {
-bool next_dirty = !dirty;
-
-if (dirty) {
-end = bdrv_dirty_bitmap_next_zero(bitmap, begin, INT64_MAX);
-} else {
-bdrv_set_dirty_iter(it, begin);
-end = bdrv_dirty_iter_next(it);
-}
-if (end == -1 || end - begin > UINT32_MAX) {
-/* Cap to an aligned value < 4G beyond begin. */
-end = MIN(bdrv_dirty_bitmap_size(bitmap),
- begin + UINT32_MAX + 1 -
- bdrv_dirty_bitmap_granularity(bitmap));
-next_dirty = dirty;
-}
-if (dont_fragment && end > overall_end) {
-end = overall_end;
-}
-
-if (nbd_extent_array_add(ea, end - begin,
- dirty ? NBD_STATE_DIRTY : 0) < 0) {
+for (start = offset;
+ bdrv_dirty_bitmap_next_dirty_area(bitmap, start, end, INT32_MAX,
+ &dirty_start, &dirty_count);
+ start = dirty_start + dirty_count)
+{
+if ((nbd_extent_array_add(es, dirty_start - start, 0) < 0) ||
+(nbd_extent_array_add(es, dirty_count, NBD_STATE_DIRTY) < 0))
+{
+full = true;
break;
}
-begin = end;
-dirty = next_dirty;
}
-bdrv_dirty_iter_free(it);
+if (!full) {
+/* last non dirty extent */
+nbd_extent_array_add(es, end - start, 0);
+}
bdrv_dirty_bitmap_unlock(bitmap);
-
-assert(offset < end);
}
static int nbd_co_send_bitmap(NBDClient *client, uint64_t handle,
@@ -2129,7 +2108,7 @@ static int nbd_co_send_bitmap(NBDClient *client, uint64_t
handle,
unsigned int nb_extents = dont_fragment ? 1 : NBD_MAX_BLOCK_STATUS_EXTENTS;
g_autoptr(NBDExtentArray) ea = nbd_extent_array_new(nb_extents);
-bitmap_to_extents(bitmap, offset, length, ea, dont_fragment);
+bitmap_to_extents(bitmap, offset, length, ea);
return nbd_co_send_extents(client, handle, ea, last, context_id, errp);
}
--
2.21.1
[PULL v2 11/11] block/qcow2-bitmap: use bdrv_dirty_bitmap_next_dirty
From: Vladimir Sementsov-Ogievskiy
store_bitmap_data() loop does bdrv_set_dirty_iter() on each iteration,
which means that we actually don't need iterator itself and we can use
simpler bitmap API.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
Reviewed-by: John Snow
Message-id: 20200205112041.6003-11-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
block/qcow2-bitmap.c | 15 +--
1 file changed, 5 insertions(+), 10 deletions(-)
diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
index 82c9f3..cb06954b4a 100644
--- a/block/qcow2-bitmap.c
+++ b/block/qcow2-bitmap.c
@@ -1288,7 +1288,6 @@ static uint64_t *store_bitmap_data(BlockDriverState *bs,
uint64_t bm_size = bdrv_dirty_bitmap_size(bitmap);
const char *bm_name = bdrv_dirty_bitmap_name(bitmap);
uint8_t *buf = NULL;
-BdrvDirtyBitmapIter *dbi;
uint64_t *tb;
uint64_t tb_size =
size_to_clusters(s,
@@ -1307,12 +1306,14 @@ static uint64_t *store_bitmap_data(BlockDriverState *bs,
return NULL;
}
-dbi = bdrv_dirty_iter_new(bitmap);
buf = g_malloc(s->cluster_size);
limit = bytes_covered_by_bitmap_cluster(s, bitmap);
assert(DIV_ROUND_UP(bm_size, limit) == tb_size);
-while ((offset = bdrv_dirty_iter_next(dbi)) >= 0) {
+offset = 0;
+while ((offset = bdrv_dirty_bitmap_next_dirty(bitmap, offset, INT64_MAX))
+ >= 0)
+{
uint64_t cluster = offset / limit;
uint64_t end, write_size;
int64_t off;
@@ -1355,23 +1356,17 @@ static uint64_t *store_bitmap_data(BlockDriverState *bs,
goto fail;
}
-if (end >= bm_size) {
-break;
-}
-
-bdrv_set_dirty_iter(dbi, end);
+offset = end;
}
*bitmap_table_size = tb_size;
g_free(buf);
-bdrv_dirty_iter_free(dbi);
return tb;
fail:
clear_bitmap_table(bs, tb, tb_size);
g_free(buf);
-bdrv_dirty_iter_free(dbi);
g_free(tb);
return NULL;
--
2.21.1
[PULL v2 04/11] hbitmap: unpublish hbitmap_iter_skip_words
From: Vladimir Sementsov-Ogievskiy
Function is internal and even commented as internal. Drop its
definition from .h file.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
Reviewed-by: John Snow
Message-id: 20200205112041.6003-4-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
include/qemu/hbitmap.h | 7 ---
util/hbitmap.c | 2 +-
2 files changed, 1 insertion(+), 8 deletions(-)
diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
index ab227b117f..15837a0e2d 100644
--- a/include/qemu/hbitmap.h
+++ b/include/qemu/hbitmap.h
@@ -297,13 +297,6 @@ void hbitmap_free(HBitmap *hb);
*/
void hbitmap_iter_init(HBitmapIter *hbi, const HBitmap *hb, uint64_t first);
-/* hbitmap_iter_skip_words:
- * @hbi: HBitmapIter to operate on.
- *
- * Internal function used by hbitmap_iter_next and hbitmap_iter_next_word.
- */
-unsigned long hbitmap_iter_skip_words(HBitmapIter *hbi);
-
/* hbitmap_next_zero:
*
* Find next not dirty bit within selected range. If not found, return -1.
diff --git a/util/hbitmap.c b/util/hbitmap.c
index a368dc5ef7..26145d4b9e 100644
--- a/util/hbitmap.c
+++ b/util/hbitmap.c
@@ -104,7 +104,7 @@ struct HBitmap {
/* Advance hbi to the next nonzero word and return it. hbi->pos
* is updated. Returns zero if we reach the end of the bitmap.
*/
-unsigned long hbitmap_iter_skip_words(HBitmapIter *hbi)
+static unsigned long hbitmap_iter_skip_words(HBitmapIter *hbi)
{
size_t pos = hbi->pos;
const HBitmap *hb = hbi->hb;
--
2.21.1
[PULL v2 05/11] hbitmap: drop meta bitmaps as they are unused
From: Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
Reviewed-by: John Snow
Message-id: 20200205112041.6003-5-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
include/qemu/hbitmap.h | 21
tests/test-hbitmap.c | 115 -
util/hbitmap.c | 16 --
3 files changed, 152 deletions(-)
diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
index 15837a0e2d..df922d8517 100644
--- a/include/qemu/hbitmap.h
+++ b/include/qemu/hbitmap.h
@@ -325,27 +325,6 @@ int64_t hbitmap_next_zero(const HBitmap *hb, uint64_t
start, uint64_t count);
bool hbitmap_next_dirty_area(const HBitmap *hb, uint64_t *start,
uint64_t *count);
-/* hbitmap_create_meta:
- * Create a "meta" hbitmap to track dirtiness of the bits in this HBitmap.
- * The caller owns the created bitmap and must call hbitmap_free_meta(hb) to
- * free it.
- *
- * Currently, we only guarantee that if a bit in the hbitmap is changed it
- * will be reflected in the meta bitmap, but we do not yet guarantee the
- * opposite.
- *
- * @hb: The HBitmap to operate on.
- * @chunk_size: How many bits in @hb does one bit in the meta track.
- */
-HBitmap *hbitmap_create_meta(HBitmap *hb, int chunk_size);
-
-/* hbitmap_free_meta:
- * Free the meta bitmap of @hb.
- *
- * @hb: The HBitmap whose meta bitmap should be freed.
- */
-void hbitmap_free_meta(HBitmap *hb);
-
/**
* hbitmap_iter_next:
* @hbi: HBitmapIter to operate on.
diff --git a/tests/test-hbitmap.c b/tests/test-hbitmap.c
index e1f867085f..aeaa0b3f22 100644
--- a/tests/test-hbitmap.c
+++ b/tests/test-hbitmap.c
@@ -22,7 +22,6 @@
typedef struct TestHBitmapData {
HBitmap *hb;
-HBitmap *meta;
unsigned long *bits;
size_t size;
size_t old_size;
@@ -94,14 +93,6 @@ static void hbitmap_test_init(TestHBitmapData *data,
}
}
-static void hbitmap_test_init_meta(TestHBitmapData *data,
- uint64_t size, int granularity,
- int meta_chunk)
-{
-hbitmap_test_init(data, size, granularity);
-data->meta = hbitmap_create_meta(data->hb, meta_chunk);
-}
-
static inline size_t hbitmap_test_array_size(size_t bits)
{
size_t n = DIV_ROUND_UP(bits, BITS_PER_LONG);
@@ -144,9 +135,6 @@ static void hbitmap_test_teardown(TestHBitmapData *data,
const void *unused)
{
if (data->hb) {
-if (data->meta) {
-hbitmap_free_meta(data->hb);
-}
hbitmap_free(data->hb);
data->hb = NULL;
}
@@ -648,96 +636,6 @@ static void
test_hbitmap_truncate_shrink_large(TestHBitmapData *data,
hbitmap_test_truncate(data, size, -diff, 0);
}
-static void hbitmap_check_meta(TestHBitmapData *data,
- int64_t start, int count)
-{
-int64_t i;
-
-for (i = 0; i < data->size; i++) {
-if (i >= start && i < start + count) {
-g_assert(hbitmap_get(data->meta, i));
-} else {
-g_assert(!hbitmap_get(data->meta, i));
-}
-}
-}
-
-static void hbitmap_test_meta(TestHBitmapData *data,
- int64_t start, int count,
- int64_t check_start, int check_count)
-{
-hbitmap_reset_all(data->hb);
-hbitmap_reset_all(data->meta);
-
-/* Test "unset" -> "unset" will not update meta. */
-hbitmap_reset(data->hb, start, count);
-hbitmap_check_meta(data, 0, 0);
-
-/* Test "unset" -> "set" will update meta */
-hbitmap_set(data->hb, start, count);
-hbitmap_check_meta(data, check_start, check_count);
-
-/* Test "set" -> "set" will not update meta */
-hbitmap_reset_all(data->meta);
-hbitmap_set(data->hb, start, count);
-hbitmap_check_meta(data, 0, 0);
-
-/* Test "set" -> "unset" will update meta */
-hbitmap_reset_all(data->meta);
-hbitmap_reset(data->hb, start, count);
-hbitmap_check_meta(data, check_start, check_count);
-}
-
-static void hbitmap_test_meta_do(TestHBitmapData *data, int chunk_size)
-{
-uint64_t size = chunk_size * 100;
-hbitmap_test_init_meta(data, size, 0, chunk_size);
-
-hbitmap_test_meta(data, 0, 1, 0, chunk_size);
-hbitmap_test_meta(data, 0, chunk_size, 0, chunk_size);
-hbitmap_test_meta(data, chunk_size - 1, 1, 0, chunk_size);
-hbitmap_test_meta(data, chunk_size - 1, 2, 0, chunk_size * 2);
-hbitmap_test_meta(data, chunk_size - 1, chunk_size + 1, 0, chunk_size * 2);
-hbitmap_test_meta(data, chunk_size - 1, chunk_size + 2, 0, chunk_size * 3);
-hbitmap_test_meta(data, 7 * chunk_size - 1, chunk_size + 2,
- 6 * chunk_size, chunk_size * 3);
-hbitmap_test_meta(data, si
[PULL v2 01/11] build: Silence clang warning on older glib autoptr usage
From: Eric Blake
glib's G_DEFINE_AUTOPTR_CLEANUP_FUNC() macro defines several static
inline functions, often with some of them unused, but prior to 2.57.2
did not mark the functions as such. As a result, clang (but not gcc)
fails to build with older glib unless -Wno-unused-function is enabled.
Reported-by: Peter Maydell
Signed-off-by: Eric Blake
Reviewed-by: John Snow
Message-id: 20200317175534.196295-1-ebl...@redhat.com
Signed-off-by: John Snow
---
configure | 20
1 file changed, 20 insertions(+)
diff --git a/configure b/configure
index 06fcd070fb..479336bf6e 100755
--- a/configure
+++ b/configure
@@ -3851,6 +3851,26 @@ if ! compile_prog "$glib_cflags -Werror" "$glib_libs" ;
then
fi
fi
+# Silence clang warnings triggered by glib < 2.57.2
+cat > $TMPC << EOF
+#include
+typedef struct Foo {
+int i;
+} Foo;
+static void foo_free(Foo *f)
+{
+g_free(f);
+}
+G_DEFINE_AUTOPTR_CLEANUP_FUNC(Foo, foo_free);
+int main(void) { return 0; }
+EOF
+if ! compile_prog "$glib_cflags -Werror" "$glib_libs" ; then
+if cc_has_warning_flag "-Wno-unused-function"; then
+glib_cflags="$glib_cflags -Wno-unused-function"
+CFLAGS="$CFLAGS -Wno-unused-function"
+fi
+fi
+
#
# zlib check
--
2.21.1
[PULL v2 08/11] block/dirty-bitmap: improve _next_dirty_area API
From: Vladimir Sementsov-Ogievskiy
Firstly, _next_dirty_area is for scenarios when we may contiguously
search for next dirty area inside some limited region, so it is more
comfortable to specify "end" which should not be recalculated on each
iteration.
Secondly, let's add a possibility to limit resulting area size, not
limiting searching area. This will be used in NBD code in further
commit. (Note that now bdrv_dirty_bitmap_next_dirty_area is unused)
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
Reviewed-by: John Snow
Message-id: 20200205112041.6003-8-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
include/block/dirty-bitmap.h | 3 ++-
include/qemu/hbitmap.h | 23 ++
block/dirty-bitmap.c | 6 +++--
tests/test-hbitmap.c | 45 +++-
util/hbitmap.c | 44 +--
5 files changed, 75 insertions(+), 46 deletions(-)
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index b1f0de12db..8a10029418 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -110,7 +110,8 @@ int64_t bdrv_dirty_bitmap_next_dirty(BdrvDirtyBitmap
*bitmap, int64_t offset,
int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap *bitmap, int64_t offset,
int64_t bytes);
bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap *bitmap,
- int64_t *offset, int64_t *bytes);
+int64_t start, int64_t end, int64_t max_dirty_count,
+int64_t *dirty_start, int64_t *dirty_count);
BdrvDirtyBitmap *bdrv_reclaim_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap,
Error **errp);
diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
index 6e9ae51ed3..5e71b6d6f7 100644
--- a/include/qemu/hbitmap.h
+++ b/include/qemu/hbitmap.h
@@ -324,18 +324,21 @@ int64_t hbitmap_next_zero(const HBitmap *hb, int64_t
start, int64_t count);
/* hbitmap_next_dirty_area:
* @hb: The HBitmap to operate on
- * @start: in-out parameter.
- * in: the offset to start from
- * out: (if area found) start of found area
- * @count: in-out parameter.
- * in: length of requested region
- * out: length of found area
+ * @start: the offset to start from
+ * @end: end of requested area
+ * @max_dirty_count: limit for out parameter dirty_count
+ * @dirty_start: on success: start of found area
+ * @dirty_count: on success: length of found area
*
- * If dirty area found within [@start, @start + @count), returns true and sets
- * @offset and @bytes appropriately. Otherwise returns false and leaves @offset
- * and @bytes unchanged.
+ * If dirty area found within [@start, @end), returns true and sets
+ * @dirty_start and @dirty_count appropriately. @dirty_count will not exceed
+ * @max_dirty_count.
+ * If dirty area was not found, returns false and leaves @dirty_start and
+ * @dirty_count unchanged.
*/
-bool hbitmap_next_dirty_area(const HBitmap *hb, int64_t *start, int64_t
*count);
+bool hbitmap_next_dirty_area(const HBitmap *hb, int64_t start, int64_t end,
+ int64_t max_dirty_count,
+ int64_t *dirty_start, int64_t *dirty_count);
/**
* hbitmap_iter_next:
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 1b14c8eb26..063793e316 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -873,9 +873,11 @@ int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap
*bitmap, int64_t offset,
}
bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap *bitmap,
- int64_t *offset, int64_t *bytes)
+int64_t start, int64_t end, int64_t max_dirty_count,
+int64_t *dirty_start, int64_t *dirty_count)
{
-return hbitmap_next_dirty_area(bitmap->bitmap, offset, bytes);
+return hbitmap_next_dirty_area(bitmap->bitmap, start, end, max_dirty_count,
+ dirty_start, dirty_count);
}
/**
diff --git a/tests/test-hbitmap.c b/tests/test-hbitmap.c
index 8905b8a351..b6726cf76b 100644
--- a/tests/test-hbitmap.c
+++ b/tests/test-hbitmap.c
@@ -920,18 +920,19 @@ static void
test_hbitmap_next_x_after_truncate(TestHBitmapData *data,
test_hbitmap_next_x_check(data, 0);
}
-static void test_hbitmap_next_dirty_area_check(TestHBitmapData *data,
- int64_t offset,
- int64_t count)
+static void test_hbitmap_next_dirty_area_check_limited(TestHBitmapData *data,
+ int64_t offset,
+ int64_t count,
+ int64_t max_dirty)
{
int64_t off1, off2;
int64_t len1 = 0, len2;
bool ret1, ret2;
int64_t end;
-off1 = offs
[PULL v2 02/11] hbitmap: assert that we don't create bitmap larger than INT64_MAX
From: Vladimir Sementsov-Ogievskiy We have APIs which returns signed int64_t, to be able to return error. Therefore we can't handle bitmaps with absolute size larger than (INT64_MAX+1). Still, keep maximum to be INT64_MAX which is a bit safer. Note, that bitmaps are used to represent disk images, which can't exceed INT64_MAX anyway. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: John Snow Message-id: 20200205112041.6003-2-vsement...@virtuozzo.com Signed-off-by: John Snow --- util/hbitmap.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/util/hbitmap.c b/util/hbitmap.c index 242c6e519c..7f9b3e0cd7 100644 --- a/util/hbitmap.c +++ b/util/hbitmap.c @@ -716,6 +716,7 @@ HBitmap *hbitmap_alloc(uint64_t size, int granularity) HBitmap *hb = g_new0(struct HBitmap, 1); unsigned i; +assert(size <= INT64_MAX); hb->orig_size = size; assert(granularity >= 0 && granularity < 64); @@ -746,6 +747,7 @@ void hbitmap_truncate(HBitmap *hb, uint64_t size) uint64_t num_elements = size; uint64_t old; +assert(size <= INT64_MAX); hb->orig_size = size; /* Size comes in as logical elements, adjust for granularity. */ -- 2.21.1
[PULL v2 03/11] hbitmap: move hbitmap_iter_next_word to hbitmap.c
From: Vladimir Sementsov-Ogievskiy
The function is definitely internal (it's not used by third party and
it has complicated interface). Move it to .c file.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
Reviewed-by: John Snow
Message-id: 20200205112041.6003-3-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
include/qemu/hbitmap.h | 30 --
util/hbitmap.c | 29 +
2 files changed, 29 insertions(+), 30 deletions(-)
diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
index 1bf944ca3d..ab227b117f 100644
--- a/include/qemu/hbitmap.h
+++ b/include/qemu/hbitmap.h
@@ -362,34 +362,4 @@ void hbitmap_free_meta(HBitmap *hb);
*/
int64_t hbitmap_iter_next(HBitmapIter *hbi);
-/**
- * hbitmap_iter_next_word:
- * @hbi: HBitmapIter to operate on.
- * @p_cur: Location where to store the next non-zero word.
- *
- * Return the index of the next nonzero word that is set in @hbi's
- * associated HBitmap, and set *p_cur to the content of that word
- * (bits before the index that was passed to hbitmap_iter_init are
- * trimmed on the first call). Return -1, and set *p_cur to zero,
- * if all remaining words are zero.
- */
-static inline size_t hbitmap_iter_next_word(HBitmapIter *hbi, unsigned long
*p_cur)
-{
-unsigned long cur = hbi->cur[HBITMAP_LEVELS - 1];
-
-if (cur == 0) {
-cur = hbitmap_iter_skip_words(hbi);
-if (cur == 0) {
-*p_cur = 0;
-return -1;
-}
-}
-
-/* The next call will resume work from the next word. */
-hbi->cur[HBITMAP_LEVELS - 1] = 0;
-*p_cur = cur;
-return hbi->pos;
-}
-
-
#endif
diff --git a/util/hbitmap.c b/util/hbitmap.c
index 7f9b3e0cd7..a368dc5ef7 100644
--- a/util/hbitmap.c
+++ b/util/hbitmap.c
@@ -298,6 +298,35 @@ uint64_t hbitmap_count(const HBitmap *hb)
return hb->count << hb->granularity;
}
+/**
+ * hbitmap_iter_next_word:
+ * @hbi: HBitmapIter to operate on.
+ * @p_cur: Location where to store the next non-zero word.
+ *
+ * Return the index of the next nonzero word that is set in @hbi's
+ * associated HBitmap, and set *p_cur to the content of that word
+ * (bits before the index that was passed to hbitmap_iter_init are
+ * trimmed on the first call). Return -1, and set *p_cur to zero,
+ * if all remaining words are zero.
+ */
+static size_t hbitmap_iter_next_word(HBitmapIter *hbi, unsigned long *p_cur)
+{
+unsigned long cur = hbi->cur[HBITMAP_LEVELS - 1];
+
+if (cur == 0) {
+cur = hbitmap_iter_skip_words(hbi);
+if (cur == 0) {
+*p_cur = 0;
+return -1;
+}
+}
+
+/* The next call will resume work from the next word. */
+hbi->cur[HBITMAP_LEVELS - 1] = 0;
+*p_cur = cur;
+return hbi->pos;
+}
+
/* Count the number of set bits between start and end, not accounting for
* the granularity. Also an example of how to use hbitmap_iter_next_word.
*/
--
2.21.1
[PULL v2 09/11] nbd/server: introduce NBDExtentArray
From: Vladimir Sementsov-Ogievskiy
Introduce NBDExtentArray class, to handle extents list creation in more
controlled way and with fewer OUT parameters in functions.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Eric Blake
Message-id: 20200205112041.6003-9-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
nbd/server.c | 210 +--
1 file changed, 118 insertions(+), 92 deletions(-)
diff --git a/nbd/server.c b/nbd/server.c
index 3106aaf3b4..f90bb33a75 100644
--- a/nbd/server.c
+++ b/nbd/server.c
@@ -1909,27 +1909,98 @@ static int coroutine_fn
nbd_co_send_sparse_read(NBDClient *client,
return ret;
}
+typedef struct NBDExtentArray {
+NBDExtent *extents;
+unsigned int nb_alloc;
+unsigned int count;
+uint64_t total_length;
+bool can_add;
+bool converted_to_be;
+} NBDExtentArray;
+
+static NBDExtentArray *nbd_extent_array_new(unsigned int nb_alloc)
+{
+NBDExtentArray *ea = g_new0(NBDExtentArray, 1);
+
+ea->nb_alloc = nb_alloc;
+ea->extents = g_new(NBDExtent, nb_alloc);
+ea->can_add = true;
+
+return ea;
+}
+
+static void nbd_extent_array_free(NBDExtentArray *ea)
+{
+g_free(ea->extents);
+g_free(ea);
+}
+G_DEFINE_AUTOPTR_CLEANUP_FUNC(NBDExtentArray, nbd_extent_array_free);
+
+/* Further modifications of the array after conversion are abandoned */
+static void nbd_extent_array_convert_to_be(NBDExtentArray *ea)
+{
+int i;
+
+assert(!ea->converted_to_be);
+ea->can_add = false;
+ea->converted_to_be = true;
+
+for (i = 0; i < ea->count; i++) {
+ea->extents[i].flags = cpu_to_be32(ea->extents[i].flags);
+ea->extents[i].length = cpu_to_be32(ea->extents[i].length);
+}
+}
+
/*
- * Populate @extents from block status. Update @bytes to be the actual
- * length encoded (which may be smaller than the original), and update
- * @nb_extents to the number of extents used.
- *
- * Returns zero on success and -errno on bdrv_block_status_above failure.
+ * Add extent to NBDExtentArray. If extent can't be added (no available space),
+ * return -1.
+ * For safety, when returning -1 for the first time, .can_add is set to false,
+ * further call to nbd_extent_array_add() will crash.
+ * (to avoid the situation, when after failing to add an extent (returned -1),
+ * user miss this failure and add another extent, which is successfully added
+ * (array is full, but new extent may be squashed into the last one), then we
+ * have invalid array with skipped extent)
*/
+static int nbd_extent_array_add(NBDExtentArray *ea,
+uint32_t length, uint32_t flags)
+{
+assert(ea->can_add);
+
+if (!length) {
+return 0;
+}
+
+/* Extend previous extent if flags are the same */
+if (ea->count > 0 && flags == ea->extents[ea->count - 1].flags) {
+uint64_t sum = (uint64_t)length + ea->extents[ea->count - 1].length;
+
+if (sum <= UINT32_MAX) {
+ea->extents[ea->count - 1].length = sum;
+ea->total_length += length;
+return 0;
+}
+}
+
+if (ea->count >= ea->nb_alloc) {
+ea->can_add = false;
+return -1;
+}
+
+ea->total_length += length;
+ea->extents[ea->count] = (NBDExtent) {.length = length, .flags = flags};
+ea->count++;
+
+return 0;
+}
+
static int blockstatus_to_extents(BlockDriverState *bs, uint64_t offset,
- uint64_t *bytes, NBDExtent *extents,
- unsigned int *nb_extents)
+ uint64_t bytes, NBDExtentArray *ea)
{
-uint64_t remaining_bytes = *bytes;
-NBDExtent *extent = extents, *extents_end = extents + *nb_extents;
-bool first_extent = true;
-
-assert(*nb_extents);
-while (remaining_bytes) {
+while (bytes) {
uint32_t flags;
int64_t num;
-int ret = bdrv_block_status_above(bs, NULL, offset, remaining_bytes,
- &num, NULL, NULL);
+int ret = bdrv_block_status_above(bs, NULL, offset, bytes, &num,
+ NULL, NULL);
if (ret < 0) {
return ret;
@@ -1938,60 +2009,37 @@ static int blockstatus_to_extents(BlockDriverState *bs,
uint64_t offset,
flags = (ret & BDRV_BLOCK_ALLOCATED ? 0 : NBD_STATE_HOLE) |
(ret & BDRV_BLOCK_ZERO ? NBD_STATE_ZERO : 0);
-if (first_extent) {
-extent->flags = flags;
-extent->length = num;
-first_extent = false;
-} else if (flags == extent->flags) {
-/* extend current extent */
-extent->length += num;
-} else {
-if (extent + 1 == extents_end) {
-break;
-}
-
-
[PULL v2 07/11] block/dirty-bitmap: add _next_dirty API
From: Vladimir Sementsov-Ogievskiy
We have bdrv_dirty_bitmap_next_zero, let's add corresponding
bdrv_dirty_bitmap_next_dirty, which is more comfortable to use than
bitmap iterators in some cases.
For test modify test_hbitmap_next_zero_check_range to check both
next_zero and next_dirty and add some new checks.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
Reviewed-by: John Snow
Message-id: 20200205112041.6003-7-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
include/block/dirty-bitmap.h | 2 +
include/qemu/hbitmap.h | 13
block/dirty-bitmap.c | 6 ++
tests/test-hbitmap.c | 130 ---
util/hbitmap.c | 60
5 files changed, 126 insertions(+), 85 deletions(-)
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index 27c72cc56a..b1f0de12db 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -105,6 +105,8 @@ for (bitmap = bdrv_dirty_bitmap_first(bs); bitmap; \
bitmap = bdrv_dirty_bitmap_next(bitmap))
char *bdrv_dirty_bitmap_sha256(const BdrvDirtyBitmap *bitmap, Error **errp);
+int64_t bdrv_dirty_bitmap_next_dirty(BdrvDirtyBitmap *bitmap, int64_t offset,
+ int64_t bytes);
int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap *bitmap, int64_t offset,
int64_t bytes);
bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap *bitmap,
diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
index b6e85f3d5d..6e9ae51ed3 100644
--- a/include/qemu/hbitmap.h
+++ b/include/qemu/hbitmap.h
@@ -297,6 +297,19 @@ void hbitmap_free(HBitmap *hb);
*/
void hbitmap_iter_init(HBitmapIter *hbi, const HBitmap *hb, uint64_t first);
+/*
+ * hbitmap_next_dirty:
+ *
+ * Find next dirty bit within selected range. If not found, return -1.
+ *
+ * @hb: The HBitmap to operate on
+ * @start: The bit to start from.
+ * @count: Number of bits to proceed. If @start+@count > bitmap size, the whole
+ * bitmap is looked through. You can use INT64_MAX as @count to search up to
+ * the bitmap end.
+ */
+int64_t hbitmap_next_dirty(const HBitmap *hb, int64_t start, int64_t count);
+
/* hbitmap_next_zero:
*
* Find next not dirty bit within selected range. If not found, return -1.
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index af9f5411a6..1b14c8eb26 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -860,6 +860,12 @@ char *bdrv_dirty_bitmap_sha256(const BdrvDirtyBitmap
*bitmap, Error **errp)
return hbitmap_sha256(bitmap->bitmap, errp);
}
+int64_t bdrv_dirty_bitmap_next_dirty(BdrvDirtyBitmap *bitmap, int64_t offset,
+ int64_t bytes)
+{
+return hbitmap_next_dirty(bitmap->bitmap, offset, bytes);
+}
+
int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap *bitmap, int64_t offset,
int64_t bytes)
{
diff --git a/tests/test-hbitmap.c b/tests/test-hbitmap.c
index 9d210dc18c..8905b8a351 100644
--- a/tests/test-hbitmap.c
+++ b/tests/test-hbitmap.c
@@ -816,92 +816,108 @@ static void test_hbitmap_iter_and_reset(TestHBitmapData
*data,
hbitmap_iter_next(&hbi);
}
-static void test_hbitmap_next_zero_check_range(TestHBitmapData *data,
- int64_t start,
- int64_t count)
+static void test_hbitmap_next_x_check_range(TestHBitmapData *data,
+int64_t start,
+int64_t count)
{
-int64_t ret1 = hbitmap_next_zero(data->hb, start, count);
-int64_t ret2 = start;
+int64_t next_zero = hbitmap_next_zero(data->hb, start, count);
+int64_t next_dirty = hbitmap_next_dirty(data->hb, start, count);
+int64_t next;
int64_t end = start >= data->size || data->size - start < count ?
data->size : start + count;
+bool first_bit = hbitmap_get(data->hb, start);
-for ( ; ret2 < end && hbitmap_get(data->hb, ret2); ret2++) {
+for (next = start;
+ next < end && hbitmap_get(data->hb, next) == first_bit;
+ next++)
+{
;
}
-if (ret2 == end) {
-ret2 = -1;
+
+if (next == end) {
+next = -1;
}
-g_assert_cmpint(ret1, ==, ret2);
+g_assert_cmpint(next_dirty, ==, first_bit ? start : next);
+g_assert_cmpint(next_zero, ==, first_bit ? next : start);
}
-static void test_hbitmap_next_zero_check(TestHBitmapData *data, int64_t start)
+static void test_hbitmap_next_x_check(TestHBitmapData *data, int64_t start)
{
-test_hbitmap_next_zero_check_range(data, start, INT64_MAX);
+test_hbitmap_next_x_check_range(data, start, INT64_MAX);
}
-static void test_hbitmap_next_zero_do(TestHBitmapData *data, int granularity)
+static void test_hbitmap_next_x_
[PULL v2 00/11] Bitmaps patches
The following changes since commit d649689a8ecb2e276cc20d3af6d416e3c299cb17: Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging (2020-03-17 18:33:05 +) are available in the Git repository at: https://github.com/jnsnow/qemu.git tags/bitmaps-pull-request for you to fetch changes up to 2d00cbd8e222a4adc08f415c399e84590ee8ff9a: block/qcow2-bitmap: use bdrv_dirty_bitmap_next_dirty (2020-03-18 14:03:46 -0400) Pull request Eric Blake (1): build: Silence clang warning on older glib autoptr usage Vladimir Sementsov-Ogievskiy (10): hbitmap: assert that we don't create bitmap larger than INT64_MAX hbitmap: move hbitmap_iter_next_word to hbitmap.c hbitmap: unpublish hbitmap_iter_skip_words hbitmap: drop meta bitmaps as they are unused block/dirty-bitmap: switch _next_dirty_area and _next_zero to int64_t block/dirty-bitmap: add _next_dirty API block/dirty-bitmap: improve _next_dirty_area API nbd/server: introduce NBDExtentArray nbd/server: use bdrv_dirty_bitmap_next_dirty_area block/qcow2-bitmap: use bdrv_dirty_bitmap_next_dirty configure| 20 +++ include/block/dirty-bitmap.h | 9 +- include/qemu/hbitmap.h | 95 +++ block/dirty-bitmap.c | 16 +- block/qcow2-bitmap.c | 15 +- nbd/server.c | 251 ++-- tests/test-hbitmap.c | 316 +-- util/hbitmap.c | 134 +-- 8 files changed, 395 insertions(+), 461 deletions(-) -- 2.21.1
[PULL v2 06/11] block/dirty-bitmap: switch _next_dirty_area and _next_zero to int64_t
From: Vladimir Sementsov-Ogievskiy
We are going to introduce bdrv_dirty_bitmap_next_dirty so that same
variable may be used to store its return value and to be its parameter,
so it would int64_t.
Similarly, we are going to refactor hbitmap_next_dirty_area to use
hbitmap_next_dirty together with hbitmap_next_zero, therefore we want
hbitmap_next_zero parameter type to be int64_t too.
So, for convenience update all parameters of *_next_zero and
*_next_dirty_area to be int64_t.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
Reviewed-by: John Snow
Message-id: 20200205112041.6003-6-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
include/block/dirty-bitmap.h | 6 +++---
include/qemu/hbitmap.h | 7 +++
block/dirty-bitmap.c | 6 +++---
nbd/server.c | 2 +-
tests/test-hbitmap.c | 36 ++--
util/hbitmap.c | 13 -
6 files changed, 36 insertions(+), 34 deletions(-)
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index e2b20ecab9..27c72cc56a 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -105,10 +105,10 @@ for (bitmap = bdrv_dirty_bitmap_first(bs); bitmap; \
bitmap = bdrv_dirty_bitmap_next(bitmap))
char *bdrv_dirty_bitmap_sha256(const BdrvDirtyBitmap *bitmap, Error **errp);
-int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap *bitmap, uint64_t offset,
-uint64_t bytes);
+int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap *bitmap, int64_t offset,
+int64_t bytes);
bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap *bitmap,
- uint64_t *offset, uint64_t *bytes);
+ int64_t *offset, int64_t *bytes);
BdrvDirtyBitmap *bdrv_reclaim_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap,
Error **errp);
diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
index df922d8517..b6e85f3d5d 100644
--- a/include/qemu/hbitmap.h
+++ b/include/qemu/hbitmap.h
@@ -304,10 +304,10 @@ void hbitmap_iter_init(HBitmapIter *hbi, const HBitmap
*hb, uint64_t first);
* @hb: The HBitmap to operate on
* @start: The bit to start from.
* @count: Number of bits to proceed. If @start+@count > bitmap size, the whole
- * bitmap is looked through. You can use UINT64_MAX as @count to search up to
+ * bitmap is looked through. You can use INT64_MAX as @count to search up to
* the bitmap end.
*/
-int64_t hbitmap_next_zero(const HBitmap *hb, uint64_t start, uint64_t count);
+int64_t hbitmap_next_zero(const HBitmap *hb, int64_t start, int64_t count);
/* hbitmap_next_dirty_area:
* @hb: The HBitmap to operate on
@@ -322,8 +322,7 @@ int64_t hbitmap_next_zero(const HBitmap *hb, uint64_t
start, uint64_t count);
* @offset and @bytes appropriately. Otherwise returns false and leaves @offset
* and @bytes unchanged.
*/
-bool hbitmap_next_dirty_area(const HBitmap *hb, uint64_t *start,
- uint64_t *count);
+bool hbitmap_next_dirty_area(const HBitmap *hb, int64_t *start, int64_t
*count);
/**
* hbitmap_iter_next:
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 7039e82520..af9f5411a6 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -860,14 +860,14 @@ char *bdrv_dirty_bitmap_sha256(const BdrvDirtyBitmap
*bitmap, Error **errp)
return hbitmap_sha256(bitmap->bitmap, errp);
}
-int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap *bitmap, uint64_t offset,
-uint64_t bytes)
+int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap *bitmap, int64_t offset,
+int64_t bytes)
{
return hbitmap_next_zero(bitmap->bitmap, offset, bytes);
}
bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap *bitmap,
- uint64_t *offset, uint64_t *bytes)
+ int64_t *offset, int64_t *bytes)
{
return hbitmap_next_dirty_area(bitmap->bitmap, offset, bytes);
}
diff --git a/nbd/server.c b/nbd/server.c
index 11a31094ff..3106aaf3b4 100644
--- a/nbd/server.c
+++ b/nbd/server.c
@@ -2055,7 +2055,7 @@ static unsigned int bitmap_to_extents(BdrvDirtyBitmap
*bitmap, uint64_t offset,
bool next_dirty = !dirty;
if (dirty) {
-end = bdrv_dirty_bitmap_next_zero(bitmap, begin, UINT64_MAX);
+end = bdrv_dirty_bitmap_next_zero(bitmap, begin, INT64_MAX);
} else {
bdrv_set_dirty_iter(it, begin);
end = bdrv_dirty_iter_next(it);
diff --git a/tests/test-hbitmap.c b/tests/test-hbitmap.c
index aeaa0b3f22..9d210dc18c 100644
--- a/tests/test-hbitmap.c
+++ b/tests/test-hbitmap.c
@@ -817,8 +817,8 @@ static void test_hbitmap_iter_and_reset(TestHBitmapData
*data,
}
static void test_hbitmap_next_zero
[PULL 07/10] block/dirty-bitmap: improve _next_dirty_area API
From: Vladimir Sementsov-Ogievskiy
Firstly, _next_dirty_area is for scenarios when we may contiguously
search for next dirty area inside some limited region, so it is more
comfortable to specify "end" which should not be recalculated on each
iteration.
Secondly, let's add a possibility to limit resulting area size, not
limiting searching area. This will be used in NBD code in further
commit. (Note that now bdrv_dirty_bitmap_next_dirty_area is unused)
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
Reviewed-by: John Snow
Message-id: 20200205112041.6003-8-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
include/block/dirty-bitmap.h | 3 ++-
include/qemu/hbitmap.h | 23 ++
block/dirty-bitmap.c | 6 +++--
tests/test-hbitmap.c | 45 +++-
util/hbitmap.c | 44 +--
5 files changed, 75 insertions(+), 46 deletions(-)
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index b1f0de12db..8a10029418 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -110,7 +110,8 @@ int64_t bdrv_dirty_bitmap_next_dirty(BdrvDirtyBitmap
*bitmap, int64_t offset,
int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap *bitmap, int64_t offset,
int64_t bytes);
bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap *bitmap,
- int64_t *offset, int64_t *bytes);
+int64_t start, int64_t end, int64_t max_dirty_count,
+int64_t *dirty_start, int64_t *dirty_count);
BdrvDirtyBitmap *bdrv_reclaim_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap,
Error **errp);
diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
index 6e9ae51ed3..5e71b6d6f7 100644
--- a/include/qemu/hbitmap.h
+++ b/include/qemu/hbitmap.h
@@ -324,18 +324,21 @@ int64_t hbitmap_next_zero(const HBitmap *hb, int64_t
start, int64_t count);
/* hbitmap_next_dirty_area:
* @hb: The HBitmap to operate on
- * @start: in-out parameter.
- * in: the offset to start from
- * out: (if area found) start of found area
- * @count: in-out parameter.
- * in: length of requested region
- * out: length of found area
+ * @start: the offset to start from
+ * @end: end of requested area
+ * @max_dirty_count: limit for out parameter dirty_count
+ * @dirty_start: on success: start of found area
+ * @dirty_count: on success: length of found area
*
- * If dirty area found within [@start, @start + @count), returns true and sets
- * @offset and @bytes appropriately. Otherwise returns false and leaves @offset
- * and @bytes unchanged.
+ * If dirty area found within [@start, @end), returns true and sets
+ * @dirty_start and @dirty_count appropriately. @dirty_count will not exceed
+ * @max_dirty_count.
+ * If dirty area was not found, returns false and leaves @dirty_start and
+ * @dirty_count unchanged.
*/
-bool hbitmap_next_dirty_area(const HBitmap *hb, int64_t *start, int64_t
*count);
+bool hbitmap_next_dirty_area(const HBitmap *hb, int64_t start, int64_t end,
+ int64_t max_dirty_count,
+ int64_t *dirty_start, int64_t *dirty_count);
/**
* hbitmap_iter_next:
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 1b14c8eb26..063793e316 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -873,9 +873,11 @@ int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap
*bitmap, int64_t offset,
}
bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap *bitmap,
- int64_t *offset, int64_t *bytes)
+int64_t start, int64_t end, int64_t max_dirty_count,
+int64_t *dirty_start, int64_t *dirty_count)
{
-return hbitmap_next_dirty_area(bitmap->bitmap, offset, bytes);
+return hbitmap_next_dirty_area(bitmap->bitmap, start, end, max_dirty_count,
+ dirty_start, dirty_count);
}
/**
diff --git a/tests/test-hbitmap.c b/tests/test-hbitmap.c
index 8905b8a351..b6726cf76b 100644
--- a/tests/test-hbitmap.c
+++ b/tests/test-hbitmap.c
@@ -920,18 +920,19 @@ static void
test_hbitmap_next_x_after_truncate(TestHBitmapData *data,
test_hbitmap_next_x_check(data, 0);
}
-static void test_hbitmap_next_dirty_area_check(TestHBitmapData *data,
- int64_t offset,
- int64_t count)
+static void test_hbitmap_next_dirty_area_check_limited(TestHBitmapData *data,
+ int64_t offset,
+ int64_t count,
+ int64_t max_dirty)
{
int64_t off1, off2;
int64_t len1 = 0, len2;
bool ret1, ret2;
int64_t end;
-off1 = offs
[PULL 08/10] nbd/server: introduce NBDExtentArray
From: Vladimir Sementsov-Ogievskiy
Introduce NBDExtentArray class, to handle extents list creation in more
controlled way and with fewer OUT parameters in functions.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Eric Blake
Message-id: 20200205112041.6003-9-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
nbd/server.c | 210 +--
1 file changed, 118 insertions(+), 92 deletions(-)
diff --git a/nbd/server.c b/nbd/server.c
index 3106aaf3b4..f90bb33a75 100644
--- a/nbd/server.c
+++ b/nbd/server.c
@@ -1909,27 +1909,98 @@ static int coroutine_fn
nbd_co_send_sparse_read(NBDClient *client,
return ret;
}
+typedef struct NBDExtentArray {
+NBDExtent *extents;
+unsigned int nb_alloc;
+unsigned int count;
+uint64_t total_length;
+bool can_add;
+bool converted_to_be;
+} NBDExtentArray;
+
+static NBDExtentArray *nbd_extent_array_new(unsigned int nb_alloc)
+{
+NBDExtentArray *ea = g_new0(NBDExtentArray, 1);
+
+ea->nb_alloc = nb_alloc;
+ea->extents = g_new(NBDExtent, nb_alloc);
+ea->can_add = true;
+
+return ea;
+}
+
+static void nbd_extent_array_free(NBDExtentArray *ea)
+{
+g_free(ea->extents);
+g_free(ea);
+}
+G_DEFINE_AUTOPTR_CLEANUP_FUNC(NBDExtentArray, nbd_extent_array_free);
+
+/* Further modifications of the array after conversion are abandoned */
+static void nbd_extent_array_convert_to_be(NBDExtentArray *ea)
+{
+int i;
+
+assert(!ea->converted_to_be);
+ea->can_add = false;
+ea->converted_to_be = true;
+
+for (i = 0; i < ea->count; i++) {
+ea->extents[i].flags = cpu_to_be32(ea->extents[i].flags);
+ea->extents[i].length = cpu_to_be32(ea->extents[i].length);
+}
+}
+
/*
- * Populate @extents from block status. Update @bytes to be the actual
- * length encoded (which may be smaller than the original), and update
- * @nb_extents to the number of extents used.
- *
- * Returns zero on success and -errno on bdrv_block_status_above failure.
+ * Add extent to NBDExtentArray. If extent can't be added (no available space),
+ * return -1.
+ * For safety, when returning -1 for the first time, .can_add is set to false,
+ * further call to nbd_extent_array_add() will crash.
+ * (to avoid the situation, when after failing to add an extent (returned -1),
+ * user miss this failure and add another extent, which is successfully added
+ * (array is full, but new extent may be squashed into the last one), then we
+ * have invalid array with skipped extent)
*/
+static int nbd_extent_array_add(NBDExtentArray *ea,
+uint32_t length, uint32_t flags)
+{
+assert(ea->can_add);
+
+if (!length) {
+return 0;
+}
+
+/* Extend previous extent if flags are the same */
+if (ea->count > 0 && flags == ea->extents[ea->count - 1].flags) {
+uint64_t sum = (uint64_t)length + ea->extents[ea->count - 1].length;
+
+if (sum <= UINT32_MAX) {
+ea->extents[ea->count - 1].length = sum;
+ea->total_length += length;
+return 0;
+}
+}
+
+if (ea->count >= ea->nb_alloc) {
+ea->can_add = false;
+return -1;
+}
+
+ea->total_length += length;
+ea->extents[ea->count] = (NBDExtent) {.length = length, .flags = flags};
+ea->count++;
+
+return 0;
+}
+
static int blockstatus_to_extents(BlockDriverState *bs, uint64_t offset,
- uint64_t *bytes, NBDExtent *extents,
- unsigned int *nb_extents)
+ uint64_t bytes, NBDExtentArray *ea)
{
-uint64_t remaining_bytes = *bytes;
-NBDExtent *extent = extents, *extents_end = extents + *nb_extents;
-bool first_extent = true;
-
-assert(*nb_extents);
-while (remaining_bytes) {
+while (bytes) {
uint32_t flags;
int64_t num;
-int ret = bdrv_block_status_above(bs, NULL, offset, remaining_bytes,
- &num, NULL, NULL);
+int ret = bdrv_block_status_above(bs, NULL, offset, bytes, &num,
+ NULL, NULL);
if (ret < 0) {
return ret;
@@ -1938,60 +2009,37 @@ static int blockstatus_to_extents(BlockDriverState *bs,
uint64_t offset,
flags = (ret & BDRV_BLOCK_ALLOCATED ? 0 : NBD_STATE_HOLE) |
(ret & BDRV_BLOCK_ZERO ? NBD_STATE_ZERO : 0);
-if (first_extent) {
-extent->flags = flags;
-extent->length = num;
-first_extent = false;
-} else if (flags == extent->flags) {
-/* extend current extent */
-extent->length += num;
-} else {
-if (extent + 1 == extents_end) {
-break;
-}
-
-
[PULL 10/10] block/qcow2-bitmap: use bdrv_dirty_bitmap_next_dirty
From: Vladimir Sementsov-Ogievskiy
store_bitmap_data() loop does bdrv_set_dirty_iter() on each iteration,
which means that we actually don't need iterator itself and we can use
simpler bitmap API.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
Reviewed-by: John Snow
Message-id: 20200205112041.6003-11-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
block/qcow2-bitmap.c | 15 +--
1 file changed, 5 insertions(+), 10 deletions(-)
diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
index 82c9f3..cb06954b4a 100644
--- a/block/qcow2-bitmap.c
+++ b/block/qcow2-bitmap.c
@@ -1288,7 +1288,6 @@ static uint64_t *store_bitmap_data(BlockDriverState *bs,
uint64_t bm_size = bdrv_dirty_bitmap_size(bitmap);
const char *bm_name = bdrv_dirty_bitmap_name(bitmap);
uint8_t *buf = NULL;
-BdrvDirtyBitmapIter *dbi;
uint64_t *tb;
uint64_t tb_size =
size_to_clusters(s,
@@ -1307,12 +1306,14 @@ static uint64_t *store_bitmap_data(BlockDriverState *bs,
return NULL;
}
-dbi = bdrv_dirty_iter_new(bitmap);
buf = g_malloc(s->cluster_size);
limit = bytes_covered_by_bitmap_cluster(s, bitmap);
assert(DIV_ROUND_UP(bm_size, limit) == tb_size);
-while ((offset = bdrv_dirty_iter_next(dbi)) >= 0) {
+offset = 0;
+while ((offset = bdrv_dirty_bitmap_next_dirty(bitmap, offset, INT64_MAX))
+ >= 0)
+{
uint64_t cluster = offset / limit;
uint64_t end, write_size;
int64_t off;
@@ -1355,23 +1356,17 @@ static uint64_t *store_bitmap_data(BlockDriverState *bs,
goto fail;
}
-if (end >= bm_size) {
-break;
-}
-
-bdrv_set_dirty_iter(dbi, end);
+offset = end;
}
*bitmap_table_size = tb_size;
g_free(buf);
-bdrv_dirty_iter_free(dbi);
return tb;
fail:
clear_bitmap_table(bs, tb, tb_size);
g_free(buf);
-bdrv_dirty_iter_free(dbi);
g_free(tb);
return NULL;
--
2.21.1
[PULL 02/10] hbitmap: move hbitmap_iter_next_word to hbitmap.c
From: Vladimir Sementsov-Ogievskiy
The function is definitely internal (it's not used by third party and
it has complicated interface). Move it to .c file.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
Reviewed-by: John Snow
Message-id: 20200205112041.6003-3-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
include/qemu/hbitmap.h | 30 --
util/hbitmap.c | 29 +
2 files changed, 29 insertions(+), 30 deletions(-)
diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
index 1bf944ca3d..ab227b117f 100644
--- a/include/qemu/hbitmap.h
+++ b/include/qemu/hbitmap.h
@@ -362,34 +362,4 @@ void hbitmap_free_meta(HBitmap *hb);
*/
int64_t hbitmap_iter_next(HBitmapIter *hbi);
-/**
- * hbitmap_iter_next_word:
- * @hbi: HBitmapIter to operate on.
- * @p_cur: Location where to store the next non-zero word.
- *
- * Return the index of the next nonzero word that is set in @hbi's
- * associated HBitmap, and set *p_cur to the content of that word
- * (bits before the index that was passed to hbitmap_iter_init are
- * trimmed on the first call). Return -1, and set *p_cur to zero,
- * if all remaining words are zero.
- */
-static inline size_t hbitmap_iter_next_word(HBitmapIter *hbi, unsigned long
*p_cur)
-{
-unsigned long cur = hbi->cur[HBITMAP_LEVELS - 1];
-
-if (cur == 0) {
-cur = hbitmap_iter_skip_words(hbi);
-if (cur == 0) {
-*p_cur = 0;
-return -1;
-}
-}
-
-/* The next call will resume work from the next word. */
-hbi->cur[HBITMAP_LEVELS - 1] = 0;
-*p_cur = cur;
-return hbi->pos;
-}
-
-
#endif
diff --git a/util/hbitmap.c b/util/hbitmap.c
index 7f9b3e0cd7..a368dc5ef7 100644
--- a/util/hbitmap.c
+++ b/util/hbitmap.c
@@ -298,6 +298,35 @@ uint64_t hbitmap_count(const HBitmap *hb)
return hb->count << hb->granularity;
}
+/**
+ * hbitmap_iter_next_word:
+ * @hbi: HBitmapIter to operate on.
+ * @p_cur: Location where to store the next non-zero word.
+ *
+ * Return the index of the next nonzero word that is set in @hbi's
+ * associated HBitmap, and set *p_cur to the content of that word
+ * (bits before the index that was passed to hbitmap_iter_init are
+ * trimmed on the first call). Return -1, and set *p_cur to zero,
+ * if all remaining words are zero.
+ */
+static size_t hbitmap_iter_next_word(HBitmapIter *hbi, unsigned long *p_cur)
+{
+unsigned long cur = hbi->cur[HBITMAP_LEVELS - 1];
+
+if (cur == 0) {
+cur = hbitmap_iter_skip_words(hbi);
+if (cur == 0) {
+*p_cur = 0;
+return -1;
+}
+}
+
+/* The next call will resume work from the next word. */
+hbi->cur[HBITMAP_LEVELS - 1] = 0;
+*p_cur = cur;
+return hbi->pos;
+}
+
/* Count the number of set bits between start and end, not accounting for
* the granularity. Also an example of how to use hbitmap_iter_next_word.
*/
--
2.21.1
[PULL 09/10] nbd/server: use bdrv_dirty_bitmap_next_dirty_area
From: Vladimir Sementsov-Ogievskiy
Use bdrv_dirty_bitmap_next_dirty_area for bitmap_to_extents. Since
bdrv_dirty_bitmap_next_dirty_area is very accurate in its interface,
we'll never exceed requested region with last chunk. So, we don't need
dont_fragment, and bitmap_to_extents() interface becomes clean enough
to not require any comment.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Eric Blake
Message-id: 20200205112041.6003-10-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
nbd/server.c | 59 +---
1 file changed, 19 insertions(+), 40 deletions(-)
diff --git a/nbd/server.c b/nbd/server.c
index f90bb33a75..02b1ed0801 100644
--- a/nbd/server.c
+++ b/nbd/server.c
@@ -2068,57 +2068,36 @@ static int nbd_co_send_block_status(NBDClient *client,
uint64_t handle,
return nbd_co_send_extents(client, handle, ea, last, context_id, errp);
}
-/*
- * Populate @ea from a dirty bitmap. Unless @dont_fragment, the
- * final extent may exceed the original @length.
- */
+/* Populate @ea from a dirty bitmap. */
static void bitmap_to_extents(BdrvDirtyBitmap *bitmap,
uint64_t offset, uint64_t length,
- NBDExtentArray *ea, bool dont_fragment)
+ NBDExtentArray *es)
{
-uint64_t begin = offset, end = offset;
-uint64_t overall_end = offset + length;
-BdrvDirtyBitmapIter *it;
-bool dirty;
+int64_t start, dirty_start, dirty_count;
+int64_t end = offset + length;
+bool full = false;
bdrv_dirty_bitmap_lock(bitmap);
-it = bdrv_dirty_iter_new(bitmap);
-dirty = bdrv_dirty_bitmap_get_locked(bitmap, offset);
-
-while (begin < overall_end) {
-bool next_dirty = !dirty;
-
-if (dirty) {
-end = bdrv_dirty_bitmap_next_zero(bitmap, begin, INT64_MAX);
-} else {
-bdrv_set_dirty_iter(it, begin);
-end = bdrv_dirty_iter_next(it);
-}
-if (end == -1 || end - begin > UINT32_MAX) {
-/* Cap to an aligned value < 4G beyond begin. */
-end = MIN(bdrv_dirty_bitmap_size(bitmap),
- begin + UINT32_MAX + 1 -
- bdrv_dirty_bitmap_granularity(bitmap));
-next_dirty = dirty;
-}
-if (dont_fragment && end > overall_end) {
-end = overall_end;
-}
-
-if (nbd_extent_array_add(ea, end - begin,
- dirty ? NBD_STATE_DIRTY : 0) < 0) {
+for (start = offset;
+ bdrv_dirty_bitmap_next_dirty_area(bitmap, start, end, INT32_MAX,
+ &dirty_start, &dirty_count);
+ start = dirty_start + dirty_count)
+{
+if ((nbd_extent_array_add(es, dirty_start - start, 0) < 0) ||
+(nbd_extent_array_add(es, dirty_count, NBD_STATE_DIRTY) < 0))
+{
+full = true;
break;
}
-begin = end;
-dirty = next_dirty;
}
-bdrv_dirty_iter_free(it);
+if (!full) {
+/* last non dirty extent */
+nbd_extent_array_add(es, end - start, 0);
+}
bdrv_dirty_bitmap_unlock(bitmap);
-
-assert(offset < end);
}
static int nbd_co_send_bitmap(NBDClient *client, uint64_t handle,
@@ -2129,7 +2108,7 @@ static int nbd_co_send_bitmap(NBDClient *client, uint64_t
handle,
unsigned int nb_extents = dont_fragment ? 1 : NBD_MAX_BLOCK_STATUS_EXTENTS;
g_autoptr(NBDExtentArray) ea = nbd_extent_array_new(nb_extents);
-bitmap_to_extents(bitmap, offset, length, ea, dont_fragment);
+bitmap_to_extents(bitmap, offset, length, ea);
return nbd_co_send_extents(client, handle, ea, last, context_id, errp);
}
--
2.21.1
[PULL 04/10] hbitmap: drop meta bitmaps as they are unused
From: Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
Reviewed-by: John Snow
Message-id: 20200205112041.6003-5-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
include/qemu/hbitmap.h | 21
tests/test-hbitmap.c | 115 -
util/hbitmap.c | 16 --
3 files changed, 152 deletions(-)
diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
index 15837a0e2d..df922d8517 100644
--- a/include/qemu/hbitmap.h
+++ b/include/qemu/hbitmap.h
@@ -325,27 +325,6 @@ int64_t hbitmap_next_zero(const HBitmap *hb, uint64_t
start, uint64_t count);
bool hbitmap_next_dirty_area(const HBitmap *hb, uint64_t *start,
uint64_t *count);
-/* hbitmap_create_meta:
- * Create a "meta" hbitmap to track dirtiness of the bits in this HBitmap.
- * The caller owns the created bitmap and must call hbitmap_free_meta(hb) to
- * free it.
- *
- * Currently, we only guarantee that if a bit in the hbitmap is changed it
- * will be reflected in the meta bitmap, but we do not yet guarantee the
- * opposite.
- *
- * @hb: The HBitmap to operate on.
- * @chunk_size: How many bits in @hb does one bit in the meta track.
- */
-HBitmap *hbitmap_create_meta(HBitmap *hb, int chunk_size);
-
-/* hbitmap_free_meta:
- * Free the meta bitmap of @hb.
- *
- * @hb: The HBitmap whose meta bitmap should be freed.
- */
-void hbitmap_free_meta(HBitmap *hb);
-
/**
* hbitmap_iter_next:
* @hbi: HBitmapIter to operate on.
diff --git a/tests/test-hbitmap.c b/tests/test-hbitmap.c
index e1f867085f..aeaa0b3f22 100644
--- a/tests/test-hbitmap.c
+++ b/tests/test-hbitmap.c
@@ -22,7 +22,6 @@
typedef struct TestHBitmapData {
HBitmap *hb;
-HBitmap *meta;
unsigned long *bits;
size_t size;
size_t old_size;
@@ -94,14 +93,6 @@ static void hbitmap_test_init(TestHBitmapData *data,
}
}
-static void hbitmap_test_init_meta(TestHBitmapData *data,
- uint64_t size, int granularity,
- int meta_chunk)
-{
-hbitmap_test_init(data, size, granularity);
-data->meta = hbitmap_create_meta(data->hb, meta_chunk);
-}
-
static inline size_t hbitmap_test_array_size(size_t bits)
{
size_t n = DIV_ROUND_UP(bits, BITS_PER_LONG);
@@ -144,9 +135,6 @@ static void hbitmap_test_teardown(TestHBitmapData *data,
const void *unused)
{
if (data->hb) {
-if (data->meta) {
-hbitmap_free_meta(data->hb);
-}
hbitmap_free(data->hb);
data->hb = NULL;
}
@@ -648,96 +636,6 @@ static void
test_hbitmap_truncate_shrink_large(TestHBitmapData *data,
hbitmap_test_truncate(data, size, -diff, 0);
}
-static void hbitmap_check_meta(TestHBitmapData *data,
- int64_t start, int count)
-{
-int64_t i;
-
-for (i = 0; i < data->size; i++) {
-if (i >= start && i < start + count) {
-g_assert(hbitmap_get(data->meta, i));
-} else {
-g_assert(!hbitmap_get(data->meta, i));
-}
-}
-}
-
-static void hbitmap_test_meta(TestHBitmapData *data,
- int64_t start, int count,
- int64_t check_start, int check_count)
-{
-hbitmap_reset_all(data->hb);
-hbitmap_reset_all(data->meta);
-
-/* Test "unset" -> "unset" will not update meta. */
-hbitmap_reset(data->hb, start, count);
-hbitmap_check_meta(data, 0, 0);
-
-/* Test "unset" -> "set" will update meta */
-hbitmap_set(data->hb, start, count);
-hbitmap_check_meta(data, check_start, check_count);
-
-/* Test "set" -> "set" will not update meta */
-hbitmap_reset_all(data->meta);
-hbitmap_set(data->hb, start, count);
-hbitmap_check_meta(data, 0, 0);
-
-/* Test "set" -> "unset" will update meta */
-hbitmap_reset_all(data->meta);
-hbitmap_reset(data->hb, start, count);
-hbitmap_check_meta(data, check_start, check_count);
-}
-
-static void hbitmap_test_meta_do(TestHBitmapData *data, int chunk_size)
-{
-uint64_t size = chunk_size * 100;
-hbitmap_test_init_meta(data, size, 0, chunk_size);
-
-hbitmap_test_meta(data, 0, 1, 0, chunk_size);
-hbitmap_test_meta(data, 0, chunk_size, 0, chunk_size);
-hbitmap_test_meta(data, chunk_size - 1, 1, 0, chunk_size);
-hbitmap_test_meta(data, chunk_size - 1, 2, 0, chunk_size * 2);
-hbitmap_test_meta(data, chunk_size - 1, chunk_size + 1, 0, chunk_size * 2);
-hbitmap_test_meta(data, chunk_size - 1, chunk_size + 2, 0, chunk_size * 3);
-hbitmap_test_meta(data, 7 * chunk_size - 1, chunk_size + 2,
- 6 * chunk_size, chunk_size * 3);
-hbitmap_test_meta(data, si
[PULL 06/10] block/dirty-bitmap: add _next_dirty API
From: Vladimir Sementsov-Ogievskiy
We have bdrv_dirty_bitmap_next_zero, let's add corresponding
bdrv_dirty_bitmap_next_dirty, which is more comfortable to use than
bitmap iterators in some cases.
For test modify test_hbitmap_next_zero_check_range to check both
next_zero and next_dirty and add some new checks.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
Reviewed-by: John Snow
Message-id: 20200205112041.6003-7-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
include/block/dirty-bitmap.h | 2 +
include/qemu/hbitmap.h | 13
block/dirty-bitmap.c | 6 ++
tests/test-hbitmap.c | 130 ---
util/hbitmap.c | 60
5 files changed, 126 insertions(+), 85 deletions(-)
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index 27c72cc56a..b1f0de12db 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -105,6 +105,8 @@ for (bitmap = bdrv_dirty_bitmap_first(bs); bitmap; \
bitmap = bdrv_dirty_bitmap_next(bitmap))
char *bdrv_dirty_bitmap_sha256(const BdrvDirtyBitmap *bitmap, Error **errp);
+int64_t bdrv_dirty_bitmap_next_dirty(BdrvDirtyBitmap *bitmap, int64_t offset,
+ int64_t bytes);
int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap *bitmap, int64_t offset,
int64_t bytes);
bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap *bitmap,
diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
index b6e85f3d5d..6e9ae51ed3 100644
--- a/include/qemu/hbitmap.h
+++ b/include/qemu/hbitmap.h
@@ -297,6 +297,19 @@ void hbitmap_free(HBitmap *hb);
*/
void hbitmap_iter_init(HBitmapIter *hbi, const HBitmap *hb, uint64_t first);
+/*
+ * hbitmap_next_dirty:
+ *
+ * Find next dirty bit within selected range. If not found, return -1.
+ *
+ * @hb: The HBitmap to operate on
+ * @start: The bit to start from.
+ * @count: Number of bits to proceed. If @start+@count > bitmap size, the whole
+ * bitmap is looked through. You can use INT64_MAX as @count to search up to
+ * the bitmap end.
+ */
+int64_t hbitmap_next_dirty(const HBitmap *hb, int64_t start, int64_t count);
+
/* hbitmap_next_zero:
*
* Find next not dirty bit within selected range. If not found, return -1.
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index af9f5411a6..1b14c8eb26 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -860,6 +860,12 @@ char *bdrv_dirty_bitmap_sha256(const BdrvDirtyBitmap
*bitmap, Error **errp)
return hbitmap_sha256(bitmap->bitmap, errp);
}
+int64_t bdrv_dirty_bitmap_next_dirty(BdrvDirtyBitmap *bitmap, int64_t offset,
+ int64_t bytes)
+{
+return hbitmap_next_dirty(bitmap->bitmap, offset, bytes);
+}
+
int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap *bitmap, int64_t offset,
int64_t bytes)
{
diff --git a/tests/test-hbitmap.c b/tests/test-hbitmap.c
index 9d210dc18c..8905b8a351 100644
--- a/tests/test-hbitmap.c
+++ b/tests/test-hbitmap.c
@@ -816,92 +816,108 @@ static void test_hbitmap_iter_and_reset(TestHBitmapData
*data,
hbitmap_iter_next(&hbi);
}
-static void test_hbitmap_next_zero_check_range(TestHBitmapData *data,
- int64_t start,
- int64_t count)
+static void test_hbitmap_next_x_check_range(TestHBitmapData *data,
+int64_t start,
+int64_t count)
{
-int64_t ret1 = hbitmap_next_zero(data->hb, start, count);
-int64_t ret2 = start;
+int64_t next_zero = hbitmap_next_zero(data->hb, start, count);
+int64_t next_dirty = hbitmap_next_dirty(data->hb, start, count);
+int64_t next;
int64_t end = start >= data->size || data->size - start < count ?
data->size : start + count;
+bool first_bit = hbitmap_get(data->hb, start);
-for ( ; ret2 < end && hbitmap_get(data->hb, ret2); ret2++) {
+for (next = start;
+ next < end && hbitmap_get(data->hb, next) == first_bit;
+ next++)
+{
;
}
-if (ret2 == end) {
-ret2 = -1;
+
+if (next == end) {
+next = -1;
}
-g_assert_cmpint(ret1, ==, ret2);
+g_assert_cmpint(next_dirty, ==, first_bit ? start : next);
+g_assert_cmpint(next_zero, ==, first_bit ? next : start);
}
-static void test_hbitmap_next_zero_check(TestHBitmapData *data, int64_t start)
+static void test_hbitmap_next_x_check(TestHBitmapData *data, int64_t start)
{
-test_hbitmap_next_zero_check_range(data, start, INT64_MAX);
+test_hbitmap_next_x_check_range(data, start, INT64_MAX);
}
-static void test_hbitmap_next_zero_do(TestHBitmapData *data, int granularity)
+static void test_hbitmap_next_x_
[PULL 01/10] hbitmap: assert that we don't create bitmap larger than INT64_MAX
From: Vladimir Sementsov-Ogievskiy We have APIs which returns signed int64_t, to be able to return error. Therefore we can't handle bitmaps with absolute size larger than (INT64_MAX+1). Still, keep maximum to be INT64_MAX which is a bit safer. Note, that bitmaps are used to represent disk images, which can't exceed INT64_MAX anyway. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: John Snow Message-id: 20200205112041.6003-2-vsement...@virtuozzo.com Signed-off-by: John Snow --- util/hbitmap.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/util/hbitmap.c b/util/hbitmap.c index 242c6e519c..7f9b3e0cd7 100644 --- a/util/hbitmap.c +++ b/util/hbitmap.c @@ -716,6 +716,7 @@ HBitmap *hbitmap_alloc(uint64_t size, int granularity) HBitmap *hb = g_new0(struct HBitmap, 1); unsigned i; +assert(size <= INT64_MAX); hb->orig_size = size; assert(granularity >= 0 && granularity < 64); @@ -746,6 +747,7 @@ void hbitmap_truncate(HBitmap *hb, uint64_t size) uint64_t num_elements = size; uint64_t old; +assert(size <= INT64_MAX); hb->orig_size = size; /* Size comes in as logical elements, adjust for granularity. */ -- 2.21.1
[PULL 03/10] hbitmap: unpublish hbitmap_iter_skip_words
From: Vladimir Sementsov-Ogievskiy
Function is internal and even commented as internal. Drop its
definition from .h file.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
Reviewed-by: John Snow
Message-id: 20200205112041.6003-4-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
include/qemu/hbitmap.h | 7 ---
util/hbitmap.c | 2 +-
2 files changed, 1 insertion(+), 8 deletions(-)
diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
index ab227b117f..15837a0e2d 100644
--- a/include/qemu/hbitmap.h
+++ b/include/qemu/hbitmap.h
@@ -297,13 +297,6 @@ void hbitmap_free(HBitmap *hb);
*/
void hbitmap_iter_init(HBitmapIter *hbi, const HBitmap *hb, uint64_t first);
-/* hbitmap_iter_skip_words:
- * @hbi: HBitmapIter to operate on.
- *
- * Internal function used by hbitmap_iter_next and hbitmap_iter_next_word.
- */
-unsigned long hbitmap_iter_skip_words(HBitmapIter *hbi);
-
/* hbitmap_next_zero:
*
* Find next not dirty bit within selected range. If not found, return -1.
diff --git a/util/hbitmap.c b/util/hbitmap.c
index a368dc5ef7..26145d4b9e 100644
--- a/util/hbitmap.c
+++ b/util/hbitmap.c
@@ -104,7 +104,7 @@ struct HBitmap {
/* Advance hbi to the next nonzero word and return it. hbi->pos
* is updated. Returns zero if we reach the end of the bitmap.
*/
-unsigned long hbitmap_iter_skip_words(HBitmapIter *hbi)
+static unsigned long hbitmap_iter_skip_words(HBitmapIter *hbi)
{
size_t pos = hbi->pos;
const HBitmap *hb = hbi->hb;
--
2.21.1
[PULL 05/10] block/dirty-bitmap: switch _next_dirty_area and _next_zero to int64_t
From: Vladimir Sementsov-Ogievskiy
We are going to introduce bdrv_dirty_bitmap_next_dirty so that same
variable may be used to store its return value and to be its parameter,
so it would int64_t.
Similarly, we are going to refactor hbitmap_next_dirty_area to use
hbitmap_next_dirty together with hbitmap_next_zero, therefore we want
hbitmap_next_zero parameter type to be int64_t too.
So, for convenience update all parameters of *_next_zero and
*_next_dirty_area to be int64_t.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
Reviewed-by: John Snow
Message-id: 20200205112041.6003-6-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
include/block/dirty-bitmap.h | 6 +++---
include/qemu/hbitmap.h | 7 +++
block/dirty-bitmap.c | 6 +++---
nbd/server.c | 2 +-
tests/test-hbitmap.c | 36 ++--
util/hbitmap.c | 13 -
6 files changed, 36 insertions(+), 34 deletions(-)
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index e2b20ecab9..27c72cc56a 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -105,10 +105,10 @@ for (bitmap = bdrv_dirty_bitmap_first(bs); bitmap; \
bitmap = bdrv_dirty_bitmap_next(bitmap))
char *bdrv_dirty_bitmap_sha256(const BdrvDirtyBitmap *bitmap, Error **errp);
-int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap *bitmap, uint64_t offset,
-uint64_t bytes);
+int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap *bitmap, int64_t offset,
+int64_t bytes);
bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap *bitmap,
- uint64_t *offset, uint64_t *bytes);
+ int64_t *offset, int64_t *bytes);
BdrvDirtyBitmap *bdrv_reclaim_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap,
Error **errp);
diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
index df922d8517..b6e85f3d5d 100644
--- a/include/qemu/hbitmap.h
+++ b/include/qemu/hbitmap.h
@@ -304,10 +304,10 @@ void hbitmap_iter_init(HBitmapIter *hbi, const HBitmap
*hb, uint64_t first);
* @hb: The HBitmap to operate on
* @start: The bit to start from.
* @count: Number of bits to proceed. If @start+@count > bitmap size, the whole
- * bitmap is looked through. You can use UINT64_MAX as @count to search up to
+ * bitmap is looked through. You can use INT64_MAX as @count to search up to
* the bitmap end.
*/
-int64_t hbitmap_next_zero(const HBitmap *hb, uint64_t start, uint64_t count);
+int64_t hbitmap_next_zero(const HBitmap *hb, int64_t start, int64_t count);
/* hbitmap_next_dirty_area:
* @hb: The HBitmap to operate on
@@ -322,8 +322,7 @@ int64_t hbitmap_next_zero(const HBitmap *hb, uint64_t
start, uint64_t count);
* @offset and @bytes appropriately. Otherwise returns false and leaves @offset
* and @bytes unchanged.
*/
-bool hbitmap_next_dirty_area(const HBitmap *hb, uint64_t *start,
- uint64_t *count);
+bool hbitmap_next_dirty_area(const HBitmap *hb, int64_t *start, int64_t
*count);
/**
* hbitmap_iter_next:
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 7039e82520..af9f5411a6 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -860,14 +860,14 @@ char *bdrv_dirty_bitmap_sha256(const BdrvDirtyBitmap
*bitmap, Error **errp)
return hbitmap_sha256(bitmap->bitmap, errp);
}
-int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap *bitmap, uint64_t offset,
-uint64_t bytes)
+int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap *bitmap, int64_t offset,
+int64_t bytes)
{
return hbitmap_next_zero(bitmap->bitmap, offset, bytes);
}
bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap *bitmap,
- uint64_t *offset, uint64_t *bytes)
+ int64_t *offset, int64_t *bytes)
{
return hbitmap_next_dirty_area(bitmap->bitmap, offset, bytes);
}
diff --git a/nbd/server.c b/nbd/server.c
index 11a31094ff..3106aaf3b4 100644
--- a/nbd/server.c
+++ b/nbd/server.c
@@ -2055,7 +2055,7 @@ static unsigned int bitmap_to_extents(BdrvDirtyBitmap
*bitmap, uint64_t offset,
bool next_dirty = !dirty;
if (dirty) {
-end = bdrv_dirty_bitmap_next_zero(bitmap, begin, UINT64_MAX);
+end = bdrv_dirty_bitmap_next_zero(bitmap, begin, INT64_MAX);
} else {
bdrv_set_dirty_iter(it, begin);
end = bdrv_dirty_iter_next(it);
diff --git a/tests/test-hbitmap.c b/tests/test-hbitmap.c
index aeaa0b3f22..9d210dc18c 100644
--- a/tests/test-hbitmap.c
+++ b/tests/test-hbitmap.c
@@ -817,8 +817,8 @@ static void test_hbitmap_iter_and_reset(TestHBitmapData
*data,
}
static void test_hbitmap_next_zero
[PULL 00/10] Bitmaps patches
The following changes since commit 6e8a73e911f066527e775e04b98f31ebd19db600: Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging (2020-03-11 14:41:27 +) are available in the Git repository at: https://github.com/jnsnow/qemu.git tags/bitmaps-pull-request for you to fetch changes up to 34b456d485a4df3a88116fb5ef0c418f2f12990d: block/qcow2-bitmap: use bdrv_dirty_bitmap_next_dirty (2020-03-12 16:36:46 -0400) Pull request Vladimir Sementsov-Ogievskiy (10): hbitmap: assert that we don't create bitmap larger than INT64_MAX hbitmap: move hbitmap_iter_next_word to hbitmap.c hbitmap: unpublish hbitmap_iter_skip_words hbitmap: drop meta bitmaps as they are unused block/dirty-bitmap: switch _next_dirty_area and _next_zero to int64_t block/dirty-bitmap: add _next_dirty API block/dirty-bitmap: improve _next_dirty_area API nbd/server: introduce NBDExtentArray nbd/server: use bdrv_dirty_bitmap_next_dirty_area block/qcow2-bitmap: use bdrv_dirty_bitmap_next_dirty include/block/dirty-bitmap.h | 9 +- include/qemu/hbitmap.h | 95 +++ block/dirty-bitmap.c | 16 +- block/qcow2-bitmap.c | 15 +- nbd/server.c | 251 ++-- tests/test-hbitmap.c | 316 +-- util/hbitmap.c | 134 +-- 7 files changed, 375 insertions(+), 461 deletions(-) -- 2.21.1
Re: [libvirt] Offline manipulation of Dirty Bitmaps by qemu-img
On 12/6/19 5:37 AM, Vladimir Sementsov-Ogievskiy wrote: > 06.12.2019 1:37, John Snow wrote: >> This has come up in the past, and I believe we discussed this at KVM >> Forum, too: >> >> There have been requests from oVirt (via Nir Soffer) to add some offline >> bitmap manipulation functionality. In the past, our stance has generally >> been "Use QEMU without an accelerator, and use QMP to manipulate the >> images." >> >> We like this for a few reasons: >> >> 1. It keeps bitmap management code tightly centralized >> 2. It allows for the full suite of bitmap manipulations in either >> offline or online mode with one tool >> 3. We wouldn't have to write new code. >> 4. Or design new CLIs and duplicate our existing work. >> 5. Or write even more tests. >> >> However, tools like oVirt may or may not be fully equipped to launch >> QEMU in this context, and there is always a desire for qemu-img to be >> able to "do more", so existing management suites could extend >> functionality more easily. > > I think, all guys, who don't want to use Qemu directly for image > manipulations, > should hope for Kevin's "[RFC PATCH 00/18] Add qemu-storage-daemon", which is > the correct solution of their problem. Still, I'm not one of these guys. > >> >> (Or so I am imagining.) >> >> I am still leaning heavily against adding any more CLI commands or >> options to qemu-img right now. Even if we do add some of the fundamental >> ones like "add" or "remove", it seems only a matter of time before we >> have to add "clear", "merge", etc. Is this just a race to code duplication? >> >> On the other hand, one of the other suggestions is to have qemu-img >> check --repair optionally delete corrupted bitmaps. I kind of like this >> idea: it's a buyer beware operation that might make management layers >> unhappy, but then again ... repair is always something that could make >> things worse. >> >> Plus, if you manage to corrupt bitmaps badly enough that they can't even >> be parsed, you might need a heavyweight repair operation. >> > > Improving "check" is a correct thing, because it's done inside qcow2 driver > itself. We just don't have corresponding qmp command or command line option > for Qemu to use this thing (or I missed it). > OK, I agree. I made a redhat BZ to track that we want this: 1780416 - RFE: qemu-img check --repair should optionally remove any corrupted bitmaps I'll work on a patch and we can debate the details there. --js -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH for-4.2?] block/qcow2-bitmap: fix crash bug in qcow2_co_remove_persistent_dirty_bitmap
On 12/6/19 9:36 AM, Eric Blake wrote: > [adding in Peter Maydell, as there is now potential talk of other > 4.2-worthy patches] > > On 12/6/19 4:18 AM, Vladimir Sementsov-Ogievskiy wrote: >> 05.12.2019 23:16, John Snow wrote: >>> >>> >>> On 12/5/19 3:09 PM, Eric Blake wrote: >>>> On 12/5/19 1:30 PM, Vladimir Sementsov-Ogievskiy wrote: >>>>> Here is double bug: >>>>> >>>>> First, return error but not set errp. This may lead to: >>>>> qmp block-dirty-bitmap-remove may report success when actually failed >>>>> >>>>> block-dirty-bitmap-remove used in a transaction will crash, as >>>>> qmp_transaction will think that it returned success and will cal >>>> >>>> call >>>> >>>>> block_dirty_bitmap_remove_commit which will crash, as state->bitmap is >>>>> NULL >>>>> >>>>> Second (like in anecdote), this case is not an error at all. As it is >>>>> documented in the comment above bdrv_co_remove_persistent_dirty_bitmap >>>>> definition, absence of bitmap is not an error, and similar case >>>>> handled >>>>> at start of qcow2_co_remove_persistent_dirty_bitmap, it returns 0 when >>>>> there is no bitmaps at all.. >>>> >>>> double . >>>> >>>>> >>>>> But when there are some bitmaps, but not the requested one, it return >>>>> error with errp unset. >>>>> >>>>> Fix that. >>>>> >>>>> Fixes: b56a1e31759b750 >>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy >>>>> --- >>>>> >>>>> Hi all! >>>>> >>>>> Ohm, suddenly we faced this bug. It's a regression in 4.2. I'm very >>>>> sorry for introducing it, and it sad that it's found so late.. >>>>> >>>>> Personally, I think that this one worth rc5, as it makes new bitmap >>>>> interfaces unusable. But the decision is yours. >>>>> >>>>> Last minute edit: hmm, actually, transaction action introduced in >>>>> 4.2, so crash is not a regression, only broken >>>>> block-dirty-bitmap-remove >>>>> command is a regression... Maybe it's OK for stable. >>>> >>>> Libvirt REALLY wants to use transaction bitmap management (and require >>>> qemu 4.2) for its incremental backup patches that Peter is almost ready >>>> to merge in. I'm trying to ascertain: >>>> >>>> When exactly can this bug hit? Can you give a sequence of QMP commands >>>> that will trigger it for easy reproduction? Are there workarounds >>>> (such >>>> as checking that a bitmap exists prior to attempting to remove it)? If >>>> it does NOT get fixed in rc5, how will libvirt be able to probe whether >>>> the fix is in place? >>>> >>> >>> It looks like: >>> >>> - You need to have at least one bitmap >>> - You need to use transactional remove >>> - you need to misspell the bitmap name >>> - The remove will fail (return -EINVAL) but doesn't set errp >>> - The caller chokes on incomplete information, state->bitmap is NULL >> >> >> No, that would be too simple, the thing is worse. Absolutele correct >> removing is broken, without any misspelling >> >> Bug triggers when we are removing persistent bitmap that is not stored >> yet in the image AND at least one (another) bitmap already stored in >> the image. So, something like: >> >> 1. create persistent bitmap A >> 2. shutdown vm (bitmap A is synced) >> 3. start vm >> 4. create persistent bitmap B >> 5. remove bitmap B - it fails (and crashes if in transaction) >> >> >> >> Hmm, workaround.. >> >> I'm afraid that the only thing is not remove persistent bitmaps, which >> were never synced to the image. So, instead the sequence above, we need >> >> >> 1. create persistent bitmap A >> 2. shutdown vm >> 3. start vm >> 4. create persistent bitmap B >> 5. remember, that we want to remove bitmap B after vm shutdown >> ... >>   some other operations >> ... >> 6. vm shutdown >> 7. start vm in stopped mode, and remove all bitmaps marked for removing >> 8. stop vm >> >> But, I think that in real circumstances, vm shutdown is rare thing... > > This is sounding a bit more serious. As I said earlier, it shouldn't > delay 4.2 on its own, but if the fix is obvious (and other than > comments, it is a single change from 'ret = -EINVAL' to 'ret = 0' which > fixes a definite reproducible crash), I think it rises to the level of > acceptable. > > I've been so worried about the question of which release, that I don't > know if I've previously offered: > Reviewed-by: Eric Blake > Oh, that is quite a bit more serious than I thought too. Yeah, I want this in 4.2 if at all possible. Reviewed-by: John Snow -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] Offline manipulation of Dirty Bitmaps by qemu-img
This has come up in the past, and I believe we discussed this at KVM Forum, too: There have been requests from oVirt (via Nir Soffer) to add some offline bitmap manipulation functionality. In the past, our stance has generally been "Use QEMU without an accelerator, and use QMP to manipulate the images." We like this for a few reasons: 1. It keeps bitmap management code tightly centralized 2. It allows for the full suite of bitmap manipulations in either offline or online mode with one tool 3. We wouldn't have to write new code. 4. Or design new CLIs and duplicate our existing work. 5. Or write even more tests. However, tools like oVirt may or may not be fully equipped to launch QEMU in this context, and there is always a desire for qemu-img to be able to "do more", so existing management suites could extend functionality more easily. (Or so I am imagining.) I am still leaning heavily against adding any more CLI commands or options to qemu-img right now. Even if we do add some of the fundamental ones like "add" or "remove", it seems only a matter of time before we have to add "clear", "merge", etc. Is this just a race to code duplication? On the other hand, one of the other suggestions is to have qemu-img check --repair optionally delete corrupted bitmaps. I kind of like this idea: it's a buyer beware operation that might make management layers unhappy, but then again ... repair is always something that could make things worse. Plus, if you manage to corrupt bitmaps badly enough that they can't even be parsed, you might need a heavyweight repair operation. Nir, do you think that'd be sufficient for your needs for now, or would you still like to see more granular offline management? --js -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH for-4.2?] block/qcow2-bitmap: fix crash bug in qcow2_co_remove_persistent_dirty_bitmap
On 12/5/19 4:53 PM, Eric Blake wrote: > On 12/5/19 2:16 PM, John Snow wrote: > >>>> Last minute edit: hmm, actually, transaction action introduced in >>>> 4.2, so crash is not a regression, only broken >>>> block-dirty-bitmap-remove >>>> command is a regression... Maybe it's OK for stable. >>> >>> Libvirt REALLY wants to use transaction bitmap management (and require >>> qemu 4.2) for its incremental backup patches that Peter is almost ready >>> to merge in. I'm trying to ascertain: >>> >>> When exactly can this bug hit? Can you give a sequence of QMP commands >>> that will trigger it for easy reproduction? Are there workarounds (such >>> as checking that a bitmap exists prior to attempting to remove it)? If >>> it does NOT get fixed in rc5, how will libvirt be able to probe whether >>> the fix is in place? >>> >> >> It looks like: >> >> - You need to have at least one bitmap >> - You need to use transactional remove >> - you need to misspell the bitmap name >> - The remove will fail (return -EINVAL) but doesn't set errp >> - The caller chokes on incomplete information, state->bitmap is NULL > > So in libvirt's case, as long as libvirt manages bitmaps completely, > it's a bug in libvirt to request deletion of a bitmap that doesn't > exist. Or, someone messes with a qcow2 image of an offline guest behind > libvirt's back without updating libvirt's metadata of what bitmaps > should exist, and then if libvirt fails to check that a bitmap actually > exists, a user may be able to coerce libvirt into requesting a bitmap > deletion that will cause a qemu crash, but that's the user's fault for > going behind libvirt's back. Or, libvirt could add code that instead of > trying to blindly delete a bitmap, it first makes a QMP call to ensure > the bitmap still exists (annoying, but harmless even when the bug is > fixed), instead of blaming the bug on the user operating behind > libvirt's back. > > The bug is nasty, but feels to be enough of a corner case that I think > deferring to 5.0 with CC: stable (and then downstreams can backport it > at will) is the right approach; no need to hold up 4.2 if this is the > only flaw. But I'm also not opposed to it going in 4.2 if we have > anything else serious. > Further, the NASTIEST problem is with transactional remove, which is new to 4.2. Normal remove is also broken, but won't choke because it doesn't hold undo information. Vladimir, do you agree with this assessment? Do we have it correct? --js -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH for-4.2?] block/qcow2-bitmap: fix crash bug in qcow2_co_remove_persistent_dirty_bitmap
On 12/5/19 3:09 PM, Eric Blake wrote:
> On 12/5/19 1:30 PM, Vladimir Sementsov-Ogievskiy wrote:
>> Here is double bug:
>>
>> First, return error but not set errp. This may lead to:
>> qmp block-dirty-bitmap-remove may report success when actually failed
>>
>> block-dirty-bitmap-remove used in a transaction will crash, as
>> qmp_transaction will think that it returned success and will cal
>
> call
>
>> block_dirty_bitmap_remove_commit which will crash, as state->bitmap is
>> NULL
>>
>> Second (like in anecdote), this case is not an error at all. As it is
>> documented in the comment above bdrv_co_remove_persistent_dirty_bitmap
>> definition, absence of bitmap is not an error, and similar case handled
>> at start of qcow2_co_remove_persistent_dirty_bitmap, it returns 0 when
>> there is no bitmaps at all..
>
> double .
>
>>
>> But when there are some bitmaps, but not the requested one, it return
>> error with errp unset.
>>
>> Fix that.
>>
>> Fixes: b56a1e31759b750
>> Signed-off-by: Vladimir Sementsov-Ogievskiy
>> ---
>>
>> Hi all!
>>
>> Ohm, suddenly we faced this bug. It's a regression in 4.2. I'm very
>> sorry for introducing it, and it sad that it's found so late..
>>
>> Personally, I think that this one worth rc5, as it makes new bitmap
>> interfaces unusable. But the decision is yours.
>>
>> Last minute edit: hmm, actually, transaction action introduced in
>> 4.2, so crash is not a regression, only broken block-dirty-bitmap-remove
>> command is a regression... Maybe it's OK for stable.
>
> Libvirt REALLY wants to use transaction bitmap management (and require
> qemu 4.2) for its incremental backup patches that Peter is almost ready
> to merge in. I'm trying to ascertain:
>
> When exactly can this bug hit? Can you give a sequence of QMP commands
> that will trigger it for easy reproduction? Are there workarounds (such
> as checking that a bitmap exists prior to attempting to remove it)? If
> it does NOT get fixed in rc5, how will libvirt be able to probe whether
> the fix is in place?
>
It looks like:
- You need to have at least one bitmap
- You need to use transactional remove
- you need to misspell the bitmap name
- The remove will fail (return -EINVAL) but doesn't set errp
- The caller chokes on incomplete information, state->bitmap is NULL
>>
>> Â block/qcow2-bitmap.c | 9 ++---
>> Â 1 file changed, 6 insertions(+), 3 deletions(-)
>>
>> diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
>> index 8abaf632fc..c6c8ebbe89 100644
>> --- a/block/qcow2-bitmap.c
>> +++ b/block/qcow2-bitmap.c
>> @@ -1469,8 +1469,10 @@ int coroutine_fn
>> qcow2_co_remove_persistent_dirty_bitmap(BlockDriverState *bs,
>> Â Qcow2BitmapList *bm_list;
>> Â Â if (s->nb_bitmaps == 0) {
>> -Â Â Â /* Absence of the bitmap is not an error: see explanation above
>> - * bdrv_remove_persistent_dirty_bitmap() definition. */
>> +Â Â Â /*
>> + * Absence of the bitmap is not an error: see explanation above
>> + * bdrv_co_remove_persistent_dirty_bitmap() definition.
>> + */
>> Â return 0;
>> Â }
>> Â @@ -1485,7 +1487,8 @@ int coroutine_fn
>> qcow2_co_remove_persistent_dirty_bitmap(BlockDriverState *bs,
>> Â Â bm = find_bitmap_by_name(bm_list, name);
>> Â if (bm == NULL) {
>> -Â Â Â ret = -EINVAL;
>> +Â Â Â /* Absence of the bitmap is not an error, see above. */
>> +Â Â Â ret = 0;
>> Â goto out;
>> Â }
>> Â
>
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL v2 9/9] hd-geo-test: Add tests for lchs override
From: Sam Eiderman
Add QTest tests to check the logical geometry override option.
The tests in hd-geo-test are out of date - they only test IDE and do not
test interesting MBRs.
Creating qcow2 disks with specific size and MBR layout is currently
unused - we only use a default empty MBR.
Reviewed-by: Karl Heubaum
Reviewed-by: Arbel Moshe
Signed-off-by: Sam Eiderman
Signed-off-by: Sam Eiderman
Signed-off-by: John Snow
---
tests/hd-geo-test.c| 551 +
tests/Makefile.include | 2 +-
2 files changed, 552 insertions(+), 1 deletion(-)
diff --git a/tests/hd-geo-test.c b/tests/hd-geo-test.c
index 62eb624726..7e86c5416c 100644
--- a/tests/hd-geo-test.c
+++ b/tests/hd-geo-test.c
@@ -17,7 +17,12 @@
#include "qemu/osdep.h"
#include "qemu-common.h"
+#include "qemu/bswap.h"
+#include "qapi/qmp/qlist.h"
#include "libqtest.h"
+#include "libqos/fw_cfg.h"
+#include "libqos/libqos.h"
+#include "standard-headers/linux/qemu_fw_cfg.h"
#define ARGV_SIZE 256
@@ -388,6 +393,537 @@ static void test_ide_drive_cd_0(void)
qtest_quit(qts);
}
+typedef struct {
+bool active;
+uint32_t head;
+uint32_t sector;
+uint32_t cyl;
+uint32_t end_head;
+uint32_t end_sector;
+uint32_t end_cyl;
+uint32_t start_sect;
+uint32_t nr_sects;
+} MBRpartitions[4];
+
+static MBRpartitions empty_mbr = { {false, 0, 0, 0, 0, 0, 0, 0, 0},
+ {false, 0, 0, 0, 0, 0, 0, 0, 0},
+ {false, 0, 0, 0, 0, 0, 0, 0, 0},
+ {false, 0, 0, 0, 0, 0, 0, 0, 0} };
+
+static char *create_qcow2_with_mbr(MBRpartitions mbr, uint64_t sectors)
+{
+const char *template = "/tmp/qtest.XX";
+char *raw_path = strdup(template);
+char *qcow2_path = strdup(template);
+char cmd[100 + 2 * PATH_MAX];
+uint8_t buf[512];
+int i, ret, fd, offset;
+uint64_t qcow2_size = sectors * 512;
+uint8_t status, parttype, head, sector, cyl;
+char *qemu_img_path;
+char *qemu_img_abs_path;
+
+offset = 0xbe;
+
+for (i = 0; i < 4; i++) {
+status = mbr[i].active ? 0x80 : 0x00;
+g_assert(mbr[i].head < 256);
+g_assert(mbr[i].sector < 64);
+g_assert(mbr[i].cyl < 1024);
+head = mbr[i].head;
+sector = mbr[i].sector + ((mbr[i].cyl & 0x300) >> 2);
+cyl = mbr[i].cyl & 0xff;
+
+buf[offset + 0x0] = status;
+buf[offset + 0x1] = head;
+buf[offset + 0x2] = sector;
+buf[offset + 0x3] = cyl;
+
+parttype = 0;
+g_assert(mbr[i].end_head < 256);
+g_assert(mbr[i].end_sector < 64);
+g_assert(mbr[i].end_cyl < 1024);
+head = mbr[i].end_head;
+sector = mbr[i].end_sector + ((mbr[i].end_cyl & 0x300) >> 2);
+cyl = mbr[i].end_cyl & 0xff;
+
+buf[offset + 0x4] = parttype;
+buf[offset + 0x5] = head;
+buf[offset + 0x6] = sector;
+buf[offset + 0x7] = cyl;
+
+(*(uint32_t *)&buf[offset + 0x8]) = cpu_to_le32(mbr[i].start_sect);
+(*(uint32_t *)&buf[offset + 0xc]) = cpu_to_le32(mbr[i].nr_sects);
+
+offset += 0x10;
+}
+
+fd = mkstemp(raw_path);
+g_assert(fd);
+close(fd);
+
+fd = open(raw_path, O_WRONLY);
+g_assert(fd >= 0);
+ret = write(fd, buf, sizeof(buf));
+g_assert(ret == sizeof(buf));
+close(fd);
+
+fd = mkstemp(qcow2_path);
+g_assert(fd);
+close(fd);
+
+qemu_img_path = getenv("QTEST_QEMU_IMG");
+g_assert(qemu_img_path);
+qemu_img_abs_path = realpath(qemu_img_path, NULL);
+g_assert(qemu_img_abs_path);
+
+ret = snprintf(cmd, sizeof(cmd),
+ "%s convert -f raw -O qcow2 %s %s > /dev/null",
+ qemu_img_abs_path,
+ raw_path, qcow2_path);
+g_assert((0 < ret) && (ret <= sizeof(cmd)));
+ret = system(cmd);
+g_assert(ret == 0);
+
+ret = snprintf(cmd, sizeof(cmd),
+ "%s resize %s %" PRIu64 " > /dev/null",
+ qemu_img_abs_path,
+ qcow2_path, qcow2_size);
+g_assert((0 < ret) && (ret <= sizeof(cmd)));
+ret = system(cmd);
+g_assert(ret == 0);
+
+free(qemu_img_abs_path);
+
+unlink(raw_path);
+free(raw_path);
+
+return qcow2_path;
+}
+
+#define BIOS_GEOMETRY_MAX_SIZE 1
+
+typedef struct {
+uint32_t c;
+uint32_t h;
+uint32_t s;
+} CHS;
+
+typedef struct {
+const char *dev_path;
+CHS chs;
+} CHSResult;
+
+static void read_bootdevices(QFWCFG *fw_cfg, CHSResult expected[])
+{
+char *buf = g_malloc0(BIOS_GEOMETRY_MAX_SIZE);
+char *cur;
+GList *results = NULL, *cur_result;
+CHSResult *r;
+int i;
+int res;
+bool found;
+
+
[libvirt] [PULL v2 8/9] bootdevice: FW_CFG interface for LCHS values
From: Sam Eiderman
Using fw_cfg, supply logical CHS values directly from QEMU to the BIOS.
Non-standard logical geometries break under QEMU.
A virtual disk which contains an operating system which depends on
logical geometries (consistent values being reported from BIOS INT13
AH=08) will most likely break under QEMU/SeaBIOS if it has non-standard
logical geometries - for example 56 SPT (sectors per track).
No matter what QEMU will report - SeaBIOS, for large enough disks - will
use LBA translation, which will report 63 SPT instead.
In addition we cannot force SeaBIOS to rely on physical geometries at
all. A virtio-blk-pci virtual disk with 255 phyiscal heads cannot
report more than 16 physical heads when moved to an IDE controller,
since the ATA spec allows a maximum of 16 heads - this is an artifact of
virtualization.
By supplying the logical geometries directly we are able to support such
"exotic" disks.
We serialize this information in a similar way to the "bootorder"
interface.
The new fw_cfg entry is "bios-geometry".
Reviewed-by: Karl Heubaum
Reviewed-by: Arbel Moshe
Reviewed-by: Philippe Mathieu-Daudé
Signed-off-by: Sam Eiderman
Signed-off-by: Sam Eiderman
Tested-by: Philippe Mathieu-Daudé
Signed-off-by: John Snow
---
include/sysemu/sysemu.h | 1 +
bootdevice.c| 31 +++
hw/nvram/fw_cfg.c | 14 +++---
3 files changed, 43 insertions(+), 3 deletions(-)
diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
index 5bc5c79cbc..80c57fdc4e 100644
--- a/include/sysemu/sysemu.h
+++ b/include/sysemu/sysemu.h
@@ -106,6 +106,7 @@ void validate_bootdevices(const char *devices, Error
**errp);
void add_boot_device_lchs(DeviceState *dev, const char *suffix,
uint32_t lcyls, uint32_t lheads, uint32_t lsecs);
void del_boot_device_lchs(DeviceState *dev, const char *suffix);
+char *get_boot_devices_lchs_list(size_t *size);
/* handler to set the boot_device order for a specific type of MachineClass */
typedef void QEMUBootSetHandler(void *opaque, const char *boot_order,
diff --git a/bootdevice.c b/bootdevice.c
index 2cf6b37c57..03aaffcc8d 100644
--- a/bootdevice.c
+++ b/bootdevice.c
@@ -405,3 +405,34 @@ void del_boot_device_lchs(DeviceState *dev, const char
*suffix)
}
}
}
+
+char *get_boot_devices_lchs_list(size_t *size)
+{
+FWLCHSEntry *i;
+size_t total = 0;
+char *list = NULL;
+
+QTAILQ_FOREACH(i, &fw_lchs, link) {
+char *bootpath;
+char *chs_string;
+size_t len;
+
+bootpath = get_boot_device_path(i->dev, false, i->suffix);
+chs_string = g_strdup_printf("%s %" PRIu32 " %" PRIu32 " %" PRIu32,
+ bootpath, i->lcyls, i->lheads, i->lsecs);
+
+if (total) {
+list[total - 1] = '\n';
+}
+len = strlen(chs_string) + 1;
+list = g_realloc(list, total + len);
+memcpy(&list[total], chs_string, len);
+total += len;
+g_free(chs_string);
+g_free(bootpath);
+}
+
+*size = total;
+
+return list;
+}
diff --git a/hw/nvram/fw_cfg.c b/hw/nvram/fw_cfg.c
index aef1727250..44a3c19326 100644
--- a/hw/nvram/fw_cfg.c
+++ b/hw/nvram/fw_cfg.c
@@ -949,13 +949,21 @@ void *fw_cfg_modify_file(FWCfgState *s, const char
*filename,
static void fw_cfg_machine_reset(void *opaque)
{
+MachineClass *mc = MACHINE_GET_CLASS(qdev_get_machine());
+FWCfgState *s = opaque;
void *ptr;
size_t len;
-FWCfgState *s = opaque;
-char *bootindex = get_boot_devices_list(&len);
+char *buf;
-ptr = fw_cfg_modify_file(s, "bootorder", (uint8_t *)bootindex, len);
+buf = get_boot_devices_list(&len);
+ptr = fw_cfg_modify_file(s, "bootorder", (uint8_t *)buf, len);
g_free(ptr);
+
+if (!mc->legacy_fw_cfg_order) {
+buf = get_boot_devices_lchs_list(&len);
+ptr = fw_cfg_modify_file(s, "bios-geometry", (uint8_t *)buf, len);
+g_free(ptr);
+}
}
static void fw_cfg_machine_ready(struct Notifier *n, void *data)
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL v2 6/9] bootdevice: Gather LCHS from all relevant devices
From: Sam Eiderman
Relevant devices are:
* ide-hd (and ide-cd, ide-drive)
* scsi-hd (and scsi-cd, scsi-disk, scsi-block)
* virtio-blk-pci
We do not call del_boot_device_lchs() for ide-* since we don't need to -
IDE block devices do not support unplugging.
Reviewed-by: Karl Heubaum
Reviewed-by: Arbel Moshe
Signed-off-by: Sam Eiderman
Signed-off-by: Sam Eiderman
Reviewed-by: Philippe Mathieu-Daudé
Tested-by: Philippe Mathieu-Daudé
Signed-off-by: John Snow
---
hw/block/virtio-blk.c | 6 ++
hw/ide/qdev.c | 5 +
hw/scsi/scsi-disk.c | 12
3 files changed, 23 insertions(+)
diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
index 9fa2eaf890..4c357d2928 100644
--- a/hw/block/virtio-blk.c
+++ b/hw/block/virtio-blk.c
@@ -1200,6 +1200,11 @@ static void virtio_blk_device_realize(DeviceState *dev,
Error **errp)
blk_set_guest_block_size(s->blk, s->conf.conf.logical_block_size);
blk_iostatus_enable(s->blk);
+
+add_boot_device_lchs(dev, "/disk@0,0",
+ conf->conf.lcyls,
+ conf->conf.lheads,
+ conf->conf.lsecs);
}
static void virtio_blk_device_unrealize(DeviceState *dev, Error **errp)
@@ -1210,6 +1215,7 @@ static void virtio_blk_device_unrealize(DeviceState *dev,
Error **errp)
unsigned i;
blk_drain(s->blk);
+del_boot_device_lchs(dev, "/disk@0,0");
virtio_blk_data_plane_destroy(s->dataplane);
s->dataplane = NULL;
for (i = 0; i < conf->num_queues; i++) {
diff --git a/hw/ide/qdev.c b/hw/ide/qdev.c
index 85cca6ec38..374a791a45 100644
--- a/hw/ide/qdev.c
+++ b/hw/ide/qdev.c
@@ -220,6 +220,11 @@ static void ide_dev_initfn(IDEDevice *dev, IDEDriveKind
kind, Error **errp)
add_boot_device_path(dev->conf.bootindex, &dev->qdev,
dev->unit ? "/disk@1" : "/disk@0");
+
+add_boot_device_lchs(&dev->qdev, dev->unit ? "/disk@1" : "/disk@0",
+ dev->conf.lcyls,
+ dev->conf.lheads,
+ dev->conf.lsecs);
}
static void ide_dev_get_bootindex(Object *obj, Visitor *v, const char *name,
diff --git a/hw/scsi/scsi-disk.c b/hw/scsi/scsi-disk.c
index 68b1675fd9..07fb5ebdf1 100644
--- a/hw/scsi/scsi-disk.c
+++ b/hw/scsi/scsi-disk.c
@@ -35,6 +35,7 @@
#include "hw/block/block.h"
#include "hw/qdev-properties.h"
#include "sysemu/dma.h"
+#include "sysemu/sysemu.h"
#include "qemu/cutils.h"
#include "trace.h"
@@ -2414,6 +2415,16 @@ static void scsi_realize(SCSIDevice *dev, Error **errp)
blk_set_guest_block_size(s->qdev.conf.blk, s->qdev.blocksize);
blk_iostatus_enable(s->qdev.conf.blk);
+
+add_boot_device_lchs(&dev->qdev, NULL,
+ dev->conf.lcyls,
+ dev->conf.lheads,
+ dev->conf.lsecs);
+}
+
+static void scsi_unrealize(SCSIDevice *dev, Error **errp)
+{
+del_boot_device_lchs(&dev->qdev, NULL);
}
static void scsi_hd_realize(SCSIDevice *dev, Error **errp)
@@ -3018,6 +3029,7 @@ static void scsi_hd_class_initfn(ObjectClass *klass, void
*data)
SCSIDeviceClass *sc = SCSI_DEVICE_CLASS(klass);
sc->realize = scsi_hd_realize;
+sc->unrealize= scsi_unrealize;
sc->alloc_req= scsi_new_request;
sc->unit_attention_reported = scsi_disk_unit_attention_reported;
dc->desc = "virtual SCSI disk";
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL v2 7/9] bootdevice: Refactor get_boot_devices_list
From: Sam Eiderman
Move device name construction to a separate function.
We will reuse this function in the following commit to pass logical CHS
parameters through fw_cfg much like we currently pass bootindex.
Reviewed-by: Karl Heubaum
Reviewed-by: Arbel Moshe
Reviewed-by: Philippe Mathieu-Daudé
Signed-off-by: Sam Eiderman
Signed-off-by: Sam Eiderman
Tested-by: Philippe Mathieu-Daudé
Signed-off-by: John Snow
---
bootdevice.c | 61 +---
1 file changed, 34 insertions(+), 27 deletions(-)
diff --git a/bootdevice.c b/bootdevice.c
index bc5e1c2de4..2cf6b37c57 100644
--- a/bootdevice.c
+++ b/bootdevice.c
@@ -202,6 +202,39 @@ DeviceState *get_boot_device(uint32_t position)
return res;
}
+static char *get_boot_device_path(DeviceState *dev, bool ignore_suffixes,
+ const char *suffix)
+{
+char *devpath = NULL, *s = NULL, *d, *bootpath;
+
+if (dev) {
+devpath = qdev_get_fw_dev_path(dev);
+assert(devpath);
+}
+
+if (!ignore_suffixes) {
+if (dev) {
+d = qdev_get_own_fw_dev_path_from_handler(dev->parent_bus, dev);
+if (d) {
+assert(!suffix);
+s = d;
+} else {
+s = g_strdup(suffix);
+}
+} else {
+s = g_strdup(suffix);
+}
+}
+
+bootpath = g_strdup_printf("%s%s",
+ devpath ? devpath : "",
+ s ? s : "");
+g_free(devpath);
+g_free(s);
+
+return bootpath;
+}
+
/*
* This function returns null terminated string that consist of new line
* separated device paths.
@@ -218,36 +251,10 @@ char *get_boot_devices_list(size_t *size)
bool ignore_suffixes = mc->ignore_boot_device_suffixes;
QTAILQ_FOREACH(i, &fw_boot_order, link) {
-char *devpath = NULL, *suffix = NULL;
char *bootpath;
-char *d;
size_t len;
-if (i->dev) {
-devpath = qdev_get_fw_dev_path(i->dev);
-assert(devpath);
-}
-
-if (!ignore_suffixes) {
-if (i->dev) {
-d = qdev_get_own_fw_dev_path_from_handler(i->dev->parent_bus,
- i->dev);
-if (d) {
-assert(!i->suffix);
-suffix = d;
-} else {
-suffix = g_strdup(i->suffix);
-}
-} else {
-suffix = g_strdup(i->suffix);
-}
-}
-
-bootpath = g_strdup_printf("%s%s",
- devpath ? devpath : "",
- suffix ? suffix : "");
-g_free(devpath);
-g_free(suffix);
+bootpath = get_boot_device_path(i->dev, ignore_suffixes, i->suffix);
if (total) {
list[total-1] = '\n';
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL v2 5/9] scsi: Propagate unrealize() callback to scsi-hd
From: Sam Eiderman
We will need to add LCHS removal logic to scsi-hd's unrealize() in the
next commit.
Reviewed-by: Karl Heubaum
Reviewed-by: Arbel Moshe
Reviewed-by: Philippe Mathieu-Daudé
Signed-off-by: Sam Eiderman
Signed-off-by: Sam Eiderman
Tested-by: Philippe Mathieu-Daudé
Signed-off-by: John Snow
---
include/hw/scsi/scsi.h | 1 +
hw/scsi/scsi-bus.c | 16
2 files changed, 17 insertions(+)
diff --git a/include/hw/scsi/scsi.h b/include/hw/scsi/scsi.h
index d77a92361b..332ef602f4 100644
--- a/include/hw/scsi/scsi.h
+++ b/include/hw/scsi/scsi.h
@@ -59,6 +59,7 @@ struct SCSIRequest {
typedef struct SCSIDeviceClass {
DeviceClass parent_class;
void (*realize)(SCSIDevice *dev, Error **errp);
+void (*unrealize)(SCSIDevice *dev, Error **errp);
int (*parse_cdb)(SCSIDevice *dev, SCSICommand *cmd, uint8_t *buf,
void *hba_private);
SCSIRequest *(*alloc_req)(SCSIDevice *s, uint32_t tag, uint32_t lun,
diff --git a/hw/scsi/scsi-bus.c b/hw/scsi/scsi-bus.c
index bccb7cc4c6..359d50d6d0 100644
--- a/hw/scsi/scsi-bus.c
+++ b/hw/scsi/scsi-bus.c
@@ -59,6 +59,14 @@ static void scsi_device_realize(SCSIDevice *s, Error **errp)
}
}
+static void scsi_device_unrealize(SCSIDevice *s, Error **errp)
+{
+SCSIDeviceClass *sc = SCSI_DEVICE_GET_CLASS(s);
+if (sc->unrealize) {
+sc->unrealize(s, errp);
+}
+}
+
int scsi_bus_parse_cdb(SCSIDevice *dev, SCSICommand *cmd, uint8_t *buf,
void *hba_private)
{
@@ -217,12 +225,20 @@ static void scsi_qdev_realize(DeviceState *qdev, Error
**errp)
static void scsi_qdev_unrealize(DeviceState *qdev, Error **errp)
{
SCSIDevice *dev = SCSI_DEVICE(qdev);
+Error *local_err = NULL;
if (dev->vmsentry) {
qemu_del_vm_change_state_handler(dev->vmsentry);
}
scsi_device_purge_requests(dev, SENSE_CODE(NO_SENSE));
+
+scsi_device_unrealize(dev, &local_err);
+if (local_err) {
+error_propagate(errp, local_err);
+return;
+}
+
blockdev_mark_auto_del(dev->conf.blk);
}
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL v2 3/9] block: Support providing LCHS from user
From: Sam Eiderman
Add logical geometry variables to BlockConf.
A user can now supply "lcyls", "lheads" & "lsecs" for any HD device
that supports CHS ("cyls", "heads", "secs").
These devices include:
* ide-hd
* scsi-hd
* virtio-blk-pci
In future commits we will use the provided LCHS and pass it to the BIOS
through fw_cfg to be supplied using INT13 routines.
Reviewed-by: Karl Heubaum
Reviewed-by: Arbel Moshe
Signed-off-by: Sam Eiderman
Signed-off-by: Sam Eiderman
Reviewed-by: Philippe Mathieu-Daudé
Tested-by: Philippe Mathieu-Daudé
Signed-off-by: John Snow
---
include/hw/block/block.h | 6 +-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/include/hw/block/block.h b/include/hw/block/block.h
index fd55a30bca..d7246f3862 100644
--- a/include/hw/block/block.h
+++ b/include/hw/block/block.h
@@ -26,6 +26,7 @@ typedef struct BlockConf {
uint32_t discard_granularity;
/* geometry, not all devices use this */
uint32_t cyls, heads, secs;
+uint32_t lcyls, lheads, lsecs;
OnOffAuto wce;
bool share_rw;
BlockdevOnError rerror;
@@ -65,7 +66,10 @@ static inline unsigned int get_physical_block_exp(BlockConf
*conf)
#define DEFINE_BLOCK_CHS_PROPERTIES(_state, _conf) \
DEFINE_PROP_UINT32("cyls", _state, _conf.cyls, 0), \
DEFINE_PROP_UINT32("heads", _state, _conf.heads, 0),\
-DEFINE_PROP_UINT32("secs", _state, _conf.secs, 0)
+DEFINE_PROP_UINT32("secs", _state, _conf.secs, 0), \
+DEFINE_PROP_UINT32("lcyls", _state, _conf.lcyls, 0),\
+DEFINE_PROP_UINT32("lheads", _state, _conf.lheads, 0), \
+DEFINE_PROP_UINT32("lsecs", _state, _conf.lsecs, 0)
#define DEFINE_BLOCK_ERROR_PROPERTIES(_state, _conf)\
DEFINE_PROP_BLOCKDEV_ON_ERROR("rerror", _state, _conf.rerror, \
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL v2 4/9] bootdevice: Add interface to gather LCHS
From: Sam Eiderman
Add an interface to provide direct logical CHS values for boot devices.
We will use this interface in the next commits.
Reviewed-by: Karl Heubaum
Reviewed-by: Arbel Moshe
Signed-off-by: Sam Eiderman
Signed-off-by: Sam Eiderman
Reviewed-by: Philippe Mathieu-Daudé
Tested-by: Philippe Mathieu-Daudé
Signed-off-by: John Snow
---
include/sysemu/sysemu.h | 3 +++
bootdevice.c| 55 +
2 files changed, 58 insertions(+)
diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
index 44f18eb739..5bc5c79cbc 100644
--- a/include/sysemu/sysemu.h
+++ b/include/sysemu/sysemu.h
@@ -103,6 +103,9 @@ void device_add_bootindex_property(Object *obj, int32_t
*bootindex,
DeviceState *dev, Error **errp);
void restore_boot_order(void *opaque);
void validate_bootdevices(const char *devices, Error **errp);
+void add_boot_device_lchs(DeviceState *dev, const char *suffix,
+ uint32_t lcyls, uint32_t lheads, uint32_t lsecs);
+void del_boot_device_lchs(DeviceState *dev, const char *suffix);
/* handler to set the boot_device order for a specific type of MachineClass */
typedef void QEMUBootSetHandler(void *opaque, const char *boot_order,
diff --git a/bootdevice.c b/bootdevice.c
index 1d225202f9..bc5e1c2de4 100644
--- a/bootdevice.c
+++ b/bootdevice.c
@@ -343,3 +343,58 @@ void device_add_bootindex_property(Object *obj, int32_t
*bootindex,
/* initialize devices' bootindex property to -1 */
object_property_set_int(obj, -1, name, NULL);
}
+
+typedef struct FWLCHSEntry FWLCHSEntry;
+
+struct FWLCHSEntry {
+QTAILQ_ENTRY(FWLCHSEntry) link;
+DeviceState *dev;
+char *suffix;
+uint32_t lcyls;
+uint32_t lheads;
+uint32_t lsecs;
+};
+
+static QTAILQ_HEAD(, FWLCHSEntry) fw_lchs =
+QTAILQ_HEAD_INITIALIZER(fw_lchs);
+
+void add_boot_device_lchs(DeviceState *dev, const char *suffix,
+ uint32_t lcyls, uint32_t lheads, uint32_t lsecs)
+{
+FWLCHSEntry *node;
+
+if (!lcyls && !lheads && !lsecs) {
+return;
+}
+
+assert(dev != NULL || suffix != NULL);
+
+node = g_malloc0(sizeof(FWLCHSEntry));
+node->suffix = g_strdup(suffix);
+node->dev = dev;
+node->lcyls = lcyls;
+node->lheads = lheads;
+node->lsecs = lsecs;
+
+QTAILQ_INSERT_TAIL(&fw_lchs, node, link);
+}
+
+void del_boot_device_lchs(DeviceState *dev, const char *suffix)
+{
+FWLCHSEntry *i;
+
+if (dev == NULL) {
+return;
+}
+
+QTAILQ_FOREACH(i, &fw_lchs, link) {
+if ((!suffix || !g_strcmp0(i->suffix, suffix)) &&
+ i->dev == dev) {
+QTAILQ_REMOVE(&fw_lchs, i, link);
+g_free(i->suffix);
+g_free(i);
+
+break;
+}
+}
+}
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL v2 2/9] block: Refactor macros - fix tabbing
From: Sam Eiderman
Fixing tabbing in block related macros.
Signed-off-by: Sam Eiderman
Signed-off-by: Sam Eiderman
Reviewed-by: Karl Heubaum
Reviewed-by: Arbel Moshe
Reviewed-by: Philippe Mathieu-Daudé
Signed-off-by: John Snow
---
include/hw/block/block.h | 16
hw/ide/qdev.c| 2 +-
2 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/include/hw/block/block.h b/include/hw/block/block.h
index 607539057a..fd55a30bca 100644
--- a/include/hw/block/block.h
+++ b/include/hw/block/block.h
@@ -50,21 +50,21 @@ static inline unsigned int get_physical_block_exp(BlockConf
*conf)
_conf.logical_block_size),\
DEFINE_PROP_BLOCKSIZE("physical_block_size", _state,\
_conf.physical_block_size), \
-DEFINE_PROP_UINT16("min_io_size", _state, _conf.min_io_size, 0), \
+DEFINE_PROP_UINT16("min_io_size", _state, _conf.min_io_size, 0),\
DEFINE_PROP_UINT32("opt_io_size", _state, _conf.opt_io_size, 0),\
-DEFINE_PROP_UINT32("discard_granularity", _state, \
- _conf.discard_granularity, -1), \
-DEFINE_PROP_ON_OFF_AUTO("write-cache", _state, _conf.wce, \
-ON_OFF_AUTO_AUTO), \
+DEFINE_PROP_UINT32("discard_granularity", _state, \
+ _conf.discard_granularity, -1), \
+DEFINE_PROP_ON_OFF_AUTO("write-cache", _state, _conf.wce, \
+ON_OFF_AUTO_AUTO), \
DEFINE_PROP_BOOL("share-rw", _state, _conf.share_rw, false)
#define DEFINE_BLOCK_PROPERTIES(_state, _conf) \
DEFINE_PROP_DRIVE("drive", _state, _conf.blk), \
DEFINE_BLOCK_PROPERTIES_BASE(_state, _conf)
-#define DEFINE_BLOCK_CHS_PROPERTIES(_state, _conf) \
-DEFINE_PROP_UINT32("cyls", _state, _conf.cyls, 0), \
-DEFINE_PROP_UINT32("heads", _state, _conf.heads, 0), \
+#define DEFINE_BLOCK_CHS_PROPERTIES(_state, _conf) \
+DEFINE_PROP_UINT32("cyls", _state, _conf.cyls, 0), \
+DEFINE_PROP_UINT32("heads", _state, _conf.heads, 0),\
DEFINE_PROP_UINT32("secs", _state, _conf.secs, 0)
#define DEFINE_BLOCK_ERROR_PROPERTIES(_state, _conf)\
diff --git a/hw/ide/qdev.c b/hw/ide/qdev.c
index 3666e59721..85cca6ec38 100644
--- a/hw/ide/qdev.c
+++ b/hw/ide/qdev.c
@@ -293,7 +293,7 @@ static void ide_drive_realize(IDEDevice *dev, Error **errp)
DEFINE_BLOCK_PROPERTIES(IDEDrive, dev.conf),\
DEFINE_BLOCK_ERROR_PROPERTIES(IDEDrive, dev.conf), \
DEFINE_PROP_STRING("ver", IDEDrive, dev.version), \
-DEFINE_PROP_UINT64("wwn", IDEDrive, dev.wwn, 0),\
+DEFINE_PROP_UINT64("wwn", IDEDrive, dev.wwn, 0), \
DEFINE_PROP_STRING("serial", IDEDrive, dev.serial),\
DEFINE_PROP_STRING("model", IDEDrive, dev.model)
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL v2 0/9] Ide patches
The following changes since commit 68d8ef4ec540682c3538d4963e836e43a211dd17: Merge remote-tracking branch 'remotes/stsquad/tags/pull-tcg-plugins-281019-4' into staging (2019-10-30 14:10:32 +) are available in the Git repository at: https://github.com/jnsnow/qemu.git tags/ide-pull-request for you to fetch changes up to dc237c45aee52f268369dc6a485c623f1232e1d3: hd-geo-test: Add tests for lchs override (2019-10-31 11:47:43 -0400) Pull request ---- John Snow (1): IDE: deprecate ide-drive Sam Eiderman (8): block: Refactor macros - fix tabbing block: Support providing LCHS from user bootdevice: Add interface to gather LCHS scsi: Propagate unrealize() callback to scsi-hd bootdevice: Gather LCHS from all relevant devices bootdevice: Refactor get_boot_devices_list bootdevice: FW_CFG interface for LCHS values hd-geo-test: Add tests for lchs override qemu-deprecated.texi | 5 + include/hw/block/block.h | 22 +- include/hw/scsi/scsi.h| 1 + include/sysemu/sysemu.h | 4 + bootdevice.c | 147 +++-- hw/block/virtio-blk.c | 6 + hw/ide/qdev.c | 10 +- hw/nvram/fw_cfg.c | 14 +- hw/scsi/scsi-bus.c| 16 + hw/scsi/scsi-disk.c | 12 + tests/hd-geo-test.c | 551 ++ tests/Makefile.include| 2 +- tests/qemu-iotests/051.pc.out | 6 +- 13 files changed, 753 insertions(+), 43 deletions(-) -- 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL v2 1/9] IDE: deprecate ide-drive
It's an old compatibility shim that just delegates to ide-cd or ide-hd.
I'd like to refactor these some day, and getting rid of the super-object
will make that easier.
Either way, we don't need this.
Signed-off-by: John Snow
Reviewed-by: Thomas Huth
Reviewed-by: Markus Armbruster
ACKed-by: Peter Krempa
Message-id: 20191009224303.10232-2-js...@redhat.com
Signed-off-by: John Snow
---
qemu-deprecated.texi | 5 +
hw/ide/qdev.c | 3 +++
tests/qemu-iotests/051.pc.out | 6 --
3 files changed, 12 insertions(+), 2 deletions(-)
diff --git a/qemu-deprecated.texi b/qemu-deprecated.texi
index f727bd3932..296bfc93a3 100644
--- a/qemu-deprecated.texi
+++ b/qemu-deprecated.texi
@@ -254,6 +254,11 @@ quite a bit. It will be removed without replacement unless
some users speaks
up at the @email{qemu-devel@@nongnu.org} mailing list with information about
their usecases.
+@subsection ide-drive (since 4.2)
+
+The 'ide-drive' device is deprecated. Users should use 'ide-hd' or
+'ide-cd' as appropriate to get an IDE hard disk or CD-ROM as needed.
+
@section System emulator machines
@subsection pc-0.12, pc-0.13, pc-0.14 and pc-0.15 (since 4.0)
diff --git a/hw/ide/qdev.c b/hw/ide/qdev.c
index 6fba6b62b8..3666e59721 100644
--- a/hw/ide/qdev.c
+++ b/hw/ide/qdev.c
@@ -279,6 +279,9 @@ static void ide_drive_realize(IDEDevice *dev, Error **errp)
{
DriveInfo *dinfo = NULL;
+warn_report("'ide-drive' is deprecated, "
+"please use 'ide-hd' or 'ide-cd' instead");
+
if (dev->conf.blk) {
dinfo = blk_legacy_dinfo(dev->conf.blk);
}
diff --git a/tests/qemu-iotests/051.pc.out b/tests/qemu-iotests/051.pc.out
index 000557c7c8..34849dd172 100644
--- a/tests/qemu-iotests/051.pc.out
+++ b/tests/qemu-iotests/051.pc.out
@@ -158,7 +158,8 @@ QEMU X.Y.Z monitor - type 'help' for more information
Testing: -drive if=none,id=disk -device ide-drive,drive=disk
QEMU X.Y.Z monitor - type 'help' for more information
-(qemu) QEMU_PROG: -device ide-drive,drive=disk: Device needs media, but drive
is empty
+(qemu) QEMU_PROG: -device ide-drive,drive=disk: warning: 'ide-drive' is
deprecated, please use 'ide-hd' or 'ide-cd' instead
+QEMU_PROG: -device ide-drive,drive=disk: Device needs media, but drive is empty
Testing: -drive if=none,id=disk -device ide-hd,drive=disk
QEMU X.Y.Z monitor - type 'help' for more information
@@ -228,7 +229,8 @@ QEMU X.Y.Z monitor - type 'help' for more information
Testing: -drive file=TEST_DIR/t.qcow2,if=none,id=disk,readonly=on -device
ide-drive,drive=disk
QEMU X.Y.Z monitor - type 'help' for more information
-(qemu) QEMU_PROG: -device ide-drive,drive=disk: Block node is read-only
+(qemu) QEMU_PROG: -device ide-drive,drive=disk: warning: 'ide-drive' is
deprecated, please use 'ide-hd' or 'ide-cd' instead
+QEMU_PROG: -device ide-drive,drive=disk: Block node is read-only
Testing: -drive file=TEST_DIR/t.qcow2,if=none,id=disk,readonly=on -device
ide-hd,drive=disk
QEMU X.Y.Z monitor - type 'help' for more information
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PULL 0/9] Ide patches
On 10/31/19 11:02 AM, Peter Maydell wrote: > On Thu, 31 Oct 2019 at 10:59, John Snow wrote: >> >> The following changes since commit 68d8ef4ec540682c3538d4963e836e43a211dd17: >> >> Merge remote-tracking branch >> 'remotes/stsquad/tags/pull-tcg-plugins-281019-4' into staging (2019-10-30 >> 14:10:32 +) >> >> are available in the Git repository at: >> >> https://github.com/jnsnow/qemu.git tags/ide-pull-request >> >> for you to fetch changes up to c35564caf20e8d3431786dddf0fa513daa7d7f3c: >> >> hd-geo-test: Add tests for lchs override (2019-10-31 06:11:34 -0400) >> >> >> Pull request >> > > Hi -- this passed the merge tests but it looks like you forgot > to add your signed-off by line as the submaintainer to Sam's > patches. Could you fix that up and resend, please? > > thanks > -- PMM > Haha. I re-applied them to grab Phil's SOBs and that dropped mine. OK, re-spinning. (Note to self: add a check to git-publish --pull that looks for my SOB.) -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL 9/9] hd-geo-test: Add tests for lchs override
From: Sam Eiderman
Add QTest tests to check the logical geometry override option.
The tests in hd-geo-test are out of date - they only test IDE and do not
test interesting MBRs.
Creating qcow2 disks with specific size and MBR layout is currently
unused - we only use a default empty MBR.
Reviewed-by: Karl Heubaum
Reviewed-by: Arbel Moshe
Signed-off-by: Sam Eiderman
Signed-off-by: Sam Eiderman
Signed-off-by: John Snow
---
tests/hd-geo-test.c| 551 +
tests/Makefile.include | 2 +-
2 files changed, 552 insertions(+), 1 deletion(-)
diff --git a/tests/hd-geo-test.c b/tests/hd-geo-test.c
index 62eb624726..7e86c5416c 100644
--- a/tests/hd-geo-test.c
+++ b/tests/hd-geo-test.c
@@ -17,7 +17,12 @@
#include "qemu/osdep.h"
#include "qemu-common.h"
+#include "qemu/bswap.h"
+#include "qapi/qmp/qlist.h"
#include "libqtest.h"
+#include "libqos/fw_cfg.h"
+#include "libqos/libqos.h"
+#include "standard-headers/linux/qemu_fw_cfg.h"
#define ARGV_SIZE 256
@@ -388,6 +393,537 @@ static void test_ide_drive_cd_0(void)
qtest_quit(qts);
}
+typedef struct {
+bool active;
+uint32_t head;
+uint32_t sector;
+uint32_t cyl;
+uint32_t end_head;
+uint32_t end_sector;
+uint32_t end_cyl;
+uint32_t start_sect;
+uint32_t nr_sects;
+} MBRpartitions[4];
+
+static MBRpartitions empty_mbr = { {false, 0, 0, 0, 0, 0, 0, 0, 0},
+ {false, 0, 0, 0, 0, 0, 0, 0, 0},
+ {false, 0, 0, 0, 0, 0, 0, 0, 0},
+ {false, 0, 0, 0, 0, 0, 0, 0, 0} };
+
+static char *create_qcow2_with_mbr(MBRpartitions mbr, uint64_t sectors)
+{
+const char *template = "/tmp/qtest.XX";
+char *raw_path = strdup(template);
+char *qcow2_path = strdup(template);
+char cmd[100 + 2 * PATH_MAX];
+uint8_t buf[512];
+int i, ret, fd, offset;
+uint64_t qcow2_size = sectors * 512;
+uint8_t status, parttype, head, sector, cyl;
+char *qemu_img_path;
+char *qemu_img_abs_path;
+
+offset = 0xbe;
+
+for (i = 0; i < 4; i++) {
+status = mbr[i].active ? 0x80 : 0x00;
+g_assert(mbr[i].head < 256);
+g_assert(mbr[i].sector < 64);
+g_assert(mbr[i].cyl < 1024);
+head = mbr[i].head;
+sector = mbr[i].sector + ((mbr[i].cyl & 0x300) >> 2);
+cyl = mbr[i].cyl & 0xff;
+
+buf[offset + 0x0] = status;
+buf[offset + 0x1] = head;
+buf[offset + 0x2] = sector;
+buf[offset + 0x3] = cyl;
+
+parttype = 0;
+g_assert(mbr[i].end_head < 256);
+g_assert(mbr[i].end_sector < 64);
+g_assert(mbr[i].end_cyl < 1024);
+head = mbr[i].end_head;
+sector = mbr[i].end_sector + ((mbr[i].end_cyl & 0x300) >> 2);
+cyl = mbr[i].end_cyl & 0xff;
+
+buf[offset + 0x4] = parttype;
+buf[offset + 0x5] = head;
+buf[offset + 0x6] = sector;
+buf[offset + 0x7] = cyl;
+
+(*(uint32_t *)&buf[offset + 0x8]) = cpu_to_le32(mbr[i].start_sect);
+(*(uint32_t *)&buf[offset + 0xc]) = cpu_to_le32(mbr[i].nr_sects);
+
+offset += 0x10;
+}
+
+fd = mkstemp(raw_path);
+g_assert(fd);
+close(fd);
+
+fd = open(raw_path, O_WRONLY);
+g_assert(fd >= 0);
+ret = write(fd, buf, sizeof(buf));
+g_assert(ret == sizeof(buf));
+close(fd);
+
+fd = mkstemp(qcow2_path);
+g_assert(fd);
+close(fd);
+
+qemu_img_path = getenv("QTEST_QEMU_IMG");
+g_assert(qemu_img_path);
+qemu_img_abs_path = realpath(qemu_img_path, NULL);
+g_assert(qemu_img_abs_path);
+
+ret = snprintf(cmd, sizeof(cmd),
+ "%s convert -f raw -O qcow2 %s %s > /dev/null",
+ qemu_img_abs_path,
+ raw_path, qcow2_path);
+g_assert((0 < ret) && (ret <= sizeof(cmd)));
+ret = system(cmd);
+g_assert(ret == 0);
+
+ret = snprintf(cmd, sizeof(cmd),
+ "%s resize %s %" PRIu64 " > /dev/null",
+ qemu_img_abs_path,
+ qcow2_path, qcow2_size);
+g_assert((0 < ret) && (ret <= sizeof(cmd)));
+ret = system(cmd);
+g_assert(ret == 0);
+
+free(qemu_img_abs_path);
+
+unlink(raw_path);
+free(raw_path);
+
+return qcow2_path;
+}
+
+#define BIOS_GEOMETRY_MAX_SIZE 1
+
+typedef struct {
+uint32_t c;
+uint32_t h;
+uint32_t s;
+} CHS;
+
+typedef struct {
+const char *dev_path;
+CHS chs;
+} CHSResult;
+
+static void read_bootdevices(QFWCFG *fw_cfg, CHSResult expected[])
+{
+char *buf = g_malloc0(BIOS_GEOMETRY_MAX_SIZE);
+char *cur;
+GList *results = NULL, *cur_result;
+CHSResult *r;
+int i;
+int res;
+bool found;
+
+
[libvirt] [PULL 8/9] bootdevice: FW_CFG interface for LCHS values
From: Sam Eiderman
Using fw_cfg, supply logical CHS values directly from QEMU to the BIOS.
Non-standard logical geometries break under QEMU.
A virtual disk which contains an operating system which depends on
logical geometries (consistent values being reported from BIOS INT13
AH=08) will most likely break under QEMU/SeaBIOS if it has non-standard
logical geometries - for example 56 SPT (sectors per track).
No matter what QEMU will report - SeaBIOS, for large enough disks - will
use LBA translation, which will report 63 SPT instead.
In addition we cannot force SeaBIOS to rely on physical geometries at
all. A virtio-blk-pci virtual disk with 255 phyiscal heads cannot
report more than 16 physical heads when moved to an IDE controller,
since the ATA spec allows a maximum of 16 heads - this is an artifact of
virtualization.
By supplying the logical geometries directly we are able to support such
"exotic" disks.
We serialize this information in a similar way to the "bootorder"
interface.
The new fw_cfg entry is "bios-geometry".
Reviewed-by: Karl Heubaum
Reviewed-by: Arbel Moshe
Reviewed-by: Philippe Mathieu-Daudé
Signed-off-by: Sam Eiderman
Signed-off-by: Sam Eiderman
Tested-by: Philippe Mathieu-Daudé
Signed-off-by: John Snow
---
include/sysemu/sysemu.h | 1 +
bootdevice.c| 31 +++
hw/nvram/fw_cfg.c | 14 +++---
3 files changed, 43 insertions(+), 3 deletions(-)
diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
index 5bc5c79cbc..80c57fdc4e 100644
--- a/include/sysemu/sysemu.h
+++ b/include/sysemu/sysemu.h
@@ -106,6 +106,7 @@ void validate_bootdevices(const char *devices, Error
**errp);
void add_boot_device_lchs(DeviceState *dev, const char *suffix,
uint32_t lcyls, uint32_t lheads, uint32_t lsecs);
void del_boot_device_lchs(DeviceState *dev, const char *suffix);
+char *get_boot_devices_lchs_list(size_t *size);
/* handler to set the boot_device order for a specific type of MachineClass */
typedef void QEMUBootSetHandler(void *opaque, const char *boot_order,
diff --git a/bootdevice.c b/bootdevice.c
index 2cf6b37c57..03aaffcc8d 100644
--- a/bootdevice.c
+++ b/bootdevice.c
@@ -405,3 +405,34 @@ void del_boot_device_lchs(DeviceState *dev, const char
*suffix)
}
}
}
+
+char *get_boot_devices_lchs_list(size_t *size)
+{
+FWLCHSEntry *i;
+size_t total = 0;
+char *list = NULL;
+
+QTAILQ_FOREACH(i, &fw_lchs, link) {
+char *bootpath;
+char *chs_string;
+size_t len;
+
+bootpath = get_boot_device_path(i->dev, false, i->suffix);
+chs_string = g_strdup_printf("%s %" PRIu32 " %" PRIu32 " %" PRIu32,
+ bootpath, i->lcyls, i->lheads, i->lsecs);
+
+if (total) {
+list[total - 1] = '\n';
+}
+len = strlen(chs_string) + 1;
+list = g_realloc(list, total + len);
+memcpy(&list[total], chs_string, len);
+total += len;
+g_free(chs_string);
+g_free(bootpath);
+}
+
+*size = total;
+
+return list;
+}
diff --git a/hw/nvram/fw_cfg.c b/hw/nvram/fw_cfg.c
index aef1727250..44a3c19326 100644
--- a/hw/nvram/fw_cfg.c
+++ b/hw/nvram/fw_cfg.c
@@ -949,13 +949,21 @@ void *fw_cfg_modify_file(FWCfgState *s, const char
*filename,
static void fw_cfg_machine_reset(void *opaque)
{
+MachineClass *mc = MACHINE_GET_CLASS(qdev_get_machine());
+FWCfgState *s = opaque;
void *ptr;
size_t len;
-FWCfgState *s = opaque;
-char *bootindex = get_boot_devices_list(&len);
+char *buf;
-ptr = fw_cfg_modify_file(s, "bootorder", (uint8_t *)bootindex, len);
+buf = get_boot_devices_list(&len);
+ptr = fw_cfg_modify_file(s, "bootorder", (uint8_t *)buf, len);
g_free(ptr);
+
+if (!mc->legacy_fw_cfg_order) {
+buf = get_boot_devices_lchs_list(&len);
+ptr = fw_cfg_modify_file(s, "bios-geometry", (uint8_t *)buf, len);
+g_free(ptr);
+}
}
static void fw_cfg_machine_ready(struct Notifier *n, void *data)
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL 7/9] bootdevice: Refactor get_boot_devices_list
From: Sam Eiderman
Move device name construction to a separate function.
We will reuse this function in the following commit to pass logical CHS
parameters through fw_cfg much like we currently pass bootindex.
Reviewed-by: Karl Heubaum
Reviewed-by: Arbel Moshe
Reviewed-by: Philippe Mathieu-Daudé
Signed-off-by: Sam Eiderman
Signed-off-by: Sam Eiderman
Tested-by: Philippe Mathieu-Daudé
Signed-off-by: John Snow
---
bootdevice.c | 61 +---
1 file changed, 34 insertions(+), 27 deletions(-)
diff --git a/bootdevice.c b/bootdevice.c
index bc5e1c2de4..2cf6b37c57 100644
--- a/bootdevice.c
+++ b/bootdevice.c
@@ -202,6 +202,39 @@ DeviceState *get_boot_device(uint32_t position)
return res;
}
+static char *get_boot_device_path(DeviceState *dev, bool ignore_suffixes,
+ const char *suffix)
+{
+char *devpath = NULL, *s = NULL, *d, *bootpath;
+
+if (dev) {
+devpath = qdev_get_fw_dev_path(dev);
+assert(devpath);
+}
+
+if (!ignore_suffixes) {
+if (dev) {
+d = qdev_get_own_fw_dev_path_from_handler(dev->parent_bus, dev);
+if (d) {
+assert(!suffix);
+s = d;
+} else {
+s = g_strdup(suffix);
+}
+} else {
+s = g_strdup(suffix);
+}
+}
+
+bootpath = g_strdup_printf("%s%s",
+ devpath ? devpath : "",
+ s ? s : "");
+g_free(devpath);
+g_free(s);
+
+return bootpath;
+}
+
/*
* This function returns null terminated string that consist of new line
* separated device paths.
@@ -218,36 +251,10 @@ char *get_boot_devices_list(size_t *size)
bool ignore_suffixes = mc->ignore_boot_device_suffixes;
QTAILQ_FOREACH(i, &fw_boot_order, link) {
-char *devpath = NULL, *suffix = NULL;
char *bootpath;
-char *d;
size_t len;
-if (i->dev) {
-devpath = qdev_get_fw_dev_path(i->dev);
-assert(devpath);
-}
-
-if (!ignore_suffixes) {
-if (i->dev) {
-d = qdev_get_own_fw_dev_path_from_handler(i->dev->parent_bus,
- i->dev);
-if (d) {
-assert(!i->suffix);
-suffix = d;
-} else {
-suffix = g_strdup(i->suffix);
-}
-} else {
-suffix = g_strdup(i->suffix);
-}
-}
-
-bootpath = g_strdup_printf("%s%s",
- devpath ? devpath : "",
- suffix ? suffix : "");
-g_free(devpath);
-g_free(suffix);
+bootpath = get_boot_device_path(i->dev, ignore_suffixes, i->suffix);
if (total) {
list[total-1] = '\n';
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL 5/9] scsi: Propagate unrealize() callback to scsi-hd
From: Sam Eiderman
We will need to add LCHS removal logic to scsi-hd's unrealize() in the
next commit.
Reviewed-by: Karl Heubaum
Reviewed-by: Arbel Moshe
Reviewed-by: Philippe Mathieu-Daudé
Signed-off-by: Sam Eiderman
Signed-off-by: Sam Eiderman
Tested-by: Philippe Mathieu-Daudé
Signed-off-by: John Snow
---
include/hw/scsi/scsi.h | 1 +
hw/scsi/scsi-bus.c | 16
2 files changed, 17 insertions(+)
diff --git a/include/hw/scsi/scsi.h b/include/hw/scsi/scsi.h
index d77a92361b..332ef602f4 100644
--- a/include/hw/scsi/scsi.h
+++ b/include/hw/scsi/scsi.h
@@ -59,6 +59,7 @@ struct SCSIRequest {
typedef struct SCSIDeviceClass {
DeviceClass parent_class;
void (*realize)(SCSIDevice *dev, Error **errp);
+void (*unrealize)(SCSIDevice *dev, Error **errp);
int (*parse_cdb)(SCSIDevice *dev, SCSICommand *cmd, uint8_t *buf,
void *hba_private);
SCSIRequest *(*alloc_req)(SCSIDevice *s, uint32_t tag, uint32_t lun,
diff --git a/hw/scsi/scsi-bus.c b/hw/scsi/scsi-bus.c
index bccb7cc4c6..359d50d6d0 100644
--- a/hw/scsi/scsi-bus.c
+++ b/hw/scsi/scsi-bus.c
@@ -59,6 +59,14 @@ static void scsi_device_realize(SCSIDevice *s, Error **errp)
}
}
+static void scsi_device_unrealize(SCSIDevice *s, Error **errp)
+{
+SCSIDeviceClass *sc = SCSI_DEVICE_GET_CLASS(s);
+if (sc->unrealize) {
+sc->unrealize(s, errp);
+}
+}
+
int scsi_bus_parse_cdb(SCSIDevice *dev, SCSICommand *cmd, uint8_t *buf,
void *hba_private)
{
@@ -217,12 +225,20 @@ static void scsi_qdev_realize(DeviceState *qdev, Error
**errp)
static void scsi_qdev_unrealize(DeviceState *qdev, Error **errp)
{
SCSIDevice *dev = SCSI_DEVICE(qdev);
+Error *local_err = NULL;
if (dev->vmsentry) {
qemu_del_vm_change_state_handler(dev->vmsentry);
}
scsi_device_purge_requests(dev, SENSE_CODE(NO_SENSE));
+
+scsi_device_unrealize(dev, &local_err);
+if (local_err) {
+error_propagate(errp, local_err);
+return;
+}
+
blockdev_mark_auto_del(dev->conf.blk);
}
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL 6/9] bootdevice: Gather LCHS from all relevant devices
From: Sam Eiderman
Relevant devices are:
* ide-hd (and ide-cd, ide-drive)
* scsi-hd (and scsi-cd, scsi-disk, scsi-block)
* virtio-blk-pci
We do not call del_boot_device_lchs() for ide-* since we don't need to -
IDE block devices do not support unplugging.
Reviewed-by: Karl Heubaum
Reviewed-by: Arbel Moshe
Signed-off-by: Sam Eiderman
Signed-off-by: Sam Eiderman
Reviewed-by: Philippe Mathieu-Daudé
Tested-by: Philippe Mathieu-Daudé
Signed-off-by: John Snow
---
hw/block/virtio-blk.c | 6 ++
hw/ide/qdev.c | 5 +
hw/scsi/scsi-disk.c | 12
3 files changed, 23 insertions(+)
diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c
index 9fa2eaf890..4c357d2928 100644
--- a/hw/block/virtio-blk.c
+++ b/hw/block/virtio-blk.c
@@ -1200,6 +1200,11 @@ static void virtio_blk_device_realize(DeviceState *dev,
Error **errp)
blk_set_guest_block_size(s->blk, s->conf.conf.logical_block_size);
blk_iostatus_enable(s->blk);
+
+add_boot_device_lchs(dev, "/disk@0,0",
+ conf->conf.lcyls,
+ conf->conf.lheads,
+ conf->conf.lsecs);
}
static void virtio_blk_device_unrealize(DeviceState *dev, Error **errp)
@@ -1210,6 +1215,7 @@ static void virtio_blk_device_unrealize(DeviceState *dev,
Error **errp)
unsigned i;
blk_drain(s->blk);
+del_boot_device_lchs(dev, "/disk@0,0");
virtio_blk_data_plane_destroy(s->dataplane);
s->dataplane = NULL;
for (i = 0; i < conf->num_queues; i++) {
diff --git a/hw/ide/qdev.c b/hw/ide/qdev.c
index 85cca6ec38..374a791a45 100644
--- a/hw/ide/qdev.c
+++ b/hw/ide/qdev.c
@@ -220,6 +220,11 @@ static void ide_dev_initfn(IDEDevice *dev, IDEDriveKind
kind, Error **errp)
add_boot_device_path(dev->conf.bootindex, &dev->qdev,
dev->unit ? "/disk@1" : "/disk@0");
+
+add_boot_device_lchs(&dev->qdev, dev->unit ? "/disk@1" : "/disk@0",
+ dev->conf.lcyls,
+ dev->conf.lheads,
+ dev->conf.lsecs);
}
static void ide_dev_get_bootindex(Object *obj, Visitor *v, const char *name,
diff --git a/hw/scsi/scsi-disk.c b/hw/scsi/scsi-disk.c
index 68b1675fd9..07fb5ebdf1 100644
--- a/hw/scsi/scsi-disk.c
+++ b/hw/scsi/scsi-disk.c
@@ -35,6 +35,7 @@
#include "hw/block/block.h"
#include "hw/qdev-properties.h"
#include "sysemu/dma.h"
+#include "sysemu/sysemu.h"
#include "qemu/cutils.h"
#include "trace.h"
@@ -2414,6 +2415,16 @@ static void scsi_realize(SCSIDevice *dev, Error **errp)
blk_set_guest_block_size(s->qdev.conf.blk, s->qdev.blocksize);
blk_iostatus_enable(s->qdev.conf.blk);
+
+add_boot_device_lchs(&dev->qdev, NULL,
+ dev->conf.lcyls,
+ dev->conf.lheads,
+ dev->conf.lsecs);
+}
+
+static void scsi_unrealize(SCSIDevice *dev, Error **errp)
+{
+del_boot_device_lchs(&dev->qdev, NULL);
}
static void scsi_hd_realize(SCSIDevice *dev, Error **errp)
@@ -3018,6 +3029,7 @@ static void scsi_hd_class_initfn(ObjectClass *klass, void
*data)
SCSIDeviceClass *sc = SCSI_DEVICE_CLASS(klass);
sc->realize = scsi_hd_realize;
+sc->unrealize= scsi_unrealize;
sc->alloc_req= scsi_new_request;
sc->unit_attention_reported = scsi_disk_unit_attention_reported;
dc->desc = "virtual SCSI disk";
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL 4/9] bootdevice: Add interface to gather LCHS
From: Sam Eiderman
Add an interface to provide direct logical CHS values for boot devices.
We will use this interface in the next commits.
Reviewed-by: Karl Heubaum
Reviewed-by: Arbel Moshe
Signed-off-by: Sam Eiderman
Signed-off-by: Sam Eiderman
Reviewed-by: Philippe Mathieu-Daudé
Tested-by: Philippe Mathieu-Daudé
Signed-off-by: John Snow
---
include/sysemu/sysemu.h | 3 +++
bootdevice.c| 55 +
2 files changed, 58 insertions(+)
diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
index 44f18eb739..5bc5c79cbc 100644
--- a/include/sysemu/sysemu.h
+++ b/include/sysemu/sysemu.h
@@ -103,6 +103,9 @@ void device_add_bootindex_property(Object *obj, int32_t
*bootindex,
DeviceState *dev, Error **errp);
void restore_boot_order(void *opaque);
void validate_bootdevices(const char *devices, Error **errp);
+void add_boot_device_lchs(DeviceState *dev, const char *suffix,
+ uint32_t lcyls, uint32_t lheads, uint32_t lsecs);
+void del_boot_device_lchs(DeviceState *dev, const char *suffix);
/* handler to set the boot_device order for a specific type of MachineClass */
typedef void QEMUBootSetHandler(void *opaque, const char *boot_order,
diff --git a/bootdevice.c b/bootdevice.c
index 1d225202f9..bc5e1c2de4 100644
--- a/bootdevice.c
+++ b/bootdevice.c
@@ -343,3 +343,58 @@ void device_add_bootindex_property(Object *obj, int32_t
*bootindex,
/* initialize devices' bootindex property to -1 */
object_property_set_int(obj, -1, name, NULL);
}
+
+typedef struct FWLCHSEntry FWLCHSEntry;
+
+struct FWLCHSEntry {
+QTAILQ_ENTRY(FWLCHSEntry) link;
+DeviceState *dev;
+char *suffix;
+uint32_t lcyls;
+uint32_t lheads;
+uint32_t lsecs;
+};
+
+static QTAILQ_HEAD(, FWLCHSEntry) fw_lchs =
+QTAILQ_HEAD_INITIALIZER(fw_lchs);
+
+void add_boot_device_lchs(DeviceState *dev, const char *suffix,
+ uint32_t lcyls, uint32_t lheads, uint32_t lsecs)
+{
+FWLCHSEntry *node;
+
+if (!lcyls && !lheads && !lsecs) {
+return;
+}
+
+assert(dev != NULL || suffix != NULL);
+
+node = g_malloc0(sizeof(FWLCHSEntry));
+node->suffix = g_strdup(suffix);
+node->dev = dev;
+node->lcyls = lcyls;
+node->lheads = lheads;
+node->lsecs = lsecs;
+
+QTAILQ_INSERT_TAIL(&fw_lchs, node, link);
+}
+
+void del_boot_device_lchs(DeviceState *dev, const char *suffix)
+{
+FWLCHSEntry *i;
+
+if (dev == NULL) {
+return;
+}
+
+QTAILQ_FOREACH(i, &fw_lchs, link) {
+if ((!suffix || !g_strcmp0(i->suffix, suffix)) &&
+ i->dev == dev) {
+QTAILQ_REMOVE(&fw_lchs, i, link);
+g_free(i->suffix);
+g_free(i);
+
+break;
+}
+}
+}
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL 3/9] block: Support providing LCHS from user
From: Sam Eiderman
Add logical geometry variables to BlockConf.
A user can now supply "lcyls", "lheads" & "lsecs" for any HD device
that supports CHS ("cyls", "heads", "secs").
These devices include:
* ide-hd
* scsi-hd
* virtio-blk-pci
In future commits we will use the provided LCHS and pass it to the BIOS
through fw_cfg to be supplied using INT13 routines.
Reviewed-by: Karl Heubaum
Reviewed-by: Arbel Moshe
Signed-off-by: Sam Eiderman
Signed-off-by: Sam Eiderman
Reviewed-by: Philippe Mathieu-Daudé
Tested-by: Philippe Mathieu-Daudé
Signed-off-by: John Snow
---
include/hw/block/block.h | 6 +-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/include/hw/block/block.h b/include/hw/block/block.h
index fd55a30bca..d7246f3862 100644
--- a/include/hw/block/block.h
+++ b/include/hw/block/block.h
@@ -26,6 +26,7 @@ typedef struct BlockConf {
uint32_t discard_granularity;
/* geometry, not all devices use this */
uint32_t cyls, heads, secs;
+uint32_t lcyls, lheads, lsecs;
OnOffAuto wce;
bool share_rw;
BlockdevOnError rerror;
@@ -65,7 +66,10 @@ static inline unsigned int get_physical_block_exp(BlockConf
*conf)
#define DEFINE_BLOCK_CHS_PROPERTIES(_state, _conf) \
DEFINE_PROP_UINT32("cyls", _state, _conf.cyls, 0), \
DEFINE_PROP_UINT32("heads", _state, _conf.heads, 0),\
-DEFINE_PROP_UINT32("secs", _state, _conf.secs, 0)
+DEFINE_PROP_UINT32("secs", _state, _conf.secs, 0), \
+DEFINE_PROP_UINT32("lcyls", _state, _conf.lcyls, 0),\
+DEFINE_PROP_UINT32("lheads", _state, _conf.lheads, 0), \
+DEFINE_PROP_UINT32("lsecs", _state, _conf.lsecs, 0)
#define DEFINE_BLOCK_ERROR_PROPERTIES(_state, _conf)\
DEFINE_PROP_BLOCKDEV_ON_ERROR("rerror", _state, _conf.rerror, \
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL 2/9] block: Refactor macros - fix tabbing
From: Sam Eiderman
Fixing tabbing in block related macros.
Signed-off-by: Sam Eiderman
Signed-off-by: Sam Eiderman
Reviewed-by: Karl Heubaum
Reviewed-by: Arbel Moshe
Reviewed-by: Philippe Mathieu-Daudé
Signed-off-by: John Snow
---
include/hw/block/block.h | 16
hw/ide/qdev.c| 2 +-
2 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/include/hw/block/block.h b/include/hw/block/block.h
index 607539057a..fd55a30bca 100644
--- a/include/hw/block/block.h
+++ b/include/hw/block/block.h
@@ -50,21 +50,21 @@ static inline unsigned int get_physical_block_exp(BlockConf
*conf)
_conf.logical_block_size),\
DEFINE_PROP_BLOCKSIZE("physical_block_size", _state,\
_conf.physical_block_size), \
-DEFINE_PROP_UINT16("min_io_size", _state, _conf.min_io_size, 0), \
+DEFINE_PROP_UINT16("min_io_size", _state, _conf.min_io_size, 0),\
DEFINE_PROP_UINT32("opt_io_size", _state, _conf.opt_io_size, 0),\
-DEFINE_PROP_UINT32("discard_granularity", _state, \
- _conf.discard_granularity, -1), \
-DEFINE_PROP_ON_OFF_AUTO("write-cache", _state, _conf.wce, \
-ON_OFF_AUTO_AUTO), \
+DEFINE_PROP_UINT32("discard_granularity", _state, \
+ _conf.discard_granularity, -1), \
+DEFINE_PROP_ON_OFF_AUTO("write-cache", _state, _conf.wce, \
+ON_OFF_AUTO_AUTO), \
DEFINE_PROP_BOOL("share-rw", _state, _conf.share_rw, false)
#define DEFINE_BLOCK_PROPERTIES(_state, _conf) \
DEFINE_PROP_DRIVE("drive", _state, _conf.blk), \
DEFINE_BLOCK_PROPERTIES_BASE(_state, _conf)
-#define DEFINE_BLOCK_CHS_PROPERTIES(_state, _conf) \
-DEFINE_PROP_UINT32("cyls", _state, _conf.cyls, 0), \
-DEFINE_PROP_UINT32("heads", _state, _conf.heads, 0), \
+#define DEFINE_BLOCK_CHS_PROPERTIES(_state, _conf) \
+DEFINE_PROP_UINT32("cyls", _state, _conf.cyls, 0), \
+DEFINE_PROP_UINT32("heads", _state, _conf.heads, 0),\
DEFINE_PROP_UINT32("secs", _state, _conf.secs, 0)
#define DEFINE_BLOCK_ERROR_PROPERTIES(_state, _conf)\
diff --git a/hw/ide/qdev.c b/hw/ide/qdev.c
index 3666e59721..85cca6ec38 100644
--- a/hw/ide/qdev.c
+++ b/hw/ide/qdev.c
@@ -293,7 +293,7 @@ static void ide_drive_realize(IDEDevice *dev, Error **errp)
DEFINE_BLOCK_PROPERTIES(IDEDrive, dev.conf),\
DEFINE_BLOCK_ERROR_PROPERTIES(IDEDrive, dev.conf), \
DEFINE_PROP_STRING("ver", IDEDrive, dev.version), \
-DEFINE_PROP_UINT64("wwn", IDEDrive, dev.wwn, 0),\
+DEFINE_PROP_UINT64("wwn", IDEDrive, dev.wwn, 0), \
DEFINE_PROP_STRING("serial", IDEDrive, dev.serial),\
DEFINE_PROP_STRING("model", IDEDrive, dev.model)
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL 1/9] IDE: deprecate ide-drive
It's an old compatibility shim that just delegates to ide-cd or ide-hd.
I'd like to refactor these some day, and getting rid of the super-object
will make that easier.
Either way, we don't need this.
Signed-off-by: John Snow
Reviewed-by: Thomas Huth
Reviewed-by: Markus Armbruster
ACKed-by: Peter Krempa
Message-id: 20191009224303.10232-2-js...@redhat.com
Signed-off-by: John Snow
---
qemu-deprecated.texi | 5 +
hw/ide/qdev.c | 3 +++
tests/qemu-iotests/051.pc.out | 6 --
3 files changed, 12 insertions(+), 2 deletions(-)
diff --git a/qemu-deprecated.texi b/qemu-deprecated.texi
index f727bd3932..296bfc93a3 100644
--- a/qemu-deprecated.texi
+++ b/qemu-deprecated.texi
@@ -254,6 +254,11 @@ quite a bit. It will be removed without replacement unless
some users speaks
up at the @email{qemu-devel@@nongnu.org} mailing list with information about
their usecases.
+@subsection ide-drive (since 4.2)
+
+The 'ide-drive' device is deprecated. Users should use 'ide-hd' or
+'ide-cd' as appropriate to get an IDE hard disk or CD-ROM as needed.
+
@section System emulator machines
@subsection pc-0.12, pc-0.13, pc-0.14 and pc-0.15 (since 4.0)
diff --git a/hw/ide/qdev.c b/hw/ide/qdev.c
index 6fba6b62b8..3666e59721 100644
--- a/hw/ide/qdev.c
+++ b/hw/ide/qdev.c
@@ -279,6 +279,9 @@ static void ide_drive_realize(IDEDevice *dev, Error **errp)
{
DriveInfo *dinfo = NULL;
+warn_report("'ide-drive' is deprecated, "
+"please use 'ide-hd' or 'ide-cd' instead");
+
if (dev->conf.blk) {
dinfo = blk_legacy_dinfo(dev->conf.blk);
}
diff --git a/tests/qemu-iotests/051.pc.out b/tests/qemu-iotests/051.pc.out
index 000557c7c8..34849dd172 100644
--- a/tests/qemu-iotests/051.pc.out
+++ b/tests/qemu-iotests/051.pc.out
@@ -158,7 +158,8 @@ QEMU X.Y.Z monitor - type 'help' for more information
Testing: -drive if=none,id=disk -device ide-drive,drive=disk
QEMU X.Y.Z monitor - type 'help' for more information
-(qemu) QEMU_PROG: -device ide-drive,drive=disk: Device needs media, but drive
is empty
+(qemu) QEMU_PROG: -device ide-drive,drive=disk: warning: 'ide-drive' is
deprecated, please use 'ide-hd' or 'ide-cd' instead
+QEMU_PROG: -device ide-drive,drive=disk: Device needs media, but drive is empty
Testing: -drive if=none,id=disk -device ide-hd,drive=disk
QEMU X.Y.Z monitor - type 'help' for more information
@@ -228,7 +229,8 @@ QEMU X.Y.Z monitor - type 'help' for more information
Testing: -drive file=TEST_DIR/t.qcow2,if=none,id=disk,readonly=on -device
ide-drive,drive=disk
QEMU X.Y.Z monitor - type 'help' for more information
-(qemu) QEMU_PROG: -device ide-drive,drive=disk: Block node is read-only
+(qemu) QEMU_PROG: -device ide-drive,drive=disk: warning: 'ide-drive' is
deprecated, please use 'ide-hd' or 'ide-cd' instead
+QEMU_PROG: -device ide-drive,drive=disk: Block node is read-only
Testing: -drive file=TEST_DIR/t.qcow2,if=none,id=disk,readonly=on -device
ide-hd,drive=disk
QEMU X.Y.Z monitor - type 'help' for more information
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL 0/9] Ide patches
The following changes since commit 68d8ef4ec540682c3538d4963e836e43a211dd17: Merge remote-tracking branch 'remotes/stsquad/tags/pull-tcg-plugins-281019-4' into staging (2019-10-30 14:10:32 +) are available in the Git repository at: https://github.com/jnsnow/qemu.git tags/ide-pull-request for you to fetch changes up to c35564caf20e8d3431786dddf0fa513daa7d7f3c: hd-geo-test: Add tests for lchs override (2019-10-31 06:11:34 -0400) Pull request ---- John Snow (1): IDE: deprecate ide-drive Sam Eiderman (8): block: Refactor macros - fix tabbing block: Support providing LCHS from user bootdevice: Add interface to gather LCHS scsi: Propagate unrealize() callback to scsi-hd bootdevice: Gather LCHS from all relevant devices bootdevice: Refactor get_boot_devices_list bootdevice: FW_CFG interface for LCHS values hd-geo-test: Add tests for lchs override qemu-deprecated.texi | 5 + include/hw/block/block.h | 22 +- include/hw/scsi/scsi.h| 1 + include/sysemu/sysemu.h | 4 + bootdevice.c | 147 +++-- hw/block/virtio-blk.c | 6 + hw/ide/qdev.c | 10 +- hw/nvram/fw_cfg.c | 14 +- hw/scsi/scsi-bus.c| 16 + hw/scsi/scsi-disk.c | 12 + tests/hd-geo-test.c | 551 ++ tests/Makefile.include| 2 +- tests/qemu-iotests/051.pc.out | 6 +- 13 files changed, 753 insertions(+), 43 deletions(-) -- 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL v3 16/19] block/qcow2-bitmap: fix and improve qcow2_reopen_bitmaps_rw
From: Vladimir Sementsov-Ogievskiy
- Correct check for write access to file child, and in correct place
(only if we want to write).
- Support reopen rw -> rw (which will be used in following commit),
for example, !bdrv_dirty_bitmap_readonly() is not a corruption if
bitmap is marked IN_USE in the image.
- Consider unexpected bitmap as a corruption and check other
combinations of in-image and in-RAM bitmaps.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Message-id: 20190927122355.7344-9-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
block/qcow2-bitmap.c | 77 +---
1 file changed, 58 insertions(+), 19 deletions(-)
diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
index f7dfb40256..98294a7696 100644
--- a/block/qcow2-bitmap.c
+++ b/block/qcow2-bitmap.c
@@ -1108,18 +1108,14 @@ int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error
**errp)
Qcow2BitmapList *bm_list;
Qcow2Bitmap *bm;
GSList *ro_dirty_bitmaps = NULL;
-int ret = 0;
+int ret = -EINVAL;
+bool need_header_update = false;
if (s->nb_bitmaps == 0) {
/* No bitmaps - nothing to do */
return 0;
}
-if (!can_write(bs)) {
-error_setg(errp, "Can't write to the image on reopening bitmaps rw");
-return -EINVAL;
-}
-
bm_list = bitmap_list_load(bs, s->bitmap_directory_offset,
s->bitmap_directory_size, errp);
if (bm_list == NULL) {
@@ -1128,32 +1124,75 @@ int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error
**errp)
QSIMPLEQ_FOREACH(bm, bm_list, entry) {
BdrvDirtyBitmap *bitmap = bdrv_find_dirty_bitmap(bs, bm->name);
-if (bitmap == NULL) {
-continue;
-}
-if (!bdrv_dirty_bitmap_readonly(bitmap)) {
-error_setg(errp, "Bitmap %s was loaded prior to rw-reopen, but was
"
- "not marked as readonly. This is a bug, something went "
- "wrong. All of the bitmaps may be corrupted", bm->name);
-ret = -EINVAL;
+if (!bitmap) {
+error_setg(errp, "Unexpected bitmap '%s' in image '%s'",
+ bm->name, bs->filename);
goto out;
}
-bm->flags |= BME_FLAG_IN_USE;
-ro_dirty_bitmaps = g_slist_append(ro_dirty_bitmaps, bitmap);
+if (!(bm->flags & BME_FLAG_IN_USE)) {
+if (!bdrv_dirty_bitmap_readonly(bitmap)) {
+error_setg(errp, "Corruption: bitmap '%s' is not marked IN_USE
"
+ "in the image '%s' and not marked readonly in RAM",
+ bm->name, bs->filename);
+goto out;
+}
+if (bdrv_dirty_bitmap_inconsistent(bitmap)) {
+error_setg(errp, "Corruption: bitmap '%s' is inconsistent but "
+ "is not marked IN_USE in the image '%s'", bm->name,
+ bs->filename);
+goto out;
+}
+
+bm->flags |= BME_FLAG_IN_USE;
+need_header_update = true;
+} else {
+/*
+ * What if flags already has BME_FLAG_IN_USE ?
+ *
+ * 1. if we are reopening RW -> RW it's OK, of course.
+ * 2. if we are reopening RO -> RW:
+ * 2.1 if @bitmap is inconsistent, it's OK. It means that it was
+ * inconsistent (IN_USE) when we loaded it
+ * 2.2 if @bitmap is not inconsistent. This seems to be
impossible
+ * and implies third party interaction. Let's error-out for
+ * safety.
+ */
+if (bdrv_dirty_bitmap_readonly(bitmap) &&
+!bdrv_dirty_bitmap_inconsistent(bitmap))
+{
+error_setg(errp, "Corruption: bitmap '%s' is marked IN_USE "
+ "in the image '%s' but it is readonly and "
+ "consistent in RAM",
+ bm->name, bs->filename);
+goto out;
+}
+}
+
+if (bdrv_dirty_bitmap_readonly(bitmap)) {
+ro_dirty_bitmaps = g_slist_append(ro_dirty_bitmaps, bitmap);
+}
}
-if (ro_dirty_bitmaps != NULL) {
+if (need_header_update) {
+if (!can_write(bs->file->bs) || !(bs->file->perm & BLK_PERM_WRITE)) {
+error_setg(errp, "Failed to reopen bitmaps rw: no write access "
+ "the protocol file");
+goto out;
+}
+
/* in_use flags must be updated */
ret = upda
[libvirt] [PULL v3 15/19] iotests: add test 260 to check bitmap life after snapshot + commit
From: Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy
Message-id: 20190927122355.7344-8-vsement...@virtuozzo.com
[Maintainer edit: removed 260 from auto group per Peter Maydell. --js]
Signed-off-by: John Snow
---
tests/qemu-iotests/260 | 89 ++
tests/qemu-iotests/260.out | 52 ++
tests/qemu-iotests/group | 1 +
3 files changed, 142 insertions(+)
create mode 100755 tests/qemu-iotests/260
create mode 100644 tests/qemu-iotests/260.out
diff --git a/tests/qemu-iotests/260 b/tests/qemu-iotests/260
new file mode 100755
index 00..4f6082c9d2
--- /dev/null
+++ b/tests/qemu-iotests/260
@@ -0,0 +1,89 @@
+#!/usr/bin/env python
+#
+# Tests for temporary external snapshot when we have bitmaps.
+#
+# Copyright (c) 2019 Virtuozzo International GmbH. All rights reserved.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+
+import iotests
+from iotests import qemu_img_create, file_path, log, filter_qmp_event
+
+iotests.verify_image_format(supported_fmts=['qcow2'])
+
+base, top = file_path('base', 'top')
+size = 64 * 1024 * 3
+
+
+def print_bitmap(msg, vm):
+result = vm.qmp('query-block')['return'][0]
+if 'dirty-bitmaps' in result:
+bitmap = result['dirty-bitmaps'][0]
+log('{}: name={} dirty-clusters={}'.format(msg, bitmap['name'],
+bitmap['count'] // 64 // 1024))
+else:
+log(msg + ': not found')
+
+
+def test(persistent, restart):
+assert persistent or not restart
+log("\nTestcase {}persistent {} restart\n".format(
+'' if persistent else 'non-', 'with' if restart else 'without'))
+
+qemu_img_create('-f', iotests.imgfmt, base, str(size))
+
+vm = iotests.VM().add_drive(base)
+vm.launch()
+
+vm.qmp_log('block-dirty-bitmap-add', node='drive0', name='bitmap0',
+ persistent=persistent)
+vm.hmp_qemu_io('drive0', 'write 0 64K')
+print_bitmap('initial bitmap', vm)
+
+vm.qmp_log('blockdev-snapshot-sync', device='drive0', snapshot_file=top,
+ format=iotests.imgfmt, filters=[iotests.filter_qmp_testfiles])
+vm.hmp_qemu_io('drive0', 'write 64K 512')
+print_bitmap('check that no bitmaps are in snapshot', vm)
+
+if restart:
+log("... Restart ...")
+vm.shutdown()
+vm = iotests.VM().add_drive(top)
+vm.launch()
+
+vm.qmp_log('block-commit', device='drive0', top=top,
+ filters=[iotests.filter_qmp_testfiles])
+ev = vm.events_wait((('BLOCK_JOB_READY', None),
+ ('BLOCK_JOB_COMPLETED', None)))
+log(filter_qmp_event(ev))
+if (ev['event'] == 'BLOCK_JOB_COMPLETED'):
+vm.shutdown()
+log(vm.get_log())
+exit()
+
+vm.qmp_log('block-job-complete', device='drive0')
+ev = vm.event_wait('BLOCK_JOB_COMPLETED')
+log(filter_qmp_event(ev))
+print_bitmap('check bitmap after commit', vm)
+
+vm.hmp_qemu_io('drive0', 'write 128K 64K')
+print_bitmap('check updated bitmap', vm)
+
+vm.shutdown()
+
+
+test(persistent=False, restart=False)
+test(persistent=True, restart=False)
+test(persistent=True, restart=True)
diff --git a/tests/qemu-iotests/260.out b/tests/qemu-iotests/260.out
new file mode 100644
index 00..2f0d98d036
--- /dev/null
+++ b/tests/qemu-iotests/260.out
@@ -0,0 +1,52 @@
+
+Testcase non-persistent without restart
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"name": "bitmap0", "node":
"drive0", "persistent": false}}
+{"return": {}}
+initial bitmap: name=bitmap0 dirty-clusters=1
+{"execute": "blockdev-snapshot-sync", "arguments": {"device": "drive0",
"format": "qcow2", "snapshot-file": "TEST_DIR/PID-top"}}
+{"return": {}}
+check that no bitmaps are in snapshot: not found
+{&
[libvirt] [PULL v3 18/19] MAINTAINERS: Add Vladimir as a reviewer for bitmaps
I already try to make sure all bitmaps patches have been reviewed by both Red Hat and Virtuozzo anyway, so this formalizes the arrangement. Fam meanwhile is no longer as active, so I am removing him as a co-maintainer simply to reflect the current practice. Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy Message-id: 20191005194448.16629-2-js...@redhat.com --- MAINTAINERS | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/MAINTAINERS b/MAINTAINERS index fe4dc51b08..250ce8e7a1 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1816,8 +1816,8 @@ F: qapi/transaction.json T: git https://repo.or.cz/qemu/armbru.git block-next Dirty Bitmaps -M: Fam Zheng M: John Snow +R: Vladimir Sementsov-Ogievskiy L: qemu-bl...@nongnu.org S: Supported F: util/hbitmap.c @@ -1826,7 +1826,6 @@ F: include/qemu/hbitmap.h F: include/block/dirty-bitmap.h F: tests/test-hbitmap.c F: docs/interop/bitmaps.rst -T: git https://github.com/famz/qemu.git bitmaps T: git https://github.com/jnsnow/qemu.git bitmaps Character device backends -- 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL v3 17/19] qcow2-bitmap: move bitmap reopen-rw code to qcow2_reopen_commit
From: Vladimir Sementsov-Ogievskiy
The only reason I can imagine for this strange code at the very-end of
bdrv_reopen_commit is the fact that bs->read_only updated after
calling drv->bdrv_reopen_commit in bdrv_reopen_commit. And in the same
time, prior to previous commit, qcow2_reopen_bitmaps_rw did a wrong
check for being writable, when actually it only need writable file
child not self.
So, as it's fixed, let's move things to correct place.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: John Snow
Acked-by: Max Reitz
Message-id: 20190927122355.7344-10-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
include/block/block_int.h | 6 --
block.c | 19 ---
block/qcow2.c | 15 ++-
3 files changed, 14 insertions(+), 26 deletions(-)
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 32fb493cbb..ca4ccac4c1 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -547,12 +547,6 @@ struct BlockDriver {
uint64_t parent_perm, uint64_t parent_shared,
uint64_t *nperm, uint64_t *nshared);
-/**
- * Bitmaps should be marked as 'IN_USE' in the image on reopening image
- * as rw. This handler should realize it. It also should unset readonly
- * field of BlockDirtyBitmap's in case of success.
- */
-int (*bdrv_reopen_bitmaps_rw)(BlockDriverState *bs, Error **errp);
bool (*bdrv_co_can_store_new_dirty_bitmap)(BlockDriverState *bs,
const char *name,
uint32_t granularity,
diff --git a/block.c b/block.c
index cf312258a9..dad5a3d8e0 100644
--- a/block.c
+++ b/block.c
@@ -3935,16 +3935,12 @@ void bdrv_reopen_commit(BDRVReopenState *reopen_state)
BlockDriver *drv;
BlockDriverState *bs;
BdrvChild *child;
-bool old_can_write, new_can_write;
assert(reopen_state != NULL);
bs = reopen_state->bs;
drv = bs->drv;
assert(drv != NULL);
-old_can_write =
-!bdrv_is_read_only(bs) && !(bdrv_get_flags(bs) & BDRV_O_INACTIVE);
-
/* If there are any driver level actions to take */
if (drv->bdrv_reopen_commit) {
drv->bdrv_reopen_commit(reopen_state);
@@ -3988,21 +3984,6 @@ void bdrv_reopen_commit(BDRVReopenState *reopen_state)
}
bdrv_refresh_limits(bs, NULL);
-
-new_can_write =
-!bdrv_is_read_only(bs) && !(bdrv_get_flags(bs) & BDRV_O_INACTIVE);
-if (!old_can_write && new_can_write && drv->bdrv_reopen_bitmaps_rw) {
-Error *local_err = NULL;
-if (drv->bdrv_reopen_bitmaps_rw(bs, &local_err) < 0) {
-/* This is not fatal, bitmaps just left read-only, so all following
- * writes will fail. User can remove read-only bitmaps to unblock
- * writes.
- */
-error_reportf_err(local_err,
- "%s: Failed to make dirty bitmaps writable: ",
- bdrv_get_node_name(bs));
-}
-}
}
/*
diff --git a/block/qcow2.c b/block/qcow2.c
index 53a025703e..8b05933565 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -1835,6 +1835,20 @@ fail:
static void qcow2_reopen_commit(BDRVReopenState *state)
{
qcow2_update_options_commit(state->bs, state->opaque);
+if (state->flags & BDRV_O_RDWR) {
+Error *local_err = NULL;
+
+if (qcow2_reopen_bitmaps_rw(state->bs, &local_err) < 0) {
+/*
+ * This is not fatal, bitmaps just left read-only, so all following
+ * writes will fail. User can remove read-only bitmaps to unblock
+ * writes or retry reopen.
+ */
+error_reportf_err(local_err,
+ "%s: Failed to make dirty bitmaps writable: ",
+ bdrv_get_node_name(state->bs));
+}
+}
g_free(state->opaque);
}
@@ -5406,7 +5420,6 @@ BlockDriver bdrv_qcow2 = {
.bdrv_detach_aio_context = qcow2_detach_aio_context,
.bdrv_attach_aio_context = qcow2_attach_aio_context,
-.bdrv_reopen_bitmaps_rw = qcow2_reopen_bitmaps_rw,
.bdrv_co_can_store_new_dirty_bitmap = qcow2_co_can_store_new_dirty_bitmap,
.bdrv_co_remove_persistent_dirty_bitmap =
qcow2_co_remove_persistent_dirty_bitmap,
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL v3 19/19] dirty-bitmaps: remove deprecated autoload parameter
This parameter has been deprecated since 2.12.0 and is eligible for
removal. Remove this parameter as it is actually completely ignored;
let's not give false hope.
Signed-off-by: John Snow
Reviewed-by: Eric Blake
Reviewed-by: Vladimir Sementsov-Ogievskiy
Message-id: 20191002232411.29968-1-js...@redhat.com
---
qemu-deprecated.texi | 20 +++-
qapi/block-core.json | 6 +-
blockdev.c | 6 --
3 files changed, 16 insertions(+), 16 deletions(-)
diff --git a/qemu-deprecated.texi b/qemu-deprecated.texi
index 01245e0b1c..7239e0959d 100644
--- a/qemu-deprecated.texi
+++ b/qemu-deprecated.texi
@@ -149,11 +149,6 @@ QEMU 4.1 has three options, please migrate to one of these
three:
@section QEMU Machine Protocol (QMP) commands
-@subsection block-dirty-bitmap-add "autoload" parameter (since 2.12.0)
-
-"autoload" parameter is now ignored. All bitmaps are automatically loaded
-from qcow2 images.
-
@subsection query-block result field dirty-bitmaps[i].status (since 4.0)
The ``status'' field of the ``BlockDirtyInfo'' structure, returned by
@@ -356,3 +351,18 @@ existing CPU models. Management software that needs
runnability
guarantees must resolve the CPU model aliases using te
``alias-of'' field returned by the ``query-cpu-definitions'' QMP
command.
+
+
+@node Recently removed features
+@appendix Recently removed features
+
+What follows is a record of recently removed, formerly deprecated
+features that serves as a record for users who have encountered
+trouble after a recent upgrade.
+
+@section QEMU Machine Protocol (QMP) commands
+
+@subsection block-dirty-bitmap-add "autoload" parameter (since 4.2.0)
+
+The "autoload" parameter has been ignored since 2.12.0. All bitmaps
+are automatically loaded from qcow2 images.
diff --git a/qapi/block-core.json b/qapi/block-core.json
index f66553aac7..b274aef713 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -2052,10 +2052,6 @@
# Qcow2 disks support persistent bitmaps. Default is false for
# block-dirty-bitmap-add. (Since: 2.10)
#
-# @autoload: ignored and deprecated since 2.12.
-#Currently, all dirty tracking bitmaps are loaded from Qcow2 on
-#open.
-#
# @disabled: the bitmap is created in the disabled state, which means that
#it will not track drive changes. The bitmap may be enabled with
#block-dirty-bitmap-enable. Default is false. (Since: 4.0)
@@ -2064,7 +2060,7 @@
##
{ 'struct': 'BlockDirtyBitmapAdd',
'data': { 'node': 'str', 'name': 'str', '*granularity': 'uint32',
-'*persistent': 'bool', '*autoload': 'bool', '*disabled': 'bool' } }
+'*persistent': 'bool', '*disabled': 'bool' } }
##
# @BlockDirtyBitmapMergeSource:
diff --git a/blockdev.c b/blockdev.c
index d77e809623..03c7cd7651 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -1966,7 +1966,6 @@ static void block_dirty_bitmap_add_prepare(BlkActionState
*common,
qmp_block_dirty_bitmap_add(action->node, action->name,
action->has_granularity, action->granularity,
action->has_persistent, action->persistent,
- action->has_autoload, action->autoload,
action->has_disabled, action->disabled,
&local_err);
@@ -2858,7 +2857,6 @@ out:
void qmp_block_dirty_bitmap_add(const char *node, const char *name,
bool has_granularity, uint32_t granularity,
bool has_persistent, bool persistent,
-bool has_autoload, bool autoload,
bool has_disabled, bool disabled,
Error **errp)
{
@@ -2890,10 +2888,6 @@ void qmp_block_dirty_bitmap_add(const char *node, const
char *name,
persistent = false;
}
-if (has_autoload) {
-warn_report("Autoload option is deprecated and its value is ignored");
-}
-
if (!has_disabled) {
disabled = false;
}
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL v3 13/19] block/qcow2-bitmap: drop qcow2_reopen_bitmaps_rw_hint()
From: Vladimir Sementsov-Ogievskiy
The function is unused, drop it.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: John Snow
Message-id: 20190927122355.7344-6-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
block/qcow2.h| 2 --
block/qcow2-bitmap.c | 15 +--
2 files changed, 1 insertion(+), 16 deletions(-)
diff --git a/block/qcow2.h b/block/qcow2.h
index 0f3d9b088e..23a9898a54 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -740,8 +740,6 @@ int qcow2_check_bitmaps_refcounts(BlockDriverState *bs,
BdrvCheckResult *res,
bool qcow2_load_dirty_bitmaps(BlockDriverState *bs, Error **errp);
Qcow2BitmapInfoList *qcow2_get_bitmap_info_list(BlockDriverState *bs,
Error **errp);
-int qcow2_reopen_bitmaps_rw_hint(BlockDriverState *bs, bool *header_updated,
- Error **errp);
int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error **errp);
int qcow2_truncate_bitmaps_check(BlockDriverState *bs, Error **errp);
void qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp);
diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
index 6dfc083548..ebc1afccd3 100644
--- a/block/qcow2-bitmap.c
+++ b/block/qcow2-bitmap.c
@@ -1102,8 +1102,7 @@ Qcow2BitmapInfoList
*qcow2_get_bitmap_info_list(BlockDriverState *bs,
return list;
}
-int qcow2_reopen_bitmaps_rw_hint(BlockDriverState *bs, bool *header_updated,
- Error **errp)
+int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error **errp)
{
BDRVQcow2State *s = bs->opaque;
Qcow2BitmapList *bm_list;
@@ -,10 +1110,6 @@ int qcow2_reopen_bitmaps_rw_hint(BlockDriverState *bs,
bool *header_updated,
GSList *ro_dirty_bitmaps = NULL;
int ret = 0;
-if (header_updated != NULL) {
-*header_updated = false;
-}
-
if (s->nb_bitmaps == 0) {
/* No bitmaps - nothing to do */
return 0;
@@ -1156,9 +1151,6 @@ int qcow2_reopen_bitmaps_rw_hint(BlockDriverState *bs,
bool *header_updated,
error_setg_errno(errp, -ret, "Can't update bitmap directory");
goto out;
}
-if (header_updated != NULL) {
-*header_updated = true;
-}
g_slist_foreach(ro_dirty_bitmaps, set_readonly_helper, false);
}
@@ -1169,11 +1161,6 @@ out:
return ret;
}
-int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error **errp)
-{
-return qcow2_reopen_bitmaps_rw_hint(bs, NULL, errp);
-}
-
/* Checks to see if it's safe to resize bitmaps */
int qcow2_truncate_bitmaps_check(BlockDriverState *bs, Error **errp)
{
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL v3 14/19] block/qcow2-bitmap: do not remove bitmaps on reopen-ro
From: Vladimir Sementsov-Ogievskiy
qcow2_reopen_bitmaps_ro wants to store bitmaps and then mark them all
readonly. But the latter don't work, as
qcow2_store_persistent_dirty_bitmaps removes bitmaps after storing.
It's OK for inactivation but bad idea for reopen-ro. And this leads to
the following bug:
Assume we have persistent bitmap 'bitmap0'.
Create external snapshot
bitmap0 is stored and therefore removed
Commit snapshot
now we have no bitmaps
Do some writes from guest (*)
they are not marked in bitmap
Shutdown
Start
bitmap0 is loaded as valid, but it is actually broken! It misses
writes (*)
Incremental backup
it will be inconsistent
So, let's stop removing bitmaps on reopen-ro. But don't rejoice:
reopening bitmaps to rw is broken too, so the whole scenario will not
work after this patch and we can't enable corresponding test cases in
260 iotests still. Reopening bitmaps rw will be fixed in the following
patches.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: John Snow
Message-id: 20190927122355.7344-7-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
block/qcow2.h| 3 ++-
block/qcow2-bitmap.c | 49 ++--
block/qcow2.c| 2 +-
3 files changed, 37 insertions(+), 17 deletions(-)
diff --git a/block/qcow2.h b/block/qcow2.h
index 23a9898a54..5cccd87162 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -742,7 +742,8 @@ Qcow2BitmapInfoList
*qcow2_get_bitmap_info_list(BlockDriverState *bs,
Error **errp);
int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error **errp);
int qcow2_truncate_bitmaps_check(BlockDriverState *bs, Error **errp);
-void qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp);
+void qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs,
+ bool release_stored, Error **errp);
int qcow2_reopen_bitmaps_ro(BlockDriverState *bs, Error **errp);
bool qcow2_co_can_store_new_dirty_bitmap(BlockDriverState *bs,
const char *name,
diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
index ebc1afccd3..f7dfb40256 100644
--- a/block/qcow2-bitmap.c
+++ b/block/qcow2-bitmap.c
@@ -1440,7 +1440,32 @@ out:
return ret;
}
-void qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp)
+/*
+ * qcow2_store_persistent_dirty_bitmaps
+ *
+ * Stores persistent BdrvDirtyBitmap objects.
+ *
+ * @release_stored: if true, release BdrvDirtyBitmap's after storing to the
+ * image. This is used in two cases, both via qcow2_inactivate:
+ * 1. bdrv_close: It's correct to remove bitmaps on close.
+ * 2. migration: If bitmaps are migrated through migration channel via
+ *'dirty-bitmaps' migration capability they are not handled by this code.
+ *Otherwise, it's OK to drop BdrvDirtyBitmap's and reload them on
+ *invalidation.
+ *
+ * Anyway, it's correct to remove BdrvDirtyBitmap's on inactivation, as
+ * inactivation means that we lose control on disk, and therefore on bitmaps,
+ * we should sync them and do not touch more.
+ *
+ * Contrariwise, we don't want to release any bitmaps on just reopen-to-ro,
+ * when we need to store them, as image is still under our control, and it's
+ * good to keep all the bitmaps in read-only mode. Moreover, keeping them
+ * read-only is correct because this is what would happen if we opened the node
+ * readonly to begin with, and whether we opened directly or reopened to that
+ * state shouldn't matter for the state we get afterward.
+ */
+void qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs,
+ bool release_stored, Error **errp)
{
BdrvDirtyBitmap *bitmap;
BDRVQcow2State *s = bs->opaque;
@@ -1551,20 +1576,14 @@ void
qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp)
g_free(tb);
}
-QSIMPLEQ_FOREACH(bm, bm_list, entry) {
-/* For safety, we remove bitmap after storing.
- * We may be here in two cases:
- * 1. bdrv_close. It's ok to drop bitmap.
- * 2. inactivation. It means migration without 'dirty-bitmaps'
- *capability, so bitmaps are not marked with
- *BdrvDirtyBitmap.migration flags. It's not bad to drop them too,
- *and reload on invalidation.
- */
-if (bm->dirty_bitmap == NULL) {
-continue;
-}
+if (release_stored) {
+QSIMPLEQ_FOREACH(bm, bm_list, entry) {
+if (bm->dirty_bitmap == NULL) {
+continue;
+}
-bdrv_release_dirty_bitmap(bm->dirty_bitmap);
+bdrv_release_dirty_bitmap(bm->dirty_bitmap);
+}
}
success:
@@ -1592,7 +1611,7 @@ int qcow2_reopen_bitmaps_ro(BlockDriverState *bs, Error
*
[libvirt] [PULL v3 00/19] Bitmaps patches
The following changes since commit f22f553efffd083ff624be116726f843a39f1148: Merge remote-tracking branch 'remotes/rth/tags/pull-tcg-20191013' into staging (2019-10-17 16:48:56 +0100) are available in the Git repository at: https://github.com/jnsnow/qemu.git tags/bitmaps-pull-request for you to fetch changes up to 3264ffced3d087bbe72d759639ef84fd5bd924cc: dirty-bitmaps: remove deprecated autoload parameter (2019-10-17 17:53:28 -0400) pull request ---- John Snow (2): MAINTAINERS: Add Vladimir as a reviewer for bitmaps dirty-bitmaps: remove deprecated autoload parameter Vladimir Sementsov-Ogievskiy (17): util/hbitmap: strict hbitmap_reset block: move bdrv_can_store_new_dirty_bitmap to block/dirty-bitmap.c block/dirty-bitmap: return int from bdrv_remove_persistent_dirty_bitmap block/qcow2: proper locking on bitmap add/remove paths block/dirty-bitmap: drop meta block/dirty-bitmap: add bs link block/dirty-bitmap: drop BdrvDirtyBitmap.mutex block/dirty-bitmap: refactor bdrv_dirty_bitmap_next block: switch reopen queue from QSIMPLEQ to QTAILQ block: reverse order for reopen commits iotests: add test-case to 165 to test reopening qcow2 bitmaps to RW block/qcow2-bitmap: get rid of bdrv_has_changed_persistent_bitmaps block/qcow2-bitmap: drop qcow2_reopen_bitmaps_rw_hint() block/qcow2-bitmap: do not remove bitmaps on reopen-ro iotests: add test 260 to check bitmap life after snapshot + commit block/qcow2-bitmap: fix and improve qcow2_reopen_bitmaps_rw qcow2-bitmap: move bitmap reopen-rw code to qcow2_reopen_commit qemu-deprecated.texi | 20 ++- qapi/block-core.json | 6 +- block/qcow2.h | 19 +-- include/block/block.h | 2 +- include/block/block_int.h | 20 +-- include/block/dirty-bitmap.h | 34 ++-- include/qemu/hbitmap.h | 5 + block.c| 79 +++-- block/backup.c | 8 +- block/block-copy.c | 2 +- block/dirty-bitmap.c | 290 +++-- block/mirror.c | 4 +- block/qcow2-bitmap.c | 212 +++- block/qcow2.c | 22 ++- blockdev.c | 40 ++--- migration/block-dirty-bitmap.c | 11 +- migration/block.c | 4 +- tests/test-hbitmap.c | 2 +- util/hbitmap.c | 4 + MAINTAINERS| 3 +- tests/qemu-iotests/165 | 57 ++- tests/qemu-iotests/165.out | 4 +- tests/qemu-iotests/260 | 89 ++ tests/qemu-iotests/260.out | 52 ++ tests/qemu-iotests/group | 1 + 25 files changed, 623 insertions(+), 367 deletions(-) create mode 100755 tests/qemu-iotests/260 create mode 100644 tests/qemu-iotests/260.out -- 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL v3 07/19] block/dirty-bitmap: drop BdrvDirtyBitmap.mutex
From: Vladimir Sementsov-Ogievskiy
mutex field is just a pointer to bs->dirty_bitmap_mutex, so no needs
to store it in BdrvDirtyBitmap when we have bs pointer in it (since
previous patch).
Drop mutex field. Constantly use bdrv_dirty_bitmaps_lock/unlock in
block/dirty-bitmap.c to make it more obvious that it's not per-bitmap
lock. Still, for simplicity, leave bdrv_dirty_bitmap_lock/unlock
functions as an external API.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: John Snow
Message-id: 20190916141911.5255-4-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
block/dirty-bitmap.c | 84 +---
1 file changed, 41 insertions(+), 43 deletions(-)
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 44453ff824..4e5c87a907 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -29,7 +29,6 @@
#include "qemu/main-loop.h"
struct BdrvDirtyBitmap {
-QemuMutex *mutex;
BlockDriverState *bs;
HBitmap *bitmap;/* Dirty bitmap implementation */
bool busy; /* Bitmap is busy, it can't be used via QMP */
@@ -72,12 +71,12 @@ static inline void
bdrv_dirty_bitmaps_unlock(BlockDriverState *bs)
void bdrv_dirty_bitmap_lock(BdrvDirtyBitmap *bitmap)
{
-qemu_mutex_lock(bitmap->mutex);
+bdrv_dirty_bitmaps_lock(bitmap->bs);
}
void bdrv_dirty_bitmap_unlock(BdrvDirtyBitmap *bitmap)
{
-qemu_mutex_unlock(bitmap->mutex);
+bdrv_dirty_bitmaps_unlock(bitmap->bs);
}
/* Called with BQL or dirty_bitmap lock taken. */
@@ -117,7 +116,6 @@ BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverState
*bs,
}
bitmap = g_new0(BdrvDirtyBitmap, 1);
bitmap->bs = bs;
-bitmap->mutex = &bs->dirty_bitmap_mutex;
bitmap->bitmap = hbitmap_alloc(bitmap_size, ctz32(granularity));
bitmap->size = bitmap_size;
bitmap->name = g_strdup(name);
@@ -151,9 +149,9 @@ static bool bdrv_dirty_bitmap_busy(const BdrvDirtyBitmap
*bitmap)
void bdrv_dirty_bitmap_set_busy(BdrvDirtyBitmap *bitmap, bool busy)
{
-qemu_mutex_lock(bitmap->mutex);
+bdrv_dirty_bitmaps_lock(bitmap->bs);
bitmap->busy = busy;
-qemu_mutex_unlock(bitmap->mutex);
+bdrv_dirty_bitmaps_unlock(bitmap->bs);
}
/* Called with BQL taken. */
@@ -278,10 +276,10 @@ void bdrv_enable_dirty_bitmap_locked(BdrvDirtyBitmap
*bitmap)
/* Called with BQL taken. */
void bdrv_dirty_bitmap_enable_successor(BdrvDirtyBitmap *bitmap)
{
-assert(bitmap->mutex == bitmap->successor->mutex);
-qemu_mutex_lock(bitmap->mutex);
+assert(bitmap->bs == bitmap->successor->bs);
+bdrv_dirty_bitmaps_lock(bitmap->bs);
bdrv_enable_dirty_bitmap_locked(bitmap->successor);
-qemu_mutex_unlock(bitmap->mutex);
+bdrv_dirty_bitmaps_unlock(bitmap->bs);
}
/* Called within bdrv_dirty_bitmap_lock..unlock and with BQL taken. */
@@ -361,9 +359,9 @@ BdrvDirtyBitmap *bdrv_reclaim_dirty_bitmap(BdrvDirtyBitmap
*parent,
{
BdrvDirtyBitmap *ret;
-qemu_mutex_lock(parent->mutex);
+bdrv_dirty_bitmaps_lock(parent->bs);
ret = bdrv_reclaim_dirty_bitmap_locked(parent, errp);
-qemu_mutex_unlock(parent->mutex);
+bdrv_dirty_bitmaps_unlock(parent->bs);
return ret;
}
@@ -543,16 +541,16 @@ bool bdrv_can_store_new_dirty_bitmap(BlockDriverState
*bs, const char *name,
void bdrv_disable_dirty_bitmap(BdrvDirtyBitmap *bitmap)
{
-bdrv_dirty_bitmap_lock(bitmap);
+bdrv_dirty_bitmaps_lock(bitmap->bs);
bitmap->disabled = true;
-bdrv_dirty_bitmap_unlock(bitmap);
+bdrv_dirty_bitmaps_unlock(bitmap->bs);
}
void bdrv_enable_dirty_bitmap(BdrvDirtyBitmap *bitmap)
{
-bdrv_dirty_bitmap_lock(bitmap);
+bdrv_dirty_bitmaps_lock(bitmap->bs);
bdrv_enable_dirty_bitmap_locked(bitmap);
-bdrv_dirty_bitmap_unlock(bitmap);
+bdrv_dirty_bitmaps_unlock(bitmap->bs);
}
BlockDirtyInfoList *bdrv_query_dirty_bitmaps(BlockDriverState *bs)
@@ -593,9 +591,9 @@ bool bdrv_dirty_bitmap_get_locked(BdrvDirtyBitmap *bitmap,
int64_t offset)
bool bdrv_dirty_bitmap_get(BdrvDirtyBitmap *bitmap, int64_t offset)
{
bool ret;
-bdrv_dirty_bitmap_lock(bitmap);
+bdrv_dirty_bitmaps_lock(bitmap->bs);
ret = bdrv_dirty_bitmap_get_locked(bitmap, offset);
-bdrv_dirty_bitmap_unlock(bitmap);
+bdrv_dirty_bitmaps_unlock(bitmap->bs);
return ret;
}
@@ -660,9 +658,9 @@ void bdrv_set_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap,
void bdrv_set_dirty_bitmap(BdrvDirtyBitmap *bitmap,
int64_t offset, int64_t bytes)
{
-bdrv_dirty_bitmap_lock(bitmap);
+bdrv_dirty_bitmaps_lock(bitmap->bs);
bdrv_set_dirty_bitmap_locked(bitmap, offset, bytes);
-bdrv_dirty_bitmap_unlock(bitmap);
+bdrv_dirty_bitmaps_unlock(bitmap->bs);
}
/* Called within bdrv_dirty_bitmap_lock..unl
[libvirt] [PULL v3 01/19] util/hbitmap: strict hbitmap_reset
From: Vladimir Sementsov-Ogievskiy hbitmap_reset has an unobvious property: it rounds requested region up. It may provoke bugs, like in recently fixed write-blocking mode of mirror: user calls reset on unaligned region, not keeping in mind that there are possible unrelated dirty bytes, covered by rounded-up region and information of this unrelated "dirtiness" will be lost. Make hbitmap_reset strict: assert that arguments are aligned, allowing only one exception when @start + @count == hb->orig_size. It's needed to comfort users of hbitmap_next_dirty_area, which cares about hb->orig_size. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-Id: <20190806152611.280389-1-vsement...@virtuozzo.com> [Maintainer edit: Max's suggestions from on-list. --js] [Maintainer edit: Eric's suggestion for aligned macro. --js] Signed-off-by: John Snow --- include/qemu/hbitmap.h | 5 + tests/test-hbitmap.c | 2 +- util/hbitmap.c | 4 3 files changed, 10 insertions(+), 1 deletion(-) diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h index 4afbe6292e..1bf944ca3d 100644 --- a/include/qemu/hbitmap.h +++ b/include/qemu/hbitmap.h @@ -132,6 +132,11 @@ void hbitmap_set(HBitmap *hb, uint64_t start, uint64_t count); * @count: Number of bits to reset. * * Reset a consecutive range of bits in an HBitmap. + * @start and @count must be aligned to bitmap granularity. The only exception + * is resetting the tail of the bitmap: @count may be equal to hb->orig_size - + * @start, in this case @count may be not aligned. The sum of @start + @count is + * allowed to be greater than hb->orig_size, but only if @start < hb->orig_size + * and @start + @count = ALIGN_UP(hb->orig_size, granularity). */ void hbitmap_reset(HBitmap *hb, uint64_t start, uint64_t count); diff --git a/tests/test-hbitmap.c b/tests/test-hbitmap.c index eed5d288cb..e1f867085f 100644 --- a/tests/test-hbitmap.c +++ b/tests/test-hbitmap.c @@ -423,7 +423,7 @@ static void test_hbitmap_granularity(TestHBitmapData *data, hbitmap_test_check(data, 0); hbitmap_test_set(data, 0, 3); g_assert_cmpint(hbitmap_count(data->hb), ==, 4); -hbitmap_test_reset(data, 0, 1); +hbitmap_test_reset(data, 0, 2); g_assert_cmpint(hbitmap_count(data->hb), ==, 2); } diff --git a/util/hbitmap.c b/util/hbitmap.c index fd44c897ab..66db87c6ff 100644 --- a/util/hbitmap.c +++ b/util/hbitmap.c @@ -476,6 +476,10 @@ void hbitmap_reset(HBitmap *hb, uint64_t start, uint64_t count) /* Compute range in the last layer. */ uint64_t first; uint64_t last = start + count - 1; +uint64_t gran = 1ULL << hb->granularity; + +assert(QEMU_IS_ALIGNED(start, gran)); +assert(QEMU_IS_ALIGNED(count, gran) || (start + count == hb->orig_size)); trace_hbitmap_reset(hb, start, count, start >> hb->granularity, last >> hb->granularity); -- 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL v3 04/19] block/qcow2: proper locking on bitmap add/remove paths
From: Vladimir Sementsov-Ogievskiy
qmp_block_dirty_bitmap_add and do_block_dirty_bitmap_remove do acquire
aio context since 0a6c86d024c52b. But this is not enough: we also must
lock qcow2 mutex when access in-image metadata. Especially it concerns
freeing qcow2 clusters.
To achieve this, move qcow2_can_store_new_dirty_bitmap and
qcow2_remove_persistent_dirty_bitmap to coroutine context.
Since we work in coroutines in correct aio context, we don't need
context acquiring in blockdev.c anymore, drop it.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: John Snow
Message-id: 20190920082543.23444-4-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
block/qcow2.h | 11 ++--
include/block/block_int.h | 10 ++--
block/dirty-bitmap.c | 102 +++---
block/qcow2-bitmap.c | 24 ++---
block/qcow2.c | 5 +-
blockdev.c| 27 +++---
6 files changed, 131 insertions(+), 48 deletions(-)
diff --git a/block/qcow2.h b/block/qcow2.h
index 08b4c15dc4..0f3d9b088e 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -746,12 +746,13 @@ int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error
**errp);
int qcow2_truncate_bitmaps_check(BlockDriverState *bs, Error **errp);
void qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp);
int qcow2_reopen_bitmaps_ro(BlockDriverState *bs, Error **errp);
-bool qcow2_can_store_new_dirty_bitmap(BlockDriverState *bs,
- const char *name,
- uint32_t granularity,
- Error **errp);
-int qcow2_remove_persistent_dirty_bitmap(BlockDriverState *bs, const char
*name,
+bool qcow2_co_can_store_new_dirty_bitmap(BlockDriverState *bs,
+ const char *name,
+ uint32_t granularity,
Error **errp);
+int qcow2_co_remove_persistent_dirty_bitmap(BlockDriverState *bs,
+const char *name,
+Error **errp);
ssize_t coroutine_fn
qcow2_co_compress(BlockDriverState *bs, void *dest, size_t dest_size,
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 6b511dd889..32fb493cbb 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -553,13 +553,13 @@ struct BlockDriver {
* field of BlockDirtyBitmap's in case of success.
*/
int (*bdrv_reopen_bitmaps_rw)(BlockDriverState *bs, Error **errp);
-bool (*bdrv_can_store_new_dirty_bitmap)(BlockDriverState *bs,
-const char *name,
-uint32_t granularity,
-Error **errp);
-int (*bdrv_remove_persistent_dirty_bitmap)(BlockDriverState *bs,
+bool (*bdrv_co_can_store_new_dirty_bitmap)(BlockDriverState *bs,
const char *name,
+ uint32_t granularity,
Error **errp);
+int (*bdrv_co_remove_persistent_dirty_bitmap)(BlockDriverState *bs,
+ const char *name,
+ Error **errp);
/**
* Register/unregister a buffer for I/O. For example, when the driver is
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index d1ae2e1922..03e0872b97 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -26,6 +26,7 @@
#include "trace.h"
#include "block/block_int.h"
#include "block/blockjob.h"
+#include "qemu/main-loop.h"
struct BdrvDirtyBitmap {
QemuMutex *mutex;
@@ -455,18 +456,59 @@ void bdrv_release_named_dirty_bitmaps(BlockDriverState
*bs)
* not fail.
* This function doesn't release corresponding BdrvDirtyBitmap.
*/
+static int coroutine_fn
+bdrv_co_remove_persistent_dirty_bitmap(BlockDriverState *bs, const char *name,
+ Error **errp)
+{
+if (bs->drv && bs->drv->bdrv_co_remove_persistent_dirty_bitmap) {
+return bs->drv->bdrv_co_remove_persistent_dirty_bitmap(bs, name, errp);
+}
+
+return 0;
+}
+
+typedef struct BdrvRemovePersistentDirtyBitmapCo {
+BlockDriverState *bs;
+const char *name;
+Error **errp;
+int ret;
+} BdrvRemovePersistentDirtyBitmapCo;
+
+static void coroutine_fn
+bdrv_co_remove_persistent_dirty_bitmap_entry(void *opaque)
+{
+BdrvRemovePersistentDirtyBitmapCo *s = opaque;
+
+s->ret = bdrv_co_remove_persistent_dirty_bitmap(s->bs, s->name, s->errp);
+aio_wait_kick();
+}
+
int bdrv_remove_persistent_dirty_bitmap(BlockDriverState *bs, const char *name,
Error **errp)
{
-
[libvirt] [PULL v3 05/19] block/dirty-bitmap: drop meta
From: Vladimir Sementsov-Ogievskiy
Drop meta bitmaps, as they are unused.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: John Snow
Message-id: 20190916141911.5255-2-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
include/block/dirty-bitmap.h | 5
block/dirty-bitmap.c | 46
2 files changed, 51 deletions(-)
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index 07503b03b5..973056778a 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -18,9 +18,6 @@ BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverState
*bs,
uint32_t granularity,
const char *name,
Error **errp);
-void bdrv_create_meta_dirty_bitmap(BdrvDirtyBitmap *bitmap,
- int chunk_size);
-void bdrv_release_meta_dirty_bitmap(BdrvDirtyBitmap *bitmap);
int bdrv_dirty_bitmap_create_successor(BlockDriverState *bs,
BdrvDirtyBitmap *bitmap,
Error **errp);
@@ -54,7 +51,6 @@ void bdrv_set_dirty_bitmap(BdrvDirtyBitmap *bitmap,
int64_t offset, int64_t bytes);
void bdrv_reset_dirty_bitmap(BdrvDirtyBitmap *bitmap,
int64_t offset, int64_t bytes);
-BdrvDirtyBitmapIter *bdrv_dirty_meta_iter_new(BdrvDirtyBitmap *bitmap);
BdrvDirtyBitmapIter *bdrv_dirty_iter_new(BdrvDirtyBitmap *bitmap);
void bdrv_dirty_iter_free(BdrvDirtyBitmapIter *iter);
@@ -96,7 +92,6 @@ void bdrv_reset_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap,
int64_t bdrv_dirty_iter_next(BdrvDirtyBitmapIter *iter);
void bdrv_set_dirty_iter(BdrvDirtyBitmapIter *hbi, int64_t offset);
int64_t bdrv_get_dirty_count(BdrvDirtyBitmap *bitmap);
-int64_t bdrv_get_meta_dirty_count(BdrvDirtyBitmap *bitmap);
void bdrv_dirty_bitmap_truncate(BlockDriverState *bs, int64_t bytes);
bool bdrv_dirty_bitmap_readonly(const BdrvDirtyBitmap *bitmap);
bool bdrv_has_readonly_bitmaps(BlockDriverState *bs);
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 03e0872b97..4ecf18d5df 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -31,7 +31,6 @@
struct BdrvDirtyBitmap {
QemuMutex *mutex;
HBitmap *bitmap;/* Dirty bitmap implementation */
-HBitmap *meta; /* Meta dirty bitmap */
bool busy; /* Bitmap is busy, it can't be used via QMP */
BdrvDirtyBitmap *successor; /* Anonymous child, if any. */
char *name; /* Optional non-empty unique ID */
@@ -127,36 +126,6 @@ BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverState
*bs,
return bitmap;
}
-/* bdrv_create_meta_dirty_bitmap
- *
- * Create a meta dirty bitmap that tracks the changes of bits in @bitmap. I.e.
- * when a dirty status bit in @bitmap is changed (either from reset to set or
- * the other way around), its respective meta dirty bitmap bit will be marked
- * dirty as well.
- *
- * @bitmap: the block dirty bitmap for which to create a meta dirty bitmap.
- * @chunk_size: how many bytes of bitmap data does each bit in the meta bitmap
- * track.
- */
-void bdrv_create_meta_dirty_bitmap(BdrvDirtyBitmap *bitmap,
- int chunk_size)
-{
-assert(!bitmap->meta);
-qemu_mutex_lock(bitmap->mutex);
-bitmap->meta = hbitmap_create_meta(bitmap->bitmap,
- chunk_size * BITS_PER_BYTE);
-qemu_mutex_unlock(bitmap->mutex);
-}
-
-void bdrv_release_meta_dirty_bitmap(BdrvDirtyBitmap *bitmap)
-{
-assert(bitmap->meta);
-qemu_mutex_lock(bitmap->mutex);
-hbitmap_free_meta(bitmap->bitmap);
-bitmap->meta = NULL;
-qemu_mutex_unlock(bitmap->mutex);
-}
-
int64_t bdrv_dirty_bitmap_size(const BdrvDirtyBitmap *bitmap)
{
return bitmap->size;
@@ -320,7 +289,6 @@ static void
bdrv_release_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap)
assert(!bitmap->active_iterators);
assert(!bdrv_dirty_bitmap_busy(bitmap));
assert(!bdrv_dirty_bitmap_has_successor(bitmap));
-assert(!bitmap->meta);
QLIST_REMOVE(bitmap, list);
hbitmap_free(bitmap->bitmap);
g_free(bitmap->name);
@@ -666,15 +634,6 @@ BdrvDirtyBitmapIter *bdrv_dirty_iter_new(BdrvDirtyBitmap
*bitmap)
return iter;
}
-BdrvDirtyBitmapIter *bdrv_dirty_meta_iter_new(BdrvDirtyBitmap *bitmap)
-{
-BdrvDirtyBitmapIter *iter = g_new(BdrvDirtyBitmapIter, 1);
-hbitmap_iter_init(&iter->hbi, bitmap->meta, 0);
-iter->bitmap = bitmap;
-bitmap->active_iterators++;
-return iter;
-}
-
void bdrv_dirty_iter_free(BdrvDirtyBitmapIter *iter)
{
if (!iter) {
@@ -821,11 +780,6 @@ int64_t bdrv_get_dirty_count(BdrvDirtyBitmap *bitmap)
return hbitmap_count(bitmap->bitmap);
}
-int64_t bdrv_get_meta_dir
[libvirt] [PULL v3 12/19] block/qcow2-bitmap: get rid of bdrv_has_changed_persistent_bitmaps
From: Vladimir Sementsov-Ogievskiy
Firstly, no reason to optimize failure path. Then, function name is
ambiguous: it checks for readonly and similar things, but someone may
think that it will ignore normal bitmaps which was just unchanged, and
this is in bad relation with the fact that we should drop IN_USE flag
for unchanged bitmaps in the image.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: John Snow
Message-id: 20190927122355.7344-5-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
include/block/dirty-bitmap.h | 1 -
block/dirty-bitmap.c | 12
block/qcow2-bitmap.c | 23 +--
3 files changed, 13 insertions(+), 23 deletions(-)
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index 257f0f6704..958e7474fb 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -95,7 +95,6 @@ bool bdrv_has_readonly_bitmaps(BlockDriverState *bs);
bool bdrv_dirty_bitmap_get_autoload(const BdrvDirtyBitmap *bitmap);
bool bdrv_dirty_bitmap_get_persistence(BdrvDirtyBitmap *bitmap);
bool bdrv_dirty_bitmap_inconsistent(const BdrvDirtyBitmap *bitmap);
-bool bdrv_has_changed_persistent_bitmaps(BlockDriverState *bs);
BdrvDirtyBitmap *bdrv_dirty_bitmap_first(BlockDriverState *bs);
BdrvDirtyBitmap *bdrv_dirty_bitmap_next(BdrvDirtyBitmap *bitmap);
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 6065db8094..4bbb251b2c 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -839,18 +839,6 @@ bool bdrv_dirty_bitmap_inconsistent(const BdrvDirtyBitmap
*bitmap)
return bitmap->inconsistent;
}
-bool bdrv_has_changed_persistent_bitmaps(BlockDriverState *bs)
-{
-BdrvDirtyBitmap *bm;
-QLIST_FOREACH(bm, &bs->dirty_bitmaps, list) {
-if (bm->persistent && !bm->readonly && !bm->skip_store) {
-return true;
-}
-}
-
-return false;
-}
-
BdrvDirtyBitmap *bdrv_dirty_bitmap_first(BlockDriverState *bs)
{
return QLIST_FIRST(&bs->dirty_bitmaps);
diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
index 99812b418b..6dfc083548 100644
--- a/block/qcow2-bitmap.c
+++ b/block/qcow2-bitmap.c
@@ -1464,16 +1464,7 @@ void
qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp)
Qcow2Bitmap *bm;
QSIMPLEQ_HEAD(, Qcow2BitmapTable) drop_tables;
Qcow2BitmapTable *tb, *tb_next;
-
-if (!bdrv_has_changed_persistent_bitmaps(bs)) {
-/* nothing to do */
-return;
-}
-
-if (!can_write(bs)) {
-error_setg(errp, "No write access");
-return;
-}
+bool need_write = false;
QSIMPLEQ_INIT(&drop_tables);
@@ -1499,6 +1490,8 @@ void
qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp)
continue;
}
+need_write = true;
+
if (check_constraints_on_bitmap(bs, name, granularity, errp) < 0) {
error_prepend(errp, "Bitmap '%s' doesn't satisfy the constraints:
",
name);
@@ -1537,6 +1530,15 @@ void
qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp)
bm->dirty_bitmap = bitmap;
}
+if (!need_write) {
+goto success;
+}
+
+if (!can_write(bs)) {
+error_setg(errp, "No write access");
+goto fail;
+}
+
/* allocate clusters and store bitmaps */
QSIMPLEQ_FOREACH(bm, bm_list, entry) {
if (bm->dirty_bitmap == NULL) {
@@ -1578,6 +1580,7 @@ void
qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp)
bdrv_release_dirty_bitmap(bm->dirty_bitmap);
}
+success:
bitmap_list_free(bm_list);
return;
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL v3 08/19] block/dirty-bitmap: refactor bdrv_dirty_bitmap_next
From: Vladimir Sementsov-Ogievskiy
bdrv_dirty_bitmap_next is always used in same pattern. So, split it
into _next and _first, instead of combining two functions into one and
add FOR_EACH_DIRTY_BITMAP macro.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: John Snow
Message-id: 20190916141911.5255-5-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
include/block/dirty-bitmap.h | 9 +++--
block.c| 4 +---
block/dirty-bitmap.c | 11 +++
block/qcow2-bitmap.c | 8 ++--
migration/block-dirty-bitmap.c | 4 +---
5 files changed, 18 insertions(+), 18 deletions(-)
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index 2f9b088e11..257f0f6704 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -96,8 +96,13 @@ bool bdrv_dirty_bitmap_get_autoload(const BdrvDirtyBitmap
*bitmap);
bool bdrv_dirty_bitmap_get_persistence(BdrvDirtyBitmap *bitmap);
bool bdrv_dirty_bitmap_inconsistent(const BdrvDirtyBitmap *bitmap);
bool bdrv_has_changed_persistent_bitmaps(BlockDriverState *bs);
-BdrvDirtyBitmap *bdrv_dirty_bitmap_next(BlockDriverState *bs,
-BdrvDirtyBitmap *bitmap);
+
+BdrvDirtyBitmap *bdrv_dirty_bitmap_first(BlockDriverState *bs);
+BdrvDirtyBitmap *bdrv_dirty_bitmap_next(BdrvDirtyBitmap *bitmap);
+#define FOR_EACH_DIRTY_BITMAP(bs, bitmap) \
+for (bitmap = bdrv_dirty_bitmap_first(bs); bitmap; \
+ bitmap = bdrv_dirty_bitmap_next(bitmap))
+
char *bdrv_dirty_bitmap_sha256(const BdrvDirtyBitmap *bitmap, Error **errp);
int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap *bitmap, uint64_t offset,
uint64_t bytes);
diff --git a/block.c b/block.c
index d19a4781a3..5721441697 100644
--- a/block.c
+++ b/block.c
@@ -5390,9 +5390,7 @@ static void coroutine_fn
bdrv_co_invalidate_cache(BlockDriverState *bs,
}
}
-for (bm = bdrv_dirty_bitmap_next(bs, NULL); bm;
- bm = bdrv_dirty_bitmap_next(bs, bm))
-{
+FOR_EACH_DIRTY_BITMAP(bs, bm) {
bdrv_dirty_bitmap_skip_store(bm, false);
}
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 4e5c87a907..6065db8094 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -851,11 +851,14 @@ bool bdrv_has_changed_persistent_bitmaps(BlockDriverState
*bs)
return false;
}
-BdrvDirtyBitmap *bdrv_dirty_bitmap_next(BlockDriverState *bs,
-BdrvDirtyBitmap *bitmap)
+BdrvDirtyBitmap *bdrv_dirty_bitmap_first(BlockDriverState *bs)
{
-return bitmap == NULL ? QLIST_FIRST(&bs->dirty_bitmaps) :
-QLIST_NEXT(bitmap, list);
+return QLIST_FIRST(&bs->dirty_bitmaps);
+}
+
+BdrvDirtyBitmap *bdrv_dirty_bitmap_next(BdrvDirtyBitmap *bitmap)
+{
+return QLIST_NEXT(bitmap, list);
}
char *bdrv_dirty_bitmap_sha256(const BdrvDirtyBitmap *bitmap, Error **errp)
diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
index 687087d2bc..99812b418b 100644
--- a/block/qcow2-bitmap.c
+++ b/block/qcow2-bitmap.c
@@ -1488,9 +1488,7 @@ void
qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp)
}
/* check constraints and names */
-for (bitmap = bdrv_dirty_bitmap_next(bs, NULL); bitmap != NULL;
- bitmap = bdrv_dirty_bitmap_next(bs, bitmap))
-{
+FOR_EACH_DIRTY_BITMAP(bs, bitmap) {
const char *name = bdrv_dirty_bitmap_name(bitmap);
uint32_t granularity = bdrv_dirty_bitmap_granularity(bitmap);
Qcow2Bitmap *bm;
@@ -1610,9 +1608,7 @@ int qcow2_reopen_bitmaps_ro(BlockDriverState *bs, Error
**errp)
return -EINVAL;
}
-for (bitmap = bdrv_dirty_bitmap_next(bs, NULL); bitmap != NULL;
- bitmap = bdrv_dirty_bitmap_next(bs, bitmap))
-{
+FOR_EACH_DIRTY_BITMAP(bs, bitmap) {
if (bdrv_dirty_bitmap_get_persistence(bitmap)) {
bdrv_dirty_bitmap_set_readonly(bitmap, true);
}
diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c
index 793f249aa5..7eafface61 100644
--- a/migration/block-dirty-bitmap.c
+++ b/migration/block-dirty-bitmap.c
@@ -283,9 +283,7 @@ static int init_dirty_bitmap_migration(void)
for (bs = bdrv_next_all_states(NULL); bs; bs = bdrv_next_all_states(bs)) {
const char *name = bdrv_get_device_or_node_name(bs);
-for (bitmap = bdrv_dirty_bitmap_next(bs, NULL); bitmap;
- bitmap = bdrv_dirty_bitmap_next(bs, bitmap))
-{
+FOR_EACH_DIRTY_BITMAP(bs, bitmap) {
if (!bdrv_dirty_bitmap_name(bitmap)) {
continue;
}
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL v3 11/19] iotests: add test-case to 165 to test reopening qcow2 bitmaps to RW
From: Vladimir Sementsov-Ogievskiy
Reopening bitmaps to RW was broken prior to previous commit. Check that
it works now.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Message-id: 20190927122355.7344-4-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
tests/qemu-iotests/165 | 57 --
tests/qemu-iotests/165.out | 4 +--
2 files changed, 57 insertions(+), 4 deletions(-)
diff --git a/tests/qemu-iotests/165 b/tests/qemu-iotests/165
index 5650dc7c87..951ea011a2 100755
--- a/tests/qemu-iotests/165
+++ b/tests/qemu-iotests/165
@@ -43,10 +43,10 @@ class TestPersistentDirtyBitmap(iotests.QMPTestCase):
os.remove(disk)
def mkVm(self):
-return iotests.VM().add_drive(disk)
+return iotests.VM().add_drive(disk, opts='node-name=node0')
def mkVmRo(self):
-return iotests.VM().add_drive(disk, opts='readonly=on')
+return iotests.VM().add_drive(disk, opts='readonly=on,node-name=node0')
def getSha256(self):
result = self.vm.qmp('x-debug-block-dirty-bitmap-sha256',
@@ -102,6 +102,59 @@ class TestPersistentDirtyBitmap(iotests.QMPTestCase):
self.vm.shutdown()
+def test_reopen_rw(self):
+self.vm = self.mkVm()
+self.vm.launch()
+self.qmpAddBitmap()
+
+# Calculate hashes
+
+self.writeRegions(regions1)
+sha256_1 = self.getSha256()
+
+self.writeRegions(regions2)
+sha256_2 = self.getSha256()
+assert sha256_1 != sha256_2 # Otherwise, it's not very interesting.
+
+result = self.vm.qmp('block-dirty-bitmap-clear', node='drive0',
+ name='bitmap0')
+self.assert_qmp(result, 'return', {})
+
+# Start with regions1
+
+self.writeRegions(regions1)
+assert sha256_1 == self.getSha256()
+
+self.vm.shutdown()
+
+self.vm = self.mkVmRo()
+self.vm.launch()
+
+assert sha256_1 == self.getSha256()
+
+# Check that we are in RO mode and can't modify bitmap.
+self.writeRegions(regions2)
+assert sha256_1 == self.getSha256()
+
+# Reopen to RW
+result = self.vm.qmp('x-blockdev-reopen', **{
+'node-name': 'node0',
+'driver': iotests.imgfmt,
+'file': {
+'driver': 'file',
+'filename': disk
+},
+'read-only': False
+})
+self.assert_qmp(result, 'return', {})
+
+# Check that bitmap is reopened to RW and we can write to it.
+self.writeRegions(regions2)
+assert sha256_2 == self.getSha256()
+
+self.vm.shutdown()
+
+
if __name__ == '__main__':
iotests.main(supported_fmts=['qcow2'],
supported_protocols=['file'])
diff --git a/tests/qemu-iotests/165.out b/tests/qemu-iotests/165.out
index ae1213e6f8..fbc63e62f8 100644
--- a/tests/qemu-iotests/165.out
+++ b/tests/qemu-iotests/165.out
@@ -1,5 +1,5 @@
-.
+..
--
-Ran 1 tests
+Ran 2 tests
OK
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL v3 09/19] block: switch reopen queue from QSIMPLEQ to QTAILQ
From: Vladimir Sementsov-Ogievskiy
We'll need reverse-foreach in the following commit, QTAILQ support it,
so move to QTAILQ.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
Message-id: 20190927122355.7344-2-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
include/block/block.h | 2 +-
block.c | 24
2 files changed, 13 insertions(+), 13 deletions(-)
diff --git a/include/block/block.h b/include/block/block.h
index 792bb826db..89606bd9f8 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -195,7 +195,7 @@ typedef struct HDGeometry {
#define BDRV_BLOCK_EOF 0x20
#define BDRV_BLOCK_RECURSE 0x40
-typedef QSIMPLEQ_HEAD(BlockReopenQueue, BlockReopenQueueEntry)
BlockReopenQueue;
+typedef QTAILQ_HEAD(BlockReopenQueue, BlockReopenQueueEntry) BlockReopenQueue;
typedef struct BDRVReopenState {
BlockDriverState *bs;
diff --git a/block.c b/block.c
index 5721441697..0347632c6c 100644
--- a/block.c
+++ b/block.c
@@ -1719,7 +1719,7 @@ typedef struct BlockReopenQueueEntry {
bool prepared;
bool perms_checked;
BDRVReopenState state;
- QSIMPLEQ_ENTRY(BlockReopenQueueEntry) entry;
+ QTAILQ_ENTRY(BlockReopenQueueEntry) entry;
} BlockReopenQueueEntry;
/*
@@ -1732,7 +1732,7 @@ static int bdrv_reopen_get_flags(BlockReopenQueue *q,
BlockDriverState *bs)
BlockReopenQueueEntry *entry;
if (q != NULL) {
-QSIMPLEQ_FOREACH(entry, q, entry) {
+QTAILQ_FOREACH(entry, q, entry) {
if (entry->state.bs == bs) {
return entry->state.flags;
}
@@ -3249,7 +3249,7 @@ static bool bdrv_recurse_has_child(BlockDriverState *bs,
* Adds a BlockDriverState to a simple queue for an atomic, transactional
* reopen of multiple devices.
*
- * bs_queue can either be an existing BlockReopenQueue that has had
QSIMPLE_INIT
+ * bs_queue can either be an existing BlockReopenQueue that has had QTAILQ_INIT
* already performed, or alternatively may be NULL a new BlockReopenQueue will
* be created and initialized. This newly created BlockReopenQueue should be
* passed back in for subsequent calls that are intended to be of the same
@@ -3290,7 +3290,7 @@ static BlockReopenQueue
*bdrv_reopen_queue_child(BlockReopenQueue *bs_queue,
if (bs_queue == NULL) {
bs_queue = g_new0(BlockReopenQueue, 1);
-QSIMPLEQ_INIT(bs_queue);
+QTAILQ_INIT(bs_queue);
}
if (!options) {
@@ -3298,7 +3298,7 @@ static BlockReopenQueue
*bdrv_reopen_queue_child(BlockReopenQueue *bs_queue,
}
/* Check if this BlockDriverState is already in the queue */
-QSIMPLEQ_FOREACH(bs_entry, bs_queue, entry) {
+QTAILQ_FOREACH(bs_entry, bs_queue, entry) {
if (bs == bs_entry->state.bs) {
break;
}
@@ -3354,7 +3354,7 @@ static BlockReopenQueue
*bdrv_reopen_queue_child(BlockReopenQueue *bs_queue,
if (!bs_entry) {
bs_entry = g_new0(BlockReopenQueueEntry, 1);
-QSIMPLEQ_INSERT_TAIL(bs_queue, bs_entry, entry);
+QTAILQ_INSERT_TAIL(bs_queue, bs_entry, entry);
} else {
qobject_unref(bs_entry->state.options);
qobject_unref(bs_entry->state.explicit_options);
@@ -3455,7 +3455,7 @@ int bdrv_reopen_multiple(BlockReopenQueue *bs_queue,
Error **errp)
assert(bs_queue != NULL);
-QSIMPLEQ_FOREACH(bs_entry, bs_queue, entry) {
+QTAILQ_FOREACH(bs_entry, bs_queue, entry) {
assert(bs_entry->state.bs->quiesce_counter > 0);
if (bdrv_reopen_prepare(&bs_entry->state, bs_queue, errp)) {
goto cleanup;
@@ -3463,7 +3463,7 @@ int bdrv_reopen_multiple(BlockReopenQueue *bs_queue,
Error **errp)
bs_entry->prepared = true;
}
-QSIMPLEQ_FOREACH(bs_entry, bs_queue, entry) {
+QTAILQ_FOREACH(bs_entry, bs_queue, entry) {
BDRVReopenState *state = &bs_entry->state;
ret = bdrv_check_perm(state->bs, bs_queue, state->perm,
state->shared_perm, NULL, NULL, errp);
@@ -3489,13 +3489,13 @@ int bdrv_reopen_multiple(BlockReopenQueue *bs_queue,
Error **errp)
/* If we reach this point, we have success and just need to apply the
* changes
*/
-QSIMPLEQ_FOREACH(bs_entry, bs_queue, entry) {
+QTAILQ_FOREACH(bs_entry, bs_queue, entry) {
bdrv_reopen_commit(&bs_entry->state);
}
ret = 0;
cleanup_perm:
-QSIMPLEQ_FOREACH_SAFE(bs_entry, bs_queue, entry, next) {
+QTAILQ_FOREACH_SAFE(bs_entry, bs_queue, entry, next) {
BDRVReopenState *state = &bs_entry->state;
if (!bs_entry->perms_checked) {
@@ -3512,7 +3512,7 @@ cleanup_perm:
}
}
cleanup:
-QSIMPLEQ_FOREACH_SAFE(bs_entry, bs_queue, entry, next) {
+QTAILQ_FOREACH_SAFE(bs_entry, bs_queue, entry, next) {
if (ret) {
if (bs_entry->prepared) {
