On 06/27/2018 03:17 AM, Ghanshyam Mann wrote:
---- On Tue, 26 Jun 2018 23:12:30 +0900 Doug Hellmann <d...@doughellmann.com>
wrote ----
> Excerpts from Matthew Treinish's message of 2018-06-26 09:52:09 -0400:
> > On Tue, Jun 26, 2018 at 08:53:21AM -0400, Doug Hellmann wrote:
> > > Excerpts from Andrea Frittoli's message of 2018-06-26 13:35:11 +0100:
> > > > On Tue, 26 Jun 2018, 1:08 pm Thierry Carrez, <thie...@openstack.org>
wrote:
> > > >
> > > > > Dmitry Tantsur wrote:
> > > > > > [...]
> > > > > > My suggestion: tempest has to be compatible with all supported
releases
> > > > > > (of both services and plugins) OR be branched.
> > > > > > [...]
> > > > > I tend to agree with Dmitry... We have a model for things that need
> > > > > release alignment, and that's the cycle-bound series. The reason
tempest
> > > > > is branchless was because there was no compatibility issue. If the
split
> > > > > of tempest plugins introduces a potential incompatibility, then I
would
> > > > > prefer aligning tempest to the existing model rather than introduce
a
> > > > > parallel tempest-specific cycle just so that tempest can stay
> > > > > release-independent...
> > > > >
> > > > > I seem to remember there were drawbacks in branching tempest,
though...
> > > > > Can someone with functioning memory brain cells summarize them
again ?
> > > > >
> > > >
> > > >
> > > > Branchless Tempest enforces api stability across branches.
> > >
> > > I'm sorry, but I'm having a hard time taking this statement seriously
> > > when the current source of tension is that the Tempest API itself
> > > is breaking for its plugins.
> > >
> > > Maybe rather than talking about how to release compatible things
> > > together, we should go back and talk about why Tempest's API is changing
> > > in a way that can't be made backwards-compatible. Can you give some more
> > > detail about that?
> > >
> >
> > Well it's not, if it did that would violate all the stability guarantees
> > provided by Tempest's library and plugin interface. I've not ever heard of
> > these kind of backwards incompatibilities in those interfaces and we go to
> > all effort to make sure we don't break them. Where did the idea that
> > backwards incompatible changes where being introduced come from?
>
> In his original post, gmann said, "There might be some changes in
> Tempest which might not work with older version of Tempest Plugins."
> I was surprised to hear that, but I'm not sure how else to interpret
> that statement.
I did not mean to say that Tempest will introduce the changes in backward
incompatible way which can break plugins. That cannot happen as all plugins and
tempest are branchless and they are being tested with master Tempest so if we
change anything backward incompatible then it break the plugins gate. Even we
have to remove any deprecated interfaces from Tempest, we fix all plugins first
like -
https://review.openstack.org/#/q/topic:remove-support-of-cinder-v1-api+(status:open+OR+status:merged)
What I mean to say here is that adding new or removing deprecated interface in
Tempest might not work with all released version or unreleased Plugins. That
point is from point of view of using Tempest and Plugins in production cloud
testing not gate(where we keep the compatibility). Production Cloud user use
Tempest cycle based version. Pike based Cloud will be tested by Tempest 17.0.0
not latest version (though latest version might work).
This thread is not just for gate testing point of view (which seems to be
always interpreted), this is more for user using Tempest and Plugins for their
cloud testing. I am looping operator mail list also which i forgot in initial
post.
We do not have any tag/release from plugins to know what version of plugin can
work with what version of tempest. For Example If There is new interface
introduced by Tempest 19.0.0 and pluginX start using it. Now it can create
issues for pluginX in both release model 1. plugins with no release (I will
call this PluginNR), 2. plugins with independent release (I will call it
PluginIR).
Users (Not Gate) will face below issues:
- User cannot use PluginNR with Tempest <19.0.0 (where that new interface was
not present). And there is no PluginNR release/tag as this is unreleased and not
branched software.
- User cannot find a PluginIR particular tag/release which can work with tempest
<19.0.0 (where that new interface was not present). Only way for user to make
it work is to manually find out the PluginIR tag/commit before PluginIR started
consuming the new interface.
In these discussions I always think: how is it solved outside of the openstack
world. And the solutions seem to be:
1. for PluginNR - do releases
2. for PluginIR - declare their minimum version of tempest in requirements.txt
Why isn't it sufficient for us?
Dmitry
Let me make it more clear via diagram:
PluginNR
PluginIR
Tempest 19.0.0
Add NewInterface
Use NewInterface
Use NewInterface
Tempest 18.0.0
NewInterface not present
No version of PluginNR
Unknown version of PluginIR
GATE (No Issue as latest things always being tested live ):
OK
OK
User issues:
X (does not
work) Hard to find
compatible version
We need a particular tag from Plugins for OpenStack release, EOL of OpenStack
release like Tempest does so that user can test their old release Cloud in easy
way.
-gmann
>
> > As for this whole thread I don't understand any of the points being
brought up
> > in the original post or any of the follow ons, things seem to have been
confused
> > from the start. The ask from users at the summit was simple. When a new
OpenStack
> > release is pushed we push a tempest release to mark that (the next one
will be
> > 19.0.0 to mark Rocky). Users were complaining that many plugins don't
have a
> > corresponding version to mark support for a new release. So when trying
to run
> > against a rocky cloud you get tempest 19.0.0 and then a bunch of plugins
for
> > various services at different sha1s which have to be manually looked up
based
> > on dates. All users wanted at the summit was a tag for plugins like
tempest
> > does with the first number in:
> >
> > https://docs.openstack.org/tempest/latest/overview.html#release-versioning
> >
> > which didn't seem like a bad idea to me. I'm not sure the best mechanism
to
> > accomplish this, because I agree with much of what plugin maintainers were
> > saying on the thread about wanting to control their own releases. But the
> > desire to make sure users have a tag they can pull for the addition or
> > removal of a supported release makes sense as something a plugin should
do.
>
> We don't coordinate versions across projects anywhere else, for a
> bunch of reasons including the complexity of coordinating the details
> and the confusion it causes when the first version of something is
> 19.0.0. Instead, we list the compatible versions of everything
> together on a series-specific page on releases.o.o. That seems to
> be enough to help anyone wanting to know which versions of tools
> work together. The data is also available in YAML files, so it's easy
> enough to consume by automation.
>
> Would that work for tempest and it's plugins, too?
>
> Is the problem that the versions are not the same, or that some of the
> plugins are not being tagged at all?
>
> Doug
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev