Re: [openstack-dev] [python3] Enabling py37 unit tests

[Corey Bryant]

On Wed, Nov 7, 2018 at 11:12 AM Clark Boylan  wrote:

> On Wed, Nov 7, 2018, at 4:47 AM, Mohammed Naser wrote:
> > On Wed, Nov 7, 2018 at 1:37 PM Doug Hellmann 
> wrote:
> > >
> > > Corey Bryant  writes:
> > >
> > > > On Wed, Oct 10, 2018 at 8:45 AM Corey Bryant <
> corey.bry...@canonical.com>
> > > > wrote:
> > > >
> > > > I'd like to start moving forward with enabling py37 unit tests for a
> subset
> > > > of projects. Rather than putting too much load on infra by enabling
> 3 x py3
> > > > unit tests for every project, this would just focus on enablement of
> py37
> > > > unit tests for a subset of projects in the Stein cycle. And just to
> be
> > > > clear, I would not be disabling any unit tests (such as py35). I'd
> just be
> > > > enabling py37 unit tests.
> > > >
> > > > As some background, this ML thread originally led to updating the
> > > > python3-first governance goal (
> https://review.openstack.org/#/c/610708/)
> > > > but has now led back to this ML thread for a +1 rather than updating
> the
> > > > governance goal.
> > > >
> > > > I'd like to get an official +1 here on the ML from parties such as
> the TC
> > > > and infra in particular but anyone else's input would be welcomed
> too.
> > > > Obviously individual projects would have the right to reject proposed
> > > > changes that enable py37 unit tests. Hopefully they wouldn't, of
> course,
> > > > but they could individually vote that way.
> > > >
> > > > Thanks,
> > > > Corey
> > >
> > > This seems like a good way to start. It lets us make incremental
> > > progress while we take the time to think about the python version
> > > management question more broadly. We can come back to the other
> projects
> > > to add 3.7 jobs and remove 3.5 jobs when we have that plan worked out.
> >
> > What's the impact on the number of consumption in upstream CI node usage?
> >
>
> For period from 2018-10-25 15:16:32,079 to 2018-11-07 15:59:04,994,
> openstack-tox-py35 jobs in aggregate represent 0.73% of our total capacity
> usage.
>
> I don't expect py37 to significantly deviate from that. Again the major
> resource consumption is dominated by a small number of projects/repos/jobs.
> Generally testing outside of that bubble doesn't represent a significant
> resource cost.
>
> I see no problem with adding python 3.7 unit testing from an
> infrastructure perspective.
>
> Clark
>
>
>
Thanks all for the input on this. It seems like we have no objections to
moving forward so I'll plan on getting started soon.

Thanks,
Corey
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Clark Boylan]

On Wed, Nov 7, 2018, at 4:47 AM, Mohammed Naser wrote:
> On Wed, Nov 7, 2018 at 1:37 PM Doug Hellmann  wrote:
> >
> > Corey Bryant  writes:
> >
> > > On Wed, Oct 10, 2018 at 8:45 AM Corey Bryant 
> > > wrote:
> > >
> > > I'd like to start moving forward with enabling py37 unit tests for a 
> > > subset
> > > of projects. Rather than putting too much load on infra by enabling 3 x 
> > > py3
> > > unit tests for every project, this would just focus on enablement of py37
> > > unit tests for a subset of projects in the Stein cycle. And just to be
> > > clear, I would not be disabling any unit tests (such as py35). I'd just be
> > > enabling py37 unit tests.
> > >
> > > As some background, this ML thread originally led to updating the
> > > python3-first governance goal (https://review.openstack.org/#/c/610708/)
> > > but has now led back to this ML thread for a +1 rather than updating the
> > > governance goal.
> > >
> > > I'd like to get an official +1 here on the ML from parties such as the TC
> > > and infra in particular but anyone else's input would be welcomed too.
> > > Obviously individual projects would have the right to reject proposed
> > > changes that enable py37 unit tests. Hopefully they wouldn't, of course,
> > > but they could individually vote that way.
> > >
> > > Thanks,
> > > Corey
> >
> > This seems like a good way to start. It lets us make incremental
> > progress while we take the time to think about the python version
> > management question more broadly. We can come back to the other projects
> > to add 3.7 jobs and remove 3.5 jobs when we have that plan worked out.
> 
> What's the impact on the number of consumption in upstream CI node usage?
> 

For period from 2018-10-25 15:16:32,079 to 2018-11-07 15:59:04,994, 
openstack-tox-py35 jobs in aggregate represent 0.73% of our total capacity 
usage.

I don't expect py37 to significantly deviate from that. Again the major 
resource consumption is dominated by a small number of projects/repos/jobs. 
Generally testing outside of that bubble doesn't represent a significant 
resource cost.

I see no problem with adding python 3.7 unit testing from an infrastructure 
perspective.

Clark

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Chris Dent]

On Tue, 6 Nov 2018, Corey Bryant wrote:


I'd like to get an official +1 here on the ML from parties such as the TC
and infra in particular but anyone else's input would be welcomed too.
Obviously individual projects would have the right to reject proposed
changes that enable py37 unit tests. Hopefully they wouldn't, of course,
but they could individually vote that way.


Speaking as someone on the TC but not "the TC" as well as someone
active in a few projects: +1. As shown elsewhere in the thread the
impact on node consumption and queue lengths shouldn't be a huge
amount and the benefits are high.


From an openstack/placement standpoint, please go for it if nobody

else beats you to it.

To me the benefits are simply that we find bugs sooner. It's bizarre
to me that we even need to think about this. The sooner we find
them, the less they impact people who want to use our code. Will it
cause breakage and extra work us now? Possibly, but it's like making
an early payment on the mortgage: We are saving cost later.

--
Chris Dent   ٩◔̯◔۶   https://anticdent.org/
freenode: cdent tw: @anticdent__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Doug Hellmann]

Mohammed Naser  writes:

> On Wed, Nov 7, 2018 at 2:07 PM Dr. Jens Harbott (frickler)
>  wrote:
>>
>> 2018-11-07 12:47 GMT+00:00 Mohammed Naser :
>> > On Wed, Nov 7, 2018 at 1:37 PM Doug Hellmann  wrote:
>> >>
>> >> Corey Bryant  writes:
>> >>
>> >> > On Wed, Oct 10, 2018 at 8:45 AM Corey Bryant 
>> >> > 
>> >> > wrote:
>> >> >
>> >> > I'd like to start moving forward with enabling py37 unit tests for a 
>> >> > subset
>> >> > of projects. Rather than putting too much load on infra by enabling 3 x 
>> >> > py3
>> >> > unit tests for every project, this would just focus on enablement of 
>> >> > py37
>> >> > unit tests for a subset of projects in the Stein cycle. And just to be
>> >> > clear, I would not be disabling any unit tests (such as py35). I'd just 
>> >> > be
>> >> > enabling py37 unit tests.
>> >> >
>> >> > As some background, this ML thread originally led to updating the
>> >> > python3-first governance goal (https://review.openstack.org/#/c/610708/)
>> >> > but has now led back to this ML thread for a +1 rather than updating the
>> >> > governance goal.
>> >> >
>> >> > I'd like to get an official +1 here on the ML from parties such as the 
>> >> > TC
>> >> > and infra in particular but anyone else's input would be welcomed too.
>> >> > Obviously individual projects would have the right to reject proposed
>> >> > changes that enable py37 unit tests. Hopefully they wouldn't, of course,
>> >> > but they could individually vote that way.
>> >> >
>> >> > Thanks,
>> >> > Corey
>> >>
>> >> This seems like a good way to start. It lets us make incremental
>> >> progress while we take the time to think about the python version
>> >> management question more broadly. We can come back to the other projects
>> >> to add 3.7 jobs and remove 3.5 jobs when we have that plan worked out.
>> >
>> > What's the impact on the number of consumption in upstream CI node usage?
>>
>> I think the relevant metric here will be nodes_used * time_used.
>> nodes_used will increase by one, time_used for usual unit test jobs
>> seems to be < 10 minutes, so I'd think that the total increase in CI
>> usage should be neglegible compared to full tempest or similar jobs
>> that take 1-2 hours.
>
> Indeed it doesn't look too bad:
>
> http://zuul.openstack.org/builds?job_name=openstack-tox-py35
>
> It'll be good to try and aim to transition as quickly as possible to
> avoid extra 'wasted' resources in the infrastructure side

Right, I think we can live with it for a few weeks.

-- 
Doug

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Thomas Goirand]

On 10/11/18 1:35 AM, Goutham Pacha Ravi wrote:
> Thanks Corey for starting this effort. I proposed changes to
> manila repos to use your template [1] [2], but the interpreter's not
> being installed,
> do you need to make any bindep changes to enable the "universe" ppa and 
> install
> python3.7 and python3.7-dev?

Your best luck in Debian based distribution is probably to attempt
installing python3-all *or* python3-dev-all. The -all means all
available versions (so, in case of Debian Sid currently, it will install
both Python 3.6 and 3.7). The -dev-all will also install the -all
package, so you never need *both*. You also don't need the -dev if you
don't have Python packages that needs Python.h (ie: embedded C code in a
Python module).

So, just switch to that, and you're good to go *forever*, independently
of what python version is available in a given OS version! :)

I hope this helps,
Cheers,

Thomas Goirand (zigo)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Mohammed Naser]

On Wed, Nov 7, 2018 at 2:07 PM Dr. Jens Harbott (frickler)
 wrote:
>
> 2018-11-07 12:47 GMT+00:00 Mohammed Naser :
> > On Wed, Nov 7, 2018 at 1:37 PM Doug Hellmann  wrote:
> >>
> >> Corey Bryant  writes:
> >>
> >> > On Wed, Oct 10, 2018 at 8:45 AM Corey Bryant 
> >> > wrote:
> >> >
> >> > I'd like to start moving forward with enabling py37 unit tests for a 
> >> > subset
> >> > of projects. Rather than putting too much load on infra by enabling 3 x 
> >> > py3
> >> > unit tests for every project, this would just focus on enablement of py37
> >> > unit tests for a subset of projects in the Stein cycle. And just to be
> >> > clear, I would not be disabling any unit tests (such as py35). I'd just 
> >> > be
> >> > enabling py37 unit tests.
> >> >
> >> > As some background, this ML thread originally led to updating the
> >> > python3-first governance goal (https://review.openstack.org/#/c/610708/)
> >> > but has now led back to this ML thread for a +1 rather than updating the
> >> > governance goal.
> >> >
> >> > I'd like to get an official +1 here on the ML from parties such as the TC
> >> > and infra in particular but anyone else's input would be welcomed too.
> >> > Obviously individual projects would have the right to reject proposed
> >> > changes that enable py37 unit tests. Hopefully they wouldn't, of course,
> >> > but they could individually vote that way.
> >> >
> >> > Thanks,
> >> > Corey
> >>
> >> This seems like a good way to start. It lets us make incremental
> >> progress while we take the time to think about the python version
> >> management question more broadly. We can come back to the other projects
> >> to add 3.7 jobs and remove 3.5 jobs when we have that plan worked out.
> >
> > What's the impact on the number of consumption in upstream CI node usage?
>
> I think the relevant metric here will be nodes_used * time_used.
> nodes_used will increase by one, time_used for usual unit test jobs
> seems to be < 10 minutes, so I'd think that the total increase in CI
> usage should be neglegible compared to full tempest or similar jobs
> that take 1-2 hours.

Indeed it doesn't look too bad:

http://zuul.openstack.org/builds?job_name=openstack-tox-py35

It'll be good to try and aim to transition as quickly as possible to
avoid extra 'wasted' resources in the infrastructure side

> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Mohammed Naser — vexxhost
-
D. 514-316-8872
D. 800-910-1726 ext. 200
E. mna...@vexxhost.com
W. http://vexxhost.com

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Dr. Jens Harbott (frickler)]

2018-11-07 12:47 GMT+00:00 Mohammed Naser :
> On Wed, Nov 7, 2018 at 1:37 PM Doug Hellmann  wrote:
>>
>> Corey Bryant  writes:
>>
>> > On Wed, Oct 10, 2018 at 8:45 AM Corey Bryant 
>> > wrote:
>> >
>> > I'd like to start moving forward with enabling py37 unit tests for a subset
>> > of projects. Rather than putting too much load on infra by enabling 3 x py3
>> > unit tests for every project, this would just focus on enablement of py37
>> > unit tests for a subset of projects in the Stein cycle. And just to be
>> > clear, I would not be disabling any unit tests (such as py35). I'd just be
>> > enabling py37 unit tests.
>> >
>> > As some background, this ML thread originally led to updating the
>> > python3-first governance goal (https://review.openstack.org/#/c/610708/)
>> > but has now led back to this ML thread for a +1 rather than updating the
>> > governance goal.
>> >
>> > I'd like to get an official +1 here on the ML from parties such as the TC
>> > and infra in particular but anyone else's input would be welcomed too.
>> > Obviously individual projects would have the right to reject proposed
>> > changes that enable py37 unit tests. Hopefully they wouldn't, of course,
>> > but they could individually vote that way.
>> >
>> > Thanks,
>> > Corey
>>
>> This seems like a good way to start. It lets us make incremental
>> progress while we take the time to think about the python version
>> management question more broadly. We can come back to the other projects
>> to add 3.7 jobs and remove 3.5 jobs when we have that plan worked out.
>
> What's the impact on the number of consumption in upstream CI node usage?

I think the relevant metric here will be nodes_used * time_used.
nodes_used will increase by one, time_used for usual unit test jobs
seems to be < 10 minutes, so I'd think that the total increase in CI
usage should be neglegible compared to full tempest or similar jobs
that take 1-2 hours.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Mohammed Naser]

On Wed, Nov 7, 2018 at 1:37 PM Doug Hellmann  wrote:
>
> Corey Bryant  writes:
>
> > On Wed, Oct 10, 2018 at 8:45 AM Corey Bryant 
> > wrote:
> >
> > I'd like to start moving forward with enabling py37 unit tests for a subset
> > of projects. Rather than putting too much load on infra by enabling 3 x py3
> > unit tests for every project, this would just focus on enablement of py37
> > unit tests for a subset of projects in the Stein cycle. And just to be
> > clear, I would not be disabling any unit tests (such as py35). I'd just be
> > enabling py37 unit tests.
> >
> > As some background, this ML thread originally led to updating the
> > python3-first governance goal (https://review.openstack.org/#/c/610708/)
> > but has now led back to this ML thread for a +1 rather than updating the
> > governance goal.
> >
> > I'd like to get an official +1 here on the ML from parties such as the TC
> > and infra in particular but anyone else's input would be welcomed too.
> > Obviously individual projects would have the right to reject proposed
> > changes that enable py37 unit tests. Hopefully they wouldn't, of course,
> > but they could individually vote that way.
> >
> > Thanks,
> > Corey
>
> This seems like a good way to start. It lets us make incremental
> progress while we take the time to think about the python version
> management question more broadly. We can come back to the other projects
> to add 3.7 jobs and remove 3.5 jobs when we have that plan worked out.

What's the impact on the number of consumption in upstream CI node usage?

> Doug
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Mohammed Naser — vexxhost
-
D. 514-316-8872
D. 800-910-1726 ext. 200
E. mna...@vexxhost.com
W. http://vexxhost.com

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Doug Hellmann]

Corey Bryant  writes:

> On Wed, Oct 10, 2018 at 8:45 AM Corey Bryant 
> wrote:
>
> I'd like to start moving forward with enabling py37 unit tests for a subset
> of projects. Rather than putting too much load on infra by enabling 3 x py3
> unit tests for every project, this would just focus on enablement of py37
> unit tests for a subset of projects in the Stein cycle. And just to be
> clear, I would not be disabling any unit tests (such as py35). I'd just be
> enabling py37 unit tests.
>
> As some background, this ML thread originally led to updating the
> python3-first governance goal (https://review.openstack.org/#/c/610708/)
> but has now led back to this ML thread for a +1 rather than updating the
> governance goal.
>
> I'd like to get an official +1 here on the ML from parties such as the TC
> and infra in particular but anyone else's input would be welcomed too.
> Obviously individual projects would have the right to reject proposed
> changes that enable py37 unit tests. Hopefully they wouldn't, of course,
> but they could individually vote that way.
>
> Thanks,
> Corey

This seems like a good way to start. It lets us make incremental
progress while we take the time to think about the python version
management question more broadly. We can come back to the other projects
to add 3.7 jobs and remove 3.5 jobs when we have that plan worked out.

Doug

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Corey Bryant]

On Wed, Oct 10, 2018 at 8:45 AM Corey Bryant 
wrote:

> Hi All,
>
> I'd like to enable py37 unit tests in the gate.
>
> == Background ==
>
> I work on OpenStack packaging for Ubuntu. During the Rocky release (Ubuntu
> Cosmic) I tried to fix py37 bugs upstream as I came across them. There
> ended up being a lot of py37 issues and after a while, due to time
> constraints, I resorted to just opening bugs and disabling py37 unit tests
> that were failing in our package builds. Luckily enough, even though Cosmic
> ships with python3.6 and python3.7, python3.6 ended up being chosen as the
> default for Cosmic.
>
> == Defaulting to python3.7 ==
>
> The next release of Ubuntu opens in just a few weeks. It will default to
> python3.7 and will not include python3.6. My hope is that if I can help
> enable py37 unit tests upstream now, we can get a wider view at fixing
> issues soon.
>
> == Enabling py37 unit tests ==
>
> Ubuntu Bionic (18.04 LTS) has the 3.7.0 interpreter and I have reviews up
> to define the py37 zuul job and templates here:
> https://review.openstack.org/#/c/609066
>
> I'd like to start submitting reviews to projects to enable
> openstack-python37-jobs (or variant) for projects that already have
> openstack-python36-jobs in their .zuul.yaml, zuul.yaml,
> .zuul.d/project.yaml.
>
> == Coinciding work ==
>
> There is python3-first work going on now and I completely understand that
> this is going to cause more work for some projects. It seems that now is as
> good of a time as ever to catch up and test with a recent python3 version.
> I'm sure python3.8 and beyond will be here before we know it.
>
> Any thoughts or concerns?
>
> Thanks,
> Corey
>

I'd like to start moving forward with enabling py37 unit tests for a subset
of projects. Rather than putting too much load on infra by enabling 3 x py3
unit tests for every project, this would just focus on enablement of py37
unit tests for a subset of projects in the Stein cycle. And just to be
clear, I would not be disabling any unit tests (such as py35). I'd just be
enabling py37 unit tests.

As some background, this ML thread originally led to updating the
python3-first governance goal (https://review.openstack.org/#/c/610708/)
but has now led back to this ML thread for a +1 rather than updating the
governance goal.

I'd like to get an official +1 here on the ML from parties such as the TC
and infra in particular but anyone else's input would be welcomed too.
Obviously individual projects would have the right to reject proposed
changes that enable py37 unit tests. Hopefully they wouldn't, of course,
but they could individually vote that way.

Thanks,
Corey
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Corey Bryant]

On Wed, Oct 17, 2018 at 3:43 PM William M Edmonds 
wrote:

>
> Corey Bryant  wrote on 10/15/2018 05:34:24 PM:
> ...
> > From an ubuntu perspective, ubuntu is going to support stein on 18.
> > 04 LTS (3.6) and 19.04 (3.7) only.
> ...
>
> So folks with Ubuntu 16.04 LTS compute nodes will have to upgrade them all
> to 18.04 before upgrading to Stein? Of course this would be a distro
> statement,and would not preclude someone from building their own
> environment from source/pypi on Ubuntu 16.04. And 16.04 is still pretty
> heavily used, right?
>
> All true statements, and the answers to your questions is yes.

> Ubuntu 18.04 LTS is not supported on PowerVM compute nodes, so the PowerVM
> CI will not be able to switch to running under py3 if code that doesn't
> work in py35 is introduced. At least until RHEL 8 comes out, at which point
> we could switch to using that in our CI. But please don't allow such
> changes before the RHEL 8 release.
>
This sounds like an orthogonal problem but maybe I'm confused.

Corey

>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[William M Edmonds]

Corey Bryant  wrote on 10/15/2018 05:34:24 PM:
...
> From an ubuntu perspective, ubuntu is going to support stein on 18.
> 04 LTS (3.6) and 19.04 (3.7) only.
...

So folks with Ubuntu 16.04 LTS compute nodes will have to upgrade them all
to 18.04 before upgrading to Stein? Of course this would be a distro
statement, and would not preclude someone from building their own
environment from source/pypi on Ubuntu 16.04. And 16.04 is still pretty
heavily used, right?

Ubuntu 18.04 LTS is not supported on PowerVM compute nodes, so the PowerVM
CI will not be able to switch to running under py3 if code that doesn't
work in py35 is introduced. At least until RHEL 8 comes out, at which point
we could switch to using that in our CI. But please don't allow such
changes before the RHEL 8 release.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Corey Bryant]

On Tue, Oct 16, 2018 at 10:58 AM Zane Bitter  wrote:

> On 15/10/18 8:00 PM, Monty Taylor wrote:
> > On 10/15/2018 06:39 PM, Zane Bitter wrote:
> >>
> >> In fact, as far as we know the version we have to support in CentOS
> >> may actually be 3.5, which seems like a good reason to keep it working
> >> for long enough that we can find out for sure one way or the other.
> >
> > I certainly hope this is not what ends up happening, but seeing as how I
> > actually do not know - I agree, I cannot discount the possibility that
> > such a thing would happen.
>
> I'm right there with ya.
>
> > That said - until such a time as we get to actually drop python2, I
> > don't see it as an actual issue. The reason being - if we test with 2.7
> > and 3.7 - the things in 3.6 that would break 3.5 get gated by the
> > existence of 2.7 for our codebase.
> >
> > Case in point- the instant 3.6 is our min, I'm going to start replacing
> > every instance of:
> >
> >"foo {bar}".format(bar=bar)
> >
> > in any code I spend time in with:
> >
> >f"foo {bar}"
> >
> > It TOTALLY won't parse on 3.5 ... but it also won't parse on 2.7.
> >
> > If we decide as a community to shift our testing of python3 to be 3.6 -
> > or even 3.7 - as long as we still are testing 2.7, I'd argue we're
> > adequately covered for 3.5.
>
> Yeah, that is a good point. There are only a couple of edge-case
> scenarios where that might not prove to be the case. One is where we
> install a different (or a different version of a) 3rd-party library on
> py2 vs. py3. The other would be where you have some code like:
>
>if six.PY3:
>   some_std_lib_function_added_in_3_6()
>else:
>   py2_code()
>
> It may well be that we can say this is niche enough that we don't care.
>
> In theory the same thing could happen between versions of python3 (e.g.
> if we only tested on 3.5 & 3.7, and not 3.6). There certainly exist
> places where we check the minor version.* However, that's so much less
> likely again that it definitely seems negligible.
>
> * e.g.
>
> https://git.openstack.org/cgit/openstack/oslo.service/tree/oslo_service/service.py#n207
>
> > The day we decide we can drop 2.7 - if we've been testing 3.7 for
> > python3 and it turns out RHEL/CentOS 8 ship with python 3.5, then
> > instead of just deleting all of the openstack-tox-py27 jobs, we'd
> > probably just need to replace them with openstack-tox-py35 jobs, as that
> > would be our new low-water mark.
> >
> > Now, maybe we'll get lucky and RHEL/CentOS 8 will be a future-looking
> > release and will ship with python 3.7 AND so will the corresponding
> > Ubuntu LTS - and we'll get to only care about one release of python for
> > a minute. :)
> >
> > Come on - I can dream, right?
>
> Sure, but let's not get complacent - 3.8 is right around the corner :)
>
>
Btw I confirmed this morning that the plan for 20.04 LTS is to have 3.8, so
it really is around the corner.

Thanks,
Corey

cheers,
> Zane.
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Zane Bitter]

On 15/10/18 8:00 PM, Monty Taylor wrote:

On 10/15/2018 06:39 PM, Zane Bitter wrote:


In fact, as far as we know the version we have to support in CentOS 
may actually be 3.5, which seems like a good reason to keep it working 
for long enough that we can find out for sure one way or the other.


I certainly hope this is not what ends up happening, but seeing as how I 
actually do not know - I agree, I cannot discount the possibility that 
such a thing would happen.


I'm right there with ya.

That said - until such a time as we get to actually drop python2, I 
don't see it as an actual issue. The reason being - if we test with 2.7 
and 3.7 - the things in 3.6 that would break 3.5 get gated by the 
existence of 2.7 for our codebase.


Case in point- the instant 3.6 is our min, I'm going to start replacing 
every instance of:


   "foo {bar}".format(bar=bar)

in any code I spend time in with:

   f"foo {bar}"

It TOTALLY won't parse on 3.5 ... but it also won't parse on 2.7.

If we decide as a community to shift our testing of python3 to be 3.6 - 
or even 3.7 - as long as we still are testing 2.7, I'd argue we're 
adequately covered for 3.5.


Yeah, that is a good point. There are only a couple of edge-case 
scenarios where that might not prove to be the case. One is where we 
install a different (or a different version of a) 3rd-party library on 
py2 vs. py3. The other would be where you have some code like:


  if six.PY3:
 some_std_lib_function_added_in_3_6()
  else:
 py2_code()

It may well be that we can say this is niche enough that we don't care.

In theory the same thing could happen between versions of python3 (e.g. 
if we only tested on 3.5 & 3.7, and not 3.6). There certainly exist 
places where we check the minor version.* However, that's so much less 
likely again that it definitely seems negligible.


* e.g. 
https://git.openstack.org/cgit/openstack/oslo.service/tree/oslo_service/service.py#n207


The day we decide we can drop 2.7 - if we've been testing 3.7 for 
python3 and it turns out RHEL/CentOS 8 ship with python 3.5, then 
instead of just deleting all of the openstack-tox-py27 jobs, we'd 
probably just need to replace them with openstack-tox-py35 jobs, as that 
would be our new low-water mark.


Now, maybe we'll get lucky and RHEL/CentOS 8 will be a future-looking 
release and will ship with python 3.7 AND so will the corresponding 
Ubuntu LTS - and we'll get to only care about one release of python for 
a minute. :)


Come on - I can dream, right?


Sure, but let's not get complacent - 3.8 is right around the corner :)

cheers,
Zane.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Monty Taylor]

On 10/15/2018 08:24 PM, Tim Burke wrote:


On Oct 15, 2018, at 5:00 PM, Monty Taylor > wrote:


If we decide as a community to shift our testing of python3 to be 3.6 
- or even 3.7 - as long as we still are testing 2.7, I'd argue we're 
adequately covered for 3.5.


That's not enough for me to be willing to declare support. I'll grant 
that we'd catch the obvious SyntaxErrors, but that could be achieved 
just as easily (and probably more cheaply, resource-wise) with multiple 
linter jobs. The reason you want unit tests to actually run is to catch 
the not-so-obvious bugs.


For example: there are a bunch of places in Swift's proxy-server where 
we get a JSON response from a backend server, loads() it up, and do some 
work based on it. As I've been trying to get the proxy ported to py3, I 
keep writing json.loads(rest.body.decode()). I'll sometimes get pushback 
from reviewers saying this shouldn't be necessary, and then I need to 
point out that while json.loads() is happy to accept either bytes or 
unicode on both py27 and py36, bytes will cause a TypeError on py35. And 
since https://bugs.python.org/issue17909 was termed an enhancement and 
not a regression (I guess the contract is str-or-unicode, for whatever 
str is?), I'm not expecting a backport.


TLDR; if we want to say that something works, best to actually test that 
it works. I might be willing to believe that py35 and py37 working 
implies that py36 will work, but py27 -> py3x tells me little about 
whether py3w works for any w < x.


Fair point - you've convinced me!


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Tim Burke]

> On Oct 15, 2018, at 5:00 PM, Monty Taylor  wrote:
> 
> If we decide as a community to shift our testing of python3 to be 3.6 - or 
> even 3.7 - as long as we still are testing 2.7, I'd argue we're adequately 
> covered for 3.5.

That's not enough for me to be willing to declare support. I'll grant that we'd 
catch the obvious SyntaxErrors, but that could be achieved just as easily (and 
probably more cheaply, resource-wise) with multiple linter jobs. The reason you 
want unit tests to actually run is to catch the not-so-obvious bugs.

For example: there are a bunch of places in Swift's proxy-server where we get a 
JSON response from a backend server, loads() it up, and do some work based on 
it. As I've been trying to get the proxy ported to py3, I keep writing 
json.loads(rest.body.decode()). I'll sometimes get pushback from reviewers 
saying this shouldn't be necessary, and then I need to point out that while 
json.loads() is happy to accept either bytes or unicode on both py27 and py36, 
bytes will cause a TypeError on py35. And since 
https://bugs.python.org/issue17909  was 
termed an enhancement and not a regression (I guess the contract is 
str-or-unicode, for whatever str is?), I'm not expecting a backport.

TLDR; if we want to say that something works, best to actually test that it 
works. I might be willing to believe that py35 and py37 working implies that 
py36 will work, but py27 -> py3x tells me little about whether py3w works for 
any w < x.

Tim__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Jeremy Stanley]

On 2018-10-15 17:34:24 -0400 (-0400), Corey Bryant wrote:
[...]
> I was assuming the desire to keep 3.5 stein support was for other distros
> that plan to support stein with 3.5.

If there are other distros which are planning to support OpenStack
Stein on Python 3.5, then they should work with us to get testing in
place on systems running their distro. That said, I'd be surprised
if a new LTS server release of a distro targets Python 3.5... it's
already over 3 years old, and 3.3 and 3.4 each got roughly 5 years
before EOL so I'd expect 3.5 to no longer be supported upstream past
September 2020.
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Monty Taylor]

On 10/15/2018 06:39 PM, Zane Bitter wrote:


In fact, as far as we know the version we have to support in CentOS may 
actually be 3.5, which seems like a good reason to keep it working for 
long enough that we can find out for sure one way or the other.


I certainly hope this is not what ends up happening, but seeing as how I 
actually do not know - I agree, I cannot discount the possibility that 
such a thing would happen.


That said - until such a time as we get to actually drop python2, I 
don't see it as an actual issue. The reason being - if we test with 2.7 
and 3.7 - the things in 3.6 that would break 3.5 get gated by the 
existence of 2.7 for our codebase.


Case in point- the instant 3.6 is our min, I'm going to start replacing 
every instance of:


  "foo {bar}".format(bar=bar)

in any code I spend time in with:

  f"foo {bar}"

It TOTALLY won't parse on 3.5 ... but it also won't parse on 2.7.

If we decide as a community to shift our testing of python3 to be 3.6 - 
or even 3.7 - as long as we still are testing 2.7, I'd argue we're 
adequately covered for 3.5.


The day we decide we can drop 2.7 - if we've been testing 3.7 for 
python3 and it turns out RHEL/CentOS 8 ship with python 3.5, then 
instead of just deleting all of the openstack-tox-py27 jobs, we'd 
probably just need to replace them with openstack-tox-py35 jobs, as that 
would be our new low-water mark.


Now, maybe we'll get lucky and RHEL/CentOS 8 will be a future-looking 
release and will ship with python 3.7 AND so will the corresponding 
Ubuntu LTS - and we'll get to only care about one release of python for 
a minute. :)


Come on - I can dream, right?

Monty

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Zane Bitter]

On 15/10/18 4:10 PM, Jeremy Stanley wrote:

On 2018-10-15 15:00:07 -0400 (-0400), Zane Bitter wrote:
[...]

That said, I don't think we should be dropping support/testing for 3.5.
According to:

   https://governance.openstack.org/tc/reference/pti/python.html

3.5 is the only Python3 version that we require all projects to run tests
for.


Until we update it to refer to the version provided by the test
platforms we document at:

https://governance.openstack.org/tc/reference/project-testing-interface.html#linux-distributions


I'm sure we will update it, but as of now we haven't. People shouldn't 
have to guess which TC-maintained documentation is serious and which 
stuff they should just ignore on an ad-hoc basis. If it says 3.5 then 
the answer is 3.5 until somebody submits a patch and the TC approves it.



Out goal is to get everyone running 3.6 unit tests by the end of Stein:

https://governance.openstack.org/tc/goals/stein/python3-first.html#python-3-6-unit-test-jobs

but we explicitly said there that we were not dropping support for 3.5 as
part of the goal, and should continue to do so until we can effect an
orderly transition later.

[...]

We're not dropping support for 3.5 as part of the python3-first
goal, but would be dropping it as part of the switch from Ubuntu
16.04 LTS (which provides Python 3.5) to 18.04 LTS (which provides
Python 3.6). In the past the OpenStack Infra team has prodded us to
follow our documented testing platform policies as new versions
become available, but now with a move to providing infrastructure
services to other OSF projects as well we're on our own to police
this.

We _could_ decide that we're going to start running tests on
multiple versions of Python 3 indefinitely (rather than as a
transitional state during the switch from Ubuntu Xenial to Bionic)


This is inevitable at some point - we say that we'll support both the 
latest release of Ubuntu LTS *and* CentOS. So far that's been irrelevant 
for Python3 because CentOS has only Python2, but we know that the next 
CentOS release will have Python3 and from that point on we will for sure 
be in a situation where we are supporting multiple Python3 versions, not 
always contiguous, for the indefinite future (because the release cycles 
of Ubuntu & CentOS are not aligned in any way).


In fact, as far as we know the version we have to support in CentOS may 
actually be 3.5, which seems like a good reason to keep it working for 
long enough that we can find out for sure one way or the other.



but that does necessarily mean running more jobs. We could also
decide to start targeting different versions of Python than provided
by the distros on which we run our tests (and build it from source
ourselves or something) but I think that's only reasonable if we're
going to also recommend that users deploy OpenStack on top of
custom-compiled Python interpreters rather than the interpreters
provided by server distros like RHEL and Ubuntu.


I am definitely spoiled by Fedora, where I have every version from 3.3 
to 3.7 installed from the distro packages.



So to sum up the above, it's less a question of whether we're
dropping Python 3.5 testing in Stein, and more a question of whether
we're going to continue requiring OpenStack to also be able to run
on Ubuntu 16.04 LTS (which wasn't the latest LTS even at the start
of the cycle).


There's actually another whole level of discussion we probably need to 
have. So far we've talked about unit tests, but running functional tests 
is whole other thing, and one we really do probably want to pick a 
single version of Ubuntu to run on for the sake of the gate (and I'd 
suggest that that version should probably by Bionic, if we can get 
everything working on 3.6 early enough in the cycle).


That process would have been a lot easier if we were earlier on 3.6, so 
I'm grateful to the folks who are already working on 3.7 (which is a 
much more substantial change) to hopefully make this less painful in the 
future.


cheers,
Zane.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Corey Bryant]

On Mon, Oct 15, 2018, 4:11 PM Jeremy Stanley  wrote:

> On 2018-10-15 15:00:07 -0400 (-0400), Zane Bitter wrote:
> [...]
> > That said, I don't think we should be dropping support/testing for 3.5.
> > According to:
> >
> >   https://governance.openstack.org/tc/reference/pti/python.html
> >
> > 3.5 is the only Python3 version that we require all projects to run tests
> > for.
>
> Until we update it to refer to the version provided by the test
> platforms we document at:
>
>
> https://governance.openstack.org/tc/reference/project-testing-interface.html#linux-distributions
>
> > Out goal is to get everyone running 3.6 unit tests by the end of Stein:
> >
> >
> https://governance.openstack.org/tc/goals/stein/python3-first.html#python-3-6-unit-test-jobs
> >
> > but we explicitly said there that we were not dropping support for 3.5 as
> > part of the goal, and should continue to do so until we can effect an
> > orderly transition later.
> [...]
>
> We're not dropping support for 3.5 as part of the python3-first
> goal, but would be dropping it as part of the switch from Ubuntu
> 16.04 LTS (which provides Python 3.5) to 18.04 LTS (which provides
> Python 3.6). In the past the OpenStack Infra team has prodded us to
> follow our documented testing platform policies as new versions
> become available, but now with a move to providing infrastructure
> services to other OSF projects as well we're on our own to police
> this.
>
> We _could_ decide that we're going to start running tests on
> multiple versions of Python 3 indefinitely (rather than as a
> transitional state during the switch from Ubuntu Xenial to Bionic)
> but that does necessarily mean running more jobs. We could also
> decide to start targeting different versions of Python than provided
> by the distros on which we run our tests (and build it from source
> ourselves or something) but I think that's only reasonable if we're
> going to also recommend that users deploy OpenStack on top of
> custom-compiled Python interpreters rather than the interpreters
> provided by server distros like RHEL and Ubuntu.
>
> So to sum up the above, it's less a question of whether we're
> dropping Python 3.5 testing in Stein, and more a question of whether
> we're going to continue requiring OpenStack to also be able to run
> on Ubuntu 16.04 LTS (which wasn't the latest LTS even at the start
> of the cycle).
>

>From an ubuntu perspective, ubuntu is going to support stein on 18.04 LTS
(3.6) and 19.04 (3.7) only. With that said does upstream still want to
ensure stein runs on 16.04 if ubuntu itself has no plans to?

I was assuming the desire to keep 3.5 stein support was for other distros
that plan to support stein with 3.5.

Corey

-- 
> Jeremy Stanley
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Jeremy Stanley]

On 2018-10-15 15:00:07 -0400 (-0400), Zane Bitter wrote:
[...]
> That said, I don't think we should be dropping support/testing for 3.5.
> According to:
> 
>   https://governance.openstack.org/tc/reference/pti/python.html
> 
> 3.5 is the only Python3 version that we require all projects to run tests
> for.

Until we update it to refer to the version provided by the test
platforms we document at:

https://governance.openstack.org/tc/reference/project-testing-interface.html#linux-distributions

> Out goal is to get everyone running 3.6 unit tests by the end of Stein:
> 
> https://governance.openstack.org/tc/goals/stein/python3-first.html#python-3-6-unit-test-jobs
> 
> but we explicitly said there that we were not dropping support for 3.5 as
> part of the goal, and should continue to do so until we can effect an
> orderly transition later.
[...]

We're not dropping support for 3.5 as part of the python3-first
goal, but would be dropping it as part of the switch from Ubuntu
16.04 LTS (which provides Python 3.5) to 18.04 LTS (which provides
Python 3.6). In the past the OpenStack Infra team has prodded us to
follow our documented testing platform policies as new versions
become available, but now with a move to providing infrastructure
services to other OSF projects as well we're on our own to police
this.

We _could_ decide that we're going to start running tests on
multiple versions of Python 3 indefinitely (rather than as a
transitional state during the switch from Ubuntu Xenial to Bionic)
but that does necessarily mean running more jobs. We could also
decide to start targeting different versions of Python than provided
by the distros on which we run our tests (and build it from source
ourselves or something) but I think that's only reasonable if we're
going to also recommend that users deploy OpenStack on top of
custom-compiled Python interpreters rather than the interpreters
provided by server distros like RHEL and Ubuntu.

So to sum up the above, it's less a question of whether we're
dropping Python 3.5 testing in Stein, and more a question of whether
we're going to continue requiring OpenStack to also be able to run
on Ubuntu 16.04 LTS (which wasn't the latest LTS even at the start
of the cycle).
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Corey Bryant]

On Mon, Oct 15, 2018 at 3:15 PM Corey Bryant 
wrote:

>
>
> On Mon, Oct 15, 2018 at 3:01 PM Zane Bitter  wrote:
>
>> On 12/10/18 8:59 AM, Corey Bryant wrote:
>> >
>> >
>> > On Thu, Oct 11, 2018 at 10:19 AM Andreas Jaeger > > > wrote:
>> >
>> > On 10/10/2018 23.10, Jeremy Stanley wrote:
>> >  > I might have only pointed this out on IRC so far, but the
>> >  > expectation is that testing 3.5 and 3.6 at the same time was
>> merely
>> >  > transitional since official OpenStack projects should be moving
>> >  > their testing from Ubuntu Xenial (which provides 3.5) to Ubuntu
>> >  > Bionic (which provides 3.6 and, now, 3.7 as well) during the
>> Stein
>> >  > cycle and so will drop 3.5 testing on master in the process.
>> >
>> > Agreed, this needs some larger communication and explanation on what
>> > to do,
>> >
>> >
>> > The good news is we now have an initial change underway and successful,
>> > dropping py35 and enabling py37:
>> https://review.openstack.org/#/c/609557/
>>
>> Hey Corey,
>> Thanks for getting this underway, it's really important that we keep
>> moving forward (we definitely got behind on the 3.6 transition and are
>> paying for it now).
>>
>> That said, I don't think we should be dropping support/testing for 3.5.
>> According to:
>>
>>https://governance.openstack.org/tc/reference/pti/python.html
>>
>> 3.5 is the only Python3 version that we require all projects to run
>> tests for.
>>
>> Out goal is to get everyone running 3.6 unit tests by the end of Stein:
>>
>>
>>
>> https://governance.openstack.org/tc/goals/stein/python3-first.html#python-3-6-unit-test-jobs
>>
>> but we explicitly said there that we were not dropping support for 3.5
>> as part of the goal, and should continue to do so until we can effect an
>> orderly transition later. Personally, I would see that including waiting
>> for all the 3.5-supporting projects to add 3.6 jobs (which has been
>> blocked up until ~this point, as we are only just now close to getting
>> all of the repos using local Zuul config).
>>
>> I do agree that anything that works on 3.5 and 3.7 will almost certainly
>> work on 3.6, so if you wanted to submit a patch to that goal saying that
>> projects could add a unit test job for *either* 3.6 or 3.7 (in addition
>> to 3.5) then I would probably support that. We could then switch all the
>> 3.5 jobs to 3.6 later when we eventually drop 3.5 support. That would
>> mean we'd only ever run 3 unit test jobs (and 2 once 2.7 is eventually
>> dropped) - for the oldest and newest versions of Python 3 that a project
>> supports.
>>
>
> This seems like a reasonable approach to me. I'll get a review up and we
> can see what others think.
>
>
I have the following up for review to modify the python3-first goal to
allow for python3.6 or python3.7 unit test enablement:
https://review.openstack.org/#/c/610708/

Thanks,
Corey


>> cheers,
>> Zane.
>>
>> [This thread was also discussed on IRC starting here:
>>
>> http://eavesdrop.openstack.org/irclogs/%23openstack-tc/%23openstack-tc.2018-10-15.log.html#t2018-10-15T18:09:05
>> ]
>>
>> > I'm happy to get things moving along and start proposing changes like
>> > this to other projects and communicating with PTLs along the way. Do
>> you
>> > think we need more discussion/communication on this or should I get
>> started?
>> >
>> > Thanks,
>> > Corey
>> >
>> >
>> > Andreas
>> > --
>> >   Andreas Jaeger aj@{suse.com ,opensuse.org
>> > } Twitter: jaegerandi
>> >SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
>> > GF: Felix Imendörffer, Jane Smithard, Graham Norton,
>> > HRB 21284 (AG Nürnberg)
>> >  GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272
>> > A126
>> >
>> >
>> >
>>  __
>> > OpenStack Development Mailing List (not for usage questions)
>> > Unsubscribe:
>> > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> > <
>> http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>> >
>> >
>> __
>> > OpenStack Development Mailing List (not for usage questions)
>> > Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
__
OpenStack Development Mailing List (not for usage 

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Corey Bryant]

On Mon, Oct 15, 2018 at 3:01 PM Zane Bitter  wrote:

> On 12/10/18 8:59 AM, Corey Bryant wrote:
> >
> >
> > On Thu, Oct 11, 2018 at 10:19 AM Andreas Jaeger  > > wrote:
> >
> > On 10/10/2018 23.10, Jeremy Stanley wrote:
> >  > I might have only pointed this out on IRC so far, but the
> >  > expectation is that testing 3.5 and 3.6 at the same time was
> merely
> >  > transitional since official OpenStack projects should be moving
> >  > their testing from Ubuntu Xenial (which provides 3.5) to Ubuntu
> >  > Bionic (which provides 3.6 and, now, 3.7 as well) during the Stein
> >  > cycle and so will drop 3.5 testing on master in the process.
> >
> > Agreed, this needs some larger communication and explanation on what
> > to do,
> >
> >
> > The good news is we now have an initial change underway and successful,
> > dropping py35 and enabling py37:
> https://review.openstack.org/#/c/609557/
>
> Hey Corey,
> Thanks for getting this underway, it's really important that we keep
> moving forward (we definitely got behind on the 3.6 transition and are
> paying for it now).
>
> That said, I don't think we should be dropping support/testing for 3.5.
> According to:
>
>https://governance.openstack.org/tc/reference/pti/python.html
>
> 3.5 is the only Python3 version that we require all projects to run
> tests for.
>
> Out goal is to get everyone running 3.6 unit tests by the end of Stein:
>
>
>
> https://governance.openstack.org/tc/goals/stein/python3-first.html#python-3-6-unit-test-jobs
>
> but we explicitly said there that we were not dropping support for 3.5
> as part of the goal, and should continue to do so until we can effect an
> orderly transition later. Personally, I would see that including waiting
> for all the 3.5-supporting projects to add 3.6 jobs (which has been
> blocked up until ~this point, as we are only just now close to getting
> all of the repos using local Zuul config).
>
> I do agree that anything that works on 3.5 and 3.7 will almost certainly
> work on 3.6, so if you wanted to submit a patch to that goal saying that
> projects could add a unit test job for *either* 3.6 or 3.7 (in addition
> to 3.5) then I would probably support that. We could then switch all the
> 3.5 jobs to 3.6 later when we eventually drop 3.5 support. That would
> mean we'd only ever run 3 unit test jobs (and 2 once 2.7 is eventually
> dropped) - for the oldest and newest versions of Python 3 that a project
> supports.
>

This seems like a reasonable approach to me. I'll get a review up and we
can see what others think.

Thanks,
Corey


> cheers,
> Zane.
>
> [This thread was also discussed on IRC starting here:
>
> http://eavesdrop.openstack.org/irclogs/%23openstack-tc/%23openstack-tc.2018-10-15.log.html#t2018-10-15T18:09:05
> ]
>
> > I'm happy to get things moving along and start proposing changes like
> > this to other projects and communicating with PTLs along the way. Do you
> > think we need more discussion/communication on this or should I get
> started?
> >
> > Thanks,
> > Corey
> >
> >
> > Andreas
> > --
> >   Andreas Jaeger aj@{suse.com ,opensuse.org
> > } Twitter: jaegerandi
> >SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
> > GF: Felix Imendörffer, Jane Smithard, Graham Norton,
> > HRB 21284 (AG Nürnberg)
> >  GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272
> > A126
> >
> >
> >
>  __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > <
> http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> >
> >
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Zane Bitter]

On 12/10/18 8:59 AM, Corey Bryant wrote:



On Thu, Oct 11, 2018 at 10:19 AM Andreas Jaeger > wrote:


On 10/10/2018 23.10, Jeremy Stanley wrote:
 > I might have only pointed this out on IRC so far, but the
 > expectation is that testing 3.5 and 3.6 at the same time was merely
 > transitional since official OpenStack projects should be moving
 > their testing from Ubuntu Xenial (which provides 3.5) to Ubuntu
 > Bionic (which provides 3.6 and, now, 3.7 as well) during the Stein
 > cycle and so will drop 3.5 testing on master in the process.

Agreed, this needs some larger communication and explanation on what
to do,


The good news is we now have an initial change underway and successful, 
dropping py35 and enabling py37: https://review.openstack.org/#/c/609557/


Hey Corey,
Thanks for getting this underway, it's really important that we keep 
moving forward (we definitely got behind on the 3.6 transition and are 
paying for it now).


That said, I don't think we should be dropping support/testing for 3.5. 
According to:


  https://governance.openstack.org/tc/reference/pti/python.html

3.5 is the only Python3 version that we require all projects to run 
tests for.


Out goal is to get everyone running 3.6 unit tests by the end of Stein:


https://governance.openstack.org/tc/goals/stein/python3-first.html#python-3-6-unit-test-jobs

but we explicitly said there that we were not dropping support for 3.5 
as part of the goal, and should continue to do so until we can effect an 
orderly transition later. Personally, I would see that including waiting 
for all the 3.5-supporting projects to add 3.6 jobs (which has been 
blocked up until ~this point, as we are only just now close to getting 
all of the repos using local Zuul config).


I do agree that anything that works on 3.5 and 3.7 will almost certainly 
work on 3.6, so if you wanted to submit a patch to that goal saying that 
projects could add a unit test job for *either* 3.6 or 3.7 (in addition 
to 3.5) then I would probably support that. We could then switch all the 
3.5 jobs to 3.6 later when we eventually drop 3.5 support. That would 
mean we'd only ever run 3 unit test jobs (and 2 once 2.7 is eventually 
dropped) - for the oldest and newest versions of Python 3 that a project 
supports.


cheers,
Zane.

[This thread was also discussed on IRC starting here: 
http://eavesdrop.openstack.org/irclogs/%23openstack-tc/%23openstack-tc.2018-10-15.log.html#t2018-10-15T18:09:05]


I'm happy to get things moving along and start proposing changes like 
this to other projects and communicating with PTLs along the way. Do you 
think we need more discussion/communication on this or should I get started?


Thanks,
Corey


Andreas
-- 
  Andreas Jaeger aj@{suse.com ,opensuse.org

} Twitter: jaegerandi
   SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
    GF: Felix Imendörffer, Jane Smithard, Graham Norton,
        HRB 21284 (AG Nürnberg)
     GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272
A126


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Corey Bryant]

On Fri, Oct 12, 2018 at 8:59 AM Corey Bryant 
wrote:

>
>
> On Thu, Oct 11, 2018 at 10:19 AM Andreas Jaeger  wrote:
>
>> On 10/10/2018 23.10, Jeremy Stanley wrote:
>> > I might have only pointed this out on IRC so far, but the
>> > expectation is that testing 3.5 and 3.6 at the same time was merely
>> > transitional since official OpenStack projects should be moving
>> > their testing from Ubuntu Xenial (which provides 3.5) to Ubuntu
>> > Bionic (which provides 3.6 and, now, 3.7 as well) during the Stein
>> > cycle and so will drop 3.5 testing on master in the process.
>>
>> Agreed, this needs some larger communication and explanation on what to
>> do,
>>
>>
> The good news is we now have an initial change underway and successful,
> dropping py35 and enabling py37: https://review.openstack.org/#/c/609557/
>
> I'm happy to get things moving along and start proposing changes like this
> to other projects and communicating with PTLs along the way. Do you think
> we need more discussion/communication on this or should I get started?
>
> Thanks,
> Corey
>

We have a story to track this now at:
https://storyboard.openstack.org/#!/story/2004073

I think we will just get started on proposing changes. I've had a couple of
folks ask if they can help out which is great so we will start to chip away
at the story above. We'll also contact PTLs as we start working on projects
in case they haven't seen this thread.

Of course if anyone objects to us moving forward, please feel free to let
us know.

Thanks,
Corey


>
> Andreas
>> --
>>  Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
>>   SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
>>GF: Felix Imendörffer, Jane Smithard, Graham Norton,
>>HRB 21284 (AG Nürnberg)
>> GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272 A126
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Corey Bryant]

On Thu, Oct 11, 2018 at 10:19 AM Andreas Jaeger  wrote:

> On 10/10/2018 23.10, Jeremy Stanley wrote:
> > I might have only pointed this out on IRC so far, but the
> > expectation is that testing 3.5 and 3.6 at the same time was merely
> > transitional since official OpenStack projects should be moving
> > their testing from Ubuntu Xenial (which provides 3.5) to Ubuntu
> > Bionic (which provides 3.6 and, now, 3.7 as well) during the Stein
> > cycle and so will drop 3.5 testing on master in the process.
>
> Agreed, this needs some larger communication and explanation on what to do,
>
>
The good news is we now have an initial change underway and successful,
dropping py35 and enabling py37: https://review.openstack.org/#/c/609557/

I'm happy to get things moving along and start proposing changes like this
to other projects and communicating with PTLs along the way. Do you think
we need more discussion/communication on this or should I get started?

Thanks,
Corey


Andreas
> --
>  Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
>   SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
>GF: Felix Imendörffer, Jane Smithard, Graham Norton,
>HRB 21284 (AG Nürnberg)
> GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272 A126
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Corey Bryant]

On Wed, Oct 10, 2018 at 7:36 PM Goutham Pacha Ravi 
wrote:

> On Wed, Oct 10, 2018 at 2:10 PM Jeremy Stanley  wrote:
> >
> > On 2018-10-10 16:00:40 -0500 (-0500), Sean McGinnis wrote:
> > [...]
> > > I would rather see us testing 3.5 and 3.7 versus 3.5, 3.6, and
> > > 3.7.
> > [...]
> >
> > I might have only pointed this out on IRC so far, but the
> > expectation is that testing 3.5 and 3.6 at the same time was merely
> > transitional since official OpenStack projects should be moving
> > their testing from Ubuntu Xenial (which provides 3.5) to Ubuntu
> > Bionic (which provides 3.6 and, now, 3.7 as well) during the Stein
> > cycle and so will drop 3.5 testing on master in the process.
>
> ++ on switching python3.5 jobs to testing with python3.7 on Bionic.
> python3.5 wasn't supported on all distros [1][2][3][4][5]. Xenial had it,
> so it was nice to test with it when developing Queens and Rocky.
>
>
> Thanks Corey for starting this effort. I proposed changes to
> manila repos to use your template [1] [2], but the interpreter's not
> being installed,
> do you need to make any bindep changes to enable the "universe" ppa and
> install
> python3.7 and python3.7-dev?
>

Following up on this for anyone else who's following along. The python3.7
interpreter and development files are now correctly being installed after
some additional changes in zuul jobs. And we have our first py37 SUCCESS!
https://review.openstack.org/#/c/609557/

Goutham, thanks again for jumping in and adding py37 tests to your projects.

Thanks,
Corey


>
> [1] OpenSuse https://software.opensuse.org/package/python3
> [2] Ubuntu https://packages.ubuntu.com/search?keywords=python3
> [3] Fedora https://apps.fedoraproject.org/packages/python3
> [4] Arch https://www.archlinux.org/packages/extra/x86_64/python/
> [5] Gentoo https://wiki.gentoo.org/wiki/Project:Python/Implementations
> [6] manila https://review.openstack.org/#/c/609558
> [7] python-manilaclient https://review.openstack.org/609557
>
> --
> Goutham
>
> > --
> > Jeremy Stanley
> >
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Andreas Jaeger]

On 10/10/2018 23.10, Jeremy Stanley wrote:
> I might have only pointed this out on IRC so far, but the
> expectation is that testing 3.5 and 3.6 at the same time was merely
> transitional since official OpenStack projects should be moving
> their testing from Ubuntu Xenial (which provides 3.5) to Ubuntu
> Bionic (which provides 3.6 and, now, 3.7 as well) during the Stein
> cycle and so will drop 3.5 testing on master in the process.

Agreed, this needs some larger communication and explanation on what to do,

Andreas
-- 
 Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
  SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
   GF: Felix Imendörffer, Jane Smithard, Graham Norton,
   HRB 21284 (AG Nürnberg)
GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272 A126


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Corey Bryant]

On Wed, Oct 10, 2018 at 7:36 PM Goutham Pacha Ravi 
wrote:

> On Wed, Oct 10, 2018 at 2:10 PM Jeremy Stanley  wrote:
> >
> > On 2018-10-10 16:00:40 -0500 (-0500), Sean McGinnis wrote:
> > [...]
> > > I would rather see us testing 3.5 and 3.7 versus 3.5, 3.6, and
> > > 3.7.
> > [...]
> >
> > I might have only pointed this out on IRC so far, but the
> > expectation is that testing 3.5 and 3.6 at the same time was merely
> > transitional since official OpenStack projects should be moving
> > their testing from Ubuntu Xenial (which provides 3.5) to Ubuntu
> > Bionic (which provides 3.6 and, now, 3.7 as well) during the Stein
> > cycle and so will drop 3.5 testing on master in the process.
>
> ++ on switching python3.5 jobs to testing with python3.7 on Bionic.
> python3.5 wasn't supported on all distros [1][2][3][4][5]. Xenial had it,
> so it was nice to test with it when developing Queens and Rocky.
>
>
> Thanks Corey for starting this effort. I proposed changes to
> manila repos to use your template [1] [2], but the interpreter's not
> being installed,
> do you need to make any bindep changes to enable the "universe" ppa and
> install
> python3.7 and python3.7-dev?
>
>
Great, thanks for doing that! I'll look into what's needed to get python3.7
installed by the CI job.

Corey


> [1] OpenSuse https://software.opensuse.org/package/python3
> [2] Ubuntu https://packages.ubuntu.com/search?keywords=python3
> [3] Fedora https://apps.fedoraproject.org/packages/python3
> [4] Arch https://www.archlinux.org/packages/extra/x86_64/python/
> [5] Gentoo https://wiki.gentoo.org/wiki/Project:Python/Implementations
> [6] manila https://review.openstack.org/#/c/609558
> [7] python-manilaclient https://review.openstack.org/609557
>
> --
> Goutham
>
> > --
> > Jeremy Stanley
> >
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Jeremy Stanley]

On 2018-10-10 16:35:38 -0700 (-0700), Goutham Pacha Ravi wrote:
[...]
> Thanks Corey for starting this effort. I proposed changes to
> manila repos to use your template [1] [2], but the interpreter's
> not being installed, do you need to make any bindep changes to
> enable the "universe" ppa and install python3.7 and python3.7-dev?
[...]

I think we need to just make sure that the standard Python jobs
install the intended version of the interpreter. Using bindep for
that particular purpose is mildly silly. The bindep.txt file is,
first and foremost, a local developer convenience to let people know
what unexpected packages they might need to install on their systems
to run certain kinds of local tests. I really doubt any reasonable
developer will be surprised that they need to install python3.7
before being able to successfully run `tox -e py37` nor is the error
message confusing if they forget to do so.

A couple projects have added python-version-specific bindep profiles
which do nothing but install the corresponding interpreter, but
adding things to bindep.txt purely to satisfy the CI system is
backwards. Our CI jobs should do what we expect them to do by
default. If the job says it's going to run unit tests under Python
3.7 then the job should make sure a suitable interpreter is
installed to do so.
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Goutham Pacha Ravi]

On Wed, Oct 10, 2018 at 2:10 PM Jeremy Stanley  wrote:
>
> On 2018-10-10 16:00:40 -0500 (-0500), Sean McGinnis wrote:
> [...]
> > I would rather see us testing 3.5 and 3.7 versus 3.5, 3.6, and
> > 3.7.
> [...]
>
> I might have only pointed this out on IRC so far, but the
> expectation is that testing 3.5 and 3.6 at the same time was merely
> transitional since official OpenStack projects should be moving
> their testing from Ubuntu Xenial (which provides 3.5) to Ubuntu
> Bionic (which provides 3.6 and, now, 3.7 as well) during the Stein
> cycle and so will drop 3.5 testing on master in the process.

++ on switching python3.5 jobs to testing with python3.7 on Bionic.
python3.5 wasn't supported on all distros [1][2][3][4][5]. Xenial had it,
so it was nice to test with it when developing Queens and Rocky.


Thanks Corey for starting this effort. I proposed changes to
manila repos to use your template [1] [2], but the interpreter's not
being installed,
do you need to make any bindep changes to enable the "universe" ppa and install
python3.7 and python3.7-dev?


[1] OpenSuse https://software.opensuse.org/package/python3
[2] Ubuntu https://packages.ubuntu.com/search?keywords=python3
[3] Fedora https://apps.fedoraproject.org/packages/python3
[4] Arch https://www.archlinux.org/packages/extra/x86_64/python/
[5] Gentoo https://wiki.gentoo.org/wiki/Project:Python/Implementations
[6] manila https://review.openstack.org/#/c/609558
[7] python-manilaclient https://review.openstack.org/609557

--
Goutham

> --
> Jeremy Stanley
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Jeremy Stanley]

On 2018-10-10 16:00:40 -0500 (-0500), Sean McGinnis wrote:
[...]
> I would rather see us testing 3.5 and 3.7 versus 3.5, 3.6, and
> 3.7.
[...]

I might have only pointed this out on IRC so far, but the
expectation is that testing 3.5 and 3.6 at the same time was merely
transitional since official OpenStack projects should be moving
their testing from Ubuntu Xenial (which provides 3.5) to Ubuntu
Bionic (which provides 3.6 and, now, 3.7 as well) during the Stein
cycle and so will drop 3.5 testing on master in the process.
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Sean McGinnis]

> >
> >
> > What I mean is that we run too into a situation where we have a large
> > backlog of CI jobs since we have to many changes and jobs in flight.
> >
> > So, I'm asking whether there is a good way to not duplicating all jobs
> > to run on all three interpreters. Do we really need testing of all three
> > versions? Or is testing with a subset a manageable risk?
> >
> 
> Fair enough. I'm probably not the right person to answer so perhaps someone
> else can chime in. One thing worth pointing out is that it seems the jump
> from 3.5 to 3.6 wasn't nearly as painful as the jump from 3.6 to 3.7, at
> least in my experience.
> 
> Corey
> 

I share Andreas's concerns. I would rather see us testing 3.5 and 3.7 versus
3.5, 3.6, and 3.7. I would expect anything that passes on 3.7 to be fairly safe
when it comes to 3.6 runtimes.

Maybe a periodic job that exercies 3.6?

Sean

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Corey Bryant]

On Wed, Oct 10, 2018 at 10:18 AM Jeremy Stanley  wrote:

> On 2018-10-10 16:09:21 +0200 (+0200), Andreas Jaeger wrote:
> [...]
> > So, I'm asking whether there is a good way to not duplicating all
> > jobs to run on all three interpreters. Do we really need testing
> > of all three versions? Or is testing with a subset a manageable
> > risk?
>
> OpenStack projects are hopefully switching to testing on Bionic
> instead of Xenial during the Stein cycle, so will stop testing with
> Python 3.5 on master when that happens (since Bionic provides
> 3.6/3.7 and no 3.5).
>

That would be ideal, in which case dropping py35 and adding py37 for master
in Stein shouldn't require any more resources.

Corey

-- 
> Jeremy Stanley
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Jeremy Stanley]

On 2018-10-10 16:09:21 +0200 (+0200), Andreas Jaeger wrote:
[...]
> So, I'm asking whether there is a good way to not duplicating all
> jobs to run on all three interpreters. Do we really need testing
> of all three versions? Or is testing with a subset a manageable
> risk?

OpenStack projects are hopefully switching to testing on Bionic
instead of Xenial during the Stein cycle, so will stop testing with
Python 3.5 on master when that happens (since Bionic provides
3.6/3.7 and no 3.5).
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Corey Bryant]

On Wed, Oct 10, 2018 at 10:09 AM Andreas Jaeger  wrote:

> On 10/10/2018 15.42, Corey Bryant wrote:
> >
> >
> > On Wed, Oct 10, 2018 at 9:26 AM Andreas Jaeger  > > wrote:
> >
> > On 10/10/2018 14.45, Corey Bryant wrote:
> >  > [...]
> >  > == Enabling py37 unit tests ==
> >  >
> >  > Ubuntu Bionic (18.04 LTS) has the 3.7.0 interpreter and I have
> > reviews
> >  > up to define the py37 zuul job and templates here:
> >  > https://review.openstack.org/#/c/609066
> >  >
> >  > I'd like to start submitting reviews to projects to enable
> >  > openstack-python37-jobs (or variant) for projects that already
> have
> >  > openstack-python36-jobs in their .zuul.yaml, zuul.yaml,
> >  > .zuul.d/project.yaml.
> >
> > We have projects testing python 3.5 and 3.6 already. Adding 3.7 to
> > it is
> > a lot of wasted VMs. Can we limit testing and not test all three,
> > please?
> >
> >
> > Well, I wouldn't call any of them wasted if they're testing against a
> > supported Python version.
>
>
> What I mean is that we run too into a situation where we have a large
> backlog of CI jobs since we have to many changes and jobs in flight.
>
> So, I'm asking whether there is a good way to not duplicating all jobs
> to run on all three interpreters. Do we really need testing of all three
> versions? Or is testing with a subset a manageable risk?
>

Fair enough. I'm probably not the right person to answer so perhaps someone
else can chime in. One thing worth pointing out is that it seems the jump
from 3.5 to 3.6 wasn't nearly as painful as the jump from 3.6 to 3.7, at
least in my experience.

Corey


> Andreas
> --
>   Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
>SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
> GF: Felix Imendörffer, Jane Smithard, Graham Norton,
> HRB 21284 (AG Nürnberg)
>  GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272 A126
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Andreas Jaeger]

On 10/10/2018 15.42, Corey Bryant wrote:



On Wed, Oct 10, 2018 at 9:26 AM Andreas Jaeger > wrote:


On 10/10/2018 14.45, Corey Bryant wrote:
 > [...]
 > == Enabling py37 unit tests ==
 >
 > Ubuntu Bionic (18.04 LTS) has the 3.7.0 interpreter and I have
reviews
 > up to define the py37 zuul job and templates here:
 > https://review.openstack.org/#/c/609066
 >
 > I'd like to start submitting reviews to projects to enable
 > openstack-python37-jobs (or variant) for projects that already have
 > openstack-python36-jobs in their .zuul.yaml, zuul.yaml,
 > .zuul.d/project.yaml.

We have projects testing python 3.5 and 3.6 already. Adding 3.7 to
it is
a lot of wasted VMs. Can we limit testing and not test all three,
please?


Well, I wouldn't call any of them wasted if they're testing against a 
supported Python version.



What I mean is that we run too into a situation where we have a large 
backlog of CI jobs since we have to many changes and jobs in flight.


So, I'm asking whether there is a good way to not duplicating all jobs 
to run on all three interpreters. Do we really need testing of all three 
versions? Or is testing with a subset a manageable risk?


Andreas
--
 Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
  SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
   GF: Felix Imendörffer, Jane Smithard, Graham Norton,
   HRB 21284 (AG Nürnberg)
GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272 A126


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Jay Pipes]

On 10/10/2018 09:42 AM, Corey Bryant wrote:
On Wed, Oct 10, 2018 at 9:26 AM Andreas Jaeger > wrote:


On 10/10/2018 14.45, Corey Bryant wrote:
 > [...]
 > == Enabling py37 unit tests ==
 >
 > Ubuntu Bionic (18.04 LTS) has the 3.7.0 interpreter and I have
reviews
 > up to define the py37 zuul job and templates here:
 > https://review.openstack.org/#/c/609066
 >
 > I'd like to start submitting reviews to projects to enable
 > openstack-python37-jobs (or variant) for projects that already have
 > openstack-python36-jobs in their .zuul.yaml, zuul.yaml,
 > .zuul.d/project.yaml.

We have projects testing python 3.5 and 3.6 already. Adding 3.7 to
it is
a lot of wasted VMs. Can we limit testing and not test all three,
please?

Well, I wouldn't call any of them wasted if they're testing against a 
supported Python version.


++

-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Jeremy Stanley]

On 2018-10-10 09:38:14 -0400 (-0400), Corey Bryant wrote:
[...]
> Another option could be to use a non-LTS image to use a supported
> release.

Let's avoid creating additional images unless there is a strong
reason (every additional image means more load on our image
builders, more space consumed in our providers, et cetera). Bionic
seems like it will serve fine for this purpose now that it's got
more than a pre-release of 3.7.
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Corey Bryant]

On Wed, Oct 10, 2018 at 9:26 AM Andreas Jaeger  wrote:

> On 10/10/2018 14.45, Corey Bryant wrote:
> > [...]
> > == Enabling py37 unit tests ==
> >
> > Ubuntu Bionic (18.04 LTS) has the 3.7.0 interpreter and I have reviews
> > up to define the py37 zuul job and templates here:
> > https://review.openstack.org/#/c/609066
> >
> > I'd like to start submitting reviews to projects to enable
> > openstack-python37-jobs (or variant) for projects that already have
> > openstack-python36-jobs in their .zuul.yaml, zuul.yaml,
> > .zuul.d/project.yaml.
>
> We have projects testing python 3.5 and 3.6 already. Adding 3.7 to it is
> a lot of wasted VMs. Can we limit testing and not test all three, please?
>
>
Well, I wouldn't call any of them wasted if they're testing against a
supported Python version.

Corey

Andreas
> --
>   Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
>SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
> GF: Felix Imendörffer, Jane Smithard, Graham Norton,
> HRB 21284 (AG Nürnberg)
>  GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272 A126
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Corey Bryant]

On Wed, Oct 10, 2018 at 9:27 AM Jeremy Stanley  wrote:

> On 2018-10-10 08:45:39 -0400 (-0400), Corey Bryant wrote:
> [...]
> > Ubuntu Bionic (18.04 LTS) has the 3.7.0 interpreter
> [...]
>
> Thanks for the heads up! Last time I looked it was still a pre-3.7.0
> beta package, but looks like that has finally been updated to a
> proper release of the interpreter for Bionic in the last few weeks?
>

Yes, it was recently updated. It was originally a sync from Debian and we
got what we got in Bionic but the foundations folks were kind enough to
update it for us. This is a universe package for Bionic so it's not
officially supported but it should be good enough for unit testing. Another
option could be to use a non-LTS image to use a supported release.

Corey

-- 
> Jeremy Stanley
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-devbut t
> 
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Andreas Jaeger]

On 10/10/2018 14.45, Corey Bryant wrote:

[...]
== Enabling py37 unit tests ==

Ubuntu Bionic (18.04 LTS) has the 3.7.0 interpreter and I have reviews 
up to define the py37 zuul job and templates here: 
https://review.openstack.org/#/c/609066


I'd like to start submitting reviews to projects to enable 
openstack-python37-jobs (or variant) for projects that already have 
openstack-python36-jobs in their .zuul.yaml, zuul.yaml, 
.zuul.d/project.yaml.


We have projects testing python 3.5 and 3.6 already. Adding 3.7 to it is 
a lot of wasted VMs. Can we limit testing and not test all three, please?


Andreas
--
 Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
  SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
   GF: Felix Imendörffer, Jane Smithard, Graham Norton,
   HRB 21284 (AG Nürnberg)
GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272 A126


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[Jeremy Stanley]

On 2018-10-10 08:45:39 -0400 (-0400), Corey Bryant wrote:
[...]
> Ubuntu Bionic (18.04 LTS) has the 3.7.0 interpreter
[...]

Thanks for the heads up! Last time I looked it was still a pre-3.7.0
beta package, but looks like that has finally been updated to a
proper release of the interpreter for Bionic in the last few weeks?
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [python3] Enabling py37 unit tests

[Corey Bryant]

Hi All,

I'd like to enable py37 unit tests in the gate.

== Background ==

I work on OpenStack packaging for Ubuntu. During the Rocky release (Ubuntu
Cosmic) I tried to fix py37 bugs upstream as I came across them. There
ended up being a lot of py37 issues and after a while, due to time
constraints, I resorted to just opening bugs and disabling py37 unit tests
that were failing in our package builds. Luckily enough, even though Cosmic
ships with python3.6 and python3.7, python3.6 ended up being chosen as the
default for Cosmic.

== Defaulting to python3.7 ==

The next release of Ubuntu opens in just a few weeks. It will default to
python3.7 and will not include python3.6. My hope is that if I can help
enable py37 unit tests upstream now, we can get a wider view at fixing
issues soon.

== Enabling py37 unit tests ==

Ubuntu Bionic (18.04 LTS) has the 3.7.0 interpreter and I have reviews up
to define the py37 zuul job and templates here:
https://review.openstack.org/#/c/609066

I'd like to start submitting reviews to projects to enable
openstack-python37-jobs (or variant) for projects that already have
openstack-python36-jobs in their .zuul.yaml, zuul.yaml,
.zuul.d/project.yaml.

== Coinciding work ==

There is python3-first work going on now and I completely understand that
this is going to cause more work for some projects. It seems that now is as
good of a time as ever to catch up and test with a recent python3 version.
I'm sure python3.8 and beyond will be here before we know it.

Any thoughts or concerns?

Thanks,
Corey
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev