Re: [openstack-dev] [all][tc][ptls] final stages of python 3 transition

[John Dickinson]

On 20 May 2018, at 17:19, Thomas Goirand wrote:


On 05/20/2018 06:24 PM, Matthew Treinish wrote:

On Sun, May 20, 2018 at 03:05:34PM +0200, Thomas Goirand wrote:

Thanks for these details. What exactly is the trouble with the Swift
backend? Do you know? Is anyone working on fixing it? At my company,
we'd be happy to work on that (if of course, it's not too time 
demanding).




Sorry I didn't mean the swift backend, but swift itself under 
python3:


https://wiki.openstack.org/wiki/Python3#OpenStack_applications_.28tc:approved-release.29

If you're trying to deploy everything under python3 I don't think 
you'll be
able to deploy swift. But if you already have a swift running then 
the glance

backend should work fine under pythom 3.


Of course I know Swift isn't Python 3 ready. And that's sad... :/


yep. we're still working on it. slowly.



However, we did also experience issues with the swift backend last 
week.

Hopefully, with the switch to uwsgi it's going to work. I'll let you
know if that's not the case.


Is the "switch to uwsgi" something about how you're running swift or 
something about how you're running glance?


FWIW, my experience with putting TLS in front of Swift is to run Swift 
as "normal" (ie run `swift-proxy-server /etc/swift/proxy-server.conf` 
itself instead of under apache or nginx or something else). Then use 
HAProxy or hitch to terminate TLS and forward that internally to the 
proxy server.




Cheers,

Thomas Goirand (zigo)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc][ptls] final stages of python 3 transition

[Thomas Goirand]

On 05/20/2018 06:24 PM, Matthew Treinish wrote:
> On Sun, May 20, 2018 at 03:05:34PM +0200, Thomas Goirand wrote:
>> Thanks for these details. What exactly is the trouble with the Swift
>> backend? Do you know? Is anyone working on fixing it? At my company,
>> we'd be happy to work on that (if of course, it's not too time demanding).
>>
> 
> Sorry I didn't mean the swift backend, but swift itself under python3:
> 
> https://wiki.openstack.org/wiki/Python3#OpenStack_applications_.28tc:approved-release.29
> 
> If you're trying to deploy everything under python3 I don't think you'll be
> able to deploy swift. But if you already have a swift running then the glance
> backend should work fine under pythom 3.

Of course I know Swift isn't Python 3 ready. And that's sad... :/

However, we did also experience issues with the swift backend last week.
Hopefully, with the switch to uwsgi it's going to work. I'll let you
know if that's not the case.

Cheers,

Thomas Goirand (zigo)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc][ptls] final stages of python 3 transition

[Thomas Goirand]

On 05/07/2018 01:36 PM, Jean-Philippe Evrard wrote:
> We've been juggling with python3, ansible and multiple distros for a while 
> now.
> That dance hasn't been fruitful: many hidden issues, either due to
> ansible modules, or our own modules, or upgrade issues.
> 
> I've recently decided to simplify the python2/3 story.
> 
> Queens and all the stable branches will be python2 only (python3 will
> not be used anymore, to simplify the code)
> 
> For Rocky, we plan to use as much as possible the distribution
> packages for the python stack, if it's recent enough for our source
> installs.
> Ubuntu 16.04 will have python2, SUSE has python2, CentOS has no
> appropriate package, so we are pip installing things (and using
> python2).
> So... If people work on Ubuntu 18.04 support, we could try a python3
> only system. Nobody worked on it right now.

/me raises hand!

At the moment, I got a nearly full Queens stack up and running on top of
Debian Stretch, using Python 3 only (and of course all of that is also
uploaded to Debian Sid). The puppet-openstack scenario001 works fully
without SSL, last week I could fix cinder, today Glance (thanks to Matt)
and Nova, and it's looking like I got one remaining issue with Neutron
that I didn't have time to investigate fully yet (got to dig in the
logs). But it's looking good. Hopefully, next week I'll be able to tell
everything works.

So, have a try with Debian? :)

Cheers,

Thomas Goirand (zigo)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc][ptls] final stages of python 3 transition

[Matthew Treinish]

On Sun, May 20, 2018 at 03:05:34PM +0200, Thomas Goirand wrote:
> On 05/19/2018 07:54 PM, Matthew Treinish wrote:
> > On Sat, May 19, 2018 at 07:04:53PM +0200, Thomas Goirand wrote:
> >> using:
> >> - RBD backend
> >> - swift backend
> >> - swift+rgw
> > 
> > As for the backend store choice I don't have any personal experience using
> > either of these 3 as a backend store. That being said your choice of store
> > should be independent from the getting glance-api deployed behind uwsgi
> > and a webserver. 
> > 
> > Although, you might have trouble with swift on py3, because IIRC that still
> > isn't working. (unless something changed recently) But, the store config is
> > really independent from getting the api to receive and handle api requests
> > properly. 
> 
> Thanks for these details. What exactly is the trouble with the Swift
> backend? Do you know? Is anyone working on fixing it? At my company,
> we'd be happy to work on that (if of course, it's not too time demanding).
> 

Sorry I didn't mean the swift backend, but swift itself under python3:

https://wiki.openstack.org/wiki/Python3#OpenStack_applications_.28tc:approved-release.29

If you're trying to deploy everything under python3 I don't think you'll be
able to deploy swift. But if you already have a swift running then the glance
backend should work fine under pythom 3.

> >>> The issues glance has with running in a wsgi app are related to it's
> >>> use of async tasks via taskflow. (which includes the tasks api and
> >>> image import stuff) This shouldn't be hard to fix, and I've had
> >>> patches up to address these for months:
> >>>
> >>> https://review.openstack.org/#/c/531498/
> >>> https://review.openstack.org/#/c/549743/
> >>
> >> Do I need to backport these patches to Queens to run Glance the way I
> >> described? Will it also fix running Glance with mod_wsgi?
> > 
> > These patches are independent of getting things working for you. They
> > are only required for 2 API features in glance to work. The tasks api and
> > the image import api (which was added in queens). You don't need either
> > to upload images by default, and the patches will only ever be necessary
> > if you have something using those APIs (which personally I've never
> > encountered in the wild). There is also no test coverage in tempest or
> > any external test suite using these apis that I'm aware of so your CI
> > likely won't even be blocked by this. (which is how this situation
> > arose in the first place)
> 
> Allright, So hopefully, I'm very close from having Debian to gate
> properly in puppet-openstack upstream. As much as I could tell, Glance
> and Cinder are the only pieces that are still failing with SSL (and
> everything works already without SSL), so I must be very close to a nice
> result (after a course of nearly 2 months already).
> 
> Thanks again for all the very valuable details that you provided. I have
> to admit that I was starting to loose faith in the project, because of
> all the frustration of not finding a working solution.
> 
> I'll let the list knows when I have something that fully works and
> gating with puppet-openstack, of course.
> 
> Cheers,
> 
> Thomas Goirand (zigo)


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc][ptls] final stages of python 3 transition

[Thomas Goirand]

On 05/19/2018 07:54 PM, Matthew Treinish wrote:
> On Sat, May 19, 2018 at 07:04:53PM +0200, Thomas Goirand wrote:
>> using:
>> - RBD backend
>> - swift backend
>> - swift+rgw
> 
> As for the backend store choice I don't have any personal experience using
> either of these 3 as a backend store. That being said your choice of store
> should be independent from the getting glance-api deployed behind uwsgi
> and a webserver. 
> 
> Although, you might have trouble with swift on py3, because IIRC that still
> isn't working. (unless something changed recently) But, the store config is
> really independent from getting the api to receive and handle api requests
> properly. 

Thanks for these details. What exactly is the trouble with the Swift
backend? Do you know? Is anyone working on fixing it? At my company,
we'd be happy to work on that (if of course, it's not too time demanding).

>>> The issues glance has with running in a wsgi app are related to it's
>>> use of async tasks via taskflow. (which includes the tasks api and
>>> image import stuff) This shouldn't be hard to fix, and I've had
>>> patches up to address these for months:
>>>
>>> https://review.openstack.org/#/c/531498/
>>> https://review.openstack.org/#/c/549743/
>>
>> Do I need to backport these patches to Queens to run Glance the way I
>> described? Will it also fix running Glance with mod_wsgi?
> 
> These patches are independent of getting things working for you. They
> are only required for 2 API features in glance to work. The tasks api and
> the image import api (which was added in queens). You don't need either
> to upload images by default, and the patches will only ever be necessary
> if you have something using those APIs (which personally I've never
> encountered in the wild). There is also no test coverage in tempest or
> any external test suite using these apis that I'm aware of so your CI
> likely won't even be blocked by this. (which is how this situation
> arose in the first place)

Allright, So hopefully, I'm very close from having Debian to gate
properly in puppet-openstack upstream. As much as I could tell, Glance
and Cinder are the only pieces that are still failing with SSL (and
everything works already without SSL), so I must be very close to a nice
result (after a course of nearly 2 months already).

Thanks again for all the very valuable details that you provided. I have
to admit that I was starting to loose faith in the project, because of
all the frustration of not finding a working solution.

I'll let the list knows when I have something that fully works and
gating with puppet-openstack, of course.

Cheers,

Thomas Goirand (zigo)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc][ptls] final stages of python 3 transition

[Matthew Treinish]

On Sat, May 19, 2018 at 07:04:53PM +0200, Thomas Goirand wrote:
> On 05/08/2018 06:22 PM, Matthew Treinish wrote:
> >> Glance - Has issues with image upload + uwsgi + eventlet [1]
> > 
> > This actually is a bit misleading. Glance works fine with image upload and 
> > uwsgi.
> > That's the only configuration of glance in a wsgi app that works because
> > of chunked transfer encoding not being in the WSGI protocol. [2] uwsgi 
> > provides
> > an alternate interface to read chunked requests which enables this to work.
> > If you look at the bugs linked off that release note about image upload
> > you'll see they're all fixed.
> 
> Hi Matt,
> 
> I'm quite happy to read the above. Just to make sure...
> 
> Can you confirm that Glance + Python 3 + uwsgi with SSL will work using
> the below setup?

So glance with uwsgi, python3, and ssl works fine. (with the caveats I
mentioned below) We test that on every commit in the integrated gate
today in the tempest-full-py3 job. It's been that way for almost a year
at this point.

> 
> using:
> - RBD backend
> - swift backend
> - swift+rgw

As for the backend store choice I don't have any personal experience using
either of these 3 as a backend store. That being said your choice of store
should be independent from the getting glance-api deployed behind uwsgi
and a webserver. 

Although, you might have trouble with swift on py3, because IIRC that still
isn't working. (unless something changed recently) But, the store config is
really independent from getting the api to receive and handle api requests
properly. 

> 
> If so, then I'll probably end up pushing for such uwsgi setup.
> 
> If I understand you correctly, it wont work with Apache mod_wsgi,
> because of these chcked transfer encoding, which is what made if fail
> when I tried using the RBD backend. Right?

This is correct, you can not use glance and mod_wsgi together because
it will not handle requests with chunked transfer encoding by default.
So it will fail on any image upload request made to glance that uses
chunked transfer encoding.

> 
> > The issues glance has with running in a wsgi app are related to it's
> > use of async tasks via taskflow. (which includes the tasks api and
> > image import stuff) This shouldn't be hard to fix, and I've had
> > patches up to address these for months:
> >
> > https://review.openstack.org/#/c/531498/
> > https://review.openstack.org/#/c/549743/
> 
> Do I need to backport these patches to Queens to run Glance the way I
> described? Will it also fix running Glance with mod_wsgi?

These patches are independent of getting things working for you. They
are only required for 2 API features in glance to work. The tasks api and
the image import api (which was added in queens). You don't need either
to upload images by default, and the patches will only ever be necessary
if you have something using those APIs (which personally I've never
encountered in the wild). There is also no test coverage in tempest or
any external test suite using these apis that I'm aware of so your CI
likely won't even be blocked by this. (which is how this situation
arose in the first place)

-Matt Treinish


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc][ptls] final stages of python 3 transition

[Thomas Goirand]

On 05/08/2018 06:22 PM, Matthew Treinish wrote:
>> Glance - Has issues with image upload + uwsgi + eventlet [1]
> 
> This actually is a bit misleading. Glance works fine with image upload and 
> uwsgi.
> That's the only configuration of glance in a wsgi app that works because
> of chunked transfer encoding not being in the WSGI protocol. [2] uwsgi 
> provides
> an alternate interface to read chunked requests which enables this to work.
> If you look at the bugs linked off that release note about image upload
> you'll see they're all fixed.

Hi Matt,

I'm quite happy to read the above. Just to make sure...

Can you confirm that Glance + Python 3 + uwsgi with SSL will work using
the below setup?

using:
- RBD backend
- swift backend
- swift+rgw

If so, then I'll probably end up pushing for such uwsgi setup.

If I understand you correctly, it wont work with Apache mod_wsgi,
because of these chcked transfer encoding, which is what made if fail
when I tried using the RBD backend. Right?

> The issues glance has with running in a wsgi app are related to it's
> use of async tasks via taskflow. (which includes the tasks api and
> image import stuff) This shouldn't be hard to fix, and I've had
> patches up to address these for months:
>
> https://review.openstack.org/#/c/531498/
> https://review.openstack.org/#/c/549743/

Do I need to backport these patches to Queens to run Glance the way I
described? Will it also fix running Glance with mod_wsgi?

Cheers,

Thomas Goirand (zigo)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc][ptls] final stages of python 3 transition

[Thomas Goirand]

On 05/08/2018 06:01 PM, Graham Hayes wrote:
> Glance - Has issues with image upload + uwsgi + eventlet [1]

Yeah, as much as I can see, my experience with last week is that there's
no working mode of operation with Glance, Python3 and SSL:

- It doesn't work with eventlet, with SSL handshake failure if we don't
remove that line:
https://github.com/eventlet/eventlet/blob/master/eventlet/green/ssl.py#L342

(of course, removing the set_nonblocking() line in eventlet is *not* a
solution)

- It doesn't work with uwsgi (connection reset by peer, IIRC)

- It doesn't work with Apache (Content-Length issue when uploading images)

The only mode that I didn't test (yet) is using eventlet without SSL,
and then using HA-Proxy to do the SSL part. Maybe using Apache with
mod_proxy will work too, I probably will test that too, and see which
one integrates the more easily with puppet-openstack.

I don't see why the above mod_proxy or haproxy deployment wouldn't work,
but after all the frustrations I had with Glance last week, I'm
expecting anything...

So, to generalize, yeah, we definitively need to fix this issue with
Eventlet ASAP. But also fix Glance so that it can work with uwsgi and
Apache mod_wsgi.

Cheers,

Thomas Goirand (zigo)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc][ptls] final stages of python 3 transition

[Doug Hellmann]

Excerpts from Graham Hayes's message of 2018-05-08 17:01:36 +0100:
> On 08/05/18 16:53, Doug Hellmann wrote:
> > Excerpts from Graham Hayes's message of 2018-05-08 16:28:46 +0100:
> >> On 08/05/18 16:09, Zane Bitter wrote:
> >>> On 30/04/18 17:16, Ben Nemec wrote:
> > Excerpts from Doug Hellmann's message of 2018-04-25 16:54:46 -0400:
> >> 1. Fix oslo.service functional tests -- the Oslo team needs help
> >>     maintaining this library. Alternatively, we could move all
> >>     services to use cotyledon (https://pypi.org/project/cotyledon/).
> >>>
> >>> I submitted a patch that fixes the py35 gate (which was broken due to
> >>> changes between CPython 3.4 and 3.5), so once that merges we can flip
> >>> the gate back to voting:
> >>>
> >>> https://review.openstack.org/566714
> >>>
>  For everyone's awareness, we discussed this in the Oslo meeting today
>  and our first step is to see how many, if any, services are actually
>  relying on the oslo.service functionality that doesn't work in Python
>  3 today.  From there we will come up with a plan for how to move forward.
> 
>  https://bugs.launchpad.net/manila/+bug/1482633 is the original bug.
> >>>
> >>> These tests are currently skipped in both oslo_service and nova.
> >>> (Equivalent tests were removed from Neutron and Manila on the principle
> >>> that they're now oslo_service's responsibility.)
> >>>
> >>> This appears to be a series of long-standing bugs in eventlet:
> >>>
> >>> Python 3.5 failure mode:
> >>> https://github.com/eventlet/eventlet/issues/308
> >>> https://github.com/eventlet/eventlet/issues/189
> >>>
> >>> Python 3.4 failure mode:
> >>> https://github.com/eventlet/eventlet/issues/476
> >>> https://github.com/eventlet/eventlet/issues/145
> >>>
> >>> There are also more problems coming down the pipeline in Python 3.6:
> >>>
> >>> https://github.com/eventlet/eventlet/issues/371
> >>>
> >>> That one is resolved in eventlet 0.21, but we have that blocked by
> >>> upper-constraints:
> >>> http://git.openstack.org/cgit/openstack/requirements/tree/upper-constraints.txt#n135
> >>>
> >>>
> >>> Given that the code in question relates solely to standalone WSGI
> >>> servers with SSL and everything should have already migrated to Apache,
> >>> and that the upstream is clearly overworked and unlikely to merge fixes
> >>> any time soon (plus we would have to deal with the fallout of moving the
> >>> upper constraint), I agree that it would be preferable if we could just
> >>> ditch this functionality.
> >>
> >> There are a few projects that have not migrated, and some that have
> >> issues running in non standalone WSGI mode (due, ironically to eventlet)
> >>
> >> We should probably get people to run these projects behind an reverse
> >> proxy, and terminate SSL there, but right now we don't have that
> >> documented.
> > 
> > Do you know which projects?
> 
> I know of 2:
> 
> Designate - mainly due to the major lack of resources available during
> the uwsgi goal period, and the level of work needed to unravel our
> tooling to support it.
> 
> Glance - Has issues with image upload + uwsgi + eventlet [1]
> 
> I am sure there are probably others, but I know of these 2.
> 
> [1] https://docs.openstack.org/releasenotes/glance/unreleased.html#b1

OK, so we need to put these things on the red flags list for moving to
Python 3. I've updated the status for oslo.service, designate, and
glance in https://wiki.openstack.org/wiki/Python3 to reflect that.

Doug

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc][ptls] final stages of python 3 transition

[Matthew Treinish]

On Tue, May 08, 2018 at 05:01:36PM +0100, Graham Hayes wrote:
> On 08/05/18 16:53, Doug Hellmann wrote:
> > Excerpts from Graham Hayes's message of 2018-05-08 16:28:46 +0100:
> >> On 08/05/18 16:09, Zane Bitter wrote:
> >>> On 30/04/18 17:16, Ben Nemec wrote:
> > Excerpts from Doug Hellmann's message of 2018-04-25 16:54:46 -0400:
> >> 1. Fix oslo.service functional tests -- the Oslo team needs help
> >>     maintaining this library. Alternatively, we could move all
> >>     services to use cotyledon (https://pypi.org/project/cotyledon/).
> >>>
> >>> I submitted a patch that fixes the py35 gate (which was broken due to
> >>> changes between CPython 3.4 and 3.5), so once that merges we can flip
> >>> the gate back to voting:
> >>>
> >>> https://review.openstack.org/566714
> >>>
>  For everyone's awareness, we discussed this in the Oslo meeting today
>  and our first step is to see how many, if any, services are actually
>  relying on the oslo.service functionality that doesn't work in Python
>  3 today.  From there we will come up with a plan for how to move forward.
> 
>  https://bugs.launchpad.net/manila/+bug/1482633 is the original bug.
> >>>
> >>> These tests are currently skipped in both oslo_service and nova.
> >>> (Equivalent tests were removed from Neutron and Manila on the principle
> >>> that they're now oslo_service's responsibility.)
> >>>
> >>> This appears to be a series of long-standing bugs in eventlet:
> >>>
> >>> Python 3.5 failure mode:
> >>> https://github.com/eventlet/eventlet/issues/308
> >>> https://github.com/eventlet/eventlet/issues/189
> >>>
> >>> Python 3.4 failure mode:
> >>> https://github.com/eventlet/eventlet/issues/476
> >>> https://github.com/eventlet/eventlet/issues/145
> >>>
> >>> There are also more problems coming down the pipeline in Python 3.6:
> >>>
> >>> https://github.com/eventlet/eventlet/issues/371
> >>>
> >>> That one is resolved in eventlet 0.21, but we have that blocked by
> >>> upper-constraints:
> >>> http://git.openstack.org/cgit/openstack/requirements/tree/upper-constraints.txt#n135
> >>>
> >>>
> >>> Given that the code in question relates solely to standalone WSGI
> >>> servers with SSL and everything should have already migrated to Apache,
> >>> and that the upstream is clearly overworked and unlikely to merge fixes
> >>> any time soon (plus we would have to deal with the fallout of moving the
> >>> upper constraint), I agree that it would be preferable if we could just
> >>> ditch this functionality.
> >>
> >> There are a few projects that have not migrated, and some that have
> >> issues running in non standalone WSGI mode (due, ironically to eventlet)
> >>
> >> We should probably get people to run these projects behind an reverse
> >> proxy, and terminate SSL there, but right now we don't have that
> >> documented.
> > 
> > Do you know which projects?
> 
> I know of 2:
> 
> Designate - mainly due to the major lack of resources available during
> the uwsgi goal period, and the level of work needed to unravel our
> tooling to support it.
> 
> Glance - Has issues with image upload + uwsgi + eventlet [1]

This actually is a bit misleading. Glance works fine with image upload and 
uwsgi.
That's the only configuration of glance in a wsgi app that works because
of chunked transfer encoding not being in the WSGI protocol. [2] uwsgi provides
an alternate interface to read chunked requests which enables this to work.
If you look at the bugs linked off that release note about image upload
you'll see they're all fixed.

The issues glance has with running in a wsgi app are related to it's use of
async tasks via taskflow. (which includes the tasks api and image import stuff)
This shouldn't be hard to fix, and I've had patches up to address these for
months:

https://review.openstack.org/#/c/531498/
https://review.openstack.org/#/c/549743/

Part of the issue is that there is no api driven testing for these async api
functions or any documented way to test them. Which is why I marked the 2nd
one WIP, since I have no method to test it and after asking several times
for a test case or some other method to validate these APIs without an answer.

In fact people are running glance under uwsgi in production already because it 
makes a lot of things easier and the current issues don't effect most users.

-Matt Treinish


> 
> I am sure there are probably others, but I know of these 2.
> 
> [1] https://docs.openstack.org/releasenotes/glance/unreleased.html#b1
> 

[2] There are a few other ways, as some other wsgi servers have grafted on
support for chunked transfer encoding. But, most wsgi servers have not
implemented a method.


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

Re: [openstack-dev] [all][tc][ptls] final stages of python 3 transition

[Graham Hayes]

On 08/05/18 16:53, Doug Hellmann wrote:
> Excerpts from Graham Hayes's message of 2018-05-08 16:28:46 +0100:
>> On 08/05/18 16:09, Zane Bitter wrote:
>>> On 30/04/18 17:16, Ben Nemec wrote:
> Excerpts from Doug Hellmann's message of 2018-04-25 16:54:46 -0400:
>> 1. Fix oslo.service functional tests -- the Oslo team needs help
>>     maintaining this library. Alternatively, we could move all
>>     services to use cotyledon (https://pypi.org/project/cotyledon/).
>>>
>>> I submitted a patch that fixes the py35 gate (which was broken due to
>>> changes between CPython 3.4 and 3.5), so once that merges we can flip
>>> the gate back to voting:
>>>
>>> https://review.openstack.org/566714
>>>
 For everyone's awareness, we discussed this in the Oslo meeting today
 and our first step is to see how many, if any, services are actually
 relying on the oslo.service functionality that doesn't work in Python
 3 today.  From there we will come up with a plan for how to move forward.

 https://bugs.launchpad.net/manila/+bug/1482633 is the original bug.
>>>
>>> These tests are currently skipped in both oslo_service and nova.
>>> (Equivalent tests were removed from Neutron and Manila on the principle
>>> that they're now oslo_service's responsibility.)
>>>
>>> This appears to be a series of long-standing bugs in eventlet:
>>>
>>> Python 3.5 failure mode:
>>> https://github.com/eventlet/eventlet/issues/308
>>> https://github.com/eventlet/eventlet/issues/189
>>>
>>> Python 3.4 failure mode:
>>> https://github.com/eventlet/eventlet/issues/476
>>> https://github.com/eventlet/eventlet/issues/145
>>>
>>> There are also more problems coming down the pipeline in Python 3.6:
>>>
>>> https://github.com/eventlet/eventlet/issues/371
>>>
>>> That one is resolved in eventlet 0.21, but we have that blocked by
>>> upper-constraints:
>>> http://git.openstack.org/cgit/openstack/requirements/tree/upper-constraints.txt#n135
>>>
>>>
>>> Given that the code in question relates solely to standalone WSGI
>>> servers with SSL and everything should have already migrated to Apache,
>>> and that the upstream is clearly overworked and unlikely to merge fixes
>>> any time soon (plus we would have to deal with the fallout of moving the
>>> upper constraint), I agree that it would be preferable if we could just
>>> ditch this functionality.
>>
>> There are a few projects that have not migrated, and some that have
>> issues running in non standalone WSGI mode (due, ironically to eventlet)
>>
>> We should probably get people to run these projects behind an reverse
>> proxy, and terminate SSL there, but right now we don't have that
>> documented.
> 
> Do you know which projects?

I know of 2:

Designate - mainly due to the major lack of resources available during
the uwsgi goal period, and the level of work needed to unravel our
tooling to support it.

Glance - Has issues with image upload + uwsgi + eventlet [1]

I am sure there are probably others, but I know of these 2.

[1] https://docs.openstack.org/releasenotes/glance/unreleased.html#b1

> 
> Doug
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 



signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc][ptls] final stages of python 3 transition

[Doug Hellmann]

Excerpts from Graham Hayes's message of 2018-05-08 16:28:46 +0100:
> On 08/05/18 16:09, Zane Bitter wrote:
> > On 30/04/18 17:16, Ben Nemec wrote:
> >>> Excerpts from Doug Hellmann's message of 2018-04-25 16:54:46 -0400:
>  1. Fix oslo.service functional tests -- the Oslo team needs help
>      maintaining this library. Alternatively, we could move all
>      services to use cotyledon (https://pypi.org/project/cotyledon/).
> > 
> > I submitted a patch that fixes the py35 gate (which was broken due to
> > changes between CPython 3.4 and 3.5), so once that merges we can flip
> > the gate back to voting:
> > 
> > https://review.openstack.org/566714
> > 
> >> For everyone's awareness, we discussed this in the Oslo meeting today
> >> and our first step is to see how many, if any, services are actually
> >> relying on the oslo.service functionality that doesn't work in Python
> >> 3 today.  From there we will come up with a plan for how to move forward.
> >>
> >> https://bugs.launchpad.net/manila/+bug/1482633 is the original bug.
> > 
> > These tests are currently skipped in both oslo_service and nova.
> > (Equivalent tests were removed from Neutron and Manila on the principle
> > that they're now oslo_service's responsibility.)
> > 
> > This appears to be a series of long-standing bugs in eventlet:
> > 
> > Python 3.5 failure mode:
> > https://github.com/eventlet/eventlet/issues/308
> > https://github.com/eventlet/eventlet/issues/189
> > 
> > Python 3.4 failure mode:
> > https://github.com/eventlet/eventlet/issues/476
> > https://github.com/eventlet/eventlet/issues/145
> > 
> > There are also more problems coming down the pipeline in Python 3.6:
> > 
> > https://github.com/eventlet/eventlet/issues/371
> > 
> > That one is resolved in eventlet 0.21, but we have that blocked by
> > upper-constraints:
> > http://git.openstack.org/cgit/openstack/requirements/tree/upper-constraints.txt#n135
> > 
> > 
> > Given that the code in question relates solely to standalone WSGI
> > servers with SSL and everything should have already migrated to Apache,
> > and that the upstream is clearly overworked and unlikely to merge fixes
> > any time soon (plus we would have to deal with the fallout of moving the
> > upper constraint), I agree that it would be preferable if we could just
> > ditch this functionality.
> 
> There are a few projects that have not migrated, and some that have
> issues running in non standalone WSGI mode (due, ironically to eventlet)
> 
> We should probably get people to run these projects behind an reverse
> proxy, and terminate SSL there, but right now we don't have that
> documented.

Do you know which projects?

Doug


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc][ptls] final stages of python 3 transition

[Graham Hayes]

On 08/05/18 16:09, Zane Bitter wrote:
> On 30/04/18 17:16, Ben Nemec wrote:
>>> Excerpts from Doug Hellmann's message of 2018-04-25 16:54:46 -0400:
 1. Fix oslo.service functional tests -- the Oslo team needs help
     maintaining this library. Alternatively, we could move all
     services to use cotyledon (https://pypi.org/project/cotyledon/).
> 
> I submitted a patch that fixes the py35 gate (which was broken due to
> changes between CPython 3.4 and 3.5), so once that merges we can flip
> the gate back to voting:
> 
> https://review.openstack.org/566714
> 
>> For everyone's awareness, we discussed this in the Oslo meeting today
>> and our first step is to see how many, if any, services are actually
>> relying on the oslo.service functionality that doesn't work in Python
>> 3 today.  From there we will come up with a plan for how to move forward.
>>
>> https://bugs.launchpad.net/manila/+bug/1482633 is the original bug.
> 
> These tests are currently skipped in both oslo_service and nova.
> (Equivalent tests were removed from Neutron and Manila on the principle
> that they're now oslo_service's responsibility.)
> 
> This appears to be a series of long-standing bugs in eventlet:
> 
> Python 3.5 failure mode:
> https://github.com/eventlet/eventlet/issues/308
> https://github.com/eventlet/eventlet/issues/189
> 
> Python 3.4 failure mode:
> https://github.com/eventlet/eventlet/issues/476
> https://github.com/eventlet/eventlet/issues/145
> 
> There are also more problems coming down the pipeline in Python 3.6:
> 
> https://github.com/eventlet/eventlet/issues/371
> 
> That one is resolved in eventlet 0.21, but we have that blocked by
> upper-constraints:
> http://git.openstack.org/cgit/openstack/requirements/tree/upper-constraints.txt#n135
> 
> 
> Given that the code in question relates solely to standalone WSGI
> servers with SSL and everything should have already migrated to Apache,
> and that the upstream is clearly overworked and unlikely to merge fixes
> any time soon (plus we would have to deal with the fallout of moving the
> upper constraint), I agree that it would be preferable if we could just
> ditch this functionality.

There are a few projects that have not migrated, and some that have
issues running in non standalone WSGI mode (due, ironically to eventlet)

We should probably get people to run these projects behind an reverse
proxy, and terminate SSL there, but right now we don't have that
documented.

> cheers,
> Zane.
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc][ptls] final stages of python 3 transition

[Zane Bitter]

On 30/04/18 17:16, Ben Nemec wrote:

Excerpts from Doug Hellmann's message of 2018-04-25 16:54:46 -0400:

1. Fix oslo.service functional tests -- the Oslo team needs help
    maintaining this library. Alternatively, we could move all
    services to use cotyledon (https://pypi.org/project/cotyledon/).


I submitted a patch that fixes the py35 gate (which was broken due to 
changes between CPython 3.4 and 3.5), so once that merges we can flip 
the gate back to voting:


https://review.openstack.org/566714

For everyone's awareness, we discussed this in the Oslo meeting today 
and our first step is to see how many, if any, services are actually 
relying on the oslo.service functionality that doesn't work in Python 3 
today.  From there we will come up with a plan for how to move forward.


https://bugs.launchpad.net/manila/+bug/1482633 is the original bug.


These tests are currently skipped in both oslo_service and nova. 
(Equivalent tests were removed from Neutron and Manila on the principle 
that they're now oslo_service's responsibility.)


This appears to be a series of long-standing bugs in eventlet:

Python 3.5 failure mode:
https://github.com/eventlet/eventlet/issues/308
https://github.com/eventlet/eventlet/issues/189

Python 3.4 failure mode:
https://github.com/eventlet/eventlet/issues/476
https://github.com/eventlet/eventlet/issues/145

There are also more problems coming down the pipeline in Python 3.6:

https://github.com/eventlet/eventlet/issues/371

That one is resolved in eventlet 0.21, but we have that blocked by 
upper-constraints: 
http://git.openstack.org/cgit/openstack/requirements/tree/upper-constraints.txt#n135


Given that the code in question relates solely to standalone WSGI 
servers with SSL and everything should have already migrated to Apache, 
and that the upstream is clearly overworked and unlikely to merge fixes 
any time soon (plus we would have to deal with the fallout of moving the 
upper constraint), I agree that it would be preferable if we could just 
ditch this functionality.


cheers,
Zane.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc][ptls] final stages of python 3 transition

[Doug Hellmann]

Excerpts from Jean-Philippe Evrard's message of 2018-05-07 13:36:34 +0200:
> We've been juggling with python3, ansible and multiple distros for a while 
> now.
> That dance hasn't been fruitful: many hidden issues, either due to
> ansible modules, or our own modules, or upgrade issues.
> 
> I've recently decided to simplify the python2/3 story.
> 
> Queens and all the stable branches will be python2 only (python3 will
> not be used anymore, to simplify the code)
> 
> For Rocky, we plan to use as much as possible the distribution
> packages for the python stack, if it's recent enough for our source
> installs.
> Ubuntu 16.04 will have python2, SUSE has python2, CentOS has no
> appropriate package, so we are pip installing things (and using
> python2).
> So... If people work on Ubuntu 18.04 support, we could try a python3
> only system. Nobody worked on it right now.
> Same for CentOS, because there is no usage of packages, we could use
> Software Collections and have centos as a python3 only system. Sadly,
> nobody has the cycles to do it now.
> 
> I am expecting we'll be using/seeing a lot more python3 in the future
> with S, and wish for a python3 only "S" release.
> But this solely depends on the work done in R to make sure 18.04 is
> ready, as this would be our example, and "stepping stone" towards both
> python2 and python3.
> 
> I am not sure this answers your question, as this is more gray than a
> black or white answer.
> But I am hoping we'll stabilize python3 this cycle, for ubuntu 18.04
> at least, and other distros as a stretch goal.
> 
> Best regards,
> Jean-Philippe Evrard (evrardjp)
> 

I think your answer does help. It sounds like, unsurprisingly, you are
depending on work upstream in two different directions: You need the
OpenStack contributor community to ensure the code works on the platform
using Python 3, and then you need the OS vendors to provide appropriate
packages using Python 3.

Doug

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc][ptls] final stages of python 3 transition

[Jean-Philippe Evrard]

We've been juggling with python3, ansible and multiple distros for a while now.
That dance hasn't been fruitful: many hidden issues, either due to
ansible modules, or our own modules, or upgrade issues.

I've recently decided to simplify the python2/3 story.

Queens and all the stable branches will be python2 only (python3 will
not be used anymore, to simplify the code)

For Rocky, we plan to use as much as possible the distribution
packages for the python stack, if it's recent enough for our source
installs.
Ubuntu 16.04 will have python2, SUSE has python2, CentOS has no
appropriate package, so we are pip installing things (and using
python2).
So... If people work on Ubuntu 18.04 support, we could try a python3
only system. Nobody worked on it right now.
Same for CentOS, because there is no usage of packages, we could use
Software Collections and have centos as a python3 only system. Sadly,
nobody has the cycles to do it now.

I am expecting we'll be using/seeing a lot more python3 in the future
with S, and wish for a python3 only "S" release.
But this solely depends on the work done in R to make sure 18.04 is
ready, as this would be our example, and "stepping stone" towards both
python2 and python3.

I am not sure this answers your question, as this is more gray than a
black or white answer.
But I am hoping we'll stabilize python3 this cycle, for ubuntu 18.04
at least, and other distros as a stretch goal.

Best regards,
Jean-Philippe Evrard (evrardjp)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc][ptls] final stages of python 3 transition

[Doug Hellmann]

Excerpts from Alex Schultz's message of 2018-04-30 15:43:16 -0600:
> On Mon, Apr 30, 2018 at 3:16 PM, Ben Nemec  wrote:
> > Resending from an address that is subscribed to the list.  Apologies to
> > those of you who get this twice.
> >
> > On 04/30/2018 10:06 AM, Doug Hellmann wrote:
> >>
> >> It would be useful to have more input from PTLs on this issue, so I'm
> >> CCing all of them to get their attention.
> >>
> >> Excerpts from Doug Hellmann's message of 2018-04-25 16:54:46 -0400:
> >>>
> >>> It's time to talk about the next steps in our migration from python
> >>> 2 to python 3.
> >>>
> >>> Up to this point we have mostly focused on reaching a state where
> >>> we support both versions of the language. We are not quite there
> >>> with all projects, as you can see by reviewing the test coverage
> >>> status information at
> >>>
> >>> https://wiki.openstack.org/wiki/Python3#Python_3_Status_of_OpenStack_projects
> >>>
> >>> Still, we need to press on to the next phase of the migration, which
> >>> I have been calling "Python 3 first". This is where we use python
> >>> 3 as the default, for everything, and set up the exceptions we need
> >>> for anything that still requires python 2.
> >>>
> >>> To reach that stage, we need to:
> >>>
> >>> 1. Change the documentation and release notes jobs to use python 3.
> >>> (The Oslo team recently completed this, and found that we did
> >>> need to make a few small code changes to get them to work.)
> >>> 2. Change (or duplicate) all functional test jobs to run under
> >>> python 3.
> >>> 3. Change the packaging jobs to use python 3.
> >>> 4. Update devstack to use 3 by default and require setting a flag to
> >>> use 2. (This may trigger other job changes.)
> >>>
> >>> At that point, all of our deliverables will be produced using python
> >>> 3, and we can be relatively confident that if we no longer had
> >>> access to python 2 we could still continue operating. We could also
> >>> start updating deployment tools to use either python 3 or 2, so
> >>> that users could actually deploy using the python 3 versions of
> >>> services.
> >>>
> >>> Somewhere in that time frame our third-party CI systems will need
> >>> to ensure they have python 3 support as well.
> >>>
> >>> After the "Python 3 first" phase is completed we should release
> >>> one series using the packages built with python 3. Perhaps Stein?
> >>> Or is that too ambitious?
> >>>
> >>> Next, we will be ready to address the prerequisites for "Python 3
> >>> only," which will allow us to drop Python 2 support.
> >>>
> >>> We need to wait to drop python 2 support as a community, rather
> >>> than going one project at a time, to avoid doubling the work of
> >>> downstream consumers such as distros and independent deployers. We
> >>> don't want them to have to package all (or even a large number) of
> >>> the dependencies of OpenStack twice because they have to install
> >>> some services running under python 2 and others under 3. Ideally
> >>> they would be able to upgrade all of the services on a node together
> >>> as part of their transition to the new version, without ending up
> >>> with a python 2 version of a dependency along side a python 3 version
> >>> of the same package.
> >>>
> >>> The remaining items could be fixed earlier, but this is the point
> >>> at which they would block us:
> >>>
> >>> 1. Fix oslo.service functional tests -- the Oslo team needs help
> >>> maintaining this library. Alternatively, we could move all
> >>> services to use cotyledon (https://pypi.org/project/cotyledon/).
> >
> >
> > For everyone's awareness, we discussed this in the Oslo meeting today and
> > our first step is to see how many, if any, services are actually relying on
> > the oslo.service functionality that doesn't work in Python 3 today.  From
> > there we will come up with a plan for how to move forward.
> >
> > https://bugs.launchpad.net/manila/+bug/1482633 is the original bug.
> >
> >>>
> >>> 2. Finish the unit test and functional test ports so that all of
> >>> our tests can run under python 3 (this implies that the services
> >>> all run under python 3, so there is no more porting to do).
> >
> >
> > And integration tests?  I know for the initial python 3 goal we said just
> > unit and functional, but it seems to me that we can't claim full python 3
> > compatibility until we can run our tempest jobs against python 3-based
> > OpenStack.
> >
> >>>
> >>> Finally, after we have *all* tests running on python 3, we can
> >>> safely drop python 2.
> >>>
> >>> We have previously discussed the end of the T cycle as the point
> >>> at which we would have all of those tests running, and if that holds
> >>> true we could reasonably drop python 2 during the beginning of the
> >>> U cycle, in late 2019 and before the 2020 cut-off point when upstream
> >>> python 2 support will be dropped.
> >>>
> >>> I need some info from the deployment tool teams to understand 

Re: [openstack-dev] [all][tc][ptls] final stages of python 3 transition

[Doug Hellmann]

Excerpts from Ben Nemec's message of 2018-04-30 16:16:35 -0500:
> Resending from an address that is subscribed to the list.  Apologies to 
> those of you who get this twice.
> 
> On 04/30/2018 10:06 AM, Doug Hellmann wrote:
> > It would be useful to have more input from PTLs on this issue, so I'm
> > CCing all of them to get their attention.
> > 
> > Excerpts from Doug Hellmann's message of 2018-04-25 16:54:46 -0400:
> >> It's time to talk about the next steps in our migration from python
> >> 2 to python 3.
> >>
> >> Up to this point we have mostly focused on reaching a state where
> >> we support both versions of the language. We are not quite there
> >> with all projects, as you can see by reviewing the test coverage
> >> status information at
> >> https://wiki.openstack.org/wiki/Python3#Python_3_Status_of_OpenStack_projects
> >>
> >> Still, we need to press on to the next phase of the migration, which
> >> I have been calling "Python 3 first". This is where we use python
> >> 3 as the default, for everything, and set up the exceptions we need
> >> for anything that still requires python 2.
> >>
> >> To reach that stage, we need to:
> >>
> >> 1. Change the documentation and release notes jobs to use python 3.
> >> (The Oslo team recently completed this, and found that we did
> >> need to make a few small code changes to get them to work.)
> >> 2. Change (or duplicate) all functional test jobs to run under
> >> python 3.
> >> 3. Change the packaging jobs to use python 3.
> >> 4. Update devstack to use 3 by default and require setting a flag to
> >> use 2. (This may trigger other job changes.)
> >>
> >> At that point, all of our deliverables will be produced using python
> >> 3, and we can be relatively confident that if we no longer had
> >> access to python 2 we could still continue operating. We could also
> >> start updating deployment tools to use either python 3 or 2, so
> >> that users could actually deploy using the python 3 versions of
> >> services.
> >>
> >> Somewhere in that time frame our third-party CI systems will need
> >> to ensure they have python 3 support as well.
> >>
> >> After the "Python 3 first" phase is completed we should release
> >> one series using the packages built with python 3. Perhaps Stein?
> >> Or is that too ambitious?
> >>
> >> Next, we will be ready to address the prerequisites for "Python 3
> >> only," which will allow us to drop Python 2 support.
> >>
> >> We need to wait to drop python 2 support as a community, rather
> >> than going one project at a time, to avoid doubling the work of
> >> downstream consumers such as distros and independent deployers. We
> >> don't want them to have to package all (or even a large number) of
> >> the dependencies of OpenStack twice because they have to install
> >> some services running under python 2 and others under 3. Ideally
> >> they would be able to upgrade all of the services on a node together
> >> as part of their transition to the new version, without ending up
> >> with a python 2 version of a dependency along side a python 3 version
> >> of the same package.
> >>
> >> The remaining items could be fixed earlier, but this is the point
> >> at which they would block us:
> >>
> >> 1. Fix oslo.service functional tests -- the Oslo team needs help
> >> maintaining this library. Alternatively, we could move all
> >> services to use cotyledon (https://pypi.org/project/cotyledon/).
> 
> For everyone's awareness, we discussed this in the Oslo meeting today 
> and our first step is to see how many, if any, services are actually 
> relying on the oslo.service functionality that doesn't work in Python 3 
> today.  From there we will come up with a plan for how to move forward.
> 
> https://bugs.launchpad.net/manila/+bug/1482633 is the original bug.
> 
> >>
> >> 2. Finish the unit test and functional test ports so that all of
> >> our tests can run under python 3 (this implies that the services
> >> all run under python 3, so there is no more porting to do).
> 
> And integration tests?  I know for the initial python 3 goal we said 
> just unit and functional, but it seems to me that we can't claim full 
> python 3 compatibility until we can run our tempest jobs against python 
> 3-based OpenStack.

Good point. The wiki page lists the integrated-gate-py35 job for
many projects, but not all will use that particular test job. I'm
not sure what other sort of integration jobs we do have, but I agree
we should have versions of them working for python 3.

> 
> >>
> >> Finally, after we have *all* tests running on python 3, we can
> >> safely drop python 2.
> >>
> >> We have previously discussed the end of the T cycle as the point
> >> at which we would have all of those tests running, and if that holds
> >> true we could reasonably drop python 2 during the beginning of the
> >> U cycle, in late 2019 and before the 2020 cut-off point when upstream
> >> python 2 support will be dropped.
> >>
> >> I need 

Re: [openstack-dev] [all][tc][ptls] final stages of python 3 transition

[Alex Schultz]

On Mon, Apr 30, 2018 at 3:16 PM, Ben Nemec  wrote:
> Resending from an address that is subscribed to the list.  Apologies to
> those of you who get this twice.
>
> On 04/30/2018 10:06 AM, Doug Hellmann wrote:
>>
>> It would be useful to have more input from PTLs on this issue, so I'm
>> CCing all of them to get their attention.
>>
>> Excerpts from Doug Hellmann's message of 2018-04-25 16:54:46 -0400:
>>>
>>> It's time to talk about the next steps in our migration from python
>>> 2 to python 3.
>>>
>>> Up to this point we have mostly focused on reaching a state where
>>> we support both versions of the language. We are not quite there
>>> with all projects, as you can see by reviewing the test coverage
>>> status information at
>>>
>>> https://wiki.openstack.org/wiki/Python3#Python_3_Status_of_OpenStack_projects
>>>
>>> Still, we need to press on to the next phase of the migration, which
>>> I have been calling "Python 3 first". This is where we use python
>>> 3 as the default, for everything, and set up the exceptions we need
>>> for anything that still requires python 2.
>>>
>>> To reach that stage, we need to:
>>>
>>> 1. Change the documentation and release notes jobs to use python 3.
>>> (The Oslo team recently completed this, and found that we did
>>> need to make a few small code changes to get them to work.)
>>> 2. Change (or duplicate) all functional test jobs to run under
>>> python 3.
>>> 3. Change the packaging jobs to use python 3.
>>> 4. Update devstack to use 3 by default and require setting a flag to
>>> use 2. (This may trigger other job changes.)
>>>
>>> At that point, all of our deliverables will be produced using python
>>> 3, and we can be relatively confident that if we no longer had
>>> access to python 2 we could still continue operating. We could also
>>> start updating deployment tools to use either python 3 or 2, so
>>> that users could actually deploy using the python 3 versions of
>>> services.
>>>
>>> Somewhere in that time frame our third-party CI systems will need
>>> to ensure they have python 3 support as well.
>>>
>>> After the "Python 3 first" phase is completed we should release
>>> one series using the packages built with python 3. Perhaps Stein?
>>> Or is that too ambitious?
>>>
>>> Next, we will be ready to address the prerequisites for "Python 3
>>> only," which will allow us to drop Python 2 support.
>>>
>>> We need to wait to drop python 2 support as a community, rather
>>> than going one project at a time, to avoid doubling the work of
>>> downstream consumers such as distros and independent deployers. We
>>> don't want them to have to package all (or even a large number) of
>>> the dependencies of OpenStack twice because they have to install
>>> some services running under python 2 and others under 3. Ideally
>>> they would be able to upgrade all of the services on a node together
>>> as part of their transition to the new version, without ending up
>>> with a python 2 version of a dependency along side a python 3 version
>>> of the same package.
>>>
>>> The remaining items could be fixed earlier, but this is the point
>>> at which they would block us:
>>>
>>> 1. Fix oslo.service functional tests -- the Oslo team needs help
>>> maintaining this library. Alternatively, we could move all
>>> services to use cotyledon (https://pypi.org/project/cotyledon/).
>
>
> For everyone's awareness, we discussed this in the Oslo meeting today and
> our first step is to see how many, if any, services are actually relying on
> the oslo.service functionality that doesn't work in Python 3 today.  From
> there we will come up with a plan for how to move forward.
>
> https://bugs.launchpad.net/manila/+bug/1482633 is the original bug.
>
>>>
>>> 2. Finish the unit test and functional test ports so that all of
>>> our tests can run under python 3 (this implies that the services
>>> all run under python 3, so there is no more porting to do).
>
>
> And integration tests?  I know for the initial python 3 goal we said just
> unit and functional, but it seems to me that we can't claim full python 3
> compatibility until we can run our tempest jobs against python 3-based
> OpenStack.
>
>>>
>>> Finally, after we have *all* tests running on python 3, we can
>>> safely drop python 2.
>>>
>>> We have previously discussed the end of the T cycle as the point
>>> at which we would have all of those tests running, and if that holds
>>> true we could reasonably drop python 2 during the beginning of the
>>> U cycle, in late 2019 and before the 2020 cut-off point when upstream
>>> python 2 support will be dropped.
>>>
>>> I need some info from the deployment tool teams to understand whether
>>> they would be ready to take the plunge during T or U and start
>>> deploying only the python 3 version. Are there other upgrade issues
>>> that need to be addressed to support moving from 2 to 3? Something
>>> that might be part of the platform(s), rather 

Re: [openstack-dev] [all][tc][ptls] final stages of python 3 transition

[Matthew Treinish]

On Mon, Apr 30, 2018 at 04:16:35PM -0500, Ben Nemec wrote:
> Resending from an address that is subscribed to the list.  Apologies to
> those of you who get this twice.
> 
> On 04/30/2018 10:06 AM, Doug Hellmann wrote:
> > It would be useful to have more input from PTLs on this issue, so I'm
> > CCing all of them to get their attention.
> > 
> > Excerpts from Doug Hellmann's message of 2018-04-25 16:54:46 -0400:
> > > It's time to talk about the next steps in our migration from python
> > > 2 to python 3.
> > > 
> > > Up to this point we have mostly focused on reaching a state where
> > > we support both versions of the language. We are not quite there
> > > with all projects, as you can see by reviewing the test coverage
> > > status information at
> > > https://wiki.openstack.org/wiki/Python3#Python_3_Status_of_OpenStack_projects
> > > 
> > > Still, we need to press on to the next phase of the migration, which
> > > I have been calling "Python 3 first". This is where we use python
> > > 3 as the default, for everything, and set up the exceptions we need
> > > for anything that still requires python 2.
> > > 
> > > To reach that stage, we need to:
> > > 
> > > 1. Change the documentation and release notes jobs to use python 3.
> > > (The Oslo team recently completed this, and found that we did
> > > need to make a few small code changes to get them to work.)
> > > 2. Change (or duplicate) all functional test jobs to run under
> > > python 3.
> > > 3. Change the packaging jobs to use python 3.
> > > 4. Update devstack to use 3 by default and require setting a flag to
> > > use 2. (This may trigger other job changes.)
> > > 
> > > At that point, all of our deliverables will be produced using python
> > > 3, and we can be relatively confident that if we no longer had
> > > access to python 2 we could still continue operating. We could also
> > > start updating deployment tools to use either python 3 or 2, so
> > > that users could actually deploy using the python 3 versions of
> > > services.
> > > 
> > > Somewhere in that time frame our third-party CI systems will need
> > > to ensure they have python 3 support as well.
> > > 
> > > After the "Python 3 first" phase is completed we should release
> > > one series using the packages built with python 3. Perhaps Stein?
> > > Or is that too ambitious?
> > > 
> > > Next, we will be ready to address the prerequisites for "Python 3
> > > only," which will allow us to drop Python 2 support.
> > > 
> > > We need to wait to drop python 2 support as a community, rather
> > > than going one project at a time, to avoid doubling the work of
> > > downstream consumers such as distros and independent deployers. We
> > > don't want them to have to package all (or even a large number) of
> > > the dependencies of OpenStack twice because they have to install
> > > some services running under python 2 and others under 3. Ideally
> > > they would be able to upgrade all of the services on a node together
> > > as part of their transition to the new version, without ending up
> > > with a python 2 version of a dependency along side a python 3 version
> > > of the same package.
> > > 
> > > The remaining items could be fixed earlier, but this is the point
> > > at which they would block us:
> > > 
> > > 1. Fix oslo.service functional tests -- the Oslo team needs help
> > > maintaining this library. Alternatively, we could move all
> > > services to use cotyledon (https://pypi.org/project/cotyledon/).
> 
> For everyone's awareness, we discussed this in the Oslo meeting today and
> our first step is to see how many, if any, services are actually relying on
> the oslo.service functionality that doesn't work in Python 3 today.  From
> there we will come up with a plan for how to move forward.
> 
> https://bugs.launchpad.net/manila/+bug/1482633 is the original bug.
> 
> > > 
> > > 2. Finish the unit test and functional test ports so that all of
> > > our tests can run under python 3 (this implies that the services
> > > all run under python 3, so there is no more porting to do).
> 
> And integration tests?  I know for the initial python 3 goal we said just
> unit and functional, but it seems to me that we can't claim full python 3
> compatibility until we can run our tempest jobs against python 3-based
> OpenStack.

They already are running, and have been since the Atlanta PTG (which was the
same cycle as the goal):

https://review.openstack.org/#/c/436540/

You can see the gate jobs history here:

http://status.openstack.org/openstack-health/#/job/tempest-full-py3

-Matt Treinish

> 
> > > 
> > > Finally, after we have *all* tests running on python 3, we can
> > > safely drop python 2.
> > > 
> > > We have previously discussed the end of the T cycle as the point
> > > at which we would have all of those tests running, and if that holds
> > > true we could reasonably drop python 2 during the beginning of the
> > > U cycle, in late 2019 and before the 

Re: [openstack-dev] [all][tc][ptls] final stages of python 3 transition

[Ben Nemec]

Resending from an address that is subscribed to the list.  Apologies to 
those of you who get this twice.


On 04/30/2018 10:06 AM, Doug Hellmann wrote:

It would be useful to have more input from PTLs on this issue, so I'm
CCing all of them to get their attention.

Excerpts from Doug Hellmann's message of 2018-04-25 16:54:46 -0400:

It's time to talk about the next steps in our migration from python
2 to python 3.

Up to this point we have mostly focused on reaching a state where
we support both versions of the language. We are not quite there
with all projects, as you can see by reviewing the test coverage
status information at
https://wiki.openstack.org/wiki/Python3#Python_3_Status_of_OpenStack_projects

Still, we need to press on to the next phase of the migration, which
I have been calling "Python 3 first". This is where we use python
3 as the default, for everything, and set up the exceptions we need
for anything that still requires python 2.

To reach that stage, we need to:

1. Change the documentation and release notes jobs to use python 3.
(The Oslo team recently completed this, and found that we did
need to make a few small code changes to get them to work.)
2. Change (or duplicate) all functional test jobs to run under
python 3.
3. Change the packaging jobs to use python 3.
4. Update devstack to use 3 by default and require setting a flag to
use 2. (This may trigger other job changes.)

At that point, all of our deliverables will be produced using python
3, and we can be relatively confident that if we no longer had
access to python 2 we could still continue operating. We could also
start updating deployment tools to use either python 3 or 2, so
that users could actually deploy using the python 3 versions of
services.

Somewhere in that time frame our third-party CI systems will need
to ensure they have python 3 support as well.

After the "Python 3 first" phase is completed we should release
one series using the packages built with python 3. Perhaps Stein?
Or is that too ambitious?

Next, we will be ready to address the prerequisites for "Python 3
only," which will allow us to drop Python 2 support.

We need to wait to drop python 2 support as a community, rather
than going one project at a time, to avoid doubling the work of
downstream consumers such as distros and independent deployers. We
don't want them to have to package all (or even a large number) of
the dependencies of OpenStack twice because they have to install
some services running under python 2 and others under 3. Ideally
they would be able to upgrade all of the services on a node together
as part of their transition to the new version, without ending up
with a python 2 version of a dependency along side a python 3 version
of the same package.

The remaining items could be fixed earlier, but this is the point
at which they would block us:

1. Fix oslo.service functional tests -- the Oslo team needs help
maintaining this library. Alternatively, we could move all
services to use cotyledon (https://pypi.org/project/cotyledon/).


For everyone's awareness, we discussed this in the Oslo meeting today 
and our first step is to see how many, if any, services are actually 
relying on the oslo.service functionality that doesn't work in Python 3 
today.  From there we will come up with a plan for how to move forward.


https://bugs.launchpad.net/manila/+bug/1482633 is the original bug.



2. Finish the unit test and functional test ports so that all of
our tests can run under python 3 (this implies that the services
all run under python 3, so there is no more porting to do).


And integration tests?  I know for the initial python 3 goal we said 
just unit and functional, but it seems to me that we can't claim full 
python 3 compatibility until we can run our tempest jobs against python 
3-based OpenStack.




Finally, after we have *all* tests running on python 3, we can
safely drop python 2.

We have previously discussed the end of the T cycle as the point
at which we would have all of those tests running, and if that holds
true we could reasonably drop python 2 during the beginning of the
U cycle, in late 2019 and before the 2020 cut-off point when upstream
python 2 support will be dropped.

I need some info from the deployment tool teams to understand whether
they would be ready to take the plunge during T or U and start
deploying only the python 3 version. Are there other upgrade issues
that need to be addressed to support moving from 2 to 3? Something
that might be part of the platform(s), rather than OpenStack itself?


Alex can probably expand on this, but I know TripleO has some challenges 
in this area.  Specifically the fact that CentOS 7 will only ever 
support Python 2 and CentOS 8 is planned to only support Python 3. Since 
CentOS 8 is not a thing yet and no release dates are announced they're 
having to use Fedora for Python 3 testing, which isn't something that 
will be supported long-term.  

Re: [openstack-dev] [all][tc][ptls] final stages of python 3 transition

[Andrey Pavlov]

Hi Doug,

There is no ec2-api project on the link.
But we have voted job  openstack-tox-py35


Regards,
Andrey Pavlov.

On Mon, Apr 30, 2018 at 6:06 PM, Doug Hellmann 
wrote:

> It would be useful to have more input from PTLs on this issue, so I'm
> CCing all of them to get their attention.
>
> Excerpts from Doug Hellmann's message of 2018-04-25 16:54:46 -0400:
> > It's time to talk about the next steps in our migration from python
> > 2 to python 3.
> >
> > Up to this point we have mostly focused on reaching a state where
> > we support both versions of the language. We are not quite there
> > with all projects, as you can see by reviewing the test coverage
> > status information at
> > https://wiki.openstack.org/wiki/Python3#Python_3_Status_
> of_OpenStack_projects
> >
> > Still, we need to press on to the next phase of the migration, which
> > I have been calling "Python 3 first". This is where we use python
> > 3 as the default, for everything, and set up the exceptions we need
> > for anything that still requires python 2.
> >
> > To reach that stage, we need to:
> >
> > 1. Change the documentation and release notes jobs to use python 3.
> >(The Oslo team recently completed this, and found that we did
> >need to make a few small code changes to get them to work.)
> > 2. Change (or duplicate) all functional test jobs to run under
> >python 3.
> > 3. Change the packaging jobs to use python 3.
> > 4. Update devstack to use 3 by default and require setting a flag to
> >use 2. (This may trigger other job changes.)
> >
> > At that point, all of our deliverables will be produced using python
> > 3, and we can be relatively confident that if we no longer had
> > access to python 2 we could still continue operating. We could also
> > start updating deployment tools to use either python 3 or 2, so
> > that users could actually deploy using the python 3 versions of
> > services.
> >
> > Somewhere in that time frame our third-party CI systems will need
> > to ensure they have python 3 support as well.
> >
> > After the "Python 3 first" phase is completed we should release
> > one series using the packages built with python 3. Perhaps Stein?
> > Or is that too ambitious?
> >
> > Next, we will be ready to address the prerequisites for "Python 3
> > only," which will allow us to drop Python 2 support.
> >
> > We need to wait to drop python 2 support as a community, rather
> > than going one project at a time, to avoid doubling the work of
> > downstream consumers such as distros and independent deployers. We
> > don't want them to have to package all (or even a large number) of
> > the dependencies of OpenStack twice because they have to install
> > some services running under python 2 and others under 3. Ideally
> > they would be able to upgrade all of the services on a node together
> > as part of their transition to the new version, without ending up
> > with a python 2 version of a dependency along side a python 3 version
> > of the same package.
> >
> > The remaining items could be fixed earlier, but this is the point
> > at which they would block us:
> >
> > 1. Fix oslo.service functional tests -- the Oslo team needs help
> >maintaining this library. Alternatively, we could move all
> >services to use cotyledon (https://pypi.org/project/cotyledon/).
> >
> > 2. Finish the unit test and functional test ports so that all of
> >our tests can run under python 3 (this implies that the services
> >all run under python 3, so there is no more porting to do).
> >
> > Finally, after we have *all* tests running on python 3, we can
> > safely drop python 2.
> >
> > We have previously discussed the end of the T cycle as the point
> > at which we would have all of those tests running, and if that holds
> > true we could reasonably drop python 2 during the beginning of the
> > U cycle, in late 2019 and before the 2020 cut-off point when upstream
> > python 2 support will be dropped.
> >
> > I need some info from the deployment tool teams to understand whether
> > they would be ready to take the plunge during T or U and start
> > deploying only the python 3 version. Are there other upgrade issues
> > that need to be addressed to support moving from 2 to 3? Something
> > that might be part of the platform(s), rather than OpenStack itself?
> >
> > What else have I missed in these phases? Other jobs? Other blocking
> > conditions?
> >
> > Doug
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev