Re: [openstack-dev] [swift] Plan to port Swift to Python 3

[vishal yadav]

Victor,

I appreciate for your effort.

However I was just checking if you considered using 2to3. I can understand
that translation using this tool might not cover every area in the code
more specifically custom/3rd party libraries (non-standard python
libraries) but IMO it can do fixer translations to much extent. If needed
custom fixers can also be defined for 2to3.

- https://docs.python.org/2/library/2to3.html
- https://docs.python.org/3/howto/pyporting.html

Thanks,
Vishal

On Fri, Oct 9, 2015 at 2:34 PM, Thierry Carrez 
wrote:

> Victor Stinner wrote:
> > Good news, we made good progress last weeks on porting Swift to Python
> > 3, a few changes were merged and all dependencies now work on Python 3.
> > We only need two more simple changes to have a working pyhon34 check job:
> >
> > * "py3: Update pbr and dnspython requirements"
> >   https://review.openstack.org/#/c/217423/
> > * "py3: Add py34 test environment to tox"
> >   https://review.openstack.org/#/c/199034/
> >
> > With these changes, it will be possible to make the python34 check job
> > voting to avoid Python 3 regressions. It's very important to avoid
> > regressions, so we cannot go backward again in Python 3 support.
> >
> > On IRC, it was said that it's better to merge Python 3 changes at the
> > beginning of the Mitaka cycle, because Python 3 requires a lot of small
> > changes which can likely introduce (subtle) bugs, and it's better to
> > catch them early during the development cycle.
> >
> > John Dickinson prefers incremental and small changes, whereas clayg
> > looks to like giant patches to fix all Python 3 issues at once to avoid
> > conflicts in other (non-Python3) changes. (Sorry, if I didn't summarized
> > correctly the discussion we had yesterday.)
> >
> > The problem is that it's hard to fix "all" Python 3 issues in a single
> > patch, the patch would be super giant and just impossible to review.
> > It's also annoying to have to write dozens of small patches: we loose
> > time on merge conflicts, rebasing, random gate failures, etc.
> >
> > I proposed a first patch serie of 6 changes to fix a lot of simple
> > Python 3 issues "at once":
> > [...]
> >
> > The overall diff is impressive: "61 files changed, 233 insertions(+),
> > 189 deletions(-)" ... but each change is quite simple. It's only one
> > pattern replaced with a different pattern. For example, replace
> > "unicode" with "six.text_type" (and add "import six" if needed). So
> > these changes should be easy to review.
> >
> > With a working (and voting?) python34 check job and these 6 changes, it
> > will be (much) easier to work on porting Swift to Python 3. Following
> > patches will be validated by the python34 check job, shorter and
> > restricted to a few files.
> >
> > Victor
>
> That's great news. Thanks so much for your tireless efforts to get
> Python 3 supported everywhere in OpenStack, Victor !
>
> --
> Thierry Carrez (ttx)
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [swift] Plan to port Swift to Python 3

[Thierry Carrez]

Victor Stinner wrote:
> Good news, we made good progress last weeks on porting Swift to Python
> 3, a few changes were merged and all dependencies now work on Python 3.
> We only need two more simple changes to have a working pyhon34 check job:
> 
> * "py3: Update pbr and dnspython requirements"
>   https://review.openstack.org/#/c/217423/
> * "py3: Add py34 test environment to tox"
>   https://review.openstack.org/#/c/199034/
> 
> With these changes, it will be possible to make the python34 check job
> voting to avoid Python 3 regressions. It's very important to avoid
> regressions, so we cannot go backward again in Python 3 support.
> 
> On IRC, it was said that it's better to merge Python 3 changes at the
> beginning of the Mitaka cycle, because Python 3 requires a lot of small
> changes which can likely introduce (subtle) bugs, and it's better to
> catch them early during the development cycle.
> 
> John Dickinson prefers incremental and small changes, whereas clayg
> looks to like giant patches to fix all Python 3 issues at once to avoid
> conflicts in other (non-Python3) changes. (Sorry, if I didn't summarized
> correctly the discussion we had yesterday.)
> 
> The problem is that it's hard to fix "all" Python 3 issues in a single
> patch, the patch would be super giant and just impossible to review.
> It's also annoying to have to write dozens of small patches: we loose
> time on merge conflicts, rebasing, random gate failures, etc.
> 
> I proposed a first patch serie of 6 changes to fix a lot of simple
> Python 3 issues "at once":
> [...]
> 
> The overall diff is impressive: "61 files changed, 233 insertions(+),
> 189 deletions(-)" ... but each change is quite simple. It's only one
> pattern replaced with a different pattern. For example, replace
> "unicode" with "six.text_type" (and add "import six" if needed). So
> these changes should be easy to review.
> 
> With a working (and voting?) python34 check job and these 6 changes, it
> will be (much) easier to work on porting Swift to Python 3. Following
> patches will be validated by the python34 check job, shorter and
> restricted to a few files.
> 
> Victor

That's great news. Thanks so much for your tireless efforts to get
Python 3 supported everywhere in OpenStack, Victor !

-- 
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [swift] Plan to port Swift to Python 3

[Victor Stinner]

Hi,

Le 09/10/2015 12:12, vishal yadav a écrit :

However I was just checking if you considered using 2to3. I can
understand that translation using this tool might not cover every area
in the code more specifically custom/3rd party libraries (non-standard
python libraries) but IMO it can do fixer translations to much extent.


I tried 2to3, modernize and 2to6 tools in the past, but they produce a 
single giant patch with unwanted changes. These tools are written to add 
compatibility for all Python versions including Python 2.6 and Python 
3.0. In OpenStack, we only care of Python 2.7 and 3.4, so the code can 
be simpler. For example, we can simply write u"unicode" instead of 
six.b("unicode").


I wrote the sixer tool for OpenStack. Basically, it's the same than 
2to6, except that:


- sixer respects OpenStack Coding Style on imports: it groups and sorts 
imports. It avoids to have to manually fix individual modified import 
which takes a lot of time


- sixer can produce a patch for a single pattern. For example, replace 
all unicode with six.text_type but nothing else. Since all changes are 
reviewed carefully in OpenStack, it's important to produce "reviewable" 
(small) changes.


See also my blog article which explains the full rationale:
http://haypo.github.io/python3-sixer.html

My patch serie of 6 changes to fix most Python 3 issues was almost fully 
generated by sixer. Sometimes, I had to manually fix a few lines because 
no tool is perfect ;-) The patch serie:


* "py3: Replace unicode with six.text_type"
  https://review.openstack.org/#/c/232476/

* "py3: Replace urllib imports with six.moves.urllib"
  https://review.openstack.org/#/c/232536/

* "py3: Use six.reraise() to reraise an exception"
  https://review.openstack.org/#/c/232537/

* "py3: Replace gen.next() with next(gen)"
  https://review.openstack.org/#/c/232538/

* "Replace itertools.ifilter with six.moves.filter"
  https://review.openstack.org/#/c/232539/

* "py3: Replace basestring with six.string_types"
  https://review.openstack.org/#/c/232540/

Then I will then use sixer on individual files to fix all Python 3 at once.

Victor

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [swift] Plan to port Swift to Python 3

[Victor Stinner]

Hi,

Good news, we made good progress last weeks on porting Swift to Python 
3, a few changes were merged and all dependencies now work on Python 3. 
We only need two more simple changes to have a working pyhon34 check job:


* "py3: Update pbr and dnspython requirements"
  https://review.openstack.org/#/c/217423/
* "py3: Add py34 test environment to tox"
  https://review.openstack.org/#/c/199034/

With these changes, it will be possible to make the python34 check job 
voting to avoid Python 3 regressions. It's very important to avoid 
regressions, so we cannot go backward again in Python 3 support.


On IRC, it was said that it's better to merge Python 3 changes at the 
beginning of the Mitaka cycle, because Python 3 requires a lot of small 
changes which can likely introduce (subtle) bugs, and it's better to 
catch them early during the development cycle.


John Dickinson prefers incremental and small changes, whereas clayg 
looks to like giant patches to fix all Python 3 issues at once to avoid 
conflicts in other (non-Python3) changes. (Sorry, if I didn't summarized 
correctly the discussion we had yesterday.)


The problem is that it's hard to fix "all" Python 3 issues in a single 
patch, the patch would be super giant and just impossible to review. 
It's also annoying to have to write dozens of small patches: we loose 
time on merge conflicts, rebasing, random gate failures, etc.


I proposed a first patch serie of 6 changes to fix a lot of simple 
Python 3 issues "at once":


* "py3: Replace unicode with six.text_type"
  https://review.openstack.org/#/c/232476/

* "py3: Replace urllib imports with six.moves.urllib"
  https://review.openstack.org/#/c/232536/

* "py3: Use six.reraise() to reraise an exception"
  https://review.openstack.org/#/c/232537/

* "py3: Replace gen.next() with next(gen)"
  https://review.openstack.org/#/c/232538/

* "Replace itertools.ifilter with six.moves.filter"
  https://review.openstack.org/#/c/232539/

* "py3: Replace basestring with six.string_types"
  https://review.openstack.org/#/c/232540/

The overall diff is impressive: "61 files changed, 233 insertions(+), 
189 deletions(-)" ... but each change is quite simple. It's only one 
pattern replaced with a different pattern. For example, replace 
"unicode" with "six.text_type" (and add "import six" if needed). So 
these changes should be easy to review.


With a working (and voting?) python34 check job and these 6 changes, it 
will be (much) easier to work on porting Swift to Python 3. Following 
patches will be validated by the python34 check job, shorter and 
restricted to a few files.


Victor

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev