Re: What is your experience porting Python 2.7.x scripts to Python 3.x?
After chickening out a couple of times over the past few years, about eight months ago we migrated our small code base from 2.7.14 to 3.6.5. Some notes: On 2.7 we spent a couple of years coding with 3.x in mind, using import from __future__ and coding to Python 3 standards wherever possible. This greatly reduced the number of changes put into production during cutover and got us used to Python 3. We used 2to3.py and found that it handled easy cases (print) well and provided a good check on my work when I hand-converted. The string versus bytes issue was the most difficult. We receive many files from other financial institutions, who receive some of the data from yet other FIs. Most of them seem to have little understanding of, or interest in, encodings, and claim to send the files in 'ASCII'. Python 3 is intolerant of dirty data, whereas previous Pythons let anything go. I because very familiar with encoding= and errors= on open(). And network protocols want bytes. I also suggest paying special attention to division (/). We used the __future__ import to handle this early. We had some problems installing Python 3.6.5 on Windows Server 2012 that already had 2.7, particularly Registry settings that wound up being wrong. The problems seemed to occur erratically, so YMMV. After some trial and error, we chose to leave Python 2.7 installed and install Python launcher for Windows (PyLauncher), to allow for easy fallback (which we never had to do) and because this provided the smoothest install. We have found Python 3.6.5 to be very reliable on Windows Server 2012. My next project is to upgrade to Windows Server 2016 and Python 3.7 on a new machine. We will not be installing Python 2.7 on this machine. HTH, Steve J. Martin -- https://mail.python.org/mailman/listinfo/python-list
Re: What is your experience porting Python 2.7.x scripts to Python 3.x?
On 2019-01-22 19:20, Grant Edwards wrote: > > For anyone who has moved a substantial bunch of Python 2 to Python > > 3, can you please reply with your experience? > > If you used bytes (or raw binary strings) at all (e.g. for doing > things like network or serial protocols) you're in for a lot of > pain. This was my biggest pain point, but it was a good pain. At $DAYJOB we had files coming from customers and telecom providers where the encoding had never been specified. By going through the conversion process, we were able to formalize the encoding of various files meaning fewer crashes when some unexpected character would slip in and fail to convert. A painful process, but the end result was better in a multitude of ways. -tkc -- https://mail.python.org/mailman/listinfo/python-list
Re: What is your experience porting Python 2.7.x scripts to Python 3.x?
Robin Becker writes: > On 22/01/2019 19:00, Schachner, Joseph wrote: > .. >> For anyone who has moved a substantial bunch of Python 2 to Python 3, >> can you please reply with your experience? Did you run into any >> issues? Did 2to3 do its job well, or did you have to review its >> output to eliminate some working but silly thing? >> > ,.. > I did the port of reportlab (www.reportlab.com) from code supporting > 2.x only x>=3 to a version which supported 2.7.z & >=3.3. The > reportlab toolkit was then about 14 years old and had (and still has > lots of cruft). The port effort began 20130215 and ended 20140326 ie > 13 months. There were 333 commits on the branch. I used 2to3, but not > six. Because we needed to maintain 2.7 and >=3.3 there were quite a > few issues related to simple things like iterkeys/values/items <--> > keys/values/items removal of xrange etc etc. > > Maintaining compatible extensions is also hard. Pdf production > requires byte output and was already quite messy because reportlab > supported both utf8 and unicode inputs; it hasn't got a lot easier. > > As for performance the 3.x runs were generally slower than 2.7, but I > think that situation has changed with 3.6 & 3.7 (for the reportlab > tests on windows 2.7 takes 68.7", 3.4 83.8", 3.5 77.0", 3.6 61.5" & > 3.7 60.9"). > > At some point reportlab will be made 3.x only which will require more > effort. Packages like reportlab with a need to support both Python 2 and 3 end up with the worst of both worlds. The initial drive for Py3k was to drop cruft that had accumulated over the years. Mixing old and new hampers your ability to write clean 3 code. -- Pete Forman -- https://mail.python.org/mailman/listinfo/python-list
Re: What is your experience porting Python 2.7.x scripts to Python 3.x?
On 23/01/2019 21:51, Ian Kelly wrote: On Wed, Jan 23, 2019 at 1:36 PM Stefan Behnel wrote: . All right, but apart from absolute imports, the print function, and true division, what has Python 3.x ever done for us? *ducks* headaches :) -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: What is your experience porting Python 2.7.x scripts to Python 3.x?
On 22/01/2019 19:00, Schachner, Joseph wrote: .. For anyone who has moved a substantial bunch of Python 2 to Python 3, can you please reply with your experience? Did you run into any issues? Did 2to3 do its job well, or did you have to review its output to eliminate some working but silly thing? ,.. I did the port of reportlab (www.reportlab.com) from code supporting 2.x only x>=3 to a version which supported 2.7.z & >=3.3. The reportlab toolkit was then about 14 years old and had (and still has lots of cruft). The port effort began 20130215 and ended 20140326 ie 13 months. There were 333 commits on the branch. I used 2to3, but not six. Because we needed to maintain 2.7 and >=3.3 there were quite a few issues related to simple things like iterkeys/values/items <--> keys/values/items removal of xrange etc etc. Maintaining compatible extensions is also hard. Pdf production requires byte output and was already quite messy because reportlab supported both utf8 and unicode inputs; it hasn't got a lot easier. As for performance the 3.x runs were generally slower than 2.7, but I think that situation has changed with 3.6 & 3.7 (for the reportlab tests on windows 2.7 takes 68.7", 3.4 83.8", 3.5 77.0", 3.6 61.5" & 3.7 60.9"). At some point reportlab will be made 3.x only which will require more effort. -- Robin Becker -- https://mail.python.org/mailman/listinfo/python-list
Re: What is your experience porting Python 2.7.x scripts to Python 3.x?
On Wed, Jan 23, 2019 at 1:36 PM Stefan Behnel wrote: > > Cameron Simpson schrieb am 23.01.19 um 00:21: > > from __future__ import absolute_imports, print_function > > > > gets you a long way. > > ... and: division. All right, but apart from absolute imports, the print function, and true division, what has Python 3.x ever done for us? *ducks* -- https://mail.python.org/mailman/listinfo/python-list
Re: What is your experience porting Python 2.7.x scripts to Python 3.x?
Cameron Simpson schrieb am 23.01.19 um 00:21: > from __future__ import absolute_imports, print_function > > gets you a long way. ... and: division. Stefan -- https://mail.python.org/mailman/listinfo/python-list
Re: What is your experience porting Python 2.7.x scripts to Python 3.x?
On 23Jan2019 14:15, Grant Edwards wrote: On 2019-01-22, Cameron Simpson wrote: On 22Jan2019 19:20, Grant Edwards wrote: On 2019-01-22, Schachner, Joseph wrote: For anyone who has moved a substantial bunch of Python 2 to Python 3, can you please reply with your experience? If you used bytes (or raw binary strings) at all (e.g. for doing things like network or serial protocols) you're in for a lot of pain. Yes, but you will be the better for it afterwards. It's a lot better if you're leaving Python2 behind. If you're maintaing 2/3 compatible code that uses bytes, the pain is chronic rather than acute. Um, yes, very true. I've had a few programs which worked with binary data, and often also "text". In Python 2 there was _constant_ uncertanty when these were mixed (writing text into binary fields and related). In Python 3 I am never confused. It is a huge win. For whatever reason, I guess I never ran into that very much. The instance I have in mind was a fairly complex bit of code doing lots of binary and also strings in Python 2. It is now Python 3 only and far cleaner. As as you suggest, _not_ Python 2 compatible. Cheers, Cameron Simpson -- https://mail.python.org/mailman/listinfo/python-list
Re: What is your experience porting Python 2.7.x scripts to Python 3.x?
On 2019-01-22, Cameron Simpson wrote: > On 22Jan2019 19:20, Grant Edwards wrote: >>On 2019-01-22, Schachner, Joseph wrote: >>> For anyone who has moved a substantial bunch of Python 2 to Python >>> 3, can you please reply with your experience? >> >>If you used bytes (or raw binary strings) at all (e.g. for doing >>things like network or serial protocols) you're in for a lot of pain. > > Yes, but you will be the better for it afterwards. It's a lot better if you're leaving Python2 behind. If you're maintaing 2/3 compatible code that uses bytes, the pain is chronic rather than acute. > I've had a few programs which worked with binary data, and often > also "text". In Python 2 there was _constant_ uncertanty when these > were mixed (writing text into binary fields and related). In Python > 3 I am never confused. It is a huge win. For whatever reason, I guess I never ran into that very much. -- Grant Edwards grant.b.edwardsYow! Here I am in the at POSTERIOR OLFACTORY LOBULE gmail.combut I don't see CARL SAGAN anywhere!! -- https://mail.python.org/mailman/listinfo/python-list
Re: What is your experience porting Python 2.7.x scripts to Python 3.x?
On Wed, Jan 23, 2019 at 6:10 PM dieter wrote:
> > Did 2to3 do its job well
>
> I have not used "2to3" -- because I doubt, that it can handle
> important cases, i.e. when a Python 2 "str" must become a Python 3 bytes
> or when a "dict.{keys, values, items}" must be listified.
Have you tried? It errs on the side of safety. Your code should be
functional, and then if you decide that they _don't_ need to be
listified, you can remove the list() call.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: What is your experience porting Python 2.7.x scripts to Python 3.x?
"Schachner, Joseph" writes:
> ...
> For anyone who has moved a substantial bunch of Python 2 to Python 3, can
> you please reply with your experience?
It can be simple and it can be difficult.
I have found "http://python-future.org/compatible_idioms.html";
especially useful.
> Did you run into any issues?
Besides the already mentioned "urllib*" and "bytes/str" problems,
I have had significant issues with "dict" (missing "iter*" methods, missing
"has_key").
Another important problem domain are doctests with unicode results.
> Did 2to3 do its job well
I have not used "2to3" -- because I doubt, that it can handle
important cases, i.e. when a Python 2 "str" must become a Python 3 bytes
or when a "dict.{keys, values, items}" must be listified.
--
https://mail.python.org/mailman/listinfo/python-list
Re: What is your experience porting Python 2.7.x scripts to Python 3.x?
On 22Jan2019 19:20, Grant Edwards wrote: On 2019-01-22, Schachner, Joseph wrote: For anyone who has moved a substantial bunch of Python 2 to Python 3, can you please reply with your experience? If you used bytes (or raw binary strings) at all (e.g. for doing things like network or serial protocols) you're in for a lot of pain. Yes, but you will be the better for it afterwards. I've had a few programs which worked with binary data, and often also "text". In Python 2 there was _constant_ uncertanty when these were mixed (writing text into binary fields and related). In Python 3 I am never confused. It is a huge win. The pain here is completely offset by the relief which comes later. Everything else is pretty minor. Largely. It is also possible to write a _lot_ of code compatible with both 2 and 3. from __future__ import absolute_imports, print_function gets you a long way. It will force these 2 things on your Python 2, making it Python 3 ready in that regard before you cut over. I gather the 2to3 tool is useful, but it generates _separate_ Python 3 code from your python 2 codebase as I understand it. I don't like that maintenance burden. I went with portability myself for most things, and a small personal library of python 3 flavoured routines with python 2 ports for some differing behaviour (because that gets me native Python 3 performance in Python 3, post cutover - the library is basicly "import the Python 3 names" in Python 3, and "implement the same names in Python 2" where needed for the Python 2). The "six" library is apparently the go to one for this kind of thing, I gather. The pain level is really pretty low. The bytes vs strings stuff is the most difficult, but the outcome is vastly better afterwards. Cheers, Cameron Simpson -- https://mail.python.org/mailman/listinfo/python-list
Re: What is your experience porting Python 2.7.x scripts to Python 3.x?
On Wed, Jan 23, 2019 at 9:43 AM Akkana Peck wrote: > > Grant Edwards writes: > > On 2019-01-22, Schachner, Joseph wrote: > > > > > For anyone who has moved a substantial bunch of Python 2 to Python > > > 3, can you please reply with your experience? > > > > If you used bytes (or raw binary strings) at all (e.g. for doing > > things like network or serial protocols) you're in for a lot of pain. > > > > Everything else is pretty minor. > > 2to3 handles most porting issues nicely. The three pain points I've > hit are .. networking, anything that > uses urllib, urllib2 or related libraries. For this last one, I would recommend first porting your code to using 'requests' as much as possible, if you aren't already. The requests API is consistent across Py2 and Py3, and then all you need urllib itself for is a handful of utility functions like urllib.parse, which I believe 2to3 can handle. Yes, requests is third-party. But it's such a good library that I recommend it fully any time you aren't utterly unable to use non-core libraries. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: What is your experience porting Python 2.7.x scripts to Python 3.x?
Grant Edwards writes: > On 2019-01-22, Schachner, Joseph wrote: > > > For anyone who has moved a substantial bunch of Python 2 to Python > > 3, can you please reply with your experience? > > If you used bytes (or raw binary strings) at all (e.g. for doing > things like network or serial protocols) you're in for a lot of pain. > > Everything else is pretty minor. 2to3 handles most porting issues nicely. The three pain points I've hit are the bytes vs strings vs unicode problem Grant mentions; GUIs in either Qt or (especially) GTK; and networking, anything that uses urllib, urllib2 or related libraries. For those three issues I usually have to do a lot of porting and debugging by hand, and a lot of testing afterward because there are always problems that crop up later upon discovering that the Python3 version of some library is returning bytes when Python3 wants chars, or some such. If you don't have automated tests set up, consider writing some now, before you start porting, with particular emphasis on anything that gets data from a network or writes it to a file. ...Akkana -- https://mail.python.org/mailman/listinfo/python-list
Re: What is your experience porting Python 2.7.x scripts to Python 3.x?
On 2019-01-22, Schachner, Joseph wrote: > For anyone who has moved a substantial bunch of Python 2 to Python > 3, can you please reply with your experience? If you used bytes (or raw binary strings) at all (e.g. for doing things like network or serial protocols) you're in for a lot of pain. Everything else is pretty minor. -- Grant Edwards grant.b.edwardsYow! Are we live or on at tape? gmail.com -- https://mail.python.org/mailman/listinfo/python-list
