[issue5901] missing meta-info in documentation pdf
Change by C.A.M. Gerlach : -- keywords: +patch nosy: +CAM-Gerlach nosy_count: 5.0 -> 6.0 pull_requests: +30424 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/32386 ___ Python tracker <https://bugs.python.org/issue5901> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5993] python produces zombie in webbrowser.open
C.A.M. Gerlach added the comment: I can verify that this still occurs on Linux under an up to date Python 3.9 env (unknown browser; I can inquire), while on my Windows machine running an essentially identical Python 3.9 env, I don't get the warning. The practical problem arises when we are using `-X dev`/`-W error` and/or strict `filterwarnings` with Pytest, and the user passes a custom `--open-browser` flag to our test suite to open the resulting output in their web browser to inspect it for themselves. See [spyder-ide/docrepr#33](https://github.com/spyder-ide/docrepr/pull/33#discussion_r772164817) for details. I'm concerned this is going to be difficult to work around without doing over-broad silencing of warnings on the Pytest, because the warning is raised when the object is GC'ed, which may happen at a non-deterministic time or even after or even Python invocation level, and I can't reproduce it on my machine to test it. -- nosy: +CAM-Gerlach status: pending -> open ___ Python tracker <https://bugs.python.org/issue5993> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40358] pathlib's relative_to should behave like os.path.relpath
C.A.M. Gerlach added the comment: For the record, requested [on Discourse](https://discuss.python.org/t/how-to-go-on-with-a-tiny-improvement-idea-for-pathlib/12472) as well, with a fairly similar proposal. -- nosy: +CAM-Gerlach ___ Python tracker <https://bugs.python.org/issue40358> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32592] Drop support of Windows Vista and Windows 7
C.A.M. Gerlach added the comment: In my limited understanding, all PR #15951 does is set a compiler macro to allow use of Win API calls present on >=Win8 instead of >=Win7. If Win8-only calls are not used, then presumably it should still build and run on Windows 7, presumably with the flag flipped back to Win7. And if there are Win8-only calls used and the flag is set to Win7+, I assume that the MSVC compiler will catch any instances of Win8-only calls at build time? Is this understanding correct? If the latter is true, its very likely a lost cause. However, if the former is the case, then at least devs using Python 3.9+ on Windows 7 will be able to easily build their own versions with support, though that could change at any time. If the upside to supporting the latest Python versions on an EoL platform that is no longer receiving security updates but still has significant use is greater than the (seemingly pretty small) benefits of dropping support, then I imagine it would at least be conceivable to switch the flag back, though it probably is a lost cause now that a major version has been released without support and the buildbots have been presumably retired. At least third parties should be able to easily release patched versions, for now. For those in the know, have there been a lot of reports/discontent surrounding the dropping of support in Py3.9, by anyone aware of that? However, I don't think it should be tied to the ESU date at all, as opposed to real-world usage of bleeding-edge Python on that version; ESU is a paid-only service for large corporate customers; if they're willing to pay through the nose for updates to their EoL OS and also need the latest bleeding-edge Python, surely they can fund the pittance required for maintenance or building it themselves. All that said, I'll hold off on doing anything further until there's consensus. I'd think whatever we do it should be consistent, at least in the docs and installer; if dropping support in the code itself is more risk and pain than benefit, then that needn't be done. -- ___ Python tracker <https://bugs.python.org/issue32592> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32592] Drop support of Windows Vista and 7 in Python 3.9
C.A.M. Gerlach added the comment: Sorry I never got around to this, I was planning on it and then life and COVID intervened. I can give it another look now to get it in before the 3.10 feature freeze (since I at least documented in detail what each change should be, so it should be relatively straightforward) unless someone else would rather take it. > PEP 11 says that "[a] new feature release X.Y.0 will support all Windows > releases whose extended support phase is not yet expired". There was no such > thing as Extended Security Updates (ESU) when that provision was accepted. As > defined by PEP 11, extended support for Windows 7 ended on 2020-01-14. Yeah, "Extended Support" has a very specific, defined meaning [1] in Microsoft standard terminology (and other vendors as well), as the support phase after "Mainstream Support" but before the product end of life, where security updates and potentially limited bugfixes are still offered to all customers and the product is still considered supported, but new features and phone support is not. This ended 2020-01-24. [1] https://docs.microsoft.com/en-us/lifecycle/policies/fixed ESU [2] is a specialized, paid program available to high volume enterprise customers as a "last resort option" that pay big $$$ to continue receiving critical security updates for a limited time. This fulfills neither the letter nor the spirit of the policy defined in PEP 11; if such vendors need support for bleeding-edge Python versions for some reason, it certainly should be well within their means to fund someone to do so. [2] https://docs.microsoft.com/en-us/lifecycle/faq/extended-security-updates -- versions: +Python 3.10 -Python 3.9 ___ Python tracker <https://bugs.python.org/issue32592> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29982] tempfile.TemporaryDirectory fails to delete itself
C.A.M. Gerlach added the comment: Thanks! Submitted as PR #24793 https://github.com/python/cpython/pull/24793 -- ___ Python tracker <https://bugs.python.org/issue29982> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25024] Allow passing "delete=False" to TemporaryDirectory
Change by C.A.M. Gerlach : -- pull_requests: +23561 pull_request: https://github.com/python/cpython/pull/24793 ___ Python tracker <https://bugs.python.org/issue25024> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29982] tempfile.TemporaryDirectory fails to delete itself
Change by C.A.M. Gerlach : -- keywords: +patch pull_requests: +23560 stage: -> patch review pull_request: https://github.com/python/cpython/pull/24793 ___ Python tracker <https://bugs.python.org/issue29982> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25024] Allow passing "delete=False" to TemporaryDirectory
C.A.M. Gerlach added the comment: To note, the proposal on [BPO-29982](https://bugs.python.org/issue29982) should hopefully address one of the two use-cases described by Anthony Sotittle, `ignore_errors=True`, in a cleaner fashion that takes advantage of the existing higher-level functionality rather than duplicating `mkdtemp`. Opinions and feedback welcome over there. -- nosy: +CAM-Gerlach ___ Python tracker <https://bugs.python.org/issue25024> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29982] tempfile.TemporaryDirectory fails to delete itself
C.A.M. Gerlach added the comment: In addition to transient failures, this can also occur when, for example, files opened in the temporary directory (perhaps by library or application code not under direct control of the caller) haven't been properly cleaned up and their file handles don't get closed, resulting in permissions errors trying to delete them (particularly on platforms like Windows, that automatically lock files when opening them). This case came up in e.g. [this recent PR](https://github.com/regebro/pyroma/pull/57) and rendered `tempfile.TemporaryDirectory` unusable for such use cases, forcing a reversion to the lower-level `tempfile.mkdtemp` without the cleaner, more robust and easier to interpret high-level interface that the former provides. Wrapping a `with` statement in a try-finally is syntactically ugly and semantically incongruous, and requires a second shutil.rmtree(..., ignore_errors=True)` call to clean up in a best-effort manner, while when manually calling `cleanup()` in a try-except, the finalizer still gets executed at a a non-deterministic later time when Python exits, raising an error. Therefore, in the spirit of Guido's statements above in terms of providing a "best-effort" cleanup, I propose (and am willing to submit a PR to implement) adding an `ignore_errors` bool parameter (defaulting to False, of course, for backward compat--and should it be keyword only like `errors` to `TemporaryFile`?) to the `tempfile.TemporaryDirectory` constructor, which gets passed to `shutil.rmtree` on cleanup. This would not only address both cases here, but also one of the two discussed by Anthony Sotitle on [BPO-25024](https://bugs.python.org/issue25024), in a cleaner and simpler fashion that would take advantage of existing `tempfile.TemporaryDirectory` functionality and behavior. Would a PR be accepted on this? If so, any specific guidance on tests and whether to mention it in What's New, etc., would be appreciated. Thanks! -- nosy: +CAM-Gerlach ___ Python tracker <https://bugs.python.org/issue29982> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32592] Drop support of Windows Vista and 7 in Python 3.9
C.A.M. Gerlach added the comment: Thanks, will do as requested. I was planning on knocking it out yesterday, but some unexpected things came up so I might have to push it all the way to next weekend, but I'll get it done. -- ___ Python tracker <https://bugs.python.org/issue32592> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32592] Drop support of Windows Vista and 7 in Python 3.9
C.A.M. Gerlach added the comment: Hey, any update on this? As I mentioned, I'm willing to implement everything I mention here in one or more PRs, but I wanted to wait for a go-ahead in case some of the changes were not desired. Let me know and I'll go ahead. Thanks! -- ___ Python tracker <https://bugs.python.org/issue32592> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32592] Drop support of Windows Vista and 7 in Python 3.9
C.A.M. Gerlach added the comment: What Eryk said wrt Windows 8 seems sound, looking at some additional data. Per Statcounter [0], Windows 8 market share has dropped to 1.32% from 2.20% of all Windows users over the past 12 months, or 1.75% -> 1.01% of all desktop users. Extrapolating a constant linear decrease, this would imply a usage share of 0.55-0.60% at Python 3.9 release, or considering a more conservative (and realistic) constant-percent decrease, this would put the amount at 0.75%. For comparison, by Statcounter's same metrics, Windows XP which was EoL 5 versions ago is still at 1.13% of the Windows or 0.90% of the total desktop market today, WinVista is at 0.41%/0.33%, Win8.1 at 4.7%/3.8% (~4x Win8) and of course the EoL and also dropped Windows 7 is at no less than 23.2%/18.5%, nearly 20 times the amount of Win 8 today and dropping more slowly besides. [0] https://gs.statcounter.com/os-version-market-share/windows/desktop/worldwide Conducting a similar analysis with the NetMarketshare data [1], we see a 0.78% -> 0.66% year on year share for Win8, which extrapolates to a 0.57% share at Py2.9 release. Vista is much lower at only 0.15%, but XP is over 2x higher at 1.35%, and Win8.1 well over 5x as large at 3.5%. Windows 7 is, again, at no less than 25.2%, nearly 40x the Win 8 marketshare. [1] https://netmarketshare.com/operating-system-market-share.aspx Users likely to be most concerned with wanting or needing to use the latest and greatest version of Python are more than likely to be power users, enthusiasts and developers, a group that was particularly averse to adopting Windows 8 when it came out, was more likely to upgrade to 8.1 , and is more aware of EoL timeframes than the average consumer. This is empirically supported by the Steam survey [2], which samples a population substantially more similar to the profile of a developer than the average web user captured by the previous two surveys. Here, the numbers are much more stark: A mere 0.17% of their >96% Windows userbase runs Windows 8, vs. 2.10% (>10x) for Windows 8.1 and 12.4% for Windows 7 (~75x). Given an infinitesimal fraction of such remaining users are likely going to require upgrading to a bleeding-edge Python version but not their OS that would have been EoL for nearly 5 years, over a year longer than Vista itself and 4 years longer than 7, I don't think supporting that EoL OS per PEP 11 for another >5+ years with another new Python version need be a priority. [2] https://store.steampowered.com/hwsurvey/Steam-Hardware-Software-Survey-Welcome-to-Steam -- ___ Python tracker <https://bugs.python.org/issue32592> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32592] Drop support of Windows Vista and 7 in Python 3.9
C.A.M. Gerlach added the comment: Here's a further conservative list of low-hanging https://github.com/python/cpython/blob/master/Lib/ntpath.py#L675 * multiprocesing/connection.py | >=Win 8 | Branch can be inlined and outdated comments removed: https://github.com/python/cpython/blob/master/Lib/multiprocessing/connection.py#L866 * pathlib.py | >= Vista | Branch can be inlined and else block removed: https://github.com/python/cpython/blob/master/Lib/pathlib.py#L19 Tests: * test_winreg.py | Vista | Multiple tests and constants (All the checks and tests requiring registry reflection/`HAS_REFLECTION`): https://github.com/python/cpython/blob/master/Lib/test/test_winreg.py * test_winreg.py | https://github.com/python/cpython/blob/master/Lib/test/test_winreg.py#L309 * test/support/__init__ | <=Win 9x (!) | Test initialization branch can be inlined and out of date comments removed: https://github.com/python/cpython/blob/master/Lib/test/support/__init__.py#L987 * test_winconsoleio.py | <=Win7 | Test branch: https://github.com/python/cpython/blob/master/Lib/test/test_winconsoleio.py#L99 * test_winconsoleio.py | <=Vista | Skipif decorator : https://github.com/python/cpython/blob/master/Lib/test/test_winconsoleio.py#L99 * test_import/__init__.py | https://github.com/python/cpython/blob/master/Lib/test/test_import/__init__.py#L1025 Other: * "Since we limit WINVER to Windows 7 anyway, it doesn't really matter which WinSDK version we use." -> can be updated to 8? https://github.com/python/cpython/blob/master/PCbuild/python.props#L101 -- ___ Python tracker <https://bugs.python.org/issue32592> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32592] Drop support of Windows Vista and 7 in Python 3.9
C.A.M. Gerlach added the comment: I'd be happy to submit a PR to resolve all the issues that I note here. Just say the word and I'll go right ahead. I presume the Windows installer code [0a] should be updated to refuse to install on Windows <8 and given an informative warning instead letting the user know their system is not supported, as well as logging the appropriate messages in each case? Appropriate failure IDs would also be defined in the associated WXL [0b]. [0a] https://github.com/python/cpython/blob/master/Tools/msi/bundle/bootstrap/PythonBootstrapperApplication.cpp#L3041 [0b] https://github.com/python/cpython/blob/master/Tools/msi/bundle/Default.wxl The relevant Using Python on Windows docs [1a] should be updated to reflect this, along with PCBuild/readme.txt [1b]. Also, given it is dropping 2 major Windows versions, including one seeing widespread use, it should probably be mentioned in What's New; I don't see it there as it stands now. [1a] https://github.com/python/cpython/blob/master/Doc/using/windows.rst [1b] https://github.com/python/cpython/blob/master/PCbuild/readme.txt As a more trivial docs change, references to "Vista's" UAC ([2a], [2b], [2c]) could be updated to simply refer to "Windows'" UAC to avoid confusion since it presumably applies to all >=Vista versions which have said feature. [2a] https://github.com/python/cpython/blob/master/Lib/distutils/command/bdist_wininst.py#L52 [2b] https://github.com/python/cpython/blob/master/PC/bdist_wininst/install.c#L2187 [2c] https://github.com/python/cpython/blob/master/Doc/distutils/builtdist.rst#vista-user-access-control-uac Finally, maybe this is stretching the scope a bit, but the relevant part of [1a], namely [3a], should also be updated to state that Windows CE is not supported (it currently states the opposite), as this was removed in 3.6.0b1 [3ai], and additionally the Sourceforge (!) project it links to [3bii] is on an legacy archived CVS repo that hasn't been meaningfully updated since 2008 and the current versions of CE (5 and 6) at the time is long EoL now (in fact, 7 is coming up on its EoL in a year). Out of date references to Windows CE support could also be removed from the ctypes doc [3b] and the condavar code comments [3c], [3d]. [3a] https://github.com/python/cpython/blob/master/Doc/using/windows.rst#L1165 [3ai] https://github.com/python/cpython/blob/master/Misc/NEWS.d/3.6.0b1.rst#L217 [3aii] https://sourceforge.net/projects/pythonce/files/ [3b] https://github.com/python/cpython/blob/master/Doc/library/ctypes.rst [3c] https://github.com/python/cpython/blob/master/Include/internal/pycore_condvar.h#L30 [3d] https://github.com/python/cpython/blob/master/Python/condvar.h#L85 -- nosy: +CAM-Gerlach ___ Python tracker <https://bugs.python.org/issue32592> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36490] Modernize function signature in Archiving section of shutil doc
C.A.M. Gerlach added the comment: Closing on grounds of apparently not really being an issue as opposed to a misunderstanding on my part, as mentioned above. -- resolution: -> not a bug stage: needs patch -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue36490> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36490] Modernize function signature in Archiving section of shutil doc
C.A.M. Gerlach added the comment: Sorry for the lack of response on this one; it just slipped off my radar. It seems quite clear that my understanding was incomplete as to why the signatures were expressed this way. As such, I presume this can just be closed as NOTABUG? I'll close in a week (assuming I remember) if I don't hear otherwise or someone else doesn't first. Thanks and sorry for bothering folks on this. -- ___ Python tracker <https://bugs.python.org/issue36490> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30661] Support tarfile.PAX_FORMAT in shutil.make_archive
C.A.M. Gerlach added the comment: Thanks for the confirmation! -- ___ Python tracker <https://bugs.python.org/issue30661> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36490] Modernize function signature format in Archiving section of shutil doc
New submission from C.A.M. Gerlach : In the process of updating the documentation for another issue, I noticed that unlike the rest of the shutil doc (and the Python docs in general, not to mention those of virtually every Python package), all the functions in the [Archiving operations section](https://docs.python.org/3/library/shutil.html#archiving-operations) uses the old style, difficult to parse nested-bracket notation for the function signatures, rather then the modern style displaying them as they would be expected to appear in Python code, with clearly and explicitly indicated defaults. Therefore, given all bracketed items are keyword arguments with defaults, and there are no cases more complex then the standard linearly-nested brackets, is there a particular reason why this was retained? Otherwise, I can go ahead and submit a PR to update this. -- assignee: docs@python components: Documentation messages: 339243 nosy: CAM-Gerlach, docs@python priority: normal severity: normal status: open title: Modernize function signature format in Archiving section of shutil doc versions: Python 3.8 ___ Python tracker <https://bugs.python.org/issue36490> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30661] Support tarfile.PAX_FORMAT in shutil.make_archive
C.A.M. Gerlach added the comment: I opened a PR to implement both those changes, and also added some minor related clarifications and fixes to the format section of the tarfile docs. > how tarfile.Tarfile behaves if you tell it to open a PAX_FORMAT archive using > GNU_FORMAT or vice-versa I tested tarfile.Tarfile() and extract_all() on the resulting object with several different simple- to moderately-complex (including Unicode filenames) real-world pax- and GNU-format archives packed with different archivers, with both format=GNU_FORMAT and format=PAX_FORMAT for each one, got no warnings or errors with debug=3 and errorlevel=2, and extraction was successful and yielded identical results for either format argument, and did not get a PAXHEADERS file output for either one. Furthermore, tracing the code, its not clear that Tarfile() (with 'r') and extract, etc. use the passed `format`. Even if so, in order to produce an error after this change but not before, all of the following would seem to have to be the case: * The tarfile being read would have to be in GNU format, i.e. from an external source or produced with an older version of Python * The tarfile would have to make use of specific extended/non-standard GNU tar features not tested above * The user would have to use Tarfile() to open the tarfile, rather than one of the other, more common/higher-level methods * The user's call to Tarfile() would have to have used DEFAULT_FORMAT rather than being explicitly specified. and implicitly relied DEFAULT_FORMAT == GNU_FORMAT Therefore, this seems like a very specific corner-case. However, if you think I should include it, I'll go ahead with it. Also, let me know if these doc changes should have a separate NEWS entry or the previous one adequately covers it. -- ___ Python tracker <https://bugs.python.org/issue30661> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36268] Change default tar format to modern POSIX 2001 (pax) for better portability/interop, support and standards conformance
Change by C.A.M. Gerlach : -- pull_requests: +12567 ___ Python tracker <https://bugs.python.org/issue36268> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30661] Support tarfile.PAX_FORMAT in shutil.make_archive
Change by C.A.M. Gerlach : -- pull_requests: +12566 stage: needs patch -> patch review ___ Python tracker <https://bugs.python.org/issue30661> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36268] Change default tar format to modern POSIX 2001 (pax) for better portability/interop, support and standards conformance
C.A.M. Gerlach added the comment: Also, one additional minor note (since I apparently can't edit comments here). Windows 10 (since the April 2018 update a year ago) now includes libarchive-based bsdtar built-in by default and accessible from the standard command prompt, which as mentioned fully supports pax. Therefore, all modern platforms should support extracting them out of the box (aside from Windows 7/Server 2008, for which extended support will end within two months from Python 3.8's initial release, Windows 10 pre-1803 for which enterprise support will end a few months after that, and Windows 8.1/Server 2012, which will be in extended support for a few more years but very low enterprise/developer/power user adoption; of course, these don't include any built-in tar support at all anyway). -- ___ Python tracker <https://bugs.python.org/issue36268> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36307] Upgrade Travis CI config to Xenial from near-EoL Trusty and remove obsolete sudo: false key
C.A.M. Gerlach added the comment: Ah yes, I actually saw that before opening this issue; not sure why I completely forgot about it. Sorry about that and thanks for the reminder. -- ___ Python tracker <https://bugs.python.org/issue36307> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36307] Upgrade Travis CI config to Xenial from near-EoL Trusty and remove obsolete sudo: false key
C.A.M. Gerlach added the comment: Yes, absolutely. While were at it, should we also bump the OpenSSL version up to the latest 1.1.1b, which is the current LTS branch (with 1.0.2 EoL in Dec. 2019 and 1.1.0 EoL in a few months, in September)? -- components: +Build ___ Python tracker <https://bugs.python.org/issue36307> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36307] Upgrade Travis CI config to Xenial from near-EoL Trusty and remove obsolete sudo: false key
C.A.M. Gerlach added the comment: Sure, thing, sorry—I realized I should have added one but apparently couldn't edit my post to include it. [Here's](https://docs.travis-ci.com/user/reference/overview/#deprecated-virtualization-environments) a description of the removal of the obsolete sudo: false key in the Travis docs, and [here's](https://blog.ubuntu.com/2019/02/05/ubuntu-14-04-trusty-tahr) a blog post on the imminent EoL of [Trusty Tar](https://blog.ubuntu.com/2019/02/05/ubuntu-14-04-trusty-tahr). -- ___ Python tracker <https://bugs.python.org/issue36307> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36307] Upgrade Travis CI config to Xenial from near-EoL Trusty and remove obsolete sudo: false key
Change by C.A.M. Gerlach : -- keywords: +patch pull_requests: +12323 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue36307> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36307] Upgrade Travis CI config to Xenial from near-EoL Trusty and remove obsolete sudo: false key
New submission from C.A.M. Gerlach : Not sure how to categorize this one, or if this should even be a bpo. Currently, the .travis.yml config file still has a sudo: false key, which previously triggered the Container-based environment on Travis. However, that was removed in December 2018, and so the key is now obsolete. Furthermore, the builds are still running in the Trusty environment, which will be at the end of its 5-year development lifecycle in April 2019 (next month). Therefore, sudo: false should be removed, and the distribution should be upgraded to the modern dist: xenial. If non-trivial errors result, the change will likely be well beyond my minimal expertise as a brand-new contributor, but I figured I'd nevertheless make the attempt in a PR if not, which others can take over/supersede if necessary. -- messages: 338022 nosy: CAM-Gerlach priority: normal severity: normal status: open title: Upgrade Travis CI config to Xenial from near-EoL Trusty and remove obsolete sudo: false key ___ Python tracker <https://bugs.python.org/issue36307> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30661] Support tarfile.PAX_FORMAT in shutil.make_archive
Change by C.A.M. Gerlach : -- versions: +Python 3.8 -Python 3.7 ___ Python tracker <https://bugs.python.org/issue30661> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30661] Support tarfile.PAX_FORMAT in shutil.make_archive
C.A.M. Gerlach added the comment: FYI, [GH-12355](https://github.com/python/cpython/pull/12355) will implement pax as default, as discussed in [bpo-36268](https://bugs.python.org/issue36268), which should be equivalent to option 1 here, thus also resolving this issue. Could you confirm that this is the case, and do you have any other comments on the change? Thanks! -- components: +Library (Lib) nosy: +CAM-Gerlach ___ Python tracker <https://bugs.python.org/issue30661> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36268] Change default tar format to modern POSIX 2001 (pax) for better portability/interop, support and standards conformance
C.A.M. Gerlach added the comment: PR is up with CI checks green as [GH-12355](https://github.com/python/cpython/pull/12355). I also had to fix one test which implicitly assumed that DEFAULT_FORMAT == GNU_FORMAT. -- ___ Python tracker <https://bugs.python.org/issue36268> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36268] Change default tar format to modern POSIX 2001 (pax) for better portability/interop, support and standards conformance
Change by C.A.M. Gerlach : -- keywords: +patch pull_requests: +12320 stage: -> patch review ___ Python tracker <https://bugs.python.org/issue36268> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30661] Support tarfile.PAX_FORMAT in shutil.make_archive
Change by C.A.M. Gerlach : -- keywords: +patch pull_requests: +12321 stage: needs patch -> patch review ___ Python tracker <https://bugs.python.org/issue30661> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36268] Change default tar format to modern POSIX 2001 (pax) for better portability/interop, support and standards conformance
C.A.M. Gerlach added the comment: Sure, in work now. Its my first contribution to CPython, so bear with me. I presume this is too trivial to go in the What's New in Python article, but does merit a NEWS entry so users are aware of the change? Aside from changing [this line](https://github.com/python/cpython/blob/3fe7fa316f74ed630fbbcdf54564f15cda7cb045/Lib/tarfile.py#L108), updating the documentation to reflect the change, and possibly adding a NEWS entry, is there anything else that needs to be done? Thanks. -- ___ Python tracker <https://bugs.python.org/issue36268> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36268] Change default tar format to modern POSIX 2001 (pax) for better portability/interop, support and standards conformance
C.A.M. Gerlach added the comment: In general, since pax is a backwards-compatible superset of the standard, portable ustar unlike the vendor-specific GNU format that even GNU tar itself no longer recommends in favor of switching to pax by default, it is to my understanding essentially always the better choice. The only exception would be systems that support GNU tar but not POSIX 2001 and where the limitations of the old ustar must be bypassed, which as far I'm aware is basically just really old (>10-15 years) GNU/Linux. NetBSD and OpenBSD both use bsdtar implementations, which as far as I could find means they support the POSIX 2001-standard pax format, and (unless they use libarchive which supports all three) likely *don't* support the current GNU format which is specific to GNU tar. Even if they don't, their ustar support means they can read pax archives as legacy ustar archives (as pax is backwards-compatible), while the same is not necessarily true of GNU tar archives. Therefore, pax is strictly a better choice than GNU or ustar. Most other programming languages I could find did not have internal/standard library implementations, instead relying on the aforementioned libraries or varying third party packages: * For C/C++, Libarchive and GNU tar are the modern two heavy hitters, and they both have supported it for a very long long. Modern version of old-style bsdtar should, but if not then they don't support GNU tar either. These are commonly used when needed with C/C++, or programmers implement their own bespoke solutions. * Libtar (C) does not, but it hasn't been updated for 6 years (and has been in minimal maintenance mode for over 15) so I'm not sure its really relevant anymore. Virtually any platform will also have one of the previous. * The major implementation for Java, Apache Commons Compress, added support for both pax and GNU in its 1.2 version, back in 2011 (8 years ago) * R uses the system's tar executable (or bundled modern tar), so will have the same support as that (i.e. any remotely modern system should be compatible). Their documentation explicitly recommends against GNU tar in favor of pax or ustar instead for portability: https://stat.ethz.ch/R-manual/R-devel/library/utils/html/tar.html * git-archive uses pax exclusively * PHP supports ustar only, not pax or GNU; in that case pax is generally the more compatible of the two extended formats * The node-tar library, the apparent standard for Javascript, support it * The standard tar package for Go supports it * What seems to be the major current implementation for C#, SharpZipLib, supports it * Ruby has no apparent standard implementation; a few third-party libraries have a mix of support -- ___ Python tracker <https://bugs.python.org/issue36268> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36268] Change default tar format to modern POSIX 2001 (pax) for better portability/interop, support and standards conformance
New submission from C.A.M. Gerlach : I propose changing tarfile.DEFAULT_FORMAT to be tarfile.PAX_FORMAT , rather than the legacy tarfile.GNU_FORMAT for Python 3.8. This would offer several benefits: • Removes limitations of the old GNU tar format, including in max UID/GID values and bits in device major and minor numbers, and is the most flexible and feature-rich tar format currently • Encodes all filenames as UTF-8 in a portable way, ensuring consistent and correct handling on all platforms, avoid errors like [this one](https://stackoverflow.com/questions/19902544/tarfile-produce-garbled-file-name-in-the-tar-gz-archivement) and generally ensure expected, sensible defaults • Is the current interoperable POSIX standard, used by all modern platforms (Linux, Unix, macOS, and third-party unarchivers on Windows) rather than a vendor-specific extension like GNU tar • Backwards compatible with any unarchiver capable of reading ustar format, unlike GNU tar as the extended pax headers will just be ignored • Fixes bpo-30661, support tarfile.PAX_FORMAT in shutil.make_archive (was proposed as a fix to the same, but it was never followed up on and the issue remains open) This change would have no effect on reading existing archives, only writing new ones, and should be broadly compatible with any remotely modern system, as pax support is included in all the widely used libraries/systems: * POSIX 2001 (major Unix vendors), released in 2001 (18 years ago) * GNU tar 1.14 (Linux, etc), released in 2004 (15 years ago) * bsdtar/libtar ~1.2.51 (BSD, macOS, etc), at least as of 2006 (13 years ago), with significant bug fixes up through 2011 (8 years ago) * 7-zip (Windows) at some point before 2011 (>8 years ago), with significant bug fixes up to 2011 (8 years ago) * Python 2.6, released in 2008 (11 years ago) Furthermore, essentially every existing archiver supports ustar format, which would allow interoperability on very old/exotic platforms that don't support pax for some reason (and would certainly not support GNU). Therefore, it should be more than safe to make the change now, with archivers on the three major platforms supporting the modern standard for nearly a decade, and any esoteric ones at least as likely to support the POSIX standard as the vendor-specific GNU extension. Is there any particular reason why we shouldn't make this change? Is there a particular group/list I should contact to follow up about seeing this implemented? It seems it should only require a one-line change [here](https://stackoverflow.com/questions/19902544/tarfile-produce-garbled-file-name-in-the-tar-gz-archivement), aside from updating the docs and presumably the NEWS file, which I would be willing to do (I would think it should make a fairly straightforward first contribution). -- components: Library (Lib) messages: 337710 nosy: CAM-Gerlach priority: normal severity: normal status: open title: Change default tar format to modern POSIX 2001 (pax) for better portability/interop, support and standards conformance type: behavior versions: Python 3.8 ___ Python tracker <https://bugs.python.org/issue36268> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
