[issue43902] ssl module: add getter for SSL_CTX* and SSL*
Hans-Christoph Steiner added the comment:
This general idea sounds nice to have, I hope it can be included.
`ctx._call_with_ctypes("SSL_CTX_set_ciphersuites"...` also sounds totally
workable to me, if that has the best security profile.
Defense in depth is important, but it is not a reason to prevent key
functionality from landing. For example, "export_keying_material" is an RFC
and widely implemented (Go crypto/tls, Rustls, Conscrypt, nodejs, boringssl,
openssl, BouncyCastle, etc see links here
https://github.com/python/cpython/pull/25255#issuecomment-1073256270). It is
used in IETF protocols like SRTP and NTS.
Perhaps that could be a concrete use case here for thinking about the security
profile?
--
nosy: +eighthave
___
Python tracker
<https://bugs.python.org/issue43902>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37952] Add support for export_keying_material to SSL library
Hans-Christoph Steiner added the comment: I understand the frustrations here, but this is really not a place to vent, since that only harms everyone's interests. When a core maintainer voices concerns or questions, they need to be addressed. This goes for any project. I'll see if I can contribute to https://bugs.python.org/issue43902, that would also work for exporting keying material. -- ___ Python tracker <https://bugs.python.org/issue37952> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37952] Add support for export_keying_material to SSL library
Hans-Christoph Steiner added the comment: We're working on the HTTP Transport Auth draft (https://www.ietf.org/archive/id/draft-schinazi-httpbis-transport-auth-05.html) in the IETF that also needs this method. I would really love to see this land, any advice? If it is just a matter of updating the patch for the current Python, I can probably handle that. -- nosy: +eighthave ___ Python tracker <https://bugs.python.org/issue37952> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45567] Support TLS Encrypted ClientHello (ECH)
Hans-Christoph Steiner added the comment: I agree with all you say, but I think it is important to not rule out handling HTTPS/SVCB DNS here. It can happen at a later stage though. What you propose works great for the first step. If handling the DNS is punted to some external library, that means that each client app will have to implement this itself, and its non-trivial. In that scenario, very few clients will implement it. This makes me believe that this should be implemented in the core. Granted, its an open question whether the ssl module is the right place. For example, it could makes more sense to handle HTTPS/SVCB DNS in the HTTP libs (urllib, requests, etc), but that means non-HTTP uses (XMPP, Matrix, etc) will be disadvantaged. -- ___ Python tracker <https://bugs.python.org/issue45567> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45567] Support TLS Encrypted ClientHello (ECH)
New submission from Hans-Christoph Steiner : The next version of the IETF-standardized TLS protocol is known as Encrypted ClientHello (ECH) [1] formerly known as Encrypted SNI (ESNI). This ticket collects information for ECH support, and tracks which APIs have to be added to Python in order to implement ECH in Python's ssl module. ECH is built on top of TLSv1.3 and completes the unfinished work from the TLSv1.3 effort. It is now in draft-13 and there are many implementations that are interoperating. ECH is working for openssl[2], boringssl[3], nginx, Apache HTTPD, lighttpd, HAProxy, Conscrypt[4], curl, and more. There is work underway in Firefox [5] and Chromium [6]. It has been sketched out for OkHTTP [7]. Early versions of the standard, known as ESNI, have been deployed in Firefox releases and some production web services. ECH works in conjunction with the new DNS RR Types HTTPS and SVCB [8]. This means that DNS needs to be handled a bit differently. As far as I understand it, the ssl module has to gain additional features: 1. HTTPS/SVCB DNS queries for setting up TLS connection with ECH. 2. A way to provide ECH Config Lists as bytes directly to ssl clients. 3. A callback that gets called whenever ECH negotiation fails and the server offers a "Retry Config". 4. A method to ensure encrypted DNS is used so all metadata is encrypted. OpenSSL does not implement the necessary APIs yet. Stephen Farrell's development OpenSSL fork [9] implements ECH and has been used in Apache HTTPD, nginx, lighttpd, HAProxy, and curl implementations. The TLS WG maintain a page with information about other implementations: https://github.com/tlswg/draft-ietf-tls-esni/wiki/Implementations [1] https://www.ietf.org/archive/id/draft-ietf-tls-esni-13.html [2] https://github.com/openssl/openssl/issues/7482 [3] https://bugs.chromium.org/p/boringssl/issues/detail?id=275 [5] https://bugzilla.mozilla.org/show_bug.cgi?id=1725938 [6] https://bugs.chromium.org/p/chromium/issues/detail?id=1091403 [6] https://github.com/google/conscrypt/issues/730 [7] https://github.com/square/okhttp/issues/6539 [8] https://www.ietf.org/archive/id/draft-ietf-dnsop-svcb-https-07.html [9] https://github.com/sftcd/openssl -- assignee: christian.heimes components: SSL messages: 404732 nosy: christian.heimes, eighthave priority: normal severity: normal status: open title: Support TLS Encrypted ClientHello (ECH) type: enhancement ___ Python tracker <https://bugs.python.org/issue45567> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43547] support ZIP files with zeroed out fields (e.g. for reproducible builds)
Hans-Christoph Steiner added the comment: > - For full reproducible builds you may have to write files to zipfiles in a > well-defined order. That already works fine now, we've been doing that with Python for years. But that leaves it up to the implemented to do. I suppose zipfile could provide a method to sort entries, but that's out of scope for this issue IMHO. -- ___ Python tracker <https://bugs.python.org/issue43547> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43547] support ZIP files with zeroed out fields (e.g. for reproducible builds)
Hans-Christoph Steiner added the comment: I just found another specific example in _open_to_write(). 0 is a valid value for zinfo.external_attr. But this code always forces 0 to something else: if not zinfo.external_attr: zinfo.external_attr = 0o600 << 16 # permissions: ?rw--- -- ___ Python tracker <https://bugs.python.org/issue43547> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43547] support ZIP files with zeroed out fields (e.g. for reproducible builds)
New submission from Hans-Christoph Steiner : It is now standard for Java JARs and Android APKs (both ZIP files) to zero out lots of the fields in the ZIP header. For example: * each file entry has the date set to zero * the create_system is always set to zero on all platforms zipfile currently cannot create such ZIPs because of two small restrictions that it introduced: * must use a tuple of 6 values to set the date * forced create_system value based on sys.platform == 'win32' * maybe other fields? I lump these together because it might make sense to handle this with a single argument, something like zero_header=True. The use case is for working with ZIP, JAR, APK, AAR files for reproducible builds. The whole build system for F-Droid is built in Python. We need to be able to copy the JAR/APK signatures in order to reproduce signed builds using only the source code and the signature files themselves. Right now, that's not possible because building a ZIP with Python's zipfile cannot zero out the ZIP header like other tools can, including Java. -- components: IO, Library (Lib) messages: 389040 nosy: eighthave priority: normal severity: normal status: open title: support ZIP files with zeroed out fields (e.g. for reproducible builds) versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue43547> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27875] Syslogs /usr/sbin/foo as /foo instead of as foo
Change by Hans Deragon : -- versions: +Python 3.8 ___ Python tracker <https://bugs.python.org/issue27875> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33327] Add a method to move messages to IMAPlib
Hans-Peter Jansen added the comment:
If I'm not mistaken, this is applied to the openSUSE TW version of Python.
For some reason, this seems to not play well with .uid('move',...)
on a cyrus imap server (v2.4.19). Is that to be expected?
```
2020-07-03 18:04:05 INFO: [imap_reorg] move b'10399' from 2017-01-01
06:30:35+02:00 to INBOX.2017
Traceback (most recent call last):
File "./imap_reorg.py", line 431, in
sys.exit(main())
File "./imap_reorg.py", line 425, in main
return process()
File "./imap_reorg.py", line 133, in trace_and_call
result = func(*args, **kwargs)
File "./imap_reorg.py", line 358, in process
ret |= reorg.run_expr(expr)
File "./imap_reorg.py", line 345, in run_expr
return method(*args)
File "./imap_reorg.py", line 328, in yearly
ret = self.imap.uid('move', uid, dest)
File "/usr/lib64/python3.8/imaplib.py", line 881, in uid
typ, dat = self._simple_command(name, command, *args)
File "/usr/lib64/python3.8/imaplib.py", line 1205, in _simple_command
return self._command_complete(name, self._command(name, *args))
File "/usr/lib64/python3.8/imaplib.py", line 1030, in _command_complete
raise self.error('%s command error: %s %s' % (name, typ, data))
imaplib.error: UID command error: BAD [b'Unrecognized UID subcommand']
```
--
nosy: +frispete
___
Python tracker
<https://bugs.python.org/issue33327>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37999] No longer use implicit convertion to int with loss
Change by Hans Petter Jansson : -- nosy: +hpj nosy_count: 2.0 -> 3.0 pull_requests: +20417 pull_request: https://github.com/python/cpython/pull/17536 ___ Python tracker <https://bugs.python.org/issue37999> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39306] Lib/configparser.py - RawConfigParser.set does not pass non-truthy values through to Interpolation.before_set
New submission from Hans Strijker : Method ```configparser.RawConfigParser.set()``` has optional parameter *value* with default value ```None``` resulting in the behavior that actually trying to set a config parameter to ```None``` will not be propagated to ```Interpolation.before_set()```. In fact, since it uses ```if value:``` and not ```if value is None:``` none of the non-truthy values will be passed through. Suggested commit [8e008be](https://github.com/HStry/cpython/commit/8e008bea0cf6bd3c698b333fd39a383e124fe026) using already established ```_UNSET``` singleton, but that appears to break compatibility elsewhere. -- components: Library (Lib) messages: 359820 nosy: Strijker, taleinat priority: normal pull_requests: 17362 severity: normal status: open title: Lib/configparser.py - RawConfigParser.set does not pass non-truthy values through to Interpolation.before_set type: behavior versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue39306> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37720] Crash of python3.7 with virt-manager
Hans Peter added the comment: How to fix that? See above post. -- ___ Python tracker <https://bugs.python.org/issue37720> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37720] Crash of python3.7 with virt-manager
Hans Peter added the comment: PYTHONDEVMODE=1 virt-manager --no-fork Fatal Python error: Segmentation fault Current thread 0x7f835bafd740 (most recent call first): File "", line 219 in _call_with_frames_removed File "", line 1043 in create_module File "", line 583 in module_from_spec File "", line 670 in _load_unlocked File "", line 967 in _find_and_load_unlocked File "", line 983 in _find_and_load File "/usr/lib/python3.7/lzma.py", line 27 in File "", line 219 in _call_with_frames_removed File "", line 728 in exec_module File "", line 677 in _load_unlocked File "", line 967 in _find_and_load_unlocked File "", line 983 in _find_and_load File "/usr/lib/python3.7/shutil.py", line 29 in File "", line 219 in _call_with_frames_removed File "", line 728 in exec_module File "", line 677 in _load_unlocked File "", line 967 in _find_and_load_unlocked File "", line 983 in _find_and_load File "/usr/lib/python3.7/tempfile.py", line 44 in File "", line 219 in _call_with_frames_removed File "", line 728 in exec_module File "", line 677 in _load_unlocked File "", line 967 in _find_and_load_unlocked File "", line 983 in _find_and_load File "/usr/lib/python3/dist-packages/apport/report.py", line 12 in File "", line 219 in _call_with_frames_removed File "", line 728 in exec_module File "", line 677 in _load_unlocked File "", line 967 in _find_and_load_unlocked File "", line 983 in _find_and_load File "/usr/lib/python3/dist-packages/apport/__init__.py", line 5 in File "", line 219 in _call_with_frames_removed File "", line 728 in exec_module File "", line 677 in _load_unlocked File "", line 967 in _find_and_load_unlocked File "", line 983 in _find_and_load File "", line 219 in _call_with_frames_removed File "", line 953 in _find_and_load_unlocked File "", line 983 in _find_and_load File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 63 in apport_excepthook Segmentation fault -- ___ Python tracker <https://bugs.python.org/issue37720> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37720] Crash of python3.7 with virt-manager
New submission from Hans Peter : Hi! A few days ago, I upgraded to UbuntuMate 19.04. I can't run 'virt-manager' because of this: # virt-manager Output: Segmentation fault kernel: [ 2003.888116] virt-manager[16014]: segfault at 32d0 ip 32d0 sp 7ffeb09ac658 error 14 in python3.7[40+21000] kernel: [ 2003.888124] Code: Bad RIP value. Look at "crash_and_packages.log" I added to attachment. Please help, to fix that. Thanks! -- components: Installation files: crash_and_packages.log messages: 348730 nosy: mother-earth40 priority: normal severity: normal status: open title: Crash of python3.7 with virt-manager type: crash versions: Python 3.7 Added file: https://bugs.python.org/file48517/crash_and_packages.log ___ Python tracker <https://bugs.python.org/issue37720> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36828] Cannot install et-xmlfile
Hans Ginzel added the comment: Thank you for suggestion. I agree. Where can I find the repo for et-xmlfile or the appropriate tracker respectively, please? I have found https://github.com/dimensions11/et_xmlfile. Is it the _master_ repo? -- ___ Python tracker <https://bugs.python.org/issue36828> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36828] Cannot install et-xmlfile
Change by Hans Ginzel : -- title: Cannot install et_xmlfile -> Cannot install et-xmlfile ___ Python tracker <https://bugs.python.org/issue36828> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36828] Cannot install et_xmlfile
New submission from Hans Ginzel : See attached log file, please. How to reproduce: rem Download https://www.python.org/ftp/python/3.7.3/python-3.7.3-embed-amd64.zip mkdir C:\Tools\Python unzip python-3.7.3-embed-amd64.zip -d C:\Tools\Python\3.7.3 path C:\Tools\Python\3.7.3;C:\Tools\Python\3.7.3\Scripts;%PATH% echo Remove comment in "#import site" on the last line, please. notepad C:\Tools\Python\3.7.3\python37._pth rem Download https://bootstrap.pypa.io/get-pip.py python get-pip.py Collecting pip Downloading https://files.pythonhosted.org/packages/5c/e0/be401c003291b56efc55aeba6a80ab790d3d4cece2778288d65323009420/pip-19.1.1-py2.py3-none-any.whl (1.4MB) || 1.4MB 656kB/s Collecting setuptools Downloading https://files.pythonhosted.org/packages/ec/51/f45cea425fd5cb0b0380f5b0f048ebc1da5b417e48d304838c02d6288a1e/setuptools-41.0.1-py2.py3-none-any.whl (575kB) || 583kB 3.2MB/s Collecting wheel Downloading https://files.pythonhosted.org/packages/96/ba/a4702cbb6a3a485239fbe9525443446203f00771af9ac000fa3ef2788201/wheel-0.33.1-py2.py3-none-any.whl Installing collected packages: pip, setuptools, wheel Successfully installed pip-19.1.1 setuptools-41.0.1 wheel-0.33.1 C:\Users\Hans\Downloads>pip install --log et-xmlfile.log et-xmlfile Collecting et-xmlfile Downloading https://files.pythonhosted.org/packages/22/28/a99c42aea746e18382ad9fb36f64c1c1f04216f41797f2f0fa567da11388/et_xmlfile-1.0.1.tar.gz ERROR: Command "python setup.py egg_info" failed with error code 1 in C:\Users\Hans\AppData\Local\Temp\pip-install-qws9gvg0\et-xmlfile\ -- components: Installation files: et-xmlfile.log messages: 341707 nosy: hginzel priority: normal severity: normal status: open title: Cannot install et_xmlfile type: crash versions: Python 3.7 Added file: https://bugs.python.org/file48308/et-xmlfile.log ___ Python tracker <https://bugs.python.org/issue36828> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35329] Documentation - capitalization issue
New submission from Hans Strijker : In the documentation I noticed "from Package import specific_submodule". I recon package should be all lowercase in accordance with pep8. (https://www.python.org/dev/peps/pep-0008/#package-and-module-names) It may be far from the most important bug ever, but it's my first bug reported, and gotta start small... :-) https://docs.python.org/3/tutorial/modules.html#importing-from-a-package 6.4.1. Importing * From a Package ... Remember, there is nothing wrong with using from Package import specific_submodule! In fact, this is the recommended notation unless the importing module needs to use submodules with the same name from different packages. -- assignee: docs@python components: Documentation messages: 330537 nosy: Strijker, docs@python priority: normal severity: normal status: open title: Documentation - capitalization issue versions: Python 3.7 ___ Python tracker <https://bugs.python.org/issue35329> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Re: PyCA cryptography 2.3 released
Hi Paul, you have a version mismatch in subject and text. Cheers, Pete On Mittwoch, 18. Juli 2018 05:19:27 Paul Kehrer wrote: > PyCA cryptography 2.2.2 has been released to PyPI. cryptography includes > both high level recipes and low level interfaces to common cryptographic > algorithms such as symmetric ciphers, message digests, and key derivation > functions. We support Python 2.7, Python 3.4+, and PyPy. > > Changelog (https://cryptography.io/en/latest/changelog/#v2-3): > > * SECURITY ISSUE: finalize_with_tag() allowed tag truncation by default > which can allow tag forgery in some cases. The method now enforces the > min_tag_length provided to the GCM constructor. > * Added support for Python 3.7. > * Added extract_timestamp() to get the authenticated timestamp of a Fernet > token. > * Support for Python 2.7.x without hmac.compare_digest has been deprecated. > We will require Python 2.7.7 or higher (or 2.7.6 on Ubuntu) in the next > cryptography release. > * Fixed multiple issues preventing cryptography from compiling against > LibreSSL 2.7.x. > * Added get_revoked_certificate_by_serial_number for quick serial number > searches in CRLs. > * The RelativeDistinguishedName class now preserves the order of > attributes. Duplicate attributes now raise an error instead of silently > discarding duplicates. > * aes_key_unwrap() and aes_key_unwrap_with_padding() now raise > InvalidUnwrap if the wrapped key is an invalid length, instead of > ValueError. > > -Paul Kehrer (reaperhulk) -- https://mail.python.org/mailman/listinfo/python-list
python not working
it shows thepythonw.exe system error: The program can't start because api-ms-win-crt-runtime-|1-1-0.dll is missing from your computer. try reinstalling the program to fix this problem. I reinstall it many times try to repair it is not working -- https://mail.python.org/mailman/listinfo/python-list
Wichtig für Maximus
https://www.heise.de/newsticker/meldung/WannaCry-Microsoft-liefert-Sicherheits-Patches-fuer-veraltete-Windows-Versionen-3713417.html -- https://mail.python.org/mailman/listinfo/python-list
Re: keyrings.cryptfile released on github
On Dienstag, 14. März 2017 00:33:34 Hans-Peter Jansen wrote: > > I plan to add authenticated service and username support via associated data > as well (that protects against tampering with these values). Done. > Cheers, > Pete -- https://mail.python.org/mailman/listinfo/python-list
Re: keyrings.cryptfile released on github
On Freitag, 10. März 2017 13:31:41 Paul Rubin wrote: > Hans-Peter Jansen <h...@urpla.net> writes: > > [1] http://web.cs.ucdavis.edu/~rogaway/ocb/license.htm > > Oh that's interesting, he's expanded the free licenses. Still though, > while OCB is very clever and it was important as the first satisfactory > AEAD mode, I don't think it's that important these days. GCM is > standardized, does similar things, and while it's potentially slower, > some CPUs even have hardware support for it now. If you library doesn't > support GCM (I haven't checked yet) then it probably should. It does support GCM now, it's even the new default(!). I've added support for all AEAD modes, that PyCryptodome supports, and supplied a small encryption mode conversion tool as well. I plan to add authenticated service and username support via associated data as well (that protects against tampering with these values). It might be a good idea to record and secure the number of entries as well. Cheers, Pete -- https://mail.python.org/mailman/listinfo/python-list
Re: keyrings.cryptfile released on github
On Donnerstag, 9. März 2017 23:09:09 ng0 wrote: > Hans-Peter Jansen transcribed 3.8K bytes: > > Hi, > > > > since the PyCrypto ML is dead, I'm looking for advise/feedback from some > > cryptography aware people. > > > > I've released a keyring companion package today: > > https://github.com/frispete/keyrings.cryptfile > > > > Its primary purpose is a decent encrypted file backend for python > > keyrings. > > As such, it uses manually parameterized argon2 hashes as KDF, and AES in > > OCB mode as stream cipher (well, it just encrypts the password for a > > given service/user name). Granted, the advantages of OCB are not /that/ > > crucial here :wink:, but apart from technical factors, the exclusion of > > military uses > I was looking for some proprietary EULA or something else locked down > license and instead just saw the Expat license. Don't get you here. Yes, the package contains just a MIT license file for now. > I assume that you are > aware that no anti-mil clause exists in co-existence with free software > licenses > and your sentence was just written in an > odd way? First, bear with me, as I'm not a native speaker. Sometimes, my expressions tend to confuse people. I'm sorry about that. Second, I'm not a lawyer. As long as I use OCB in OSS and does not sue somebody else about it, License 1 is granted, if I read the OCB license options correctly [1]. Anyway, I will add the OCB license to the package and add a note in the initial password retrieval dialog about it.. In this projects state, I would like to discuss the technical issues first. As said, I sympathize with the idea of the license, since I personally believe, that military conflicts doesn't solve any problems, but many todays problems originate from them. [1] http://web.cs.ucdavis.edu/~rogaway/ocb/license.htm > > by its license is rather *attractive* from my POV(!). But I'm open for > > discussions of course. Cheers, Pete -- https://mail.python.org/mailman/listinfo/python-list
keyrings.cryptfile released on github
Hi,
since the PyCrypto ML is dead, I'm looking for advise/feedback from some
cryptography aware people.
I've released a keyring companion package today:
https://github.com/frispete/keyrings.cryptfile
Its primary purpose is a decent encrypted file backend for python keyrings.
As such, it uses manually parameterized argon2 hashes as KDF, and AES in OCB
mode as stream cipher (well, it just encrypts the password for a given
service/user name). Granted, the advantages of OCB are not /that/ crucial
here :wink:, but apart from technical factors, the exclusion of military uses
by its license is rather *attractive* from my POV(!). But I'm open for
discussions of course.
Still interested? Here we go:
To get you started, I expect you to have a python3 environment and git
available. You might want to provide the packages argon2-cffi, keyring,
pycryptodome and their dependencies (most notably SecretStorage and
cryptography, or use a local venv, but that will depend on a compiler and some
development packages.
Example session, create an encrypted keyring:
$ git clone https://github.com/frispete/keyrings.cryptfile
$ cd keyrings.cryptfile
$ pyvenv env
$ . env/bin/activate
(env) $ pip install -e .
[...] # should succeed, some development packages might be missing otherwise
(env) $ python3
Python 3.4.5 (default, Jul 03 2016, 12:57:15) [GCC] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from keyrings.cryptfile.cryptfile import CryptFileKeyring
>>> kr = CryptFileKeyring()
>>> kr.set_password("service", "user", "secret")
Please set a password for your new keyring:
Please confirm the password:
>>> ^d
Second session, retrieve the stored secret from the keyring:
(env) $ python3
Python 3.4.5 (default, Jul 03 2016, 12:57:15) [GCC] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from keyrings.cryptfile.cryptfile import CryptFileKeyring
>>> kr = CryptFileKeyring()
>>> kr.get_password("service", "user")
Please enter password for encrypted keyring:
'secret'
>>> ^d
Note, that the KDF might delay the {set,get}_password() operations for a
few seconds (~1 sec. on a capable system).
The resulting file is located here (by default) and might look similar to:
(env) $ cat ~/.local/share/python_keyring/cryptfile_pass.cfg
[keyring_2Dsetting]
password_20reference =
eyJub25jZSI6ICJQdVdWVUIwUHNYbEFqYUUxZ2l2RlxuIiwgIm1hYyI6ICIvVTFIVDBWTnRheTFl
TjA5TVlHb0dRPT1cbiIsICJzYWx0IjogIklMdDNBU1hMUENrbWZ2NzFudmtBSUE9PVxuIiwgImRh
dGEiOiAidW1EQkNvQ2dRUTk5WEVaNkZ4NWt3NXRkSUZDOHFIUE5ZOHhWXG4ifQ==
scheme = PyCryptodome [Argon2] AES OCB
version = 1.0
[service]
user =
eyJub25jZSI6ICI5SUU3UGp2eDU2SXNQdHlLUGRtaFxuIiwgIm1hYyI6ICJKcFR1NXMxaDd0UGlW
OW9XL3d5cFdBPT1cbiIsICJzYWx0IjogIlpBeEhJdXlqYnRuTkgzb3BMNTFvdkE9PVxuIiwgImRh
dGEiOiAiT2I3Z1JJbXR5aVJLXG4ifQ==
The values can be decoded like this:
(env) $ python3
>>> import base64
>>> base64.decodebytes(b"""
... eyJub25jZSI6ICI5SUU3UGp2eDU2SXNQdHlLUGRtaFxuIiwgIm1hYyI6ICJKcFR1NXMxaDd0UGlW
... OW9XL3d5cFdBPT1cbiIsICJzYWx0IjogIlpBeEhJdXlqYnRuTkgzb3BMNTFvdkE9PVxuIiwgImRh
... dGEiOiAiT2I3Z1JJbXR5aVJLXG4ifQ==""")
b'{"nonce": "9IE7Pjvx56IsPtyKPdmh\\n", "mac": "JpTu5s1h7tPiV9oW/wypWA==\\n",
"salt": "ZAxHIuyjbtnNH3opL51ovA==\\n", "data": "Ob7gRImtyiRK\\n"}'
The items should be self explanatory. In theory, it should be considerable
hard to get back to the plain values of data without knowing the password.
Any cryptography experts attending?
What do you think? The class hierarchy is inherited from keyrings.alt, and
not exactly easy to follow, but the interesting parts are all in cryptfile,
which is quite brief.
I would be glad to hear something from you about my handling of cryptography.
Is it ready for the public in that form or should I better locked away? :wink:
TIA,
Pete
--
https://mail.python.org/mailman/listinfo/python-list
Re: ANN: psutil 5.1.0 with hardware sensors released
On Mittwoch, 1. Februar 2017 21:54:06 Giampaolo Rodola' wrote: > Hello all, > I'm glad to announce the release of psutil 5.1.1: ^ Guess, you meant to say 5.1.0 here, or probably your time machine broke ;) Cheers, Pete -- https://mail.python.org/mailman/listinfo/python-list
Re: Referencing section name by interpolation in ConfigParser
On Mittwoch, 25. Januar 2017 10:01:56 Peter Otten wrote:
> Hans-Peter Jansen wrote:
> > I would like to use a interpolated section name, e.g.:
> >
> > [Section]
> > secref: %{section}s/whatever
> >
> > should result in:
> >>>> config['Section']['secref']
> >
> > 'Section/whatever'
> >
> > Any idea anybody, how to archive this with minimum fuzz?
>
> If you can live with the existing "basic interpolation", i. e. %(...)s, not
> %{...}s:
Yes, of course.. Sorry for the typo.
> $ cat config.ini
> [Foo]
> secref: %(section)s/whatever
> [Bar]
> secref: %(section)s/whatever
> $ cat demo.py
> import configparser
>
>
> class Interpolation(configparser.BasicInterpolation):
> def before_get(self, parser, section, option, value, defaults):
> defaults = defaults.copy()
> defaults["section"] = section
> return super().before_get(parser, section, option, value, defaults)
>
>
> p = configparser.ConfigParser(interpolation=Interpolation())
> p.read("config.ini")
> for section in "Foo", "Bar":
> print(section, "-->", p[section]["secref"])
> $ python3 demo.py
> Foo --> Foo/whatever
> Bar --> Bar/whatever
> $
Brilliant as usual, thank you, Peter.
This is exactly, what I was after.
Have a nice weekend,
Pete
--
https://mail.python.org/mailman/listinfo/python-list
Referencing section name by interpolation in ConfigParser
Hi,
I would like to use a interpolated section name, e.g.:
[Section]
secref: %{section}s/whatever
should result in:
>>> config['Section']['secref']
'Section/whatever'
Any idea anybody, how to archive this with minimum fuzz?
Thanks,
Pete
--
https://mail.python.org/mailman/listinfo/python-list
Re: Using sudo with pip3?
On Samstag, 7. Januar 2017 19:07:55 Clint Moyer wrote: > I would lightly advise against, assuming both Pip and your package > manager are trying to accomplish nearly the same thing. Stick with > updating through the repo. > > If you find that the version your OS provides is out-of-date compared > to what's on PyPi or Github, then you might want to remove from your > OS and re-install through Pip, for those discrete cases. That's the > platform agnostic route. Or take the ninja way, I do: Build all packages, you want (updated) for your distribution: https://build.opensuse.org/project/monitor/home:frispete:python https://build.opensuse.org/project/monitor/home:frispete:python3 The red labels are reminders... SCR, Pete -- https://mail.python.org/mailman/listinfo/python-list
Re: Choosing a Python IDE. what is your Pythonish recommendation? I
On Montag, 2. Januar 2017 03:38:53 Antonio Caminero Garcia wrote: > Hello, I am having a hard time deciding what IDE or IDE-like code editor > should I use. This can be overwhelming. > > So far, I have used Vim, Sublime, Atom, Eclipse with PyDev, Pycharm, > IntelliJ with Python plugin. Well, since nobody mentioned it, yet: eric is doing quite nice here. With on the fly error checking, jedi and qscintilla calltips and autocompletion, git integration (using a plugin), graphical debugger, it's grown to a capable IDE over the years. Given, it's a fully open source, PyQt based project, it also shows the powers of Python3 and PyQt. Pete -- https://mail.python.org/mailman/listinfo/python-list
Re: Choosing a Python IDE. what is your Pythonish recommendation? I do not know what to choose.
On Montag, 2. Januar 2017 03:38:53 Antonio Caminero Garcia wrote: > Hello, I am having a hard time deciding what IDE or IDE-like code editor > should I use. This can be overwhelming. > > So far, I have used Vim, Sublime, Atom, Eclipse with PyDev, Pycharm, > IntelliJ with Python plugin. Well, since nobody mentioned it, yet: eric is doing quite nice here. With on the fly error checking, jedi and qscintilla calltips and autocompletion, git integration (using a plugin), graphical debugger, it's grown to a capable IDE over the years. Given, it's a fully open source, PyQt based project, it also shows the powers of Python3 and PyQt. Pete -- https://mail.python.org/mailman/listinfo/python-list
Re: ctypes, memory mapped files and context manager
Dear Eryk,
thanks for chiming in.
On Donnerstag, 29. Dezember 2016 21:27:56 eryk sun wrote:
> On Thu, Dec 29, 2016 at 12:18 PM, Hans-Peter Jansen <h...@urpla.net> wrote:
> >> >>> import weakref, ctypes
> >> >>> T = ctypes.c_ubyte * 3
> >> >>> t = T()
> >> >>> bytes(t) == b"\0" * 3
> >>
> >> True
> >>
> >> >>> bytes(weakref.proxy(t)) == b"\0" * 3
> >>
> >> Traceback (most recent call last):
> >> File "", line 1, in
> >>
> >> AttributeError: 'c_ubyte_Array_3' object has no attribute '__bytes__'
> >>
> >> That looks like a leaky abstraction. While I found a workaround
> >>
> >> >>> bytes(weakref.proxy(t)[:]) == b"\0" * 3
> >>
> >> True
> >
> > I found a couple of other rough corners already, when working with the
> > ctypes module. Obviously, this module is lacking some love.
>
> That's not the fault of ctypes. There's no requirement for objects
> that implement the buffer protocol to also implement __bytes__. You'd
> have the same problem if you tried to proxy a memoryview.
>
> However, using a proxy seems particularly wHaorthless for ctypes. Type
> checking is integral to the design of ctypes, and a weakproxy won't
Did you follow the discussion?
I'm trying to make context manager work with ctypes.from_buffer on mmapped
files:
import ctypes
import mmap
import weakref
NOPROB=False
#NOPROB=True
from contextlib import contextmanager
class T(ctypes.Structure):
_fields_ = [("foo", ctypes.c_uint32)]
@contextmanager
def map_struct(m, n, struct, offset = 0):
m.resize(n * mmap.PAGESIZE)
inst = struct.from_buffer(m, offset)
yield inst
SIZE = mmap.PAGESIZE * 2
f = open("tmp.dat", "w+b")
f.write(b"\0" * SIZE)
f.seek(0)
m = mmap.mmap(f.fileno(), mmap.PAGESIZE)
with map_struct(m, 1, T) as a:
a.foo = 1
if NOPROB:
del a
with map_struct(m, 2, T) as b:
b.foo = 2
if NOPROB:
del b
offset = ctypes.sizeof(T)
rest = m.size() - offset
overhang = ctypes.c_ubyte * rest
with map_struct(m, 2, overhang, offset) as c:
assert(bytes(c) == bytes(rest))
if NOPROB:
del c
Without these dreaded del statements, this code doesn't work:
$ python3 mmap_test2.py
Traceback (most recent call last):
File "mmap_test2.py", line 30, in
with map_struct(m, 2, T) as b:
File "/usr/lib64/python3.4/contextlib.py", line 59, in __enter__
return next(self.gen)
File "mmap_test2.py", line 16, in map_struct
m.resize(n * mmap.PAGESIZE)
BufferError: mmap can't resize with extant buffers exported.
It will work, if you define NOPROB=True.
The weakref approach was an attempt to make this work.
Do you have an idea how to create the context manager in a way, that obsoletes
these ugly dels? Something like ctypes.from_buffer_release() that is able to
actively release the mapping is needed here, AFAICS.
This code works with Python2 due to the mmap module not checking for any
existing mappings which may lead to segfaults, if the mmap is resized.
Thanks,
Pete
--
https://mail.python.org/mailman/listinfo/python-list
Re: ctypes, memory mapped files and context manager
On Donnerstag, 29. Dezember 2016 09:33:59 Peter Otten wrote: > Hans-Peter Jansen wrote: > > On Mittwoch, 28. Dezember 2016 16:53:53 Hans-Peter Jansen wrote: > > The minimal example is > > >>> import weakref, ctypes > >>> T = ctypes.c_ubyte * 3 > >>> t = T() > >>> bytes(t) == b"\0" * 3 > > True > > >>> bytes(weakref.proxy(t)) == b"\0" * 3 > > Traceback (most recent call last): > File "", line 1, in > AttributeError: 'c_ubyte_Array_3' object has no attribute '__bytes__' > > That looks like a leaky abstraction. While I found a workaround > > >>> bytes(weakref.proxy(t)[:]) == b"\0" * 3 > > True I found a couple of other rough corners already, when working with the ctypes module. Obviously, this module is lacking some love. > to me your whole approach is beginning to look very questionable. You know, > > "If the implementation is hard to explain, it's a bad idea." > > What do you gain from using the mmap/ctypes combo instead of regular file > operations and the struct module? Your sample code seems to touch every > single byte of the file once so that there are little to no gains from > caching. And then your offset is basically a file position managed manually > instead of implicitly with read, write, and seek. Of course, the real code is a bit more complex... The code presented here is for demonstration purposes only. I'm not allowed to reveal the projects' code, but I can state, that using this combination allows for crawling through huge files (5-25GB) in unbelievable performance (without any further optimization), and updating parts of it. By delegating the whole I/O management to the kernel, one can observe, that python runs at full speed managing the data just by reference and assignment operations, all (mostly) in place. The resource usage is impressively low at the same time. Since the code is meant to be executed with many instances in parallel on a single machine, this is an important design criteria. While I would love to get rid of these dreaded and unpythonic del statements, I can accept them for now, until a better approach is found. Will dig through the ctypes module again, when I find time. Thanks again for taking your valuable time, Peter. Much appreciated. I wish you a Happy New Year! Cheers, Pete -- https://mail.python.org/mailman/listinfo/python-list
Re: ctypes, memory mapped files and context manager
On Mittwoch, 28. Dezember 2016 16:53:53 Hans-Peter Jansen wrote:
> On Mittwoch, 28. Dezember 2016 15:17:22 Hans-Peter Jansen wrote:
> > On Mittwoch, 28. Dezember 2016 13:48:48 Peter Otten wrote:
> > > Hans-Peter Jansen wrote:
> > > > Dear Peter,
> > > >
> > > > thanks for taking valuable time to look into my issue.
> > >
> > > You're welcome!
> > >
> > > @contextmanager
> > > def map_struct(m, n):
> > > m.resize(n * mmap.PAGESIZE)
> > > keep_me = T.from_buffer(m)
> > > yield weakref.proxy(keep_me)
> >
> > Hooray, that did the trick. Great solution, thank you very much!
>
> Sorry for bothering you again, Peter, but after applying it to the real
> project, that fails consistently similar to:
$ python3 mmap_test_weakref.py
Traceback (most recent call last):
File "mmap_test_weakref.py", line 32, in
assert(bytes(c) == bytes(rest))
AttributeError: 'c_ubyte_Array_8188' object has no attribute '__bytes__'
$ cat mmap_test_weakref.py
import ctypes
import mmap
import weakref
from contextlib import contextmanager
class T(ctypes.Structure):
_fields_ = [("foo", ctypes.c_uint32)]
@contextmanager
def map_struct(m, n, struct, offset = 0):
m.resize(n * mmap.PAGESIZE)
inst = struct.from_buffer(m, offset)
yield weakref.proxy(inst)
SIZE = mmap.PAGESIZE
f = open("tmp.dat", "w+b")
f.write(b"\0" * SIZE)
f.seek(0)
m = mmap.mmap(f.fileno(), mmap.PAGESIZE)
with map_struct(m, 1, T) as a:
a.foo = 1
with map_struct(m, 2, T) as b:
b.foo = 2
offset = ctypes.sizeof(T)
rest = m.size() - offset
overhang = ctypes.c_ubyte * rest
with map_struct(m, 2, overhang, offset) as c:
assert(bytes(c) == bytes(rest))
With weakref and mmap.resize() disabled, this acts as expected.
BTW: mmapped files need the first page initialized, the rest is done in the
kernel (filled with zeros on resize).
Pete
--
https://mail.python.org/mailman/listinfo/python-list
Re: ctypes, memory mapped files and context manager
On Mittwoch, 28. Dezember 2016 21:58:38 Peter Otten wrote:
> Hans-Peter Jansen wrote:
> > On Mittwoch, 28. Dezember 2016 13:48:48 Peter Otten wrote:
> >> Hans-Peter Jansen wrote:
> > It leaves the question on why is Python2 acting as one would expect
> > related to context managers, and Python3 needs this weakref juggling.
> > Maybe something, that's worth to be placed in python-dev. What do you
> > think?
>
> Well, given
>
> $ cat mmap_resize.py
> import ctypes
> import mmap
>
> class T(ctypes.Structure):
> _fields_ = [("foo", ctypes.c_uint32)]
>
> SIZE = 2 * mmap.PAGESIZE
> f = open("tmp.dat", "w+b")
> f.write(b"\0" * SIZE)
> f.seek(0)
>
> m = mmap.mmap(f.fileno(), SIZE)
>
> a = T.from_buffer(m, mmap.PAGESIZE)
> m.resize(mmap.PAGESIZE)
> a.foo = 42
> $ python3 mmap_resize.py
> Traceback (most recent call last):
> File "mmap_resize.py", line 15, in
> m.resize(mmap.PAGESIZE)
> BufferError: mmap can't resize with extant buffers exported.
> $ python2.7 mmap_resize.py
> Segmentation fault
>
> do you really prefer the Python 2 behaviour?
Hmm, so Python2 behavior was "working by chance", or better "working without
any safety net" which isn't the Real McCoy either in that area.
Now, if only the weakref version would behave with the ctypes objects...
Pete
--
https://mail.python.org/mailman/listinfo/python-list
Re: ctypes, memory mapped files and context manager
On Mittwoch, 28. Dezember 2016 15:17:22 Hans-Peter Jansen wrote:
> On Mittwoch, 28. Dezember 2016 13:48:48 Peter Otten wrote:
> > Hans-Peter Jansen wrote:
> > > Dear Peter,
> > >
> > > thanks for taking valuable time to look into my issue.
> >
> > You're welcome!
> >
> > > It might be related to my distinct silliness, but the problem persists
> > > with your code as well.
> >
> > Unfortunately I posted the broken toy example rather than the fixed one.
> > Here's the latter. Basically you have to keep a reference in the context
> > manager (whether you implement it as a class or a generator doesn't
> > matter)
> > without giving another reference away to client code:
> >
> > $ cat mmap_after.py
> > import ctypes
> > import mmap
> > import weakref
> >
> > from contextlib import contextmanager
> >
> > class T(ctypes.Structure):
> > _fields = [("foo", ctypes.c_uint32)]
> >
> > @contextmanager
> >
> > def map_struct(m, n):
> > m.resize(n * mmap.PAGESIZE)
> > keep_me = T.from_buffer(m)
> > yield weakref.proxy(keep_me)
>
> Hooray, that did the trick. Great solution, thank you very much!
Sorry for bothering you again, Peter, but after applying it to the real
project, that fails consistently similar to:
# first run, check explicitly disabled
$> python3 ctypes_mmap_ctx.py
DEBUG: starting offset: 0x10
DEBUG: add_data: header
DEBUG: add_data: 5168
DEBUG: add_data: header
DEBUG: add_data: 5168
DEBUG: add_data: header
DEBUG: add_data: 5168
DEBUG: add_data: header
DEBUG: add_data: 5168
DEBUG: add_data: header
DEBUG: add_data: 5168
DEBUG: add_data: header
DEBUG: add_data: 5168
DEBUG: add_data: header
DEBUG: add_data: 5168
DEBUG: add_data: header
DEBUG: add_data: 5168
DEBUG: final offset: 0xa1d0
$> python3 ctypes_mmap_ctx.py
Traceback (most recent call last):
File "ctypes_mmap_ctx.py", line 163, in
mf = MapFile(mapfile)
File "ctypes_mmap_ctx.py", line 109, in __init__
if bytes(blk) != bytes(rest):
AttributeError: 'c_ubyte_Array_3632' object has no attribute '__bytes__'
The new issue appears in some consistency checking code. The mapfile is
checked for zeroed out overhang on that line. The weakref proxy isn't behaving
well at that point. Any idea, what could be going wrong with it?
Updated the gist to demonstrate the issue after switching to weakref.proxy().
(has grown even more code in that process, sorry).
It looks like a minor issue, but the code quality is really degraded from
these ugly del statements.
Pete
--
https://mail.python.org/mailman/listinfo/python-list
Re: ctypes, memory mapped files and context manager
On Mittwoch, 28. Dezember 2016 13:48:48 Peter Otten wrote:
> Hans-Peter Jansen wrote:
> > Dear Peter,
> >
> > thanks for taking valuable time to look into my issue.
>
> You're welcome!
>
> > It might be related to my distinct silliness, but the problem persists
> > with your code as well.
>
> Unfortunately I posted the broken toy example rather than the fixed one.
> Here's the latter. Basically you have to keep a reference in the context
> manager (whether you implement it as a class or a generator doesn't matter)
> without giving another reference away to client code:
>
> $ cat mmap_after.py
> import ctypes
> import mmap
> import weakref
>
> from contextlib import contextmanager
>
> class T(ctypes.Structure):
> _fields = [("foo", ctypes.c_uint32)]
>
>
> @contextmanager
> def map_struct(m, n):
> m.resize(n * mmap.PAGESIZE)
> keep_me = T.from_buffer(m)
> yield weakref.proxy(keep_me)
Hooray, that did the trick. Great solution, thank you very much!
If you don't mind, I will mention you and your solution at the various places,
I placed this issue over the last weeks.
You made my day, Peter!
It leaves the question on why is Python2 acting as one would expect related to
context managers, and Python3 needs this weakref juggling. Maybe something,
that's worth to be placed in python-dev. What do you think?
Cheers,
Pete
--
https://mail.python.org/mailman/listinfo/python-list
Re: ctypes, memory mapped files and context manager
Dear Peter,
thanks for taking valuable time to look into my issue.
It might be related to my distinct silliness, but the problem persists with
your code as well. Further comments inlined.
On Dienstag, 27. Dezember 2016 21:39:51 Peter Otten wrote:
> Hans-Peter Jansen wrote:
> >
> > def __enter__(self):
> > # resize the mmap (and backing file), if structure exceeds mmap
> > # size mmap size must be aligned to mmap.PAGESIZE
> > cssize = ctypes.sizeof(self._cstruct)
> >
> > if self._offset + cssize > self._mm.size():
> > newsize = align(self._offset + cssize, mmap.PAGESIZE)
> > self._mm.resize(newsize)
> >
> > self._csinst = self._cstruct.from_buffer(self._mm, self._offset)
> > return self._csinst
>
> Here you give away a reference to the ctypes.BigEndianStructure. That means
> you no longer control the lifetime of self._csinst which in turn holds a
> reference to the underlying mmap or whatever it's called.
Here's the code, based on contextmanager:
@contextmanager
def cstructmap(cstruct, mm, offset = 0):
# resize the mmap (and backing file), if structure exceeds mmap size
# mmap size must be aligned to mmap.PAGESIZE
cssize = ctypes.sizeof(cstruct)
if offset + cssize > mm.size():
newsize = align(offset + cssize, mmap.PAGESIZE)
mm.resize(newsize)
yield cstruct.from_buffer(mm, offset)
While much more concise, I don't understand, how it should make a difference
relative to the "with" variable lifetime, when used.
> There might be a way to release the mmap reference while the wrapper
> structure is still alive, but the cleaner way is probably to not give it
> away in the first place, and create a proxy instead with
>
> return weakref.proxy(self._csinst)
This fails, as it doesn't keep the reference long enough.
> > def __exit__(self, exc_type, exc_value, exc_traceback):
> > # free all references into mmap
> > del self._csinst
>
> The line above is redundant. It removes the attribute from the instance
> __dict__ and implicitly decreases its refcount. It does not actually
> physically delete the referenced object. If you remove the del statement the
> line below will still decrease the refcount.
>
> Make sure you understand this to avoid littering your code with cargo cult
> del-s ;)
Yes, I was aware of this. It was a testing relic, that survived somehow.
Sorry. Yes, I usually try to avoid cargo cultry in my code. ;)
> > The issue: when creating a mapping via context manager, we assign a local
> > variable (with ..), that keep existing in the local context, even when the
> > manager context was left. This keeps a reference on the ctypes mapped area
> > alive, even if we try everything to destroy it in __exit__. We have to del
> > the with var manually.
> >
> > Now, I want to get rid of the ugly any error prone del statements.
> >
> > What is needed, is a ctypes operation, that removes the mapping actively,
> > and that could be added to the __exit__ part of the context manager.
Revised code (including your test code):
https://gist.github.com/frispete/97c27e24a0aae1bcaf1375e2e463d239
> > The script creates a memory mapped file in the current directory named
> > "mapfile". When started without arguments, it copies itself into this
> > file, until 10 * mmap.PAGESIZE growth is reached (or it errored out
> > before..).
> >
> > IF you change NOPROB to True, it will actively destruct the context
> > manager vars, and should work as advertized.
> >
> > Any ideas are much appreciated.
>
> You might put some more effort into composing example scripts. Something
> like the script below would have saved me some time...
I'm very sorry about this.
> import ctypes
> import mmap
>
> from contextlib import contextmanager
>
> class T(ctypes.Structure):
> _fields = [("foo", ctypes.c_uint32)]
>
>
> @contextmanager
> def map_struct(m, n):
> m.resize(n * mmap.PAGESIZE)
> yield T.from_buffer(m)
>
> SIZE = mmap.PAGESIZE * 2
> f = open("tmp.dat", "w+b")
> f.write(b"\0" * SIZE)
> f.seek(0)
> m = mmap.mmap(f.fileno(), mmap.PAGESIZE)
>
> with map_struct(m, 1) as a:
> a.foo = 1
> with map_struct(m, 2) as b:
> b.foo = 2
Unfortunately, your code behaves exactly like mine:
$> python3 mmap_test.py
Traceback (most recent call last):
File "mmap_test.py", line 23, in
with map_struct(m, 2) as b:
File "/usr/lib64/python3.4/contextlib.py", line 59, in __enter__
return next(self.gen)
File "mmap_test.py", line 12, in map_struct
m.resize(n * mmap.PAGESIZE)
BufferError: mmap can't resize with extant buffers exported.
BTW, Python2 makes a difference in this respect, but my project is Python3
based. Have you tested this with Python3? It would be interesting to explore
the reasons of this difference, which is, ähem, pretty surprising.
Thanks,
Pete
--
https://mail.python.org/mailman/listinfo/python-list
ctypes, memory mapped files and context manager
Hi, I'm using $subjects combination successfully in a project for creating/iterating over huge binary files (> 5GB) with impressive performance, while resource usage keeps pretty low, all with plain Python3 code. Nice! Environment: (Python 3.4.5, Linux 4.8.14, openSUSE/x86_64, NFS4 and XFS filesystems) The idea is: map a ctypes structure onto the file at a certain offset, act on the structure, and release the mapping. The latter is necessary for keeping the mmap file properly resizable and closable (due to the nature of mmaps and Python's posix implementation thereof). Hence, a context manager serves us well (in theory). Here's some code excerpt: class cstructmap: def __init__(self, cstruct, mm, offset = 0): self._cstruct = cstruct self._mm = mm self._offset = offset self._csinst = None def __enter__(self): # resize the mmap (and backing file), if structure exceeds mmap size # mmap size must be aligned to mmap.PAGESIZE cssize = ctypes.sizeof(self._cstruct) if self._offset + cssize > self._mm.size(): newsize = align(self._offset + cssize, mmap.PAGESIZE) self._mm.resize(newsize) self._csinst = self._cstruct.from_buffer(self._mm, self._offset) return self._csinst def __exit__(self, exc_type, exc_value, exc_traceback): # free all references into mmap del self._csinst self._csinst = None def work(): with cstructmap(ItemHeader, self._mm, self._offset) as ih: ih.identifier = ItemHeader.Identifier ih.length = ItemHeaderSize + datasize blktype = ctypes.c_char * datasize with cstructmap(blktype, self._mm, self._offset) as blk: blk.raw = data In practice, this results in: Traceback (most recent call last): File "ctypes_mmap_ctx.py", line 146, in mf.add_data(data) File "ctypes_mmap_ctx.py", line 113, in add_data with cstructmap(blktype, self._mm, self._offset) as blk: File "ctypes_mmap_ctx.py", line 42, in __enter__ self._mm.resize(newsize) BufferError: mmap can't resize with extant buffers exported. The issue: when creating a mapping via context manager, we assign a local variable (with ..), that keep existing in the local context, even when the manager context was left. This keeps a reference on the ctypes mapped area alive, even if we try everything to destroy it in __exit__. We have to del the with var manually. Now, I want to get rid of the ugly any error prone del statements. What is needed, is a ctypes operation, that removes the mapping actively, and that could be added to the __exit__ part of the context manager. Full working code example: https://gist.github.com/frispete/97c27e24a0aae1bcaf1375e2e463d239 The script creates a memory mapped file in the current directory named "mapfile". When started without arguments, it copies itself into this file, until 10 * mmap.PAGESIZE growth is reached (or it errored out before..). IF you change NOPROB to True, it will actively destruct the context manager vars, and should work as advertized. Any ideas are much appreciated. Thanks in advance, Pete -- https://mail.python.org/mailman/listinfo/python-list
[issue9253] argparse: optional subparsers
Changes by Hans-Peter Jansen <h...@urpla.net>: -- nosy: +frispete ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue9253> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28970] ctypes.from_buffer counterpart to actively remove the mapping
New submission from Hans-Peter Jansen: In an attempt of using ctypes.from_buffer() to map a structure to a memory mapped file, it is important to destroy the mapping after use, because the mmap won't be resizable or freeable correctly until then. The best approach, I was able to came up with is using a context manager, but calling the dtor of the mapping in __exit__ is not enough, which results to code like this: with StructMap(ctypes_struct, mm, offest) as smap: smap.xxx = 'blabla' del smap # necessary, but ugly Without the del, the "with" variable still exist in the local context, hence the mapping still exist, until it is explicitly destroyed. I hope, that ctypes is able to (or can be made to) actively remove the mapping in the __exit__ part of the context manager. I've put some code on stackoverflow to play with this: http://stackoverflow.com/questions/41077696/python-ctypes-from-buffer-mapping-with-context-manager-into-memory-mapped-file The problem seems to exist with the linux mmap implementation only. -- components: ctypes messages: 283188 nosy: frispete priority: normal severity: normal status: open title: ctypes.from_buffer counterpart to actively remove the mapping type: behavior versions: Python 3.7 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28970> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
distutils_ui 0.1.1 released
For those of you, who like PyQt{4,5} as much as I do, as well as for those who
don't like it that much, because of the poor integration with setuptools
et.al., here's another piece of software to bridge the gap:
A distutils build extension for PyQt{4,5} applications
that makes handling the PyQt tool chain easier than ever:
https://pypi.python.org/pypi/distutils_ui
Ahem, well, it wasn't that easy before. Most of us were using dreaded
Makefiles or other such crutches to generate translation files, .py modules of
forms, and resource modules. Scratch the crutches, here's what you're looking
for.
Feedback welcome.
Enjoy,
Pete
--
https://mail.python.org/mailman/listinfo/python-announce-list
Support the Python Software Foundation:
http://www.python.org/psf/donations/
distutils_ui 0.1.1 released
For those of you, who like PyQt{4,5} as much as I do, as well as for those who
don't like it that much, because of the poor integration with setuptools
et.al., here's another piece of software to bridge the gap:
A distutils build extension for PyQt{4,5} applications
that makes handling the PyQt tool chain easier than ever:
https://pypi.python.org/pypi/distutils_ui
Ahem, well, it wasn't that easy before. Most of us were using dreaded
Makefiles or other such crutches to generate translation files, .py modules of
forms, and resource modules. Scratch the crutches, here's what you're looking
for.
Feedback welcome.
Enjoy,
Pete
--
https://mail.python.org/mailman/listinfo/python-list
[issue27568] "HTTPoxy", use of HTTP_PROXY flag supplied by attacker in CGI scripts
Hans-Peter Jansen added the comment: > (In msg271688, I pondered if I need to backport a behavior change from > issue26804 which will allow lower cased proxies, but then, I decided against > it as it will introduce unnecessary changes to this security fix releases). Hmm, Senthil, while I understand, that you want to avoid unnecessary changes, doesn't this result in non deterministic behaviour of proxy handling without my patch? + header. If you need to use an HTTP proxy in a CGI environment, either use + ``ProxyHandler`` explicitly, or make sure the variable name is in + lowercase (or at least the ``_proxy`` suffix). Without 26804, this fix works by chance only for 3.3 and 3.4, since it depends on os.environ dictionary order, which is non deterministic by definition. 26804 resolves this by making sure, a lower case _proxy var has a higher priority over the other variants. -- nosy: +frispete ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27568> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27513] email.utils.getaddresses raises exception from erroneous message get_all input
Hans-Peter Jansen added the comment: message.get cannot decode the header correctly, and returns a Header instance instead, which makes email.utils.getaddresses stumble upon... A much better behavior for getaddresses in this case would be returning the perfectly valid address, and ignoring the invalid dtext part. -- Added file: http://bugs.python.org/file43715/decode_from_header.py ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27513> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27513] email.utils.getaddresses raises exception from erroneous message get_all input
New submission from Hans-Peter Jansen:
An unfortunate combination of get_all and getaddresses results in a Traceback:
Traceback (most recent call last):
File "misc/decode_from_header.py", line 17, in
print('From: %s' % email.utils.getaddresses(val))
File "/usr/lib64/python3.4/email/utils.py", line 112, in getaddresses
all = COMMASPACE.join(fieldvalues)
TypeError: sequence item 0: expected str instance, Header found
Here's the relevant part of it:
Content-type: text/html;charset=iso-8859-1
From: Itaú Uniclass. <comunicado.com...@atendimento.gotdns.ch>
Obviously, the From header is iso-8859-1 encoded as well, and violates RFC 2822
as such. But making it crash in the usual combination of
val = msg.get('from')
email.utils.getaddresses([val])
isn't the real McCoy either..
--
components: email
files: iso-8859-1-encoded-from-header.mail
messages: 270399
nosy: barry, frispete, r.david.murray
priority: normal
severity: normal
status: open
title: email.utils.getaddresses raises exception from erroneous message get_all
input
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file43714/iso-8859-1-encoded-from-header.mail
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue27513>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27256] email header indentation destroyed
Hans-Peter Jansen added the comment: Sorry guys for not providing this earlier. It turned out, that the sub optimal behaviour is related to a unfortunate policy choice: email.policy.SMTP. -- Added file: http://bugs.python.org/file43417/email_flatten.py ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27256> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27257] get_addresses results in traceback with an addrspec with an empty local part.
Hans-Peter Jansen added the comment: Sorry guys for not providing this earlier. It turned out, that the sub optimal behaviour is related to a unfortunate policy choice: email.policy.SMTP. -- Added file: http://bugs.python.org/file43416/email_flatten.py ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27257> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27258] Exception in BytesGenerator.flatten
Hans-Peter Jansen added the comment: Sorry guys for not providing this earlier. It turned out, that the sub optimal behaviour is related to a unfortunate policy choice: email.policy.SMTP. -- Added file: http://bugs.python.org/file43415/email_flatten.py ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27258> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26804] Prioritize lowercase proxy variables in urllib.request
Hans-Peter Jansen added the comment: In a couple of systems, I have to stick with 3.4. Is there a chance to have this patch in 3.4 as well, if a new release 3.4 is made? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26804> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27257] get_addresses results in traceback with a valid? header
Hans-Peter Jansen added the comment: Dear Stephen, thanks for your care. I'm glad, that you're able to reproduce it. This header is added from the email provider (the biggest here in Germany), so yes, it deserves an entry in the defects list, but must not traceback, of course. It is not expected to provide a sensible way of interoperability otherwise. The unlisted-recipients part is a bit more useful in this respect. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27257> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27258] Exception in BytesGenerator.flatten
New submission from Hans-Peter Jansen: Attached mail, parsed with email.message_from_binary_file results in: Traceback (most recent call last): File "./mail_filter.py", line 616, in ret = main.run() File "./mail_filter.py", line 605, in run self.process(fp) File "./mail_filter.py", line 589, in process self.save_message(msg, self._fname + '.out') File "./mail_filter.py", line 103, in save_message ofd.write(msg.as_bytes()) File "/usr/lib64/python3.4/email/message.py", line 179, in as_bytes g.flatten(self, unixfrom=unixfrom) File "/usr/lib64/python3.4/email/generator.py", line 115, in flatten self._write(msg) File "/usr/lib64/python3.4/email/generator.py", line 195, in _write self._write_headers(msg) File "/usr/lib64/python3.4/email/generator.py", line 422, in _write_headers self._fp.write(self.policy.fold_binary(h, v)) File "/usr/lib64/python3.4/email/policy.py", line 190, in fold_binary folded = self._fold(name, value, refold_binary=self.cte_type=='7bit') File "/usr/lib64/python3.4/email/policy.py", line 204, in _fold return self.header_factory(name, ''.join(lines)).fold(policy=self) File "/usr/lib64/python3.4/email/headerregistry.py", line 255, in fold return header.fold(policy=policy) File "/usr/lib64/python3.4/email/_header_value_parser.py", line 300, in fold self._fold(folded) File "/usr/lib64/python3.4/email/_header_value_parser.py", line 1228, in _fold rest._fold(folded) File "/usr/lib64/python3.4/email/_header_value_parser.py", line 338, in _fold if folded.append_if_fits(part, tstr): File "/usr/lib64/python3.4/email/_header_value_parser.py", line 149, in append_if_fits token._fold(self) File "/usr/lib64/python3.4/email/_header_value_parser.py", line 324, in _fold for part in self.parts: File "/usr/lib64/python3.4/email/_header_value_parser.py", line 254, in parts if token.startswith_fws(): File "/usr/lib64/python3.4/email/_header_value_parser.py", line 267, in startswith_fws return self[0].startswith_fws() File "/usr/lib64/python3.4/email/_header_value_parser.py", line 267, in startswith_fws return self[0].startswith_fws() File "/usr/lib64/python3.4/email/_header_value_parser.py", line 267, in startswith_fws return self[0].startswith_fws() File "/usr/lib64/python3.4/email/_header_value_parser.py", line 267, in startswith_fws return self[0].startswith_fws() File "/usr/lib64/python3.4/email/_header_value_parser.py", line 267, in startswith_fws return self[0].startswith_fws() IndexError: list index out of range when flattened with BytesGenerator. -- components: email files: flatten-exception.mail messages: 267736 nosy: barry, frispete, r.david.murray priority: normal severity: normal status: open title: Exception in BytesGenerator.flatten versions: Python 3.4, Python 3.5, Python 3.6 Added file: http://bugs.python.org/file43288/flatten-exception.mail ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27258> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27257] get_addresses results in traceback with a valid? header
New submission from Hans-Peter Jansen:
In the course of replacing an old Python 2.7 email filter tool with a rewritten
Python3 version, I stumbled across a ugly case, where such an header:
To: unlisted-recipients: ;,
""@pop.kundenserver.de (no To-header on input)
results in a Traceback (most recent call last):
File "./mail_filter.py", line 606, in
ret = main.run()
File "./mail_filter.py", line 595, in run
self.process(fp)
File "./mail_filter.py", line 520, in process
config.recipients = self.get_addresses('to', msg)
File "./mail_filter.py", line 103, in get_addresses
vals = msg.get_all(field, [])
File "/usr/lib64/python3.4/email/message.py", line 511, in get_all
values.append(self.policy.header_fetch_parse(k, v))
File "/usr/lib64/python3.4/email/policy.py", line 145, in header_fetch_parse
return self.header_factory(name, ''.join(value.splitlines()))
File "/usr/lib64/python3.4/email/headerregistry.py", line 584, in __call__
return self[name](name, value)
File "/usr/lib64/python3.4/email/headerregistry.py", line 195, in __new__
cls.parse(value, kwds)
File "/usr/lib64/python3.4/email/headerregistry.py", line 342, in parse
for mb in addr.all_mailboxes]))
File "/usr/lib64/python3.4/email/headerregistry.py", line 342, in
for mb in addr.all_mailboxes]))
File "/usr/lib64/python3.4/email/_header_value_parser.py", line 837, in
local_part
return self[0].local_part
File "/usr/lib64/python3.4/email/_header_value_parser.py", line 889, in
local_part
return self[0].local_part
File "/usr/lib64/python3.4/email/_header_value_parser.py", line 984, in
local_part
tok[0].token_type == 'cfws'):
IndexError: list index out of range
I'm not completely sure, if the Top header, as added from my email provider, is
perfectly valid, but none of the other parts of my mail infrastructure neither
complained, nor behave strange with such headers.
This happens with 3.4.4, but also with the email module from current hg for
testing.
--
components: email
files: lkml-exception.mail
messages: 267733
nosy: barry, frispete, r.david.murray
priority: normal
severity: normal
status: open
title: get_addresses results in traceback with a valid? header
versions: Python 3.4, Python 3.5, Python 3.6
Added file: http://bugs.python.org/file43287/lkml-exception.mail
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue27257>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27256] header indentation destroyed
Changes by Hans-Peter Jansen <h...@urpla.net>: Added file: http://bugs.python.org/file43286/mf.9__mi0bf.out ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27256> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27256] header indentation destroyed
New submission from Hans-Peter Jansen: In the course of replacing an old Python 2.7 email filter tool with a rewritten Python3 version, I stumbled across a ugly case, where such an header: X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtCTDJQUjAyTUI1MTQ7MjM6bEtRRlNaUHQvVTk5WCttdktlOUVrUGQvVFBH?= =?utf-8?B?cDFJemVUeXFzOGNzYnZOYWlwMDZpR0YzbXZyY09WaTBKM2pkeUl4S1VDMkxw?= =?utf-8?B?eVRkNWthRW9waUhJTzczTWd5WDZOQ3hMNU1haGFvQTVzVTdRZmxJUnZlblpW?= ... is regenerated as: X-Microsoft-Exchange-Diagnostics: 1;BL2PR02MB514;23:lKQFSZPt/U99X+mvKe9EkPd/TPG p1IzeTyqs8csbvNaip06iGF3mvrcOVi0J3jdyIxKUC2Lp yTd5kaEopiHIO73MgyX6NCxL5MahaoA5sU7QflIRvenZV which is plain wrong of course. I'm using email.message_from_binary_file for parsing and BytesGenerator.flatten for regeneration. Since those are LKML public mails, I'm attaching both versions. I'm using 3.4.4, but also the email module from current hg for testing. -- components: email files: utf8-header-failure.mail messages: 267732 nosy: barry, frispete, r.david.murray priority: normal severity: normal status: open title: header indentation destroyed versions: Python 3.4, Python 3.5, Python 3.6 Added file: http://bugs.python.org/file43285/utf8-header-failure.mail ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27256> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10808] ssl unwrap fails with Error 0
Hans-Peter Jansen added the comment:
Poor old bug.
Just being bitten from it today, while trying to package pyftpdlib on the
openSUSE build service, which creates a clean reproducible build environment
for all packages, and testing fails.
Part of the game: openssl 1.0.1k, Python 2.7.8
https://build.opensuse.org/package/show/home:frispete:python/python-pyftpdlib
It happens reproducible for i586 only, but not for x86_64, with all the same
versions, and not with a local (much faster) build host.
So it is smells like a timing problem.
[ 97s] ERROR: test_nlst (test_functional_ssl.TestFtpListingCmdsTLSMixin)
[ 97s] --
[ 97s] Traceback (most recent call last):
[ 97s] File
"/home/abuild/rpmbuild/BUILD/pyftpdlib-1.5.1/pyftpdlib/test/test_functional_ssl.py",
line 139, in test_nlst
[ 97s] super(TestFtpListingCmdsTLSMixin, self).test_nlst()
[ 97s] File
"/home/abuild/.local/lib/python2.7/site-packages/pyftpdlib-1.5.1-py2.7.egg/pyftpdlib/test/test_functional.py",
line 1187, in test_nlst
[ 97s] self._test_listing_cmds('nlst')
[ 97s] File
"/home/abuild/.local/lib/python2.7/site-packages/pyftpdlib-1.5.1-py2.7.egg/pyftpdlib/test/test_functional.py",
line 1180, in _test_listing_cmds
[ 97s] self.client.retrlines('%s %s' % (cmd, tempdir), x.append)
[ 97s] File "/usr/lib/python2.7/ftplib.py", line 735, in retrlines
[ 97s] conn.unwrap()
[ 97s] File "/usr/lib/python2.7/ssl.py", line 289, in unwrap
[ 97s] s = self._sslobj.shutdown()
[ 97s] error: [Errno 0] Error
--
nosy: +frispete
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue10808>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26804] Prioritize lowercase proxy variables in urllib.request
Hans-Peter Jansen added the comment: v7: - reorder test code in order to improve edibility -- Added file: http://bugs.python.org/file42586/python-urllib-prefer-lowercase-proxies-v7.diff ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26804> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26804] Prioritize lowercase proxy variables in urllib.request
Hans-Peter Jansen added the comment: > In Python 2, it looks like the proxy_bypass_etc() functions are defined > in urllib and imported into urllib2, so it makes sense to include the > tests in test_urllib rather than test_urllib2. The tests are in test_urllib. test_urllib2 is testing proxy behaviour on a higher level, so I think, they're in the correct module, aren't they? -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26804> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26804] Prioritize lowercase proxy variables in urllib.request
Hans-Peter Jansen added the comment: * blatant error fixed * one test case added -- Added file: http://bugs.python.org/file42582/python-urllib-prefer-lowercase-proxies-v6.diff ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26804> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26831] ConfigParser parsing failures with default_section and ExtendedInterpolation options
New submission from Hans-Peter Jansen:
ConfigParser fails in interesting ways, when using default_section and
ExtendedInterpolation options. Running the attached script results in:
ConfigParser() with expected result:
global: [('loglevel', 'WARNING'), ('logfile', '-')]
section1: [('key_a', 'value'), ('key_b', 'morevalue')]
section2: [('key_c', 'othervalue'), ('key_d', 'differentvalue')]
ConfigParser(default_section='global') mangles section separation:
section1: [('loglevel', 'WARNING'), ('logfile', '-'), ('key_a', 'value'),
('key_b', 'morevalue')]
section2: [('loglevel', 'WARNING'), ('logfile', '-'), ('key_c', 'othervalue'),
('key_d', 'differentvalue')]
ConfigParser(interpolation=ExtendedInterpolation) fails with strange error:
Traceback (most recent call last):
File "configparser-test.py", line 36, in
print_sections(cp)
File "configparser-test.py", line 21, in print_sections
cp.read_string(__doc__)
File "/usr/lib64/python3.4/configparser.py", line 696, in read_string
self.read_file(sfile, source)
File "/usr/lib64/python3.4/configparser.py", line 691, in read_file
self._read(f, source)
File "/usr/lib64/python3.4/configparser.py", line 1089, in _read
self._join_multiline_values()
File "/usr/lib64/python3.4/configparser.py", line 1101, in
_join_multiline_values
name, val)
TypeError: before_read() missing 1 required positional argument: 'value'
while it is expected to behave identical.
--
components: Library (Lib)
files: configparser-test.py
messages: 264031
nosy: frispete
priority: normal
severity: normal
status: open
title: ConfigParser parsing failures with default_section and
ExtendedInterpolation options
versions: Python 3.4
Added file: http://bugs.python.org/file42573/configparser-test.py
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue26831>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26804] Prioritize lowercase proxy variables in urllib.request
Hans-Peter Jansen added the comment: v5: don't require the proxies argument in proxy_bypass_environment() -- Added file: http://bugs.python.org/file42565/python-urllib-prefer-lowercase-proxies-v5.diff ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26804> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26804] Prioritize lowercase proxy variables in urllib.request
Hans-Peter Jansen added the comment: Here's the finalized version of this patch, including unit tests. -- Added file: http://bugs.python.org/file42552/python-urllib-prefer-lowercase-proxies-v4.diff ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26804> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26804] Prioritize lowercase proxy variables in urllib.request
Hans-Peter Jansen added the comment: Here we go: v3 fixes following issues: * prefer lowercase proxy environment settings, if multiple (disagreeing) settings are specified with differing case schemes (e.g. HTTP_PROXY vs. http_proxy) * an empty proxy variable value resets the related setting correctly * apply this logic to no_proxy, too * document this behaviour * fix misleading docstrings related to proxy_bypass -- Added file: http://bugs.python.org/file42548/python-urllib-prefer-lowercase-proxies-v3.diff ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26804> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26804] Prioritize lowercase proxy variables in urllib.request
Hans-Peter Jansen added the comment: Hi Martin, hi Senthil, please find a new patch attached, that incorporates your suggestions. * added a comment to get_proxies doc in urllib.rst * documented and fixed the mixed case scheme * added a note to proxy_bypass_environment, that behaves slightly different in this respect Yes, mixed case situations are not handled in proxy_bypass_environment, just lowercase and uppercase, while lowercase is preferred correctly. I think, that the mixed case situation is pathologic enough and deserves to be ignored here. BTW, while looking at the code, I noticed, that most docstrings of the callers of proxy_bypass_environment are wrong: they say, that the proxies dict is returned, but they return the value of proxy_bypass_environment(), not get_proxies(). A follow up patch could do this in order to clean up this mess: since there's always a call to get_proxies preceding the call to proxy_bypass_environment, we could add a second argument to the latter, passing in the proxy dict, that is thrown away at the moment. Since that carries a 'no' key already, if it exists, using it here would fix this ambiguity. While at it, fix up the affected docstrings. What do you think about the attached patch and the last paragraph? -- Added file: http://bugs.python.org/file42544/python-urllib-prefer-lowercase-proxies.diff ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26804> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26804] Prioritize lowercase proxy variables in urllib.request
Hans-Peter Jansen added the comment: Hi Martin, hi Senthil, thanks for the valuable comments. Will incorporate your suggestions later today. Yes, Martin, it's a bug, and should be fixed for 2.7 and 3.5 as well, but I was unsure, if I get some feedback at all... Hence, this is a very nice experience for me. I'm out for jogging now, Pete -- versions: +Python 2.7, Python 3.5 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26804> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26804] Prioritize lowercase proxy variables in urllib.request
Changes by Hans-Peter Jansen <h...@urpla.net>: -- versions: +Python 3.6 -Python 3.5 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26804> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26804] Prioritize lowercase proxy variables in urllib.request
New submission from Hans-Peter Jansen: During programming a function, that replaces a wget call, I noticed, that something is wrong with urllibs proxy handling. I usually use the scheme "http_proxy= wget -N -nd URL" when I need to bypass the proxy. Hence I was pretty confused, that this doesn't work with python(3). Creating an empty ProxyHandler isn't the real Mc Coy either. Diving into the issue, I found getproxies_environment, but couldn't make much sense out of its behavior, up until I noticed, that my OS (openSUSE ) creates both variants of environment variables behind the scenes: uppercase and lowercase. Consequence: python3 needs the scheme "http_proxy= HTTP_PROXY= python3 ..." Since I, like everyone else, prefer gentle tones over the loud, and want to spare this surprise for others in the future, I propose the attached patch. Process environment variables in two passes, first uppercase, then lowercase, allowing an empty lowercase value to overrule any uppercase value. Please consider applying this. -- components: Extension Modules files: prioritize_lowercase_proxy_vars_in_urllib_request.diff keywords: patch messages: 263720 nosy: frispete priority: normal severity: normal status: open title: Prioritize lowercase proxy variables in urllib.request type: behavior versions: Python 3.5 Added file: http://bugs.python.org/file42516/prioritize_lowercase_proxy_vars_in_urllib_request.diff ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26804> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26796] BaseEventLoop.run_in_executor shouldn't specify max_workers for default Executor
Hans Lawrenz added the comment: New patch attached. Includes comments and a note in the documentation. The documentation note is inside a versionchanged:: 3.5 block. Should this be more specific about the version it changed in? It could be confusing for someone using a version of 3.5 that doesn't have the change. Contributor form is signed. Thanks! -- Added file: http://bugs.python.org/file42509/run_in_executor_max_workers_with_docs.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26796> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26796] BaseEventLoop.run_in_executor shouldn't specify max_workers for default Executor
Hans Lawrenz added the comment: Thanks, that makes sense. I've attached a patch with a version check. -- Added file: http://bugs.python.org/file42507/run_in_executor_max_workers_vcheck.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26796> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26796] BaseEventLoop.run_in_executor shouldn't specify max_workers for default Executor
New submission from Hans Lawrenz: In issue 21527 <http://bugs.python.org/issue21527> the concurrent.futures.ThreadPoolExecutor was changed to have a default value for max_workers. When asyncio.base_events.BaseEventLoop.run_in_executor creates a default ThreadPoolExecutor it specifies a value of 5 for max_workers, presumably because at the time it was written ThreadPoolExecutor didn't have a default for max_workers. This is confusing because on reading the documentation for ThreadPoolExecutor one might assume that the default specified there is what will be used if a default executor isn't supplied via BaseEventLoop.set_default_executor. I propose that BaseEventLoop.run_in_executor be changed to not supply a default for max_workers. If this isn't acceptable, a note ought to be put in the run_in_executor documentation. -- components: asyncio files: run_in_executor_max_workers.patch keywords: patch messages: 263669 nosy: Hans Lawrenz, gvanrossum, haypo, yselivanov priority: normal severity: normal status: open title: BaseEventLoop.run_in_executor shouldn't specify max_workers for default Executor type: behavior versions: Python 3.5 Added file: http://bugs.python.org/file42506/run_in_executor_max_workers.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26796> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26133] asyncio: ugly error related to signal handlers at exit if the loop is not closed explicitly
Hans Lellelid added the comment: FWIW, I am experiencing the issue described here with Python 3.5.1. -- nosy: +Hans Lellelid ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26133> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25717] tempfile.TemporaryFile fails when dir option set to directory residing on host OS mount
Hans Lawrenz added the comment: Emanuel, sorry, I missed the request for sys.version earlier. The tempfile.py I attached earlier is from the python 3.5 pulled from a ppa. I wouldn't be surprised if it has some patches applied by debian/ubuntu. To be clear though the problem also presents itself with a python 3.5 I compiled. The Vagrantfile in the github repository shows exactly how I compiled it. Here it is from the version from the ppa: Python 3.5.0 (default, Sep 17 2015, 00:00:00) [GCC 4.8.4] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.version '3.5.0 (default, Sep 17 2015, 00:00:00) \n[GCC 4.8.4]' >>> sys.implementation namespace(_multiarch='x86_64-linux-gnu', cache_tag='cpython-35', hexversion=50659568, name='cpython', version=sys.version_info(major=3, minor=5, micro=0, releaselevel='final', serial=0)) Here are the same for the version I compiled: vagrant@vagrant-ubuntu-trusty-64:~$ python3.5 Python 3.5.0 (default, Nov 24 2015, 19:50:42) [GCC 4.8.4] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.version '3.5.0 (default, Nov 24 2015, 19:50:42) \n[GCC 4.8.4]' >>> sys.implementation namespace(cache_tag='cpython-35', hexversion=50659568, name='cpython', version=sys.version_info(major=3, minor=5, micro=0, releaselevel='final', serial=0)) -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25717> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25717] tempfile.TemporaryFile fails when dir option set to directory residing on host OS mount
Hans Lawrenz added the comment:
Serhiy and Emanuel, I'll paste below the surrounding code and attach the exact
tempfile.py. It is the version distributed with the 3.5.0 release. If you take
a look at the github repo I linked in the first comment you can also try it out
for yourself if you've got vagrant and virtualbox installed. I was able to
recreate the error with the provisioning compiling python and with it
installing from a ppa. You can see the details in the Vagrantfile. I also had a
coworker test in a nearly identical environment and he had the same result.
(fd, name) = _mkstemp_inner(dir, prefix, suffix, flags, output_type)
try:
_os.unlink(name)
return _io.open(fd, mode, buffering=buffering,
newline=newline, encoding=encoding)
except:
_os.close(fd)
raise
One final note. I decided to try this out with a windows host and the only
clarity this test adds is that the problem doesn't show up there but it fails
for its own reasons on all the python versions I tried. See below for reference:
Last login: Tue Nov 24 17:16:12 2015 from 10.0.2.2
vagrant@vagrant-ubuntu-trusty-64:~$ cat /vagrant/foo.py
import tempfile
with tempfile.TemporaryFile(dir="/vagrant") as tf:
tf.write("testing testing testing\n".encode('utf-8'))
print("Path 2: worked")
vagrant@vagrant-ubuntu-trusty-64:~$ python /vagrant/foo.py
Traceback (most recent call last):
File "/vagrant/foo.py", line 4, in
with tempfile.TemporaryFile(dir="/vagrant") as tf:
File "/usr/lib/python2.7/tempfile.py", line 495, in TemporaryFile
_os.unlink(name)
OSError: [Errno 26] Text file busy: '/vagrant/tmpvx7Mie'
vagrant@vagrant-ubuntu-trusty-64:~$ python2.7 /vagrant/foo.py
Traceback (most recent call last):
File "/vagrant/foo.py", line 4, in
with tempfile.TemporaryFile(dir="/vagrant") as tf:
File "/usr/lib/python2.7/tempfile.py", line 495, in TemporaryFile
_os.unlink(name)
OSError: [Errno 26] Text file busy: '/vagrant/tmpNXQ6Cf'
vagrant@vagrant-ubuntu-trusty-64:~$ python3.4 /vagrant/foo.py
Traceback (most recent call last):
File "/vagrant/foo.py", line 4, in
with tempfile.TemporaryFile(dir="/vagrant") as tf:
File "/usr/lib/python3.4/tempfile.py", line 638, in TemporaryFile
_os.unlink(name)
OSError: [Errno 26] Text file busy: '/vagrant/tmpfwhj7ip4'
vagrant@vagrant-ubuntu-trusty-64:~$ python3.5 /vagrant/foo.py
Traceback (most recent call last):
File "/vagrant/foo.py", line 4, in
with tempfile.TemporaryFile(dir="/vagrant") as tf:
File "/usr/lib/python3.5/tempfile.py", line 751, in TemporaryFile
_os.unlink(name)
OSError: [Errno 26] Text file busy: '/vagrant/tmp02s19r_a'
vagrant@vagrant-ubuntu-trusty-64:~$ python2.7 --version
Python 2.7.6
vagrant@vagrant-ubuntu-trusty-64:~$ python3.4 --version
Python 3.4.3
vagrant@vagrant-ubuntu-trusty-64:~$ python3.5 --version
Python 3.5.0
--
Added file: http://bugs.python.org/file41153/tempfile.py
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25717>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25717] tempfile.TemporaryFile fails when dir option set to directory residing on host OS mount
Hans Lawrenz added the comment: The file system causing the problem is of type vboxsf which is the Virtualbox shared folder file system type. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25717> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25717] tempfile.TemporaryFile fails when dir option set to directory residing on host OS mount
Hans Lawrenz added the comment: Unfortunately changing the tempfile call isn't an easy option for me. The situation in which I'm encountering the error is running tox on our project which is mounted in the /vagrant directory in the VM (this is standard for vagrant). Tox makes its own temp directory--presumably for test isolation--in the .tox directory in the project root. The actual call to tempfile.TemporaryFile() which is triggering the error is in a third party library that is called during a test run. I was able to work around the issue by changing the tox workdir setting to outside the mount. However, I'd like to mention that developing in vagrant in this fashion isn't uncommon and once 3.5 gains adoption I would guess this issue may affect a good number of people. I'm happy to take a stab at writing a patch but looking at the code it's somewhat out of my comfort zone and I worry I'd make a hash of it. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25717> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25717] tempfile.TemporaryFile fails when dir option set to directory residing on host OS mount
New submission from Hans Lawrenz:
Inside a virtualbox vm, calling tempfile.TemporaryFile(dir=foo) where foo is a
directory which resides on a volume mounted from the host OS, a
FileNotFoundError exception is thrown.
In the following code sample, the second block will print "Path 2: ERROR" on
Python 3.5 but not on 3.4 or 2.7 (assuming the vagrant environment provided
below):
import tempfile
try:
with tempfile.TemporaryFile() as tf:
tf.write("testing testing testing\n".encode('utf-8'))
print("Path 1: worked")
except FileNotFoundError as e:
print("Path 1: ERROR")
try:
with tempfile.TemporaryFile(dir="/vagrant") as tf:
tf.write("testing testing testing\n".encode('utf-8'))
print("Path 2: worked")
except FileNotFoundError as e:
print("Path 2: ERROR")
A runnable test case can be found here:
https://github.com/hlawrenz/py35tempfiletest
--
components: Library (Lib)
messages: 255246
nosy: Hans Lawrenz
priority: normal
severity: normal
status: open
title: tempfile.TemporaryFile fails when dir option set to directory residing
on host OS mount
type: behavior
versions: Python 3.5
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25717>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25717] tempfile.TemporaryFile fails when dir option set to directory residing on host OS mount
Hans Lawrenz added the comment:
Host OS: Mac OS 10.11.1
Guest OS: Ubuntu Trusty 64
Virtualbox: 4.3.30
Vagrant: 1.7.4
Example with trace thrown:
vagrant@vagrant-ubuntu-trusty-64:/vagrant$ cat tt.py
import tempfile
with tempfile.TemporaryFile(dir="/vagrant") as tf:
tf.write("testing testing testing\n".encode('utf-8'))
vagrant@vagrant-ubuntu-trusty-64:/vagrant$ python3.5 tt.py
Traceback (most recent call last):
File "tt.py", line 4, in
with tempfile.TemporaryFile(dir="/vagrant") as tf:
File "/usr/lib/python3.5/tempfile.py", line 753, in TemporaryFile
newline=newline, encoding=encoding)
FileNotFoundError: [Errno 2] No such file or directory
I think the underlying system call that's causing the error (I attached the
full strace in case that's helpful):
open("/vagrant", O_RDWR|O_EXCL|O_DIRECTORY|O_NOFOLLOW|O_CLOEXEC|0x40) = -1
EOPNOTSUPP (Operation not supported)
Finally, the mount point looks like this:
vagrant on /vagrant type vboxsf (uid=1000,gid=1000,rw)
--
Added file: http://bugs.python.org/file41145/tempfile-strace.txt
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25717>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20836] Pickle Nonetype
New submission from Hans Polak:
This simple code works, but I have two 'import pickle' lines.
The none working code has one 'import pickle' outside the class definition.
This raises a Nonetype error in the pickle module, in the __del__ method. I'd
like to know why and I think it's a bug, or is it?
class Login():
users = {}
def __init__(self):
try:
with open('user.data','rb') as f:
import pickle
self.users = pickle.load(f)
#print('init: {}'.format(self.users))
except Exception:
#'user.data' file doesn't exist
pass
def __del__(self):
try:
with open('user.data','wb') as f:
#print('del: {}'.format(self.users))
#print(type(self.users))
#print(type(f))
import pickle
pickle.dump(self.users, f, pickle.HIGHEST_PROTOCOL)
except Exception as e:
print('__del__ Shit happened: {}'.format(e))
--
assignee: docs@python
components: Documentation
messages: 212598
nosy: Hans.Polak, docs@python
priority: normal
severity: normal
status: open
title: Pickle Nonetype
type: behavior
versions: Python 3.2
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20836
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
Python 2.7.5: Strange and differing behavior depending on sys.setdefaultencoding being set
Hi,
I'm experiencing strange behavior with attached code, that differs depending
on sys.setdefaultencoding being set or not. If it is set, the code works as
expected, if not - what should be the usual case - the code fails with some
non-sensible traceback.
I tried to boil it down to a comprehensible state, but some LOC are still
involved. I use similar code to represent database records, where the repr's
appear in user visible logs for example.
It is greatly appreciated, if some kind soul could shed some light on this
ungrateful behavior.
Thanks in advance,
Pete#!/usr/bin/env python
# -*- coding: utf-8 -*-
u
# this code will usually crash with python 2.7.5, except some poor soul has thrown
# a file called sitecustomize.py into systems site-packages containing:
import sys
sys.setdefaultencoding('utf-8')
# question is, why does it crash without defining a default encoding?
# these are basically two classes with a __repr__ method, where one class
# refers to the other. On a repr, a two level recursion should happen.
# from the log traces, one can see, that in the good case, this is dealt
# well, in the normal case, the recursion isn't handled well: when it
# should continue, it restarts the loop for some reason.
# with default encoding set (comments starting the #):
DEBUG: ['a', 'b', 'c', 'd', 'e', 'f'] # iterate over these elements of D2Rec
DEBUG: a
DEBUG: b
DEBUG: c
DEBUG: d
DEBUG: ['bla', 'ho', 'hu', 'pi', 'tup'] # iterate over these elements of D1Rec
DEBUG: bla
DEBUG: ho
DEBUG: hu
DEBUG: pi
DEBUG: tup # D1Rec exhausted
DEBUG: e# continue with rest of D2Rec
DEBUG: f# finish
DEBUG: D2Rec( # dump structure
a: 1
b: 2
c: 3
d: [D1Rec(
bla: []
ho: u'äöü'
hu: 'hu'
pi: 3.14
tup: ()
)]
e: u'fünf'
f: 'sechs'
)
# all is good, without (the normal, but failing way):
DEBUG: ['a', 'b', 'c', 'd', 'e', 'f'] # iterate over these elements of D2Rec
DEBUG: a
DEBUG: b
DEBUG: c
DEBUG: d
DEBUG: ['bla', 'ho', 'hu', 'pi', 'tup'] # iterate over these elements of D1Rec
DEBUG: bla
DEBUG: ho
DEBUG: hu
DEBUG: pi
DEBUG: tup # D1Rec exhausted
DEBUG: ['a', 'b', 'c', 'd', 'e', 'f'] # it should continue with rest of D1Rec
DEBUG: a# but restarts the process for unknown reasons
DEBUG: b# effectively processing these items twice
DEBUG: c# only to commit suicide thereafter
DEBUG: d
DEBUG: ['bla', 'ho', 'hu', 'pi', 'tup']
DEBUG: bla
DEBUG: ho
DEBUG: hu
DEBUG: pi
DEBUG: tup
Traceback (most recent call last):
File /usr/lib64/python2.7/logging/__init__.py, line 851, in emit
msg = self.format(record)
File /usr/lib64/python2.7/logging/__init__.py, line 724, in format
return fmt.format(record)
File /usr/lib64/python2.7/logging/__init__.py, line 467, in format
s = self._fmt % record.__dict__
File reprtest.py, line 80, in __repr__
return u'%s(\n%s\n)' % (self.__class__.__name__, frec(self.__dict__))
File reprtest.py, line 66, in frec
ret.append(u'%*s: %s' % (maxklen, key, reprstr(rec[key])))
File reprtest.py, line 53, in reprstr
s = repr(s)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 22-24: ordinal not in range(128)
Logged from file reprtest.py, line 108
# Consequently, the traceback doesn't make any sense..
import sys
import logging
logconfig = {
'level': logging.DEBUG,
'format': '%(levelname)s: %(message)s',
'encoding': 'utf8',
}
logging.basicConfig(**logconfig)
log = logging.getLogger(__name__)
def isascii(s):
tests a string, if it can be represented as pure ascii
return all(ord(c) 128 for c in s)
def reprstr(s):
helper to format values in a python 2 compatible way,
using unicode only, where necessary, and quote strings
if isinstance(s, basestring):
if isascii(s):
s = repr(str(s))
else:
assert isinstance(s, unicode), only unicode for non ascii strings allowed: %r % s
s = u'%s' % s.replace(', \\')
else:
s = repr(s)
return s
def frec(rec):
'''format a dict in a easy to read sorted record presentation
'''
ret = []
keys = [key for key in rec]
maxklen = len(keys) and max([len(key) for key in keys]) or 0
log.debug(sorted(keys))
for key in sorted(keys):
log.debug(key)
ret.append(u'%*s: %s' % (maxklen, key, reprstr(rec[key])))
return u'\n'.join(ret)
def recordfactory(classname, **kwargs):
record factory, returning a class name classname,
and keyword args assigned as class members
class Record(object):
represent a Record, carrying its attributes as class members
def __init__(self, **kwargs):
self.__dict__.update(kwargs)
Re: Python 2.7.5: Strange and differing behavior depending on sys.setdefaultencoding being set
Hi Chris,
On Mittwoch, 4. Dezember 2013 10:20:31 Chris Angelico wrote:
On Wed, Dec 4, 2013 at 9:32 AM, Hans-Peter Jansen h...@urpla.net wrote:
I'm experiencing strange behavior with attached code, that differs
depending on sys.setdefaultencoding being set or not. If it is set, the
code works as expected, if not - what should be the usual case - the code
fails with some non-sensible traceback.
Interesting. You're mixing str and unicode objects a lot here. The
cleanest solution, IMO, would be to either switch to Python 3 or add
this to the top of your code:
from __future__ import unicode_literals
Either way, you'll have all your quoted strings be Unicode, rather
than byte, strings. Then take away the requirement that Unicode
strings contain non-ASCII characters, and let everything go through
that code branch.
Looking at this line in reprstr():
s = u'%s' % s.replace(', \\')
Two potential problems with that. Firstly, the representation is
flawed: a backslash in the input string won't be changed, so it's not
a true repr; but if this is just for debugging output, that's not a
big deal. Secondly, this code might produce either a str or a unicode,
depending on the type of s. That may cause messes later; since you
seem to be mostly working with the unicode type after that, it'd
probably be simpler/safer to make that always return one:
The code serves three purposes: make simple strings more readable, document
the others as being unicode, and display those correctly ;)
s = uu'%s' % s.replace(', \\')
But the actual problem, I think, is that repr() guarantees to return a
str, and you're trying to return a unicode. Here's an illustration:
# -*- coding: utf-8 -*-
class Foo(object):
def __repr__(self):
return u'äöü'
foo = Foo()
print(foo.__repr__())
print(repr(foo))
The first one succeeds, because building up that string isn't at all a
problem. The second one then tries to turn the return value of
__repr__ into a string using the default encoding - which defaults to
'ascii', hence the problem you're seeing.
Solution 1: Switch to Python 3, in which this will work fine (because
repr() in Py3 returns a Unicode string, since _everything_ is
Unicode).
Solution 2: Explicitly encode in frec, or at the end of Record.__repr__():
def __repr__(self):
s = u'%s(\n%s\n)' % (self.__class__.__name__,
frec(self.__dict__)) return s.encode(utf-8)
(that could be a one-liner, but it's already pushing 80-chars, so if
you have a length limit, breaking it helps)
Solution 3: Don't use __repr__ here, but simply have your frec
function intelligently handle Record types. Effectively, you have your
own method of generating a debug description of a Record, which could
then return a unicode instead of a str.
Thanks for all your considerations, they are very helpful indeed. Even more
helpful, that I understand the issue in question now. I will take some rest
and then decide, what to do about this with your precious help.
I personally recommend switching to Python 3 :) But presumably that's
not an option, or you'd already have considered it.
You nailed it ;)
Given the amount of special unicode handling code, that is necessary to keep
Python 2 happy, makes proceeding with it no real fun on a longer term..
And the biggest proponent for hacking in Python IS the fun part of it. Then
productivity, elegance, ..., you name it.
Have-a-good-day-ly y'rs,
Pete
--
https://mail.python.org/mailman/listinfo/python-list
Release 3.2.0 of CodeInvestigator.
CodeInvestigator 3.2.0 was released on August 21. Bug fixes: - Chrome browser: search by value - less memory usage Changes: - nested iterations remain at selected iteration as much as possible - block colors now follow a set pattern - searches are now grouped in subgroups - user can now use middle-click to open functions in a separate tab - user can use refresh back forward and bookmarks - more shortcut keys - printout position retained over iterations - iteration tabs can now be scrolled through You need Python = 2.7 and Firefox or Chrome for CodeInvestigator. CodeInvestigator is a tracing tool for Python programs. Running a program through CodeInvestigator creates a recording. Program flow, function calls, variable values and conditions are all stored for every line the program executes. The recording is then viewed with an interface consisting of the code. The code can be clicked: A clicked variable displays its value, a clicked loop displays its iterations. You read code, and have at your disposal all the run time details of that code. A computerized desk check tool and another way to learn about your program. http://sourceforge.net/project/showfiles.php?group_id=183942 -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
How can I redirect or launch a html file in wsgi coding?
Hi,
I have code like this:
root@lin-ser-1:~# cat /usr/local/www/wsgi-scripts/myapp.py
def application(environ, start_response):
import sys
...
status = '200 OK'
req_method=environ['REQUEST_METHOD']
if req_method == 'POST' :
json_received = environ['wsgi.input'].read()
resp=lib_json_http.process_json(json_received)
output = simplejson.dumps(resp)
elif req_method == 'GET' :
webbrowser.open('http://autotestnet.sourceforge.net/')
return This code does not work
else :
output = invalid request method
response_headers = [('Content-type', 'text/plain'),
('Content-Length', str(len(output)))]
start_response(status, response_headers)
return [output]
POST works OK, for GET, I hope I can redirect it to a url link, or launch a
local html file, how can I do it?
thanks!!!
--
http://mail.python.org/mailman/listinfo/python-list
how can I check if group member exist ?
Hi, I'm doing a regular expression matching, let's say a=re.search(re_str,match_str), if matching, I don't know how many str/item will be extracted from re_str, maybe a.group(1), a.group(2) exist but a.group(3) does not. Can I somehow check it? something like: if exist(a.group(1)): print a.group(1) if exist(a.group(2)): print a.group(2) if exist(a.group(3)): print a.group(3) I don't want to be hit by Indexerror: print a.group(3) Traceback (most recent call last): File stdin, line 1, in module IndexError: no such group thanks!!! -- http://mail.python.org/mailman/listinfo/python-list
Re: using split for a string : error
On 25/01/13 15:04:02, Neil Cerutti wrote: On 2013-01-25, Oscar Benjamin oscar.j.benja...@gmail.com wrote: On 24 January 2013 11:35, Chris Angelico ros...@gmail.com wrote: It's usually fine to have int() complain about any non-numerics in the string, but I must confess, I do sometimes yearn for atoi() semantics: atoi(123asd) == 123, and atoi(qqq) == 0. I've not seen a convenient Python function for doing that. Usually it involves manually getting the digits off the front. All I want is to suppress the error on finding a non-digit. Oh well. I'm interested to know what the situations are where you want the behaviour of atoi(). Right. atoi is no good even in C. You get much better control using the sprintf family. I think you meant sscanf. It's true that sscanf gives you more control. That being said, sometimes the one option atoi gives you, just happens to be what you need. int would need to return a tuple of the number it found plus the number of characters consumed to be more useful for parsing. intparse(123abc) (123, 3) But that would make it might inconvenient for general use. If the new function is nameed intparse, and the existing int function remains available, then most use cases would be served by int, and intparse would be available as a building block for other use cases. For example atoi could be defined as: def atoi(s): return intparse(s)[0] intparse(xyz) should return (0, 0), and leave it to the caller to decide whether a ValueError shoud be raised. -- HansM -- http://mail.python.org/mailman/listinfo/python-list
Re: urllib2 FTP Weirdness
On 24/01/13 00:58:04, Chris Angelico wrote: On Thu, Jan 24, 2013 at 7:07 AM, Nick Cash nick.c...@npcinternational.com wrote: Python 2.7.3 on linux This has me fairly stumped. It looks like urllib2.urlopen(ftp://some.ftp.site/path;).read() will either immediately return '' or hang indefinitely. But response = urllib2.urlopen(ftp://some.ftp.site/path;) response.read() works fine and returns what is expected. This is only an issue with urllib2, vanilla urllib doesn't do it. The site I first noticed it on is private, but I can reproduce it with ftp://ftp2.census.gov/;. Confirmed on 2.6.5 on Windows, fwiw. This is extremely weird. It works fine with 2.7.3 on my Mac. Possibly it's some kind of race condition?? If urllib2 is using active mode FTP, then a firewall on your box could explain what you're seeing. But then, that's why active mode is hardly used these days. Hope this helps, -- HansM -- http://mail.python.org/mailman/listinfo/python-list
Re: help
On 10/01/13 19:35:40, kwakukwat...@gmail.com wrote:
pls this is a code to show the pay of two people.bt I want each of to be
able to get a different money when they enter their user name,and to use
it for about six people.
database = [
['Mac'],
['Sam'],
]
pay1 = 1000
pay2 = 2000
name = raw_input('Enter your name: ')
if [name] in database:print your pay is $,pay
How about:
database = dict(Mac=1000, Sam=2000, Eric=3000, Terry=4000, Graham=5000)
name = raw_input('Enter your name: ')
if name in database: print Your pay is ${}.format(database[name])
Hope this helps,
-- HansM
--
http://mail.python.org/mailman/listinfo/python-list
Re: please i need explanation
On 11/01/13 16:35:10, kwakukwat...@gmail.com wrote: def factorial(n): if n2: return 1 f = 1 while n= 2: f *= n f -= 1 U think this line should have been: n -= 1 return f Hope this helps, -- HansM -- http://mail.python.org/mailman/listinfo/python-list
Re: Numpy outlier removal
On 6/01/13 20:44:08, Joseph L. Casale wrote: I have a dataset that consists of a dict with text descriptions and values that are integers. If required, I collect the values into a list and create a numpy array running it through a simple routine: data[abs(data - mean(data)) m * std(data)] where m is the number of std deviations to include. The problem is I loos track of which were removed so the original display of the dataset is misleading when the processed average is returned as it includes the removed key/values. Ayone know how I can maintain the relationship and when I exclude a value, remove it from the dict? Assuming your data and the dictionary are keyed by a common set of keys: for key in descriptions: if abs(data[key] - mean(data)) = m * std(data): del data[key] del descriptions[key] Hope this helps, -- HansM -- http://mail.python.org/mailman/listinfo/python-list
Re: Missing something obvious with python-requests
On 4/01/13 03:56:47, Chris Angelico wrote:
On Fri, Jan 4, 2013 at 5:53 AM, Ray Cote
rgac...@appropriatesolutions.com wrote:
proxies = {
'https': '192.168.24.25:8443',
'http': '192.168.24.25:8443', }
a = requests.get('http://google.com/', proxies=proxies)
When I look at the proxy log, I see a GET being performed -- when it should
be a CONNECT.
Does not matter if I try to get http or https google.com.
Not sure if it's related to your problem or not, but my understanding
of a non-SSL request through a proxy is that it'll be a GET request
(eg GET http://google.com/ HTTP/1.0). So the problem is only that
it's still doing GET requests when it's an https query (which is where
CONNECT is needed).
It all depends on the proxy URL.
It the proxy URL is http://192.168.24.25/, then the client should send
GET requests to the proxy in both cases, and the proxy should send GET
or CONNECT to the origin server, depending on whether origin URL uses
SSL.
If the proxy URL is https://192.168.24.25/, then the client should send
CONNECT requests to the proxy, and the proxy should send GET or CONNECT
as appropriate.
Python-requests appears to implement only the first case.
Hope this helps,
-- HansM
--
http://mail.python.org/mailman/listinfo/python-list
Re: New to python, do I need an IDE or is vim still good enough?
On 31/12/12 12:57:59, Adam Tauno Williams wrote: On Thu, 2012-12-27 at 12:01 -0800, mogul wrote: 'Aloha! I'm new to python, got 10-20 years perl and C experience, all gained on unix alike machines hacking happily in vi, and later on in vim. Now it's python, and currently mainly on my kubuntu desktop. Do I really need a real IDE, as the windows guys around me say I do, You don't need one. +1 You are crazy if you don't WANT one. ITYM: you'd be crazy if you'd want one. Check out geany http://www.geany.org/ Don't bother: Python comes with a free IDE named IDLE. Try it for a few minutes, and you'll find that most of the features that make vim so wonderful, are missing from IDLE. Just stay with vim. -- HansM -- http://mail.python.org/mailman/listinfo/python-list
Re: Question about nested loop
On 31/12/12 11:02:56, Isaac Won wrote: Hi all, I am a very novice for Python. Currently, I am trying to read continuous columns repeatedly in the form of array. my code is like below: import numpy as np b = [] c = 4 f = open(text.file, r) while c 10: c = c + 1 for columns in ( raw.strip().split() for raw in f ): b.append(columns[c]) y = np.array(b, float) print c, y I thought that can get the arrays of the columns[5] to [10], but I only could get repetition of same arrays of columns[5]. The result was something like: 5 [1 2 3 4 .., 10 9 8] 6 [1 2 3 4 .., 10 9 8] 7 [1 2 3 4 .., 10 9 8] 8 [1 2 3 4 .., 10 9 8] 9 [1 2 3 4 .., 10 9 8] 10 [1 2 3 4 .., 10 9 8] What I can't understand is that even though c increased incrementally upto 10, y arrays stay same. Would someone help me to understand this problem more? That's because the inner loop read from a file until his reaches the end of the file. Since you're not resetting the file pointer, during the second and later runs of the outer loop, the inner loop starts at the end of the file and terminates without any action. You'd get more interesting results if you rewind the file: import numpy as np b = [] c = 4 f = open(text.file, r) while c 10: c = c + 1 f.seek(0,0) for columns in ( raw.strip().split() for raw in f ): b.append(columns[c]) y = np.array(b, float) print c, y It's a bit inefficient to read the same file several times. You might consider reading it just once. For example: import numpy as np b = [] f = open(text.file, r) data = [ line.strip().split() for line in f ] f.close() for c in xrange(5, 11): for row in data: b.append(row[c]) y = np.array(b, float) print c, y Hope this helps, -- HansM -- http://mail.python.org/mailman/listinfo/python-list
Re: Tarfile and usernames
On 30/12/12 19:57:31, Nicholas Cole wrote: Dear List, I'm hoping to use the tarfile module in the standard library to move some files between computers. I can't see documented anywhere what this library does with userids and groupids. I can't guarantee that the computers involved will have the same users and groups, and would like the archives to be extracted so that the files are all owned by the extracting user. Essentially, I do *not* with to preserve the owner and groups specified in the archives. What is the right way to achieve this? I would agree that this ought to be documented. From reading the code: the way to achieve this, is to run as a user other than root. Or monkeypatch the TarFile.chown method to be a no-op. Hope this helps, -- HansM -- http://mail.python.org/mailman/listinfo/python-list
Re: Python lists
On 28/12/12 18:46:45, Alex wrote:
Manatee wrote:
On Friday, December 28, 2012 9:14:57 AM UTC-5, Manatee wrote:
I read in this:
['C100, C117', 'X7R 0.033uF 10% 25V 0603', '0603-C_L, 0603-C_N',
'10', '2', '', '30', '15463-333', 'MURATA', 'GRM188R71E333KA01D',
'Digi-Key', '490-1521-1-ND', '']
Then I need to convert it to this:
[['C100', 'C117'], 'X7R 0.033uF 10% 25V 0603', '0603-C_L',
'0603-C_N', '10', '2', '', '30', '15463-333', 'MURATA',
'GRM188R71E333KA01D', 'Digi-Key', '490-1521-1-ND', '']]
l = ['C100, C117', 'X7R 0.033uF 10% 25V 0603', '0603-C_L, 0603-C_N',
'10', '2', '', '30', '15463-333', 'MURATA', 'GRM188R71E333KA01D',
'Digi-Key', '490-1521-1-ND', '']
[item.split(',') if i == 0 else item for i, item in enumerate(l)]
If it's okay to modify the original list, you can simply do:
l[0] = split(l[0], , )
If modifying the original is not okay, the simple solution would
be to copy it first:
l2 = l
l2[0] = split(l2[0], , )
Hope this helps,
-- HansM
--
http://mail.python.org/mailman/listinfo/python-list
Re: Beginner: Trying to get REAL NUMBERS from %d command
Hello, Python does not support REAL numbers. It has float number, which are approximations of real numbers. They behave almost, but not quite, like you might expect. It also has Decimal numbers. They also approximate real numbers, but slightly differently. They might behave more like you'd expect if you're used to an electronic calculator. On 30/12/12 23:37:53, Alvaro Lacerda wrote: The code I wrote is supposed to ask the user to enter a number; Then tell the user what's going to happen to that number (x / 2 + 5) ; Then give the user an answer; I succeeded getting results from even numbers, but when I try diving an uneven number (i.e. 5) by 2, I get only the whole number (i.e. 2) I'm trying to get full number result using the %d command, I've tried to add the line from __future__ import division to the beginning of my code, but I haven't been succeeded. I also tried making my numbers decimals (i.e. 2.0 instead of just 2) Errrhm, 2.0 is a float. To get decimal numbers, you'd need to import the Decimal module. Below is my program and thanks for the help :) from __future__ import division; def decimal_percent_test(): number = input(Enter a number: ); print number, will be divided by 2 then added by 5!; print %d divided by %d is %d plus %d is... %d ! %( number,2,number/2,5,number/2+5); The %d code will convert your number to a whole number. as you found out. The simplest way to make this program work, is to use the %s code, which doesn't convert to a different type of number: from __future__ import division; def decimal_percent_test(): number = input(Enter a number: ) print number, will be divided by 2 then added by 5! print %s divided by %s is %s plus %s is... %s ! % ( number, 2, number/2, 5, number/2+5) while True: decimal_percent_test() Enter a number: 10 10 will be divided by 2 then added by 5! 10 divided by 2 is 5.0 plus 5 is... 10.0 ! Enter a number: 11 11 will be divided by 2 then added by 5! 11 divided by 2 is 5.5 plus 5 is... 10.5 ! Enter a number: x Traceback (most recent call last): File test.py, line 11, in module decimal_percent_test() File test.py, line 5, in decimal_percent_test number = input(Enter a number: ) File string, line 1, in module NameError: name 'x' is not defined Hope this helps, -- HansM -- http://mail.python.org/mailman/listinfo/python-list
Re: Python lists
On 30/12/12 23:25:39, Evan Driscoll wrote: On 12/30/2012 4:19 PM, Hans Mulder wrote: If it's okay to modify the original list, you can simply do: l[0] = split(l[0], , ) If modifying the original is not okay, the simple solution would be to copy it first: l2 = l l2[0] = split(l2[0], , ) Um, that doesn't copy the list: Oops, you're right. l = [C100, C117, X7R ...] l2 = l import string l2[0] = string.split(l2[0], , ) l [['C100', 'C117'], 'X7R ...'] l2 [['C100', 'C117'], 'X7R ...'] To make a copy of a list you can either use slice syntax (l2 = l[:]) or call the list constructor (l2 = list(l)). Thanks for correcting me, -- HansM -- http://mail.python.org/mailman/listinfo/python-list
Re: need a url that its http response header that cotain 401 status code
On 26/12/12 10:08:41, iMath wrote: I am going to do a Basic Authentication , so I need a url that its http response header that cotain 401 status code. Isn't that backwards? I mean, what's the point of implementing Basic Authentication, unless you already know a site that uses it? if you don't know any such site, you can set one up easily: just run a webserver on your laptop and password-protect a page in your server configuration. When you've found a suitable url, take a look at HTTPBasicAuthHandler in the urllib2 modules. Hope this helps, -- HansM -- http://mail.python.org/mailman/listinfo/python-list
Re: how to detect the character encoding in a web page ?
On 24/12/12 01:34:47, iMath wrote: how to detect the character encoding in a web page ? That depends on the site: different sites indicate their encoding differently. such as this page: http://python.org/ If you download that page and look at the HTML code, you'll find a line: meta http-equiv=content-type content=text/html; charset=utf-8 / So it's encoded as utf-8. Other sites declare their charset in the Content-Type HTTP header line. And then there are sites relying on the default. And sites that get it wrong, and send data in a different encoding from what they declare. Welcome to the real world, -- HansM -- http://mail.python.org/mailman/listinfo/python-list
Re: Forking into the background (Linux)
On 24/12/12 01:50:24, Olive wrote:
My goal is to write a script that 1) write something to stdout; then
fork into the background, closing the stdout (and stderr, stdin) pipe.
I have found this answer (forking - setsid - forking)
http://stackoverflow.com/a/3356154
However the standard output of the child is still connected to the
terminal. I would like that if we execute a subprocess.checkprocess on
this program, only I would like to see this is captured and that the
program terminates when the parent exits.
#! /usr/bin/python2
import os,sys,time
print I would like to see this
pid = os.fork()
if (pid == 0): # The first child.
# os.chdir(/)
os.setsid()
# os.umask(0)
pid2 = os.fork()
if (pid2 == 0): # Second child
print I would like not see this
time.sleep(5)
else:
sys.exit()#First child exists
else: # Parent Code
sys.exit() # Parent exists
You could do this before forking:
sys.stdin.close()
sys.stdin = open('/dev/null', 'r')
sys.stdout.close()
sys.stdout = open('/dev/null', 'w')
sys.stderr.close()
sys.stderr = open('/dev/null', 'w')
You may want to look at the python-daemon module on Pypy, which appears
to do what you need, including some features you haven't asked for, yet.
Hope this helps,
-- HansM
--
http://mail.python.org/mailman/listinfo/python-list
Re: redirect standard output problem
On 21/12/12 06:23:18, iMath wrote:
redirect standard output problem
why the result only print A but leave out 888 ?
import sys
class RedirectStdoutTo:
def __init__(self, out_new):
self.out_new = out_new
def __enter__(self):
sys.stdout = self.out_new
def __exit__(self, *args):
sys.stdout = sys.__stdout__
print('A')
with open('out.log', mode='w', encoding='utf-8') as a_file,
RedirectStdoutTo(a_file):
print('B')
print('C')
print(888)
On my machine it works as you'd expect.
If it doesn't work on your system, it might help to flush
sys.stdout in a few strategic places:
class RedirectStdoutTo:
def __init__(self, out_new):
self.out_new = out_new
def __enter__(self):
sys.stdout.flush()
sys.stdout = self.out_new
def __exit__(self, *args):
sys.stdout.flush()
sys.stdout = sys.__stdout__
print('A')
with open('out.log', mode='w', encoding='utf-8') as a_file,
RedirectStdoutTo(a_file):
print('B')
print('C')
print(888)
sys.stdout.flush()
Hope this helps,
-- HansM
--
http://mail.python.org/mailman/listinfo/python-list
