ANN: A new version (0.5.3) of python-gnupg has been released.
What Changed?= This is an enhancement and bug-fix release, and all users are encouraged to upgrade. Brief summary: - * Fix #117: Add WKD (Web Key Directory) support for auto-locating keys. Thanks to Myzel394 for the patch. * Fix #237: Ensure local variable is initialized even when an exception occurs. - * Fix #239: Remove logging of decryption result. This release [2] has been signed with my code signing key: Vinay Sajip (CODE SIGNING KEY) Fingerprint: CA74 9061 914E AC13 8E66 EADB 9147 B477 339A 9B86 Recent changes to PyPI don't show the GPG signature with the download links. An alternative download source where the signatures are available is at [4]. The source code repository is at [1]. Documentation is available at [5]. As always, your feedback is most welcome (especially bug reports [3], patches and suggestions for improvement, or any other points via this group). Enjoy! Cheers Vinay Sajip [1] https://github.com/vsajip/python-gnupg [2] https://pypi.org/project/python-gnupg/0.5.3 [3] https://github.com/vsajip/python-gnupg/issues [4] https://github.com/vsajip/python-gnupg/releases/ [5] python-gnupg - A Python wrapper for GnuPG -- https://mail.python.org/mailman/listinfo/python-list
ANN: A new version (0.5.2) of python-gnupg has been released.
What Changed?= This is an enhancement and bug-fix release, and all users are encouraged to upgrade. Brief summary: * Fix #228: Clarify documentation for encryption/decryption. * Make I/O buffer size configurable via buffer_size attribute on a GPG instance. This release [2] has been signed with my code signing key: Vinay Sajip (CODE SIGNING KEY) Fingerprint: CA74 9061 914E AC13 8E66 EADB 9147 B477 339A 9B86 Recent changes to PyPI don't show the GPG signature with the download links. An alternative download source where the signatures are available is at [4]. The source code repository is at [1]. Documentation is available at [5]. As always, your feedback is most welcome (especially bug reports [3], patches and suggestions for improvement, or any other points via this group). Enjoy! Cheers Vinay Sajip [1] https://github.com/vsajip/python-gnupg [2] https://pypi.org/project/python-gnupg/0.5.2 [3] https://github.com/vsajip/python-gnupg/issues [4] https://github.com/vsajip/python-gnupg/releases/ [5] https://docs.red-dove.com/python-gnupg/ -- https://mail.python.org/mailman/listinfo/python-list
Announcement: distlib 0.3.8 released on PyPI
Version 0.3.8 of distlib has recently been released on PyPI [1]. For newcomers, distlib is a library of packaging functionality which is intended to be usable as the basis for third-party packaging tools. The main changes in this release are as follows: * Fix #204: use symlinks in venv creation during test. * Fix #208: handle deprecation removals in Python 3.13. * Fix #209: use legacy version implementation for Python versions. A more detailed change log is available at [2]. Please try it out, and if you find any problems or have any suggestions for improvements, please give some feedback using the issue tracker at [3]. Regards, Vinay Sajip [1] https://pypi.org/project/distlib/0.3.8/ [2] https://distlib.readthedocs.io/en/latest/overview.html#change-log-for-distlib [3] https://github.com/pypa/distlib/issues/new/choose -- https://mail.python.org/mailman/listinfo/python-list
ANN: A new version (0.5.1) of python-gnupg has been released.
What Changed? = This is an enhancement and bug-fix release, and all users are encouraged to upgrade. Brief summary: * Added TRUST_EXPIRED to trust_keys. * Fix #206: Remove deprecated --always-trust in favour of --trust-model always * Fix #208: Add status_detail attribute to result objects which is populated when the status is 'invalid recipient' (encryption/decryption) or 'invalid signer' (signing). This attribute will be set when the result object's status attribute is set to 'invalid recipient' and will contain more information about the failure in the form of reason:ident where reason is a text description of the reason, and ident identifies the recipient key. * Add scan_keys_mem() function to scan keys in a string. * Fix #214: Handle multiple signatures when one of them is invalid or unverified. * A problems attribute was added which holds problems reported by gpg during verification. This is a list of dictionaries, one for each reported problem. Each dictionary will have status and keyid keys indicating the problem and the corresponding key; other information in the dictionaries will be error specific. * Fix #217: Use machine-readable interface to query the gpg version. * Added the ability to export keys to a file. This release [2] has been signed with my code signing key: Vinay Sajip (CODE SIGNING KEY) Fingerprint: CA74 9061 914E AC13 8E66 EADB 9147 B477 339A 9B86 Recent changes to PyPI don't show the GPG signature with the download links. An alternative download source where the signatures are available is at [4]. The source code repository is at [1]. Documentation is available at [5]. As always, your feedback is most welcome (especially bug reports [3], patches and suggestions for improvement, or any other points via this group). Enjoy! Cheers Vinay Sajip [1] https://github.com/vsajip/python-gnupg [2] https://pypi.org/project/python-gnupg/0.5.1 [3] https://github.com/vsajip/python-gnupg/issues [4] https://github.com/vsajip/python-gnupg/releases/ [5] https://docs.red-dove.com/python-gnupg/ -- https://mail.python.org/mailman/listinfo/python-list
Announcement: distlib 0.3.7 released on PyPI
Version 0.3.7 of distlib has recently been released on PyPI [1]. For newcomers, distlib is a library of packaging functionality which is intended to be usable as the basis for third-party packaging tools. The main changes in this release are as follows: * Handle bare newlines when parsing metadata. * Use version comparison logic for python_full_version. * Fix shebang computation for source builds of Python. * Extract tarfiles more safely by incorporating tarfile filters. * Check for 'has_cert' attribute before using it. * Fix #200: Improve conformance to PEP440. * Fix #203: Handle parsing of export entries to allow script names such as "," or ",foo". A more detailed change log is available at [2]. Please try it out, and if you find any problems or have any suggestions for improvements, please give some feedback using the issue tracker at [3]. Regards, Vinay Sajip [1] https://pypi.org/project/distlib/0.3.7/ [2] https://distlib.readthedocs.io/en/0.3.7/overview.html#change-log-for-distlib [3] https://github.com/pypa/distlib/issues/new/choose -- https://mail.python.org/mailman/listinfo/python-list
ANN: A new version (0.5.0) of python-gnupg has been released.
What Changed? = This is an enhancement and bug-fix release, and all users are encouraged to upgrade. Brief summary: * Fixed #181: Added the ability to pass file paths to encrypt_file, decrypt_file, sign_file, verify_file, get_recipients_file and added import_keys_file. * Fixed #183: Handle FAILURE and UNEXPECTED conditions correctly. Thanks to sebbASF for the patch. * Fixed #185: Handle VALIDSIG arguments more robustly. * Fixed #188: Remove handling of DECRYPTION_FAILED from Verify code, as not required there. Thanks to sebbASF for the patch. * Fixed #190: Handle KEY_CREATED more robustly. * Fixed #191: Handle NODATA messages during verification. * Fixed #196: Don't log chunk data by default, as it could contain sensitive information (during decryption, for example). * Added the ability to pass an environment to the gpg executable. Thanks to Edvard Rejthar for the patch. This release [2] has been signed with my code signing key: Vinay Sajip (CODE SIGNING KEY) Fingerprint: CA74 9061 914E AC13 8E66 EADB 9147 B477 339A 9B86 Recent changes to PyPI don't show the GPG signature with the download links. The source code repository is at [1]. An alternative download source where the signatures are available is at [4]. Documentation is available at [5]. As always, your feedback is most welcome (especially bug reports [3], patches and suggestions for improvement, or any other points via this group). Enjoy! Cheers Vinay Sajip [1] https://github.com/vsajip/python-gnupg [2] https://pypi.org/project/python-gnupg/0.5.0 [3] https://github.com/vsajip/python-gnupg/issues [4] https://github.com/vsajip/python-gnupg/releases/ [5] https://docs.red-dove.com/python-gnupg/ -- https://mail.python.org/mailman/listinfo/python-list
ANN: distlib 0.3.5 released on PyPI
I've recently released version 0.3.5 of distlib on PyPI [1]. For newcomers, distlib is a library of packaging functionality which is intended to be usable as the basis for third-party packaging tools. The main changes in this release are as follows: * Fixed #161: Updated test case. * Fixed #164: Improved support for reproducible builds by allowing a fixed date/time to be inserted into created .exe files. Thanks to Somber Night for the patch. * Fixed #169: Removed usage of deprecated imp module in favour of importlib. * Fixed #170: Corrected implementation of ``get_required_dists()``. * Fixed #172: Compute ABI correctly for Python < 3.8. * Changed the default locator configuration. * Made updates in support of PEP 643 / Metadata 2.2. * Updated launcher executables. Thanks to Michael Bikovitsky for his help with the launcher changes. * Updated to write archive path of RECORD to RECORD instead of staging path. Thanks to Pieter Pas for the patch. A more detailed change log is available at [2]. Please try it out, and if you find any problems or have any suggestions for improvements, please give some feedback using the issue tracker! [3] Regards, Vinay Sajip [1] https://pypi.org/project/distlib/0.3.5/ [2] https://distlib.readthedocs.io/en/0.3.5/ [3] https://github.com/pypa/distlib/issues/new/choose -- https://mail.python.org/mailman/listinfo/python-list
ANN: A new version (0.4.9) of python-gnupg has been released.
What Changed? = This is an enhancement and bug-fix release, and all users are encouraged to upgrade. Brief summary: * Fixed #161: Added a status attribute to the returned object from gen_key() which is set to 'ok' if a key was successfully created, or 'key not created' if that was reported by gpg, or None in any other case. * Fixed #164: Provided the ability to add subkeys. Thanks to Daniel Kilimnik for the feature request and patch. * Fixed #166: Added keygrip values to the information collected when keys are listed. Thanks to Daniel Kilimnik for the feature request and patch. * Fixed #173: Added extra_args to send_keys(), recv_keys() and search_keys() to allow passing options relating to key servers. This release [2] has been signed with my code signing key: Vinay Sajip (CODE SIGNING KEY) Fingerprint: CA74 9061 914E AC13 8E66 EADB 9147 B477 339A 9B86 Recent changes to PyPI don't show the GPG signature with the download links. The source code repository is at [1]. An alternative download source where the signatures are available is at [4]. Documentation is available at [5]. As always, your feedback is most welcome (especially bug reports [3], patches and suggestions for improvement, or any other points via this group). Enjoy! Cheers Vinay Sajip [1] https://github.com/vsajip/python-gnupg [2] https://pypi.org/project/python-gnupg/0.4.9 [3] https://github.com/vsajip/python-gnupg/issues [4] https://github.com/vsajip/python-gnupg/releases/ [5] https://docs.red-dove.com/python-gnupg/ -- https://mail.python.org/mailman/listinfo/python-list
ANN: distlib 0.3.4 released on PyPI
I've recently released version 0.3.4 of distlib on PyPI [1]. For newcomers, distlib is a library of packaging functionality which is intended to be usable as the basis for third-party packaging tools. The main changes in this release are as follows: * Fixed #153: Raise warnings in get_distributions() if bad metadata seen, but keep going. * Fixed #154: Determine Python versions correctly for Python >= 3.10. * Updated launcher executables with changes to handle duplication logic. Code relating to support for Python 2.6 was also removed (support for Python 2.6 was dropped in an earlier release, but supporting code wasn't removed until now). A more detailed change log is available at [2]. Please try it out, and if you find any problems or have any suggestions for improvements, please give some feedback using the issue tracker! [3] Regards, Vinay Sajip [1] https://pypi.org/project/distlib/0.3.4/ [2] https://distlib.readthedocs.io/en/0.3.4/ [3] https://bitbucket.org/pypa/distlib/issues/new -- https://mail.python.org/mailman/listinfo/python-list
ANN: A new library for encryption and signing has been released.
A new library called pagesign (Python-age-sign) has been released on PyPI [1]. It covers similar functionality to python-gnupg, but uses the modern encryption tool age [2] and the modern signing tool minisign [3]. The initial release allows you to: * Create and manage identities (which are containers for the keys used for encryption/decryption and signing/verification). * Encrypt and decrypt files to multiple recipients. * Sign files and verify signatures. This release has been signed with my code signing key: Vinay Sajip (CODE SIGNING KEY) Fingerprint: CA74 9061 914E AC13 8E66 EADB 9147 B477 339A 9B86 Recent changes to PyPI don't show the GPG signature with the download links. An alternative download source where the signatures are available is at [4]. Documentation is available at [5]. As always, your feedback is most welcome (especially bug reports [6], patches and suggestions for improvement, or any other points). Enjoy! Cheers Vinay Sajip [1] https://pypi.org/project/pagesign/0.1.0/ [2] https://age-encryption.org/ [3] https://jedisct1.github.io/minisign/ [4] https://bitbucket.org/vinay.sajip/pagesign/downloads/ [5] https://docs.red-dove.com/pagesign/ [6] https://github.com/vsajip/pagesign/issues/new/choose -- https://mail.python.org/mailman/listinfo/python-list
ANN: distlib 0.3.3 released on PyPI
I've recently released version 0.3.3 of distlib on PyPI [1]. For newcomers, distlib is a library of packaging functionality which is intended to be usable as the basis for third-party packaging tools. The main changes in this release are as follows: * Fixed #152: Removed splituser() function which wasn't used and is deprecated. * Fixed #149: Handle version comparisons correctly in environment markers. * Add ARM-64 launchers and support code to use them. Thanks to Niyas Sait and Adrian Vladu for their contributions. * Fixed #148: Handle a single trailing comma following a version. Thanks to Blazej Floch for the report and suggested fix. * Fixed #150: Fix incorrect handling of epochs. * Reverted handling of tags for Python >= 3.10 (use 310 rather than 3_10). This is because PEP 641 was rejected. * Added a GitHub Actions workflow to perform tests. A more detailed change log is available at [2]. Please try it out, and if you find any problems or have any suggestions for improvements, please give some feedback using the issue tracker! [3] Regards, Vinay Sajip [1] https://pypi.org/project/distlib/0.3.3/ [2] https://distlib.readthedocs.io/en/0.3.3/ [3] https://bitbucket.org/pypa/distlib/issues/new -- https://mail.python.org/mailman/listinfo/python-list
Re: ANN: Version 0.5.1 of the Python config module has been released.
Many of those 4.8K "users" might be using indirectly via some other dependency - I'm not sure how GitHub calculates "used by", but even if it were a direct dependency, one has no idea if it's actually being used or not. so I tend not to worry about such things. My distlib library has only 2 stars, and is used by pip and therefore by many developers every day, and yet it doesn't merit a "used by" according to GitHub. I've had a few users contact me about the config library and they're happy with it and they find it useful, and that's good enough for me :-) On Sunday, 12 September 2021, 18:11:21 BST, Abdur-Rahmaan Janhangeer wrote: Used by 4.8k but only ... 4 stars -- https://mail.python.org/mailman/listinfo/python-list
ANN: Version 0.5.1 of the Python config module has been released.
What Does It Do? The CFG configuration format is a text format for configuration files which is similar to, and a superset of, the JSON format.It has the following aims: * Allow a hierarchical configuration scheme with support for key-value mappings and lists. * Support cross-references between one part of the configuration and another. * Provide a string interpolation facility to easily build up configuration values from other configuration values. * Provide the ability to compose configurations (using include and merge facilities). * Provide the ability to access real application objects safely. * Be completely declarative. It overcomes a number of drawbacks of JSON when used as a configuration format: * JSON is more verbose than necessary. * JSON doesn’t allow comments. * JSON doesn’t provide first-class support for dates and multi-line strings. * JSON doesn’t allow trailing commas in lists and mappings. * JSON doesn’t provide easy cross-referencing, interpolation, or composition. The Python config module provides an interface to work with configuration files written in the CFG format. Comprehensive documentation is available at https://docs.red-dove.com/cfg/index.html and you can report issues / enhancement requests at https://github.com/vsajip/py-cfg-lib/issues As always, your feedback is most welcome (especially bug reports, patches and suggestions for improvement). Enjoy! Cheers, Vinay Sajip -- https://mail.python.org/mailman/listinfo/python-list
ANN: distlib 0.3.2 released on PyPI
I've recently released version 0.3.2 of distlib on PyPI [1]. For newcomers, distlib is a library of packaging functionality which is intended to be usable as the basis for third-party packaging tools. The main changes in this release are as follows: * Fixed #139: improved handling of errors related to the test PyPI server. * Fixed #140: allowed "Obsoletes" in more scenarios, to better handle faulty metadata already on PyPI. * Fixed #141: removed unused regular expression. * Fixed #143: removed normcase() to avoid some problems on Windows. * Fixed #146: added entry for SourcelessFileLoader to the finder registry. * Fixed #147: permission bits are now preserved on POSIX when installing from a wheel. * Made the generation of scripts more configurable. * Added support for manylinux wheel tags. A more detailed change log is available at [2]. Please try it out, and if you find any problems or have any suggestions for improvements, please give some feedback using the issue tracker! [3] Regards, Vinay Sajip [1] https://pypi.org/project/distlib/0.3.2/ [2] https://distlib.readthedocs.io/en/0.3.2/ [3] https://bitbucket.org/pypa/distlib/issues/new -- https://mail.python.org/mailman/listinfo/python-list
ANN: Version 0.1.6 of sarge (a subprocess wrapper library) has been released.
Version 0.1.6 of Sarge, a cross-platform library which wraps the subprocess
module in the standard library, has been released.
What changed?
-
- Fixed #44: Added an optional timeout to Command.wait() and Pipeline.wait(),
which
only takes effect on Python >= 3.3.
- Fixed #47: Added the ``replace_env`` keyword argument which allows a complete
replacement for ``os.environ`` to be passed.
- Fixed #51: Improved error handling around a Popen call.
What does Sarge do?
---
Sarge tries to make interfacing with external programs from your
Python applications easier than just using subprocess alone.
Sarge offers the following features:
* A simple way to run command lines which allows a rich subset of Bash-
style shell command syntax, but parsed and run by sarge so that you
can run on Windows without cygwin (subject to having those commands
available):
>>> from sarge import capture_stdout
>>> p = capture_stdout('echo foo | cat; echo bar')
>>> for line in p.stdout: print(repr(line))
...
'foo\n'
'bar\n'
* The ability to format shell commands with placeholders, such that
variables are quoted to prevent shell injection attacks.
* The ability to capture output streams without requiring you to
program your own threads. You just use a Capture object and then you
can read from it as and when you want.
* The ability to look for patterns in captured output and to interact
accordingly with the child process.
Advantages over subprocess
---
Sarge offers the following benefits compared to using subprocess:
* The API is very simple.
* It's easier to use command pipelines - using subprocess out of the
box often leads to deadlocks because pipe buffers get filled up.
* It would be nice to use Bash-style pipe syntax on Windows, but
Windows shells don't support some of the syntax which is useful, like
&&, ||, |& and so on. Sarge gives you that functionality on Windows,
without cygwin.
* Sometimes, subprocess.Popen.communicate() is not flexible enough for
one's needs - for example, when one needs to process output a line at
a time without buffering the entire output in memory.
* It's desirable to avoid shell injection problems by having the
ability to quote command arguments safely.
* subprocess allows you to let stderr be the same as stdout, but not
the other way around - and sometimes, you need to do that.
Python version and platform compatibility
-
Sarge is intended to be used on any Python version >= 2.6 and is
tested on Python versions 2.6, 2.7, 3.3, 3.4, 3.5, 3.6, 3.7 and 3.8 on Linux,
Windows, and Mac OS X (not all versions are tested on all platforms,
but sarge is expected to work correctly on all these versions on all
these platforms).
Finding out more
You can read the documentation at
http://sarge.readthedocs.org/
There's a lot more information, with examples, than I can put into
this post.
You can install Sarge using "pip install sarge" to try it out. The
project is hosted on BitBucket at
https://bitbucket.org/vinay.sajip/sarge/
And you can leave feedback on the issue tracker there.
I hope you find Sarge useful!
Regards,
Vinay Sajip
--
https://mail.python.org/mailman/listinfo/python-list
ANN: distlib 0.3.1 released on PyPI
I've recently released version 0.3.1 of distlib on PyPI [1]. For newcomers, distlib is a library of packaging functionality which is intended to be usable as the basis for third-party packaging tools. The main changes in this release are as follows: * Fixed #132: Added documentation to help with relative interpreter paths. Thanks to Paul Kienzle for the patch. * Fixed #134: Allowed specifying a different target Python version when generating scripts. * Fixed #135: Exposed the ``enquote_executable`` function previously implemented as an internal function. * Addressed #138: Improved metadata support for newer metadata versions. Thanks to James Tocknell for the patch. * Changed the output of flags in entry point definitions in wheels. Thanks to frostming (明希) for the patch. * Stopped writing JSON metadata. Only old-style metadata is written now. A more detailed change log is available at [2]. Please try it out, and if you find any problems or have any suggestions for improvements, please give some feedback using the issue tracker! [3] Note: Mailman3 might mishandle some of the links below. In that case, just copy and paste the links into your browser address bar - that should work. Regards, Vinay Sajip [1] https://pypi.org/project/distlib/0.3.1/ [2] https://distlib.readthedocs.io/en/0.3.1/ [3] https://bitbucket.org/pypa/distlib/issues/new -- https://mail.python.org/mailman/listinfo/python-list
ANN: distlib 0.3.0 released on PyPI
I've recently released version 0.3.0 of distlib on PyPI [1]. For newcomers,distlib is a library of packaging functionality which is intended to beusable as the basis for third-party packaging tools. The main changes in this release are as follows: * Partially addressed #102: modules attribute of InstalledDistribution was incorrectly computed as a list of bytes, rather than a list of str. This has now been corrected. * Updated Locator._get_digest to check PyPI JSON responses for a "digests" dictionary before trying "algo_digest" keys. Thanks to Jeffery To for the patch. * Fixed #123: Improved error message if a resource isn't found. * Fixed #124: Stopped norm-casing the executable written into shebangs, as it doesn't work for some non-ASCII paths. * Fixed #125: Updated launchers with versions that correctly report errors containing non-ASCII characters. The updated launchers now also support relative paths (see http://bit.ly/2JxmOoi for more information). * Fixed #127: Allowed hyphens in flags in export specifications. * Changed Python version handling to accommodate versions like e.g. 3.10 (no longer assume a version X.Y where X and Y are single digits). A more detailed change log is available at [2]. Please try it out, and if you find any problems or have any suggestions for improvements,please give some feedback using the issue tracker! [3] Regards, Vinay Sajip [1] https://pypi.org/project/distlib/0.3.0/[2] https://distlib.readthedocs.io/en/0.3.0/[3] https://bitbucket.org/pypa/distlib/issues/new -- https://mail.python.org/mailman/listinfo/python-list
ANN: A new version (0.4.5) of python-gnupg has been released.
A new version of the Python module which wraps GnuPG has been released.
What Changed?=
This is an enhancement and bug-fix release, and all users are encouraged
toupgrade.
Brief summary:
* Fixed #107: Improved documentation.
* Fixed #112: Raised a ValueError if a gnupghome is specified which is not an
existing directory.
* Fixed #113: Corrected stale link in the documentation.
* Fixed #116: Updated documentation to clarify when spurious key-expired/
signature-expired messages might be seen.
* Fixed #119: Added --yes to avoid pinentry when deleting secret keys with
GnuPG >= 2.1.
* A warning is logged if gpg returns a non-zero return code.
* Added ``extra_args`` to ``import_keys``.
* Added support for CI using AppVeyor.
This release [2] has been signed with my code signing key:
Vinay Sajip (CODE SIGNING KEY) Fingerprint: CA74
9061 914E AC13 8E66 EADB 9147 B477 339A 9B86
Recent changes to PyPI don't show the GPG signature with the download links.An
alternative download source where the signatures are available is the
project'sown downloads page [5].
What Does It Do?The gnupg module allows Python programs to make
use of thefunctionality provided by the Gnu Privacy Guard (abbreviated GPG
orGnuPG). Using this module, Python programs can encrypt and decryptdata,
digitally sign documents and verify digital signatures, manage(generate, list
and delete) encryption keys, using proven Public KeyInfrastructure (PKI)
encryption technology based on OpenPGP.
This module is expected to be used with Python versions >= 2.4, as itmakes use
of the subprocess module which appeared in that version ofPython. This module
is a newer version derived from earlier work byAndrew Kuchling, Richard Jones
and Steve Traugott.
A test suite using unittest is included with the source distribution.
Simple usage:
>>> import gnupg>>> gpg = gnupg.GPG(gnupghome='/path/to/keyring/directory')>>>
>>> gpg.list_keys()
[{...'fingerprint': 'F819EE7705497D73E3CCEE65197D5DAC68F1AAB2','keyid':
'197D5DAC68F1AAB2','length': '1024','type': 'pub','uids': ['', 'Gary Gross (A
test user) ']},{...'fingerprint':
'37F24DD4B918CC264D4F31D60C5FEFA7A921FC4A','keyid':
'0C5FEFA7A921FC4A','length': '1024',...'uids': ['', 'Danny Davis (A test user)
']}]>>> encrypted = gpg.encrypt("Hello, world!",
['0C5FEFA7A921FC4A'])>>> str(encrypted)
'-BEGIN PGP MESSAGE-\nVersion: GnuPG v1.4.9
(GNU/Linux)\n\nhQIOA/6NHMDTXUwcEAf.-END PGP MESSAGE-\n'>>> decrypted =
gpg.decrypt(str(encrypted), passphrase='secret')>>> str(decrypted)
'Hello, world!'>>> signed = gpg.sign("Goodbye, world!", passphrase='secret')>>>
verified = gpg.verify(str(signed))>>> print "Verified" if verified else "Not
verified"
'Verified'
As always, your feedback is most welcome (especially bug reports [3],patches
and suggestions for improvement, or any other points via themailing
list/discussion group [4]).
Enjoy!
Cheers
Vinay SajipRed Dove Consultants Ltd.
[1] https://bitbucket.org/vinay.sajip/python-gnupg[2]
https://pypi.org/project/python-gnupg/0.4.5[3]
https://bitbucket.org/vinay.sajip/python-gnupg/issues[4]
https://groups.google.com/forum/#!forum/python-gnupg[5]
https://bitbucket.org/vinay.sajip/python-gnupg/downloads/--
--
https://mail.python.org/mailman/listinfo/python-list
ANN: distlib 0.2.9 released on PyPI
I've recently released version 0.2.9 of distlib on PyPI [1]. For newcomers,distlib is a library of packaging functionality which is intended to beusable as the basis for third-party packaging tools. The main changes in this release are as follows: * Updated default PyPI URL to https://pypi.org/pypi. * Relaxed metadata format checks to ignore 'Provides'. * Fixed #33, #34: simplified script template. * Fixed #115: Relaxed check for '..' in wheel archive entries by not checking filename parts, only directory segments. * Fixed #116: Corrected parsing of credentials from URLs. * Fixed #122: Skipped entries in archive entries ending with '/' (directories) when verifying or installing. * Commented out Disqus comment section in documentation. A more detailed change log is available at [2]. Please try it out, and if you find any problems or have any suggestions for improvements,please give some feedback using the issue tracker! [3] Regards, Vinay Sajip [1] https://pypi.org/project/distlib/0.2.9/[2] https://distlib.readthedocs.io/en/latest/overview.html#change-log-for-distlib[3] https://bitbucket.org/pypa/distlib/issues/new -- https://mail.python.org/mailman/listinfo/python-list
ANN: A new version (0.4.4) of python-gnupg has been released. It contains a security-related change - please update to this version
A new version of the Python module which wraps GnuPG has been released.
What Changed?=This is an enhancement and security-fix release, and
all users are stronglyencouraged to upgrade.
Brief summary:
* Fixed #108: Changed how any return value from the on_data callable is
processed. In earlier versions, the return value was ignored. In this version,
if the return value is False, the data received from gpg is not buffered.
Otherwise (if the value is None or True, for example), the data is buffered as
normal. This functionality can be used to do your own buffering, or to prevent
buffering altogether.
The on_data callable is also called once with an empty byte-string to signal
the end of data from gpg.
* Fixed #97: Added an additional attribute check_fingerprint_collisions to GPG
instances, which defaults to False. It seems that gpg is happy to have
duplicate keys and fingerprints in a keyring, so we can't be too strict. A
user can set this attribute of an instance to True to trigger a check for
collisions.
* Fixed #111: With GnuPG 2.2.7 or later, provide the fingerprint of a signing
key for a failed signature verification, if available.
* Fixed #21: For verification where multiple signatures are involved, a
mapping of signature_ids to fingerprint, keyid, username, creation date,
creation timestamp and expiry timestamp is provided.
* Added a check to disallow certain control characters ('\r', '\n', NUL) in
passphrases. This fix mitigates against CVE-2019-6690.
This release [2] has been signed with my code signing key:
Vinay Sajip (CODE SIGNING KEY) Fingerprint: CA74
9061 914E AC13 8E66 EADB 9147 B477 339A 9B86
Recent changes to PyPI don't show the GPG signature with the download links.An
alternative download source where the signatures are available is the
project'sown downloads page [5].
What Does It Do?The gnupg module allows Python programs to make
use of thefunctionality provided by the Gnu Privacy Guard (abbreviated GPG
orGnuPG). Using this module, Python programs can encrypt and decryptdata,
digitally sign documents and verify digital signatures, manage(generate, list
and delete) encryption keys, using proven Public KeyInfrastructure (PKI)
encryption technology based on OpenPGP.
This module is expected to be used with Python versions >= 2.4, as itmakes use
of the subprocess module which appeared in that version ofPython. This module
is a newer version derived from earlier work byAndrew Kuchling, Richard Jones
and Steve Traugott.
A test suite using unittest is included with the source distribution.
Simple usage:
>>> import gnupg>>> gpg = gnupg.GPG(gnupghome='/path/to/keyring/directory')>>>
>>> gpg.list_keys()
[{...'fingerprint': 'F819EE7705497D73E3CCEE65197D5DAC68F1AAB2','keyid':
'197D5DAC68F1AAB2','length': '1024','type': 'pub','uids': ['', 'Gary Gross (A
test user) ']},{...'fingerprint':
'37F24DD4B918CC264D4F31D60C5FEFA7A921FC4A','keyid':
'0C5FEFA7A921FC4A','length': '1024',...'uids': ['', 'Danny Davis (A test user)
']}]>>> encrypted = gpg.encrypt("Hello, world!",
['0C5FEFA7A921FC4A'])>>> str(encrypted)
'-BEGIN PGP MESSAGE-\nVersion: GnuPG v1.4.9
(GNU/Linux)\n\nhQIOA/6NHMDTXUwcEAf.-END PGP MESSAGE-\n'>>> decrypted =
gpg.decrypt(str(encrypted), passphrase='secret')>>> str(decrypted)
'Hello, world!'>>> signed = gpg.sign("Goodbye, world!", passphrase='secret')>>>
verified = gpg.verify(str(signed))>>> print "Verified" if verified else "Not
verified"
'Verified'
As always, your feedback is most welcome (especially bug reports [3],patches
and suggestions for improvement, or any other points via themailing
list/discussion group [4]).
Enjoy!
Cheers
Vinay SajipRed Dove Consultants Ltd.
[1] https://bitbucket.org/vinay.sajip/python-gnupg[2]
https://pypi.python.org/pypi/python-gnupg/0.4.4[3]
https://bitbucket.org/vinay.sajip/python-gnupg/issues[4]
https://groups.google.com/forum/#!forum/python-gnupg[5]
https://bitbucket.org/vinay.sajip/python-gnupg/downloads/
--
https://mail.python.org/mailman/listinfo/python-list
ANN: distlib 0.2.8 released on PyPI
I've recently released version 0.2.8 of distlib on PyPI [1]. For newcomers,distlib is a library of packaging functionality which is intended to beusable as the basis for third-party packaging tools. The main changes in this release are as follows: * Fixed #107: Updated documentation on testing to include information on setting PYTHONHASHSEED. * Fixed #108: Updated metadata scan to look for the METADATA file as well as the JSON formats. * Fixed #109: Removed existing files (which might have been symlinks) before overwriting. * Fixed #111: Avoided unnecessary newlines in script preambles, which caused problems with detecting encoding declarations. Thanks to Wim Glenn for the report and patch. * Fixed #112: Handled wheel tags and platform-dependent downloads correctly in SimpleScrapingLocator. A more detailed change log is available at [2]. Please try it out, and if you find any problems or have any suggestions for improvements,please give some feedback using the issue tracker! [3] Regards, Vinay Sajip [1] https://pypi.org/project/distlib/0.2.8/[2] https://goo.gl/tVzKUc[3] https://bitbucket.org/pypa/distlib/issues/new -- https://mail.python.org/mailman/listinfo/python-list
ANN: Version 0.1.5 of sarge (a subprocess wrapper library) has been released.
Version 0.1.5 of Sarge, a cross-platform library which wraps the
subprocessmodule in the standard library, has been released.
What changed?-
- Fixed #37: Instead of an OSError with a "no such file or directory" message,
a ValueError is raised with a more informative "Command not found" message.
- Fixed #38: Replaced ``async`` keyword argument with ``async_``, as ``async``
has become a keyword in Python 3.7.
- Fixed #39: Updated tutorial example on progress monitoring.
What does Sarge do?---
Sarge tries to make interfacing with external programs from yourPython
applications easier than just using subprocess alone.
Sarge offers the following features:
* A simple way to run command lines which allows a rich subset of Bash-style
shell command syntax, but parsed and run by sarge so that youcan run on Windows
without cygwin (subject to having those commandsavailable):
>>> from sarge import capture_stdout>>> p = capture_stdout('echo foo | cat;
>>> echo bar')>>> for line in p.stdout: print(repr(line))...'foo\n''bar\n'
* The ability to format shell commands with placeholders, such thatvariables
are quoted to prevent shell injection attacks.
* The ability to capture output streams without requiring you toprogram your
own threads. You just use a Capture object and then youcan read from it as and
when you want.
* The ability to look for patterns in captured output and to
interactaccordingly with the child process.
Advantages over subprocess---
Sarge offers the following benefits compared to using subprocess:
* The API is very simple.
* It's easier to use command pipelines - using subprocess out of thebox often
leads to deadlocks because pipe buffers get filled up.
* It would be nice to use Bash-style pipe syntax on Windows, butWindows shells
don't support some of the syntax which is useful, like&&, ||, |& and so on.
Sarge gives you that functionality on Windows,without cygwin.
* Sometimes, subprocess.Popen.communicate() is not flexible enough forone's
needs - for example, when one needs to process output a line ata time without
buffering the entire output in memory.
* It's desirable to avoid shell injection problems by having theability to
quote command arguments safely.
* subprocess allows you to let stderr be the same as stdout, but notthe other
way around - and sometimes, you need to do that.
Python version and platform
compatibility-
Sarge is intended to be used on any Python version >= 2.6 and istested on
Python versions 2.6, 2.7, 3.3, 3.4, 3.5, 3.6 and 3.7 on Linux,Windows, and Mac
OS X (not all versions are tested on all platforms,but sarge is expected to
work correctly on all these versions on allthese platforms).
Finding out more
You can read the documentation at
http://sarge.readthedocs.org/
There's a lot more information, with examples, than I can put intothis post.
You can install Sarge using "pip install sarge" to try it out. Theproject is
hosted on BitBucket at
https://bitbucket.org/vinay.sajip/sarge/
And you can leave feedback on the issue tracker there.
I hope you find Sarge useful!
Regards,
Vinay Sajip
--
https://mail.python.org/mailman/listinfo/python-list
ANN: A new version (0.4.3) of python-gnupg has been released. It contains a security-related change - please update to this version
A new version of the Python module which wraps GnuPG has been released.
What Changed?=This is a security-fix release, and all users are
strongly encouraged to upgrade.This fix mitigates against CVE-2018-12020. See
the discoverer's blog post [6] formore information.
Brief summary:
* Added --no-verbose to the gpg command line, in case verbose is specified in
gpg.conf - we don't need verbose output.
This release [2] has been signed with my code signing key:
Vinay Sajip (CODE SIGNING KEY) Fingerprint: CA74
9061 914E AC13 8E66 EADB 9147 B477 339A 9B86
Recent changes to PyPI don't show the GPG signature with the download links.An
alternative download source where the signatures are available is the
project'sown downloads page [5].
What Does It Do?The gnupg module allows Python programs to make
use of thefunctionality provided by the Gnu Privacy Guard (abbreviated GPG
orGnuPG). Using this module, Python programs can encrypt and decryptdata,
digitally sign documents and verify digital signatures, manage(generate, list
and delete) encryption keys, using proven Public KeyInfrastructure (PKI)
encryption technology based on OpenPGP.
This module is expected to be used with Python versions >= 2.4, as itmakes use
of the subprocess module which appeared in that version ofPython. This module
is a newer version derived from earlier work byAndrew Kuchling, Richard Jones
and Steve Traugott.
A test suite using unittest is included with the source distribution.
Simple usage:
>>> import gnupg>>> gpg = gnupg.GPG(gnupghome='/path/to/keyring/directory')>>>
>>> gpg.list_keys()
[{...'fingerprint': 'F819EE7705497D73E3CCEE65197D5DAC68F1AAB2','keyid':
'197D5DAC68F1AAB2','length': '1024','type': 'pub','uids': ['', 'Gary Gross (A
test user) ']},{...'fingerprint':
'37F24DD4B918CC264D4F31D60C5FEFA7A921FC4A','keyid':
'0C5FEFA7A921FC4A','length': '1024',...'uids': ['', 'Danny Davis (A test user)
']}]>>> encrypted = gpg.encrypt("Hello, world!",
['0C5FEFA7A921FC4A'])>>> str(encrypted)
'-BEGIN PGP MESSAGE-\nVersion: GnuPG v1.4.9
(GNU/Linux)\n\nhQIOA/6NHMDTXUwcEAf.-END PGP MESSAGE-\n'>>> decrypted =
gpg.decrypt(str(encrypted), passphrase='secret')>>> str(decrypted)
'Hello, world!'>>> signed = gpg.sign("Goodbye, world!", passphrase='secret')>>>
verified = gpg.verify(str(signed))>>> print "Verified" if verified else "Not
verified"
'Verified'
As always, your feedback is most welcome (especially bug reports [3],patches
and suggestions for improvement, or any other points via themailing
list/discussion group [4]).
Enjoy!
Cheers
Vinay SajipRed Dove Consultants Ltd.
[1] https://bitbucket.org/vinay.sajip/python-gnupg[2]
https://pypi.python.org/pypi/python-gnupg/0.4.3[3]
https://bitbucket.org/vinay.sajip/python-gnupg/issues[4]
https://groups.google.com/forum/#!forum/python-gnupg[5]
https://bitbucket.org/vinay.sajip/python-gnupg/downloads/[6]
https://neopg.io/blog/gpg-signature-spoof/
--
https://mail.python.org/mailman/listinfo/python-list
ANN: distlib 0.2.7 released on PyPI
I've just released version 0.2.7 of distlib on PyPI [1]. For newcomers, distlib is a library of packaging functionality which is intended to be usable as the basis for third-party packaging tools. The main changes in this release are as follows: * Addressed #102: InstalledDistributions now have a modules attribute which is a list of top-level modules as read from top_level.txt, if that is in the distribution info. * Fixed #103: Now https downloads are preferred to those over http. Thanks to Saulius Žemaitaitis for the patch. * Fixed #104: Updated launcher binaries to properly handle interpreter paths with spaces. Thanks to Atsushi Odagiri for the diagnosis and fix. * Fixed #105: cache_from_source is now imported from importlib.util where available. * Added support for PEP 566 / Metadata 2.1. A more detailed change log is available at [2]. Please try it out, and if you find any problems or have any suggestions for improvements, please give some feedback using the issue tracker! [3] Regards, Vinay Sajip [1] https://pypi.org/project/distlib/0.2.7/ [2] https://goo.gl/M3kQzR [3] https://bitbucket.org/pypa/distlib/issues/new -- https://mail.python.org/mailman/listinfo/python-list
ANN: A new version (0.4.2) of python-gnupg has been released.
A new version of the Python module which wraps GnuPG has been released.
What Changed? = This is an enhancement and bug-fix release, and all
users are encouraged to upgrade. See the project website [1] for more
information.
Brief summary:
* Subkey information is now collected and returned in a subkey_info dictionary
keyed by the subkey's ID.
* GPG2 version is now correctly detected on OS X.
* Added expect_passphrase keyword argument for use on GnuPG >= 2.1 when
passing passphrase to gpg via pinentry.
* Provided a trust_keys method to allow setting the trust level for keys.
Thanks to William Foster for a suggested implementation.
* Made the exception message when the gpg executable is not found contain the
path of the executable that was tried. Thanks to Kostis Anagnostopoulos for
the suggestion.
* Made the error message less categorical in the case of a failure with an
unspecified reason, adding some information from gpg error codes when
available.
This release [2] has been signed with my code signing key:
Vinay Sajip (CODE SIGNING KEY) Fingerprint: CA74 9061
914E AC13 8E66 EADB 9147 B477 339A 9B86
What Does It Do? The gnupg module allows Python programs to
make use of the functionality provided by the Gnu Privacy Guard (abbreviated
GPG or GnuPG). Using this module, Python programs can encrypt and decrypt data,
digitally sign documents and verify digital signatures, manage (generate, list
and delete) encryption keys, using proven Public Key Infrastructure (PKI)
encryption technology based on OpenPGP.
This module is expected to be used with Python versions >= 2.4, as it makes use
of the subprocess module which appeared in that version of Python. This module
is a newer version derived from earlier work by Andrew Kuchling, Richard Jones
and Steve Traugott.
A test suite using unittest is included with the source distribution.
Simple usage:
>>> import gnupg >>> gpg = gnupg.GPG(gnupghome='/path/to/keyring/directory')
>>> >>> gpg.list_keys()
[{ ... 'fingerprint': 'F819EE7705497D73E3CCEE65197D5DAC68F1AAB2', 'keyid':
'197D5DAC68F1AAB2', 'length': '1024', 'type': 'pub', 'uids': ['', 'Gary Gross
(A test user) ']}, { ... 'fingerprint':
'37F24DD4B918CC264D4F31D60C5FEFA7A921FC4A', 'keyid': '0C5FEFA7A921FC4A',
'length': '1024', ... 'uids': ['', 'Danny Davis (A test user) ']}] >>> encrypted = gpg.encrypt("Hello, world!",
['0C5FEFA7A921FC4A']) >>> str(encrypted)
'-BEGIN PGP MESSAGE-\nVersion: GnuPG v1.4.9 (GNU/Linux)\n
\nhQIOA/6NHMDTXUwcEAf . -END PGP MESSAGE-\n' >>> decrypted =
gpg.decrypt(str(encrypted), passphrase='secret') >>> str(decrypted)
'Hello, world!' >>> signed = gpg.sign("Goodbye, world!", passphrase='secret')
>>> verified = gpg.verify(str(signed)) >>> print "Verified" if verified else
"Not verified"
'Verified'
As always, your feedback is most welcome (especially bug reports [3], patches
and suggestions for improvement, or any other points via the mailing
list/discussion group [4]).
Please refer to the documentation [5] for more information.
Enjoy!
Cheers
Vinay Sajip Red Dove Consultants Ltd.
[1] https://bitbucket.org/vinay.sajip/python-gnupg [2]
https://pypi.python.org/pypi/python-gnupg/0.4.2 [3]
https://bitbucket.org/vinay.sajip/python-gnupg/issues [4]
https://groups.google.com/forum/#!forum/python-gnupg [5]
https://gnupg.readthedocs.io/en/latest/
--
https://mail.python.org/mailman/listinfo/python-list
ANN: distlib 0.2.6 released on PyPI
I've just released version 0.2.6 of distlib on PyPI [1]. For newcomers,distlib is a library of packaging functionality which is intended to beusable as the basis for third-party packaging tools. The main changes in this release are as follows: * Fixed #99: Updated to handle a case where sys.getfilesystemencoding() returns None. * Fixed #97: Eliminated a crash in EggInfoDistribution.list_distinfo_files() which was caused by trying to open a non-existent file. * Fixed #96: SimpleScrapingLocator no longer fails prematurely when scraping links due to invalid versions. * Improved error messages issued when interpreting markers. * Improved the shebangs written into installed scripts when the interpreter path is very long or contains spaces (to cater for a limitation in shebang line parsing on Linux). * Updated launcher binaries. A more detailed change log is available at [2]. Please try it out, and if you find any problems or have any suggestions forimprovements, please give some feedback using the issue tracker! [3] Regards, Vinay Sajip [1] https://pypi.python.org/pypi/distlib/0.2.6[2] https://goo.gl/M3kQzR[3] https://bitbucket.org/pypa/distlib/issues/new -- https://mail.python.org/mailman/listinfo/python-list
ANN: A new version (0.4.1) of python-gnupg has been released.
A new version of the Python module which wraps GnuPG has been released.
What Changed?
=
This is an enhancement and bug-fix release, and all users are encouraged to
upgrade.
See the project website [1] for more information.
Brief summary:
* Updated message handling logic to no longer raise exceptions when a message
isn't
recognised. Thanks to Daniel Kahn Gillmor for the patch.
* Always use always use --fixed-list-mode, --batch and --with-colons. Thanks to
Daniel
Kahn Gillmor for the patch.
* Improved scan_keys() handling on GnuPG >= 2.1. Thanks to Daniel Kahn Gillmor
for the
patch.
* Improved test behaviour with GnuPG >= 2.1. Failures when deleting test
directory trees
are now ignored. Thanks to Daniel Kahn Gillmor for the patch.
* Added close_file keyword argument to verify_file to allow the file closing to
be made
optional. Current behaviour is maintained - close_file=False can be passed to
skip
closing the file being verified.
* Added the extra_args keyword parameter to allow custom arguments to be passed
to the
gpg executable.
* Instances of the GPG class now have an additional on_data attribute, which
defaults to
None. It can be set to a callable which will be called with a single argument
- a binary
chunk of data received from the gpg executable. The callable can do whatever
it likes
with the chunks passed to it - e.g. write them to a separate stream. The
callable should
not raise any exceptions (unless it wants the current operation to fail).
This release [2] has been signed with my code signing key:
Vinay Sajip (CODE SIGNING KEY)
Fingerprint: CA74 9061 914E AC13 8E66 EADB 9147 B477 339A 9B86
What Does It Do?
The gnupg module allows Python programs to make use of the
functionality provided by the Gnu Privacy Guard (abbreviated GPG or
GnuPG). Using this module, Python programs can encrypt and decrypt
data, digitally sign documents and verify digital signatures, manage
(generate, list and delete) encryption keys, using proven Public Key
Infrastructure (PKI) encryption technology based on OpenPGP.
This module is expected to be used with Python versions >= 2.4, as it
makes use of the subprocess module which appeared in that version of
Python. This module is a newer version derived from earlier work by
Andrew Kuchling, Richard Jones and Steve Traugott.
A test suite using unittest is included with the source distribution.
Simple usage:
>>> import gnupg
>>> gpg = gnupg.GPG(gnupghome='/path/to/keyring/directory')
>>> gpg.list_keys()
[{
...
'fingerprint': 'F819EE7705497D73E3CCEE65197D5DAC68F1AAB2',
'keyid': '197D5DAC68F1AAB2',
'length': '1024',
'type': 'pub',
'uids': ['', 'Gary Gross (A test user) ']},
{
...
'fingerprint': '37F24DD4B918CC264D4F31D60C5FEFA7A921FC4A',
'keyid': '0C5FEFA7A921FC4A',
'length': '1024',
...
'uids': ['', 'Danny Davis (A test user) ']}]
>>> encrypted = gpg.encrypt("Hello, world!", ['0C5FEFA7A921FC4A'])
>>> str(encrypted)
'-BEGIN PGP MESSAGE-\nVersion: GnuPG v1.4.9 (GNU/Linux)\n
\nhQIOA/6NHMDTXUwcEAf
.
-END PGP MESSAGE-\n'
>>> decrypted = gpg.decrypt(str(encrypted), passphrase='secret')
>>> str(decrypted)
'Hello, world!'
>>> signed = gpg.sign("Goodbye, world!", passphrase='secret')
>>> verified = gpg.verify(str(signed))
>>> print "Verified" if verified else "Not verified"
'Verified'
As always, your feedback is most welcome (especially bug reports [3],
patches and suggestions for improvement, or any other points via the
mailing list/discussion group [4]).
Please refer to the documentation [5] for more information.
Enjoy!
Cheers
Vinay Sajip
Red Dove Consultants Ltd.
[1] https://bitbucket.org/vinay.sajip/python-gnupg
[2] https://pypi.python.org/pypi/python-gnupg/0.4.1
[3] https://bitbucket.org/vinay.sajip/python-gnupg/issues
[4] https://groups.google.com/forum/#!forum/python-gnupg
[5] https://gnupg.readthedocs.io/en/latest/
--
https://mail.python.org/mailman/listinfo/python-list
ANN: distlib 0.2.5 released on PyPI
I've just released version 0.2.5 of distlib on PyPI [1]. For newcomers, distlib is a library of packaging functionality which is intended to be usable as the basis for third-party packaging tools. The main changes in this release are as follows: * Changed regular expressions to be compatible with 3.6 as regards escape sequences. * Closed some resource leaks related to XML-RPC proxies. * Changed wheel processing to look for metadata in metadata.json as well as pydist.json. * Updated requirement and marker parsing to be compatible with PEP 508. * Made downloadability a factor in scoring URLs for preferences. * Removed Python 2.6 from the support list. A more detailed change log is available at [2]. Please try it out, and if you find any problems or have any suggestions for improvements, please give some feedback using the issue tracker! [3] Regards, Vinay Sajip [1] https://pypi.python.org/pypi/distlib/0.2.5 [2] https://goo.gl/M3kQzR [3] https://bitbucket.org/pypa/distlib/issues/new -- https://mail.python.org/mailman/listinfo/python-list
ANN: A new version (0.4.0) of python-gnupg has been released.
A new version of the Python module which wraps GnuPG has been released.
What Changed?
=
This is an enhancement and bug-fix release, and all users are encouraged to
upgrade.
See the project website [1] for more information.
Brief summary:
* Added support for ``KEY_CONSIDERED`` in more places - encryption /
decryption, signing, key generation and key import.
* Partial fix for #32 (GPG 2.1 compatibility). Unfortunately, better
support cannot be provided at this point, unless there are certain
changes (relating to pinentry popups) in how GPG 2.1 works.
* Fixed #60: An IndexError was being thrown by ``scan_keys()``.
* Ensured that utf-8 encoding is used when the ``--with-column`` mode is
used. Thanks to Yann Leboulanger for the patch.
* ``list_keys()`` now uses ``--fixed-list-mode``. Thanks to Werner Koch
for the pointer.
This release [2] has been signed with my code signing key:
Vinay Sajip (CODE SIGNING KEY)
Fingerprint: CA74 9061 914E AC13 8E66 EADB 9147 B477 339A 9B86
What Does It Do?
The gnupg module allows Python programs to make use of the
functionality provided by the Gnu Privacy Guard (abbreviated GPG or
GnuPG). Using this module, Python programs can encrypt and decrypt
data, digitally sign documents and verify digital signatures, manage
(generate, list and delete) encryption keys, using proven Public Key
Infrastructure (PKI) encryption technology based on OpenPGP.
This module is expected to be used with Python versions >= 2.4, as it
makes use of the subprocess module which appeared in that version of
Python. This module is a newer version derived from earlier work by
Andrew Kuchling, Richard Jones and Steve Traugott.
A test suite using unittest is included with the source distribution.
Simple usage:
>>> import gnupg
>>> gpg = gnupg.GPG(gnupghome='/path/to/keyring/directory')
>>> gpg.list_keys()
[{
...
'fingerprint': 'F819EE7705497D73E3CCEE65197D5DAC68F1AAB2',
'keyid': '197D5DAC68F1AAB2',
'length': '1024',
'type': 'pub',
'uids': ['', 'Gary Gross (A test user) ']},
{
...
'fingerprint': '37F24DD4B918CC264D4F31D60C5FEFA7A921FC4A',
'keyid': '0C5FEFA7A921FC4A',
'length': '1024',
...
'uids': ['', 'Danny Davis (A test user) ']}]
>>> encrypted = gpg.encrypt("Hello, world!", ['0C5FEFA7A921FC4A'])
>>> str(encrypted)
'-BEGIN PGP MESSAGE-\nVersion: GnuPG v1.4.9 (GNU/Linux)\n
\nhQIOA/6NHMDTXUwcEAf
.
-END PGP MESSAGE-\n'
>>> decrypted = gpg.decrypt(str(encrypted), passphrase='secret')
>>> str(decrypted)
'Hello, world!'
>>> signed = gpg.sign("Goodbye, world!", passphrase='secret')
>>> verified = gpg.verify(str(signed))
>>> print "Verified" if verified else "Not verified"
'Verified'
As always, your feedback is most welcome (especially bug reports [3],
patches and suggestions for improvement, or any other points via the
mailing list/discussion group [4]).
Enjoy!
Cheers
Vinay Sajip
Red Dove Consultants Ltd.
[1] https://bitbucket.org/vinay.sajip/python-gnupg
[2] https://pypi.python.org/pypi/python-gnupg/0.4.0
[3] https://bitbucket.org/vinay.sajip/python-gnupg/issues
[4] https://groups.google.com/forum/#!forum/python-gnupg
--
https://mail.python.org/mailman/listinfo/python-list
ANN: distlib 0.2.4 released on PyPI
I've just released version 0.2.4 of distlib on PyPI [1]. For newcomers, distlib is a library of packaging functionality which is intended to be usable as the basis for third-party packaging tools. The main changes in this release are as follows: * Updated to not fail during import if SSL is not available. * Changed project name comparisons to follow PEP 503. * Changed manifest and resources logic to work correctly under (upcoming) Python 3.6. * Updated Windows launchers with fixes to bugs related to argument-passing in shebang lines. A more detailed change log is available at [2]. Please try it out, and if you find any problems or have any suggestions for improvements, please give some feedback using the issue tracker! [3] Regards, Vinay Sajip [1] https://pypi.python.org/pypi/distlib/0.2.4 [2] https://goo.gl/M3kQzR [3] https://bitbucket.org/pypa/distlib/issues/new -- https://mail.python.org/mailman/listinfo/python-list
ANN: distlib 0.2.2 released on PyPI
I've just released version 0.2.2 of distlib on PyPI [1]. For newcomers, distlib is a library of packaging functionality which is intended to be usable as the basis for third-party packaging tools. The main changes in this release are as follows: * Fixed issue #81: Added support for detecting distributions installed by wheel versions >= 0.23 (which use metadata.json rather than pydist.json). * Updated default PyPI URL to https://pypi.python.org/pypi * Updated to use different formatting for description field for V1.1 metadata. * Corrected “classifier” to “classifiers” in the mapping for V1.0 metadata. * Improved support for Jython when quoting executables in output scripts. * Fixed issue #77: Made the internal URL used for extended metadata fetches configurable via a module attribute. * Fixed issue #78: Improved entry point parsing to handle leading spaces in ini-format files. A more detailed change log is available at [2]. Please try it out, and if you find any problems or have any suggestions for improvements, please give some feedback using the issue tracker! [3] Regards, Vinay Sajip [1] https://pypi.python.org/pypi/distlib/0.2.2 [2] https://goo.gl/M3kQzR [3] https://bitbucket.org/pypa/distlib/issues/new -- https://mail.python.org/mailman/listinfo/python-list
ANN: A new version (0.3.8) of python-gnupg has been released.
A new version of the Python module which wraps GnuPG has been
released.
What Changed?
=
This is an enhancement and bug-fix release, and all users are encouraged to
upgrade.
See the project website [1] for more information.
Brief summary:
* Fixed #22: handled PROGRESS messages during verification and signing.
* Fixed #26: handled PINENTRY_LAUNCHED messages during verification,
decryption and key generation.
* Fixed #28: Allowed a default Name-Email to be computed even when neither of
LOGNAME and USERNAME are in the environment.
* Fixed #29: Included test files missing from the tarball in previous versions.
* Fixed #39: On Python 3.x, passing a text instead of a binary stream caused
file decryption to hang due to a UnicodeDecodeError. This has now been
correctly handled: The decryption fails with a "no data" status.
* Fixed #41: Handled Unicode filenames correctly by encoding them on 2.x using
the file system encoding.
* Fixed #43: handled PINENTRY_LAUNCHED messages during key export. Thanks to
Ian Denhardt for looking into this.
* Hide the console window which appears on Windows when gpg is spawned.
Thanks to Kévin Bernard-Allies for the patch.
* Subkey fingerprints are now captured.
* The returned value from the list_keys method now has a new attribute,
key_map, which is a dictionary mapping key and subkey fingerprints to
the corresponding key's dictionary. With this change, you don't need to
iterate over the (potentially large) returned list to search for a key with
a given fingerprint - the key_map dict will take you straight to the key
info, whether the fingerprint you have is for a key or a subkey. Thanks to
Nick Daly for the initial suggestion.
This release [2] has been signed with my code signing key:
Vinay Sajip (CODE SIGNING KEY)
Fingerprint: CA74 9061 914E AC13 8E66 EADB 9147 B477 339A 9B86
However, due to some error the signature for the source distribution (.tar.gz)
didn't get uploaded. To rectify this, I have pasted it into the PyPI page for
the release [2].
What Does It Do?
The gnupg module allows Python programs to make use of the
functionality provided by the Gnu Privacy Guard (abbreviated GPG or
GnuPG). Using this module, Python programs can encrypt and decrypt
data, digitally sign documents and verify digital signatures, manage
(generate, list and delete) encryption keys, using proven Public Key
Infrastructure (PKI) encryption technology based on OpenPGP.
This module is expected to be used with Python versions >= 2.4, as it
makes use of the subprocess module which appeared in that version of
Python. This module is a newer version derived from earlier work by
Andrew Kuchling, Richard Jones and Steve Traugott.
A test suite using unittest is included with the source distribution.
Simple usage:
>>> import gnupg
>>> gpg = gnupg.GPG(gnupghome='/path/to/keyring/directory')
>>> gpg.list_keys()
[{
...
'fingerprint': 'F819EE7705497D73E3CCEE65197D5DAC68F1AAB2',
'keyid': '197D5DAC68F1AAB2',
'length': '1024',
'type': 'pub',
'uids': ['', 'Gary Gross (A test user) ']},
{
...
'fingerprint': '37F24DD4B918CC264D4F31D60C5FEFA7A921FC4A',
'keyid': '0C5FEFA7A921FC4A',
'length': '1024',
...
'uids': ['', 'Danny Davis (A test user) ']}]
>>> encrypted = gpg.encrypt("Hello, world!", ['0C5FEFA7A921FC4A'])
>>> str(encrypted)
'-BEGIN PGP MESSAGE-\nVersion: GnuPG v1.4.9 (GNU/Linux)\n
\nhQIOA/6NHMDTXUwcEAf
.
-END PGP MESSAGE-\n'
>>> decrypted = gpg.decrypt(str(encrypted), passphrase='secret')
>>> str(decrypted)
'Hello, world!'
>>> signed = gpg.sign("Goodbye, world!", passphrase='secret')
>>> verified = gpg.verify(str(signed))
>>> print "Verified" if verified else "Not verified"
'Verified'
As always, your feedback is most welcome (especially bug reports [3],
patches and suggestions for improvement, or any other points via the
mailing list/discussion group [4]).
Enjoy!
Cheers
Vinay Sajip
Red Dove Consultants Ltd.
[1] https://bitbucket.org/vinay.sajip/python-gnupg
[2] https://pypi.python.org/pypi/python-gnupg/0.3.8
[3] https://bitbucket.org/vinay.sajip/python-gnupg/issues
[4] https://groups.google.com/forum/#!forum/python-gnupg
--
https://mail.python.org/mailman/listinfo/python-list
ANN: distlib 0.2.1 released on PyPI
I've just released version 0.2.1 of distlib on PyPI [1]. For newcomers, distlib is a library of packaging functionality which is intended to be usable as the basis for third-party packaging tools. The main changes in this release are as follows: Fixed issue #58: Return a Distribution instance or None from locate(). Fixed issue #59: Skipped special keys when looking for versions. Improved behaviour of PyPIJSONLocator to be analogous to that of other locators. Added resource iterator functionality. Fixed issue #71: Updated launchers to decode shebangs using UTF-8. This allows non-ASCII pathnames to be correctly handled. Ensured that the executable written to shebangs is normcased. Changed ScriptMaker to work better under Jython. Changed the mode setting method to work better under Jython. Changed get_executable() to return a normcased value. Handled multiple-architecture wheel filenames correctly. A more detailed change log is available at [2]. Please try it out, and if you find any problems or have any suggestions for improvements, please give some feedback using the issue tracker! [3] Regards, Vinay Sajip [1] https://pypi.python.org/pypi/distlib/0.2.1 [2] https://goo.gl/K5Spsp [3] https://bitbucket.org/pypa/distlib/issues/new -- https://mail.python.org/mailman/listinfo/python-list
ANN: Version 0.1.4 of sarge (a subprocess wrapper library) has been released.
Version 0.1.4 of Sarge, a cross-platform library which wraps the subprocess
module in the standard library, has been released.
What changed?
-
- Fixed issue #20: Now runs a pipeline in a separate thread if async.
- Fixed issue #21: The command line isn't parsed if shell=True is specified.
- Added Coveralls to Travis configuration.
- Tests added to cover added functionality and reported issues.
- Numerous documentation updates.
What does Sarge do?
---
Sarge tries to make interfacing with external programs from your
Python applications easier than just using subprocess alone.
Sarge offers the following features:
* A simple way to run command lines which allows a rich subset of Bash-
style shell command syntax, but parsed and run by sarge so that you
can run on Windows without cygwin (subject to having those commands
available):
>>> from sarge import capture_stdout
>>> p = capture_stdout('echo foo | cat; echo bar')
>>> for line in p.stdout: print(repr(line))
...
'foo\n'
'bar\n'
* The ability to format shell commands with placeholders, such that
variables are quoted to prevent shell injection attacks.
* The ability to capture output streams without requiring you to
program your own threads. You just use a Capture object and then you
can read from it as and when you want.
* The ability to look for patterns in captured output and to interact
accordingly with the child process.
Advantages over subprocess
---
Sarge offers the following benefits compared to using subprocess:
* The API is very simple.
* It's easier to use command pipelines - using subprocess out of the
box often leads to deadlocks because pipe buffers get filled up.
* It would be nice to use Bash-style pipe syntax on Windows, but
Windows shells don't support some of the syntax which is useful, like
&&, ||, |& and so on. Sarge gives you that functionality on Windows,
without cygwin.
* Sometimes, subprocess.Popen.communicate() is not flexible enough for
one's needs - for example, when one needs to process output a line at
a time without buffering the entire output in memory.
* It's desirable to avoid shell injection problems by having the
ability to quote command arguments safely.
* subprocess allows you to let stderr be the same as stdout, but not
the other way around - and sometimes, you need to do that.
Python version and platform compatibility
-
Sarge is intended to be used on any Python version >= 2.6 and is
tested on Python versions 2.6, 2.7, 3.1, 3.2, 3.3 and 3.4 on Linux,
Windows, and Mac OS X (not all versions are tested on all platforms,
but sarge is expected to work correctly on all these versions on all
these platforms).
Finding out more
You can read the documentation at
http://sarge.readthedocs.org/
There's a lot more information, with examples, than I can put into
this post.
You can install Sarge using "pip install sarge" to try it out. The
project is hosted on BitBucket at
https://bitbucket.org/vinay.sajip/sarge/
And you can leave feedback on the issue tracker there.
I hope you find Sarge useful!
Regards,
Vinay Sajip
--
https://mail.python.org/mailman/listinfo/python-list
[ANN]: distlib 0.2.0 released on PyPI
I've just released version 0.2.0 of distlib on PyPI [1]. For newcomers, distlib is a library of packaging functionality which is intended to be usable as the basis for third-party packaging tools. The main changes in this release are as follows: Updated match_hostname to use the latest Python implementation. Updates to better support PEP 426 / PEP 440. You can now provide interpreter arguments in shebang lines written by distlib. Removed reference to __PYVENV_LAUNCHER__ (relevant to OS X only). A more detailed change log is available at [2]. Please try it out, and if you find any problems or have any suggestions for improvements, please give some feedback using the issue tracker! [3] Regards, Vinay Sajip [1] https://pypi.python.org/pypi/distlib/0.2.0 [2] http://pythonhosted.org/distlib/overview.html#change-log-for-distlib [3] https://bitbucket.org/pypa/distlib/issues/new -- https://mail.python.org/mailman/listinfo/python-list
ANN: A new version (0.3.7) of python-gnupg has been released.
A new version of the Python module which wraps GnuPG has been
released.
What Changed?
=
This is an enhancement and bug-fix release, but the bug-fixes
include some security improvements, so all users are encouraged
to upgrade. See the project website [1] for more information.
Brief summary:
* Added an 'output' keyword parameter to the 'sign' and
'sign_file' methods, to allow writing the signature to a file.
* Allowed specifying 'True' for the 'sign' keyword parameter,
which allows use of the default key for signing and avoids
having to specify a key id when it's desired to use the default.
* Used a uniform approach with subprocess on Windows and POSIX:
shell=True is not used on either.
* When signing/verifying, the status is updated to reflect any
expired or revoked keys or signatures.
* Handled 'NOTATION_NAME' and 'NOTATION_DATA' during verification.
* Fixed #1, #16, #18, #20: Quoting approach changed, since now
shell=False.
* Fixed #14: Handled 'NEED_PASSPHRASE_PIN' message.
* Fixed #8: Added a scan_keys method to allow scanning of keys
without the need to import into a keyring.
* Fixed #5: Added '0x' prefix when searching for keys.
* Fixed #4: Handled 'PROGRESS' message during encryption.
* Fixed #3: Changed default encoding to Latin-1.
* Fixed #2: Raised ValueError if no recipients were specified
for an asymmetric encryption request.
* Handled 'UNEXPECTED' message during verification.
* Replaced old range(len(X)) idiom with enumerate().
* Refactored ``ListKeys`` / ``SearchKeys`` classes to maximise
use of common functions.
* Fixed GC94: Added ``export-minimal`` and ``armor`` options
when exporting keys. This addition was inadvertently left out
of 0.3.6.
This release [2] has been signed with my code signing key:
Vinay Sajip (CODE SIGNING KEY)
Fingerprint: CA74 9061 914E AC13 8E66 EADB 9147 B477 339A 9B86
What Does It Do?
The gnupg module allows Python programs to make use of the
functionality provided by the Gnu Privacy Guard (abbreviated GPG or
GnuPG). Using this module, Python programs can encrypt and decrypt
data, digitally sign documents and verify digital signatures, manage
(generate, list and delete) encryption keys, using proven Public Key
Infrastructure (PKI) encryption technology based on OpenPGP.
This module is expected to be used with Python versions >= 2.4, as it
makes use of the subprocess module which appeared in that version of
Python. This module is a newer version derived from earlier work by
Andrew Kuchling, Richard Jones and Steve Traugott.
A test suite using unittest is included with the source distribution.
Simple usage:
>>> import gnupg
>>> gpg = gnupg.GPG(gnupghome='/path/to/keyring/directory')
>>> gpg.list_keys()
[{
...
'fingerprint': 'F819EE7705497D73E3CCEE65197D5DAC68F1AAB2',
'keyid': '197D5DAC68F1AAB2',
'length': '1024',
'type': 'pub',
'uids': ['', 'Gary Gross (A test user) ']},
{
...
'fingerprint': '37F24DD4B918CC264D4F31D60C5FEFA7A921FC4A',
'keyid': '0C5FEFA7A921FC4A',
'length': '1024',
...
'uids': ['', 'Danny Davis (A test user) ']}]
>>> encrypted = gpg.encrypt("Hello, world!", ['0C5FEFA7A921FC4A'])
>>> str(encrypted)
'-BEGIN PGP MESSAGE-\nVersion: GnuPG v1.4.9 (GNU/Linux)\n
\nhQIOA/6NHMDTXUwcEAf
...
-END PGP MESSAGE-\n'
>>> decrypted = gpg.decrypt(str(encrypted), passphrase='secret')
>>> str(decrypted)
'Hello, world!'
>>> signed = gpg.sign("Goodbye, world!", passphrase='secret')
>>> verified = gpg.verify(str(signed))
>>> print "Verified" if verified else "Not verified"
'Verified'
As always, your feedback is most welcome (especially bug reports [3],
patches and suggestions for improvement, or any other points via the
mailing list/discussion group [4]).
Enjoy!
Cheers
Vinay Sajip
Red Dove Consultants Ltd.
[1] https://bitbucket.org/vinay.sajip/python-gnupg
[2] https://pypi.python.org/pypi/python-gnupg/0.3.7
[3] https://bitbucket.org/vinay.sajip/python-gnupg/issues
[4] https://groups.google.com/forum/#!forum/python-gnupg
--
https://mail.python.org/mailman/listinfo/python-list
[ANN]: distlib 0.1.9 released on PyPI
I've just released version 0.1.9 of distlib on PyPI [1]. For newcomers, distlib is a library of packaging functionality which is intended to be usable as the basis for third-party packaging tools. The main changes in this release are as follows: Fixed issue #47: Updated binary launchers to fix double-quoting bug where script executable paths have spaces. Added ``keystore`` keyword argument to signing and verification APIs. A more detailed change log is available at [2]. Please try it out, and if you find any problems or have any suggestions for improvements, please give some feedback using the issue tracker! [3] Regards, Vinay Sajip [1] https://pypi.python.org/pypi/distlib/0.1.9 [2] http://pythonhosted.org/distlib/overview.html#change-log-for-distlib [3] https://bitbucket.org/pypa/distlib/issues/new -- https://mail.python.org/mailman/listinfo/python-list
[ANN]: distlib 0.1.8 released on PyPI
I've released version 0.1.8 of distlib on PyPI [1]. For newcomers, distlib is a library of packaging functionality which is intended to be usable as the basis for third-party packaging tools. The main changes in this release are as follows: * Fixed issue #45: Improved thread-safety in SimpleScrapingLocator. * Fixed issue #42: Handling of pre-release legacy version numbers now mirrors setuptools logic. * Added exists, verify, update, is_compatible and is_mountable methods to the Wheel class (the update method fixed issue #41). * Added a search method to the PackageIndex class. * Fixed a bug in the Metadata.add_requirements method. * Allowed versions with a single numeric component and a local version component (tracking changes to PEP 440).* Corrected spelling of environment variable used for the stub launcher on OS X. * Avoided using pydist.json in 1.0 wheels (bdist_wheel writes a non- conforming pydist.json). * Improved computation of ABI tags on Python versions where SOABI is not available, and improved computation of compatibility tags on OS X to allow for multiple architectures and older OS X versions. A more detailed change log is available at [2]. Please try it out, and if you find any problems or have any suggestions for improvements, please give some feedback using the issue tracker! [3] Regards, Vinay Sajip [1] https://pypi.python.org/pypi/distlib/0.1.8 [2] http://pythonhosted.org/distlib/overview.html#change-log-for-distlib [3] https://bitbucket.org/pypa/distlib/issues/new -- https://mail.python.org/mailman/listinfo/python-list
ANN: A new version (0.3.6) of python-gnupg has been released.
A new version of the Python module which wraps GnuPG has been
released.
What Changed?
=
This is an enhancement and bug-fix release, but the bug-fixes
include some security improvements, so all users are encouraged
to upgrade. See the project website ( http://code.google.com/p/python-gnupg/ )
for more information. Summary:
Enabled fast random tests on gpg as well as gpg2.
Avoided deleting temporary file to preserve its permissions.
Avoided writing passphrase to log.
Added export-minimal and armor options when exporting keys.
Added verify_data() method to allow verification of signatures in memory.
Regularised end-of-line characters in ths source code.
Rectified problems with earlier fix for shell injection.
The current version passes all tests on Windows (CPython 2.4, 2.5,
2.6, 3.1, 2.7 and Jython 2.5.1) and Ubuntu (CPython 2.4, 2.5, 2.6,
2.7, 3.0, 3.1, 3.2). On Windows, GnuPG 1.4.11 has been used for the
tests.
What Does It Do?
The gnupg module allows Python programs to make use of the
functionality provided by the Gnu Privacy Guard (abbreviated GPG or
GnuPG). Using this module, Python programs can encrypt and decrypt
data, digitally sign documents and verify digital signatures, manage
(generate, list and delete) encryption keys, using proven Public Key
Infrastructure (PKI) encryption technology based on OpenPGP.
This module is expected to be used with Python versions >= 2.4, as it
makes use of the subprocess module which appeared in that version of
Python. This module is a newer version derived from earlier work by
Andrew Kuchling, Richard Jones and Steve Traugott.
A test suite using unittest is included with the source distribution.
Simple usage:
>>> import gnupg
>>> gpg = gnupg.GPG(gnupghome='/path/to/keyring/directory')
>>> gpg.list_keys()
[{
...
'fingerprint': 'F819EE7705497D73E3CCEE65197D5DAC68F1AAB2',
'keyid': '197D5DAC68F1AAB2',
'length': '1024',
'type': 'pub',
'uids': ['', 'Gary Gross (A test user) ']},
{
...
'fingerprint': '37F24DD4B918CC264D4F31D60C5FEFA7A921FC4A',
'keyid': '0C5FEFA7A921FC4A',
'length': '1024',
...
'uids': ['', 'Danny Davis (A test user) ']}]
>>> encrypted = gpg.encrypt("Hello, world!", ['0C5FEFA7A921FC4A'])
>>> str(encrypted)
'-BEGIN PGP MESSAGE-\nVersion: GnuPG v1.4.9 (GNU/Linux)\n
\nhQIOA/6NHMDTXUwcEAf
...
-END PGP MESSAGE-\n'
>>> decrypted = gpg.decrypt(str(encrypted), passphrase='secret')
>>> str(decrypted)
'Hello, world!'
>>> signed = gpg.sign("Goodbye, world!", passphrase='secret')
>>> verified = gpg.verify(str(signed))
>>> print "Verified" if verified else "Not verified"
'Verified'
For more information, visit http://code.google.com/p/python-gnupg/ -
as always, your feedback is most welcome (especially bug reports,
patches and suggestions for improvement). Enjoy!
Cheers
Vinay Sajip
Red Dove Consultants Ltd.
--
https://mail.python.org/mailman/listinfo/python-list
ANN: Version 0.1.2 of sarge (a subprocess wrapper library) has been released.
Version 0.1.2 of Sarge, a cross-platform library which wraps the subprocess
module in the standard library, has been released.
What changed?
-
- Fixed issue #12: Prevented a hang which occurred when a redirection failed.
- Fixed issue #11: Added "+" to the characters allowed in parameters.
- Fixed issue #10: Removed a spurious debugger breakpoint.
- Fixed issue #9: Relative pathnames in redirections are now relative to the
current working directory for the redirected process.
- Added the ability to pass objects with "fileno()" methods as values
to the "input" argument of "run()", and a "Feeder" class which
facilitates passing data to child processes dynamically over time (rather
than just an initial string, byte-string or file).
- Added functionality under Windows to use PATH, PATHEXT and the
registry to find appropriate commands. This can e.g. convert a
command 'foo bar', if 'foo.py' is a Python script in the
c:\Tools directory which is on the path, to the equivalent
'c:\Python26\Python.exe c:\Tools\foo.py bar'. This is done internally
when a command is parsed, before it is passed to subprocess.
- Fixed issue #7: Corrected handling of whitespace and redirections.
- Fixed issue #8: Added a missing import.
- Added Travis integration.
- Added encoding parameter to the "Capture" initializer.
- Fixed issue #6: addressed bugs in Capture logic so that iterating over
captures is closer to subprocess behaviour.
- Tests added to cover added functionality and reported issues.
- Numerous documentation updates.
What does Sarge do?
---
Sarge tries to make interfacing with external programs from your
Python applications easier than just using subprocess alone.
Sarge offers the following features:
* A simple way to run command lines which allows a rich subset of Bash-
style shell command syntax, but parsed and run by sarge so that you
can run on Windows without cygwin (subject to having those commands
available):
>>> from sarge import capture_stdout
>>> p = capture_stdout('echo foo | cat; echo bar')
>>> for line in p.stdout: print(repr(line))
...
'foo\n'
'bar\n'
* The ability to format shell commands with placeholders, such that
variables are quoted to prevent shell injection attacks.
* The ability to capture output streams without requiring you to
program your own threads. You just use a Capture object and then you
can read from it as and when you want.
* The ability to look for patterns in captured output and to interact
accordingly with the child process.
Advantages over subprocess
---
Sarge offers the following benefits compared to using subprocess:
* The API is very simple.
* It's easier to use command pipelines - using subprocess out of the
box often leads to deadlocks because pipe buffers get filled up.
* It would be nice to use Bash-style pipe syntax on Windows, but
Windows shells don't support some of the syntax which is useful, like
&&, ||, |& and so on. Sarge gives you that functionality on Windows,
without cygwin.
* Sometimes, subprocess.Popen.communicate() is not flexible enough for
one's needs - for example, when one needs to process output a line at
a time without buffering the entire output in memory.
* It's desirable to avoid shell injection problems by having the
ability to quote command arguments safely.
* subprocess allows you to let stderr be the same as stdout, but not
the other way around - and sometimes, you need to do that.
Python version and platform compatibility
-
Sarge is intended to be used on any Python version >= 2.6 and is
tested on Python versions 2.6, 2.7, 3.1, 3.2 and 3.3 on Linux,
Windows, and Mac OS X (not all versions are tested on all platforms,
but sarge is expected to work correctly on all these versions on all
these platforms).
Finding out more
You can read the documentation at
http://sarge.readthedocs.org/
There's a lot more information, with examples, than I can put into
this post.
You can install Sarge using "pip install sarge" to try it out. The
project is hosted on BitBucket at
https://bitbucket.org/vinay.sajip/sarge/
And you can leave feedback on the issue tracker there.
I hope you find Sarge useful!
Regards,
Vinay Sajip
--
https://mail.python.org/mailman/listinfo/python-list
ANN: A new version (0.3.5) of python-gnupg has been released.
A new version of the Python module which wraps GnuPG has been
released.
What Changed?
=
This is a minor enhancement and bug-fix release. See the project
website ( http://code.google.com/p/python-gnupg/ ) for more
information. Summary:
Added improved shell quoting to guard against shell injection attacks.
Added search_keys() and send_keys() methods to interact with keyservers.
A symmetric cipher algorithm can now be specified when encrypting.
UTF-8 encoding is used as a fall back when no other encoding can be determined.
The key length now defaults to 2048 bits.
A default Name-Comment field is no longer provided during key generation.
What Does It Do?
The gnupg module allows Python programs to make use of the
functionality provided by the Gnu Privacy Guard (abbreviated GPG or
GnuPG). Using this module, Python programs can encrypt and decrypt
data, digitally sign documents and verify digital signatures, manage
(generate, list and delete) encryption keys, using proven Public Key
Infrastructure (PKI) encryption technology based on OpenPGP.
This module is expected to be used with Python versions >= 2.4, as it
makes use of the subprocess module which appeared in that version of
Python. This module is a newer version derived from earlier work by
Andrew Kuchling, Richard Jones and Steve Traugott.
A test suite using unittest is included with the source distribution.
Simple usage:
>>> import gnupg
>>> gpg = gnupg.GPG(gnupghome='/path/to/keyring/directory')
>>> gpg.list_keys()
[{
...
'fingerprint': 'F819EE7705497D73E3CCEE65197D5DAC68F1AAB2',
'keyid': '197D5DAC68F1AAB2',
'length': '1024',
'type': 'pub',
'uids': ['', 'Gary Gross (A test user) ']},
{
...
'fingerprint': '37F24DD4B918CC264D4F31D60C5FEFA7A921FC4A',
'keyid': '0C5FEFA7A921FC4A',
'length': '1024',
...
'uids': ['', 'Danny Davis (A test user) ']}]
>>> encrypted = gpg.encrypt("Hello, world!", ['0C5FEFA7A921FC4A'])
>>> str(encrypted)
'-BEGIN PGP MESSAGE-\nVersion: GnuPG v1.4.9 (GNU/Linux)\n
\nhQIOA/6NHMDTXUwcEAf
...
-END PGP MESSAGE-\n'
>>> decrypted = gpg.decrypt(str(encrypted), passphrase='secret')
>>> str(decrypted)
'Hello, world!'
>>> signed = gpg.sign("Goodbye, world!", passphrase='secret')
>>> verified = gpg.verify(str(signed))
>>> print "Verified" if verified else "Not verified"
'Verified'
For more information, visit http://code.google.com/p/python-gnupg/ -
as always, your feedback is most welcome (especially bug reports,
patches and suggestions for improvement). Enjoy!
Cheers
Vinay Sajip
Red Dove Consultants Ltd.
--
http://mail.python.org/mailman/listinfo/python-list
ANN: A new version (0.3.4) of the Python module which wraps GnuPG has been released.
A new version of the Python module which wraps GnuPG has been released.
What Changed?
=
This is a minor enhancement and bug-fix release. See the project website (
http://code.google.com/p/python-gnupg/ ) for more information. Summary:
An encoding bug which caused an exception when getting the GPG version has been
fixed.
Recipients can be passed in a set or frozenset as well as in a list or tuple.
The keyring argument now accepts a list of public keyring filenames as well as
a single filename.
A secret_keyring argument has been added which accepts either a single filename
or a list of filenames for secret keyrings.
The current version passes all tests on Windows (CPython 2.4, 2.5, 2.6, 2.7,
3.1 and Jython 2.5.1), Mac OS X (Python 2.5) and Ubuntu (CPython 2.4, 2.5, 2.6,
2.7, 3.0, 3.1, 3.2). On Windows, GnuPG 1.4.11 has been used for the tests.
What Does It Do?
The gnupg module allows Python programs to make use of the functionality
provided by the Gnu Privacy Guard (abbreviated GPG or GnuPG). Using this
module, Python programs can encrypt and decrypt data, digitally sign documents
and verify digital signatures, manage (generate, list and delete) encryption
keys, using proven Public Key Infrastructure (PKI) encryption technology based
on OpenPGP.
This module is expected to be used with Python versions >= 2.4, as it makes use
of the subprocess module which appeared in that version of Python. This module
is a newer version derived from earlier work by Andrew Kuchling, Richard Jones
and Steve Traugott.
A test suite using unittest is included with the source distribution.
Simple usage:
>>> import gnupg
>>> gpg = gnupg.GPG(gnupghome='/path/to/keyring/directory')
>>> gpg.list_keys()
[{
...
'fingerprint': 'F819EE7705497D73E3CCEE65197D5DAC68F1AAB2',
'keyid': '197D5DAC68F1AAB2',
'length': '1024',
'type': 'pub',
'uids': ['', 'Gary Gross (A test user) ']},
{
...
'fingerprint': '37F24DD4B918CC264D4F31D60C5FEFA7A921FC4A',
'keyid': '0C5FEFA7A921FC4A',
'length': '1024',
...
'uids': ['', 'Danny Davis (A test user) ']}]
>>> encrypted = gpg.encrypt("Hello, world!", ['0C5FEFA7A921FC4A'])
>>> str(encrypted)
'-BEGIN PGP MESSAGE-\nVersion: GnuPG v1.4.9
(GNU/Linux)\n\nhQIOA/6NHMDTXUwcEAf
...
-END PGP MESSAGE-\n'
>>> decrypted = gpg.decrypt(str(encrypted), passphrase='secret')
>>> str(decrypted)
'Hello, world!'
>>> signed = gpg.sign("Goodbye, world!", passphrase='secret')
>>> verified = gpg.verify(str(signed))
>>> print "Verified" if verified else "Not verified"
'Verified'
For more information, visit http://code.google.com/p/python-gnupg/ - as always,
your feedback is most welcome (especially bug reports, patches and suggestions
for improvement). Enjoy!
Cheers
Vinay Sajip
Red Dove Consultants Ltd.
--
http://mail.python.org/mailman/listinfo/python-list
ANN: Version 0.1.1 of sarge (a subprocess wrapper library) has been released.
Version 0.1.1 of Sarge, a cross-platform library which wraps the subprocess
module in the standard library, has been released.
What changed?
-
- Added the ability to scan for specific patterns in subprocess output streams.
- Added convenience methods to operate on wrapped subprocesses.
- Exceptions which occur while spawning subprocesses are now propagated.
- Fixed issues #2, #3, and #4.
- Improved shell_shlex resilience with Unicode on 2.x.
- Added get_stdout, get_stderr and get_both for when subprocess output is not
expected to be voluminous.
- Added an internal lock to serialise access to shared data.
- Added tests to cover added functionality and reported issues.
- Added numerous documentation updates.
What does Sarge do?
---
Sarge tries to make interfacing with external programs from your
Python applications easier than just using subprocess alone.
Sarge offers the following features:
* A simple way to run command lines which allows a rich subset of Bash-
style shell command syntax, but parsed and run by sarge so that you
can run on Windows without cygwin (subject to having those commands
available):
>>> from sarge import capture_stdout
>>> p = capture_stdout('echo foo | cat; echo bar')
>>> for line in p.stdout: print(repr(line))
...
'foo\n'
'bar\n'
* The ability to format shell commands with placeholders, such that
variables are quoted to prevent shell injection attacks.
* The ability to capture output streams without requiring you to
program your own threads. You just use a Capture object and then you
can read from it as and when you want.
Advantages over subprocess
---
Sarge offers the following benefits compared to using subprocess:
* The API is very simple.
* It's easier to use command pipelines - using subprocess out of the
box often leads to deadlocks because pipe buffers get filled up.
* It would be nice to use Bash-style pipe syntax on Windows, but
Windows shells don't support some of the syntax which is useful, like
&&, ||, |& and so on. Sarge gives you that functionality on Windows,
without cygwin.
* Sometimes, subprocess.Popen.communicate() is not flexible enough for
one's needs - for example, when one needs to process output a line at
a time without buffering the entire output in memory.
* It's desirable to avoid shell injection problems by having the
ability to quote command arguments safely.
* subprocess allows you to let stderr be the same as stdout, but not
the other way around - and sometimes, you need to do that.
Python version and platform compatibility
-
Sarge is intended to be used on any Python version >= 2.6 and is
tested on Python versions 2.6, 2.7, 3.1, 3.2 and 3.3 on Linux,
Windows, and Mac OS X (not all versions are tested on all platforms,
but sarge is expected to work correctly on all these versions on all
these platforms).
Finding out more
You can read the documentation at
http://sarge.readthedocs.org/
There's a lot more information, with examples, than I can put into
this post.
You can install Sarge using "pip install sarge" to try it out. The
project is hosted on BitBucket at
https://bitbucket.org/vinay.sajip/sarge/
And you can leave feedback on the issue tracker there.
I hope you find Sarge useful!
Regards,
Vinay Sajip
--
http://mail.python.org/mailman/listinfo/python-list
ANN: A new version (0.3.3) of the Python module which wraps GnuPG has been released.
A new version of the Python module which wraps GnuPG has been
released.
What Changed?
=
This is a minor enhancement and bug-fix release. See the project
website ( http://code.google.com/p/python-gnupg/ ) for more
information. Summary:
Improved support for status messages from GnuPG.
Fixed list_keys to handle control characters.
Enabled fast random for tests.
The current version passes all tests on Windows (CPython 2.4, 2.5,
2.6, 3.1, 2.7 and Jython 2.5.1) and Ubuntu (CPython 2.4, 2.5, 2.6,
2.7, 3.0, 3.1, 3.2). On Windows, GnuPG 1.4.11 has been used for the
tests.
What Does It Do?
The gnupg module allows Python programs to make use of the
functionality provided by the Gnu Privacy Guard (abbreviated GPG or
GnuPG). Using this module, Python programs can encrypt and decrypt
data, digitally sign documents and verify digital signatures, manage
(generate, list and delete) encryption keys, using proven Public Key
Infrastructure (PKI) encryption technology based on OpenPGP.
This module is expected to be used with Python versions >= 2.4, as it
makes use of the subprocess module which appeared in that version of
Python. This module is a newer version derived from earlier work by
Andrew Kuchling, Richard Jones and Steve Traugott.
A test suite using unittest is included with the source distribution.
Simple usage:
>>> import gnupg
>>> gpg = gnupg.GPG(gnupghome='/path/to/keyring/directory')
>>> gpg.list_keys()
[{
...
'fingerprint': 'F819EE7705497D73E3CCEE65197D5DAC68F1AAB2',
'keyid': '197D5DAC68F1AAB2',
'length': '1024',
'type': 'pub',
'uids': ['', 'Gary Gross (A test user) ']},
{
...
'fingerprint': '37F24DD4B918CC264D4F31D60C5FEFA7A921FC4A',
'keyid': '0C5FEFA7A921FC4A',
'length': '1024',
...
'uids': ['', 'Danny Davis (A test user) ']}]
>>> encrypted = gpg.encrypt("Hello, world!", ['0C5FEFA7A921FC4A'])
>>> str(encrypted)
'-BEGIN PGP MESSAGE-\nVersion: GnuPG v1.4.9 (GNU/Linux)\n
\nhQIOA/6NHMDTXUwcEAf
...
-END PGP MESSAGE-\n'
>>> decrypted = gpg.decrypt(str(encrypted), passphrase='secret')
>>> str(decrypted)
'Hello, world!'
>>> signed = gpg.sign("Goodbye, world!", passphrase='secret')
>>> verified = gpg.verify(str(signed))
>>> print "Verified" if verified else "Not verified"
'Verified'
For more information, visit http://code.google.com/p/python-gnupg/ -
as always, your feedback is most welcome (especially bug reports,
patches and suggestions for improvement). Enjoy!
Cheers
Vinay Sajip
Red Dove Consultants Ltd.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python launcher (PEP 397) and emacs python-mode.el
Thomas Heller ctypes.org> writes: > What I meant to write is this: > > when the shebang line in script.py contains this: >#!/usr/bin/python3.1-32 > then emacs SHOULD run >py.exe -3.1-32 script.py > and the launcher runs >c:\Python31\python.exe script.py IMO it would be better for emacs to just run py.exe script.py and py.exe can read the shebang and do the right thing. This saves the emacs code from having to duplicate the shebang line processing logic that py.exe uses (which, as we know, is unusual. So for a cross-platform you can have a shebang line of #!/usr/bin/python3.2, and on Windows it will still call the appropriate Python 3.2 even if it's not in /usr/bin, as there's no /usr/bin :-)) Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: A new script which creates Python 3.3 venvs with Distribute and pip installed in them
Ian Kelly gmail.com> writes: > > I have a shell script for this: > Sure - there's a similar one at https://gist.github.com/4591655 The main purpose of the script was to illustrate how to subclass venv.EnvBuilder, and I've added it as an example to the 3.3 and in-development documentation: http://docs.python.org/3/library/venv.html#an-example-of-extending-envbuilder Doing it in Python means that it runs cross-platform, offers a few benefits such as command line help, or the option to install Distribute but not pip. Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
A new script which creates Python 3.3 venvs with Distribute and pip installed in them
Python 3.3 includes a script, pyvenv, which is used to create virtual environments. However, Distribute and pip are not installed in such environments - because, though they are popular, they are third-party packages - not part of Python. The Python 3.3 venv machinery allows customisation of virtual environments fairly readily. To demonstrate how to do this, and to provide at the same time a script which might be useful to people, I've created a script, pyvenvex.py, at https://gist.github.com/4673395 which extends the pyvenv script to not only create virtual environments, but to also install Distribute and pip into them. The script needs Python 3.3, and one way to use it is: 1. Download the script to a directory in your path, and (on Posix platforms) make it executable. 2. Add a shebang line at the top of your script, pointing to your Python 3.3 interpreter (Posix, and also Windows if you have the PEP 397 launcher which is part of Python 3.3 on Windows). 3. Run the pyvenvex script to create your virtual environments, in place of pyvenv, when you want Distribute and pip to be installed for you (this is how virtualenv sets up environments it creates). You can run the script with -h to see the command line options available, which are a superset of the pyvenv script. Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
ANN: A new version (0.3.2) of the Python module which wraps GnuPG has been released.
A new version of the Python module which wraps GnuPG has been
released.
What Changed?
=
This is a minor enhancement and bug-fix release. See the project
website ( http://code.google.com/p/python-gnupg/ ) for more
information. Summary:
Improved support for status messages from GnuPG.
Fixed key generation to skip empty values.
Fixed list_keys to handle escaped characters.
Removed doctests which required interactive entry of passwords.
The current version passes all tests on Windows (CPython 2.4, 2.5,
2.6, 3.1, 2.7 and Jython 2.5.1) and Ubuntu (CPython 2.4, 2.5, 2.6,
2.7, 3.0, 3.1, 3.2). On Windows, GnuPG 1.4.11 has been used for the
tests. Tests also pass under CPython 2.5 and CPython 2.6 on OS X.
What Does It Do?
The gnupg module allows Python programs to make use of the
functionality provided by the Gnu Privacy Guard (abbreviated GPG or
GnuPG). Using this module, Python programs can encrypt and decrypt
data, digitally sign documents and verify digital signatures, manage
(generate, list and delete) encryption keys, using proven Public Key
Infrastructure (PKI) encryption technology based on OpenPGP.
This module is expected to be used with Python versions >= 2.4, as it
makes use of the subprocess module which appeared in that version of
Python. This module is a newer version derived from earlier work by
Andrew Kuchling, Richard Jones and Steve Traugott.
A test suite using unittest is included with the source distribution.
Simple usage:
>>> import gnupg
>>> gpg = gnupg.GPG(gnupghome='/path/to/keyring/directory')
>>> gpg.list_keys()
[{
...
'fingerprint': 'F819EE7705497D73E3CCEE65197D5DAC68F1AAB2',
'keyid': '197D5DAC68F1AAB2',
'length': '1024',
'type': 'pub',
'uids': ['', 'Gary Gross (A test user) ']},
{
...
'fingerprint': '37F24DD4B918CC264D4F31D60C5FEFA7A921FC4A',
'keyid': '0C5FEFA7A921FC4A',
'length': '1024',
...
'uids': ['', 'Danny Davis (A test user) ']}]
>>> encrypted = gpg.encrypt("Hello, world!", ['0C5FEFA7A921FC4A'])
>>> str(encrypted)
'-BEGIN PGP MESSAGE-\nVersion: GnuPG v1.4.9 (GNU/Linux)\n
\nhQIOA/6NHMDTXUwcEAf
...
-END PGP MESSAGE-\n'
>>> decrypted = gpg.decrypt(str(encrypted), passphrase='secret')
>>> str(decrypted)
'Hello, world!'
>>> signed = gpg.sign("Goodbye, world!", passphrase='secret')
>>> verified = gpg.verify(str(signed))
>>> print "Verified" if verified else "Not verified"
'Verified'
For more information, visit http://code.google.com/p/python-gnupg/ -
as always, your feedback is most welcome (especially bug reports,
patches and suggestions for improvement). Enjoy!
Cheers
Vinay Sajip
Red Dove Consultants Ltd.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Specifying two log files with one configuration file
Peter Steele gmail.com> writes: > I have been unable to get this to work. My current conf file looks like this: Try with the following changes: [logger_test] level: DEBUG handlers: test propagate: 0 qualname: test The qualname: test is what identifies the logger as the logger named 'test', and propagate: 0 prevents the test message from being passed up to the root logger. Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: Getting a TimedRotatingFileHandler not to put two dates in the same file?
David M Chess us.ibm.com> writes: > >> But now the users have noticed that if the process isn't up at > midnight, > >> they can end up with lines from two (or I guess potentially more) > dates in > >> the same log file. > >> > >> Is there some way to fix this, either with cleverer arguments > into the > >> TimedRotatingFileHandler, or by some plausible subclassing of > it or its > >> superclass? Well, of course you can subclass and override it to do what you want - there's no magic there. The behaviour is as you would expect: the default behaviour of a TimedRotatingFileHandler is to append, and roll over at midnight. So if your program isn't running at midnight, it won't rotate: Day 1. Run your program, stop it before midnight. The log file contains dates from this day. No rotation occurred. Day 2. Run your program again, stop it before midnight. The log file contains dates from this day, and Day 1. No rotation occurred. That's the symptom you're seeing, right? You could do as Dave Angel suggested - just use a FileHandler with the name derived from the date. If you don't want to or can't actually rotate files at midnight, you're using the wrong tool for the job :-) If you sometimes want to rotate at midnight (the process is running at that time) and at other times not (the process isn't running then), you might have to code startup logic in your program to deal with the vagaries of your environment, since only you would know what they are :-) Work is afoot to make the actual rollover time configurable (i.e. not forced to be literally midnight) - see http://bugs.python.org/issue9556 - but that's an enhancement request, not a bug, and so it'll see the light of day in Python 3.4, if at all. An implementation is in my sandbox repo at http://hg.python.org/sandbox/vsajip in branch fix9556. If all you need to do is rollover at a different time daily (say 7 a.m.), you might be able to use this. Feel free to use that code as inspiration for your subclass. Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Logging: Specifying converter attribute of a log formatter in config file
On Thursday, August 30, 2012 11:38:27 AM UTC+1, Radha Krishna Srimanthula wrote: > > Now, how do I specify the converter attribute (time.gmtime) in the above > section? Sadly, there is no way of doing this using the configuration file, other than having e.g. a class UTCFormatter(logging.Formatter): converter = time.gmtime and then using a UTCFormatter in the configuration. -- http://mail.python.org/mailman/listinfo/python-list
Re: Logging handler: No output
Florian Lindner xgm.de> writes: > But neither the FileHandler nor the StreamHandler produce any actual output. > The file is being created but stays empty. If I use a print output in the > while loop it works, so output is catched and the applications stdout in > working. But why the logger proclog catching nothing? > Paul Rubin's answer looks correct. In addition, you should note that every call to start_process will add a new handler to the logger (I can't tell if the logger could be the same on multiple calls, but it seems likely) and that may produce multiple messages. The rule of thumb is: most code should get loggers and log to them, but adding handlers, setting levels etc. should be only done in one place (typically invoked from a "if __name__ == '__main__'" clause. Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
ANN: A new version (0.3.1) of the Python module which wraps GnuPG has been released.
A new version of the Python module which wraps GnuPG has been
released.
What Changed?
=
This is a minor enhancement and bug-fix release. See the project
website ( http://code.google.com/p/python-gnupg/ ) for more
information. Summary:
Better support for status messages from GnuPG.
Support for additional arguments to be passed to GnuPG.
Bugs in tests which used Latin-1 encoded data have been fixed by
specifying that encoding.
On verification (including after decryption), the signer trust level
is returned in integer and text formats.
The current version passes all tests on Windows (CPython 2.4, 2.5,
2.6, 2.7, 3.1 and Jython 2.5.1), Mac OS X (Python 2.5) and Ubuntu
(CPython 2.4, 2.5, 2.6, 2.7, 3.0, 3.1, 3.2). On Windows, GnuPG 1.4.11
has been used for the tests.
What Does It Do?
The gnupg module allows Python programs to make use of the
functionality provided by the Gnu Privacy Guard (abbreviated GPG or
GnuPG). Using this module, Python programs can encrypt and decrypt
data, digitally sign documents and verify digital signatures, manage
(generate, list and delete) encryption keys, using proven Public Key
Infrastructure (PKI) encryption technology based on OpenPGP.
This module is expected to be used with Python versions >= 2.4, as it
makes use of the subprocess module which appeared in that version of
Python. This module is a newer version derived from earlier work by
Andrew Kuchling, Richard Jones and Steve Traugott.
A test suite using unittest is included with the source distribution.
Simple usage:
>>> import gnupg
>>> gpg = gnupg.GPG(gnupghome='/path/to/keyring/directory')
>>> gpg.list_keys()
[{
...
'fingerprint': 'F819EE7705497D73E3CCEE65197D5DAC68F1AAB2',
'keyid': '197D5DAC68F1AAB2',
'length': '1024',
'type': 'pub',
'uids': ['', 'Gary Gross (A test user) ']},
{
...
'fingerprint': '37F24DD4B918CC264D4F31D60C5FEFA7A921FC4A',
'keyid': '0C5FEFA7A921FC4A',
'length': '1024',
...
'uids': ['', 'Danny Davis (A test user) ']}]
>>> encrypted = gpg.encrypt("Hello, world!", ['0C5FEFA7A921FC4A'])
>>> str(encrypted)
'-BEGIN PGP MESSAGE-\nVersion: GnuPG v1.4.9 (GNU/Linux)\n
\nhQIOA/6NHMDTXUwcEAf
...
-END PGP MESSAGE-\n'
>>> decrypted = gpg.decrypt(str(encrypted), passphrase='secret')
>>> str(decrypted)
'Hello, world!'
>>> signed = gpg.sign("Goodbye, world!", passphrase='secret')
>>> verified = gpg.verify(str(signed))
>>> print "Verified" if verified else "Not verified"
'Verified'
For more information, visit http://code.google.com/p/python-gnupg/ -
as always, your feedback is most welcome (especially bug reports,
patches and suggestions for improvement). Enjoy!
Cheers
Vinay Sajip
Red Dove Consultants Ltd.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Wish: Allow all log Handlers to accept the level argument
Fayaz Yusuf Khan gmail.com> writes:
>
> ***TRIVIAL ISSUE***, but this has been irking me for a while now.
> The main logging.Handler class' __init__ accepts a level argument while none
> of its children do. The poor minions seem to be stuck with the setLevel
> method which considerably lengthens the code.
>
> In short:
> Let's do this:
> root.addHandler(FileHandler('debug.log', level=DEBUG)
> Instead of this:
> debug_file_handler = FileHandler('debug.log')
> debug_file_handler.setLevel(DEBUG)
> root.addHandler(debug_file_handler)
Levels on handlers are generally not needed (though of course they are sometimes
needed) - level filtering should be applied at the logger first, and at the
handler only when necessary. I don't especially want to encourage the pattern
you suggest, because it isn't needed much of the time.
The code above won't do any more or less than if you hadn't bothered to set the
level on the handler.
Don't forget, more complex configurations are effected even more simply using
dictConfig().
Regards,
Vinay Sajip
--
http://mail.python.org/mailman/listinfo/python-list
Re: Possible change to logging.handlers.SysLogHandler
Vinay Sajip yahoo.co.uk> writes: > I am thinking of removing the BOM insertion in 2.7 and 3.2 - although > it is a change in behaviour, the current behaviour does seem broken > with regard to RFC 5424 conformance. However, as some might disagree > with that assessment and view it as a backwards-incompatible behaviour > change, I thought I should post this to get some opinions about > whether this change is viewed as objectionable. As there have been no objections, the BOM insertion code has been removed from SysLogHandler; these changes will appear in Python 2.7.4, Python 3.2.4, Python 3.3 and later versions. If you do need a BOM inserted into the message sent to your syslog daemon, a cookbook recipe will shortly appear telling you how to do this. Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: Multiprocessing & Logging
Thibaut gmail.com> writes: > This is exactly what I wanted, it seems perfect. However I still have a > question, from what I understood, > I have to configure logging AFTER creating the process, to avoid > children process to inherits the logging config. > > Unless there is a way to "clean" logging configuration in children > processes, so they only have one handler : the QueueHandler. > > I looked at the logging code and it doesn't seems to have an easy way to > do this. The problem of configuring the logging > after the process creation is that... I can't log during process > creation. But if it's too complicated, I will just do this. > I've updated the 3.2 / 3.3 logging cookbook with an example of what I mean. There is a gist of the example script at https://gist.github.com/2331314/ and the cookbook example should show once the docs get built on docs.python.org. Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: Multiprocessing & Logging
Thibaut gmail.com> writes: > This is exactly what I wanted, it seems perfect. However I still have a > question, from what I understood, > I have to configure logging AFTER creating the process, to avoid > children process to inherits the logging config. > > Unless there is a way to "clean" logging configuration in children > processes, so they only have one handler : the QueueHandler. > > I looked at the logging code and it doesn't seems to have an easy way to > do this. The problem of configuring the logging > after the process creation is that... I can't log during process > creation. But if it's too complicated, I will just do this. You may be able to have a "clean" configuration: for example, dictConfig() allows the configuration dictionary to specify whether existing loggers are disabled. So the details depend on the details of your desired configuration. One more point: I suggested that you subclass QueueListener, but you don't actually need to do this. For example, you can do something like: class DelegatingHandler(object): def handle(self, record): logger = logging.getLogger(record.name) logger.handle(record) And then instantiate the QueueListener with an instance of DelegatingHandler. QueueListener doesn't need actual logging handlers, just something with a handle method which takes a record. Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: Multiprocessing & Logging
Thibaut gmail.com> writes: > Ok, I understand what happenned. In fact, configuring the logging before > forking works fine. Subprocess inherits the configuration, as I thought. > > The problem was that I didn't passed any handler to the QueueListener > constructor. The when the listener recieved an message, it wasn't handled. > > I'm not sure how the logging module works, but what handlers should I > pass the QueueListener constructor ? I mean, maybe that I would like > some messages (depending of the logger) to be logged to a file, while > some others message would just be printed to stdout. > > This doesn't seem to be doable with a QueueListener. Maybe I should > implement my own system, and pass a little more informations with the > record sent in the queue : the logger name for example. > > Then, in the main process I would do a logging.getLogger(loggername) and > log the record using this logger (by the way it was configured). > > What do you think ? > You probably need different logging configurations in different processes. In your multiprocessing application, nominate one of the processes as a logging listener. It should initialize a QueueListener subclass which you write. All other processes should just configure a QueueHandler, which uses the same queue as the QueueListener. All the processes with QueueHandlers just send their records to the queue. The process with the QueueListener picks these up and handles them by calling the QueueListener's handle() method. The default implementation of QueueListener.handle() is: def handle(self, record): record = self.prepare(record) for handler in self.handlers: handler.handle(record) where self.handlers is just the handlers you passed to the QueueListener constructor. However, if you want a very flexible configuration where different loggers have different handlers, this is easy to arrange. Just configure logging in the listener process however you want, and then, in your QueueListener subclass, do something like this: class MyQueueListener(logging.handlers.QueueListener): def handle(self, record): record = self.prepare(record) logger = logging.getLogger(record.name) logger.handle(record) This will pass the events to whatever handlers are configured for a particular logger. I will try to update the Cookbook in the logging docs with this approach, and a working script. Background information is available here: [1][2] Regards, Vinay Sajip [1] http://plumberjack.blogspot.co.uk/2010/09/using-logging-with-multiprocessing.html [2] http://plumberjack.blogspot.co.uk/2010/09/improved-queuehandler-queuelistener.html -- http://mail.python.org/mailman/listinfo/python-list
Possible change to logging.handlers.SysLogHandler
There is a problem with the way logging.handlers.SysLogHandler works when presented with Unicode messages. According to RFC 5424, Unicode is supposed to be sent encoded as UTF-8 and preceded by a BOM. However, the current handler implementation puts the BOM at the start of the formatted message, and this is wrong in scenarios where you want to put some additional structured data in front of the unstructured message part; the BOM is supposed to go after the structured part (which, therefore, has to be ASCII) and before the unstructured part. In that scenario, the handler's current behaviour does not strictly conform to RFC 5424. The issue is described in [1]. The BOM was originally added / position changed in response to [2] and [3]. It is not possible to achieve conformance with the current implementation of the handler, unless you subclass the handler and override the whole emit() method. This is not ideal. For 3.3, I will refactor the implementation to expose a method which creates the byte string which is sent over the wire to the syslog daemon. This method can then be overridden for specific use cases where needed. However, for 2.7 and 3.2, removing the BOM insertion would bring the implementation into conformance to the RFC, though the entire message would have to be regarded as just a set of octets. A Unicode message would still be encoded using UTF-8, but the BOM would be left out. I am thinking of removing the BOM insertion in 2.7 and 3.2 - although it is a change in behaviour, the current behaviour does seem broken with regard to RFC 5424 conformance. However, as some might disagree with that assessment and view it as a backwards-incompatible behaviour change, I thought I should post this to get some opinions about whether this change is viewed as objectionable. Regards, Vinay Sajip [1] http://bugs.python.org/issue14452 [2] http://bugs.python.org/issue7077 [3] http://bugs.python.org/issue8795 -- http://mail.python.org/mailman/listinfo/python-list
Re: Reading Live Output from a Subprocess
On Apr 6, 7:57 am, buns...@gmail.com wrote:
> I've heard that the Pexpect module works wonders, but the problem is that
> relies on pty which is available in Unix only. Additionally, because I want
> this script to be usable by others, any solution should be in the standard
> library, which means I'd have to copy the Pexpect code into my script to use
> it.
>
> Is there any such solution in the Python 3 Standard Library, and if not, how
> much of a thorn is this?
>
> "There should be one-- and preferably only one --obvious way to do it."
> Unfortunately, this is one case where the above is true for Perl but not
> Python. Such an example in Perl is
>
> open(PROG, "command |") or die "Couldn't start prog!";
> while () {
> print "$_"; }
>
> (Note that I do not know Perl and do not have any intentions to learn it; the
> above comes from the script I was previously copying and extending, but I
> imagine (due to its simplicity) that it's a common Perl idiom. Note however,
> that the above does fail if the program re-prints output to the same line, as
> many long-running C programs do. Preferably this would also be caught in a
> Python solution.)
>
> If there is a general consensus that this is a problem for lots of people, I
> might consider writing a PEP.
>
> Of course, my highest priority is solving the blasted problem, which is
> holding up my script at the moment. (I can work around this by redirecting
> the program to a tmp file and reading that, but that would be such a perilous
> and ugly kludge that I would like to avoid it if at all possible.)
Try the sarge package [1], with documentation at [2] and source code
at [3]. It's intended for your use case, works with both Python 2.x
and 3.x, and is tested on Linux, OS X and Windows. Disclosure: I'm the
maintainer.
Regards,
Vinay Sajip
[1] http://pypi.python.org/pypi/sarge/0.1
[2] http://sarge.readthedocs.org/en/latest/
[3] https://bitbucket.org/vinay.sajip/sarge/
--
http://mail.python.org/mailman/listinfo/python-list
ANN: A new version (0.2.9) of the Python module which wraps GnuPG has been released.
A new version of the Python module which wraps GnuPG has been
released.
What Changed?
=
This is a minor bug-fix release. See the project website (
http://code.google.com/p/python-gnupg/ ) for more information.
Summary:
Better support for status messages from GnuPG.
A random data file used in testing is no longer shipped with the
source distribution, but created by the test suite if needed.
The current version passes all tests on Windows (CPython 2.4, 2.5,
2.6, 3.1, 2.7 and Jython 2.5.1) and Ubuntu (CPython 2.4, 2.5, 2.6,
2.7, 3.0, 3.1, 3.2). On Windows, GnuPG 1.4.11 has been used for the
tests.
What Does It Do?
The gnupg module allows Python programs to make use of the
functionality provided by the Gnu Privacy Guard (abbreviated GPG or
GnuPG). Using this module, Python programs can encrypt and decrypt
data, digitally sign documents and verify digital signatures, manage
(generate, list and delete) encryption keys, using proven Public Key
Infrastructure (PKI) encryption technology based on OpenPGP.
This module is expected to be used with Python versions >= 2.4, as it
makes use of the subprocess module which appeared in that version of
Python. This module is a newer version derived from earlier work by
Andrew Kuchling, Richard Jones and Steve Traugott.
A test suite using unittest is included with the source distribution.
Simple usage:
>>> import gnupg
>>> gpg = gnupg.GPG(gnupghome='/path/to/keyring/directory')
>>> gpg.list_keys()
[{
...
'fingerprint': 'F819EE7705497D73E3CCEE65197D5DAC68F1AAB2',
'keyid': '197D5DAC68F1AAB2',
'length': '1024',
'type': 'pub',
'uids': ['', 'Gary Gross (A test user) ']},
{
...
'fingerprint': '37F24DD4B918CC264D4F31D60C5FEFA7A921FC4A',
'keyid': '0C5FEFA7A921FC4A',
'length': '1024',
...
'uids': ['', 'Danny Davis (A test user) ']}]
>>> encrypted = gpg.encrypt("Hello, world!", ['0C5FEFA7A921FC4A'])
>>> str(encrypted)
'-BEGIN PGP MESSAGE-\nVersion: GnuPG v1.4.9 (GNU/Linux)\n
\nhQIOA/6NHMDTXUwcEAf
...
-END PGP MESSAGE-\n'
>>> decrypted = gpg.decrypt(str(encrypted), passphrase='secret')
>>> str(decrypted)
'Hello, world!'
>>> signed = gpg.sign("Goodbye, world!", passphrase='secret')
>>> verified = gpg.verify(str(signed))
>>> print "Verified" if verified else "Not verified"
'Verified'
For more information, visit http://code.google.com/p/python-gnupg/ -
as always, your feedback is most welcome (especially bug reports,
patches and suggestions for improvement). Enjoy!
Cheers
Vinay Sajip
Red Dove Consultants Ltd.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Help me with weird logging problem
On Mar 6, 4:09 pm, J wrote: > > Any idea what I'm doing wrong? Levels can be set on loggers as well as handlers, and you're only setting levels on the handlers. The default level on the root logger is WARNING. A logger checks its level first, and only if the event passes that test will it be passed to the handlers (which will also perform level tests). So, a logger.setLevel(logging.DEBUG) should be all you need to add before logging anything. Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: Error with co_filename when loading modules from zip file
On Mar 6, 2:40 am, Bob Rossi wrote: > Darn it, this was reported in 2007 > http://bugs.python.org/issue1180193 > and it was mentioned the logging package was effected. > > Yikes. > I will think about this, but don't expect any quick resolution :-( I think the right fix would be not in the logging package, but in the module loading machinery (as mentioned on that issue). I wouldn't worry about the performance aspect - once the logging package is loaded, there's no performance impact. That's a tiny one- off hit which you will probably not notice at all. Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: Error with co_filename when loading modules from zip file
On Mar 5, 8:36 pm, Bob wrote: > The logging package gets the filename and line number > of the calling function by looking at two variables, the filename > of the frame in the stack trace and the variable logging._srcfile. > The comparison is done in logging/__init__.py:findCaller. > The _srcfile is computed in logging/__init__.py - can you see which of the paths it takes when computing _srcfile? > I've tried putting only the pyc files, only the py files > and both in the zip file. I think the filename info might be stored in the .pyc from when you ran it outside the .zip. If you delete all .pyc files and only have .py in the .zip, what happens? Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
PEP 414 has been accepted
PEP 414 has been accepted: http://mail.python.org/pipermail/python-dev/2012-February/116995.html This means that from Python 3.3 onwards, you can specify u'xxx' for Unicode as well as just 'xxx'. The u'xxx' form is not valid syntax in Python 3.2, 3.1 or 3.0. The idea is to make porting code from 2.x to 3.x easier than before. Get porting! Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: Should I acquire lock for logging.Handler.flush()?
On Feb 23, 5:55 pm, Fayaz Yusuf Khan wrote: > buffer. Shouldn't the access be synchronized? I've now updated the repos for 2.7, 3.2 and default to add locking for flush/close operations. Thanks for the suggestion. Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: Should I acquire lock for logging.Handler.flush()?
On Feb 23, 5:55 pm, Fayaz Yusuf Khan wrote: > Well, as emit() is always being called from within a lock, I assumed that > flush() should/would also be handled similarly. Afterall, they are handling > the > same underlying output stream or in case of the BufferingHandler share the > same > buffer. Shouldn't the access be synchronized? Yes, you might well be right - though no problems have been reported, it's probably best to be safe. Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: Should I acquire lock for logging.Handler.flush()?
On Feb 22, 4:44 am, Fayaz Yusuf Khan wrote: > Anyway, I read the source and found many interesting things that ought to be > mentioned in the docs. > Such as flush() should be called from close() whenever it's implemented. > (FileHandler.close() is doing it) This is entirely handler-dependent - there's no absolute rule that you *have* to call flush() before close(). Some underlying will do flushing when you close. > And how come close()/flush() isn't being called from inside a lock? Why does it need to be? Are you aware of any race conditions or other concurrency problems which will occur with the code as it is now? > (Handler.close() calls the module level _acquireLock() and _releaseLock()s but > nothing about the instance level acquire() or release()) > Or is it being locked from somewhere else? The module level acquisitions are because module-level handler lists are changed in Handler.close(). If locking is required in a particular handler class for close or flush, that can be implemented by the developer of that handler class. AFAIK there is no such need for the handler classes in the stdlib - if you have reason to believe otherwise, please give some examples of potential problems and with example code if possible. Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: Should I acquire lock for logging.Handler.flush()?
On Feb 21, 7:23 am, Fayaz Yusuf Khan wrote: > I'm writing a custom logging Handler that sends emails through AWS Simple > Email Service using the boto library. > As there's a quota cap on how many (200) emails I can send within 24hrs, I > think I need to buffer my log messages from the emit() calls (Or is that a bad > idea?). > And I was reading the Handler documentation and was confused if I should call > the acquire() and release() methods from within a flush() call. > -- > Fayaz Yusuf Khan > Cloud developer and architect > Dexetra SS, Bangalore, India > fayaz.yusuf.khan_AT_gmail_DOT_com > fayaz_AT_dexetra_DOT_com > +91-9746-830-823 > > signature.asc > < 1KViewDownload If you are using SMTPHandler, calling flush() won't do anything. You'll probably need to subclass the handler to implement rate limiting. In the stdlib, only StreamHandler and its subclasses actually implement flush(), which flushes I/O buffers to disk. Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: HTTP logging
On Feb 20, 5:47 pm, Vinay Sajip wrote: > I will reinstate it in the reference API docs, but the answer to > Jason's problem is to either subclass HTTPHandler and override > handleError to suppress the error, or set logging.raiseExceptions to > True (in which case all logging exceptions will be swallowed - not Um, that should be *False*, not True. Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: HTTP logging
On Feb 20, 5:07 pm, Jean-Michel Pichavant wrote: > However, I looked into the code and find out an (undocumented ?) > attribute of the logging module : raiseException which value is set to 1 > by default (python 2.5.2 logging.__version__ < '0.5.0.2' > ). > > When set to 1, handlerError print the traceback. > > This has been probably fixed in recent version of the module since the > handleError doc does not reference raiseException anymore. Actually, I think it's a mistake in the docs - when they were reorganised a few months ago, the text referring to raiseExceptions was moved to the tutorial: http://docs.python.org/howto/logging.html#exceptions-raised-during-logging I will reinstate it in the reference API docs, but the answer to Jason's problem is to either subclass HTTPHandler and override handleError to suppress the error, or set logging.raiseExceptions to True (in which case all logging exceptions will be swallowed - not necessarily what he wants). Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: ANN: Sarge, a library wrapping the subprocess module, has been released.
On Feb 17, 1:49 pm, Jean-Michel Pichavant wrote: > I can't use it though, I'm still using a vintage 2.5 version :-/ That's a shame. I chose 2.6 as a baseline for this package, because I need it to work on Python 2.x and 3.x with the same code base and minimal work, and that meant supporting Unicode literals via "from __future__ import unicode_literals". I'm stuck on 2.5 with other projects, so I share your pain :-( Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: re module: Nothing to repeat, but no sre_constants.error: nothing to repeat ?
On Feb 14, 4:38 am, Devin Jeanpierre wrote: > Hey Pythonistas, > > Consider the regular expression "$*". Compilation fails with the > exception, "sre_constants.error: nothing to repeat". > > Consider the regular expression "(?=$)*". As far as I know it is > equivalent. It does not fail to compile. > > Why the inconsistency? What's going on here? > > -- Devin $ is a meta character for regular expressions. Use '\$*', which does compile. Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: ANN: Sarge, a library wrapping the subprocess module, has been released.
On Feb 13, 7:08 am, Anh Hai Trinh wrote:
> > Objection! Does the defense REALLY expect this court to believe that
> > he can testify as to how MOST members of the Python community would or
> > would not favor bash over Python? And IF they do in fact prefer bash,
> > is this display of haughty arrogance nothing more than a hastily
> > stuffed straw-man presented to protect his own ego?
>
> Double objection! Relevance. The point is that the OP created another
> language that is neither Python nor Bash.
>
Triple objection! I think Rick's point was only that he didn't think
you were expressing the views of "most" people, which sort of came
across in your post.
To say I've created "another language" is misleading - it's just a
subset of Bash syntax, so you can do things like "echo foo; echo bar",
use "&&", "||" etc. (I used the Bash man page as my guide when
designing the parser.)
As an experiment on Windows, in a virtualenv, with GnuWin32 installed
on the path:
(venv) C:\temp>python
ActivePython 2.6.6.17 (ActiveState Software Inc.) based on
Python 2.6.6 (r266:84292, Nov 24 2010, 09:16:51) [MSC v.1500 32 bit
(Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from extproc import sh
>>> sh('echo foo; echo bar')
Traceback (most recent call last):
File "", line 1, in
File "C:\temp\venv\lib\site-packages\extproc.py", line 412, in sh
f = Sh(cmd, fd=fd, e=e, cd=cd).capture(1).stdout
File "C:\temp\venv\lib\site-packages\extproc.py", line 202, in
capture
p = subprocess.Popen(self.cmd, cwd=self.cd, env=self.env,
stdin=self.fd[0],
stdout=self.fd[1], stderr=self.fd[2])
File "C:\Python26\Lib\subprocess.py", line 623, in __init__
errread, errwrite)
File "C:\Python26\Lib\subprocess.py", line 833, in _execute_child
startupinfo)
WindowsError: [Error 3] The system cannot find the path specified
>>> from sarge import capture_stdout
>>> capture_stdout('echo foo; echo bar').stdout.text
u'foo\r\nbar\r\n'
>>>
That's all from a single interactive session. So as you can see, my
use cases are a little different to yours, which in turn makes a
different approach reasonable.
> My respectful opinion is that the OP's approach is fundamentally flawed.
> There are many platform-specific issues when forking and threading are fused.
> My benign intent was to warn others about unsolved problems and
> scratching-your-head situations.
>
> Obviously, the OP can always choose to continue his direction at his own
> discretion.
I think you were right to bring up the forking+threading issue, but I
have addressed the points you made in this thread - please feel free
to respond to the points I made about the Linux Programming Blog
article. I've updated the sarge docs to point to that article, and
I've added a section on API stability to highlight the fact that the
library is in alpha status and that API changes may be needed based on
feedback.
I'm not being blasé about the issue - it's just that I don't want to
be too timid, either. Python does not proscribe using subprocess and
threads together, and the issues you mention could easily occur even
without the use of sarge. You might say that sarge makes it more
likely that the issues will surface - but it'll only do that if you
pass "a & b & c & d" to sarge, and not otherwise.
The other use of threads by sarge - to read output streams from child
processes - is no different from the stdlib usage of threads in
subprocess.Popen.communicate().
Possibly Rick was objecting to the tone of your comments, but I
generally disregard any tone that seems confrontational when the
benefit of the doubt can be given - on the Internet, you can never
take for granted, and have to make allowances for, the language style
of your interlocutor ... I think you meant well when you responded,
and I have taken your posts in that spirit.
Regards,
Vinay Sajip
--
http://mail.python.org/mailman/listinfo/python-list
Re: ANN: Sarge, a library wrapping the subprocess module, has been released.
On Feb 13, 3:57 am, Anh Hai Trinh wrote:
>
> I don't disagree with it. But the solution is really easy, just call 'sh' and
> pass it a string!
>
> >>> from extproc import sh
> >>> n = int(sh(“ls /etc –1 | wc –l”))
>
> No parser needed written!
>
> Yes there is a danger of argument parsing and globs and all that. But people
> are aware of it. With string parsing, ambiguity is always there. Even when
> you have a BNF grammar, people easily make mistakes.
You're missing a few points:
* The parser is *already* written, so there's no point worrying about
saving any effort.
* Your solution is to pass shell=True, which as you point out, can
lead to shell injection problems. To say "people are aware of it" is
glossing over it a bit - how come you don't say that when it comes to
locking+forking? ;-)
* I'm aiming to offer cross-platform functionality across Windows and
Posix. Your approach will require a lower common denominator, since
the Windows shell (cmd.exe) is not as flexible as Bash. For example -
no "echo foo; echo bar"; no "a && b" or "a || b" - these are not
supported by cmd.exe.
* Your comment about people making mistakes applies just as much if
someone passes a string with a Bash syntax error, to Bash, via your
sh() function. After all, Bash contains a parser, too. For instance:
>>> from extproc import sh
>>> sh('ls >>> abc')
/bin/sh: Syntax error: redirection unexpected
''
If you're saying there might be bugs in the parser, that's something
else - I'll address those as and when they turn up.
Regards,
Vinay Sajip
--
http://mail.python.org/mailman/listinfo/python-list
Re: ANN: Sarge, a library wrapping the subprocess module, has been released.
On Feb 12, 4:19 pm, Anh Hai Trinh wrote: > If you use threads and call fork(), you'll almost guaranteed to face with > deadlocks. Perhaps not in a particular piece of code, but some others. > Perhaps not on your laptop, but on the production machine with different > kernels. Like most race conditions, they will eventually show up. You can hit deadlocks in multi-threaded programs even without the fork(), can't you? In that situation, you either pin it down to a bug in your code (and even developers experienced in writing multi- threaded programs hit these), or a bug in the underlying library (which can hopefully be fixed, but that applies to any bug you might hit in any library you use, and is something you have to consider whenever you use a library written by someone else), or an unfixable problem (e.g. due to problems in the Python or C runtime) which require a different approach. I understand your concerns, but you are just a little further along the line from people who say "If you use threads, you will have deadlock problems. Don't use threads." I'm not knocking that POV - people need to use what they're comfortable with, and to avoid things that make them uncomfortable. I'm not pushing the async feature as a major advantage of the library - it's still useful without that, IMO. Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: ANN: Sarge, a library wrapping the subprocess module, has been released.
On Feb 12, 3:35 pm, Anh Hai Trinh wrote:
> I think most users like to use Python, or they'd use Bash. I think people
> prefer not another language that is different from both, and having little
> benefits. My own opinion of course.
>
I have looked at pbs and clom: they Pythonify calls to external
programs by making spawning those look like function calls. There's
nothing wrong with that, it's just a matter of taste. I find that e.g.
wc(ls("/etc", "-1"), "-l")
is not as readable as
call(“ls /etc –1 | wc –l”)
and the attempt to Pythonify doesn't buy you much, IMO. Of course, it
is a matter of taste - I understand that there are people who will
prefer the pbs/clom way of doing things.
> Re. threads &
> fork():http://www.linuxprogrammingblog.com/threads-and-fork-think-twice-befo...
>
> For a careful impl of fork-exec with threads,
> seehttp://golang.org/src/pkg/syscall/exec_unix.go
Thanks for the links. The first seems to me to be talking about the
dangers of locking and forking; if you don't use threads, you don't
need locks, so the discussion about locking only really applies in a
threading+forking scenario.
I agree that locking+forking can be problematic because the semantics
of what happens to the state of the locks and threads in the child
(for example, as mentioned in http://bugs.python.org/issue6721).
However, it's not clear that any problem occurs if the child just
execs a new program, overwriting the old - which is the case here. The
link you pointed to says that
"It seems that calling execve(2) to start another program is the only
sane reason you would like to call fork(2) in a multi-threaded
program."
which is what we're doing in this case. Even though it goes on to
mention the dangers inherent in inherited file handles, it also
mentions that these problems have been overcome in recent Linux
kernels, and the subprocess module does contain code to handle at
least some of these conditions (e.g. preexec_fn, close_fds keyword
arguments to subprocess.Popen).
Hopefully, if there are race conditions which emerge in the subprocess
code (as has happened in the past), they will be fixed (as has
happened in the past).
> Hmm, if the extra "envelop" is the async code with threads that may deadlock,
> I would say "thanks but no thanks" :p
That is of course your privilege. I would hardly expect you to drop
extproc in favour of sarge. But there might be people who need to
tread in these dangerous waters, and hopefully sarge will make things
easier for them. As I said earlier, one doesn't *need* to use
asynchronous calls.
I agree that I may have to review the design decisions I've made,
based on feedback based on people actually trying the async
functionality out. I don't feel that shying away from difficult
problems without even trying to solve them is the best way of moving
things forward. What are the outcomes?
* Maybe people won't even try the async functionality (in which case,
they won't hit problems)
* They'll hit problems and just give up on the library (I hope not -
if I ever have a problem with a library I want to use, I always try
and engage with the developers to find a workaround or fix)
* They'll report problems which, on investigation, will turn out to be
fixable bugs - well and good
* The reported bugs will be unfixable for some reason, in which case
I'll just have to deprecate that functionality.
Remember, this is version 0.1 of the library, not version 1.0. I
expect to do some API and functionality tweaks based on feedback and
bugs which show up.
> I do think that IO redirection is much nicer with extproc.
Again, a matter of taste. You feel that it's better to pass dicts
around in the public API where integer file handles map to other
handles or streams; I feel that using a Capture instance is less
fiddly for the user. Let a thousand flowers bloom, and all that.
I do thank you for the time you've taken to make these comments, and I
found the reading you pointed me to interesting. I will update the
sarge docs to point to the link on the Linux Programming blog, to make
sure people are informed of potential pitfalls.
Regards,
Vinay Sajip
--
http://mail.python.org/mailman/listinfo/python-list
Re: ANN: Sarge, a library wrapping the subprocess module, has been released.
On Feb 12, 9:41 am, Anh Hai Trinh wrote: > Having written something with similar purpose > (https://github.com/aht/extproc), here are my comments: > > * Having command parsed from a string is complicated. Why not just have an > OOP API to construct commands? It's not hard for the user, and less work e.g. when migrating from an existing Bash script. I may have put in the effort to use a recursive descent parser under the hood, but why should the user of the library care? It doesn't make their life harder. And it's not complicated, not even particularly complex - such parsers are commonplace. > * Using threads and fork()ing process does not play nice together unless > extreme care is taken. Disasters await. By that token, disasters await if you ever use threads, unless you know what you're doing (and sometimes even then). Sarge doesn't force the use of threads with forking - you can do everything synchronously if you want. The test suite does cover the particular case of thread +fork. Do you have specific caveats, or is it just a "there be dragons" sentiment? Sarge is still in alpha status; no doubt bugs will surface, but unless a real show-stopper occurs, there's not much to be gained by throwing up our hands. BTW extproc is nice, but I wanted to push the envelope a little :-) Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
ANN: Sarge, a library wrapping the subprocess module, has been released.
Sarge, a cross-platform library which wraps the subprocess module in
the standard library, has been released.
What does it do?
Sarge tries to make interfacing with external programs from your
Python applications easier than just using subprocess alone.
Sarge offers the following features:
* A simple way to run command lines which allows a rich subset of Bash-
style shell command syntax, but parsed and run by sarge so that you
can run on Windows without cygwin (subject to having those commands
available):
>>> from sarge import capture_stdout
>>> p = capture_stdout('echo foo | cat; echo bar')
>>> for line in p.stdout: print(repr(line))
...
'foo\n'
'bar\n'
* The ability to format shell commands with placeholders, such that
variables are quoted to prevent shell injection attacks.
* The ability to capture output streams without requiring you to
program your own threads. You just use a Capture object and then you
can read from it as and when you want.
Advantages over subprocess
---
Sarge offers the following benefits compared to using subprocess:
* The API is very simple.
* It's easier to use command pipelines - using subprocess out of the
box often leads to deadlocks because pipe buffers get filled up.
* It would be nice to use Bash-style pipe syntax on Windows, but
Windows shells don't support some of the syntax which is useful, like
&&, ||, |& and so on. Sarge gives you that functionality on Windows,
without cygwin.
* Sometimes, subprocess.Popen.communicate() is not flexible enough for
one's needs - for example, when one needs to process output a line at
a time without buffering the entire output in memory.
* It's desirable to avoid shell injection problems by having the
ability to quote command arguments safely.
* subprocess allows you to let stderr be the same as stdout, but not
the other way around - and sometimes, you need to do that.
Python version and platform compatibility
-
Sarge is intended to be used on any Python version >= 2.6 and is
tested on Python versions 2.6, 2.7, 3.1, 3.2 and 3.3 on Linux,
Windows, and Mac OS X (not all versions are tested on all platforms,
but sarge is expected to work correctly on all these versions on all
these platforms).
Finding out more
You can read the documentation at
http://sarge.readthedocs.org/
There's a lot more information, with examples, than I can put into
this post.
You can install Sarge using "pip install sarge" to try it out. The
project is hosted on BitBucket at
https://bitbucket.org/vinay.sajip/sarge/
And you can leave feedback on the issue tracker there.
I hope you find Sarge useful!
Regards,
Vinay Sajip
--
http://mail.python.org/mailman/listinfo/python-list
Re: PythonWin debugger holds onto global logging objects too long
On Jan 24, 2:52 pm, Rob Richardson wrote: > I use PythonWin to debug the Python scripts we write. Our scripts often use > the log2pyloggingpackage. When running the scripts inside the debugger, we > seem to get oneloggingobject for every time we run the script. The result is > that after running the script five times, the log file contains five copies > of every message. The only way I know to clean this up and get only a single > copy of each message is to close PythonWin and restart it. > > What do I have to do in my scripts to clean up theloggingobjects so that I > never get more than one copy of each message in my log files? > I don't know what log2py is - Google didn't show up anything that looked relevant. If you're talking about the logging package in the Python standard library, I may be able to help: but a simple script that I ran in PythonWin didn't show any problems, so you'll probably need to post a short script which demonstrates the problem when run in PythonWin. Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: python logging filter limitation, looks intentional?
On Jan 28, 10:51 am, Chris Withers wrote: > To be clear, I wasn't asking for a change to existing behaviour, I was > asking for the addition of an option that would allow thelogging > framework to behave as most people would expect when it comes to filters ;-) And the evidence for "most people" would be ... ? ;-) It hasn't been raised before, despite filters working the way they do since Python 2.3 ... And I wonder, would any of those people be willing to accept the performance impact of the change? I'm not sure how big the impact would be, but it does involve another hierarchy traversal and additional calls to the ancestor filters. Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: python logging filter limitation, looks intentional?
On Jan 19, 12:50 am, Terry Reedy wrote: > > >> I don't want people to have to code differently for Python 3.3 and for > >> older versions. > > This is not a general policy, else we would never add new features ;-) > Do you plan to keep logging feature-frozen forever, or just for another > release? (I actually think is a good idea to fix bugs, tests, and docs > first.) Logging isn't by any means feature-frozen - new features are being added all the time, including non-trivial additions made in 2.7 and 3.2, and some more changes planned for 3.3 and already checked in. My comment was about the specific change proposed, which if implemented could lead to existing working third party code failing after a Python version upgrade. The proposed change isn't a new feature, it's a request for an existing feature to work differently. Apart from the behavioural change, there is also a performance implication which needs to be considered. All in all, I'm not sure there is a lot of demand for this proposed change, and I have already suggested a workable approach to Chris which should meet his needs without any need for a behavioural change in the standard library. Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: python logging filter limitation, looks intentional?
> From: Chris Withers > How breaking code? Configuration, maybe, but I can't see anyone being upset > that filtering would begin working "the same as everything else". > This just feels like a bug... Well, it means that filters that don't get called now would get called - and that's a change in behaviour. It's not a bug, because it's like that by design. I understand that you don't agree with that particular design decision, but it's out there now. What you're asking for isn't unreasonable, but not achievable without a change in behaviour. I don't want people to have to code differently for Python 3.3 and for older versions. > Actually, the filter I want to work on (log_nerf) is precisely for 3rd party > libraries; if a library author thinks a message should be error but you think > it > should be debug, and vice versa, having a filter that alters the level is the > only way to sort things out without hacking the 3rd party library. I've also > seen this technique used successfully where the filter looks to a database > for > config and nerfs messages during a certain time period. Okay, then applying a filter on handlers is a way of achieving this. > Again, this doesn't help if the __name__-ish loggers are in third party > packages that you don't want to have to fork ;-) True, but as I said earlier, you can attach a filter to your handlers. It's rather unlikely that you would have more than half-a-dozen handlers (since handlers ~= potential audiences for log events), but even so, I can't believe it's especially onerous to set up filter configuration for those handlers - and even easier if you add a DelegatingHandler to the mix. In https://gist.github.com/1623702 I've shown a DelegatingHandler; it's quite short. > Both use cases are valid, and a DelegatingHandler just feels like a hack to > work > around a deficiency in the framework... Rather, the approach I've taken is not to assume I'll meet everyone's needs out of the box, but that the right pieces are in place for people to use logging in useful ways in scenarios that I haven't thought of - which shortcomings might well be termed "deficiencies" according to your point of view - by deriving and overriding classes, or by composing using filters, or setting module-level flags. There are numerous examples on e.g. Stack Overflow where a small amount of customisation (e.g. a custom handler like DelegatingHandler) has been suggested to meet a specific user requirement. I don't think of these as hacks - I occasionally have to subclass stdlib classes in other areas to get the behaviour I want, and that's part and parcel of having OO at your disposal. Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: python loggingL filter limitation, looks intentional?
On Jan 16, 11:21 pm, Vinay Sajip wrote: > > I thought from an earlier comment - "rather than just the root logger where > my handlers live" - that you had your handlers attached to the root logger, > in which case it wouldn't be onerous at all. In place of those individual > handlers attached to the root, you simply attach your DelegatingHandler to > the root logger, and attach the filter and the other handlers to that > DelegatingHandler instance. An example of using a delegating handler: https://gist.github.com/1623702 Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: python loggingL filter limitation, looks intentional?
> Why is this? There must be some rationale for this rather than what, for me > and > others I've talked to, would seem more natural, ie: a filter on the root > logger would get messages both logged to it and any messages propagated to it > from child loggers to process. Perhaps you're right, but this can't be changed without breaking existing code, AFAICT. This hasn't come up before, so I'm not sure to what extent this is just a matter of taste. >> However, if you find your usage pattern occurring very commonly in your > applications, it's easy enough to create a DelegatingHandler handler class > which passes its events on to any number of handlers you like. > But this is just extra work when the normal built-in > delegation-upwards-ending-in-root works exactly as I want. It's not in general useful to filter on message content, when one factors in use of third-party libraries. Of course, if you are in control in the whole software stack you're using, or in certain very specific scenarios, it may be more useful - but in general it's most useful to filter on level and using logger names. The __name__ for loggers is useful to locate the source of events to a module, but you can have child loggers under that to log specific types of events in that module, and use the verbosity of those sub-module loggers to control output. Of course I don't know the precise reason you need things to work this way, so my suggestions and comments might appear somewhat general. > It's also the opposite of what I'd need, I'd need to plug one > handler into all the specific loggers that get used (and bear in mind that > many > libraries log to __name__ from modules and sub-modules) this is a pretty > onerous > task :-( I thought from an earlier comment - "rather than just the root logger where my handlers live" - that you had your handlers attached to the root logger, in which case it wouldn't be onerous at all. In place of those individual handlers attached to the root, you simply attach your DelegatingHandler to the root logger, and attach the filter and the other handlers to that DelegatingHandler instance. Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: python loggingL filter limitation, looks intentional?
Hi Chris, Filtering is intended to be for cases where integer level-based filtering is insufficient; hence it applies to individual loggers and handlers, just as the integer levels do. You are right that you would need to add a filter to all of the loggers where you want it to apply, or to all of the handlers where you want it to apply. However, if you find your usage pattern occurring very commonly in your applications, it's easy enough to create a DelegatingHandler handler class which passes its events on to any number of handlers you like. You can then apply the filter you want to an instance of this handler, which you attach to your root logger. Regards, Vinay Sajip - Original Message - > From: Chris Withers > To: Vinay Sajip > Cc: Python List > Sent: Monday, 16 January 2012, 13:10 > Subject: python loggingL filter limitation, looks intentional? > > Hi Vinay, > > It looks like this was intentional, so why was it decided that filters would > only be passed messages logged to the logger they're attached to rather than > the all messages that end up getting passed to logger? > > For example, an app and the libraries it uses log to 'some.db.driver', > 'some.lib' and 'myapp'. > > The root logger has a number of handlers attached to it. > > I want to write a filter that adjusts, up or down, the level of certain > messages > logged based on a regex match of their message. > > If I understand correctly, I have to add my filter to all of the loggers > above, > rather than just the root logger where my handlers live? > > Or, alternatively, add the filter to all of the root handlers, thereby making > the configuration of those handlers and/or more dependent on each other. > > Why is that? > > cheers, > > Chris > > -- Simplistix - Content Management, Batch Processing & Python Consulting > - http://www.simplistix.co.uk > -- http://mail.python.org/mailman/listinfo/python-list
Re: Two questions about logging
On Jan 12, 2:19 am, Matthew Pounsett wrote: > First, I'd like to be able to permit users to do more typical log > rotation, based on their OS's log rotation handler, rather than > rotating logs from inside an application. This is usually handled by > signalling an application with a HUP, whereupon it closes and then re- > opens all of its logs, getting new file handles (and new inodes). I > don't see anything in the Handler methods (or anywhere else) that > would let me tell a logger object to refresh the file handles on a log > file. Is there some standard way to deal with this? There's the WatchedFileHandler, which checks to see if a file's device or inode has changed (which happens when the external rotator does rotation) and if so, closes and reopens the file. (This handler is for Unix/Linux/OS X - it doesn't work on Windows). See http://docs.python.org/library/logging.handlers.html#watchedfilehandler for more information. Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: logging issues
On Dec 13, 5:03 pm, Andrea Crotti wrote: > The set_verbosity seems to do its job, the second is also > called and the filters are added but I don't see the output > formatted as it should.. Generally speaking, you should only do logging configuration once, as Jean-Michel said - from your main() function or "if __name__ == '__main__'" suite. Until you show more detail about what you're getting and what you're expecting, it's hard to provide more help. Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: Peculiarity of '@' in logging.Formatter
On Nov 21, 11:17 pm, Charlie Martin wrote:
> This is what seems like an odd bug, but in code I'd
> thing often-enough used it must be the expected behavior
> and I just don't understand. Please, sirs/mesdames, is
> this a bug?
It may be a bug, but if so, it's in the syslog daemon rather than
logging. Please try again with FileHandlers in place of
SysLogHandlers, and confirm whether the anomaly still occurs.
I could reproduce the problem on a Suse 11.3 machine running Python
2.6, but it doesn't occur on Ubuntu for example:
Nov 23 12:33:09 eta-jaunty [slhtest3:026]:My log message:isn't it
special?
Nov 23 12:33:09 eta-jaunty [slhtest3@026]:My log message:isn't it
special?
I get the same (expected) result on Ubuntu for both Python 2.6 and
2.7.
I noticed that on Suse the syslog daemon is rsyslogd, whereas on
Ubuntu it's plain syslogd. Daemons do differ on how they parse
messages :-(
I ran the script with strace, and logging is writing correctly to the
socket:
socket(PF_FILE, SOCK_DGRAM, 0) = 3
connect(3, {sa_family=AF_FILE, path="/dev/log"}, 10) = 0
socket(PF_FILE, SOCK_DGRAM, 0) = 4
connect(4, {sa_family=AF_FILE, path="/dev/log"}, 10) = 0
gettimeofday({1322052499, 78501}, NULL) = 0
send(3, "<14>[slhtest:026]:My log message"..., 51, 0) = 51
send(4, "<14>[slhtest@026]:My log message"..., 51, 0) = 51
close(4)= 0
close(3)= 0
This is on Suse 11.3, where the formatting anomaly does occur - so it
appears to be down to how rsyslogd does things.
Regards,
Vinay Sajip
--
http://mail.python.org/mailman/listinfo/python-list
Re: Strange logging.Formatter behaviour
On Nov 22, 3:19 pm, Tiaburn Stedd wrote: > I don't believe this behavior is normal. I expect error raised in a > Formatter.format function, should be passed all the way up, but not > consumed. > > I found a workaround: > > class CustomFileHandler(logging.FileHandler, object): > def handleError(self, record): > super(CustomFileHandler, self).handleError(record) > raise > > But it looks ugly to add Custom*Handler for any default Handler (like, > StreamHandler, etc) to make them possible to fall out on any error. > > There are raiseException switch in source of logging/__init__.py > > 92 #raiseExceptions is used to see if exceptions during handling > should be > 93 #propagated > 94 # > 95 raiseExceptions = 1 > > But all it switch is only is traceback printed or not. I think this > behaviour is against Zen of Python: "Errors should never pass > silently." They don't pass silently - raiseExceptions defaults to 1, so tracebacks are printed by default. In production, it's not a good idea for a potentially long-running process like a server to be brought down because someone put a typo in a logging format string, so the recommendation is to set raiseExceptions to 0/False in such scenarios. Since you are developing a custom formatter, you can certainly catch any formatting exceptions in your format method and decide what you want to do with them. The default handleError behaviour of printing a traceback is reasonable, in my view. If you want something else, you can subclass the relevant handler class; just setting a printTracebacks flag to false isn't going to get useful behaviour specific to individual requirements. There's no real point in throwing a very specific exception from a formatter, as a handler isn't going to automatically know how to handle a SpecificFormatException in any meaningful way. Remember that in the general case, application developers don't always have control of what handlers are configured for an application (for example, this could be set by end-user or per-deployment configuration). Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: Got some problems when using logging Filter
On Nov 21, 7:15 am, sword wrote: > > Thank you! Maybe I should find out another way to manipulate the log, > like wrap the getLogger function and add the filter at the first > time :) If you are using Python 2.7, 3.2 or later, you can use dictionary- based configuration - it's fairly painless. http://docs.python.org/library/logging.config.html#logging.config.dictConfig If you are using an earlier version of Python, the logutils project includes the same dictionary-based configuration logic. http://code.google.com/p/logutils/ Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: Got some problems when using logging Filter
On Nov 17, 9:06 am, sword wrote:
> On Nov 16, 10:50 pm, Peter Otten <__pete...@web.de> wrote:
>
>
>
>
>
>
>
>
>
> > sword wrote:
> > > Thanks for your reply. I tried to edit the source a bit, now the
> > > main.py looks like this:
> > > #main.py
> > > importlogging
> > > fromloggingimport Filter
> > > import a
> > > import b
>
> > >logging.basicConfig(level=logging.DEBUG)
> > > root =logging.getLogger()
> > > root.addFilter(Filter("GoneWithTheWind")) #so I suppose no log msg
> > > would pass this filter
>
> > > logger =logging.getLogger("main")
> > > logger.debug("main process")
> > > a.print_log()
> > > b.print_log()
>
> > >
> > > And It still prints out all the log msg. :(
>
> > Here's a little demo to explore how filtering works:
>
> > $ cat demo.py
> > importlogging
> > class Filter(logging.Filter):
> > def filter(self, record):
> > print "applying filter", self.name
> > return True
>
> >logging.basicConfig()
>
> > loggers = [logging.getLogger(path) for path in ["", "a", "a.b"]]
> > for logger in loggers:
> > logger.addFilter(Filter("filter@" + logger.name))
>
> > [handler] =logging.getLogger().handlers
> > handler.addFilter(Filter("filter@handler"))
>
> > for logger in loggers:
> > logger.critical("whatever")
> > $ python demo.py
> > applying filter filter@root
> > applying filter filter@handler
> > CRITICAL:root:whatever
> > applying filter filter@a
> > applying filter filter@handler
> > CRITICAL:a:whatever
> > applying filter fil...@a.b
> > applying filter filter@handler
> > CRITICAL:a.b:whatever
> > $
>
> > As you can infer from the output only the filter(s) of the original logger
> > and of the handler(s) are applied.
>
> Thanks, so if I want to see my own log out of all logs produced by
> different module in the project, I should addFilter to each
> corresponding logger. I thought I could add Filter in root and filter
> out only the interested info from it before.
Or you can add a filter to the handler (but then you can't use
basicConfig() to configure it - you need to do it explicitly).
Regards,
Vinay Sajip
--
http://mail.python.org/mailman/listinfo/python-list
Re: python logging multiple processes to one file (via socket server)
On Oct 27, 11:09 am, Gelonida N wrote: > "The following section documents this approach in more detail and > includes a working socket receiver which can be used as a starting point > for you to adapt in your own applications." > > Somehow I have a mental block though and fail to see the 'following > section'. You're right, the link got lost in a reorganisation of the documentation. Working example is here: http://docs.python.org/howto/logging-cookbook.html#sending-and-receiving-logging-events-across-a-network Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: logging: warn() methods and function to be deprecated.
Mike C. Fletcher vrplumber.com> writes: > More: an undocumented entry point is not "deprecated" because, after > all, it shows up in PyDoc as a regular method. Deprecated methods also show up in PyDoc. Of course, if the deprecation is mentioned in the docstring, users would see this - but if it isn't, they wouldn't know until they got a DeprecationWarning. > auto-translation via 2to3 (because you generally are calling log.warn() > rather than logging.warning, but sometimes you are doing getattr( log, > log_level ) and then passing that method around a few times), and it That doesn't sound like a good usage pattern to me, especially as loggers have a log method which takes the logging level. There shouldn't be any need to pass a bound method around. Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: logging: warn() methods and function to be deprecated.
Mike C. Fletcher vrplumber.com> writes: > I actually consider .warning() a nit :) . After all, it's 3 extra > characters :) , and *who* actually reads documentation instead of just > poking around and finding the shortest-named method in the instance? Readability counts :-) Are you saying there's no need to bother documenting stuff ??? ;-) > Anyway, I personally don't see this as worth the breakage. What breakage are we really talking about? Remember, Python 2.x will not change in this area - the proposed change is for Python 3.3 and later only, and will not be backported :-) As far as I know, Trac doesn't work with Python 3 anyway. Most of the code out there (which Mark found via Google Code Search) is Python 2.x. When porting from 2 to 3.3, it's just one extra little thing to deal with - small compared with other issues which come up when doing such ports. Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: logging: warn() methods and function to be deprecated.
> I think that is a real shame - it seems to be gratuitous breakage for almost > zero benefit. That issue shows that Trac makes heavy use of .warn, I've use > .warn almost exclusively for many years, and code.google.com shows it is used > extensively in the wild. Okay, but it's easy enough to change, and people are getting a reasonable amount of time to deal with it. > Is there still a chance to reconsider? I'm not dogmatic about things like this; warn() is just a hangover from a long time ago and bit of a nit, that's all. I suppose I should have removed it when 3.0 was released, but it went under my radar at that time. Hence my post here, to get feedback from logging users about this proposed change. Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
logging: warn() methods and function to be deprecated.
In response to an issue (#13235) raised on the Python bug tracker, I'm going to deprecate the warn() methods in the Logger and LoggerAdapter classes in the stdlib logging package, as well the module-level warn() function. The warn() variants were synonyms for the warning() methods and function, and a holdover from before the time that logging was added to Python.They were not documented; it's probably time to retire them, so I've added a DeprecationWarning to appear in 3.3, and they'll be completely removed in 3.4 (along with the WARN synonym for WARNING). With this change, all the logging levels are adjectives which apply to the logged message: DEBUG, INFO, WARNING, ERROR and CRITICAL. I don't believe the WARN/warn variants were used much, if at all - but this is just a heads up for anyone who might have used them. Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: How add class name to format of the logging module?
On Oct 12, 9:42 pm, Peng Yu wrote: > Hi, > > The following attributes does not include the class name. Is there a > way to add class name to the format string? Thanks! Not in the general case without a reasonable run-time cost. Since you can find the exact line number a logging call was made from, you can always figure out the class. Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
Re: Advise on using logging.getLogger needed.
On Oct 2, 11:12 pm, "Steven W. Orr" wrote:
> I hope I don't sound like I'm ranting :-(
You don't, but neither is it absolutely clear what you're trying to
achieve. BTW the term "root logger" in the logging docs refers to a
logger internal to the logging package, and not to a logger that you
create. For that, it's best to refer to "top level logger for my
application/library".
Supposing you want your top-level logger always to be the name of your
main script, and you want other loggers to live below this logger. In
your main script module, therefore, you do the equivalent of
pname = compute_program_name(sys.argv[0])
logger = logging.getLogger(pname)
# configure logging handlers, filters etc. for your application
In your library modules, if you want them to live under that top-level
logger, then you can do something like
pname = compute_program_name(sys.argv[0])
logger = logging.getLogger('%s.%s' % pname, __name__)
where you can use something other than __name__ for the logger name
suffix, if you want.
> My problem is that I just want the modules that I write, to be able to get the
> named root logger and still be able to refer to possible sub-loggers.
According to the scheme I describe, if you have two scripts called
"foo" and "bar" which make use of a common module "baz", then events
in the foo module would appear under logger "foo", and events in the
baz module when run from foo would be logged under logger "foo.baz".
When running "bar", events in the bar module would appear under logger
"bar", whereas events in the baz module when run from bar would be
logged under logger "bar.baz". This seems to be what you're asking
for, but I may have misunderstood.
> I am running 2.6, so I don't have access to logging.getChild, but I'm not
> clear that I even want that.
getChild is just a convenience method, you don't need it - you can
just build the logger name as in my example above.
> My modules are used by multiple programs. Does this mean that I should simply
> use __name__ all the time? (That seems inelegant, no?)
Actually __name__ is very elegant, as it is impervious to you moving
your code around and follows the Python package hierarchy. Also, if
you are using third party code (especially from various sources),
using __name__ will make things unambiguous about where exactly events
are being logged from. Third party code typically won't know about
your top-level logger, and __name__ is the one scheme that everything
can agree on - the point about logger names being that they're
supposed to codify "where" in your application events occur, and the
package hierarchy is the canonical representation of source locations
in an application.
Regards,
Vinay Sajip
--
http://mail.python.org/mailman/listinfo/python-list
Re: Making `logging.basicConfig` log to *both* `sys.stderr` and `sys.stdout`?
On Aug 30, 9:53 am, Michel Albert wrote: > Unfortunately this setup makes `logging.basicConfig` pretty useless. > However, I believe that this is something that more people could > benefit from. I also believe, that it just "makes sense" to send > warnings (and above) to `stderr`, the rest to `stdout`. > > So I was thinking: "Why does `logging.basicConfig` not behave that > way". Because what seems entirely natural and obvious to you might not seem so for someone else. The API in the stdlib tries to provide baseline functionality which others can build on. For example, if you always have a particular pattern which you use, you can always write a utility function to set things up exactly how you like, and others who want to set things up differently (for whatever reason) can do the same thing, without having to come into conflict (if that's not too strong a word) with views different from their own. > Naturally, I was thinking of writing a patch against the python > codebase and submit it as a suggestion. But before doing so, I would > like to hear your thoughts on this. Does it make sense to you too or > am I on the wrong track? Are there any downsides I am missing? Python 2.x is closed to feature changes, and Python 2.7 and Python 3.2 already support flexible configuration using dictConfig() - see http://docs.python.org/library/logging.config.html#logging.config.dictConfig Also, Python 3.3 will support passing a list of handlers to basicConfig(): see http://plumberjack.blogspot.com/2011/04/added-functionality-for-basicconfig-in.html which will allow you to do what you want quite easily. Regards, Vinay Sajip -- http://mail.python.org/mailman/listinfo/python-list
