[Python-Dev] PEP 539 v3: A new C API for Thread-Local Storage in CPython
Hi folks,
I submit PEP 539 third draft for the finish. Thank you for all the advice
and the help!
Summary of technical changes:
* Don't use `bool` types in the implementation, per discussion at
python/cpython#1362 [1]
* Update for removal of --without-threads
Best regards,
Masayuki
[1]: https://github.com/python/cpython/pull/1362#discussion_r136357583
First round:
https://mail.python.org/pipermail/python-ideas/2016-December/043983.html
Second round:
https://mail.python.org/pipermail/python-dev/2017-August/149091.html
Discussion for the issue:
https://bugs.python.org/issue25658
HTML version for PEP 539 draft:
https://www.python.org/dev/peps/pep-0539/
Diff from first round to version 3:
https://gist.github.com/ma8ma/624f9e4435ebdb26230130b11ce12d20/revisions
And the pull-request for reference implementation (work in progress):
https://github.com/python/cpython/pull/1362
PEP: 539
Title: A New C-API for Thread-Local Storage in CPython
Version: $Revision$
Last-Modified: $Date$
Author: Erik M. Bray, Masayuki Yamamoto
BDFL-Delegate: Nick Coghlan
Status: Draft
Type: Informational
Content-Type: text/x-rst
Created: 20-Dec-2016
Post-History: 16-Dec-2016
Abstract
The proposal is to add a new Thread Local Storage (TLS) API to CPython which
would supersede use of the existing TLS API within the CPython interpreter,
while deprecating the existing API. The new API is named the "Thread
Specific Storage (TSS) API" (see `Rationale for Proposed Solution`_ for the
origin of the name).
Because the existing TLS API is only used internally (it is not mentioned in
the documentation, and the header that defines it, ``pythread.h``, is not
included in ``Python.h`` either directly or indirectly), this proposal
probably only affects CPython, but might also affect other interpreter
implementations (PyPy?) that implement parts of the CPython API.
This is motivated primarily by the fact that the old API uses ``int`` to
represent TLS keys across all platforms, which is neither POSIX-compliant,
nor portable in any practical sense [1]_.
.. note::
Throughout this document the acronym "TLS" refers to Thread Local
Storage and should not be confused with "Transportation Layer Security"
protocols.
Specification
=
The current API for TLS used inside the CPython interpreter consists of 6
functions::
PyAPI_FUNC(int) PyThread_create_key(void)
PyAPI_FUNC(void) PyThread_delete_key(int key)
PyAPI_FUNC(int) PyThread_set_key_value(int key, void *value)
PyAPI_FUNC(void *) PyThread_get_key_value(int key)
PyAPI_FUNC(void) PyThread_delete_key_value(int key)
PyAPI_FUNC(void) PyThread_ReInitTLS(void)
These would be superseded by a new set of analogous functions::
PyAPI_FUNC(int) PyThread_tss_create(Py_tss_t *key)
PyAPI_FUNC(void) PyThread_tss_delete(Py_tss_t *key)
PyAPI_FUNC(int) PyThread_tss_set(Py_tss_t *key, void *value)
PyAPI_FUNC(void *) PyThread_tss_get(Py_tss_t *key)
The specification also adds a few new features:
* A new type ``Py_tss_t``--an opaque type the definition of which may
depend on the underlying TLS implementation. It is defined::
typedef struct {
int _is_initialized;
NATIVE_TSS_KEY_T _key;
} Py_tss_t;
where ``NATIVE_TSS_KEY_T`` is a macro whose value depends on the
underlying native TLS implementation (e.g. ``pthread_key_t``).
* A constant default value for ``Py_tss_t`` variables,
``Py_tss_NEEDS_INIT``.
* Three new functions::
PyAPI_FUNC(Py_tss_t *) PyThread_tss_alloc(void)
PyAPI_FUNC(void) PyThread_tss_free(Py_tss_t *key)
PyAPI_FUNC(int) PyThread_tss_is_created(Py_tss_t *key)
The first two are needed for dynamic (de-)allocation of a ``Py_tss_t``,
particularly in extension modules built with ``Py_LIMITED_API``, where
static allocation of this type is not possible due to its implementation
being opaque at build time. A value returned by ``PyThread_tss_alloc`` is
in the same state as a value initialized with ``Py_tss_NEEDS_INIT``, or
``NULL`` in the case of dynamic allocation failure. The behavior of
``PyThread_tss_free`` involves calling ``PyThread_tss_delete``
preventively, or is a no-op if the value pointed to by the ``key``
argument is ``NULL``. ``PyThread_tss_is_created`` returns non-zero if the
given ``Py_tss_t`` has been initialized (i.e. by ``PyThread_tss_create``).
The new TSS API does not provide functions which correspond to
``PyThread_delete_key_value`` and ``PyThread_ReInitTLS``, because these
functions were needed only for CPython's now defunct built-in TLS
implementation; that is the existing behavior of these functions is treated
as follows: ``PyThread_delete_key_value(key)`` is equalivalent to
``PyThread_set_key_value(key, NULL)``, and ``PyThread_ReInitTLS()`` is a
no-op [8]_.
The new ``PyThread_tss_`` functions are almost exactly analogous to their
original counterparts with a few minor differenc
Re: [Python-Dev] PEP 552: deterministic pycs
On Thu, 7 Sep 2017 18:47:20 -0700 Nick Coghlan wrote: > However, I do wonder whether we could encode *all* the mode settings > into the magic number, such that we did something like reserving the > top 3 bits for format flags: > > * number & 0x1FFF -> the traditional magic number > * number & 0x8000 -> timestamp or hash? > * number & 0x4000 -> checked or not? > * number & 0x2000 -> reserved for future format changes I'd rather a single magic number and a separate bitfield that tells what the header encodes exactly. We don't *have* to fight for a tiny size reduction of pyc files. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 552: single magic number
On Fri, 8 Sep 2017 12:04:52 +0200 Antoine Pitrou wrote: > On Thu, 7 Sep 2017 18:47:20 -0700 > Nick Coghlan wrote: > > However, I do wonder whether we could encode *all* the mode settings > > into the magic number, such that we did something like reserving the > > top 3 bits for format flags: > > > > * number & 0x1FFF -> the traditional magic number > > * number & 0x8000 -> timestamp or hash? > > * number & 0x4000 -> checked or not? > > * number & 0x2000 -> reserved for future format changes > > I'd rather a single magic number and a separate bitfield that tells > what the header encodes exactly. We don't *have* to fight for a tiny > size reduction of pyc files. Let me expand a bit on this. Currently, the format is: - bytes 0..3: magic number - bytes 4..7: source file timestamp - bytes 8..11: source file size - bytes 12+: pyc file body (marshal format) What I'm proposing is: - bytes 0..3: magic number - bytes 4..7: header options (bitfield) - bytes 8..15: header contents Depending on header options: - bytes 8..11: source file timestamp - bytes 12..15: source file size or: - bytes 8..15: 64-bit source file hash - bytes 16+: pyc file body (marshal format) This way, we keep a single magic number, a single header size, and there's only a per-build variation in the middle of the header. Of course, there are possible ways to encode information. For example, the header could be a sequence of Type-Length-Value triplets, perhaps prefixed with header size or body offset for easy seeking. My whole point here is that we can easily avoid the annoyance of dual magic numbers and encodings which must be maintained in parallel. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 553 V2 - builtin breakpoint() (was Re: PEP 553: Built-in debug())
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 It would appear to be more secure to define specific meanings for particular values of the environment variable. S -BEGIN PGP SIGNATURE- Version: CryptUp 4.4.3 Gmail Encryption https://cryptup.org Comment: Seamlessly send, receive and search encrypted email wsFcBAEBCAAQBQJZsogwCRAplraUdy0hlgAAvBoQALGf9Fx4fnp4zmoYD6Qc y5XEbRZnHdnZUg4/k51qb5U5FRoQCSnFzJMltFD+5BuSTLCG8H0Bm9w1LL+N 8Rcg5j0rgXMY5FACK//jCV11uqi7eqVUst99oLPvtYSbGd9CFH5S8yCxT9Sz fl6Cw8Err828l18phAQ1fgq+AG9Y9vMtGDwLcj4XN5OlWEcpX/rB5HPppylH /xuProaU+Xluig6Gh09meRPSnd1GT+Y6T3E90AypnwbQiUiMVLTCVOXPRFuR vlODvAULpw2Pn4D191rgu3Rwou6DcTRmeJMrNhHLOfMEwGMXAkpZdJYGkVvj vgwmr2ZOD/CafzqQHgi22en0qwPQTZUK9MhqrVcT4pIPfshNfjQ34fe3qdnt sEPxDb4KjFVUBfZVbnxEvLIIf5C3Gir9sEq8LxoCU/2eG8jkxbxNKmhDfFEi lvHF/HSyU0AYhwK/MWsagiGmE+6MHL3URpl6Lp/jaKTvVIVhIxAEp+GKxp36 04v/elaiF++p82jjMFGvUWeqIj4hJJG94RFybXZ53TE5gK6jmOkCixtW8A6E LRFRr9C0f8mKM5d3JmZSCJVtIjoIerFyTTz4prm3S482mLxEeHvcYslhTZI3 yPn0sGIaTDcMk3PvJ8psjrQaEziuOGcDOLX2LYsr9IxKk/kbShmjKs9UBcm9 FNCp =acbU -END PGP SIGNATURE- -BEGIN PGP PUBLIC KEY BLOCK- Version: CryptUp 4.3.6 Gmail Encryption https://cryptup.org Comment: Seamlessly send, receive and search encrypted email xsFNBFlt3m8BEAC/1rDdusFArgznDbnluLfvIt3Wyl6DzFshtDIGUECuuGTG xcNLxAwLt60z3pabKxFW7w5RDyHcqRqB8RDDYfVw1FiPfAiYcdc6kMFmBoa4 veADYol0maZEGWdg9hmsbmcCuUZ6bHAKAlAdFlLV8T/SmayhwGzk5qGl2WqY VxqDhVPVBkjt/YBdzE++CY+7Gjqm1EqhWMe+cm2HwrZk51+KWCWSmZj8VieG jM9qogS+atleJhF14XbRmrF/OUq5odaJcIlYPy2MFnxNz9lsjVRLvkPGs4Z7 p+LFujj7B7QpW+B9ibrrktGOfPA+gVEBukWEapgghgF7C4X/xE5VfgdX9bHu q52gEuKQWX4weHJDbBykkceZWXlMy2bmWuoXJIQ+rwerGLA3BeuQphmW841d G2tRx4QRBuEUGNfcCLf57Dp7OMf4pu6+JGnCSmf3l2Se38jLqH/c4ySw7sCA slhCKTlzjt6hvvAHlOGOpwSVTzNgSb4LRdcFlgjAYE0TXwgsVzY4bc+cdMfQ rkruoZnQ+Eg0M+XauO3hIZujvcEckiiBjON6VWGLsgLOl/qAyKwzoFvQnnxK qvh7LGnRBtXkVuT3Q/HP8lBtJ1mUOvuKkrq/L75Xjtvj3LBPOFe9knLzvz+M tzzRR6Tv2MVVd3n5NqHvAuRDMw6IyEEwxf2MpQARAQABzSJTdGV2ZSBIb2xk ZW4gPHN0ZXZlQGhvbGRlbndlYi5jb20+wsF1BBABCAApBQJZbd5yBgsJBwgD AgkQKZa2lHctIZYEFQgCCgMWAgECGQECGwMCHgEAAM+JD/0YSpIaMYSo7Wqz YL13yyzukd27352glbTQosuLG31UgjbTx7rgK02EkXAOjVJCOpt8IqRVKEyI S8dAdg4k/rS4DeAiSD1wNlyVxD3TQbFZlDE8L6Gbj7CvdVg64Dm1mby+/YQX XrRAay48fCHWQyLwsvep/sc8k9wL2l7lXVETgbmo8vz0IeYevgZenYJGYWaY hEUnThdbcnjMJQArLt4VyVVLBxMRUg1ggf/ccmVjI8JqS+V1vyGDgHqBwDLp gdJcb6q1poUasD5Zdl3MCpbe3Z/6kUiaQIDkzAKSJdmHxwP2LnOG2I6Kkosk x3fDv6hR3hkG7jv8bv1jyuZzqSgv55wdLbBMNfeDsY14e0q0IRK6cYYiFGyU /y02Yj5KnDRqrMuDXuNcJ8CrkgcfWDUlHn5k0wmBEQCaAvm6amxYSvg8pqdz PtM53GcoZZ/5mDxGCq2j3beyqmYVI1Cp9rPGc9C7TfBJm72dc+TjaGjGQWbU 3DQHH/B5Ny4EJwPu7pFwGF/wQBnym9/aud2tb6d1+l4svQtC1XDzk/m/Hedt Metx3vuqEuUnFkucTl1v+Kk56GnG0mQTBmRg+xhRStqRkK+JYC/5MIasxcE5 cOTrTkoZzsET5+chNEdnK5JKi6gCMaLZIh7zjkLEP8gNP6vTEoJouZpSjBb5 qNPSwV+b587BTQRZbd5vARAAsw8xPylNT0VFDAdX2kgKCB7RsjTp6uYy5KuW jvO8f/WoUhJxKO5yLHzdJ5+nU33U7w73UhdeMBuon5bTeddIIczO6DcZjuiP OJVQeZCpjQbx58lPdvvT4/m16duHiYeD0ONDEchd03mbx0RSVB2Qo/KPy6cb etvdBF893uIJSvJKY/vDmi8yuYsbfnxmirPGMT8DqY3O02g6nZuUp8iQbDEQ xHwp9PZZ5A3ZewG95uY7QgRv8we97QHjGc/Nzr284zHIAOJeqiRNd+xnqCZs mm31lw0ITxwJERKZTAIOgPGejpuLd4zuqS+P385fhsC1RSQJOuozlkiraOSI R76QhOgdSMsMNDr8PRMaDYOLWgXq5vIDQ+7CmH7b6EFM77Acxblf7V1LHjHm IOCgQVRCf27F4M0GulcNOEioQFAYlPGtsI4SshvOguy8eybN+T/mh3R66WrZ RVUHnoURkAkEJ1p+r6amAaQCr+OYHztkjd+6KgW6jQJWsCWcii8lzyWf1oPT g/eZsF7gqArR7H0FkWPFnKPUaJe2AEDXzLlTCf1tN/KDegx8aW1kC+wXUqFj VG0tL1s5CRwVSRhXqt4hV4ARrmeoP6R5IQVvRRomPemisvd+qNHFasWU0El7 S7LVZ+ppS6YejowjscHXDREnoux9Q+v014qTnHYR40qvr8sAEQEAAcLBXwQY AQgAEwUCWW3ecwkQKZa2lHctIZYCGwwAADPkD/9sQInKpiSxeddX0j9ygH5y RdPIZ8fX9rA/E3qp06dLIj0sQo3wpX+8LhxwTGbB/pM6yd5SjNt1A/j9tpS4 Rg6uzqe7ezmMuQTKczOL9/2yFhNpGaPOnFUs2nEFfHv4IGO96rqanDOEfUDo kz/zXoxzehnpbxEUC5+azR46laTrEyRdX9R5UhvY2EA8kuL2hKGQg5OXdaMJ dqtGLyGSa+XTOAvCD0DalJA6T4tDHjOlyUI2GaGs53it3dLAT4sUeBZMATGW vbR+FJbM/dQVvT39ogtzDilL5je2dGZUQMdqwyfPL3HkDmfkr7ESEJEjQD+I /0fzH5jGm0q67bI2B54ddLCy7/60ViFPDYDi57gMEq5lXx6uBQgQMh49vgfO 9ywulUyH2ueFijaTJulNQPpn0fhMTm/20SLI2qHYtLI/g/veMDva/UPuolr1 kQk0BRaLmR8audc4SrGWObGDgVfEmqqHdzJu97ZX6lEKtCcLh/n1u0SbmyEX SF/iStR5tnURIGrykEM+lkeJVmc0Yuh67DCiJC14Widnfy2f5FbcfyRsUs/Y 5WsqgXXroxDWXy/892paz8LmswX08y/sdxGKzr1GmMq8Ib5CixjmFuDhaKpp WhOOGXLkTeGK+W2ARm/xq2YLcmpTBm48ABwRHfTYm6WbrVMXjlWbqwhr3mWT 2g== =0xrN -END PGP PUBLIC KEY BLOCK- ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 539 v3: A new C API for Thread-Local Storage in CPython
On 8 September 2017 at 00:30, Masayuki YAMAMOTO wrote: > Hi folks, > > I submit PEP 539 third draft for the finish. Thank you for all the advice > and the help! Thank you Erik & Yamamoto-san for all of your work on this PEP! The updates look good, so I'm happy to say as BDFL-Delegate that this proposal is now accepted :) Regards, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 552: single magic number
I also like having the header fixed-size, so it might be possible to rewrite headers (e.g. to flip the source bit) without moving the rest of the file. On Fri, Sep 8, 2017 at 3:38 AM, Antoine Pitrou wrote: > On Fri, 8 Sep 2017 12:04:52 +0200 > Antoine Pitrou wrote: > > On Thu, 7 Sep 2017 18:47:20 -0700 > > Nick Coghlan wrote: > > > However, I do wonder whether we could encode *all* the mode settings > > > into the magic number, such that we did something like reserving the > > > top 3 bits for format flags: > > > > > > * number & 0x1FFF -> the traditional magic number > > > * number & 0x8000 -> timestamp or hash? > > > * number & 0x4000 -> checked or not? > > > * number & 0x2000 -> reserved for future format changes > > > > I'd rather a single magic number and a separate bitfield that tells > > what the header encodes exactly. We don't *have* to fight for a tiny > > size reduction of pyc files. > > Let me expand a bit on this. Currently, the format is: > > - bytes 0..3: magic number > - bytes 4..7: source file timestamp > - bytes 8..11: source file size > - bytes 12+: pyc file body (marshal format) > > What I'm proposing is: > > - bytes 0..3: magic number > - bytes 4..7: header options (bitfield) > - bytes 8..15: header contents >Depending on header options: > - bytes 8..11: source file timestamp > - bytes 12..15: source file size >or: > - bytes 8..15: 64-bit source file hash > - bytes 16+: pyc file body (marshal format) > > This way, we keep a single magic number, a single header size, and > there's only a per-build variation in the middle of the header. > > > Of course, there are possible ways to encode information. For > example, the header could be a sequence of Type-Length-Value triplets, > perhaps prefixed with header size or body offset for easy seeking. > > My whole point here is that we can easily avoid the annoyance of dual > magic numbers and encodings which must be maintained in parallel. > > Regards > > Antoine. > > > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > guido%40python.org > -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 552: deterministic pycs
On 8 September 2017 at 03:04, Antoine Pitrou wrote: > On Thu, 7 Sep 2017 18:47:20 -0700 > Nick Coghlan wrote: >> However, I do wonder whether we could encode *all* the mode settings >> into the magic number, such that we did something like reserving the >> top 3 bits for format flags: >> >> * number & 0x1FFF -> the traditional magic number >> * number & 0x8000 -> timestamp or hash? >> * number & 0x4000 -> checked or not? >> * number & 0x2000 -> reserved for future format changes > > I'd rather a single magic number and a separate bitfield that tells > what the header encodes exactly. We don't *have* to fight for a tiny > size reduction of pyc files. One of Benjamin's goals was for the existing timestamp-based pyc format to remain completely unchanged, so we need some kind of marker in the magic number to indicate whether the file is using the new format or nor. I'd also be fine with using a single bit for that, such that the only bitmasking needed was: * number & 0x8000 -> legacy format or new format? * number & 0x7FFF -> the magic number itself And any further flags would go in a separate field. That's essentially what PEP 552 already suggests, the only adjustment is the idea of specifically using the high order bit in the magic number field to indicate the pyc format in use rather than leaving the explanation of how the two magic numbers will differ unspecified. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Pygments in PEPs?
Hi, Is it possible to install pygments on the machine which builds and renders PEPs? It would allow syntax highlighting in PEPs, making example code more readable. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 552: deterministic pycs
On Fri, 8 Sep 2017 07:49:46 -0700 Nick Coghlan wrote: > On 8 September 2017 at 03:04, Antoine Pitrou wrote: > > On Thu, 7 Sep 2017 18:47:20 -0700 > > Nick Coghlan wrote: > >> However, I do wonder whether we could encode *all* the mode settings > >> into the magic number, such that we did something like reserving the > >> top 3 bits for format flags: > >> > >> * number & 0x1FFF -> the traditional magic number > >> * number & 0x8000 -> timestamp or hash? > >> * number & 0x4000 -> checked or not? > >> * number & 0x2000 -> reserved for future format changes > > > > I'd rather a single magic number and a separate bitfield that tells > > what the header encodes exactly. We don't *have* to fight for a tiny > > size reduction of pyc files. > > One of Benjamin's goals was for the existing timestamp-based pyc > format to remain completely unchanged, so we need some kind of marker > in the magic number to indicate whether the file is using the new > format or nor. I don't think that's a useful goal, as long as we bump the magic number. Note the header format was already changed in the past when we added a "size" field beside the "timestamp" field, to resolve collisions due to timestamp granularity. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 553 V2 - builtin breakpoint() (was Re: PEP 553: Built-in debug())
On 7 September 2017 at 20:02, Adrian Petrescu wrote: > Would that not be a security concern, if you can get Python to execute > arbitrary code just by setting an environment variable? Not really, as once someone has write access to your process environment, you've already lost (they can mess with PYTHONIOENCODING, PYTHONPATH, LD_PRELOAD, OpenSSL certificate verification settings, and more). Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 557: Data Classes
I've written a PEP for what might be thought of as "mutable namedtuples
with defaults, but not inheriting tuple's behavior" (a mouthful, but it
sounded simpler when I first thought of it). It's heavily influenced by
the attrs project. It uses PEP 526 type annotations to define fields.
From the overview section:
@dataclass
class InventoryItem:
name: str
unit_price: float
quantity_on_hand: int = 0
def total_cost(self) -> float:
return self.unit_price * self.quantity_on_hand
Will automatically add these methods:
def __init__(self, name: str, unit_price: float, quantity_on_hand:
int = 0) -> None:
self.name = name
self.unit_price = unit_price
self.quantity_on_hand = quantity_on_hand
def __repr__(self):
return
f'InventoryItem(name={self.name!r},unit_price={self.unit_price!r},quantity_on_hand={self.quantity_on_hand!r})'
def __eq__(self, other):
if other.__class__ is self.__class__:
return (self.name, self.unit_price, self.quantity_on_hand) ==
(other.name, other.unit_price, other.quantity_on_hand)
return NotImplemented
def __ne__(self, other):
if other.__class__ is self.__class__:
return (self.name, self.unit_price, self.quantity_on_hand) !=
(other.name, other.unit_price, other.quantity_on_hand)
return NotImplemented
def __lt__(self, other):
if other.__class__ is self.__class__:
return (self.name, self.unit_price, self.quantity_on_hand) <
(other.name, other.unit_price, other.quantity_on_hand)
return NotImplemented
def __le__(self, other):
if other.__class__ is self.__class__:
return (self.name, self.unit_price, self.quantity_on_hand) <=
(other.name, other.unit_price, other.quantity_on_hand)
return NotImplemented
def __gt__(self, other):
if other.__class__ is self.__class__:
return (self.name, self.unit_price, self.quantity_on_hand) >
(other.name, other.unit_price, other.quantity_on_hand)
return NotImplemented
def __ge__(self, other):
if other.__class__ is self.__class__:
return (self.name, self.unit_price, self.quantity_on_hand) >=
(other.name, other.unit_price, other.quantity_on_hand)
return NotImplemented
Data Classes saves you from writing and maintaining these functions.
The PEP is largely complete, but could use some filling out in places.
Comments welcome!
Eric.
P.S. I wrote this PEP when I was in my happy place.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pygments in PEPs?
I already made a PR that would highlight code in PEPs half-year ago, but it was rejected with the reason that they will be moved to RtD soon. -- Ivan 8 Вер 2017 16:56 "Antoine Pitrou" пише: > > Hi, > > Is it possible to install pygments on the machine which builds and > renders PEPs? It would allow syntax highlighting in PEPs, making > example code more readable. > > Regards > > Antoine. > > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > levkivskyi%40gmail.com > ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 556: Threaded garbage collection
Hello, I've written a PEP by which you can tell the GC to run in a dedicated thread. The goal is to solve reentrancy issues with finalizers: https://www.python.org/dev/peps/pep-0556/ Regards Antoine. PS: I did not come up with the idea for this PEP while other people were having nightmares. Any nightmares involved in this PEP are fictional, and any resemblance to actual nightmares is purely coincidental. No nightmares were harmed while writing this PEP. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 557: Data Classes
Oops, I forgot the link. It should show up shortly at
https://www.python.org/dev/peps/pep-0557/.
Eric.
On 9/8/17 7:57 AM, Eric V. Smith wrote:
I've written a PEP for what might be thought of as "mutable namedtuples
with defaults, but not inheriting tuple's behavior" (a mouthful, but it
sounded simpler when I first thought of it). It's heavily influenced by
the attrs project. It uses PEP 526 type annotations to define fields.
From the overview section:
@dataclass
class InventoryItem:
name: str
unit_price: float
quantity_on_hand: int = 0
def total_cost(self) -> float:
return self.unit_price * self.quantity_on_hand
Will automatically add these methods:
def __init__(self, name: str, unit_price: float, quantity_on_hand: int
= 0) -> None:
self.name = name
self.unit_price = unit_price
self.quantity_on_hand = quantity_on_hand
def __repr__(self):
return
f'InventoryItem(name={self.name!r},unit_price={self.unit_price!r},quantity_on_hand={self.quantity_on_hand!r})'
def __eq__(self, other):
if other.__class__ is self.__class__:
return (self.name, self.unit_price, self.quantity_on_hand) ==
(other.name, other.unit_price, other.quantity_on_hand)
return NotImplemented
def __ne__(self, other):
if other.__class__ is self.__class__:
return (self.name, self.unit_price, self.quantity_on_hand) !=
(other.name, other.unit_price, other.quantity_on_hand)
return NotImplemented
def __lt__(self, other):
if other.__class__ is self.__class__:
return (self.name, self.unit_price, self.quantity_on_hand) <
(other.name, other.unit_price, other.quantity_on_hand)
return NotImplemented
def __le__(self, other):
if other.__class__ is self.__class__:
return (self.name, self.unit_price, self.quantity_on_hand) <=
(other.name, other.unit_price, other.quantity_on_hand)
return NotImplemented
def __gt__(self, other):
if other.__class__ is self.__class__:
return (self.name, self.unit_price, self.quantity_on_hand) >
(other.name, other.unit_price, other.quantity_on_hand)
return NotImplemented
def __ge__(self, other):
if other.__class__ is self.__class__:
return (self.name, self.unit_price, self.quantity_on_hand) >=
(other.name, other.unit_price, other.quantity_on_hand)
return NotImplemented
Data Classes saves you from writing and maintaining these functions.
The PEP is largely complete, but could use some filling out in places.
Comments welcome!
Eric.
P.S. I wrote this PEP when I was in my happy place.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Lazy initialization of module global state
This is an idea that came out of the lazy module loading (via AST analysis), posted to python-ideas. The essential idea is to split the marshal data stored in the .pyc into smaller pieces and only load the parts as they are accessed. E.g. use a __getattr__ hook on the module to unmarshal+exec the code. I have a very early prototype: https://github.com/warsaw/lazyimport/blob/master/lazy_compile.py Would work like a "compile_all.py" tool. It writes standard .pyc files right now. It is not there yet but I should use the AST analysis, like the lazy module load stuff, to determine if things have potential side-effects on module import. Those things will get loaded eagerly, like they do now. Initially I was thinking of class definitions and functions but now I realize any global state could get this treatment. E.g. if you have a large dictionary global, don't unmarshal it until someone accesses the module attribute. This should be pretty safe to do and should give a significant benefit in startup time and memory usage. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 557: Data Classes
On 8 September 2017 at 15:57, Eric V. Smith wrote: > I've written a PEP for what might be thought of as "mutable namedtuples with > defaults, but not inheriting tuple's behavior" (a mouthful, but it sounded > simpler when I first thought of it). It's heavily influenced by the attrs > project. It uses PEP 526 type annotations to define fields. [...] > The PEP is largely complete, but could use some filling out in places. > Comments welcome! > > Eric. > > P.S. I wrote this PEP when I was in my happy place. Looks good! One minor point - apparently in your happy place, C and Python have the same syntax :-) """ field's may optionally specify a default value, using normal Python syntax: @dataclass class C: int a # 'a' has no default value int b = 0 # assign a default value for 'b' """ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 557: Data Classes
Interesting. I note that this under "Specification":
"""
field's may optionally specify a default value, using normal Python syntax:
@dataclass
class C:
int a # 'a' has no default value
int b = 0 # assign a default value for 'b'
"""
...does not look like "normal Python syntax".
On Fri, Sep 8, 2017 at 11:44 AM Eric V. Smith wrote:
> Oops, I forgot the link. It should show up shortly at
> https://www.python.org/dev/peps/pep-0557/.
>
> Eric.
>
> On 9/8/17 7:57 AM, Eric V. Smith wrote:
> > I've written a PEP for what might be thought of as "mutable namedtuples
> > with defaults, but not inheriting tuple's behavior" (a mouthful, but it
> > sounded simpler when I first thought of it). It's heavily influenced by
> > the attrs project. It uses PEP 526 type annotations to define fields.
> > From the overview section:
> >
> > @dataclass
> > class InventoryItem:
> > name: str
> > unit_price: float
> > quantity_on_hand: int = 0
> >
> > def total_cost(self) -> float:
> > return self.unit_price * self.quantity_on_hand
> >
> > Will automatically add these methods:
> >
> > def __init__(self, name: str, unit_price: float, quantity_on_hand: int
> > = 0) -> None:
> > self.name = name
> > self.unit_price = unit_price
> > self.quantity_on_hand = quantity_on_hand
> > def __repr__(self):
> > return
> > f'InventoryItem(name={self.name
> !r},unit_price={self.unit_price!r},quantity_on_hand={self.quantity_on_hand!r})'
> >
> > def __eq__(self, other):
> > if other.__class__ is self.__class__:
> > return (self.name, self.unit_price, self.quantity_on_hand) ==
> > (other.name, other.unit_price, other.quantity_on_hand)
> > return NotImplemented
> > def __ne__(self, other):
> > if other.__class__ is self.__class__:
> > return (self.name, self.unit_price, self.quantity_on_hand) !=
> > (other.name, other.unit_price, other.quantity_on_hand)
> > return NotImplemented
> > def __lt__(self, other):
> > if other.__class__ is self.__class__:
> > return (self.name, self.unit_price, self.quantity_on_hand) <
> > (other.name, other.unit_price, other.quantity_on_hand)
> > return NotImplemented
> > def __le__(self, other):
> > if other.__class__ is self.__class__:
> > return (self.name, self.unit_price, self.quantity_on_hand) <=
> > (other.name, other.unit_price, other.quantity_on_hand)
> > return NotImplemented
> > def __gt__(self, other):
> > if other.__class__ is self.__class__:
> > return (self.name, self.unit_price, self.quantity_on_hand) >
> > (other.name, other.unit_price, other.quantity_on_hand)
> > return NotImplemented
> > def __ge__(self, other):
> > if other.__class__ is self.__class__:
> > return (self.name, self.unit_price, self.quantity_on_hand) >=
> > (other.name, other.unit_price, other.quantity_on_hand)
> > return NotImplemented
> >
> > Data Classes saves you from writing and maintaining these functions.
> >
> > The PEP is largely complete, but could use some filling out in places.
> > Comments welcome!
> >
> > Eric.
> >
> > P.S. I wrote this PEP when I was in my happy place.
> >
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/jcgoble3%40gmail.com
>
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 553: Built-in debug()
Barry Warsaw writes: > and you would drop into the debugger after foo() but before bar(). > More rationale and details are provided in the PEP: > > https://www.python.org/dev/peps/pep-0553/ > > Unlike David, but like Larry, I have a prototype implementation: > > https://github.com/python/cpython/pull/3355 > > P.S. This came to me in a nightmare on Sunday night, and the more I > explored the idea the more it frightened me. I know exactly what I > was dreaming about and the only way to make it all go away was to > write this thing up. What's even scarier are the Qabalistic implications of the reversal of the PEP # and the PR #! Steve ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 539 v3: A new C API for Thread-Local Storage in CPython
Awesome! Thank you for accepting the PEP :) The only thing missing is a reference implementation, I'm going to complete. BTW, one of TSS keys is going to move to the runtime struct by bpo-30860 "Consolidate stateful C globals under a single struct". Although that's PR was merged, the issue status is still open yet. Is there a chance for rollback? Masayuki 2017-09-08 23:37 GMT+09:00 Nick Coghlan : > On 8 September 2017 at 00:30, Masayuki YAMAMOTO > wrote: > > Hi folks, > > > > I submit PEP 539 third draft for the finish. Thank you for all the advice > > and the help! > > Thank you Erik & Yamamoto-san for all of your work on this PEP! > > The updates look good, so I'm happy to say as BDFL-Delegate that this > proposal is now accepted :) > > Regards, > Nick. > > -- > Nick Coghlan | [email protected] | Brisbane, Australia > ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 557: Data Classes
Hah! Thanks for the catch. There is no C in my happy place. -- Eric. > On Sep 8, 2017, at 8:52 AM, Paul Moore wrote: > >> On 8 September 2017 at 15:57, Eric V. Smith wrote: >> I've written a PEP for what might be thought of as "mutable namedtuples with >> defaults, but not inheriting tuple's behavior" (a mouthful, but it sounded >> simpler when I first thought of it). It's heavily influenced by the attrs >> project. It uses PEP 526 type annotations to define fields. > [...] >> The PEP is largely complete, but could use some filling out in places. >> Comments welcome! >> >> Eric. >> >> P.S. I wrote this PEP when I was in my happy place. > > Looks good! One minor point - apparently in your happy place, C and > Python have the same syntax :-) > > """ > field's may optionally specify a default value, using normal Python syntax: > > @dataclass > class C: >int a # 'a' has no default value >int b = 0 # assign a default value for 'b' > """ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Summary of Python tracker Issues
ACTIVITY SUMMARY (2017-09-01 - 2017-09-08) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open6149 (-17) closed 36997 (+87) total 43146 (+70) Open issues with patches: 2321 Issues opened (45) == #31069: test_multiprocessing_spawn and test_multiprocessing_forkserver https://bugs.python.org/issue31069 reopened by haypo #31327: bug in dateutil\tz\tz.py https://bugs.python.org/issue31327 opened by jerrykramskoy #31329: Add idlelib module entry to doc https://bugs.python.org/issue31329 opened by terry.reedy #31331: IDLE: Move prompts with input. https://bugs.python.org/issue31331 opened by terry.reedy #31332: Building modules by Clang with Microsoft CodeGen https://bugs.python.org/issue31332 opened by tats.u. #31333: Implement ABCMeta in C https://bugs.python.org/issue31333 opened by levkivskyi #31334: select.poll.poll fails on BSDs with arbitrary negative timeout https://bugs.python.org/issue31334 opened by Riccardo Coccioli #31336: Speed up _PyType_Lookup() for class creation https://bugs.python.org/issue31336 opened by scoder #31338: Use abort() for code we never expect to hit https://bugs.python.org/issue31338 opened by barry #31342: test.bisect module causes tests to fail https://bugs.python.org/issue31342 opened by nascheme #31345: Backport docstring improvements to the C version of OrderedDic https://bugs.python.org/issue31345 opened by rhettinger #31346: Prefer PROTOCOL_TLS_CLIENT/SERVER https://bugs.python.org/issue31346 opened by christian.heimes #31348: webbrowser BROWSER with MacOSXOSAScript type https://bugs.python.org/issue31348 opened by michael-lazar #31349: Embedded initialization ignores Py_SetProgramName() https://bugs.python.org/issue31349 opened by chrullrich #31351: ensurepip discards pip's return code which leads to broken ven https://bugs.python.org/issue31351 opened by Igor Filatov #31353: Implement PEP 553 - built-in debug() https://bugs.python.org/issue31353 opened by barry #31354: Fixing a bug related to LTO only build https://bugs.python.org/issue31354 opened by octavian.soldea #31355: Remove Travis CI macOS job: rely on buildbots https://bugs.python.org/issue31355 opened by haypo #31356: Add context manager to temporarily disable GC https://bugs.python.org/issue31356 opened by rhettinger #31357: Expose `worker_target` and `workitem_cls` as arugments to cust https://bugs.python.org/issue31357 opened by justdoit0823 #31359: `configure` script incorrectly detects symbols as available on https://bugs.python.org/issue31359 opened by Maxime Belanger #31361: Update feedparser.py to prevent theano compiling fail in pytho https://bugs.python.org/issue31361 opened by Wei-Shun Lo #31363: __PYVENV_LAUNCHER__ breaks calling another venv's interpreter https://bugs.python.org/issue31363 opened by Ilya.Kulakov #31366: Missing terminator option when using readline with socketserve https://bugs.python.org/issue31366 opened by Thomas Feldmann #31368: RWF_NONBLOCK https://bugs.python.org/issue31368 opened by YoSTEALTH #31369: re.RegexFlag is not included in __all__ https://bugs.python.org/issue31369 opened by PJB3005 #31371: Remove deprecated tkinter.tix module in 3.7 https://bugs.python.org/issue31371 opened by ned.deily #31372: Add SSLSocket.get_verify_result() https://bugs.python.org/issue31372 opened by christian.heimes #31374: expat: warning: "_POSIX_C_SOURCE" redefined https://bugs.python.org/issue31374 opened by christian.heimes #31375: Add the interpreters module to stdlib (PEP 554). https://bugs.python.org/issue31375 opened by eric.snow #31376: test_multiprocessing_spawn randomly hangs AMD64 FreeBSD 10.x S https://bugs.python.org/issue31376 opened by haypo #31377: remove *_INTERNED opcodes from marshal https://bugs.python.org/issue31377 opened by benjamin.peterson #31378: Missing documentation for sqlite3.OperationalError https://bugs.python.org/issue31378 opened by Leonardo Taglialegne #31380: test_undecodable_filename() in Lib/test/test_httpservers.py br https://bugs.python.org/issue31380 opened by howarthjw #31381: Unable to read the project file "pythoncore.vcxproj". https://bugs.python.org/issue31381 opened by denis-osipov #31382: CGI upload error when file ~< 10kb https://bugs.python.org/issue31382 opened by mschaming #31386: Make return types of wrap_bio and wrap_socket customizable https://bugs.python.org/issue31386 opened by christian.heimes #31387: asyncio should make it easy to enable cooperative SIGINT handl https://bugs.python.org/issue31387 opened by ncoghlan #31388: Provide a way to defer SIGINT handling in the current thread https://bugs.python.org/issue31388 opened by ncoghlan #31389: Give pdb.set_trace() an optional `header` keyword argument https://bugs.python.org/issue31389 opened by barry #31390: pydoc.Helper.keywords missing async a
Re: [Python-Dev] PEP 557: Data Classes
On 9/8/17 9:07 AM, Eric V. Smith wrote: Hah! Thanks for the catch. I've pushed a fix to the PEP repo, it should show up online shortly. Eric. There is no C in my happy place. -- Eric. On Sep 8, 2017, at 8:52 AM, Paul Moore wrote: On 8 September 2017 at 15:57, Eric V. Smith wrote: I've written a PEP for what might be thought of as "mutable namedtuples with defaults, but not inheriting tuple's behavior" (a mouthful, but it sounded simpler when I first thought of it). It's heavily influenced by the attrs project. It uses PEP 526 type annotations to define fields. [...] The PEP is largely complete, but could use some filling out in places. Comments welcome! Eric. P.S. I wrote this PEP when I was in my happy place. Looks good! One minor point - apparently in your happy place, C and Python have the same syntax :-) """ field's may optionally specify a default value, using normal Python syntax: @dataclass class C: int a # 'a' has no default value int b = 0 # assign a default value for 'b' """ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 552: deterministic pycs
On 8 September 2017 at 07:55, Antoine Pitrou wrote: > On Fri, 8 Sep 2017 07:49:46 -0700 > Nick Coghlan wrote: >> > I'd rather a single magic number and a separate bitfield that tells >> > what the header encodes exactly. We don't *have* to fight for a tiny >> > size reduction of pyc files. >> >> One of Benjamin's goals was for the existing timestamp-based pyc >> format to remain completely unchanged, so we need some kind of marker >> in the magic number to indicate whether the file is using the new >> format or nor. > > I don't think that's a useful goal, as long as we bump the magic number. Yeah, we (me, Benjamin, Greg) discussed that here, and we agree - there isn't actually any benefit to keeping the timestamp based pyc's using the same layout, since the magic number is already going to change anyway. Given that, I think your suggested 16 byte header layout would be a good one: 4 byte magic number, 4 bytes reserved for format flags, 8 bytes with an interpretation that depends on the format flags. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 556: Threaded garbage collection
On 8 September 2017 at 08:05, Antoine Pitrou wrote: > > Hello, > > I've written a PEP by which you can tell the GC to run in a dedicated > thread. The goal is to solve reentrancy issues with finalizers: > https://www.python.org/dev/peps/pep-0556/ +1 from me for the general concept. (Minor naming idea: "inline" may be a clearer name for the current behaviour). One point that seems worth noting: even with cyclic GC moved out to a separate thread, __del__ methods and weakref callbacks triggered by a refcount going to zero will still be called inline in the current thread. Changing that would require a tweak to the semantics of Py_DECREF where if the GC was in threaded mode, instead of finalizing the object immediately, it would instead be placed on a FIFO queue where the GC thread would pick it up and then actually delete it. Cheers, Nick. > PS: I did not come up with the idea for this PEP while other people > were having nightmares. I dunno, debugging finalizer re-entrancy problems seems pretty nightmarish to me ;) -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Lazy initialization of module global state
On 8 September 2017 at 08:50, Neil Schemenauer wrote: > This should be pretty safe to do and should give a significant > benefit in startup time and memory usage. And given the extended pyc header being proposed in PEP 552, we'd be able to include flags in the header to indicate that the pyc file was compiled with this alternative behaviour. While that wouldn't be needed for import, it would be useful for anyone trying to confirm that the pyc file actually matches the corresponding source file. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 556: Threaded garbage collection
On Fri, 8 Sep 2017 10:03:33 -0700 Nick Coghlan wrote: > On 8 September 2017 at 08:05, Antoine Pitrou wrote: > > > > Hello, > > > > I've written a PEP by which you can tell the GC to run in a dedicated > > thread. The goal is to solve reentrancy issues with finalizers: > > https://www.python.org/dev/peps/pep-0556/ > > +1 from me for the general concept. (Minor naming idea: "inline" may > be a clearer name for the current behaviour). > > One point that seems worth noting: even with cyclic GC moved out to a > separate thread, __del__ methods and weakref callbacks triggered by a > refcount going to zero will still be called inline in the current > thread. You're right, that bears mentioning. It's much less of a problem, of course, since such calls happen at deterministic places. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 539 v3: A new C API for Thread-Local Storage in CPython
On 8 September 2017 at 08:58, Masayuki YAMAMOTO wrote: > Awesome! Thank you for accepting the PEP :) > The only thing missing is a reference implementation, I'm going to complete. > > BTW, one of TSS keys is going to move to the runtime struct by bpo-30860 > "Consolidate stateful C globals under a single struct". Although that's PR > was merged, the issue status is still open yet. Is there a chance for > rollback? No, we genuinely want to consolidate that state into a single shared location. However, the struct definition can be adjusted as needed as part of the PEP 539 implementation (and we'll get Eric Snow to be one of the PR reviewers). Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 557: Data Classes
On 8 September 2017 at 07:57, Eric V. Smith wrote: > I've written a PEP for what might be thought of as "mutable namedtuples with > defaults, but not inheriting tuple's behavior" (a mouthful, but it sounded > simpler when I first thought of it). It's heavily influenced by the attrs > project. It uses PEP 526 type annotations to define fields. From the > overview section: > > @dataclass > class InventoryItem: > name: str > unit_price: float > quantity_on_hand: int = 0 > > def total_cost(self) -> float: > return self.unit_price * self.quantity_on_hand Very nice! > def __eq__(self, other): > if other.__class__ is self.__class__: > return (self.name, self.unit_price, self.quantity_on_hand) == > (other.name, other.unit_price, other.quantity_on_hand) > return NotImplemented My one technical question about the PEP relates to the use of an exact type check in the comparison methods, rather than "isinstance(other, self.__class__)". I think I agree with that decision, but it isn't immediately obvious that the class identity is considered part of the instance value for a data class, so if you do: @dataclass class BaseItem: value: Any class DerivedItem: pass Then instances of DerivedItem *won't* be considered equivalent to instances of BaseItem, and they also won't be orderable relative to each other, even though "DerivedItem" doesn't actually add any new data fields. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 552: deterministic pycs
Thank you all for the feedback. I've now updated the PEP to specify a 4-word pyc header with a bit field in every case. On Fri, Sep 8, 2017, at 09:43, Nick Coghlan wrote: > On 8 September 2017 at 07:55, Antoine Pitrou wrote: > > On Fri, 8 Sep 2017 07:49:46 -0700 > > Nick Coghlan wrote: > >> > I'd rather a single magic number and a separate bitfield that tells > >> > what the header encodes exactly. We don't *have* to fight for a tiny > >> > size reduction of pyc files. > >> > >> One of Benjamin's goals was for the existing timestamp-based pyc > >> format to remain completely unchanged, so we need some kind of marker > >> in the magic number to indicate whether the file is using the new > >> format or nor. > > > > I don't think that's a useful goal, as long as we bump the magic number. > > Yeah, we (me, Benjamin, Greg) discussed that here, and we agree - > there isn't actually any benefit to keeping the timestamp based pyc's > using the same layout, since the magic number is already going to > change anyway. > > Given that, I think your suggested 16 byte header layout would be a > good one: 4 byte magic number, 4 bytes reserved for format flags, 8 > bytes with an interpretation that depends on the format flags. > > Cheers, > Nick. > > -- > Nick Coghlan | [email protected] | Brisbane, Australia > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/benjamin%40python.org ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 557: Data Classes
On Fri, Sep 08, 2017 at 10:37:12AM -0700, Nick Coghlan wrote: > > def __eq__(self, other): > > if other.__class__ is self.__class__: > > return (self.name, self.unit_price, self.quantity_on_hand) == > > (other.name, other.unit_price, other.quantity_on_hand) > > return NotImplemented > > My one technical question about the PEP relates to the use of an exact > type check in the comparison methods, rather than "isinstance(other, > self.__class__)". I haven't read the whole PEP in close detail, but that method stood out for me too. Only, unlike Nick, I don't think I agree with the decision. I'm also not convinced that we should be adding ordered comparisons (__lt__ __gt__ etc) by default, if these DataClasses are considered more like structs/records than tuples. The closest existing equivalent to a struct in the std lib (apart from namedtuple) is, I think, SimpleNamespace, and they are unorderable. -- Steve ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 557: Data Classes
On 09/08/2017 11:38 AM, Steven D'Aprano wrote: On Fri, Sep 08, 2017 at 10:37:12AM -0700, Nick Coghlan wrote: def __eq__(self, other): if other.__class__ is self.__class__: return (self.name, self.unit_price, self.quantity_on_hand) == (other.name, other.unit_price, other.quantity_on_hand) return NotImplemented My one technical question about the PEP relates to the use of an exact type check in the comparison methods, rather than "isinstance(other, self.__class__)". I haven't read the whole PEP in close detail, but that method stood out for me too. Only, unlike Nick, I don't think I agree with the decision. I'm also not convinced that we should be adding ordered comparisons (__lt__ __gt__ etc) by default, if these DataClasses are considered more like structs/records than tuples. The closest existing equivalent to a struct in the std lib (apart from namedtuple) is, I think, SimpleNamespace, and they are unorderable. I'll split the difference. ;) I agree with D'Aprano that an isinstance check should be used, but I disagree with him about the rich comparison methods -- please include them. I think unordered should only be the default when order is impossible, extremely difficult, or nonsensical. -- ~Ethan~ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 556: Threaded garbage collection
I like it overall. - I was wondering what happens during interpreter shutdown. I see you have that listed as a open issue. How about simply shutting down the finalization thread and not guaranteeing that finalizers are actually ever run à la Java? - Why not run all (Python) finalizers on the thread and not just ones from cycles? On Fri, Sep 8, 2017, at 08:05, Antoine Pitrou wrote: > > Hello, > > I've written a PEP by which you can tell the GC to run in a dedicated > thread. The goal is to solve reentrancy issues with finalizers: > https://www.python.org/dev/peps/pep-0556/ > > Regards > > Antoine. > > PS: I did not come up with the idea for this PEP while other people > were having nightmares. Any nightmares involved in this PEP are > fictional, and any resemblance to actual nightmares is purely > coincidental. No nightmares were harmed while writing this PEP. > > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/benjamin%40python.org ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 556: Threaded garbage collection
On 09/08/2017 12:04 PM, Benjamin Peterson wrote: - Why not run all (Python) finalizers on the thread and not just ones from cycles? Two reasons: 1. Because some code relies on the finalizer being called on the thread where the last reference is dropped. This is usually the same thread where the object was created. Some irritating third-party libraries make demands on callers, e.g. "you can only interact with / destroy X objects on your 'main thread'". This is often true of windowing / GUI libraries. (For example, I believe this was true of Direct3D, at least as of D3D8; it was also often true of Win32 USER objects.) 2. Because there's so much work there. In the Gilectomy prototype, I originally called all finalizers on the "reference count manager commit thread", the thread that also committed increfs and decrefs. The thread immediately fell behind on its queue and never caught up. I changed the Gilectomy so objects needing finalization are passed back to the thread where the last decref happened, for finalization on that thread; this was pleasingly self-balancing. Note that I turned off cyclic GC on the Gilectomy prototype a long time ago and haven't revisited it since. My very, very long-term plan for GC is to stop the world and run it from one thread. With the current system, that means all those finalizers would be run on the thread chosen to run the GC. //arry/ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 556: Threaded garbage collection
On Fri, 08 Sep 2017 12:04:10 -0700 Benjamin Peterson wrote: > I like it overall. > > - I was wondering what happens during interpreter shutdown. I see you > have that listed as a open issue. How about simply shutting down the > finalization thread and not guaranteeing that finalizers are actually > ever run à la Java? I don't know. People generally have expectations towards stuff being finalized properly (especially when talking about files etc.). Once the first implementation is devised, we will know more about what's workable (perhaps we'll have to move _PyGC_Fini earlier in the shutdown sequence? perhaps we'll want to switch back to serial mode when shutting down?). > - Why not run all (Python) finalizers on the thread and not just ones > from cycles? Because a lot of code probably expects them to be run as soon as the last visible ref disappears. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 556: Threaded garbage collection
On Fri, Sep 8, 2017, at 12:24, Larry Hastings wrote: > > > On 09/08/2017 12:04 PM, Benjamin Peterson wrote: > > - Why not run all (Python) finalizers on the thread and not just ones > > from cycles? > > Two reasons: > > 1. Because some code relies on the finalizer being called on the thread > where the last reference is dropped. This is usually the same > thread where the object was created. Some irritating third-party > libraries make demands on callers, e.g. "you can only interact with > / destroy X objects on your 'main thread'". This is often true of > windowing / GUI libraries. (For example, I believe this was true of > Direct3D, at least as of D3D8; it was also often true of Win32 USER > objects.) Is the requirement that the construction and destruction be literally on the same thread or merely non-concurrent? The GIL would provide the latter. > 2. Because there's so much work there. In the Gilectomy prototype, I > originally called all finalizers on the "reference count manager > commit thread", the thread that also committed increfs and decrefs. > The thread immediately fell behind on its queue and never caught > up. I changed the Gilectomy so objects needing finalization are > passed back to the thread where the last decref happened, for > finalization on that thread; this was pleasingly self-balancing. I'm only suggesting Python-level __del__ methods be run on the separate thread not general deallocation work. I would those would be few and far between. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 556: Threaded garbage collection
On Fri, Sep 8, 2017, at 12:13, Antoine Pitrou wrote: > On Fri, 08 Sep 2017 12:04:10 -0700 > Benjamin Peterson wrote: > > I like it overall. > > > > - I was wondering what happens during interpreter shutdown. I see you > > have that listed as a open issue. How about simply shutting down the > > finalization thread and not guaranteeing that finalizers are actually > > ever run à la Java? > > I don't know. People generally have expectations towards stuff being > finalized properly (especially when talking about files etc.). > Once the first implementation is devised, we will know more about > what's workable (perhaps we'll have to move _PyGC_Fini earlier in the > shutdown sequence? perhaps we'll want to switch back to serial mode > when shutting down?). Okay, I'm curious to know what ends up happening here then. > > > - Why not run all (Python) finalizers on the thread and not just ones > > from cycles? > > Because a lot of code probably expects them to be run as soon as the > last visible ref disappears. But this assumption is broken on PyPy and sometimes already by CPython, so I don't feel very bad moving away from it. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 556: Threaded garbage collection
On Fri, Sep 8, 2017 at 12:13 PM, Antoine Pitrou wrote: > On Fri, 08 Sep 2017 12:04:10 -0700 > Benjamin Peterson wrote: >> I like it overall. >> >> - I was wondering what happens during interpreter shutdown. I see you >> have that listed as a open issue. How about simply shutting down the >> finalization thread and not guaranteeing that finalizers are actually >> ever run à la Java? > > I don't know. People generally have expectations towards stuff being > finalized properly (especially when talking about files etc.). > Once the first implementation is devised, we will know more about > what's workable (perhaps we'll have to move _PyGC_Fini earlier in the > shutdown sequence? perhaps we'll want to switch back to serial mode > when shutting down?). PyPy just abandons everything when shutting down, instead of running finalizers. See the last paragraph of : http://doc.pypy.org/en/latest/cpython_differences.html#differences-related-to-garbage-collection-strategies So that might be a useful source of experience. On another note, I'm going to be that annoying person who suggests massively extending the scope of your proposal. Feel free to throw things at me or whatever. Would it make sense to also move signal handlers to run in this thread? Those are the other major source of nasty re-entrancy problems. -n -- Nathaniel J. Smith -- https://vorpus.org ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 549 v2: now titled Instance Descriptors
I've updated PEP 549 with a new title--"Instance Descriptors" is a better name than "Instance Properties"--and to clarify my rationale for the PEP. I've also updated the prototype with code cleanups and a new type: "collections.abc.InstanceDescriptor", a base class that allows user classes to be instance descriptors. //arry/ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 556: Threaded garbage collection
On 09/08/2017 12:30 PM, Benjamin Peterson wrote: On Fri, Sep 8, 2017, at 12:24, Larry Hastings wrote: On 09/08/2017 12:04 PM, Benjamin Peterson wrote: - Why not run all (Python) finalizers on the thread and not just ones from cycles? Two reasons: 1. Because some code relies on the finalizer being called on the thread where the last reference is dropped. This is usually the same thread where the object was created. Some irritating third-party libraries make demands on callers, e.g. "you can only interact with / destroy X objects on your 'main thread'". This is often true of windowing / GUI libraries. (For example, I believe this was true of Direct3D, at least as of D3D8; it was also often true of Win32 USER objects.) Is the requirement that the construction and destruction be literally on the same thread or merely non-concurrent? The GIL would provide the latter. Literally the same thread. My theory was that these clowntown external libraries are hiding important details in thread local storage, but I don't actually know. //arry/ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 556: Threaded garbage collection
On Fri, 8 Sep 2017 12:40:34 -0700 Nathaniel Smith wrote: > > PyPy just abandons everything when shutting down, instead of running > finalizers. See the last paragraph of : > http://doc.pypy.org/en/latest/cpython_differences.html#differences-related-to-garbage-collection-strategies > > So that might be a useful source of experience. CPython can be embedded in applications, though, and that is why we try to be a bit more thorough during the interpreter cleanup phase. > Would it make sense to also move signal handlers to run in this > thread? Those are the other major source of nasty re-entrancy > problems. See the "Non-goals" section in the PEP, they are already mentioned there :-) Note I don't think signal handlers are a major source of reentrancy problems, rather minor, since usually you don't try to do much in a signal handler. Signal handling is mostly a relic of 70s Unix design and it has less and less relevance in today's world, apart from the trivial task of telling a process to shut down. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 556: Threaded garbage collection
On Fri, Sep 8, 2017 at 12:52 PM Antoine Pitrou wrote: > On Fri, 8 Sep 2017 12:40:34 -0700 > Nathaniel Smith wrote: > > > > PyPy just abandons everything when shutting down, instead of running > > finalizers. See the last paragraph of : > > > http://doc.pypy.org/en/latest/cpython_differences.html#differences-related-to-garbage-collection-strategies > > > > So that might be a useful source of experience. > > CPython can be embedded in applications, though, and that is why we try > to be a bit more thorough during the interpreter cleanup phase. > Indeed. My gut feeling is that proposing to not run finalizers on interpreter shutdown is a non-starter and would get the pep rejected. We've previously guaranteed that they were run unless the process dies via an unhandled signal or calls os._exit() in CPython. -gps ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Memory bitmaps for the Python cyclic garbage collector
[Neil Schemenauer ] > Python objects that participate in cyclic GC (things like lists, dicts, > sets but not strings, ints and floats) have extra memory overhead. I > think it is possible to mostly eliminate this overhead. Also, while > the GC is running, this GC state is mutated, which destroys > copy-on-write optimizations. This change would mostly fix that > issue. > > All objects that participate in cyclic GC have the Py_TPFLAGS_HAVE_GC > bit set in their type. That causes an extra chunk of memory to be > allocated *before* the ob_refcnt struct member. This is the PyGC_Head > struct. > > The whole object looks like this in memory (PyObject pointer is at > arrow): > > union __gc_head *gc_next; > union __gc_head *gc_prev; > Py_ssize_t gc_refs; > --> > Py_ssize_t ob_refcnt > struct _typeobject *ob_type; > [rest of PyObject members] > > > So, 24 bytes of overhead on a 64-bit machine. The smallest Python > object that can have a pointer to another object (e.g. a single PyObject > * member) is 48 bytes. Removing PyGC_Head would cut the size of these > objects in half. > > Carl Shaprio questioned me today on why we use a double linked-list and > not the memory bitmap. I think the answer is that there is no good > reason. We use a double linked list only due to historical constraints > that are no longer present. Since you wrote this code to begin with, it will come back to you ;-) that the real purpose of the doubly-linked lists is to _partition_ (not just find) the tracked objects. Between collections, they're partitioned by generation, and within a collection equivalence classes are first merged (up through the oldest generation to be scanned in this run), and then temporarily partitioned internally in various ways (based on things like whether objects turn out to be reachable from outside, and whether they have finalizers). The linked list representation makes all the required operations cheap: iteration, merging classes, moving an object from one class to another, removing an object entirely _while_ iterating over its equivalence class. Don't know whether all that can be done efficiently with a bitmap representation instead. > Long ago, Python objects could be allocated using the system malloc or > other memory allocators. Since we could not control the memory > location, bitmaps would be inefficient. Today, we allocate all Python > objects via our own function. Python objects under a certain size are > allocated using our own malloc, obmalloc, and are stored in memory > blocks known "arenas". > > The PyGC_Head struct performs three functions. First, it allows the GC > to find all Python objects that will be checked for cycles (i.e. follow > the linked list). As above, the set of tracked objects is partitioned into more than one linked list. > Second, it stores a single bit of information to let > the GC know if it is safe to traverse the object, set with > PyObject_GC_Track(). ? An object is "tracked" if and only if it appears in _some_ doubly-linked list. There is no bit set (or cleared) for this. Untracking an object removes it entirely from whichever linked list it's in (leaving it in no linked lists), and tracking an object consists of adding it to the "generation 0" linked list. Unless the code has changed a whole lot recently. For clarity, the top N-1 bits of gc_refs (which you cover next) are also set to a special _PyGC_REFS_UNTRACKED constant when an object is untracked: """ /* True if the object is currently tracked by the GC. */ #define _PyObject_GC_IS_TRACKED(o) \ (_PyGC_REFS(o) != _PyGC_REFS_UNTRACKED) """ But I believe it could just as well check to see whether the object's gc_next is NULL. > Finally, it has a scratch area to compute the > effective reference count while tracing refs (gc_refs). As above, the top N-1 bits of that are also used between collections to record whether an object is tracked. The least significant bit of gc_refs now (not back when you or I were mucking with this code) records whether the object has a finalizer that has already been run, and that state needs to be preserved across gc runs. So that's another bit that would need to be stored somewhere else. > Here is a sketch of how we can remove the PyGC_Head struct for small > objects (say less than 512 bytes). Large objects or objects created by > a different memory allocator will still have the PyGC_Head overhead. > > * Have memory arenas that contain only objects with the > Py_TPFLAGS_HAVE_GC flag. Objects like ints, strings, etc will be > in different arenas, not have bitmaps, not be looked at by the > cyclic GC. > > * For those arenas, add a memory bitmap. The bitmap is a bit array that > has a bit for each fixed size object in the arena. The memory used by > the bitmap is a fraction of what is needed by PyGC_Head. E.g. an > arena that holds up to 1024 objects of 48 bytes in size would have a > bitmap of 1024 bits. If it's based on
Re: [Python-Dev] PEP 549 v2: now titled Instance Descriptors
On 09/08/2017 12:44 PM, Larry Hastings wrote: I've updated PEP 549 with a new title--"Instance Descriptors" is a better name than "Instance Properties"--and to clarify my rationale for the PEP. I've also updated the prototype with code cleanups and a new type: "collections.abc.InstanceDescriptor", a base class that allows user classes to be instance descriptors. I like the new title, I'm +0 on the PEP itself, and I have one correction for the PEP: we've had the ability to simulate module properties for ages: Python 2.7.6 (default, Oct 26 2016, 20:32:47) [GCC 4.8.4] on linux2 Type "help", "copyright", "credits" or "license" for more information. --> import module_class --> module_class.hello 'hello' --> module_class.hello = 'hola' --> module_class.hello 'hola' And the code: class ModuleClass(object): @property def hello(self): try: return self._greeting except AttributeError: return 'hello' @hello.setter def hello(self, value): self._greeting = value import sys sys.modules[__name__] = ModuleClass() I will admit I don't see what reassigning the __class__ attribute on a module did for us. -- ~Ethan~ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 549 v2: now titled Instance Descriptors
Assigning the module's __class__ means you can otherwise initialize your module instance normally--all your class needs to do is declare your properties or magic methods. If you overwrite the sys.modules entry for your module with a new instance of a custom subclass, all your module initialization needs to be done within--or propagated to--the new class or instance. //arry/ On 09/08/2017 01:45 PM, Ethan Furman wrote: On 09/08/2017 12:44 PM, Larry Hastings wrote: I've updated PEP 549 with a new title--"Instance Descriptors" is a better name than "Instance Properties"--and to clarify my rationale for the PEP. I've also updated the prototype with code cleanups and a new type: "collections.abc.InstanceDescriptor", a base class that allows user classes to be instance descriptors. I like the new title, I'm +0 on the PEP itself, and I have one correction for the PEP: we've had the ability to simulate module properties for ages: Python 2.7.6 (default, Oct 26 2016, 20:32:47) [GCC 4.8.4] on linux2 Type "help", "copyright", "credits" or "license" for more information. --> import module_class --> module_class.hello 'hello' --> module_class.hello = 'hola' --> module_class.hello 'hola' And the code: class ModuleClass(object): @property def hello(self): try: return self._greeting except AttributeError: return 'hello' @hello.setter def hello(self, value): self._greeting = value import sys sys.modules[__name__] = ModuleClass() I will admit I don't see what reassigning the __class__ attribute on a module did for us. -- ~Ethan~ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/larry%40hastings.org ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 549 v2: now titled Instance Descriptors
On Fri, Sep 8, 2017 at 1:45 PM, Ethan Furman wrote: > On 09/08/2017 12:44 PM, Larry Hastings wrote: > >> I've updated PEP 549 with a new title--"Instance Descriptors" is a better >> name than "Instance Properties"--and to >> clarify my rationale for the PEP. I've also updated the prototype with >> code cleanups and a new type: >> "collections.abc.InstanceDescriptor", a base class that allows user >> classes to be instance descriptors. > > > I like the new title, I'm +0 on the PEP itself, and I have one correction > for the PEP: we've had the ability to simulate module properties for ages: > > Python 2.7.6 (default, Oct 26 2016, 20:32:47) > [GCC 4.8.4] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > --> import module_class > --> module_class.hello > 'hello' > --> module_class.hello = 'hola' > --> module_class.hello > 'hola' > > And the code: > > class ModuleClass(object): > @property > def hello(self): > try: > return self._greeting > except AttributeError: > return 'hello' > @hello.setter > def hello(self, value): > self._greeting = value > > import sys > sys.modules[__name__] = ModuleClass() > > I will admit I don't see what reassigning the __class__ attribute on a > module did for us. If you have an existing package that doesn't replace itself in sys.modules, then it's difficult and risky to switch to that form -- don't think of toy examples, think of django/__init__.py or numpy/__init__.py. You have to rewrite the whole export logic, and you have to figure out what to do with things like submodules that import from the parent module before the swaparoo happens, you can get skew issues between the original module namespace and the replacement class namespace, etc. The advantage of the __class__ assignment trick (as compared to what we had before) is that it lets you easily and safely retrofit this kind of magic onto existing packages. -n -- Nathaniel J. Smith -- https://vorpus.org ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 549 v2: now titled Instance Descriptors
On 09/08/2017 01:45 PM, Ethan Furman wrote: I will admit I don't see what reassigning the __class__ attribute on a module did for us. Nathaniel Smith wrote: - > If you have an existing package that doesn't replace itself in > sys.modules, then it's difficult and risky to switch to that form -- > don't think of toy examples, think of django/__init__.py or > numpy/__init__.py. You have to rewrite the whole export logic, and you > have to figure out what to do with things like submodules that import > from the parent module before the swaparoo happens, you can get skew > issues between the original module namespace and the replacement class > namespace, etc. The advantage of the __class__ assignment trick (as > compared to what we had before) is that it lets you easily and safely > retrofit this kind of magic onto existing packages. Ah, that makes sense. Larry Hastings wrote: - > Assigning the module's __class__ means you can otherwise initialize your module instance > normally--all your class needs to do is declare your properties or magic methods. If you > overwrite the sys.modules entry for your module with a new instance of a custom subclass, > all your module initialization needs to be done within--or propagated to--the new class > or instance. So with the module level __class__ we have to use two steps to get what we want, and PEP 549 takes it back to one step. Hmm. In theory it sounds cool; on the other hand, the existing __class__ assignment method means all the magic is in one place in the module, not scattered around -- but that's a style issue. Thanks for the info! -- ~Ethan~ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] make multissltests
For your information, You can now automatically compile and check the ssl module with multiple versions of OpenSSL and LibreSSL. The multissltest script downloads tar.gz, compiles the source and installs headers + shared lib into a local directory. It takes rather long the first time because OpenSSL does not support parallel builds. Be prepared to wait an hour. After the script has downloaded and installed OpenSSL/LibreSSL, it recompiles the ssl and hashlib module with every version and runs the SSL-related test suite. make multissltests compiles and runs all tests make multisslcompile only compiles See ./python Tools/ssl/multissltests.py --help for additional options. For now the script covers: * OpenSSL 0.9.8zc * OpenSSL 0.9.8zh * OpenSSL 1.0.1u * OpenSSL 1.0.2 * OpenSSL 1.0.2l * OpenSSL 1.1.0f * LibreSSL 2.3.10 * LibreSSL 2.4.5 * LibreSSL 2.5.3 * LibreSSL 2.5.5 Christian ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pygments in PEPs?
For some unknown value of "soon". :-( On Fri, Sep 8, 2017 at 7:59 AM, Ivan Levkivskyi wrote: > I already made a PR that would highlight code in PEPs half-year ago, but > it was rejected with the reason that they will be moved to RtD soon. > > -- > Ivan > > 8 Вер 2017 16:56 "Antoine Pitrou" пише: > >> >> Hi, >> >> Is it possible to install pygments on the machine which builds and >> renders PEPs? It would allow syntax highlighting in PEPs, making >> example code more readable. >> >> Regards >> >> Antoine. >> >> ___ >> Python-Dev mailing list >> [email protected] >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: https://mail.python.org/mailman/options/python-dev/levkivsky >> i%40gmail.com >> > > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > guido%40python.org > > -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pygments in PEPs?
On Fri, 8 Sep 2017 16:59:57 +0200 Ivan Levkivskyi wrote: > I already made a PR that would highlight code in PEPs half-year ago, but it > was rejected with the reason that they will be moved to RtD soon. Perhaps we can revive that PR? Browsing through old peps PRs, I can see that other people were already bitten by the lack of pygments. (more generally, it would be nice to build the PEPs with Sphinx so as to get the whole gamut of useful markup that comes with it, but I guess that's much more involved than a simple "pip install pygments" inside the build environment) Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pygments in PEPs?
> > For some unknown value of "soon". :-( Well as soon as these tasks are done: https://github.com/python/peps/projects/1 Mariatta Wijaya ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 557: Data Classes
On 2017-09-08 07:57, Eric V. Smith wrote: I've written a PEP for… Apologies for the following list dumb questions and bikesheds: - 'Classes can be thought of as "mutable namedtuples with defaults".' - A C/C++ (struct)ure sounds like a simpler description that many more would understand. - dataclass name: - class, redundant - data, good but very common - struct, used? - Record? (best I could come up with) - Source needs blanks between functions, hard to read. - Are types required? Maybe an example or two with Any? - Intro discounts inheritance and metaclasses as "potentially interfering", but unclear why that would be the case. Inheritance is easy to override, metaclasses not sure? - Perhaps mention ORMs/metaclass approach, as prior art: https://docs.djangoproject.com/en/dev/topics/db/models/ - Perhaps mention Kivy Properties, as prior art: https://kivy.org/docs/api-kivy.properties.html - For mutable default values: @dataclass class C: x: list # = field(default_factory=list) Could it detect list as a mutable class "type" and set it as a factory automatically? The PEP/bug #3 mentions using copy, but that's not exactly what I'm asking above. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pygments in PEPs?
On 9 September 2017 at 00:14, Antoine Pitrou wrote: > On Fri, 8 Sep 2017 16:59:57 +0200 > Ivan Levkivskyi wrote: > > > I already made a PR that would highlight code in PEPs half-year ago, but > it > > was rejected with the reason that they will be moved to RtD soon. > > Perhaps we can revive that PR? > Just for the reference here is the PR https://github.com/python/pythondotorg/pull/1063 It is indeed quite simple. Unfortunately, right now (and next one-two weeks) I will be not able to work on this PR if any additional changes will be needed. But I will be happy if someone will continue with this (temporary) solution. -- Ivan ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 557: Data Classes
On 9/8/17 3:20 PM, Mike Miller wrote: On 2017-09-08 07:57, Eric V. Smith wrote: I've written a PEP for… Apologies for the following list dumb questions and bikesheds: - 'Classes can be thought of as "mutable namedtuples with defaults".' - A C/C++ (struct)ure sounds like a simpler description that many more would understand. Yes, other people have pointed out that this might not be the best "elevator pitch" example. I'm thinking about it. - dataclass name: - class, redundant - data, good but very common - struct, used? - Record? (best I could come up with) There was a bunch of discussions on this. We're delaying the name bikeshedding for later (and maybe never). - Source needs blanks between functions, hard to read. It's supposed to be hard to read! You're just supposed to think "am I glad I don't have to read or write that". But I'll look at it. - Are types required? Annotations are required, the typing module is not. Maybe an example or two with Any? I'd rather leave it like it is: typing is referenced only once, for ClassVar. - Intro discounts inheritance and metaclasses as "potentially interfering", but unclear why that would be the case. Inheritance is easy to override, metaclasses not sure? I don't really want to get in to the history of why people don't like inheritance, single and multi. Or how metaclass magic can make life difficult. I just want to point out that Data Classes don't interfere at all. - Perhaps mention ORMs/metaclass approach, as prior art: https://docs.djangoproject.com/en/dev/topics/db/models/ - Perhaps mention Kivy Properties, as prior art: https://kivy.org/docs/api-kivy.properties.html Those are all good. Thanks. - For mutable default values: @dataclass class C: x: list # = field(default_factory=list) Could it detect list as a mutable class "type" and set it as a factory automatically? The PEP/bug #3 mentions using copy, but that's not exactly what I'm asking above. The problem is: how do you know what's a mutable type? There's no general way to know. The behavior in the PEP is just mean to stop the worst of it. I guess we could have an option that says: call the type to create a new, empty instance. @dataclass class C: x: list = field(default_type_is_factory=True) Thanks for the critical reading and your comments. I'm going to push a new version early next week, when I get back from traveling. Eric. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 557: Data Classes
I think it would be useful to write 1-2 sentences about the problem with inheritance -- in that case you pretty much have to use a metaclass, and the use of a metaclass makes life harder for people who want to use their own metaclass (since metaclasses don't combine without some manual intervention). On Fri, Sep 8, 2017 at 3:40 PM, Eric V. Smith wrote: > On 9/8/17 3:20 PM, Mike Miller wrote: > >> >> On 2017-09-08 07:57, Eric V. Smith wrote: >> >>> I've written a PEP for… >>> >> >> Apologies for the following list dumb questions and bikesheds: >> >> >> - 'Classes can be thought of as "mutable namedtuples with defaults".' >> >> - A C/C++ (struct)ure sounds like a simpler description that many more >> would understand. >> > > Yes, other people have pointed out that this might not be the best > "elevator pitch" example. I'm thinking about it. > > - dataclass name: >> >> - class, redundant >> - data, good but very common >> - struct, used? >> - Record? (best I could come up with) >> > > There was a bunch of discussions on this. We're delaying the name > bikeshedding for later (and maybe never). > > - Source needs blanks between functions, hard to read. >> > > It's supposed to be hard to read! You're just supposed to think "am I glad > I don't have to read or write that". But I'll look at it. > > - Are types required? >> > > Annotations are required, the typing module is not. > > Maybe an example or two with Any? >> > > I'd rather leave it like it is: typing is referenced only once, for > ClassVar. > > - Intro discounts inheritance and metaclasses as "potentially interfering", >> but unclear why that would be the case. >> Inheritance is easy to override, metaclasses not sure? >> > > I don't really want to get in to the history of why people don't like > inheritance, single and multi. Or how metaclass magic can make life > difficult. I just want to point out that Data Classes don't interfere at > all. > > - Perhaps mention ORMs/metaclass approach, as prior art: >> >> https://docs.djangoproject.com/en/dev/topics/db/models/ >> >> - Perhaps mention Kivy Properties, as prior art: >> >> https://kivy.org/docs/api-kivy.properties.html >> > > Those are all good. Thanks. > > - For mutable default values: >> >> @dataclass >> class C: >> x: list # = field(default_factory=list) >> >> Could it detect list as a mutable class "type" and set it as a factory >> automatically? >> >> The PEP/bug #3 mentions using copy, but that's not exactly what I'm >> asking >> above. >> > > The problem is: how do you know what's a mutable type? There's no general > way to know. The behavior in the PEP is just mean to stop the worst of it. > > I guess we could have an option that says: call the type to create a new, > empty instance. > > @dataclass > class C: > x: list = field(default_type_is_factory=True) > > > Thanks for the critical reading and your comments. I'm going to push a new > version early next week, when I get back from traveling. > > Eric. > > > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido% > 40python.org > -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 554 v2 (new "interpreters" module)
I've updated the PEP in response to some excellent feedback. Thanks to all who helped. Most notably, I've added a basic object passing mechanism. I've included the PEP below. Please let me know what you think. Thanks! -eric PEP: 554 Title: Multiple Interpreters in the Stdlib Author: Eric Snow Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 2017-09-05 Python-Version: 3.7 Post-History: Abstract This proposal introduces the stdlib ``interpreters`` module. It exposes the basic functionality of subinterpreters that already exists in the C-API. Each subinterpreter runs with its own state (see ``Interpreter Isolation`` below). The module will be "provisional", as described by PEP 411. Rationale = Running code in multiple interpreters provides a useful level of isolation within the same process. This can be leveraged in number of ways. Furthermore, subinterpreters provide a well-defined framework in which such isolation may extended. CPython has supported subinterpreters, with increasing levels of support, since version 1.5. While the feature has the potential to be a powerful tool, subinterpreters have suffered from neglect because they are not available directly from Python. Exposing the existing functionality in the stdlib will help reverse the situation. This proposal is focused on enabling the fundamental capability of multiple isolated interpreters in the same Python process. This is a new area for Python so there is relative uncertainly about the best tools to provide as companions to subinterpreters. Thus we minimize the functionality we add in the proposal as much as possible. Concerns * "subinterpreters are not worth the trouble" Some have argued that subinterpreters do not add sufficient benefit to justify making them an official part of Python. Adding features to the language (or stdlib) has a cost in increasing the size of the language. So it must pay for itself. In this case, subinterpreters provide a novel concurrency model focused on isolated threads of execution. Furthermore, they present an opportunity for changes in CPython that will allow simulateous use of multiple CPU cores (currently prevented by the GIL). Alternatives to subinterpreters include threading, async, and multiprocessing. Threading is limited by the GIL and async isn't the right solution for every problem (nor for every person). Multiprocessing is likewise valuable in some but not all situations. Direct IPC (rather than via the multiprocessing module) provides similar benefits but with the same caveat. Notably, subinterpreters are not intended as a replacement for any of the above. Certainly they overlap in some areas, but the benefits of subinterpreters include isolation and (potentially) performance. In particular, subinterpreters provide a direct route to an alternate concurrency model (e.g. CSP) which has found success elsewhere and will appeal to some Python users. That is the core value that the ``interpreters`` module will provide. * "stdlib support for subinterpreters adds extra burden on C extension authors" In the ``Interpreter Isolation`` section below we identify ways in which isolation in CPython's subinterpreters is incomplete. Most notable is extension modules that use C globals to store internal state. PEP 3121 and PEP 489 provide a solution for most of the problem, but one still remains. [petr-c-ext]_ Until that is resolved, C extension authors will face extra difficulty to support subinterpreters. Consequently, projects that publish extension modules may face an increased maintenance burden as their users start using subinterpreters, where their modules may break. This situation is limited to modules that use C globals (or use libraries that use C globals) to store internal state. Ultimately this comes down to a question of how often it will be a problem in practice: how many projects would be affected, how often their users will be affected, what the additional maintenance burden will be for projects, and what the overall benefit of subinterpreters is to offset those costs. The position of this PEP is that the actual extra maintenance burden will be small and well below the threshold at which subinterpreters are worth it. Proposal The ``interpreters`` module will be added to the stdlib. It will provide a high-level interface to subinterpreters and wrap the low-level ``_interpreters`` module. The proposed API is inspired by the ``threading`` module. The module provides the following functions: ``list()``:: Return a list of all existing interpreters. ``get_current()``:: Return the currently running interpreter. ``get_main()``:: Return the main interpreter. ``create()``:: Initialize a new Python interpreter and return it. The interpreter will be created in the current thread and will remain idle until something is run in it. T
Re: [Python-Dev] PEP 557: Data Classes
On 2017-09-08 07:57, Eric V. Smith wrote: I've written a PEP for… Apologies for the following list dumb questions and bikesheds: - 'Classes can be thought of as "mutable namedtuples with defaults".' - A C/C++ (struct)ure sounds like a simpler description that many more would understand. - dataclass name: - class, redundant - data, good but very common - struct, used? - Record? (best I could come up with) - Source needs blanks between functions, hard to read. - Are types required? Maybe an example or two with Any? - Intro discounts inheritance and metaclasses as "potentially interfering", but unclear why that would be the case. Inheritance is easy to override, metaclasses not sure? - Perhaps mention ORMs/metaclass approach, as prior art: https://docs.djangoproject.com/en/dev/topics/db/models/ - Perhaps mention Kivy Properties, as prior art: https://kivy.org/docs/api-kivy.properties.html - For mutable default values: @dataclass class C: x: list # = field(default_factory=list) Could it detect list as a mutable class "type" and set it as a factory automatically? The PEP/bug #3 mentions using copy, but that's not exactly what I'm asking above. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 557: Data Classes
On 9 September 2017 at 01:00, Guido van Rossum wrote: > I think it would be useful to write 1-2 sentences about the problem with > inheritance -- in that case you pretty much have to use a metaclass, > > It is not the case now. I think __init_subclass__ has almost the same possibilities as a decorator, it just updates an already created class and can add some methods to it. This is a more subtle question, these two for example would be equivalent: from dataclass import Data, Frozen class Point(Frozen, Data): x: int y: int and from dataclass import dataclass @dataclass(frozen=True) class Point: x: int y: int But the problem with inheritance based pattern is that it cannot support automatic addition of __slots__. Also I think a decorator will be easier to maintain. But on the other hand I think inheritance based scheme is a bit more readable. -- Ivan ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 554 v2 (new "interpreters" module)
On Fri, Sep 08, 2017 at 04:04:27PM -0700, Eric Snow wrote: > * "stdlib support for subinterpreters adds extra burden > on C extension authors" > > In the ``Interpreter Isolation`` section below we identify ways in > which isolation in CPython's subinterpreters is incomplete. Most > notable is extension modules that use C globals to store internal > state. PEP 3121 and PEP 489 provide a solution for most of the > problem, but one still remains. [petr-c-ext]_ Until that is resolved, > C extension authors will face extra difficulty to support > subinterpreters. It's a bit of a hassle, and the enormous slowdown in some of the existing solutions is really a no go [1]. In the case of _decimal, the tls-context is already subinterpreter safe and reasonable fast due to caching. The most promising model to me is to put *all* globals in a tls structure and cache the whole structure. Extrapolating from my experiences with the context, this might have a slowdown of "only" 4%. Still, the argument "who uses subinterpreters?" of course still remains. Stefan Krah [1] I'm referring to the slowdown of heaptypes + module state. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Memory bitmaps for the Python cyclic garbage collector
Tim Peters wrote: In that case, it's because Python _does_ mutate the objects' refcount members under the covers, and so the OS ends up making fresh copies of the memory anyway. Has anyone ever considered addressing that by moving the refcounts out of the objects and keeping them somewhere else? -- Greg ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Memory bitmaps for the Python cyclic garbage collector
[Tim] >> In that case, it's because Python >> _does_ mutate the objects' refcount members under the covers, and so >> the OS ends up making fresh copies of the memory anyway. [Greg Ewing ] > Has anyone ever considered addressing that by moving the > refcounts out of the objects and keeping them somewhere > else? Not that I know of. I know Larry Hastings was considering doing it as part of his experiments with removing the GIL, but that had nothing to do with reducing cross-process copy-on-write surprises (it had to do with "batching" refcount operations to eliminate a need for fine-grained locking). As-is, I'd say it's "a feature" that the refcount is part of the object header. Ref count manipulations are very frequent, and as part of the object header a refcount tends to show up in cache lines "for free" as a side effect of accessing the object's type pointer. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 554 v2 (new "interpreters" module)
Le 09/09/2017 à 01:28, Stefan Krah a écrit : > On Fri, Sep 08, 2017 at 04:04:27PM -0700, Eric Snow wrote: >> * "stdlib support for subinterpreters adds extra burden >> on C extension authors" >> >> In the ``Interpreter Isolation`` section below we identify ways in >> which isolation in CPython's subinterpreters is incomplete. Most >> notable is extension modules that use C globals to store internal >> state. PEP 3121 and PEP 489 provide a solution for most of the >> problem, but one still remains. [petr-c-ext]_ Until that is resolved, >> C extension authors will face extra difficulty to support >> subinterpreters. > > It's a bit of a hassle, and the enormous slowdown in some of the existing > solutions is really a no go [1]. > > In the case of _decimal, the tls-context is already subinterpreter safe > and reasonable fast due to caching. > > > The most promising model to me is to put *all* globals in a tls structure > and cache the whole structure. Extrapolating from my experiences with the > context, this might have a slowdown of "only" 4%. > > > Still, the argument "who uses subinterpreters?" of course still remains. For now, nobody. But if we expose it and web frameworks manage to create workers as fast as multiprocessing and as cheap as threading, you will find a lot of people starting to want to use it. We can't know until we got the toy to play with. > > > > Stefan Krah > > > [1] I'm referring to the slowdown of heaptypes + module state. > > > > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/desmoulinmichel%40gmail.com > ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 553: Built-in debug()
On 9/7/2017 10:45 PM, Barry Warsaw wrote: On Sep 7, 2017, at 16:19, Terry Reedy wrote: I think breakpoint() should have a db= parameter so one can select a debugger in one removable line. The sys interface is more useful for IDEs to change the default, possible with other args (like breakpoints and colors) bound to the callable. I’m skeptical about that. I think any particular user is going to overwhelmingly use the same debugger, so having to repeat themselves every time they want to enter the debugger is going to get tedious fast. I know it would annoy *me* if I had to tell it to use pdb every time I wrote `breakpoint()`, and I’m almost never going to use anything else. OK I’m also not sure what useful semantics for `db` would be. E.g. what specifically would you set `db` to in order to invoke idle or tkdb (‘gdb’ would be an unfortunate name I think, given the popular existing GNU Debugger ;). I will stick with tkdb until there is a shed to paint. A somewhat separate point: the name breakpoint() is slightly misleading, which has consequences if it is (improperly) called more than once. While breakpoint() acts as a breakpoint, what it does (at least in the default pdb case) is *initialize* and start a *new* debugger, possibly after an import. Re-importing a module is no big deal. Replacing an existing debugger with a *new* one, and tossing away all defined aliases and breakpoints and Bdb's internal caches, is. It is likely not what most people would want or expect. I think it more likely that people will call breakpoint() multiple times than they would, for instance, call pdb.set_trace() multiple times. Multiple calls to pdb.set_trace() is fairly common in practice today, so I’m not terribly concerned about it. I am slightly surprised, but I guess if one does not set anything, one would not lose anything. With a gui debugger, having one window go and another appear might be additionally annoying. If the first window is not immediately GCed, having two windows would be confusing. Perhaps breakpoint() could be made a no-op after the first call. Your sys.breakpointhook could easily implement that, with a much better user experience than what built-in breakpoint() could do anyway. Now that I know multiple breakpoint()s are likely, I definitely would. -- Terry Jan Reedy ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 554 v2 (new "interpreters" module)
On Fri, Sep 8, 2017 at 8:54 PM, Michel Desmoulin wrote: > > Le 09/09/2017 à 01:28, Stefan Krah a écrit : >> Still, the argument "who uses subinterpreters?" of course still remains. > > For now, nobody. But if we expose it and web frameworks manage to create > workers as fast as multiprocessing and as cheap as threading, you will > find a lot of people starting to want to use it. To temper expectations a bit here, it sounds like the first version might be more like: as slow as threading (no multicore), as expensive as multiprocessing (no shared memory), and -- on Unix -- slower to start than either of them (no fork). -n -- Nathaniel J. Smith -- https://vorpus.org ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
