[Python-Dev] Re: PEP 611 -- why limit coroutines and classes?

2019-12-12 Thread Antoine Pitrou 
On Wed, 11 Dec 2019 23:17:48 -0500
Kyle Stanley  wrote:
> 
> TL;DR: It's definitely possible to have more than one client per TCP port.

Thanks for correcting me.  Not sure why, but I appear to make that
mistake once every couple years.

> I'm have no idea what the
> realistic maximum limit of global FDs would be for most modern servers
> though, but here's the upper bound limit on Linux kernel 5.3.13:
> 
> [aeros:~]$ cat /proc/sys/fs/file-max
> 9223372036854775807

Looks like 2**63 - 1 to me :-)

> I recall reading somewhere that per additional 100 file descriptors, it
> requires approximately 1MB of main memory.

More than file descriptors per se, what's relevant here is the
per-TCP connection overhead (unless you're interested in keeping closed
TCP sockets around?).  Which I guess is related to the
latency*bandwidth product.

Regards

Antoine.
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/TC5PANOZ5ROSLMGXMNXB2XSDTD72BIZ6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 611 -- why limit coroutines and classes?

2019-12-12 Thread Antoine Pitrou 
On Thu, 12 Dec 2019 00:56:41 -0500
Kyle Stanley  wrote:
> 
> IMO, the max
> number of concurrent transactions that the a database system can handle at
> once is a very clear application of Moore's Law.

I'm not quite sure that's the case.  I think in reality Moore's Law has
also helped databases become much larger and more complex, so it's not
clear-cut.

But in any case, Moore's Law is slowly dying, so I'm not sure that's a
good argument for envisioning databases supporting 1M+
concurrent transactions in 10 years.  Of course, never say never :-)

> Note: Although coroutines were already dropped from PEP 611, I felt that
> this response was still worthwhile to write. I suspect that the topic of
> "coroutine object limits" is likely to come up again in the future.

Right.

Best regards

Antoine.
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/EEFA6JK7Q2EKCANADBPLQGU6ZZTP76P2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Feedback on PEP 611 (so far) from the Steering Council

2019-12-12 Thread Nick Coghlan 
On Wed., 11 Dec. 2019, 9:16 pm Mark Shannon,  wrote:

>
>
> On 11/12/2019 12:04 am, Barry Warsaw wrote:
> > The Python Steering Council discussed PEP 611 at today’s meeting.  Here
> is our feedback so far:
> >
> > * The Steering Council reserves the right to be the BDFL-Delegate for
> this PEP
> >
> > * The PEP should clearly delineate two aspects:
> >
> >- A language generic part (i.e. applies to all implementations of
> Python) which exposes implementation limits in a way that Python code can
> read at runtime.  Think sys.getrecursionlimit(), sys.maxsize, and
> sys.implementation
> >
> >- An implementation-specific part where the PEP would propose
> specific limits for the CPython implementation.  Other implementations of
> the Python language would be free to adjust such limits up or down as they
> deem appropriate.
> >
> > * We encourage the PEP authors and proponents to gather actual
> performance data that can be used to help us evaluate whether this PEP is a
> good idea or not.
>
> I think that judging this on numbers from a handful of optimizations
> would be a mistake.
>
> There are an infinite number of potential optimizations, so it is
> impossible to put numbers on as yet undiscovered optimization.
> Conversely it is impossible to perfectly predict what limits might
> restrict possible future applications.
>

It is however likely possible to document and expose at runtime the
existing limits, regardless of what they may be.

Thus the suggestion in Barry's post to separate the two activities of
"define and expose the limits" and "potentially lower the limits in CPython
specifically".

Cheers,
Nick.


>
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/2LNUKNZKPUWQXLHNSQLRB2D6XRKYNXCQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Why limit coroutines?

2019-12-12 Thread Petr Viktorin 

On 2019-12-11 22:45, Antoine Pitrou wrote:

On Mon, 9 Dec 2019 21:42:36 -0500
Kyle Stanley  wrote:


(b) Why limit coroutines? It's just another Python object and has no

operating resources associated with it. Perhaps your definition of
coroutine is different, and you are thinking of OS threads?

This was my primary concern with the proposed PEP. At the moment, it's
rather trivial to create one million coroutines, and the total memory taken
up by each individual coroutine object is very minimal compared to each OS
thread.

There's also a practical use case for having a large number of coroutine
objects, such as for asynchronously:

[...]

2) Sending a large number of concurrent database transactions to run on a
cluster of database servers.


1M concurrent database transactions?  Does that sound reasonable at
all?  Your database administrator probably won't like you.


Right. Instead, you use a pool of DB connections, making each coroutine 
await its turn to talk to the DB.

You can still have a million coroutines waiting.

(Or as Guido put it, a coroutine "just another Python object and has no 
operating resources associated with it". I still sense some confusion 
around what was meant there.)

___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/3IJPT5OH6ISHFTSRHSJU5E6DA5WJCJ74/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please be more precise when commenting on PEP 611.

2019-12-12 Thread Mark Shannon 



On 11/12/2019 3:56 pm, Guido van Rossum wrote:
On Wed, Dec 11, 2019 at 2:14 AM Mark Shannon > wrote:



If the status quo were the result of considered decision, then it would
of course need considerable justification.
If, as is the case here, the status quo is a result of historical
accident and/or implementation details, then I think a weaker
justification is OK.


Whoa. The lack of limits in the status quo (no limits on various things 
except indirectly, through available memory) is most definitely the 
result of an intentional decision. "No arbitrary limits" was part of 
Python's initial design philosophy. We didn't always succeed (parse tree 
depth and call recursion depth come to mind) but that was definitely the 
philosophy. It was in contrast to other languages that did have 
arbitrary limits (e.g. Pascal's 255-char limit on strings, or C's 
machine-dependent integer size) and in several cases the implementation 
went through great lengths to avoid limits (e.g. we could have avoided a 
lot of dynamic memory (re)allocation if we'd limited line lengths or 
file sizes).


Sure, there are few *designed* in limits in Python at the moment. But in 
reality Python has lots of limits.


Because CPython is the reference implementation, Python is what CPython 
does; and CPython has lots of limits.


Classes.


One suggested way, by which a million classes might be created was using 
namedtuple:


>>> from collections import namedtuple
>>> l =  [ namedtuple(f"nt{i}", "a,b,c,d,e,f,g,h,i,j") for i in 
range(1000_000) ]


This takes 6Gb of resident memory on my machine (3.9alpha).
Most machines have roughly 4Gb per core (if you buy a bigger machine you 
get more CPUs and more RAM, roughly in that proportion).
So RAM effectively limits the number of classes to less than one million 
already (assuming you want to use your cores efficiently).
Imposing a limit on the number of classes and reducing memory footprint 
would allow more classes in practice, and would allow a lot more objects 
in more sensible programs.


Instructions per code-object


CPython will crash if this is in the 2**31 to 2**32 range as the 
compiler treats addresses as unsigned, but the interpreter treats them 
as signed.
Obviously this can be fixed, but it is an example of the sort of lurking 
bug that implicit limits can cause. These things are very expensive to 
test as you need a machine with hundreds of gigabytes of memory.




Explicit limits are much easier to test. Does code outside the limit 
fail in the expected fashion and code just under the limit work correctly?


What I want, is to allow more efficient use of resources without 
inconveniently low or unspecified limits. There will always be some 
limits on finite machines. If they aren't specified, they still exist, 
we just don't know what they are or how they will manifest themselves.


Cheers,
Mark.




You have an extreme need to justify why we should change now. "An 
infinite number of potential optimizations" does not cut it.

--
--Guido van Rossum (python.org/~guido )
/Pronouns: he/him //(why is my pronoun here?)/ 


___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/XVSDNGJ7YMUHG2XV5TGBY7WJ56KOEHDK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Python 3.8 error?

2019-12-12 Thread Petr Viktorin 
I'm quite interested in the rest of the story here. PySide is probably 
the biggest open-source user of the limited API, so IMO it is relevant 
to this list.


On 2019-12-11 23:48, Christian Tismer wrote:

Hi all,

Sorry for the noise, I was wrong, and I retract.
I was somehow mislead and hunted a phantom.


Does that mean that there was never a problem?
Or that a problem is still there, but we don't know if it's in CPython 
or PySide or their interaction?

Or was the problem solved in PySide?
Do we need to document something better, or just aware of some caveat?



Best - Chris


On 10.12.19 00:29, Christian Tismer wrote:

On 09.12.19 23:26, Nick Coghlan wrote:



On Tue., 10 Dec. 2019, 5:17 am MRAB, mailto:[email protected]>> wrote:

 On 2019-12-09 18:22, Christian Tismer wrote:
 >
 >
 > Hi Nick,
 >
 > after staring long at the code, I fount something funny in
 > typeobject.c #286 ff:
 >
 >
 > static void
 > type_mro_modified(PyTypeObject *type, PyObject *bases) {
 >      /*
 >         Check that all base classes or elements of the MRO of type are
 >         able to be cached.  This function is called after the base
 >         classes or mro of the type are altered.
 >
 >         Unset HAVE_VERSION_TAG and VALID_VERSION_TAG if the type
 >         has a custom MRO that includes a type which is not officially
 >         super type, or if the type implements its own mro() method.
 >
 >         Called from mro_internal, which will subsequently be called on
 >         each subclass when their mro is recursively updated.
 >       */
 >      Py_ssize_t i, n;
 >      int custom = (Py_TYPE(type) != &PyType_Type);
 >      int unbound;
 >      PyObject *mro_meth = NULL;
 >      PyObject *type_mro_meth = NULL;
 >
 >      if (!PyType_HasFeature(type, Py_TPFLAGS_HAVE_VERSION_TAG))
 >          return;
 >
 >      if (custom) {
 >          _Py_IDENTIFIER(mro);
 >          mro_meth = lookup_maybe_method(
 >              (PyObject *)type, &PyId_mro, &unbound);
 >          if (mro_meth == NULL)
 >              goto clear;
 >          type_mro_meth = lookup_maybe_method(
 >              (PyObject *)&PyType_Type, &PyId_mro, &unbound);
 >          if (type_mro_meth == NULL)
 >              goto clear;
 >          if (mro_meth != type_mro_meth)
 >              goto clear;
 >          Py_XDECREF(mro_meth);
 >          Py_XDECREF(type_mro_meth);
 >      }
 >
 >
 > Look at the "if (custom)" clause.
 > "mro_meth = lookup_maybe_method(" uses lookup_maybe_method which
 > gives a borrowed reference. The same holds for "type_mro_meth".
 >
 > But then both are decreffed, which IMHO is not correct.
 >
 Look at what happens at the label "clear": it DECREFs them.

 If mro_meth != NULL or mro_meth != type_mro_meth, they'll get DECREFed
 at "clear".


I believe Christian's point is that this entire "if (custom) {" branch
looks suspicious, as it assumes "lookup_maybe_method" will increment the
refcount on the returned object. If that assumption is incorrect, we're
going to get DECREFs without a preceding INCREF.

The specific code path is also obscure enough that it's plausible the
test suite may not currently cover it (as it requires doing something
that calls "type_mro_modified" on a type with a custom metaclass).


Thanks Nick for this nice analysis. And it's exactly this codepath that
is taken in the case of PySide: custom types all the time :-)

what-a-relief - ly y'rs -- Chris


___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/AONFRH53HED5RN7VVT375JDMORGKEFQV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 611: The one million limit.

2019-12-12 Thread Rhodri James 

On 11/12/2019 21:35, Antoine Pitrou wrote:

In any case, this will have to be judged on benchmark numbers, once
Mark (or someone else) massages the interpreter to experiment with
those runtime memory footprint reductions.


This I absolutely agree with.  Without evidence we're just waving our 
prejudices and varied experiences at one another (for example, my 
experience that the "modern processors" you were talking about have 
massive caches, so memory access isn't as much of an issue as you might 
think vs your experience that memory access matters more than doing a 
shift-and-mask all the time).  I've seen no hard evidence of any actual 
improvement of any size, and without that there really isn't a decision 
to be made.


--
Rhodri James *-* Kynesim Ltd
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/N4UNHOHCQHCOIUEFQKFJMKTKIAEHTM52/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Python 3.8 error?

2019-12-12 Thread Christian Tismer 
On 12.12.19 13:04, Petr Viktorin wrote:
> I'm quite interested in the rest of the story here. PySide is probably
> the biggest open-source user of the limited API, so IMO it is relevant
> to this list.


I guess that PyQt5 is a similar huge user, and it may be that they have
the same problem, since there is no 5.14 version yet ;-)


> On 2019-12-11 23:48, Christian Tismer wrote:
>> Hi all,
>>
>> Sorry for the noise, I was wrong, and I retract.
>> I was somehow mislead and hunted a phantom.
> 
> Does that mean that there was never a problem?
> Or that a problem is still there, but we don't know if it's in CPython
> or PySide or their interaction?
> Or was the problem solved in PySide?
> Do we need to document something better, or just aware of some caveat?


The problem is still there.
When PySide creates a new type, then PyType_Ready digs into the new
type, fetches the "mro()" method and uses the
Py_TPFLAGS_METHOD_DESCRIPTOR flag to decide how to handle it.

When I leave the flag in, we crash. So this is the current fix,
and I have no idea why this affects us at all. Here is the last change
to SbkObjectTypeTpNew() #503:

// The meta type creates a new type when the Python programmer
extends a wrapped C++ class.
auto type_new = reinterpret_cast(PyType_Type.tp_new);

// PYSIDE-939: This is a temporary patch that circumvents the problem
// with Py_TPFLAGS_METHOD_DESCRIPTOR until this is finally solved.
// PyType_Ready uses mro(). We need to temporarily remove the flag
from it's type.
// We cannot use PyMethodDescr_Type since it is not exported by
Python 2.7 .
static PyTypeObject *PyMethodDescr_TypePtr = Py_TYPE(
PyObject_GetAttr(reinterpret_cast(&PyType_Type),
Shiboken::PyName::mro()));
auto hold = PyMethodDescr_TypePtr->tp_flags;
PyMethodDescr_TypePtr->tp_flags &= ~Py_TPFLAGS_METHOD_DESCRIPTOR;
auto *newType = reinterpret_cast(type_new(metatype,
args, kwds));
PyMethodDescr_TypePtr->tp_flags = hold;


I am still not sure where the bug is and who has it.
My understanding is that this flag should have no impact on PySide
at all, but it has.

Thanks for taking care -- Chris


>> On 10.12.19 00:29, Christian Tismer wrote:
>>> On 09.12.19 23:26, Nick Coghlan wrote:


 On Tue., 10 Dec. 2019, 5:17 am MRAB, >>> > wrote:

  On 2019-12-09 18:22, Christian Tismer wrote:
  >
  >
  > Hi Nick,
  >
  > after staring long at the code, I fount something funny in
  > typeobject.c #286 ff:
  >
  >
  > static void
  > type_mro_modified(PyTypeObject *type, PyObject *bases) {
  >      /*
  >         Check that all base classes or elements of the MRO of
 type are
  >         able to be cached.  This function is called after the
 base
  >         classes or mro of the type are altered.
  >
  >         Unset HAVE_VERSION_TAG and VALID_VERSION_TAG if the type
  >         has a custom MRO that includes a type which is not
 officially
  >         super type, or if the type implements its own mro()
 method.
  >
  >         Called from mro_internal, which will subsequently be
 called on
  >         each subclass when their mro is recursively updated.
  >       */
  >      Py_ssize_t i, n;
  >      int custom = (Py_TYPE(type) != &PyType_Type);
  >      int unbound;
  >      PyObject *mro_meth = NULL;
  >      PyObject *type_mro_meth = NULL;
  >
  >      if (!PyType_HasFeature(type, Py_TPFLAGS_HAVE_VERSION_TAG))
  >          return;
  >
  >      if (custom) {
  >          _Py_IDENTIFIER(mro);
  >          mro_meth = lookup_maybe_method(
  >              (PyObject *)type, &PyId_mro, &unbound);
  >          if (mro_meth == NULL)
  >              goto clear;
  >          type_mro_meth = lookup_maybe_method(
  >              (PyObject *)&PyType_Type, &PyId_mro, &unbound);
  >          if (type_mro_meth == NULL)
  >              goto clear;
  >          if (mro_meth != type_mro_meth)
  >              goto clear;
  >          Py_XDECREF(mro_meth);
  >          Py_XDECREF(type_mro_meth);
  >      }
  >
  >
  > Look at the "if (custom)" clause.
  > "mro_meth = lookup_maybe_method(" uses lookup_maybe_method which
  > gives a borrowed reference. The same holds for "type_mro_meth".
  >
  > But then both are decreffed, which IMHO is not correct.
  >
  Look at what happens at the label "clear": it DECREFs them.

  If mro_meth != NULL or mro_meth != type_mro_meth, they'll get
 DECREFed
  at "clear".


 I believe Christian's point is that this entire "if (cust

[Python-Dev] Re: Python 3.8 error?

2019-12-12 Thread Christian Tismer 
On 12.12.19 13:04, Petr Viktorin wrote:
> I'm quite interested in the rest of the story here. PySide is probably
> the biggest open-source user of the limited API, so IMO it is relevant
> to this list.

BTW., the Python 3.8 change to type refcounting is already
breaking the Limited API a bit, because we have to dynamically
figure that out in order to be version-independent.

I am not so sure if that whole change was worth to break it?

Cheers -- Chris

--
Christian Tismer-Sperling:^)   [email protected]
Software Consulting  : http://www.stackless.com/
Strandstraße 37  : https://github.com/PySide
24217 Schönberg  : GPG key -> 0xFB7BEE0E
phone +49 173 24 18 776  fax +49 (30) 700143-0023
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/FVUU6DSHC4KRB2X7H2CJHFTHBENNPYZM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Python 3.8 error?

2019-12-12 Thread Christian Tismer 
Pardon, I meant "there is no Python 3.8 version, yet".
And this is wrong, the MacOS pip install shows

PyQt5-5.13.2-5.13.2-cp35.cp36.cp37.cp38-abi3-macosx_10_6_intel.whl

So probably we have some bad oversight, somewhere.

Cheers -- Chris


On 12.12.19 13:48, Christian Tismer wrote:
> On 12.12.19 13:04, Petr Viktorin wrote:
>> I'm quite interested in the rest of the story here. PySide is probably
>> the biggest open-source user of the limited API, so IMO it is relevant
>> to this list.
> 
> 
> I guess that PyQt5 is a similar huge user, and it may be that they have
> the same problem, since there is no 5.14 version yet ;-)
> 
> 
>> On 2019-12-11 23:48, Christian Tismer wrote:
>>> Hi all,
>>>
>>> Sorry for the noise, I was wrong, and I retract.
>>> I was somehow mislead and hunted a phantom.
>>
>> Does that mean that there was never a problem?
>> Or that a problem is still there, but we don't know if it's in CPython
>> or PySide or their interaction?
>> Or was the problem solved in PySide?
>> Do we need to document something better, or just aware of some caveat?
> 
> 
> The problem is still there.
> When PySide creates a new type, then PyType_Ready digs into the new
> type, fetches the "mro()" method and uses the
> Py_TPFLAGS_METHOD_DESCRIPTOR flag to decide how to handle it.
> 
> When I leave the flag in, we crash. So this is the current fix,
> and I have no idea why this affects us at all. Here is the last change
> to SbkObjectTypeTpNew() #503:
> 
> // The meta type creates a new type when the Python programmer
> extends a wrapped C++ class.
> auto type_new = reinterpret_cast(PyType_Type.tp_new);
> 
> // PYSIDE-939: This is a temporary patch that circumvents the problem
> // with Py_TPFLAGS_METHOD_DESCRIPTOR until this is finally solved.
> // PyType_Ready uses mro(). We need to temporarily remove the flag
> from it's type.
> // We cannot use PyMethodDescr_Type since it is not exported by
> Python 2.7 .
> static PyTypeObject *PyMethodDescr_TypePtr = Py_TYPE(
> PyObject_GetAttr(reinterpret_cast(&PyType_Type),
> Shiboken::PyName::mro()));
> auto hold = PyMethodDescr_TypePtr->tp_flags;
> PyMethodDescr_TypePtr->tp_flags &= ~Py_TPFLAGS_METHOD_DESCRIPTOR;
> auto *newType = reinterpret_cast(type_new(metatype,
> args, kwds));
> PyMethodDescr_TypePtr->tp_flags = hold;
> 
> 
> I am still not sure where the bug is and who has it.
> My understanding is that this flag should have no impact on PySide
> at all, but it has.
> 
> Thanks for taking care -- Chris
> 
> 
>>> On 10.12.19 00:29, Christian Tismer wrote:
 On 09.12.19 23:26, Nick Coghlan wrote:
>
>
> On Tue., 10 Dec. 2019, 5:17 am MRAB,  > wrote:
>
>  On 2019-12-09 18:22, Christian Tismer wrote:
>  >
>  >
>  > Hi Nick,
>  >
>  > after staring long at the code, I fount something funny in
>  > typeobject.c #286 ff:
>  >
>  >
>  > static void
>  > type_mro_modified(PyTypeObject *type, PyObject *bases) {
>  >      /*
>  >         Check that all base classes or elements of the MRO of
> type are
>  >         able to be cached.  This function is called after the
> base
>  >         classes or mro of the type are altered.
>  >
>  >         Unset HAVE_VERSION_TAG and VALID_VERSION_TAG if the type
>  >         has a custom MRO that includes a type which is not
> officially
>  >         super type, or if the type implements its own mro()
> method.
>  >
>  >         Called from mro_internal, which will subsequently be
> called on
>  >         each subclass when their mro is recursively updated.
>  >       */
>  >      Py_ssize_t i, n;
>  >      int custom = (Py_TYPE(type) != &PyType_Type);
>  >      int unbound;
>  >      PyObject *mro_meth = NULL;
>  >      PyObject *type_mro_meth = NULL;
>  >
>  >      if (!PyType_HasFeature(type, Py_TPFLAGS_HAVE_VERSION_TAG))
>  >          return;
>  >
>  >      if (custom) {
>  >          _Py_IDENTIFIER(mro);
>  >          mro_meth = lookup_maybe_method(
>  >              (PyObject *)type, &PyId_mro, &unbound);
>  >          if (mro_meth == NULL)
>  >              goto clear;
>  >          type_mro_meth = lookup_maybe_method(
>  >              (PyObject *)&PyType_Type, &PyId_mro, &unbound);
>  >          if (type_mro_meth == NULL)
>  >              goto clear;
>  >          if (mro_meth != type_mro_meth)
>  >              goto clear;
>  >          Py_XDECREF(mro_meth);
>  >          Py_XDECREF(type_mro_meth);
>  >      }
>  >
>  >
>  > Look at the "if (custom)" clause.
>  > "mro_meth = lookup_maybe_method("

[Python-Dev] Re: PEP 611: The one million limit.

2019-12-12 Thread Antoine Pitrou 
On Thu, 12 Dec 2019 11:43:58 +
Rhodri James  wrote:

> On 11/12/2019 21:35, Antoine Pitrou wrote:
> > In any case, this will have to be judged on benchmark numbers, once
> > Mark (or someone else) massages the interpreter to experiment with
> > those runtime memory footprint reductions.  
> 
> This I absolutely agree with.  Without evidence we're just waving our 
> prejudices and varied experiences at one another (for example, my 
> experience that the "modern processors" you were talking about have 
> massive caches, so memory access isn't as much of an issue as you might 
> think vs your experience that memory access matters more than doing a 
> shift-and-mask all the time).

But massive caches are not that fast.
L1 cache is typically very fast (3 or 4 cycles latency) but small (on
the order of 64 kiB).
L2 cache varies, but is generally significantly slower (typically 15
cycles latency) and medium sized (256 or 512 kiB perhaps).
L3 cache is often massive (16 MiB is not uncommon) but actually quite
slow (several dozens of cycles), by virtue of being large, further
away and generally shared between all cores.

If you have a 4-way superscalar CPU (which is an approximate
description, since the level of allowed parallelism is not the same in
all pipeline stages and depends on the instruction mix), during a
single 15-cycle L2 cache access your CPU can issue at most 60
instructions.  And during a L3 cache access there's an even larger
amount of instructions that can be scheduled and executed.

Regards

Antoine.

___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/4KA5BRDMS3ZJF4CBVITMOKVKA6PBA6NL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Emit a SyntaxWarning for unhashables literals in hashable dependant literals.

2019-12-12 Thread mental na via Python-Dev 
I'm not sure if this is the right place to bring this up, python-ideas seemed 
like language issues and python-dev seemed like CPython issues.

There are several unhashable builtin types present in CPython, as of today the 
ones I've noticed are: lists, dicts, sets, and bytearrays.
Two of these are containers that require (at least partially) their members to 
be hashable: dicts and sets

By this logic you cannot have a set of lists or a dict of sets to ints.

CPython however does not stop or warn you if you attempt to do such a thing 
until it hits `BUILD_MAP` or `BUILD_SET` during runtime.
This is reasonable behavior when it's not possible to infer the member type of 
the container i.e. ``{f(x) for x in iterable}`` or ``{f(x): y for zip(xs, ys)}``

However, given the situation where literals are nested i.e. ``{[*gen] for gen 
in gens}`` or ``{{green: eggs}, {and_: ham}}`` this presents an unavoidable 
exception at runtime.
I suggest emitting a SyntaxWarning when encountering these cases of literals 
that produce unhashable types that are used in literals that produce types 
where the members must be hashable.
I don't think it should be a SyntaxError because it's not a language issue, its 
an implementation issue.
I don't think it should be a linters responsibility because for the most part 
linters should consider language issues/idioms not side-effects from the 
running implementation.

I do understand that such cases this issue addresses may be uncommon and once 
you do get that TypeError raised its a relatively quick and easy fix, but 
consider this being present in code paths that don't get taken as frequently, 
large codebases where it becomes difficult to keep track of small one liner 
literals like this or even for the newer programmers toying with Python through 
CPython and naively using unhashables in places they shouldn't be.

Either way I'm interested in hearing what the core team thinks of this 
suggestion, thanks in advance! :D
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/IUOIEOCI22N3G67HX5BOKBS7Z26KC4FX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Emit a SyntaxWarning for unhashables literals in hashable dependant literals.

2019-12-12 Thread Guido van Rossum 
This is most definitely a language issue, not just a CPython issue -- the
rules around hashability and (im)mutability are due to the language
definition, not the whim of an implementer.

A tool like mypy will catch this for you.

As to the desirability of adding a syntax warning for such situations (when
they can be detected statically), I'm not sure -- we generally only do
syntax warnings when there is something that even experienced users get
wrong, by mistake (e.g. assert (condition, message)).

I presume this caused you some grief, or you wouldn't be posting here --
can you describe more of how this bit you, and why the runtime error did
not suffice in your case?

On Thu, Dec 12, 2019 at 7:49 AM mental na via Python-Dev <
[email protected]> wrote:

> I'm not sure if this is the right place to bring this up, python-ideas
> seemed like language issues and python-dev seemed like CPython issues.
>
> There are several unhashable builtin types present in CPython, as of today
> the ones I've noticed are: lists, dicts, sets, and bytearrays.
> Two of these are containers that require (at least partially) their
> members to be hashable: dicts and sets
>
> By this logic you cannot have a set of lists or a dict of sets to ints.
>
> CPython however does not stop or warn you if you attempt to do such a
> thing until it hits `BUILD_MAP` or `BUILD_SET` during runtime.
> This is reasonable behavior when it's not possible to infer the member
> type of the container i.e. ``{f(x) for x in iterable}`` or ``{f(x): y for
> zip(xs, ys)}``
>
> However, given the situation where literals are nested i.e. ``{[*gen] for
> gen in gens}`` or ``{{green: eggs}, {and_: ham}}`` this presents an
> unavoidable exception at runtime.
> I suggest emitting a SyntaxWarning when encountering these cases of
> literals that produce unhashable types that are used in literals that
> produce types where the members must be hashable.
> I don't think it should be a SyntaxError because it's not a language
> issue, its an implementation issue.
> I don't think it should be a linters responsibility because for the most
> part linters should consider language issues/idioms not side-effects from
> the running implementation.
>
> I do understand that such cases this issue addresses may be uncommon and
> once you do get that TypeError raised its a relatively quick and easy fix,
> but consider this being present in code paths that don't get taken as
> frequently, large codebases where it becomes difficult to keep track of
> small one liner literals like this or even for the newer programmers toying
> with Python through CPython and naively using unhashables in places they
> shouldn't be.
>
> Either way I'm interested in hearing what the core team thinks of this
> suggestion, thanks in advance! :D
> ___
> Python-Dev mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/[email protected]/message/IUOIEOCI22N3G67HX5BOKBS7Z26KC4FX/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/BMRPKUQNJNISTKGVZTHG7X5ZKRMYX7ZR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 611 -- why limit coroutines and classes?

2019-12-12 Thread Duane Griffin 
On Thu, Dec 12, 2019 at 7:05 PM Kyle Stanley  wrote:
> Antoine Pitrou wrote:
> > 1M concurrent database transactions?  Does that sound reasonable at
> > all?  Your database administrator probably won't like you.
>
> I agree that 1M concurrent transactions would not be reasonable for the vast 
> majority of database configurations, I didn't mean to specifically imply that 
> 1M would be something reasonable today. That's why I said "a large number of 
> concurrent transactions" instead of specifically saying "1M concurrent 
> transactions". I honestly don't know if any databases are close to capable of 
> handling that many transactions at once, at least not at the present time.

I have personally worked on a database back end that was processing
~800k transactions per second, about 10 years ago. It was
highly-specialised, ran on Big Iron, and was implemented in C/C++
(mostly), FORTRAN, and IIRC a smidgin of assembler. So those numbers
may not be currently realistic for Python, or for general purpose
RDBMSs, but they are not completely ridiculous.

Cheers,
Duane.

-- 
"I never could learn to drink that blood and call it wine" - Bob Dylan
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/JWQTN2CQMSUCEB6XBVXQAQH6HUUAECE4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please be more precise when commenting on PEP 611.

2019-12-12 Thread Nick Coghlan 
On Thu., 12 Dec. 2019, 10:03 pm Mark Shannon,  wrote:

>
> Explicit limits are much easier to test. Does code outside the limit
> fail in the expected fashion and code just under the limit work correctly?
>
> What I want, is to allow more efficient use of resources without
> inconveniently low or unspecified limits. There will always be some
> limits on finite machines. If they aren't specified, they still exist,
> we just don't know what they are or how they will manifest themselves.
>

This is the main reason I'd like to see the PEP split into two distinct
aspects:

* articulating & testing specific limits (we already have some bigmem tests
that can only be run at full scale on server class machines with 20+ GiB of
RAM - don't forget that CPython runs on supercomputers, not just regular
machines).

* potentially lowering some specific limits in the CPython implementation

The first part is clearly valuable as a general capability, especially if
there's an API that lets different implementations report different limits.
For example, something like a "sys.implementation.limits" informational
API, modelled on the way that the top level "sys.implementation" data
structure already works.

Even when the limits are large, we have potential access to machines that
can accommodate them for testing purposes (e.g. I know at least one Linux
vendor that we're on good terms with that has servers with more than a
terabyte of RAM in their test fleet, and I presume other OS vendors and
cloud platform providers have similar capabilities).

By contrast, the second part really needs to be tackled on a case-by-case
basis, with a clear understanding of the kinds of failure modes we're
aiming to handle more gracefully, which kinds of performance improvements
we're aiming to enable, and which existing capabilities we'd be reducing.

Cheers,
Nick.


>
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/PTRLASLIT3ZPARSLDV4I5PWAGSVEYNBX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Travis CI for backports not working.

2019-12-12 Thread Brett Cannon 
This is failing again, so I had to switch off Travis from being a requirement 
(again).

I'm not not going to flip it back on until Travis has been stable for a month 
as I don't like being the blocker on stuff when I can help it. And if Travis 
isn't stable in a month then we might need to start talking about turning it 
off entirely as flaky CI is never useful.
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/YJRPHEDV6DRVMSXCORRDUDCEFVYP4QUI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Travis CI for backports not working.

2019-12-12 Thread Victor Stinner 
What is the issue? Can someone please open a bug report at
https://bugs.python.org/ so I can try to investigate?

Victor

Le ven. 13 déc. 2019 à 02:05, Brett Cannon  a écrit :
>
> This is failing again, so I had to switch off Travis from being a requirement 
> (again).
>
> I'm not not going to flip it back on until Travis has been stable for a month 
> as I don't like being the blocker on stuff when I can help it. And if Travis 
> isn't stable in a month then we might need to start talking about turning it 
> off entirely as flaky CI is never useful.
> ___
> Python-Dev mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/[email protected]/message/YJRPHEDV6DRVMSXCORRDUDCEFVYP4QUI/
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/REAVUXBHB7RSKVG4IZTPXYFDMJJF4TWB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Travis CI for backports not working.

2019-12-12 Thread Kyle Stanley 
Victor Stinner wrote:
> What is the issue? Can someone please open a bug report at
https://bugs.python.org/ so I can try to investigate?

>From my understanding, it looks to be pyenv related and not something we
can fix on our end, at least based on the build logs:
https://travis-ci.org/python/cpython/jobs/624160244?utm_medium=notification&utm_source=github_status.
This was from a recent backport PR to 3.7 (backport also failing for 3.8
with similar issues).

On Thu, Dec 12, 2019 at 8:14 PM Victor Stinner  wrote:

> What is the issue? Can someone please open a bug report at
> https://bugs.python.org/ so I can try to investigate?
>
> Victor
>
> Le ven. 13 déc. 2019 à 02:05, Brett Cannon  a écrit :
> >
> > This is failing again, so I had to switch off Travis from being a
> requirement (again).
> >
> > I'm not not going to flip it back on until Travis has been stable for a
> month as I don't like being the blocker on stuff when I can help it. And if
> Travis isn't stable in a month then we might need to start talking about
> turning it off entirely as flaky CI is never useful.
> > ___
> > Python-Dev mailing list -- [email protected]
> > To unsubscribe send an email to [email protected]
> > https://mail.python.org/mailman3/lists/python-dev.python.org/
> > Message archived at
> https://mail.python.org/archives/list/[email protected]/message/YJRPHEDV6DRVMSXCORRDUDCEFVYP4QUI/
> > Code of Conduct: http://python.org/psf/codeofconduct/
>
>
>
> --
> Night gathers, and now my watch begins. It shall not end until my death.
> ___
> Python-Dev mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/[email protected]/message/REAVUXBHB7RSKVG4IZTPXYFDMJJF4TWB/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/34HWMTFBGXFGAAN5LCHHGX3PMO4CGZBT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Travis CI for backports not working.

2019-12-12 Thread Guido van Rossum 
Does Travis-CI serve any purpose for us still? I.e. if someone breaks some
test, is Travis-CI the only thing that keeps the breakage from landing on
master?

On Thu, Dec 12, 2019 at 5:46 PM Kyle Stanley  wrote:

> Victor Stinner wrote:
> > What is the issue? Can someone please open a bug report at
> https://bugs.python.org/ so I can try to investigate?
>
> From my understanding, it looks to be pyenv related and not something we
> can fix on our end, at least based on the build logs:
> https://travis-ci.org/python/cpython/jobs/624160244?utm_medium=notification&utm_source=github_status.
> This was from a recent backport PR to 3.7 (backport also failing for 3.8
> with similar issues).
>
> On Thu, Dec 12, 2019 at 8:14 PM Victor Stinner 
> wrote:
>
>> What is the issue? Can someone please open a bug report at
>> https://bugs.python.org/ so I can try to investigate?
>>
>> Victor
>>
>> Le ven. 13 déc. 2019 à 02:05, Brett Cannon  a écrit :
>> >
>> > This is failing again, so I had to switch off Travis from being a
>> requirement (again).
>> >
>> > I'm not not going to flip it back on until Travis has been stable for a
>> month as I don't like being the blocker on stuff when I can help it. And if
>> Travis isn't stable in a month then we might need to start talking about
>> turning it off entirely as flaky CI is never useful.
>> > ___
>> > Python-Dev mailing list -- [email protected]
>> > To unsubscribe send an email to [email protected]
>> > https://mail.python.org/mailman3/lists/python-dev.python.org/
>> > Message archived at
>> https://mail.python.org/archives/list/[email protected]/message/YJRPHEDV6DRVMSXCORRDUDCEFVYP4QUI/
>> > Code of Conduct: http://python.org/psf/codeofconduct/
>>
>>
>>
>> --
>> Night gathers, and now my watch begins. It shall not end until my death.
>> ___
>> Python-Dev mailing list -- [email protected]
>> To unsubscribe send an email to [email protected]
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/[email protected]/message/REAVUXBHB7RSKVG4IZTPXYFDMJJF4TWB/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> ___
> Python-Dev mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/[email protected]/message/34HWMTFBGXFGAAN5LCHHGX3PMO4CGZBT/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/AN4T2RLMGUMA7QPVAUBYAK2UCC5ODHUV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Travis CI for backports not working.

2019-12-12 Thread Kyle Stanley 
> I.e. if someone breaks some test, is Travis-CI the only thing that keeps
the breakage from landing on master?

We still have Azure Pipelines as a mandatory check for PRs before they can
be merged to master, which includes a few additional platforms and hasn't
had any recent issues (AFAIK). I believe Steve Dower was also recently
experimenting with adding GitHub Actions as an additional CI service.

On Thu, Dec 12, 2019 at 10:16 PM Guido van Rossum  wrote:

> Does Travis-CI serve any purpose for us still? I.e. if someone breaks some
> test, is Travis-CI the only thing that keeps the breakage from landing on
> master?
>
> On Thu, Dec 12, 2019 at 5:46 PM Kyle Stanley  wrote:
>
>> Victor Stinner wrote:
>> > What is the issue? Can someone please open a bug report at
>> https://bugs.python.org/ so I can try to investigate?
>>
>> From my understanding, it looks to be pyenv related and not something we
>> can fix on our end, at least based on the build logs:
>> https://travis-ci.org/python/cpython/jobs/624160244?utm_medium=notification&utm_source=github_status.
>> This was from a recent backport PR to 3.7 (backport also failing for 3.8
>> with similar issues).
>>
>> On Thu, Dec 12, 2019 at 8:14 PM Victor Stinner 
>> wrote:
>>
>>> What is the issue? Can someone please open a bug report at
>>> https://bugs.python.org/ so I can try to investigate?
>>>
>>> Victor
>>>
>>> Le ven. 13 déc. 2019 à 02:05, Brett Cannon  a écrit :
>>> >
>>> > This is failing again, so I had to switch off Travis from being a
>>> requirement (again).
>>> >
>>> > I'm not not going to flip it back on until Travis has been stable for
>>> a month as I don't like being the blocker on stuff when I can help it. And
>>> if Travis isn't stable in a month then we might need to start talking about
>>> turning it off entirely as flaky CI is never useful.
>>> > ___
>>> > Python-Dev mailing list -- [email protected]
>>> > To unsubscribe send an email to [email protected]
>>> > https://mail.python.org/mailman3/lists/python-dev.python.org/
>>> > Message archived at
>>> https://mail.python.org/archives/list/[email protected]/message/YJRPHEDV6DRVMSXCORRDUDCEFVYP4QUI/
>>> > Code of Conduct: http://python.org/psf/codeofconduct/
>>>
>>>
>>>
>>> --
>>> Night gathers, and now my watch begins. It shall not end until my death.
>>> ___
>>> Python-Dev mailing list -- [email protected]
>>> To unsubscribe send an email to [email protected]
>>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>>> Message archived at
>>> https://mail.python.org/archives/list/[email protected]/message/REAVUXBHB7RSKVG4IZTPXYFDMJJF4TWB/
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>>
>> ___
>> Python-Dev mailing list -- [email protected]
>> To unsubscribe send an email to [email protected]
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/[email protected]/message/34HWMTFBGXFGAAN5LCHHGX3PMO4CGZBT/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> 
>
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/TSM5LJZS2AJSFVIEW64PRJXEK7T5J3EM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Please be more precise when commenting on PEP 611.

2019-12-12 Thread Steven D'Aprano 
On Wed, Dec 11, 2019 at 10:30:15PM -0500, Richard Damon wrote:

> I will way that in my many years of
> programming experience I can't think of any great cases where a language
> as part of the language definition limited to 'size' of a program to
> good effect,

Good thing that's not what the PEP proposes then :-)

The PEP talks about CPython implementation, not hard limits on all 
Python compilers/interpreters.


> and generally such limits relegate a language into being
> seen as a 'toy language'. 

The designers of C recognised that, in practice, compilers will have 
limits. Rather than demanding "NO ARBITRARY LIMITS!!!" they specified 
*minimum* levels for compliant compilers. Often quite low limits.

Actual compilers often impose their own limits:

https://gcc.gnu.org/onlinedocs/cpp/Implementation-limits.html

https://www.cs.auckland.ac.nz/references/unix/digital/AQTLTBTE/DOCU_012.HTM

So if Mark's proposal relegates Python to a "toy language", we'll be in 
good company with other "toys" that have implementation, or even 
language, limits:


https://stackoverflow.com/questions/5689798/why-does-java-limit-the-size-of-a-method-to-65535-byte

https://web.archive.org/web/20160304023522/http://programmers.stackexchange.com/questions/108699/why-does-javas-collection-size-return-an-int

https://www.sqlite.org/limits.html


(If you read only one of those five links, please read the last.)


> The biggest issue is that computers are
> growing more powerful every day, and programs follow in getting bigger,
> so any limit that we think of as more than sufficient soon becomes too
> small (No one will need more than 640k of RAM).

The beauty of this proposal is that since its an implementation limit, 
not a law of nature, if and when computers get more powerful and 
machines routinely have multiple zettabyte memories *wink* we can always 
update the implementation.

I'm not entirely being facetious here. There's a serious point. Unlike 
languages like C and Java, where changes have to go through a long, 
slow, difficult process, we have a much more agile process. If the PEP 
is accepted, that doesn't mean we're locked into that decision for life. 
Relaxing limits in the future doesn't break backwards compatibility.

"What if computers get more powerful? Our limits will be obsolete!!!" 
Naturally. Do you still expect to be using Python 3.9 in ten years time 
with your fancy new uber-hyper-quantum ten thousand gigabyte computer? 
Probably not. As hardware grows, and our needs grow, so can the 
hypothetical limits.

What is valuable are *concrete*, actual (not theoretical) examples of 
where Mark's proposed limits are too low, so that we can get a realistic 
view of where the potential tradeoffs lie:

* lose this much functionality (code that did run, or might have run, 
  but that won't run under the PEP)

* in order to gain this much in safety, maintainability, efficiency.


And before people jump down my throat again, I've already said -- on 
multiple occassions -- that the onus is on Mark to demonstrate the 
plausibility of any such gains.


Thank you for reading :-)



-- 
Steven
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/J4YNTDMOHXDR46JMRTX6BIZAWR4NL72Q/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Emit a SyntaxWarning for unhashables literals in hashable dependant literals.

2019-12-12 Thread mental na via Python-Dev 
Guido van Rossum wrote:
> This is most definitely a language issue, not just a CPython issue -- the
> rules around hashability and (im)mutability are due to the language
> definition, not the whim of an implementer.

I was not aware of this, I assumed it was a implementation issue because 
I knew CPython's dicts use a hash table implementation and there are other
ways to implement a mapping data structure i.e. via trees.

Guido could you provide a link to the language definition that dictates these
rules about hashability and (im)mutability?

> A tool like mypy will catch this for you.

Perhaps I should raise this as a mypy issue then?
I'm using mypy version 0.750 as of today and the following code passes with no 
errors:
```
"""foo bar."""

from typing import Set, Dict

SOME = {{0}}
print(SOME)


def green_eggs() -> Set[Set[int]]:
"""foo."""
return {{1}}


def and_ham() -> Dict[Set[int], int]:
"""bar."""
return {{1}: 0}
```
Additionally the above code gets a perfect 10/10 out of pylint version 2.4.4
And as a last ditch paranoia fueled attempt formatting with black did nothing 
version 19.10b0

> As to the desirability of adding a syntax warning for such situations (when
> they can be detected statically), I'm not sure -- we generally only do
> syntax warnings when there is something that even experienced users get
> wrong, by mistake (e.g. assert (condition, message)).

Unless they're writing tests, I don't think anyone wants their code to fail.
This pattern is a guaranteed way to do that, and as shown above it's not
being caught on linters/static type checkers as far as i can tell (if it is and
I'm out of date, that's fantastic news! and I'll happily shut up about it :D .)

An experienced user wouldn't typically make this mistake, yes, but regardless
of the user mistakes can still be made and these inadvertently produce failing 
code.

> I presume this caused you some grief, or you wouldn't be posting here --
> can you describe more of how this bit you, and why the runtime error did
> not suffice in your case?

Luckily there was no grief, I was playing around with the dis module observing
how far the compiler optimized the source before depending on the runtime and
I happened upon the case ``x in {"a", "b"}`` which led me towards ``x in 
{{"a"}}``
and ``x in {{0: 1}, {1: 2}}``, and I was surprised that there was valid code 
emitted
for such a surefire way to raise an exception.

I first posted here because I thought it was an implementation issue, but as 
you've
pointed out it's most certainly a language issue; and one that can be detected 
statically.

Now I'd like to suggest that instead of relying on linters and static type 
checkers to catch
these bad patterns. Python shouldn't have allowed them in the first place, I 
see it as a
contradiction in the language's semantics ultimately deferring the job of 
denying the
programmer at the last possible moment.
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/4BVFXEP3ZQAN3I3M25246ESZGNWPLCC2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Emit a SyntaxWarning for unhashables literals in hashable dependant literals.

2019-12-12 Thread Random832 
On Fri, Dec 13, 2019, at 02:20, mental na via Python-Dev wrote:
> Guido van Rossum wrote:
> > This is most definitely a language issue, not just a CPython issue -- the
> > rules around hashability and (im)mutability are due to the language
> > definition, not the whim of an implementer.
> 
> I was not aware of this, I assumed it was a implementation issue because 
> I knew CPython's dicts use a hash table implementation and there are other
> ways to implement a mapping data structure i.e. via trees.

There are significant differences in semantics between a hash table and a 
tree-based mapping. Dict has to be a hash table on all implementations. 
(However, Java mutable types are hashable and the sky hasn't fallen for them.)

> Guido could you provide a link to the language definition that dictates these
> rules about hashability and (im)mutability?
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/KESFBWNUT2ISHH6ETR65PLWPLXSEFYA2/
Code of Conduct: http://python.org/psf/codeofconduct/