Re: [Python-Dev] Is static typing still optional?

[Eric V. Smith]

On 1/29/2018 1:55 AM, Yury Selivanov wrote:


On Mon, Jan 29, 2018 at 1:36 AM Nick Coghlan > wrote:


[...]
Currently the answers are:

- A: not hashable
- B: hashable (by identity) # Wat?
- C: hashable (by field hash)
- D: hashable (by identity) # Wat?
- E: hashable (by field hash)
- F: hashable (by field hash)
- G: hashable (by field hash)
- H: hashable (by field hash)


This is very convoluted.

+1 to make hashability an explicit opt-in.


I agree it's complicated.

I think it would be a bad design to have to opt-in to hashability if 
using frozen=True. The point of hash=None (the default) is to try and 
get the simple cases right with the simplest possible interface.


It's the intersection of "have simple defaults, but ways to override 
them" with "if the user provides some dunder methods, don't make them 
specify feature=False in order to use them" that complicated things. For 
example, maybe we no longer need eq=False now that specifying a __eq__ 
turns off dataclasses's __eq__ generation. Does dataclasses really need 
a way of using object identity for equality?


Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is static typing still optional?

[Yury Selivanov]

On Mon, Jan 29, 2018 at 1:36 AM Nick Coghlan  wrote:

> [...]
> Currently the answers are:
>
> - A: not hashable
> - B: hashable (by identity) # Wat?
> - C: hashable (by field hash)
> - D: hashable (by identity) # Wat?
> - E: hashable (by field hash)
> - F: hashable (by field hash)
> - G: hashable (by field hash)
> - H: hashable (by field hash)


This is very convoluted.

+1 to make hashability an explicit opt-in.

Yury
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is static typing still optional?

[Nick Coghlan]

On 29 January 2018 at 12:08, Guido van Rossum  wrote:
> I think this is a good candidate for fine-tuning during the beta period.
>
> Though honestly Python's own rules for when a class is hashable or not are
> the root cause for the complexity here -- since we decided to implicitly set
> __hash__ = None when you define __eq__, it's hardly surprising that
> dataclasses are having a hard time making natural rules.

In Raymond's example, the problem is the opposite: data classes are
currently interpreting "hash=False" as "Don't add a __hash__
implementation" rather than "Make this unhashable". That
interpretation isn't equivalent due to object.__hash__ existing by
default. (Reviewing Eric's table again, I believe this problem still
exists in the 3.7b1 variant as well - I just missed it the first time
I read that)

I'd say the major argument in favour of Raymond's suggestion (i.e.
always requiring an explicit "hash=True" in the dataclass decorator
call if you want the result to be hashable) is that even if we *do*
come up with a completely consistent derivation rule that the
decorator can follow, most *readers* aren't going to know that rule.
It would become a Python gotcha question for tech interviews:

=
Which of the following class definitions are hashable and what is
their hash based on?:

@dataclass
class A:
field: int

@dataclass(eq=False)
class B:
field: int

@dataclass(frozen=True)
class C:
field: int

   @dataclass(eq=False, frozen=True)
class D:
field: int

@dataclass(eq=True, frozen=True)
class E:
field: int

@dataclass(hash=True)
class F:
field: int

@dataclass(frozen=True, hash=True)
class G:
field: int

@dataclass(eq=True, frozen=True, hash=True)
class H:
field: int
=

Currently the answers are:

- A: not hashable
- B: hashable (by identity) # Wat?
- C: hashable (by field hash)
- D: hashable (by identity) # Wat?
- E: hashable (by field hash)
- F: hashable (by field hash)
- G: hashable (by field hash)
- H: hashable (by field hash)

If we instead make the default "hash=False" (and interpret that as
meaning "Inject __hash__=None"), then you end up with the following
much simpler outcome that can be mapped directly to the decorator
"hash" parameter:

- A: not hashable
- B: not hashable
- C: not hashable
- D: not hashable
- E: not hashable
- F: hashable (by field hash)
- G: hashable (by field hash)
- H: hashable (by field hash)

Inheritance of __hash__ could then be made explicitly opt-in by way of
a "dataclasses.INHERIT" constant.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Making "-j0" the default setting for the test suite?

[Glenn Linderman]

On 1/28/2018 9:15 PM, Terry Reedy wrote:

The speedup would be even better but for the last very long running test.
Could the last very long running test be started first, instead? (maybe 
it is, or maybe there are reasons not to)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Making "-j0" the default setting for the test suite?

[Nick Coghlan]

On 29 January 2018 at 14:43, Guido van Rossum  wrote:
> So why can't you just run "make test" if that's faster?

I can (and do), but I also run it the other way if I need to pass
additional options. I'll then notice that I forgot -j0, ctrl-C out,
then run it again with -j0.

That's a minor irritation for me, but for folks that don't already
know about the -j0 option, they're more likely to just go "CPython's
test suite is annoyingly slow".

To provide a bit more detail on what I'd suggest we do:

* "-j1" would explicitly turn off multiprocessing
* "-j0" and "-jN" (N >= 2) would explicitly request multiprocessing
and error out if there's a conflicting flag
* not setting the flag would be equivalent to "-j0" by default, but
"-j1" if a conflicting flag was set

The testing options that already explicitly conflict with the
multiprocessing option are:

* -T (tracing)
* -l (leak hunting)

"-j1" would likely also be a better default when the verbosity flags
are set (since the output is incredibly hard to read if you have
multiple verbose tests running in parallel).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Making "-j0" the default setting for the test suite?

[Terry Reedy]

On 1/28/2018 11:43 PM, Guido van Rossum wrote:

So why can't you just run "make test" if that's faster?


Not a standard option on Windows ;-).

On Sun, Jan 28, 2018 at 8:30 PM, Nick Coghlan > wrote:


On my current system, "make test" runs in around 3 minutes, while
"./python -m test" runs in around 16 minutes. And that's with "make
test" actually running more tests (since it enables several of the
"-u" options).

The difference is that "make test" passes "-j0" and hence not only
uses all the available cores in the machines, but will also run other
tests while some tests are sleeping.

How would folks feel about making "-j 0" the default in the test
suite, and then adjusted the handling of "-j 1" to switch back to the
current default single process mode?

My rationale for that is to improve the default edit-test cycle in
local development, while still providing a way to force single-process
execution for failure investigation purposes.


I would like this (though I could write a .bat file).  I routinely pass 
-j14 or so on a 6 core x 2 processes/core machine and get about the same 
times.  The speedup would be even better but for the last very long 
running test.  I wish each test file was limited to about 30 seconds, or 
even a minute.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Making "-j0" the default setting for the test suite?

[Guido van Rossum]

So why can't you just run "make test" if that's faster?

On Sun, Jan 28, 2018 at 8:30 PM, Nick Coghlan  wrote:

> On my current system, "make test" runs in around 3 minutes, while
> "./python -m test" runs in around 16 minutes. And that's with "make
> test" actually running more tests (since it enables several of the
> "-u" options).
>
> The difference is that "make test" passes "-j0" and hence not only
> uses all the available cores in the machines, but will also run other
> tests while some tests are sleeping.
>
> How would folks feel about making "-j 0" the default in the test
> suite, and then adjusted the handling of "-j 1" to switch back to the
> current default single process mode?
>
> My rationale for that is to improve the default edit-test cycle in
> local development, while still providing a way to force single-process
> execution for failure investigation purposes.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Making "-j0" the default setting for the test suite?

[Nick Coghlan]

On my current system, "make test" runs in around 3 minutes, while
"./python -m test" runs in around 16 minutes. And that's with "make
test" actually running more tests (since it enables several of the
"-u" options).

The difference is that "make test" passes "-j0" and hence not only
uses all the available cores in the machines, but will also run other
tests while some tests are sleeping.

How would folks feel about making "-j 0" the default in the test
suite, and then adjusted the handling of "-j 1" to switch back to the
current default single process mode?

My rationale for that is to improve the default edit-test cycle in
local development, while still providing a way to force single-process
execution for failure investigation purposes.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is static typing still optional?

[Guido van Rossum]

I think this is a good candidate for fine-tuning during the beta period.

Though honestly Python's own rules for when a class is hashable or not are
the root cause for the complexity here -- since we decided to implicitly
set __hash__ = None when you define __eq__, it's hardly surprising that
dataclasses are having a hard time making natural rules.

On Sun, Jan 28, 2018 at 5:07 PM, Raymond Hettinger <
raymond.hettin...@gmail.com> wrote:

>
> >>> 2) Change the default value for "hash" from "None" to "False".  This
> might take a little effort because there is currently an oddity where
> setting hash=False causes it to be hashable.  I'm pretty sure this wasn't
> intended ;-)
> >> I haven't looked at this yet.
> >
> > I think the hashing logic explained in https://bugs.python.org/
> issue32513#msg310830 is correct. It uses hash=None as the default, so
> that frozen=True objects are hashable, which they would not be if
> hash=False were the default.
>
> Wouldn't it be simpler to make the options orthogonal?  Frozen need not
> imply hashable.  I would think if a user wants frozen and hashable, they
> could just write frozen=True and hashable=True.  That would more explicit
> and clear than just having frozen=True imply that hashability gets
> turned-on implicitly whether you want it or not.
>
> > If there's some case there that you disagree with, I'd be interested in
> hearing about it.
> >
> > That logic is what is currently scheduled to go in to 3.7 beta 1. I have
> not updated the PEP yet, mostly because it's so difficult to explain.
>
> That might be a strong hint that this part of the API needs to be
> simplified :-)
>
> "If the implementation is hard to explain, it's a bad idea." -- Zen
>
> If for some reason, dataclasses really do need tri-state logic, it may be
> better off with enum values (NOT_HASHABLE, VALUE_HASHABLE,
> IDENTITY_HASHABLE, HASHABLE_IF_FROZEN or some such) rather than with None,
> True, and False which don't communicate enough information to understand
> what the decorator is doing.
>
> > What's the case where setting hash=False causes it to be hashable? I
> don't think that was ever the case, and I hope it's not the case now.
>
> Python 3.7.0a4+ (heads/master:631fd38dbf, Jan 28 2018, 16:20:11)
> [GCC 7.2.0] on darwin
> Type "copyright", "credits" or "license()" for more information.
>
> >>> from dataclasses import dataclass
> >>> @dataclass(hash=False)
> class A:
> x: int
>
> >>> hash(A(1))
> 285969507
>
>
> I'm hoping that this part of the API gets thought through before it gets
> set in stone.  Since dataclasses code never got a chance to live in the
> wild (on PyPI or some such), it behooves us to think through all the
> usability issues.  To me at least, the tri-state hashability was entirely
> unexpected and hard to debug -- I had to do a close reading of the source
> to figure-out what was happening.
>
>
> Raymond
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is static typing still optional?

[Raymond Hettinger]

>>> 2) Change the default value for "hash" from "None" to "False".  This might 
>>> take a little effort because there is currently an oddity where setting 
>>> hash=False causes it to be hashable.  I'm pretty sure this wasn't intended 
>>> ;-)
>> I haven't looked at this yet.
> 
> I think the hashing logic explained in 
> https://bugs.python.org/issue32513#msg310830 is correct. It uses hash=None as 
> the default, so that frozen=True objects are hashable, which they would not 
> be if hash=False were the default.

Wouldn't it be simpler to make the options orthogonal?  Frozen need not imply 
hashable.  I would think if a user wants frozen and hashable, they could just 
write frozen=True and hashable=True.  That would more explicit and clear than 
just having frozen=True imply that hashability gets turned-on implicitly 
whether you want it or not.

> If there's some case there that you disagree with, I'd be interested in 
> hearing about it.
> 
> That logic is what is currently scheduled to go in to 3.7 beta 1. I have not 
> updated the PEP yet, mostly because it's so difficult to explain.

That might be a strong hint that this part of the API needs to be simplified :-)

"If the implementation is hard to explain, it's a bad idea." -- Zen

If for some reason, dataclasses really do need tri-state logic, it may be 
better off with enum values (NOT_HASHABLE, VALUE_HASHABLE, IDENTITY_HASHABLE, 
HASHABLE_IF_FROZEN or some such) rather than with None, True, and False which 
don't communicate enough information to understand what the decorator is doing.

> What's the case where setting hash=False causes it to be hashable? I don't 
> think that was ever the case, and I hope it's not the case now.

Python 3.7.0a4+ (heads/master:631fd38dbf, Jan 28 2018, 16:20:11) 
[GCC 7.2.0] on darwin
Type "copyright", "credits" or "license()" for more information.

>>> from dataclasses import dataclass
>>> @dataclass(hash=False)
class A:
x: int

>>> hash(A(1))
285969507


I'm hoping that this part of the API gets thought through before it gets set in 
stone.  Since dataclasses code never got a chance to live in the wild (on PyPI 
or some such), it behooves us to think through all the usability issues.  To me 
at least, the tri-state hashability was entirely unexpected and hard to debug 
-- I had to do a close reading of the source to figure-out what was happening.


Raymond


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Sad buildbots

[Ned Deily]

On Jan 28, 2018, at 18:00, Victor Stinner  wrote:
> It seems like the feature freeze is close: while I usually get 2
> emails/day at maximum on buildbot-status, I got 14 emails during the
> weekend:
> https://mail.python.org/mm3/archives/list/buildbot-sta...@python.org/
> (are all buildbots red? :-p)
> 
> I will not have the bandwidth to analyze all buildbot failures. Can
> someone help to investigate all these funny new regressions?
> http://buildbot.python.org/all/#/builders
> 
> I would feel safer to cut a release if most buildbots are green again.

Never fear, we're *not* going to do a release in such a state.  That's one of 
the reasons we have release managers.  :-)

Not surprisingly, there has been a *lot* of activity over the last few days as 
core-developers work on getting features finished prior to the 3.7 feature code 
freeze coming up at the end of Monday AoE.  Some of the intermediate checkins 
cause some breakages across the board, unfortunately, that have subsequently 
been addressed.  Most of the 3.x stable buildbots are currently green with some 
builds still going on.  But, yeah, please all keep an eye of them especially 
those of you merging code.  Just because the CI tests passed doesn't mean there 
won't be problems on other platforms and configurations.

Thanks for everyone's help so far!  We're getting close.

--
  Ned Deily
  n...@python.org -- []

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Sad buildbots

[Victor Stinner]

Hi,

It seems like the feature freeze is close: while I usually get 2
emails/day at maximum on buildbot-status, I got 14 emails during the
weekend:
https://mail.python.org/mm3/archives/list/buildbot-sta...@python.org/
(are all buildbots red? :-p)

I will not have the bandwidth to analyze all buildbot failures. Can
someone help to investigate all these funny new regressions?
http://buildbot.python.org/all/#/builders

I would feel safer to cut a release if most buildbots are green again.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Guido's Python 1.0.0 Announcement from 27 Jan 1994

[Sebastian Krause]

Guido van Rossum  wrote:
> For me personally, the fondest memories are of 1.5.2, which Paul Everitt
> declared, while we were well into 2.x territory, was still the best Python
> ever. (I didn't agree, but 1.5.2 did serve us very well for a long time.)

That makes me feel better about the fact that 1.5.2 was my
employer's main Python version until late 2010. :)

(We're at 3.5 now.)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is static typing still optional?

[Eric V. Smith]

On 1/6/2018 5:13 PM, Eric V. Smith wrote:

On 12/10/2017 5:00 PM, Raymond Hettinger wrote:


...

2) Change the default value for "hash" from "None" to "False".  This 
might take a little effort because there is currently an oddity where 
setting hash=False causes it to be hashable.  I'm pretty sure this 
wasn't intended ;-)


I haven't looked at this yet.


I think the hashing logic explained in 
https://bugs.python.org/issue32513#msg310830 is correct. It uses 
hash=None as the default, so that frozen=True objects are hashable, 
which they would not be if hash=False were the default. If there's some 
case there that you disagree with, I'd be interested in hearing about it.


That logic is what is currently scheduled to go in to 3.7 beta 1. I have 
not updated the PEP yet, mostly because it's so difficult to explain.


What's the case where setting hash=False causes it to be hashable? I 
don't think that was ever the case, and I hope it's not the case now.


Eric
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com