Re: [Python-Dev] A minimal Python interpreter written in Python for experimenting with language changes

[rym...@gmail.com]

https://refi64.com/posts/the-magic-of-rpython.html

Note that this was written back when I used "like" and "really" in every
sentence, and when I used to think that Python copied tuples (don't ask).

On Feb 3, 2018 at 10:05 AM, > wrote:

> Are you aware of pypy?
>

Yes, but I do not know it well. It seems to be more focused on JIT
compiling with many of its functions written for special cases and I
did not find a way to make experimental changes easily.

I even tried (briefly) to have the main loop of this interpreter run
in RPython but didn't find enough information on which subset of
Python does RPython handle.


> Can you give an example of how you would do that? I don't mean the
> mechanism used, I mean how would a developer implement a new syntactic
> feature. Suppose I wanted to add a new clause to for...else, let's say:
>
> for ... :
>block
> otherwise:
># runs only if the for-loop was empty
>
> How would do I do that?
>

Here's one way to do that. Sorry for the lack of doc for the moment.

To alter the language's grammar, you'd create a string with the new
grammar, parse it and set the result to pyterp.parser. Then, depending
on how the grammar was changed (which affects what AST is created),
you'd change the corresponding function to handle the new semantics.

I didn't quite understand what you mean by "was empty". However, I
noticed that "else" blocks after for are ignored in the current
implement. Here's an example session adding "else".

If you tell me a bit more about the intended behaviour of "otherwise",
I'd be happy to do an example with that clause.

$ ipython -i test/python_repl.py
p>> simport simple_ast
p>> for i in [1, 2]:
... print i
...
1
2
p>> for i in [1, 2, 3]:
... print i
... else:
... print 100
...
1
2
3
p>> ^D

In the above for loop, the else block is ignored! I press Control-D to
exist the interpreter.

In [1]: grammar = python_grammar.full_definition + python_grammar.extra

In [2]: grammar += r"""
   ...: for_stmt = "for" {exprlist} "in" {testlist} ":" {suite}
{(SAME_INDENT "else" ":" {suite}) | void=pass_stmt}
   ...: """

In [3]: pyterp.parser = python.Interpreter(i3.parse("grammar", grammar))

In [4]: pyterp.repl()

Now edit for_stmt in simple_ast.py

 except StopIteration:
+evaluate(else_block)
 return

(If else_block was not already an (ignored) parameter, we'd have to
change the `def for_stmt...` line too.) And run in the REPL we just
restarted.

p>> simple_ast.reload_module(simple_ast)
p>> for i in [1, 2, 3]:
... print i
... else:
... print 100
...
1
2
3
100
p>> for i in [1, 2, 3]:
... print i
... break
... else:
... print 100
...
1
p>>



Some notes:

1. I'm using the host Python interpreter to change the grammar here
but that's not strictly necessary if we expose pyterp itself in the
global scope.
2. I'm editing simple_ast.py and reloading because its less changes
but redefining a new function and setting simple_ast.for_stmt (in
other words, monkey-patching the change) should also work.

If we wanted an "otherwise" clause on top of an "else" clause, we
probably want to make some provision for the runtime to distinguish
between the two so I'd edit the grammar with something like

for_stmt = "for" {exprlist} "in" {testlist} ":" {suite}
{((SAME_INDENT "else" ":" {suite}) | void=pass_stmt)=else_block
((SAME_INDENT "otherwise" ":" {suite}) |
void=pass_stmt)=otherwise_block}

(In this case, "otherwise" blocks have to come after the "else" block
if both are present.)

asrp
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] A minimal Python interpreter written in Python for experimenting with language changes

[Steven D'Aprano]

On Sat, Feb 03, 2018 at 11:45:15AM +0100, asrp wrote:

> > Can you give an example of how you would do that? I don't mean the 
> > mechanism used, I mean how would a developer implement a new syntactic 
> > feature. Suppose I wanted to add a new clause to for...else, let's say:
> >
> > for ... :
> >block
> > otherwise:
> ># runs only if the for-loop was empty
> > 
> > How would do I do that?
[...]
> If you tell me a bit more about the intended behaviour of "otherwise", 
> I'd be happy to do an example with that clause.


Here's a faked session showing the sort of thing I am referring to. 
(Note that this is just an example, not a proposal for a new language 
feature.)

for x in [1, 2, 3]:
print(x)
otherwise:
print("nothing there")


prints 1, 2, 3.

for x in []:
print(x)
otherwise:
print("nothing there")

prints "nothing there". In other words, the otherwise block runs if, and 
only if, the loop iterable is empty and the for block does NOT run.

Can you do something like that?



-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Immutability vs. hashability

[Steven D'Aprano]

On Mon, Feb 05, 2018 at 12:09:52AM -0600, Chris Barker wrote:

> But a bit more detail -- I'm commenting on the API, not the capability -
> that is, since users often equate hashable and immutability, they will
> expect that if they say hash=True, then will get an immutable, and if they
> say frozen=True, they will get something hashable (as long as the fields
> are hashable, just like a tuple.
> 
> That is, even though these concepts are independent, the defaults shouldn't
> reflect that.

I'm not happy about the concept of pandering to the least capable, most 
ignorant programmers by baking a miscomprehension into an important 
standard library API. The fact is that mutability and hashability ARE 
independent qualities, and the API ought to reflect reality, not 
ignorance. That's why there are two separate switches, frozen and hash, 
not just one "frozen_hashable" switch.

(Things would be different if we just outright banned mutable+hashable, 
but I don't think anyone wants that.)

Fortunately, I also believe that the number of programmers who would 
fail to draw the right conclusion from the existence of separate 
switches will actually be pretty small in practice. The fact that there 
are two separate switches is a pretty big clue that mutability and 
hashability can be controlled separately.

I believe that the proposed API is much simpler to understand than your 
revision. We have:

- frozen and hash both default to False;
- if you explicitly set one, the other uses the default.

This corresponds to a very common, Pythonic pattern that nearly 
everyone is familiar with:

def spam(frozen=False, hash=False):
...

which is easy to understand and easy to explain. Versus your proposal:

- if you set neither, then frozen and hash both default to False;
- but if you explicitly set one, the other uses True, namely the 
  opposite of the standard default.

which corresponds to something harder to describe and much less common:

def spam(frozen=None, hash=None):
if frozen is hash is None:
frozen = hash = False
elif frozen is None:
frozen = True
elif hash is None:
hash = True
...

"frozen and hash default to True, unless neither are set, in which case 
they default to False."


Let's look at the two possible scenarios you are worried about:

(1) I set frozen=True thinking that makes the class hashable so I can 
use it in a set or hash. The first time I actually do so, I get an 
explicit and obvious TypeError. Problem solved.[1]

(2) I set hash=True thinking that makes the class frozen. This scenario 
is more problematic, because there's no explicit and obvious error when 
I get it wrong. Instead, my program could silently do the wrong thing if 
my instances are quietly mutated.

The first error is self-correcting, and so I believe that the second is 
the only one we should worry about. There are two questions:

- how much should we worry? (how often will this happen?);

- what do we do about it?

I think the answers ought to be, not much and nothing. Or *at most*, 
raise a *warning* when hash=True is set without also explicitly setting 
frozen. But even that seems unnecessary to me.

I think that the intersection of events needed for this to be a real 
problem will be fairly small:

- people using DataClasses;
- who want a frozen, hashable class;
- and believe that the two are equivalent;
- and who weren't clued in by the existence of separate switches;
- and set hash=True without frozen=True;
- and don't write a unit test to confirm that their data is immutable;
- and accidentally mutate an instance which they thought was immutable;
- in such a way as to cause a silent failure.

I don't think this is a failure mode that we need to be concerned with. 
We can't protect everyone from everything.



[1] Yes, I'm glossing over the possible annoyance if not difficulty of 
actually solving the problem: somebody has to raise a bug report, 
someone has to fix the bug which in principle could involve a lot of 
disruption, there should be regression tests and maybe a new release of 
the application, etc. But this is par for the course for *any* bug -- 
there's no need to imagine that this specific bug is so terrible that 
the standard library needs to protect programmers from the possibility 
of ordinary, run-of-the-mill bugs.

-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dataclasses and correct hashability

[Ivan Levkivskyi]

Just wanted to add my 5 cents here. I am a bit surprised how people are
scared by adding `__hash__` to mutable classes.
>From my experience it is quite normal, I was always thinking about `hash()`
as hashing a _value_,
and never as hashing _identity_, if I would need the latter, there is a
different function for this, `id()`.
Moreover, I often did this in situations where dataclasses would be useful:
a class with several fields,
necessary dunders, and few other methods (record-like classes).
My motivation was mostly speed-up by memorization.
To be fair my use cases were mostly about some tree-like strictures, but
this is probably a coincidence.

FWIW it is not a super-safe practice, but neither super-dangerous.

--
Ivan



On 5 February 2018 at 22:56, Nick Coghlan  wrote:

> On 6 February 2018 at 03:47, Guido van Rossum  wrote:
> > If there's going to be an API for it, it should be in the class, not
> > something that mutates the class afterwards.
>
> Something I realised after posting the __class__ setting idea is that
> you can actually use a comparable trick to inject an unsafe hash from
> the frozen version into the mutable version:
>
> >>> from dataclasses import dataclass
> >>> @dataclass
> ... class Example:
> ... a: int
> ... b: int
> ...
> >>> c = Example(1, 2)
> >>> hash(c)
> Traceback (most recent call last):
>  File "", line 1, in 
> TypeError: unhashable type: 'Example'
>
> >>> @dataclass(frozen=True)
> ... class LockedExample(Example):
> ... pass
> ...
> >>> Example.__hash__ = LockedExample.__hash__
> >>> hash(c)
> 3713081631934410656
>
> So "unsafe_hash=True" would just be a shorthand spelling of that which
> skips creating the full frozen version of the class (and with the
> explicit parameter, we can better document the risks of making
> something hashable without also freezing it post-creation).
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> levkivskyi%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Immutability vs. hashability

[Steven D'Aprano]

On Sun, Feb 04, 2018 at 09:18:25PM -0800, Guido van Rossum wrote:

> The way I think of it generally is that immutability is a property of
> types, while hashability is a property of values.

That's a great way to look at it, thanks.


-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dataclasses and correct hashability

[Nick Coghlan]

On 6 February 2018 at 03:47, Guido van Rossum  wrote:
> If there's going to be an API for it, it should be in the class, not
> something that mutates the class afterwards.

Something I realised after posting the __class__ setting idea is that
you can actually use a comparable trick to inject an unsafe hash from
the frozen version into the mutable version:

>>> from dataclasses import dataclass
>>> @dataclass
... class Example:
... a: int
... b: int
...
>>> c = Example(1, 2)
>>> hash(c)
Traceback (most recent call last):
 File "", line 1, in 
TypeError: unhashable type: 'Example'

>>> @dataclass(frozen=True)
... class LockedExample(Example):
... pass
...
>>> Example.__hash__ = LockedExample.__hash__
>>> hash(c)
3713081631934410656

So "unsafe_hash=True" would just be a shorthand spelling of that which
skips creating the full frozen version of the class (and with the
explicit parameter, we can better document the risks of making
something hashable without also freezing it post-creation).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] libxml2 installation/binding issue

[Ethan Smith]

This list is for the discussion of development *of* Python. For discussion
of development *with* Python, you want python-list.

On Mon, Feb 5, 2018 at 2:41 PM, Priest, Matt  wrote:

> Hello,
>
>
>
> I am not sure if this is the correct place to post an issue/question like
> this, but here goes…
>
>
>
> I’ve successfully (?) installed Python 3.6.4 and libxml2, with the
> ultimate goal of installing GTK+ 3.22.0.
>
> However, I’m running into this error:
>
>
>
>
>
> python3
>
> Python 3.6.4 (default, Feb  5 2018, 13:28:04)
>
> [GCC 4.7.2] on linux
>
> Type "help", "copyright", "credits" or "license" for more information.
>
> >>> import libxml2
>
> Traceback (most recent call last):
>
>   File "", line 1, in 
>
>   File "/nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/
> development/sfwr/lib/python3.6/site-packages/libxml2.py", line 1, in
> 
>
> import libxml2mod
>
> ImportError: /nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/
> development/sfwr/lib/python3.6/site-packages/libxml2mod.so: undefined
> symbol: _PyVerify_fd
>
>
>
>
>
> Here are the details on the version, cflags, and ldflags.
>
> python3 --version ;
>
> Python 3.6.4
>
> python3-config --cflags
>
> -I/nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/
> development/sfwr/include/python3.6m
>
> -I/nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/
> development/sfwr/include/python3.6m
>
> -Wno-unused-result
>
> -Wsign-compare
>
> -fPIC -DNDEBUG
>
> -g
>
> -fwrapv
>
> -O3
>
> -Wall
>
> -Wstrict-prototypes
>
>
>
> python3-config –ldflags;
>
> -L/nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/
> development/sfwr/lib/python3.6/config-3.6m-x86_64-linux-gnu
>
> -L/nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/development/sfwr/lib
>
> -lpython3.6m
>
> -lpthread
>
> -ldl
>
> -lutil
>
> -lrt
>
> -lm
>
> -Xlinker
>
> -export-dynamic
>
>
>
> Anyhelp or hint would be appreciated…
>
>
>
>
>
>
>
> Matt
>
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> ethan%40ethanhs.me
>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] libxml2 installation/binding issue

[Priest, Matt]

Hello,

I am not sure if this is the correct place to post an issue/question like this, 
but here goes...

I've successfully (?) installed Python 3.6.4 and libxml2, with the ultimate 
goal of installing GTK+ 3.22.0.
However, I'm running into this error:


python3
Python 3.6.4 (default, Feb  5 2018, 13:28:04)
[GCC 4.7.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import libxml2
Traceback (most recent call last):
  File "", line 1, in 
  File 
"/nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/development/sfwr/lib/python3.6/site-packages/libxml2.py",
 line 1, in 
import libxml2mod
ImportError: 
/nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/development/sfwr/lib/python3.6/site-packages/libxml2mod.so:
 undefined symbol: _PyVerify_fd


Here are the details on the version, cflags, and ldflags.
python3 --version ;
Python 3.6.4
python3-config --cflags
-I/nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/development/sfwr/include/python3.6m
-I/nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/development/sfwr/include/python3.6m
-Wno-unused-result
-Wsign-compare
-fPIC -DNDEBUG
-g
-fwrapv
-O3
-Wall
-Wstrict-prototypes

python3-config -ldflags;
-L/nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/development/sfwr/lib/python3.6/config-3.6m-x86_64-linux-gnu
-L/nfs/sc/disks/slx_1353/mlpriest/sl1/work_root/a0/development/sfwr/lib
-lpython3.6m
-lpthread
-ldl
-lutil
-lrt
-lm
-Xlinker
-export-dynamic

Anyhelp or hint would be appreciated...



Matt

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dataclasses and correct hashability

[Paul G]

I don't think it matters so much whether you are stacking two decorators or a 
single decorator, but would an @add_unsafe_hash decorator be useful for 
anything *except* data classes? If not, then there's no point in having a 
*second* decorator that can *only* modify the first one - particularly 
considering @dataclass actually takes arguments.

On 02/05/2018 02:12 PM, Guido van Rossum wrote:
> Yes, that's what I meant -- "afterwards" meaning after the @dataclass
> decorator is applied.
> 
> On Mon, Feb 5, 2018 at 11:09 AM, Kirill Balunov 
> wrote:
> 
>>
>> 2018-02-05 20:47 GMT+03:00 Guido van Rossum :
>>
>>> If there's going to be an API for it, it should be in the class, not
>>> something that mutates the class afterwards.
>>>
>>
>>
>> I apologize and don't want to make unnecessary noise. But the already
>> selected design with decorator @dataclass implies that it will mutate
>> the freshly created class (which in its turn already limits some
>> possibilities), or I've missed something? If you meant that everything
>> should be defined in one place, then I basically understand your desire as
>> the least of two evils.
>>
>> With kind regards,
>> -gdg
>>
> 
> 
> 
> 
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/paul%40ganssle.io
> 



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [RELEASE] Python 3.7.0b1 is now available for testing

[Mike Miller]

On 2018-01-31 17:34, Ned Deily wrote:

Please see "What’s New In Python 3.7" for more information.
Additional documentation for these features and for other changes
will be provided during the beta phase.

https://docs.python.org/3.7/whatsnew/3.7.html



I see that the new classmethod fromisoformat in datetime arrived (thanks all) 
but it hasn't yet received a blurb in the what's new page.


https://docs.python.org/3.7/library/datetime.html#datetime.datetime.fromisoformat

-Mike
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dataclasses and correct hashability

[Guido van Rossum]

Yes, that's what I meant -- "afterwards" meaning after the @dataclass
decorator is applied.

On Mon, Feb 5, 2018 at 11:09 AM, Kirill Balunov 
wrote:

>
> 2018-02-05 20:47 GMT+03:00 Guido van Rossum :
>
>> If there's going to be an API for it, it should be in the class, not
>> something that mutates the class afterwards.
>>
>
>
> I apologize and don't want to make unnecessary noise. But the already
> selected design with decorator @dataclass implies that it will mutate
> the freshly created class (which in its turn already limits some
> possibilities), or I've missed something? If you meant that everything
> should be defined in one place, then I basically understand your desire as
> the least of two evils.
>
> With kind regards,
> -gdg
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dataclasses and correct hashability

[Kirill Balunov]

2018-02-05 20:47 GMT+03:00 Guido van Rossum :

> If there's going to be an API for it, it should be in the class, not
> something that mutates the class afterwards.
>


I apologize and don't want to make unnecessary noise. But the already
selected design with decorator @dataclass implies that it will mutate
the freshly created class (which in its turn already limits some
possibilities), or I've missed something? If you meant that everything
should be defined in one place, then I basically understand your desire as
the least of two evils.

With kind regards,
-gdg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dataclasses and correct hashability

[David Mertz]

Absolutely I agree. 'unsafe_hash' as a name is clear warning to users.

On Feb 4, 2018 10:43 PM, "Chris Barker"  wrote:



On Sun, Feb 4, 2018 at 11:57 PM, Gregory P. Smith  wrote:

> +1 using unsafe_hash as a name addresses my concern.
>
mine too -- anyone surprised by using this deserves what they get :-)

-CHB


On Sun, Feb 4, 2018, 9:50 PM Guido van Rossum  wrote:
>
>> Looks like this is turning into a major flamewar regardless of what I
>> say. :-(
>>
>> I really don't want to lose the ability to add a hash function to a
>> mutable dataclass by flipping a flag in the decorator. I'll explain below.
>> But I am fine if this flag has a name that clearly signals it's an unsafe
>> thing to do.
>>
>> I propose to replace the existing (as of 3.7.0b1) hash= keyword for the
>> @dataclass decorator with a simpler flag named unsafe_hash=. This would be
>> a simple bool (not a tri-state flag like the current hash=None|False|True).
>> The default would be False, and the behavior then would be to add a hash
>> function automatically only if it's safe (using the same rules as for
>> hash=None currently). With unsafe_hash=True, a hash function would always
>> be generated that takes all fields into account except those declared using
>> field(hash=False). If there's already a `def __hash__` in the function I
>> don't care what it does, maybe it should raise rather than quietly doing
>> nothing or quietly overwriting it.
>>
>> Here's my use case.
>>
>> A frozen class requires a lot of discipline, since you have to compute
>> the values of all fields before calling the constructor. A mutable class
>> allows other initialization patterns, e.g. manually setting some fields
>> after the instance has been constructed, or having a separate non-dunder
>> init() method. There may be good reasons for using these patterns, e.g. the
>> object may be part of a cycle (e.g. parent/child links in a tree). Or you
>> may just use one of these patterns because you're a pretty casual coder. Or
>> you're modeling something external.
>>
>> My point is that once you have one of those patterns in place, changing
>> your code to avoid them may be difficult. And yet your code may treat the
>> objects as essentially immutable after the initialization phase (e.g. a
>> parse tree). So if you create a dataclass and start coding like that for a
>> while, and much later you need to put one of these into a set or use it as
>> a dict key, switching to frozen=True may not be a quick option. And writing
>> a __hash__ method by hand may feel like a lot of busywork. So this is where
>> [unsafe_]hash=True would come in handy.
>>
>> I think naming the flag unsafe_hash should take away most objections,
>> since it will be clear that this is not a safe thing to do. People who
>> don't understand the danger are likely to copy a worse solution from
>> StackOverflow anyway. The docs can point to frozen=True and explain the
>> danger.
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: https://mail.python.org/mailman/options/python-dev/greg%
>> 40krypto.org
>>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/chris.
> barker%40noaa.gov
>
>


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: https://mail.python.org/mailman/options/python-dev/
mertz%40gnosis.cx
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dataclasses and correct hashability

[Guido van Rossum]

If there's going to be an API for it, it should be in the class, not
something that mutates the class afterwards.

On Mon, Feb 5, 2018 at 1:59 AM, Kirill Balunov 
wrote:

> On Sun, Feb 4, 2018, 9:50 PM Guido van Rossum  > wrote:
>>
>> Looks like this is turning into a major flamewar regardless of what I say.
>> :-(
>> I really don't want to lose the ability to add a hash function to a
>> mutable dataclass by flipping a flag in the decorator. I'll explain below.
>> But I am fine if this flag has a name that clearly signals it's an unsafe
>> thing to do.
>>
>> I propose to replace the existing (as of 3.7.0b1) hash= keyword for the
>> @dataclass decorator with a simpler flag named unsafe_hash=. This would be
>> a simple bool (not a tri-state flag like the current hash=None|False|True).
>> The default would be False, and the behavior then would be to add a hash
>> function automatically only if it's safe (using the same rules as for
>> hash=None currently). With unsafe_hash=True, a hash function would always
>> be generated that takes all fields into account except those declared using
>> field(hash=False). If there's already a `def __hash__` in the function I
>> don't care what it does, maybe it should raise rather than quietly doing
>> nothing or quietly overwriting it.
>>
>> Here's my use case.
>>
>>
> May be it is better to provide a special purpose function
> `make_unsafe_hash` in
> dataclass module which will patch a dataclass, instead of to clutter
> @dataclass
> API with arguments which are rather special case.
>
> This `unsafe_hash` argument will constantly raise questions among ordinary
> users
> like me, and will be possibly considered as a non-obvious design - there
> is a
> public API but it is somehow unsafe. On the other hand, with a function,
> when
> the user asks how to make a `frozen=False` dataclass hashable, you can
> suggest
> to use this `make_unsafe_hash` function with all its cautions in its docs
> or to try to
> implement __hash__ by yourself.
>
> Also taking into account the Python approach for backward compatibility it
> is
> better to stick with function and if it will be usefull to add a
> `unsafe_hash`
> argument in Python 3.8. It is easier to add later than to deprecate in the
> future.
>
> With kind regards,
> -gdg
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> guido%40python.org
>
>


-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dataclasses and correct hashability

[Guido van Rossum]

I'm sorry, but a solution that requires __class__ assignment is way too
fragile for my taste.

On Mon, Feb 5, 2018 at 6:28 AM, Nick Coghlan  wrote:

> On 5 February 2018 at 15:49, Guido van Rossum  wrote:
> > My point is that once you have one of those patterns in place, changing
> your
> > code to avoid them may be difficult. And yet your code may treat the
> objects
> > as essentially immutable after the initialization phase (e.g. a parse
> tree).
> > So if you create a dataclass and start coding like that for a while, and
> > much later you need to put one of these into a set or use it as a dict
> key,
> > switching to frozen=True may not be a quick option. And writing a
> __hash__
> > method by hand may feel like a lot of busywork. So this is where
> > [unsafe_]hash=True would come in handy.
> >
> > I think naming the flag unsafe_hash should take away most objections,
> since
> > it will be clear that this is not a safe thing to do. People who don't
> > understand the danger are likely to copy a worse solution from
> StackOverflow
> > anyway. The docs can point to frozen=True and explain the danger.
>
> Aye, calling the flag unsafe_hash would convert me from -1 to -0.
>
> The remaining -0 is because I think there's a different and more
> robust way to tackle your example use case:
>
> # Mutable initialization phase
> >>> from dataclasses import dataclass
> >>> @dataclass
> ... class Example:
> ... a: int
> ... b: int
> ...
> >>> c = Example(None, None)
> >>> c
> Example(a=None, b=None)
> >>> c.a = 1
> >>> c.b = 2
> >>> c
> Example(a=1, b=2)
>
>
> # Frozen usage phase
> >>> @dataclass(frozen=True)
> ... class LockedExample(Example):
> ... pass
> ...
> >>> c.__class__ = LockedExample
> >>> c.a = 1
> Traceback (most recent call last):
>  File "", line 1, in 
>  File "/home/ncoghlan/devel/cpython/Lib/dataclasses.py", line 448,
> in _frozen_setattr
>raise FrozenInstanceError(f'cannot assign to field {name!r}')
> dataclasses.FrozenInstanceError: cannot assign to field 'a'
> >>> c.b = 2
> Traceback (most recent call last):
>  File "", line 1, in 
>  File "/home/ncoghlan/devel/cpython/Lib/dataclasses.py", line 448,
> in _frozen_setattr
>raise FrozenInstanceError(f'cannot assign to field {name!r}')
> dataclasses.FrozenInstanceError: cannot assign to field 'b'
> >>> hash(c)
> 3713081631934410656
>
> The gist of that approach is to assume that there will be *somewhere*
> in the code where it's possible to declare the construction of the
> instance "complete", and flip the nominal class over to the frozen
> subclass to make further mutation unlikely, even though the true
> underlying type is still the mutable version.
>
> That said, if we do provide "unsafe_hash", then the documentation for
> that flag becomes a place where we can explicitly suggest using a
> frozen subclass instead.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Dataclasses and correct hashability

[Kirill Balunov]

>
> On Sun, Feb 4, 2018, 9:50 PM Guido van Rossum  > wrote:
>
> Looks like this is turning into a major flamewar regardless of what I say.
> :-(
> I really don't want to lose the ability to add a hash function to a
> mutable dataclass by flipping a flag in the decorator. I'll explain below.
> But I am fine if this flag has a name that clearly signals it's an unsafe
> thing to do.
>
> I propose to replace the existing (as of 3.7.0b1) hash= keyword for the
> @dataclass decorator with a simpler flag named unsafe_hash=. This would be
> a simple bool (not a tri-state flag like the current hash=None|False|True).
> The default would be False, and the behavior then would be to add a hash
> function automatically only if it's safe (using the same rules as for
> hash=None currently). With unsafe_hash=True, a hash function would always
> be generated that takes all fields into account except those declared using
> field(hash=False). If there's already a `def __hash__` in the function I
> don't care what it does, maybe it should raise rather than quietly doing
> nothing or quietly overwriting it.
>
> Here's my use case.
>
>
May be it is better to provide a special purpose function
`make_unsafe_hash` in
dataclass module which will patch a dataclass, instead of to clutter
@dataclass
API with arguments which are rather special case.

This `unsafe_hash` argument will constantly raise questions among ordinary
users
like me, and will be possibly considered as a non-obvious design - there is
a
public API but it is somehow unsafe. On the other hand, with a function,
when
the user asks how to make a `frozen=False` dataclass hashable, you can
suggest
to use this `make_unsafe_hash` function with all its cautions in its docs
or to try to
implement __hash__ by yourself.

Also taking into account the Python approach for backward compatibility it
is
better to stick with function and if it will be usefull to add a
`unsafe_hash`
argument in Python 3.8. It is easier to add later than to deprecate in the
future.

With kind regards,
-gdg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dataclasses and correct hashability

[Nick Coghlan]

On 5 February 2018 at 15:49, Guido van Rossum  wrote:
> My point is that once you have one of those patterns in place, changing your
> code to avoid them may be difficult. And yet your code may treat the objects
> as essentially immutable after the initialization phase (e.g. a parse tree).
> So if you create a dataclass and start coding like that for a while, and
> much later you need to put one of these into a set or use it as a dict key,
> switching to frozen=True may not be a quick option. And writing a __hash__
> method by hand may feel like a lot of busywork. So this is where
> [unsafe_]hash=True would come in handy.
>
> I think naming the flag unsafe_hash should take away most objections, since
> it will be clear that this is not a safe thing to do. People who don't
> understand the danger are likely to copy a worse solution from StackOverflow
> anyway. The docs can point to frozen=True and explain the danger.

Aye, calling the flag unsafe_hash would convert me from -1 to -0.

The remaining -0 is because I think there's a different and more
robust way to tackle your example use case:

# Mutable initialization phase
>>> from dataclasses import dataclass
>>> @dataclass
... class Example:
... a: int
... b: int
...
>>> c = Example(None, None)
>>> c
Example(a=None, b=None)
>>> c.a = 1
>>> c.b = 2
>>> c
Example(a=1, b=2)


# Frozen usage phase
>>> @dataclass(frozen=True)
... class LockedExample(Example):
... pass
...
>>> c.__class__ = LockedExample
>>> c.a = 1
Traceback (most recent call last):
 File "", line 1, in 
 File "/home/ncoghlan/devel/cpython/Lib/dataclasses.py", line 448,
in _frozen_setattr
   raise FrozenInstanceError(f'cannot assign to field {name!r}')
dataclasses.FrozenInstanceError: cannot assign to field 'a'
>>> c.b = 2
Traceback (most recent call last):
 File "", line 1, in 
 File "/home/ncoghlan/devel/cpython/Lib/dataclasses.py", line 448,
in _frozen_setattr
   raise FrozenInstanceError(f'cannot assign to field {name!r}')
dataclasses.FrozenInstanceError: cannot assign to field 'b'
>>> hash(c)
3713081631934410656

The gist of that approach is to assume that there will be *somewhere*
in the code where it's possible to declare the construction of the
instance "complete", and flip the nominal class over to the frozen
subclass to make further mutation unlikely, even though the true
underlying type is still the mutable version.

That said, if we do provide "unsafe_hash", then the documentation for
that flag becomes a place where we can explicitly suggest using a
frozen subclass instead.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Immutability vs. hashability

[Antoine Pitrou]

On Sun, 4 Feb 2018 14:31:06 -0800
Guido van Rossum  wrote:
> On Sun, Feb 4, 2018 at 11:59 AM, Chris Barker - NOAA Federal <
> chris.bar...@noaa.gov> wrote:  
> 
> > I think the folks that are concerned about this issue are quite right
> > — most Python users equate immutable and hashable—so the dataclass API
> > should reflect that.
> >  
> 
> Since they are *not* equivalent (consider a tuple containing a list) I'm
> not at all convinced that any API in the core language should "reflect"
> this misconception, depending on how you meant that.

+1 from me.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dataclasses and correct hashability

[Terry Reedy]

On 2/5/2018 2:28 AM, Glenn Linderman wrote:

This is an interesting use case. I haven't got the internals knowledge 
to know just how just different mutable and immutable classes and 
objects are under the hood.


I believe there is no internal difference.  An object is immutable if 
there is not way to mutate it with Python code that not poke into 
internals, such as one can do with ctypes or 3rd party extensions. 
Numbers and strings have no mutation methods, including no .__init__.


A tuple is a fixed sequence of objects and has no .__init__.  But if any 
object in a tuple is mutable, then the tuple is.  But the tuple does not 
know its status, and there is no 'is_mutable' function. However, 
tuple.__hash__ calls the .__hash__ method of each object and if that is 
missing for one, tuple.__hash raises.


>>> hash((1, 'a', []))
Traceback (most recent call last):
  File "", line 1, in 
hash((1, 'a', []))
TypeError: unhashable type: 'list'

The built-in immutable objects are mutated from their initial blank 
values in the C code of their .__new__ methods.  So they are only 
'immutable' once constructed.  Guido pointed out that users constructing 
objects in Python code might reasonably do so other than only with 
.__new__, but still want to treat the object as immutable once constructed.


In Lisp, for instance, lists are actually trees.  To be immutable, they 
can only be singly linked and must be constructed from leaf nodes to the 
root (or head).  Python programmers should be able to link in both 
directions and start from the root, and still treat the result as frozen 
and hashable.


But this use case makes me wonder if, even 
at the cost of some performance that "normal" immutable classes and 
objects might obtain, if it would be possible to use the various 
undisciplined initialization patterns as desired, followed by as 
declaration "This OBJECT is now immutable" which would calculate its 
HASH value, and prevent future mutations of the object?


Something like this has been proposed, at least for dicts, and rejected.

--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dataclasses and correct hashability

[Glenn Linderman]

On 2/5/2018 12:11 AM, Nathaniel Smith wrote:

On Sun, Feb 4, 2018 at 11:28 PM, Glenn Linderman  wrote:

This is an interesting use case. I haven't got the internals knowledge to
know just how just different mutable and immutable classes and objects are
under the hood. But this use case makes me wonder if, even at the cost of
some performance that "normal" immutable classes and objects might obtain,
if it would be possible to use the various undisciplined initialization
patterns as desired, followed by as declaration "This OBJECT is now
immutable" which would calculate its HASH value, and prevent future
mutations of the object?

It would be technically possible to support something like

@dataclass(freezable=True)
class Foo:
 blah: int

foo = Foo()
# Initially, object is mutable, and hash(foo) raises an error
foo.blah = 1
assertRaises(hash, foo)

# This method is automatically generated for classes with freezable=True
foo.freeze()

# Now object is immutable, and hash(foo) is allowed
assertRaises(foo.__setattr__, "blah", 2)
hash(foo)

I don't know if it's worth the complexity, but I guess it would cover
at least some of the use cases Guido raised.

-n

Thanks, Nathaniel, for confirming that what I was suggesting is not 
impossible, even if it turns out to be undesirable for some reason, or 
unwanted by anyone else. But I have encountered a subset of the use 
cases Guido mentioned, and had to make a 2nd class to gather/hold the 
values of the eventual immutable class, before I could make it, because 
pieces of the data for the class values were obtained from different 
sources at different times. Once all collected, then the immutability 
could be obtained, the rest of the processing performed. Thrashes the 
allocator pretty well doing it that way, but the job got done.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dataclasses and correct hashability

[Nathaniel Smith]

On Sun, Feb 4, 2018 at 11:28 PM, Glenn Linderman  wrote:
> This is an interesting use case. I haven't got the internals knowledge to
> know just how just different mutable and immutable classes and objects are
> under the hood. But this use case makes me wonder if, even at the cost of
> some performance that "normal" immutable classes and objects might obtain,
> if it would be possible to use the various undisciplined initialization
> patterns as desired, followed by as declaration "This OBJECT is now
> immutable" which would calculate its HASH value, and prevent future
> mutations of the object?

It would be technically possible to support something like

@dataclass(freezable=True)
class Foo:
blah: int

foo = Foo()
# Initially, object is mutable, and hash(foo) raises an error
foo.blah = 1
assertRaises(hash, foo)

# This method is automatically generated for classes with freezable=True
foo.freeze()

# Now object is immutable, and hash(foo) is allowed
assertRaises(foo.__setattr__, "blah", 2)
hash(foo)

I don't know if it's worth the complexity, but I guess it would cover
at least some of the use cases Guido raised.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com