Re: [Python-ideas] Let try-except check the exception instance

[Franklin? Lee]

Would guards (such as from the switch-case discussions) be a good fit?

try:
...
except RuntimeError as e if e.message == "tie":
...


On Thu, May 31, 2018, 00:48 Danilo J. S. Bellini 
wrote:

> Hi!
> I was working on handling some exceptions from external software
> (e.g. database constraint triggers)
> switching the handler based on the messages that had been sent.
> Today we can do something like (running on Python 3.6.5):
>
>
> >>> try:
> ... # [...]
> ... session.commit() # Here it raises!
> ... # [...]
> ... except DatabaseError as exc:
> ... msg = get_db_error_msg_from_exception(exc)
> ... if msg == "beyond_limit":
> ... # [...]
> ... elif msg == "no_funds":
> ... # [...]
> ... else:
> ... raise
>
>
> That works, but I'd like to do something like:
>
>
> >>> try:
> ... # [...]
> ... except BeyondLimit:
> ... # [...]
> ... except NoFunds:
> ... # [...]
>
>
> Creating classes to "match" the exception in their __instancecheck__.
> Well, I tried to do so. A simplified example would be:
>
>
> >>> class MsgCheckerMeta(type):
> ... def __instancecheck__(cls, instance):
> ... return str(instance) == cls.internal
> ...
> >>> class ExceptionHasMessage(Exception, metaclass=MsgCheckerMeta):
> ... internal = "message"
>
>
> Using these new classes, we would get this:
>
>
> >>> try:
> ... raise Exception("message")
> ... except ExceptionHasMessage:
> ... print("Yeah!")
> ...
> Traceback (most recent call last):
>   File "", line 2, in 
> Exception: message
>
>
> Yet,
>
>
> >>> isinstance(Exception("message"), ExceptionHasMessage)
> True
> >>> try:
> ... raise Exception("message")
> ... except Exception as exc:
> ... print(isinstance(exc, ExceptionHasMessage))
> ...
> True
>
>
> The idea is to allow catching exceptions beyond checking their MRO,
> using a class that checks the exception instance by implementing
> a custom __instancecheck__.
>
> --
> Danilo J. S. Bellini
> ---
> "*It is not our business to set up prohibitions, but to arrive at
> conventions.*" (R. Carnap)
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Let try-except check the exception instance

[Steven D'Aprano]

On Thu, May 31, 2018 at 01:47:17AM -0300, Danilo J. S. Bellini wrote:

> >>> try:
> ... # [...]
> ... session.commit() # Here it raises!
> ... # [...]
> ... except DatabaseError as exc:
> ... msg = get_db_error_msg_from_exception(exc)
> ... if msg == "beyond_limit":
> ... # [...]
> ... elif msg == "no_funds":
> ... # [...]
> ... else:
> ... raise

Since error messages are rarely part of the exception API, testing for 
them is fragile and prone to errors. For example, what if the message 
changes to "no money" or "out of funds" or "insufficient funds" or 
"keine Mittel"? Since the error message is not part of the API, that can 
happen at any time, without warning.

If you want to make such a risky check in your own code, of course you 
can do so, but we shouldn't encourage it or add functionality to make it 
easier. You have the right to shoot yourself in the foot, but don't 
expect us to load the gun, hand it to you, point it at your foot, and 
place your finger on the trigger :-)


> That works, but I'd like to do something like:
> 
> 
> >>> try:
> ... # [...]
> ... except BeyondLimit:
> ... # [...]
> ... except NoFunds:
> ... # [...]

The obvious way to do that is to create BeyondLimit and NoFunds 
subclasses of DatabaseError. Why can't you do that?


> Creating classes to "match" the exception in their __instancecheck__.
> Well, I tried to do so. A simplified example would be:
> 
> >>> class MsgCheckerMeta(type):
> ... def __instancecheck__(cls, instance):
> ... return str(instance) == cls.internal
> ...
> >>> class ExceptionHasMessage(Exception, metaclass=MsgCheckerMeta):
> ... internal = "message"


That seems like a backwards way to do it to me. If you're the author of 
the exception class, why not just subclass your exceptions the regular 
way and get a real subclass?

If you have enough control of the source to change the exception from 
DatabaseError to ExceptionHasMessage, then you ought to have enough 
control to change to a specific subclass as needed.

And if you *don't* have that control, then you're probably monkey- 
patching something you don't control and again, we shouldn't encourage 
that.

Do you have a better example of checking __instancecheck__ that doesn't 
involve something that's a risky hack?



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Let try-except check the exception instance

[Danilo J. S. Bellini]

Hi!
I was working on handling some exceptions from external software
(e.g. database constraint triggers)
switching the handler based on the messages that had been sent.
Today we can do something like (running on Python 3.6.5):


>>> try:
... # [...]
... session.commit() # Here it raises!
... # [...]
... except DatabaseError as exc:
... msg = get_db_error_msg_from_exception(exc)
... if msg == "beyond_limit":
... # [...]
... elif msg == "no_funds":
... # [...]
... else:
... raise


That works, but I'd like to do something like:


>>> try:
... # [...]
... except BeyondLimit:
... # [...]
... except NoFunds:
... # [...]


Creating classes to "match" the exception in their __instancecheck__.
Well, I tried to do so. A simplified example would be:


>>> class MsgCheckerMeta(type):
... def __instancecheck__(cls, instance):
... return str(instance) == cls.internal
...
>>> class ExceptionHasMessage(Exception, metaclass=MsgCheckerMeta):
... internal = "message"


Using these new classes, we would get this:


>>> try:
... raise Exception("message")
... except ExceptionHasMessage:
... print("Yeah!")
...
Traceback (most recent call last):
  File "", line 2, in 
Exception: message


Yet,


>>> isinstance(Exception("message"), ExceptionHasMessage)
True
>>> try:
... raise Exception("message")
... except Exception as exc:
... print(isinstance(exc, ExceptionHasMessage))
...
True


The idea is to allow catching exceptions beyond checking their MRO,
using a class that checks the exception instance by implementing
a custom __instancecheck__.

-- 
Danilo J. S. Bellini
---
"*It is not our business to set up prohibitions, but to arrive at
conventions.*" (R. Carnap)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A real life example of "given"

[Neil Girdhar]

On Wed, May 30, 2018 at 9:02 PM Steven D'Aprano  wrote:

> On Thu, May 31, 2018 at 10:05:33AM +1000, Chris Angelico wrote:
> > On Thu, May 31, 2018 at 9:53 AM, Steven D'Aprano 
> wrote:
> > >> There is no nice, equivalent := version as far as I can tell.
> > >
> > > Given (pun intended) the fact that you only use transformed_b in a
> > > single place, I don't think it is necessary to use := at all.
> > >
> > > z = {a: transform(b) for b in bs for a in as_}
> > >
> > > But if you really insist:
> > >
> > > # Pointless use of :=
> > > z = {a: (transformed_b := transform(b)) for b in bs for a in as_}
> > >
> >
> > That's the subtlety of the 'given' usage here. You fell for the same
> > trap I did: thinking "it's only used once".
>
> But it is only used once. I meant once per loop.
>
> It isn't used in the "for a in as_" inner loop, there's no "if
> transformed_b" condition, and it only is used once in the key:value part
> of the comprehension.
>
>
> > Actually, what he has is equivalent to:
> >
> > z = {a: tb for b in bs for tb in [transform(b)] for a in as_}
>
> Which also uses tb only once, making it a Useless Use Of Assignment.
>
> (I assume we're not calling transform() for some side-effect, like
> logging a message, or erasing your hard drive.)
>
>
> > which means it evaluates transform(b) once regardless of the length of
> > as_.
>
> Ah yes, I see what you mean. Expanded to a loop:
>
> for b in bs:
> tb = transform(b)
> for a in as_:
> z[a] = tb
>
>
> It's a little ugly, but there's a trick I already use today:
>
> py> [x+y for x in "abc" if print(x) or True for y in "de"]
> a
> b
> c
> ['ad', 'ae', 'bd', 'be', 'cd', 'ce']
>
> So we can adapt that to assignment instead of output:
>
> # Don't do this!
> z = {a: tb for b in bs if (tb := transform(b)) or True for a in as_}
>
> But I wouldn't do that. If I'm concerned about the call to transform
> (because it is super expensive, say) then I set up a pipeline:
>
> tbs = (transform(b) for b in bs)  # or map(transform, bs)
> z = {a: tb for tb in tbs for a in as_}
>
> The first generator comprehension can be easily embedded in the other:
>
> z = {a: tb for tb in (transform(b) for b in bs) for a in as_}
>
> This makes it super-obvious that transform is called for each b, not for
> each (b, a) pair, it works today, and there's no assignment expression
> needed at all.
>
> Assignment expressions should not be about adding yet a third way to
> solve a problem that already has a perfectly good solution! ("Expand to
> a loop statement" is not a *perfectly* good solution.) To showcase
> assignment expressions, we should be solving problems that don't have a
> good solution now.
>
> I'm still not convinced that Neil's "given" example will even work (see
> below) but *if he is right* that it does, perhaps that's a good reason
> to prefer the simpler := assignment expression syntax, since we're
> less likely to use it in confusing ways.
>
>
> > But it's really REALLY not obvious.
>
> But is it even legal?
>
> As I understand it, "given" is an expression, not an addition to
> comprehension syntax. In that case, I don't think Neil's example will
> work at all, for reasons I've already stated.
>
> If that's not the case, then until somebody tells me what this new
> comprehension syntax means, and what it looks like, I have no idea what
> is intended.
>
> Which of these can we write, and what do they do?
>

Great question.  The trick is to just write them as a sequence of
statements without changing the order except to put the expression last.


>
> [expression given name=something for x in seq]
>

retval = []
name = something
for x in seq:
retval.append(expression)
return retval

>
> [expression for x given name=something in seq]
>

this one doesn't make sense.

[expression for x in seq given name=something]
>
>
retval = []
for x in seq:
name = something
retval.append(expression)
return retval


> [expression for x in seq if given name=something condition]
>
> this one doesn't make sense.


> [expression for x in seq if condition given name=something]
>
> retval = []
for x in seq:
if condition:
name = something
retval.append(expression)
return retval

and of course, the original proposal

expression given name=something

means:

name = something
retval = expression
return retval


>
> --
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
> --
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "python-ideas" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/python-ideas/keaR3FudcwQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> python-ideas+unsubscr...@googlegroups.com.
> For more options, visit 

Re: [Python-ideas] A real life example of "given"

[Neil Girdhar]

On Wed, May 30, 2018 at 7:54 PM Steven D'Aprano  wrote:

> On Wed, May 30, 2018 at 01:59:37PM -0400, Neil Girdhar wrote:
>
> > This example shows additional flexibility:
> >
> > z = {a: transformed_b
> >  for b in bs
> >  given transformed_b = transform(b)
> >  for a in as_}
>
> Is that even legal?
>

In case you missed my earlier reply to you:

One addition to the grammar would be to "test" for something like

test: bool_test [comp_given]
bool_test: or_test ['if' or_test 'else' test] | lambdef
comp_given: 'given' testlist_star_expr annassign

The second would permit the usage in comprehensions:

comp_iter: comp_for | comp_if | comp_given


>
> Again, you're putting half of the comprehension in the middle of
> the given expression. I believe that "given" expression syntax is:
>
> expression given name = another_expression
>
> it's not a syntactic form that we can split across arbitrary chunks of
> code:
>
> # surely this won't be legal?
> def method(self, arg, x=spam):
> body
> given spam = expression
>
>
> Comprehension syntax in this case is:
>
> {key:expr for b in it1 for a in it2}
>
> (of course comprehensions can also include more loops and if clauses,
> but this example doesn't use those). So you've interleaved part of the
> given expression and part of the comprehension:
>
> {key: expression COMPRE- given name = another_expression -HENSION}
>
>
> That's the second time you've done that. Neil, if my analysis is
> correct, I think you have done us a great service: showing that the
> "given" expression syntax really encourages people to generate syntax
> errors in their comprehensions.
>

> > There is no nice, equivalent := version as far as I can tell.
>
> Given (pun intended) the fact that you only use transformed_b in a
> single place, I don't think it is necessary to use := at all.
>
> z = {a: transform(b) for b in bs for a in as_}
>
> But if you really insist:
>
> # Pointless use of :=
> z = {a: (transformed_b := transform(b)) for b in bs for a in as_}
>
>
Those call transform for every a needlessly.


>
>
> --
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
> --
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "python-ideas" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/python-ideas/keaR3FudcwQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> python-ideas+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A real life example of "given"

[Neil Girdhar]

On Wed, May 30, 2018 at 8:10 PM Steven D'Aprano  wrote:

> On Thu, May 31, 2018 at 04:06:51AM +1000, Chris Angelico wrote:
> > On Thu, May 31, 2018 at 3:59 AM, Neil Girdhar 
> wrote:
> > > This example shows additional flexibility:
> > >
> > > z = {a: transformed_b
> > >  for b in bs
> > >  given transformed_b = transform(b)
> > >  for a in as_}
> > >
> > > There is no nice, equivalent := version as far as I can tell.
> >
> > True. However, it took me several readings to understand what you were
> > doing here.
>
> Possibly you shouldn't have tried reading at 4am.
>
> Either that or I shouldn't be reading before I've had a coffee :-)
>
> Have I missed something that you have seen? Even if the syntax were
> legal, that seems to be a pointless use of an assignment expression.
> Since the new name "transformed_b" is only used once, we can and should
> just use the transform(b) in place:
>
> z = {a: transform(b) for b in bs for a in as_}
>

Chris just explained it to you.  You're calling transform too often.


>
> If we need to use it twice, we can do this:
>
> # assume "@" stands in for something useful
> z = {a: (transformed_b := transform(b)) @ transformed_b
>  for b in bs for a in as_}
>
>
> I'm not seeing the advantage of given, or any extra flexibility here,
> unless the aim is to encourage people to make syntax errors :-)
>
>
The flexibility of "given" is in giving names to elements of expressions
and comprehensions to avoid recalculation.


> What have I missed?
>
> Like you say, := and given both work for expressions.  "given" could
theoretically also be used in comprehensions.



>
>
> --
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
> --
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "python-ideas" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/python-ideas/keaR3FudcwQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> python-ideas+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A real life example of "given"

[Steven D'Aprano]

On Thu, May 31, 2018 at 10:05:33AM +1000, Chris Angelico wrote:
> On Thu, May 31, 2018 at 9:53 AM, Steven D'Aprano  wrote:
> >> There is no nice, equivalent := version as far as I can tell.
> >
> > Given (pun intended) the fact that you only use transformed_b in a
> > single place, I don't think it is necessary to use := at all.
> >
> > z = {a: transform(b) for b in bs for a in as_}
> >
> > But if you really insist:
> >
> > # Pointless use of :=
> > z = {a: (transformed_b := transform(b)) for b in bs for a in as_}
> >
> 
> That's the subtlety of the 'given' usage here. You fell for the same
> trap I did: thinking "it's only used once".

But it is only used once. I meant once per loop.

It isn't used in the "for a in as_" inner loop, there's no "if 
transformed_b" condition, and it only is used once in the key:value part 
of the comprehension.


> Actually, what he has is equivalent to:
> 
> z = {a: tb for b in bs for tb in [transform(b)] for a in as_}

Which also uses tb only once, making it a Useless Use Of Assignment.

(I assume we're not calling transform() for some side-effect, like 
logging a message, or erasing your hard drive.)


> which means it evaluates transform(b) once regardless of the length of
> as_. 

Ah yes, I see what you mean. Expanded to a loop:

for b in bs:
tb = transform(b)
for a in as_:
z[a] = tb


It's a little ugly, but there's a trick I already use today:

py> [x+y for x in "abc" if print(x) or True for y in "de"]
a
b
c
['ad', 'ae', 'bd', 'be', 'cd', 'ce']

So we can adapt that to assignment instead of output:

# Don't do this!
z = {a: tb for b in bs if (tb := transform(b)) or True for a in as_}

But I wouldn't do that. If I'm concerned about the call to transform 
(because it is super expensive, say) then I set up a pipeline:

tbs = (transform(b) for b in bs)  # or map(transform, bs)
z = {a: tb for tb in tbs for a in as_}

The first generator comprehension can be easily embedded in the other:

z = {a: tb for tb in (transform(b) for b in bs) for a in as_}

This makes it super-obvious that transform is called for each b, not for 
each (b, a) pair, it works today, and there's no assignment expression 
needed at all.

Assignment expressions should not be about adding yet a third way to 
solve a problem that already has a perfectly good solution! ("Expand to 
a loop statement" is not a *perfectly* good solution.) To showcase 
assignment expressions, we should be solving problems that don't have a 
good solution now.

I'm still not convinced that Neil's "given" example will even work (see 
below) but *if he is right* that it does, perhaps that's a good reason 
to prefer the simpler := assignment expression syntax, since we're 
less likely to use it in confusing ways.


> But it's really REALLY not obvious. 

But is it even legal?

As I understand it, "given" is an expression, not an addition to 
comprehension syntax. In that case, I don't think Neil's example will 
work at all, for reasons I've already stated.

If that's not the case, then until somebody tells me what this new 
comprehension syntax means, and what it looks like, I have no idea what 
is intended.

Which of these can we write, and what do they do?

[expression given name=something for x in seq]

[expression for x given name=something in seq]

[expression for x in seq given name=something]

[expression for x in seq if given name=something condition]

[expression for x in seq if condition given name=something]



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal: A Reduce-Map Comprehension and a "last" builtin

[Rob Cliffe via Python-ideas]

On 30/05/2018 17:05, Peter O'Connor wrote:


On Thu, May 24, 2018 at 2:49 PM, Steven D'Aprano > wrote:


On Thu, May 24, 2018 at 02:06:03PM +0200, Peter O'Connor wrote:
> We could use given for both the in-loop variable update and the
variable
> initialization:
>    smooth_signal =  [average given average=(1-decay)*average +
decay*x for
> x in signal] given average=0.

So in your example, the OUTER "given" creates a local variable in the
current scope, average=0, but the INNER "given" inside the
comprehension
exists inside a separate, sub-local comprehension scope, where you
will
get an UnboundLocalError when it tries to evaluate
(1-decay)*average the
first time.


You're right, having re-thought it, it seems that the correct way to 
write it would be to define both of them in the scope of the 
comprehension:


  smooth_signal =  [average given average=(1-decay)*average + decay*x 
for x in signal given average=0.]


This makes sense and follows a simple rule: "B given A" just causes A 
to be executed before B - that holds true whether B is a variable or a 
loop declaration like "for x in x_gen".


So

    a_gen = (g(a) given a=f(a, x) for x in x_gen given a=0)

would be a compact form of:

    def a_gen_func(x_gen):
        a=0
        for x in x_gen:
            a = f(a, x)
            yield g(a)
    a_gen = a_gen_func()
[There is a typo here - a_gen_func is defined to take 1 argument but is 
called with none.]


After - *I think* - understanding this, I would try to make the one-line 
clearer by parenthesizing it thus (whether or not the grammar required it):


    a_gen = ( ((g(a) given a=f(a, x)) for x in x_gen) given a=0)

Even then, it would make my head spin if I came across it.  I hope 
no-one would write code like that.


I'm not keen on given, but I must admit that ISTM that this example 
shows something that can only be done with given: putting some 
initialisation, viz. "a=0", into a generator expression.  With :=, it 
would need a trick:

    a_gen = (g( a:=f(a, x) ) for x in [x_gen, a:=0][0] )
or
    a_gen = (g( a:=f(a, x) ) for a in [0] for x in x_gen] )

Of course, in the case of a list comprehension (as opposed to a genexp), 
the initialisation could be done separately:

    a = 0
    a_list = [g( a:=f(a, x) ) for x in x_gen]

Rob Cliffe
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A real life example of "given"

[Chris Angelico]

On Thu, May 31, 2018 at 10:05 AM, Steven D'Aprano  wrote:
> On Thu, May 31, 2018 at 04:06:51AM +1000, Chris Angelico wrote:
>> On Thu, May 31, 2018 at 3:59 AM, Neil Girdhar  wrote:
>> > This example shows additional flexibility:
>> >
>> > z = {a: transformed_b
>> >  for b in bs
>> >  given transformed_b = transform(b)
>> >  for a in as_}
>> >
>> > There is no nice, equivalent := version as far as I can tell.
>>
>> True. However, it took me several readings to understand what you were
>> doing here.
>
> Possibly you shouldn't have tried reading at 4am.
>
> Either that or I shouldn't be reading before I've had a coffee :-)
>
> Have I missed something that you have seen?

Yep, as mentioned in the other post. The fact that you talk like this
about it - asserting that it's obvious what this does, while still
considering it to be utterly useless - is proof, IMO, that this should
be frowned upon in style guides.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A real life example of "given"

[Steven D'Aprano]

On Thu, May 31, 2018 at 04:06:51AM +1000, Chris Angelico wrote:
> On Thu, May 31, 2018 at 3:59 AM, Neil Girdhar  wrote:
> > This example shows additional flexibility:
> >
> > z = {a: transformed_b
> >  for b in bs
> >  given transformed_b = transform(b)
> >  for a in as_}
> >
> > There is no nice, equivalent := version as far as I can tell.
> 
> True. However, it took me several readings to understand what you were
> doing here.

Possibly you shouldn't have tried reading at 4am.

Either that or I shouldn't be reading before I've had a coffee :-)

Have I missed something that you have seen? Even if the syntax were 
legal, that seems to be a pointless use of an assignment expression. 
Since the new name "transformed_b" is only used once, we can and should 
just use the transform(b) in place:

z = {a: transform(b) for b in bs for a in as_}

If we need to use it twice, we can do this:

# assume "@" stands in for something useful
z = {a: (transformed_b := transform(b)) @ transformed_b 
 for b in bs for a in as_}


I'm not seeing the advantage of given, or any extra flexibility here, 
unless the aim is to encourage people to make syntax errors :-)

What have I missed?



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A real life example of "given"

[Chris Angelico]

On Thu, May 31, 2018 at 9:53 AM, Steven D'Aprano  wrote:
>> There is no nice, equivalent := version as far as I can tell.
>
> Given (pun intended) the fact that you only use transformed_b in a
> single place, I don't think it is necessary to use := at all.
>
> z = {a: transform(b) for b in bs for a in as_}
>
> But if you really insist:
>
> # Pointless use of :=
> z = {a: (transformed_b := transform(b)) for b in bs for a in as_}
>

That's the subtlety of the 'given' usage here. You fell for the same
trap I did: thinking "it's only used once". Actually, what he has is
equivalent to:

z = {a: tb for b in bs for tb in [transform(b)] for a in as_}

which means it evaluates transform(b) once regardless of the length of
as_. But it's really REALLY not obvious. That's why I actually prefer
the "interpolated 'for' loop" notation, despite it being distinctly
distasteful in general. At least it's obvious that something weird is
happening, so you don't instantly assume that you can inline the
single usage.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A real life example of "given"

[Steven D'Aprano]

On Wed, May 30, 2018 at 01:59:37PM -0400, Neil Girdhar wrote:

> This example shows additional flexibility:
> 
> z = {a: transformed_b
>  for b in bs
>  given transformed_b = transform(b)
>  for a in as_}

Is that even legal?

Again, you're putting half of the comprehension in the middle of 
the given expression. I believe that "given" expression syntax is:

expression given name = another_expression

it's not a syntactic form that we can split across arbitrary chunks of 
code:

# surely this won't be legal?
def method(self, arg, x=spam):
body
given spam = expression


Comprehension syntax in this case is:

{key:expr for b in it1 for a in it2}

(of course comprehensions can also include more loops and if clauses, 
but this example doesn't use those). So you've interleaved part of the 
given expression and part of the comprehension:

{key: expression COMPRE- given name = another_expression -HENSION}


That's the second time you've done that. Neil, if my analysis is 
correct, I think you have done us a great service: showing that the 
"given" expression syntax really encourages people to generate syntax 
errors in their comprehensions.


> There is no nice, equivalent := version as far as I can tell.

Given (pun intended) the fact that you only use transformed_b in a 
single place, I don't think it is necessary to use := at all.

z = {a: transform(b) for b in bs for a in as_}

But if you really insist:

# Pointless use of :=
z = {a: (transformed_b := transform(b)) for b in bs for a in as_}



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A real life example of "given"

[Adam Bartoš]

A side comment:

> potential_updates = {
> y: potential_update
> for x in need_initialization_nodes
> for y in [x, *x.synthetic_inputs()]
> given potential_update = command.create_potential_update(y)
> if potential_update is not None}

Probably, I would write

@make_dict
def potential_updates():
for x in need_initialization_nodes:
for y in [x, *x.synthetic_inputs()]:
potential_update = command.create_potential_update(y)
if potential_update is not None:
yield y, potential_update

If such pattern covered a lot of examples, even some syntax sugar could be
added. For example:

foo[0].potential_updates = dict(_) from:
for x in need_initialization_nodes:
for y in [x, *x.synthetic_inputs()]:
potential_update = command.create_potential_update(y)
if potential_update is not None:
yield y, potential_update

where

 from:


would be equivalent to

def _():

_ = _()


Best regards,
Adam Bartoš
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A real life example of "given"

[Neil Girdhar]

Peter wrote:

> Well you could just do:

z = {a: b
 for b in (transform(bi) for bi in bs)
 for a in as_}

That works, but I prefer the implicit nesting of a sequence of "comp_for"
expressions to a the nested generator.

On Wed, May 30, 2018 at 2:16 PM Chris Angelico  wrote:

> On Thu, May 31, 2018 at 3:59 AM, Neil Girdhar 
> wrote:
> > This example shows additional flexibility:
> >
> > z = {a: transformed_b
> >  for b in bs
> >  given transformed_b = transform(b)
> >  for a in as_}
> >
> > There is no nice, equivalent := version as far as I can tell.
>
> True. However, it took me several readings to understand what you were
> doing here. I think I actually prefer "for transformed_b in
> [transform(b)]" to this syntax, which is saying something.
>

I feel you.  I think of "given" as an assignment that is in front of the
expression, just like "for" (in comp_for) is a for loop that is in front,
and "if" (in comp_if) is a condition that is in front.


>
> ChrisA
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
> --
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "python-ideas" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/python-ideas/keaR3FudcwQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> python-ideas+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A real life example of "given"

[Chris Angelico]

On Thu, May 31, 2018 at 3:59 AM, Neil Girdhar  wrote:
> This example shows additional flexibility:
>
> z = {a: transformed_b
>  for b in bs
>  given transformed_b = transform(b)
>  for a in as_}
>
> There is no nice, equivalent := version as far as I can tell.

True. However, it took me several readings to understand what you were
doing here. I think I actually prefer "for transformed_b in
[transform(b)]" to this syntax, which is saying something.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A real life example of "given"

[Peter O'Connor]

On Wed, May 30, 2018 at 7:59 PM, Neil Girdhar  wrote:
>
> z = {a: transformed_b
>  for b in bs
>  given transformed_b = transform(b)
>  for a in as_}
>
> There is no nice, equivalent := version as far as I can tell.
>

Well you could just do:

z = {a: b
 for b in (transform(bi) for bi in bs)
 for a in as_}
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A real life example of "given"

[Neil Girdhar]

On Wed, May 30, 2018 at 1:56 PM Neil Girdhar  wrote:

> On Wed, May 30, 2018 at 1:52 PM Chris Angelico  wrote:
>
>> On Thu, May 31, 2018 at 1:23 AM, Peter O'Connor
>>  wrote:
>> >>  In comparison, I think that := is much simpler.
>> >
>> >
>> > In this case that's true, but a small modification:
>> >
>> > updates = {
>> > y: do_something_to(potential_update)
>> > for x in need_initialization_nodes
>> > for y in [x, *x.synthetic_inputs()]
>> > if potential_update is not None
>> > given potential_update = command.create_potential_update(y)
>> > }
>> >
>> > Shows the flexibility of this given syntax vs ":="
>>
>> I don't understand what you're showcasing here. With :=, you give a
>> name to something at the exact point that it happens:
>>
>> updates = {
>> y: do_something_to(potential_update)
>> for x in need_initialization_nodes
>> for y in [x, *x.synthetic_inputs()]
>> if (potential_update :=
>> command.create_potential_update(y)) is not None
>> }
>>
>> Personally, I'd use a shorter name for something that's used in such a
>> small scope (same as you use one-letter "x" and "y"). But that's the
>> only way that the 'given' syntax looks at all better - by encouraging
>> you to use yet another line, it conceals some of its immense
>> verbosity. (Note how the name "potential_update" is used twice with
>> :=, once to set and one to retrieve; but with given, it's used three
>> times - retrieve, retrieve, and set.)
>>
>> How does this show that 'given' is more flexible?
>>
>>
> Oh yeah, good point, I forgot that I could use := within the condition
> itself.  It does show that this feature is useful, but not that given is
> more flexible than :=.
>

This example shows additional flexibility:

z = {a: transformed_b
 for b in bs
 given transformed_b = transform(b)
 for a in as_}

There is no nice, equivalent := version as far as I can tell.


>
>
>> ChrisA
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>> --
>>
>> ---
>> You received this message because you are subscribed to a topic in the
>> Google Groups "python-ideas" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/python-ideas/keaR3FudcwQ/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> python-ideas+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A real life example of "given"

[Neil Girdhar]

On Wed, May 30, 2018 at 1:52 PM Chris Angelico  wrote:

> On Thu, May 31, 2018 at 1:23 AM, Peter O'Connor
>  wrote:
> >>  In comparison, I think that := is much simpler.
> >
> >
> > In this case that's true, but a small modification:
> >
> > updates = {
> > y: do_something_to(potential_update)
> > for x in need_initialization_nodes
> > for y in [x, *x.synthetic_inputs()]
> > if potential_update is not None
> > given potential_update = command.create_potential_update(y)
> > }
> >
> > Shows the flexibility of this given syntax vs ":="
>
> I don't understand what you're showcasing here. With :=, you give a
> name to something at the exact point that it happens:
>
> updates = {
> y: do_something_to(potential_update)
> for x in need_initialization_nodes
> for y in [x, *x.synthetic_inputs()]
> if (potential_update :=
> command.create_potential_update(y)) is not None
> }
>
> Personally, I'd use a shorter name for something that's used in such a
> small scope (same as you use one-letter "x" and "y"). But that's the
> only way that the 'given' syntax looks at all better - by encouraging
> you to use yet another line, it conceals some of its immense
> verbosity. (Note how the name "potential_update" is used twice with
> :=, once to set and one to retrieve; but with given, it's used three
> times - retrieve, retrieve, and set.)
>
> How does this show that 'given' is more flexible?
>
>
Oh yeah, good point, I forgot that I could use := within the condition
itself.  It does show that this feature is useful, but not that given is
more flexible than :=.


> ChrisA
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
> --
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "python-ideas" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/python-ideas/keaR3FudcwQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> python-ideas+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A real life example of "given"

[Chris Angelico]

On Thu, May 31, 2018 at 1:23 AM, Peter O'Connor
 wrote:
>>  In comparison, I think that := is much simpler.
>
>
> In this case that's true, but a small modification:
>
> updates = {
> y: do_something_to(potential_update)
> for x in need_initialization_nodes
> for y in [x, *x.synthetic_inputs()]
> if potential_update is not None
> given potential_update = command.create_potential_update(y)
> }
>
> Shows the flexibility of this given syntax vs ":="

I don't understand what you're showcasing here. With :=, you give a
name to something at the exact point that it happens:

updates = {
y: do_something_to(potential_update)
for x in need_initialization_nodes
for y in [x, *x.synthetic_inputs()]
if (potential_update :=
command.create_potential_update(y)) is not None
}

Personally, I'd use a shorter name for something that's used in such a
small scope (same as you use one-letter "x" and "y"). But that's the
only way that the 'given' syntax looks at all better - by encouraging
you to use yet another line, it conceals some of its immense
verbosity. (Note how the name "potential_update" is used twice with
:=, once to set and one to retrieve; but with given, it's used three
times - retrieve, retrieve, and set.)

How does this show that 'given' is more flexible?

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A real life example of "given"

[Neil Girdhar]

On Wed, May 30, 2018 at 11:32 AM Peter O'Connor 
wrote:

>  In comparison, I think that := is much simpler.
>
>
> In this case that's true, but a small modification:
>
> updates = {
> y: do_something_to(potential_update)
> for x in need_initialization_nodes
> for y in [x, *x.synthetic_inputs()]
> if potential_update is not None
> given potential_update = command.create_potential_update(y)
> }
>
> Shows the flexibility of this given syntax vs ":="
>
> If we think of "given" as just inserting a line with variable-definitions
> before the preceding statement, it seems clear that:
>
> updates = {
> y: potential_update
> given potential_update = command.create_potential_update(y)
> for x in need_initialization_nodes
> for y in [x, *x.synthetic_inputs()]
> if potential_update is not None
> }
>
> Should raise a NameError: name 'potential_update' is not defined, and
>
> updates = {
> y: potential_update
> for x in need_initialization_nodes
> for y in [x, *x.synthetic_inputs()]
> given potential_update = command.create_potential_update(y)
> if potential_update is not None
> }
>
>
> Should raise a NameError: name 'y' is not defined.
>

The reason I want it like that for comprehensions is that I think of it as
equivalent to:

updates = {}
for x in need_initialization_nodes:
for y in [x, *x.synthetic_inputs()]:
potential_update = command.create_potential_update(y)
if potential_update is not None:
updates[y] = potential_update

But you're right that this would be a second addition to the grammar.  One
addition would be to "test" for something like

test: bool_test [comp_given]
bool_test: or_test ['if' or_test 'else' test] | lambdef
comp_given: 'given' testlist_star_expr annassign

The second would permit the usage in comprehensions:

comp_iter: comp_for | comp_if | comp_given

Best,

Neil

For safety it seems reasonable that if a variable is "given" in a
> comprehension, trying to refer to it (even if it defined in the enclosing
> scope) before the inner-definition will result in a NameError.
>
>
> On Wed, May 30, 2018 at 2:22 PM, Steven D'Aprano 
> wrote:
>
>> On Wed, May 30, 2018 at 02:42:21AM -0700, Neil Girdhar wrote:
>>
>> > With "given", I can write:
>> >
>> > potential_updates = {
>> > y: potential_update
>> > for x in need_initialization_nodes
>> > for y in [x, *x.synthetic_inputs()]
>> > given potential_update = command.create_potential_update(y)
>> > if potential_update is not None}
>>
>> I'm not sure if that would be legal for the "given" syntax. As I
>> understand it, the "given" syntax is:
>>
>> expression given name = another_expression
>>
>> but you've got half of the comprehension stuffed in the gap between the
>> leading expression and the "given" keyword:
>>
>> expression COMPREH- given name = another_expression -ENSION
>>
>> so I think that's going to be illegal.
>>
>>
>> I think it wants to be written this way:
>>
>> potential_updates = {
>> y: potential_update
>> for x in need_initialization_nodes
>> for y in [x, *x.synthetic_inputs()]
>> if potential_update is not None
>> given potential_update = command.create_potential_update(y)
>> }
>>
>>
>> Or maybe it should be this?
>>
>> potential_updates = {
>> y: potential_update
>> given potential_update = command.create_potential_update(y)
>> for x in need_initialization_nodes
>> for y in [x, *x.synthetic_inputs()]
>> if potential_update is not None
>> }
>>
>>
>> I'm damned if I know which way is correct. Either of them? Neither?
>>
>> In comparison, I think that := is much simpler. There's only one place
>> it can go:
>>
>> potential_updates = {
>> y: potential_update
>> for x in need_initialization_nodes
>> for y in [x, *x.synthetic_inputs()]
>> if (
>> potential_update := command.create_potential_update(y)
>>) is not None
>> }
>>
>>
>> --
>> Steve
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
> --
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "python-ideas" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/python-ideas/keaR3FudcwQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> python-ideas+unsubscr...@googlegroups.com.
> For more options, visit 

[Python-ideas] Add shutil.chown(..., recursive=False)

[Wes Turner]

On Wednesday, May 30, 2018, Nick Coghlan  wrote:

> On 30 May 2018 at 02:38, Guido van Rossum  wrote:
>
>> Honestly, despite the occasional use case(1), I'm not sure that this is a
>> battery we need in the stdlib. Nobody seems too excited about writing the
>> code, and when the operation is needed, shelling out to the system chown is
>> not too onerous. (Ditto for chmod.)
>>
>> (1) Not even sure that a use case was shown -- it was just shown that the
>> operation is not necessarily useless.
>>
>
> My main use cases have been in installers and test suites, but those cases
> have also been for Linux-specific code where shelling out to "chown -R" and
> "chmod -R" was an entirely acceptable alternative.
>
> I think one of the other key points here is that "chown" and "chmod"
> inherently don't map at all well to the Windows filesystem access control
> model [1], so there's no new portability challenges arising from expecting
> the chown and chmod commands to be available.
>
> Cheers,
> Nick.
>
> [1] os.chown and shutil.chown don't exist at all there, and os.chmod only
> supports setting a file to read-only - there isn't any access to user or
> group permissions.
>

Notably, Ansible and Salt don't haven't even tried to abstract Linux/BSD
and Windows filesystem ACLs into a common API:

https://github.com/ansible/ansible/blob/devel/lib/ansible/
modules/files/acl.py
https://github.com/ansible/ansible/blob/devel/lib/ansible/
modules/windows/win_acl.py
https://github.com/ansible/ansible/blob/devel/lib/
ansible/modules/windows/win_acl.ps1


https://github.com/saltstack/salt/blob/develop/salt/states/file.py

https://github.com/saltstack/salt/blob/develop/salt/states/win_dacl.py
https://github.com/saltstack/salt/blob/develop/salt/modules/win_dacl.py
https://github.com/saltstack/salt/blob/develop/salt/utils/win_dacl.py

https://github.com/saltstack/salt/blob/develop/salt/states/linux_acl.py
https://github.com/saltstack/salt/blob/develop/salt/modules/linux_acl.py

... Looked these up and thought I'd share.

>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal: A Reduce-Map Comprehension and a "last" builtin

[Peter O'Connor]

On Thu, May 24, 2018 at 2:49 PM, Steven D'Aprano 
wrote:

> On Thu, May 24, 2018 at 02:06:03PM +0200, Peter O'Connor wrote:
> > We could use given for both the in-loop variable update and the variable
> > initialization:
> >smooth_signal =  [average given average=(1-decay)*average + decay*x
> for
> > x in signal] given average=0.
>
> So in your example, the OUTER "given" creates a local variable in the
> current scope, average=0, but the INNER "given" inside the comprehension
> exists inside a separate, sub-local comprehension scope, where you will
> get an UnboundLocalError when it tries to evaluate (1-decay)*average the
> first time.


You're right, having re-thought it, it seems that the correct way to write
it would be to define both of them in the scope of the comprehension:

smooth_signal =  [average given average=(1-decay)*average + decay*x for x
in signal given average=0.]

This makes sense and follows a simple rule: "B given A" just causes A to be
executed before B - that holds true whether B is a variable or a loop
declaration like "for x in x_gen".

So

a_gen = (g(a) given a=f(a, x) for x in x_gen given a=0)

would be a compact form of:

def a_gen_func(x_gen):
a=0
for x in x_gen:
a = f(a, x)
yield g(a)

a_gen = a_gen_func()
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A real life example of "given"

[Peter O'Connor]

>
>  In comparison, I think that := is much simpler.


In this case that's true, but a small modification:

updates = {
y: do_something_to(potential_update)
for x in need_initialization_nodes
for y in [x, *x.synthetic_inputs()]
if potential_update is not None
given potential_update = command.create_potential_update(y)
}

Shows the flexibility of this given syntax vs ":="

If we think of "given" as just inserting a line with variable-definitions
before the preceding statement, it seems clear that:

updates = {
y: potential_update
given potential_update = command.create_potential_update(y)
for x in need_initialization_nodes
for y in [x, *x.synthetic_inputs()]
if potential_update is not None
}

Should raise a NameError: name 'potential_update' is not defined, and

updates = {
y: potential_update
for x in need_initialization_nodes
for y in [x, *x.synthetic_inputs()]
given potential_update = command.create_potential_update(y)
if potential_update is not None
}


Should raise a NameError: name 'y' is not defined.

For safety it seems reasonable that if a variable is "given" in a
comprehension, trying to refer to it (even if it defined in the enclosing
scope) before the inner-definition will result in a NameError.


On Wed, May 30, 2018 at 2:22 PM, Steven D'Aprano 
wrote:

> On Wed, May 30, 2018 at 02:42:21AM -0700, Neil Girdhar wrote:
>
> > With "given", I can write:
> >
> > potential_updates = {
> > y: potential_update
> > for x in need_initialization_nodes
> > for y in [x, *x.synthetic_inputs()]
> > given potential_update = command.create_potential_update(y)
> > if potential_update is not None}
>
> I'm not sure if that would be legal for the "given" syntax. As I
> understand it, the "given" syntax is:
>
> expression given name = another_expression
>
> but you've got half of the comprehension stuffed in the gap between the
> leading expression and the "given" keyword:
>
> expression COMPREH- given name = another_expression -ENSION
>
> so I think that's going to be illegal.
>
>
> I think it wants to be written this way:
>
> potential_updates = {
> y: potential_update
> for x in need_initialization_nodes
> for y in [x, *x.synthetic_inputs()]
> if potential_update is not None
> given potential_update = command.create_potential_update(y)
> }
>
>
> Or maybe it should be this?
>
> potential_updates = {
> y: potential_update
> given potential_update = command.create_potential_update(y)
> for x in need_initialization_nodes
> for y in [x, *x.synthetic_inputs()]
> if potential_update is not None
> }
>
>
> I'm damned if I know which way is correct. Either of them? Neither?
>
> In comparison, I think that := is much simpler. There's only one place
> it can go:
>
> potential_updates = {
> y: potential_update
> for x in need_initialization_nodes
> for y in [x, *x.synthetic_inputs()]
> if (
> potential_update := command.create_potential_update(y)
>) is not None
> }
>
>
> --
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Was `os.errno` undocumented?

[Guido van Rossum]

Mypy would totally catch this. PEP 484 has specific words for whether
imports are re-exported (they aren't unless you use 'as').

On Wed, May 30, 2018 at 4:58 AM, Steven D'Aprano 
wrote:

> On Wed, May 30, 2018 at 04:18:51AM +, Steve Barnes wrote:
>
> > Maybe what we need is to add a, possibly optional, or suppressible,
> > warning whenever the import system encounters an implicit/indirect
> > import?
>
> I don't think your terminology ("implicit/indirect") is very accurate.
>
> from os import errno
>
> is as explicit as you can get.
>
>
> > If an import that is working because the package we are
> > importing it from has imported it from elsewhere, but it is not included
> > in the __init__ for that package,
>
> I think you mean __all__ for the module.
>
> I'm not sure how that check would work. For a simple module, whenever
> you call "from module import name", the interpreter has to inspect the
> object it just imported, and if it is a module itself, check whether
> "name" is in the owning module's __all__.
>
> How would it work for packages? "from package import submodule" ought to
> work without a warning even if submodule isn't listed in __all__.
>
> Even for simple modules, it is prone to false positives: if "name" is
> documented as public, but not listed in __all__ then that would wrongly
> be detected as a non-public import.
>
> But most problematic, it does nothing about this case:
>
> import os
> os.errno
>
> I think this is best left for linters.
>
>
> --
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add shutil.chown(..., recursive=False)

[Nick Coghlan]

On 30 May 2018 at 02:38, Guido van Rossum  wrote:

> Honestly, despite the occasional use case(1), I'm not sure that this is a
> battery we need in the stdlib. Nobody seems too excited about writing the
> code, and when the operation is needed, shelling out to the system chown is
> not too onerous. (Ditto for chmod.)
>
> (1) Not even sure that a use case was shown -- it was just shown that the
> operation is not necessarily useless.
>

My main use cases have been in installers and test suites, but those cases
have also been for Linux-specific code where shelling out to "chown -R" and
"chmod -R" was an entirely acceptable alternative.

I think one of the other key points here is that "chown" and "chmod"
inherently don't map at all well to the Windows filesystem access control
model [1], so there's no new portability challenges arising from expecting
the chown and chmod commands to be available.

Cheers,
Nick.

[1] os.chown and shutil.chown don't exist at all there, and os.chmod only
supports setting a file to read-only - there isn't any access to user or
group permissions.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A real life example of "given"

[Steven D'Aprano]

On Wed, May 30, 2018 at 02:42:21AM -0700, Neil Girdhar wrote:

> With "given", I can write:
> 
> potential_updates = {
> y: potential_update
> for x in need_initialization_nodes
> for y in [x, *x.synthetic_inputs()]
> given potential_update = command.create_potential_update(y)
> if potential_update is not None}

I'm not sure if that would be legal for the "given" syntax. As I 
understand it, the "given" syntax is:

expression given name = another_expression

but you've got half of the comprehension stuffed in the gap between the 
leading expression and the "given" keyword:

expression COMPREH- given name = another_expression -ENSION

so I think that's going to be illegal.


I think it wants to be written this way:

potential_updates = {
y: potential_update
for x in need_initialization_nodes
for y in [x, *x.synthetic_inputs()]
if potential_update is not None
given potential_update = command.create_potential_update(y)
}


Or maybe it should be this?

potential_updates = {
y: potential_update
given potential_update = command.create_potential_update(y)
for x in need_initialization_nodes
for y in [x, *x.synthetic_inputs()]
if potential_update is not None
}


I'm damned if I know which way is correct. Either of them? Neither?

In comparison, I think that := is much simpler. There's only one place 
it can go:

potential_updates = {
y: potential_update
for x in need_initialization_nodes
for y in [x, *x.synthetic_inputs()]
if (
potential_update := command.create_potential_update(y)
   ) is not None
}


-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Was `os.errno` undocumented?

[Steven D'Aprano]

On Wed, May 30, 2018 at 04:18:51AM +, Steve Barnes wrote:

> Maybe what we need is to add a, possibly optional, or suppressible, 
> warning whenever the import system encounters an implicit/indirect 
> import?

I don't think your terminology ("implicit/indirect") is very accurate.

from os import errno

is as explicit as you can get.


> If an import that is working because the package we are 
> importing it from has imported it from elsewhere, but it is not included 
> in the __init__ for that package,

I think you mean __all__ for the module.

I'm not sure how that check would work. For a simple module, whenever 
you call "from module import name", the interpreter has to inspect the 
object it just imported, and if it is a module itself, check whether 
"name" is in the owning module's __all__.

How would it work for packages? "from package import submodule" ought to 
work without a warning even if submodule isn't listed in __all__.

Even for simple modules, it is prone to false positives: if "name" is 
documented as public, but not listed in __all__ then that would wrongly 
be detected as a non-public import.

But most problematic, it does nothing about this case:

import os
os.errno

I think this is best left for linters.


-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] A real life example of "given"

[Neil Girdhar]

I thought I would share a recent use I had for "given":

I have this comprehension:

potential_updates = {y: command.create_potential_update(y)
 for x in need_initialization_nodes
 for y in [x, *x.synthetic_inputs()]}

I want to filter out values that are None.  I don't want to call the 
function call twice, so I have to resort to using a loop and appending or 
the for z in [y] trick.  With "given", I can write:

potential_updates = {
y: potential_update
for x in need_initialization_nodes
for y in [x, *x.synthetic_inputs()]
given potential_update = command.create_potential_update(y)
if potential_update is not None}

I also wanted to point out that code like this is self-commenting because 
the key and value of the comprehension can be given names.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add shutil.chown(..., recursive=False)

[Wes Turner]

Configuration management tools with far more code than this are regularly
run with root privileges.

OTOH, Salt and Ansible, for example, both support recursive chown and
chmod; and report what actually changed. Yum/dnf probably do, too.

Supporting recursive chmod/chown OOB may be useful. That it might be run as
root is not the criteria, AFAIU.

On Tuesday, May 29, 2018, Chris Angelico  wrote:

> On Wed, May 30, 2018 at 9:11 AM, Greg Ewing 
> wrote:
> > BTW, I wouldn't argue that Python shouldn't provide things
> > that are only useful to root. While writing setuid utilities
> > in Python is a bad idea for lots of reasons, I don't think
> > there's anything wrong with becoming root by another means
> > and then running a Python program that you know well enough
> > to trust.
>
> I'd go further. Once a shell script gets longer than about a page or
> two of code, it often needs to be rewritten in a different language,
> and Python is well situated to be that language. That doesn't change
> when the script is to be run as root. I've written many Python scripts
> to do sysadminning jobs for me - usually one-shot scripts, but also
> some that stick around. Since I wrote the scripts myself, the trust
> issue doesn't come up; I trust the Python interpreter the same way
> that I trust /bin/bash.
>
> ChrisA
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/