Re: [Python-ideas] With expressions

[Robert Vanden Eynde]

>
>
>
>
> A with-statement is great for when you care about the
> implementation details. Somebody has to care about the process of
> opening a file, reading from it and closing it. But when you *don't*
> care about those implementation details, a simple interface like a
> read() function is superior to a with-statement, *or* a with-expression,
> which shoves those details in your face.
>

Totally agree. In the case where you care about the implementation details.
You can explain to a beginner:

something = (f.read() with open('...') as f)

Is another way to write:

with open('filename') as f:
something = f.read()

Exactly like the ternary if.

And I agree that because with is often relevant for "implementation
details" or beginning use as a "enter.. exit' statement (like a cpp
destructor), it matches more imperative programming.

with stack.push_matrix(Scale(4)):
triangle = draw_triangle_using(stack)

Is used for it's "enter/close" functionality, but can still be used like:

triangle = (draw_triangle_using(stack) with stack.push_matrix(Scale(4)) as
NotUsed)

But the "as close" is useless here.


>
> > and in most cases when
> > reading the code you then have to look at the function to see if
> > anything else is done there.
>
> I doubt that many people really spend a lot of time digging into
> functions to see whether they do more than what they say. Unless and
> until I ran into unexpected problems, I'd be no more inclined to look
> into a function called "read" than I would be to do the same to len or
> math.sin. I'm sure it does what it says it does.
>

Fair point, the "def" statements generally are passed when read so they
don't really count as overhead.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] With expressions

[Robert Vanden Eynde]

> I know what functional programming is. What I don't understand is what
> you mean when you say that the F.P. community "seems to be more
> interested in python". Surely they are more interested in functional
> languages than a multi-paradigm language like Python which does not
> privilege functional idioms over imperative idioms.
>
>
Just a personal feeling, it's not really thought out.

> > try...except exceptions have been proposed before and rejected.
> >
> > I'm wondering why, that must have been the same reasons of not accepting
> > "with".
>
> Read the PEP. Or the (long!) discussions on Python-Ideas and Python-Dev.
>
> https://www.python.org/dev/peps/pep-0463/
>
> >


I'm reading it. The prelude at the beginning then says there are no
convincing use case if I get it right?


>
> Which is why I said it was not a good example. If you're going to
> propose syntax, you ought to give good examples, not bad examples.
>

When showing toy examples I thought some people would think "indeed, that
happens to me often".


> In any case, this is not a proposal for a "where" expression. You aren't
> going to convince people to add a "with" expression by showing them
> expression forms of "if", "for" or hypothetical "where". Each feature
> must justify itself, not sneak in behind another feature.
>

Indeed, I'm talking about it because I think it all relates to
"expressionalize simple statements", that's also why I speak about FP,
because that's an "expression-first" paradigm.


>
> Or factor your code into named functions with isolated namespaces, so
> the f in one function doesn't clobber the f in another function.
> Structured programming won the debate against unstructured programming a
> long time ago.
>
> https://en.wikipedia.org/wiki/Structured_programming#History
>
>
> [...]


Of course I would do that, i completely agree that refactoring is useful,
but as shown in the end of my post:

filename = compute_filename(...)
lines = compute_lines(...)
parsed_data = compute_parsed_data(...)

The functions body being a bit too small to be refactored and doesn't
really have another meaning of "code to compute filename", I feel like the
"filename =" already catches the idea, I feel like repeating myself (DRY).

And the function body in those cases is not very reusable so it doesn't
make sense to give it a meaningful name.

I would do a named function otherwise indeed.


> > One difficultly of finding use cases, it that it's about changing the
> > paradigm, probably all cases have a really readable implementation in
> > current python / imperative style. But when I see:
> >
> > try:
> > a_variable = int(input("..."))
> > except ValueError:
> > try:
> >a_variable = fetch_db()
> > except DbError:
> >a_variable = default
> >
> > I really think "why can't I put it one one line like I do with if".
> >
> > a_variable = (int(input("...")) except ValueError:
> >fetch_db() except DbError:
> >default)
>
> Whereas when I see somebody using a double if...else expression in a
> single line, I wish they would spread it out over multiple lines so it
> is readable and understandable, instead of trying to squeeze the maximum
> amount of code per line they can.
>

Multiline is clearly better yes. When I say 'one line' I'm just saying
"using the expression-statement but probably gonna span it on multiple
lines because the Expressions are not short.


>
> > For "with", I'm just wondering "why do I have to indent, it will lose the
> > focus of the reader on the result itself".
>
> If this is a problem, then you can easily focus the reader on the
> result by factoring the code into a named function.
>
> text = read('filename')
>
> requires no indentation, no complicated new syntax, and it is easily
> extensible to more complex examples:
>
> text = (prefix + (read('filename') or 'default') + moretext).upper()
>
> This argument isn't about whether it might occasionally be useful to use
> a with expression, but whether it is useful *enough* to justify adding
> new syntax to the language.
>

For 'with', as you all say, it really boils down to finding use cases other
than "file manipulation". It's true I can't think of any (expect the fact
it does create a function that may not have a meaningful name).


>
> --
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] With expressions

[Jeroen Demeyer]

On 2018-08-04 09:43, Steven D'Aprano wrote:

If you want to
propose a general exception suppressing mechanism (aside from the
existing try...except statement) then propose it as an independent PEP.


This was already tried in PEP 463 (which IMHO is a pity that it wasn't 
accepted because it seems really useful).

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] With expressions

[Steven D'Aprano]

On Fri, Aug 03, 2018 at 06:57:40PM -0500, Abe Dillon wrote:

> tmp = None
> with suppress(AttributeError):
> tmp = person.name[0]
> initial = tmp
> 
> Then it would solve many of the common use cases for the None-aware
> operators proposed in PEP 505

No it would not. The None-aware operators are not about suppressing 
AttributeError, please stop suggesting that it is. If you want to 
propose a general exception suppressing mechanism (aside from the 
existing try...except statement) then propose it as an independent PEP.

Even if we had a general purpose exception-suppressing expression, it 
wouldn't meet the functional requirements for PEP 505. It would do too 
much, like offering somebody a bulldozer when all we want is a dustpan 
and broom.



> especially if we made it easy to filter out
> None-specific errors:
> 
> class NoneError(BaseException):
> pass
> 
> class NoneAttributeError(AttributeError, NoneError):
> pass

They're not None specific. Any object can raise them.



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] With expressions

[Steven D'Aprano]

On Sat, Aug 04, 2018 at 12:40:13AM +0200, Benedikt Werner wrote:

> Overall I like the idea of the with-expression as it allows you to make 
> some common use cases like the open/read example more readable. 

For some definition of "readable".

> It's 
> clear at first sight that what is actually done is reading the file.

Only because the file.read() comes early in the expression. But that 
applies equally to 

text = read('filename')


> While it's true that it's usually easy to write a simple function to do 
> the job that still isn't as simple to understand 


(1) read(filename)

Explanation to a beginner:

"It reads text from the named file."


(2) f.read() with open(filename) as f

Explanation to a beginner:

"It opens a file -- I'll explain what it means to open a file 
later -- in a context manager -- that's advanced Python programming, 
don't worry about context managers for now -- and binds the context 
manager to the name f, then calls the read method on the object f -- 
I'll explain object oriented programming later -- which reads text from 
the named file and closes the file... oh I forgot to mention that the 
context manager is also a file object."

Do you still think that explaining a with expression is simpler than 
explaining a self-descriptively named function?


The beauty of the named function is that it hides a lot of irrelevant 
detail and focuses *only* on the important feature, which is reading 
from the file.

A with-statement is great for when you care about the 
implementation details. Somebody has to care about the process of 
opening a file, reading from it and closing it. But when you *don't* 
care about those implementation details, a simple interface like a 
read() function is superior to a with-statement, *or* a with-expression, 
which shoves those details in your face.


> and in most cases when 
> reading the code you then have to look at the function to see if 
> anything else is done there.

I doubt that many people really spend a lot of time digging into 
functions to see whether they do more than what they say. Unless and 
until I ran into unexpected problems, I'd be no more inclined to look 
into a function called "read" than I would be to do the same to len or 
math.sin. I'm sure it does what it says it does.

In Python 2, one of the problems with the input() function was that it 
was used by people who didn't read the docs and where taken completely 
by surprise by the fact that it called eval on the input. That shows 
that people don't make a habit of digging into functions "just in case".


> Also what if you then do readlines 
> somewhere? Then you need another function.

Indeed. But not everything has to be a built-in feature, and refactoring 
common code into a named function will often be a good idea *even if* 
``with`` is an exception:

def read(filename, size=-1, **kwargs):
return f.read(size) with open(filename, **kwargs) as f


-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] With expressions

[Steven D'Aprano]

On Sat, Aug 04, 2018 at 12:28:40AM +0200, Robert Vanden Eynde wrote:

> As the code I showed, being just that:
> 
> filename = ...
> lines = ...
> parsed_data = ...
> 
> With implementation details? The highest value is there, the alg is clear.,
> one fetches the filename, one preprocess the lines, then parse the data.

As programmers, surely we have to care about implementation details of 
the code we are maintaining or writing. If we don't care about 
implementation, who does?

Of course there is a tension between having to deal with implementation 
details too earlier, or hiding them too deeply. We have to balance too 
little refactoring from too much refactoring, and people can 
legitimately disagree as to when a function, method or class carries its 
own weight.

The same applies to syntactic features. That's why *concrete use-cases* 
are important, not pretend, made-up toy examples.

The Python community as a whole is not very receptive to arguments from 
functional programming purity. I know Python veterans who still don't 
like list comprehensions (although they're a minority). As a whole, the 
community does not believe that using multiple statements is always a 
problem to be fixed.

The question is not whether it is *possible* to have a with expression, 
or whether we might find some toy examples that kinda look good, but how 
much it improves *real code*.

And arguments that we should have a "with" expression because we already 
have "if" expressions and comprehensions will just fall flat. Arguments 
by consistency ("we have A, so we ought to have E too, because they're 
both vowels") are not very productive.


-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] With expressions

[Steven D'Aprano]

On Fri, Aug 03, 2018 at 12:49:24PM +0200, Robert Vanden Eynde wrote:

> Le ven. 3 août 2018 à 03:07, Steven D'Aprano  a écrit :
> 
> > On Thu, Aug 02, 2018 at 03:13:25PM +0200, Robert Vanden Eynde wrote:
> >
> > > This brings the discussion of variable assignement in Expression.
> > Functional
> > > programming community seems to be more interested in python.
> >
> > I'm not sure what you mean there. Your English grammar is just slightly
> > off, enough to make your meaning unclear, sorry.
> >
> 
> When I say "functional programming", I speak about the paradigm used in
> language like Haskell.

I know what functional programming is. What I don't understand is what 
you mean when you say that the F.P. community "seems to be more 
interested in python". Surely they are more interested in functional 
languages than a multi-paradigm language like Python which does not 
privilege functional idioms over imperative idioms.


[...]
> > try...except exceptions have been proposed before and rejected.
> 
> I'm wondering why, that must have been the same reasons of not accepting
> "with".

Read the PEP. Or the (long!) discussions on Python-Ideas and Python-Dev.

https://www.python.org/dev/peps/pep-0463/


> > > value = (x+y**2 where x,y = (2,4))
> >
> > A "where" *statement* is interesting, but this is not a good example of
> > it. The above is better written in the normal syntax:
> >
> > value = 2 + 4**2
> 
> 
> That's the discussion we had on the list called "variable assignement in
> expressions". What you did here is inlining the variables, technically it's
> not possible if you're calling a function and using the variable more than
> once.

Which is why I said it was not a good example. If you're going to 
propose syntax, you ought to give good examples, not bad examples.

In any case, this is not a proposal for a "where" expression. You aren't 
going to convince people to add a "with" expression by showing them 
expression forms of "if", "for" or hypothetical "where". Each feature 
must justify itself, not sneak in behind another feature.


[...]
> > > with open('hello') as f:
> > > lines = f.readlines()
> > > del f  # f is leaked !
> >
> > 99% of the time, I would think that "del f" was a waste of time. If that
> > code is in function, then f will be closed when the "with" block is
> > exited, and garbage collected when the function returns.
> 
> Yes of course, but what if "f" was defined before? We lose its value, even
> if "f" was created only as a temporary variable to have the variables lines.
[...]
> But maybe we are in a script and we have a lots of
> variables? That kind of questions arise, when we wanted a temporary
> variable.

Then don't use f. Use F, fi, fp, fileobj, f_, f1, ϕ, фп, or 
myfileobject. Surely you haven't used them all. There is no shortage of 
useful identifier names.

Or factor your code into named functions with isolated namespaces, so 
the f in one function doesn't clobber the f in another function. 
Structured programming won the debate against unstructured programming a 
long time ago.

https://en.wikipedia.org/wiki/Structured_programming#History


[...]
> One difficultly of finding use cases, it that it's about changing the
> paradigm, probably all cases have a really readable implementation in
> current python / imperative style. But when I see:
> 
> try:
> a_variable = int(input("..."))
> except ValueError:
> try:
>a_variable = fetch_db()
> except DbError:
>a_variable = default
> 
> I really think "why can't I put it one one line like I do with if".
> 
> a_variable = (int(input("...")) except ValueError:
>fetch_db() except DbError:
>default)

Whereas when I see somebody using a double if...else expression in a 
single line, I wish they would spread it out over multiple lines so it 
is readable and understandable, instead of trying to squeeze the maximum 
amount of code per line they can.


> For "with", I'm just wondering "why do I have to indent, it will lose the
> focus of the reader on the result itself".

If this is a problem, then you can easily focus the reader on the 
result by factoring the code into a named function.

text = read('filename')

requires no indentation, no complicated new syntax, and it is easily 
extensible to more complex examples:

text = (prefix + (read('filename') or 'default') + moretext).upper()

This argument isn't about whether it might occasionally be useful to use 
a with expression, but whether it is useful *enough* to justify adding 
new syntax to the language.


-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] With expressions

[Eric Fahlgren]

On Fri, Aug 3, 2018 at 5:26 PM Abe Dillon  wrote:

> One last thing:
>
> Since expressions tend to have pretty limited space, it might be worth it
> to consider more concise options like maybe instead of:
>
>  except (Exception[-list])[ as e]): 
>

You're reinventing PEP 463:

https://www.python.org/dev/peps/pep-0463/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] With expressions

[Abe Dillon]

One last thing:

Since expressions tend to have pretty limited space, it might be worth it
to consider more concise options like maybe instead of:

 except (Exception[-list])[ as e]): 

it could be:

 (Exception[-list])[ as e])! 

So:

y = x[0] IndexError! default

instead of

y = x[0] except IndexError: default

or:

y = spam.ham[0] (AttributeError, IndexError) as e! foo(e)

instead of

y = spam.ham[0] except (AttributeError, IndexError) as e: foo(e)

Or something to that effect. (thought I think it might be best to err on
the side of 'except' because it's more clear what it does and it only adds
7 characters).

On Fri, Aug 3, 2018 at 6:57 PM, Abe Dillon  wrote:

> [Benedikt Werner]
>
>> As the with-expression mimics the with-statement I would say this is
>> similar to:
>> with supress(AttributeError):
>> tmp = person.name[0]
>> initial = tmp # Error on assignment wouldn't get suppressed. Not relevant
>> for this case but still.
>
>
> Ah, yes. That makes much more sense now. I would say the error on
> assignment IS relevant for that case and it's not clear to me how you would
> present that error to the user without causing a lot of confusion.
>
> If it were implemented as:
>
> tmp = None
> with suppress(AttributeError):
> tmp = person.name[0]
> initial = tmp
>
> Then it would solve many of the common use cases for the None-aware
> operators proposed in PEP 505, especially if we made it easy to filter out
> None-specific errors:
>
> class NoneError(BaseException):
> pass
>
> class NoneAttributeError(AttributeError, NoneError):
> pass
>
> ... code to make erroneous attribute access on NoneType objects throw
> NoneAttributeErrors ...
> ... shouldn't break much code other than maybe some doc-string tests ...
>
> initial = person.name[0] with suppress(NoneError)
>
> [Robert Vanden Eynde]
>
>> With implementation details? The highest value is there, the alg is
>> clear., one fetches the filename, one preprocess the lines, then parse the
>> data.
>
>
> Yes. Sorry for using confusing language. I was trying to say that I like
> your proposed syntax (for both with and except) because it follows the same
> principal. At the same time, I was trying to demonstrate part of the value
> of expression-ized statements to everyone else. Since Python is strictly
> evaluated statement by statement, there is no notion of looking ahead and
> re-ordering code. You can't use a variable in one statement then define
> that variable later on (like you do in comprehensions). Expressions, on the
> other hand; are parsed in a more complex way, so you can do things like put
> the relevant logic up front and move all the book-keeping to the end.
>
> [Robert Vanden Eynde]
>
>> About if elif elif else, ternary if does have that:
>> y = (x+1 if x < 0 else
>>  x-1 if x > 0 else
>>  0)
>
>
> True, and your except syntax could chain in a similar manner. In-fact it
> might be possible to make a separate "finally" operator that executes the
> left side then executes the right side no matter what:
>
> y = ((x[0] except InxexError: x.default) except AttributeError: None)
> finally print("hello!")
>
> [Robert Vanden Eynde]
>
>> Limiting the scope is interesting, for "with" the only limitation in that
>> the body must have exactly one assignment, like in the ternary if case?
>> a = ([l.strip() for l in f.readlines()] with open ('name') as f)
>> By cutting the edge cases, "with something():" is not possible, only
>> "with ... as" being possible?
>
>
> I think it's better to forget what I said about limiting the scope of the
> operator. It was a half-baked thought.
>
> As for assignment expressions, they're already on their way thanks to PEP
> 572. I like the "where" version you proposed because it places the logic
> first, but unfortunately PEP 572 (4th alternate spelling
> ) states
> that they had considered that version and rejected it because "where" is a
> pretty common variable name. From the PEP:
>
> SQLAlchemy and numpy have where methods, as does tkinter.dnd.Icon in the
>> standard library
>
>
> So adding a new "where" keyword would break backwards compatibility in a
> major way.
>
> On Fri, Aug 3, 2018 at 5:40 PM, Benedikt Werner <1benediktwer...@gmail.com
> > wrote:
>
>> For instance, what would the following do?
>>
>> initial = person.name[0] with suppress(AttributeError)  # Hangover from
>> PEP 505 discussion...
>>
>> As the with-expression mimics the with-statement I would say this is
>> similar to:
>>
>> with supress(AttributeError):
>> tmp = person.name[0]
>> initial = tmp# Error on assignment wouldn't get 
>> suppressed. Not relevant for this case but still.
>>
>> I don't understand where this is ambigous?
>>
>> So maybe it only makes sense to use expression assignment (PEP 572):
>>
>> data = (d := file.read() with open(...) as file)
>>
>> To which I say, "Eww..."
>>
>> I definitely agree that this looks a bit bad but 

Re: [Python-ideas] With expressions

[Abe Dillon]

[Benedikt Werner]

> As the with-expression mimics the with-statement I would say this is
> similar to:
> with supress(AttributeError):
> tmp = person.name[0]
> initial = tmp # Error on assignment wouldn't get suppressed. Not relevant
> for this case but still.


Ah, yes. That makes much more sense now. I would say the error on
assignment IS relevant for that case and it's not clear to me how you would
present that error to the user without causing a lot of confusion.

If it were implemented as:

tmp = None
with suppress(AttributeError):
tmp = person.name[0]
initial = tmp

Then it would solve many of the common use cases for the None-aware
operators proposed in PEP 505, especially if we made it easy to filter out
None-specific errors:

class NoneError(BaseException):
pass

class NoneAttributeError(AttributeError, NoneError):
pass

... code to make erroneous attribute access on NoneType objects throw
NoneAttributeErrors ...
... shouldn't break much code other than maybe some doc-string tests ...

initial = person.name[0] with suppress(NoneError)

[Robert Vanden Eynde]

> With implementation details? The highest value is there, the alg is
> clear., one fetches the filename, one preprocess the lines, then parse the
> data.


Yes. Sorry for using confusing language. I was trying to say that I like
your proposed syntax (for both with and except) because it follows the same
principal. At the same time, I was trying to demonstrate part of the value
of expression-ized statements to everyone else. Since Python is strictly
evaluated statement by statement, there is no notion of looking ahead and
re-ordering code. You can't use a variable in one statement then define
that variable later on (like you do in comprehensions). Expressions, on the
other hand; are parsed in a more complex way, so you can do things like put
the relevant logic up front and move all the book-keeping to the end.

[Robert Vanden Eynde]

> About if elif elif else, ternary if does have that:
> y = (x+1 if x < 0 else
>  x-1 if x > 0 else
>  0)


True, and your except syntax could chain in a similar manner. In-fact it
might be possible to make a separate "finally" operator that executes the
left side then executes the right side no matter what:

y = ((x[0] except InxexError: x.default) except AttributeError: None)
finally print("hello!")

[Robert Vanden Eynde]

> Limiting the scope is interesting, for "with" the only limitation in that
> the body must have exactly one assignment, like in the ternary if case?
> a = ([l.strip() for l in f.readlines()] with open ('name') as f)
> By cutting the edge cases, "with something():" is not possible, only "with
> ... as" being possible?


I think it's better to forget what I said about limiting the scope of the
operator. It was a half-baked thought.

As for assignment expressions, they're already on their way thanks to PEP
572. I like the "where" version you proposed because it places the logic
first, but unfortunately PEP 572 (4th alternate spelling
) states
that they had considered that version and rejected it because "where" is a
pretty common variable name. From the PEP:

SQLAlchemy and numpy have where methods, as does tkinter.dnd.Icon in the
> standard library


So adding a new "where" keyword would break backwards compatibility in a
major way.

On Fri, Aug 3, 2018 at 5:40 PM, Benedikt Werner <1benediktwer...@gmail.com>
wrote:

> For instance, what would the following do?
>
> initial = person.name[0] with suppress(AttributeError)  # Hangover from
> PEP 505 discussion...
>
> As the with-expression mimics the with-statement I would say this is
> similar to:
>
> with supress(AttributeError):
> tmp = person.name[0]
> initial = tmp # Error on assignment wouldn't get suppressed. 
> Not relevant for this case but still.
>
> I don't understand where this is ambigous?
>
> So maybe it only makes sense to use expression assignment (PEP 572):
>
> data = (d := file.read() with open(...) as file)
>
> To which I say, "Eww..."
>
> I definitely agree that this looks a bit bad but I don't get why you would
> consider using an expression assignment there.
>
> data = file.read with open(...) as file
>
> works perfectly fine so why would you want to additonaly assign it to
> another variable "d"?
>
>
> Overall I like the idea of the with-expression as it allows you to make
> some common use cases like the open/read example more readable. It's clear
> at first sight that what is actually done is reading the file. While it's
> true that it's usually easy to write a simple function to do the job that
> still isn't as simple to understand and in most cases when reading the code
> you then have to look at the function to see if anything else is done
> there. Also what if you then do readlines somewhere? Then you need another
> function.
>
> ___
> Python-ideas mailing list
> Python

Re: [Python-ideas] With expressions

[Benedikt Werner]

For instance, what would the following do?

initial = person.name [0] with 
suppress(AttributeError)  # Hangover from PEP 505 discussion...
As the with-expression mimics the with-statement I would say this is 
similar to:


with supress(AttributeError):
tmp = person.name[0]
initial = tmp   # Error on assignment wouldn't get suppressed. 
Not relevant for this case but still.

I don't understand where this is ambigous?


So maybe it only makes sense to use expression assignment (PEP 572):

data = (d := file.read() with open(...) as file)

To which I say, "Eww..."
I definitely agree that this looks a bit bad but I don't get why you 
would consider using an expression assignment there.


data = file.read with open(...) as file

works perfectly fine so why would you want to additonaly assign it to 
another variable "d"?



Overall I like the idea of the with-expression as it allows you to make 
some common use cases like the open/read example more readable. It's 
clear at first sight that what is actually done is reading the file. 
While it's true that it's usually easy to write a simple function to do 
the job that still isn't as simple to understand and in most cases when 
reading the code you then have to look at the function to see if 
anything else is done there. Also what if you then do readlines 
somewhere? Then you need another function.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] With expressions

[Robert Vanden Eynde]

>
> Expressionization may break the "one and only on obvious way" guideline,
> but it can offer concise, readable code in a lot of instances where a
> statement-based version would be clumsy and noisy, and there's already some
> precedent for it:
>
> function declaration => lambda
> for-loops => generator expressions and comprehensions
> if-else => ternary statements
>

Totally agree. That's the problem when being multi paradigm, but we already
have that "problem" and that's alright.


> With the exception of lambda, expressionized statements usually allow one
> to put the "meat before the vegetables" so to speak. That is; the highest
> value part of the expression comes first and all the book-keeping follows
>

As the code I showed, being just that:

filename = ...
lines = ...
parsed_data = ...

With implementation details? The highest value is there, the alg is clear.,
one fetches the filename, one preprocess the lines, then parse the data.



> One tactic that other expressionizations have taken is to limit the scope.
> For instance, the ternary operation only covers expressionization of
> "if-else" not "just if" or "if-elif-..." or "if-elif-...-else", and
> generator expressions don't allow the 'else' clause
>  of normal
> for-loops. So maybe you can obviate some of the edge cases by requiring an
> as clause or something. I don't know how that would help with the
> suppress(AttributeError) case thought...
>

About if elif elif else, ternary if does have that:

y = (x+1 if x < 0 else
   x-1 if x > 0 else
   0)

Limiting the scope is interesting, for "with" the only limitation in that
the body must have exactly one assignment, like in the ternary if case?

a = ([l.strip() for l in f.readlines()] with open ('name') as f)

By cutting the edge cases, "with something():" is not possible, only "with
... as" being possible?

But the "scope limitation" for try except in expression concept would be
not to have "else" or "finally"? The else and finally clauses do not make
sense in Expression style assignment anyway.

a = (int('stuff') except ValueError: 5)


> On Fri, Aug 3, 2018 at 12:56 PM, Todd  wrote:
>
>> On Thu, Aug 2, 2018 at 5:35 AM, Ken Hilton  wrote:
>>
>>> Hi, I don't know if someone has already suggested this before, but here
>>> goes:
>>>
>>> With expressions allow using the enter/exit semantics of the with
>>> statement inside an expression context. Examples:
>>>
>>> contents = f.read() with open('file') as f #the most obvious one
>>> multiplecontents = [f.read() with open(name) as f for name in names]
>>> #reading multiple files
>>>
>>> I don't know if it's worth making the "as NAME" part of the with
>>> mandatory in an expression - is this a valid use case?
>>>
>>> data = database.selectrows() with threadlock
>>>
>>> Where this would benefit: I think the major use case is `f.read() with
>>> open('file') as f`. Previous documentation has suggested
>>> `open('file').read()` and rely on garbage collection; as the disadvantages
>>> of that became obvious, it transitioned to a method that couldn't be done
>>> in an expression:
>>>
>>> with open('file') as f:
>>> contents = f.read()
>>>
>>> Therefore `f.read() with open('file') as f`, I think, would be much
>>> welcomed as the best way to read a file in an expression.
>>>
>>> For those wondering about the scope semantics of the "as NAME", I think
>>> they would be identical to the scope semantics of the "for" expression -
>>> i.e. these are legal:
>>>
>>> contents = f.read() with open('file') as f
>>> grid = [[i] * 4 for i in range(4)]
>>>
>>> But these are not:
>>>
>>> contents = f.read() with open('file') as f
>>> f.seek(0)
>>> grid = [[i] * 4 for i in range(4)]
>>> grid[i][i] = 4
>>>
>>> Is this a good idea? Are there some subtleties I've failed to explain?
>>> Please let me know.
>>>
>>> Sharing,
>>> Ken Hilton
>>>
>>>
>> If this is a common enough operation for you, it would be trivially easy
>> to just write a function that does this.  There is already a module on pypi
>> that has this function: read_and_close.
>>
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] With expressions

[Abe Dillon]

I like this idea in theory, but I'm not sold yet.

I think there's a lot of draw to the concept of "expressionizing"
statements because many statements require an unnatural ordering in-which
the most important code, the logic, comes after some necessary but
ultimately noisy (from the readers perspective) preamble. So I expect
people to keep asking for expressionized statements and slowly, but,
surely, they'll make their way into the language. They just need to be very
carefully thought out.

Expressionization may break the "one and only on obvious way" guideline,
but it can offer concise, readable code in a lot of instances where a
statement-based version would be clumsy and noisy, and there's already some
precedent for it:

function declaration => lambda
for-loops => generator expressions and comprehensions
if-else => ternary statements

With the exception of lambda, expressionized statements usually allow one
to put the "meat before the vegitables" so to speak. That is; the highest
value part of the expression comes first and all the book-keeping follows. To
illustrate this, I'll write a part of an expression and gradually reveal
the complete expression, you can see how progressively easier it is to
predict the next reveal:

def initials(people):
return {"".join(name[0] ...

# The identifier "name" isn't in scope so it must be assigned in the for
clause of a comprehension.

def initials(people):
return {"".join(name[0] for name in ...

# This is a nested comprehension so it's not much of a surprise that the
iterator might be related to another
# yet-to-be assigned identifier.

def initials(people):
return {"".join(name[0] for name in person.names ...

# Blindly accessing the first element of an empty sequence could cause
problems

def initials(people):
return {"".join(name[0] for name in person.names if name) ...

# The inner generator is closed but we still need a binding for "person"

def initials(people):
return {"".join(name[0] for name in person.names if name) for person in
...

# There's not much left to iterate over and decent variable names point to
one obvious choice

def initials(people):
return {"".join(name[0] for name in person.names if name) for person in
people}


The same could be said for lambdas if they were defined logic-first because
they're usually used in a context where the call signature is obvious:

hand = sorted(cards, key=(card.suit if card is not wild else max_value <==
card))[-5:]

Of course, no such thing exists so the '<==' syntax is made up (in-fact a
possibly better alternative is, "with"), but it doesn't really matter
because a reverse lambda isn't going to happen. You can see, however; that
the function's signature is pretty obvious from context, so it's more for
the computer's sake than the reader's sake and would be better placed out
of the way.

I like the current proposal because it follows that design idea, however; I
haven't taken the time to think about all the edge cases yet.
For instance, what would the following do?

initial = person.name[0] with suppress(AttributeError)  # Hangover from PEP
505 discussion...

Now that I think of it, this seems to inherently make assignment part of
the expression:

data = file.read() with open(...) as file

is supposed to be equivalent to:

with open(...) as file:
data = file.read()

Right?

So maybe it only makes sense to use expression assignment (PEP 572):

data = (d := file.read() with open(...) as file)

To which I say, "Eww..."

Also:

initial = (i := person.name[0] with suppress(AttributeError))

Is still ambiguous (and still eww).

One tactic that other expressionizations have taken is to limit the scope.
For instance, the ternary operation only covers expressionization of
"if-else" not "just if" or "if-elif-..." or "if-elif-...-else", and
generator expressions don't allow the 'else' clause
 of normal for-loops.
So maybe you can obviate some of the edge cases by requiring an as clause
or something. I don't know how that would help with the
suppress(AttributeError) case thought...

On Fri, Aug 3, 2018 at 12:56 PM, Todd  wrote:

> On Thu, Aug 2, 2018 at 5:35 AM, Ken Hilton  wrote:
>
>> Hi, I don't know if someone has already suggested this before, but here
>> goes:
>>
>> With expressions allow using the enter/exit semantics of the with
>> statement inside an expression context. Examples:
>>
>> contents = f.read() with open('file') as f #the most obvious one
>> multiplecontents = [f.read() with open(name) as f for name in names]
>> #reading multiple files
>>
>> I don't know if it's worth making the "as NAME" part of the with
>> mandatory in an expression - is this a valid use case?
>>
>> data = database.selectrows() with threadlock
>>
>> Where this would benefit: I think the major use case is `f.read() with
>> open('file') as f`. Previous documentation has suggested
>> `open('file').read()` and rely on garbage collection; as the 

Re: [Python-ideas] With expressions

[Todd]

On Thu, Aug 2, 2018 at 5:35 AM, Ken Hilton  wrote:

> Hi, I don't know if someone has already suggested this before, but here
> goes:
>
> With expressions allow using the enter/exit semantics of the with
> statement inside an expression context. Examples:
>
> contents = f.read() with open('file') as f #the most obvious one
> multiplecontents = [f.read() with open(name) as f for name in names]
> #reading multiple files
>
> I don't know if it's worth making the "as NAME" part of the with mandatory
> in an expression - is this a valid use case?
>
> data = database.selectrows() with threadlock
>
> Where this would benefit: I think the major use case is `f.read() with
> open('file') as f`. Previous documentation has suggested
> `open('file').read()` and rely on garbage collection; as the disadvantages
> of that became obvious, it transitioned to a method that couldn't be done
> in an expression:
>
> with open('file') as f:
> contents = f.read()
>
> Therefore `f.read() with open('file') as f`, I think, would be much
> welcomed as the best way to read a file in an expression.
>
> For those wondering about the scope semantics of the "as NAME", I think
> they would be identical to the scope semantics of the "for" expression -
> i.e. these are legal:
>
> contents = f.read() with open('file') as f
> grid = [[i] * 4 for i in range(4)]
>
> But these are not:
>
> contents = f.read() with open('file') as f
> f.seek(0)
> grid = [[i] * 4 for i in range(4)]
> grid[i][i] = 4
>
> Is this a good idea? Are there some subtleties I've failed to explain?
> Please let me know.
>
> Sharing,
> Ken Hilton
>
>
If this is a common enough operation for you, it would be trivially easy to
just write a function that does this.  There is already a module on pypi
that has this function: read_and_close.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] With expressions

[Chris Barker via Python-ideas]

On Fri, Aug 3, 2018 at 3:49 AM, Robert Vanden Eynde 
wrote:

>
> When I say "functional programming", I speak about the paradigm used in
> language like Haskell. In language like those, all constructs are
> "expression-based".
>

sure -- but Python is explicitly NOT a functional language in that sense.
It does support some functional paradigms, but features are not added to
conform to the functional paradigm per-se, but because they are considered
useful features.

So one needs to defend a new proposal with arguments about how it makes
python programming better (by SOME definition of better) on its own merits.

-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] With expressions

[Robert Vanden Eynde]

Thanks for answering each line. If someone wants "too long didn't read",
just check my code at the paragraph "readlines is a toy example, but maybe
the code would be more creative".

Le ven. 3 août 2018 à 03:07, Steven D'Aprano  a écrit :

> On Thu, Aug 02, 2018 at 03:13:25PM +0200, Robert Vanden Eynde wrote:
>
> > This brings the discussion of variable assignement in Expression.
> Functional
> > programming community seems to be more interested in python.
>
> I'm not sure what you mean there. Your English grammar is just slightly
> off, enough to make your meaning unclear,

sorry.
>

When I say "functional programming", I speak about the paradigm used in
language like Haskell. In language like those, all constructs are
"expression-based". I consider the code "result = 5 if condition else 2"
more "functional style" than "if condition: result = 5; else: result = 2".
Functional style focus on the result, uses expressions. Imperative focus on
the process, "we must do a condition, then we set the variable, else, we
set a variable to something else".


>
> > lines = (f.readlines() with open('hello') as f)
>
> readlines has the same problems as read, as I described earlier, and the
> same trivial three-line solution.
>
>
> > digit = (int('hello') except ValueError: 5)
>
> try...except exceptions have been proposed before and rejected.
>

I'm wondering why, that must have been the same reasons of not accepting
"with".

if condition:
something = x
else:
something = y

Can be refactored

something = x if condition else y

Or

something = (x if condition else
y)

But,

try:
something = x
except:
something = y

Can't?

The use cases seems similar.

One can use the first form, using more of a imperative programming, or the
second line, which is more "functional programming", more expressions
oriented.


>
> > value = (x+y**2 where x,y = (2,4))
>
> A "where" *statement* is interesting, but this is not a good example of
> it. The above is better written in the normal syntax:
>
> value = 2 + 4**2


That's the discussion we had on the list called "variable assignement in
expressions". What you did here is inlining the variables, technically it's
not possible if you're calling a function and using the variable more than
once.

So we're really comparing it to :

x,y = (2,4)
value = x+y**2

Or

x = 2
y = 4
value = x+y**2

Where the idea is to separate the steps of a computation, introducing
temporary variables with a meaningful name is useful (And as mentioned, if
a function is called and the variable reused, it's called once, but that's
not the main point).

In Haskell there is "value = (x = 2 in y = 4 in x+y**2)" or similar.

position = initial + speed * time
position_inch = distance / 2.54

Vs

position_inch = (initial + speed * time) / 2.54

The programmer chooses what's more clear given the context and the audience.

Or maybe he wants to emphasize that the code creates a position_inch
variable usable in the code after ?

Or both, he wants to explain how he computes position_inch using a
temporary variable but doesn't want the rest of the code depend o
"position" ?).

Yes the del is generally useless, more about  leaking below.


> no need to introduce temporary variables that exist only to obfuscate
> the code.
>
>
> > values = [x+y**2 for x in range(5) for y in range(7)]
> > values = [x+y**2 for x,y in product (range(5), range(7))]
> > y = 5 if condition else 2
>
> These already exist, because they are useful.
>

I see those as a refactoring of imperative programming style as well
(values = []; for ...: values.append(...))

>
> > y = (lambda x: x+2)(x=5)
>
> This is not a good example of the use of a lambda. Better:
>
> y = 5 + 2
>

Same thing as in the where syntax. However, some constructs are easier to
refactor as

def meaningfulname(x):
 return x+2

y = meaningfulname(5)


> Why bother writing a function with such a simple body if you are going
> to immediately call it on the spot? Unless the body is more complex, or
> you are going to call it elsewhere, or call it repeatedly, the lambda
> adds nothing.
>

Indeed, in complex expressions.

Or if I want to separate the steps of a computation.

position = initial + speed * time
position_inch = distance / 2.54


> Nobody denies that *some* statements are well-suited and useful as
> expressions. The question is whether "with" is one of those.
>

I'm just pointing out those constructs are very similar, it kind of makes
sense to compare them.

Of course I don't know about real world examples that would simplify a
code. But often as I'm a "expression first" guy

I write:

result = ...
# code using result and that doesn't care about the temporary variable it
uses.

Then I figure how to compute result, without polluting the namespace. Again
adding temporary variables before "result = ..." Is totally fine and that's
the way to go in imperative programming.


>
> > with open('hello') as f:
> > lines = f.readlines()
> > de

Re: [Python-ideas] With expressions

[Steven D'Aprano]

On Thu, Aug 02, 2018 at 03:13:25PM +0200, Robert Vanden Eynde wrote:

> This brings the discussion of variable assignement in Expression. Functional
> programming community seems to be more interested in python.

I'm not sure what you mean there. Your English grammar is just slightly 
off, enough to make your meaning unclear, sorry.


> lines = (f.readlines() with open('hello') as f)

readlines has the same problems as read, as I described earlier, and the 
same trivial three-line solution.


> digit = (int('hello') except ValueError: 5)

try...except exceptions have been proposed before and rejected.


> value = (x+y**2 where x,y = (2,4))

A "where" *statement* is interesting, but this is not a good example of 
it. The above is better written in the normal syntax:

value = 2 + 4**2

no need to introduce temporary variables that exist only to obfuscate 
the code.


> values = [x+y**2 for x in range(5) for y in range(7)]
> values = [x+y**2 for x,y in product (range(5), range(7))]
> y = 5 if condition else 2

These already exist, because they are useful.


> y = (lambda x: x+2)(x=5)

This is not a good example of the use of a lambda. Better:

y = 5 + 2

Why bother writing a function with such a simple body if you are going 
to immediately call it on the spot? Unless the body is more complex, or 
you are going to call it elsewhere, or call it repeatedly, the lambda 
adds nothing.

Nobody denies that *some* statements are well-suited and useful as 
expressions. The question is whether "with" is one of those.


> with open('hello') as f:
> lines = f.readlines()
> del f  # f is leaked !

99% of the time, I would think that "del f" was a waste of time. If that 
code is in function, then f will be closed when the "with" block is 
exited, and garbage collected when the function returns.

If you are worried about the memory efficiency of one tiny closed file 
object, then Python is the wrong language for you.

If that code is in the top-level of a script, who cares about f? You 
surely don't delete all your variables when you are done with them:

name = input("what is your name?")
print("Hello,", name)
del name
play_game()


The only time I would explicitly delete f like you do above was if I was 
writing a library which specifically and explicitly supported the "from 
module import *" syntax, AND there were too many exportable names to 
list them all in __all__ by hand. Only in that very rare case would I 
care about tidying up the global namespace by using del.


> x,y = 2,4
> value = x+y**2
> del x, y  # x,y are leaked !

If you are writing code like this, you are just obscuring the code. Much 
better to just use the values where you need them, not to invent 
unnecessary temporary variables that you don't need!

value = 2 + 4**2


[...]
> If we add one, it's Logical to add the others to be consistent.

My car has round wheels. Since we use one shape (circle) for wheels, it 
must be "logical" to make wheels in all other shapes, to be consistent:

- triangular wheels
- square wheels

etc. Consistency just for the sake of consistency is *not* a virtue. 
Unless those expression forms justify *themselves* on their own merits, 
it is just a waste of time and effort to let them sneak into the 
language "for consistency".

 
> Of course, one can always write functions like read_text but the ide of
> those construction is like the lambda, we want anonymous.

We can say:

text = read('filename')

text = f.read() with open('filename') as f

and both are equally unanonymous (both use a named variable), or we can 
say:

process(spam, eggs, read('filename'), foo, bar)

process(spam, eggs, f.read with open('filename') as f, foo, bar)

and both are equally anonymous.

If Python had a built-in function "read", surely you wouldn't refuse to 
use it because it was a named function? We don't complain that map() and 
filter() are named functions and demand "anonymous" ways to do the same 
thing. A read function should be no different.

The only point of difference is that it is not built-in, you have to 
write it yourself. But not every trivial three-line function must be 
built-in.



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] With expressions

[Terry Reedy]

On 8/2/2018 7:53 AM, Thomas Nyberg via Python-ideas wrote:

On 08/02/2018 12:43 PM, Paul Moore wrote:

But if someone wanted to raise a doc bug suggesting that we mention
this, I'm not going to bother objecting...
Paul



I opened a bug here:

 https://bugs.python.org/issue34319

We can see what others think.


I suggested on the issue that we add "The file is opened and then 
closed." before "The optional parameters have the same meaning as in 
open()."


Another option would be to add "The file is closed after reading." after 
the latter sentence.



--
Terry Jan Reedy


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] With expressions

[Robert Vanden Eynde]

This brings the discussion of variable assignement in Expression. Functional
programming community seems to be more interested in python.

lines = (f.readlines() with open('hello') as f)
digit = (int('hello') except ValueError: 5)
value = (x+y**2 where x,y = (2,4))
values = [x+y**2 for x in range(5) for y in range(7)]
values = [x+y**2 for x,y in product (range(5), range(7))]
y = 5 if condition else 2
y = (lambda x: x+2)(x=5)

vs

with open('hello') as f:
lines = f.readlines()
del f  # f is leaked !

x,y = 2,4
value = x+y**2
del x, y  # x,y are leaked !

try:
digit = (int('hello')
except ValueError:
digit = 5

if condition:
y = 5
else:
y = 2

def f(x):
return x+2
y = f(x=2)
del f  # we want an anonymous function !

Those "oneliners" is not only the will to be quicker in interactive mode,
it's the way functional programming Thinks.

If we add one, it's Logical to add the others to be consistent.

Of course, one can always write functions like read_text but the ide of
those construction is like the lambda, we want anonymous.

Le jeu. 2 août 2018 à 13:56, Steven D'Aprano  a écrit :

> On Thu, Aug 02, 2018 at 11:35:11AM +0200, Ken Hilton wrote:
>
> > Where this would benefit: I think the major use case is `f.read() with
> > open('file') as f`.
> [...]
> > Therefore `f.read() with open('file') as f`, I think, would be much
> > welcomed as the best way to read a file in an expression.
>
> Perhaps so, but do we want to encourage that to the point of adding
> syntax to make it easier?
>
> f.read() is a (mild) code-smell. Unless your file is guaranteed to be
> smaller than the amount of free memory, it risks starting your OS
> thrashing. IMO that makes this an idiom only suitable for quick and
> dirty scripts where the the user knows the limitations of the script and
> can abide by them.
>
> (In these days of computers with multiple gigabytes of RAM, reading in
> an entire file is not as risky as it used to be. But on the other hand,
> in these days of terrabyte and even petabyte storage devices, there are
> more *really large* files too.)
>
> Don't get me wrong -- f.read() is not necessarily bad. I often write
> scripts that slurp in an entire file at once, but they're typically
> throw-away scripts, and I'm also the user of the script and I know not
> to call it on files above a certain size. (As Miss Piggy once said,
> "Never eat more in one sitting than you can lift.")
>
> But I'm not sure if this sort of thing is something we want to
> *encourage* rather than merely *allow*. Best practice for reading files
> is, after all, a with statement for a reason: we expect to read text
> files line by line, often wrapped in a try...except to handle
> exceptions.
>
> For your use-case, I suspect the best thing is a utility function:
>
> def read(name, size=-1, **kwargs):
> with open(name, **kwargs) as f:
> return f.read(size)
>
> Not every three-line function needs to be a built-in, let alone
> given syntax :-)
>
>
> > For those wondering about the scope semantics of the "as NAME", I think
> > they would be identical to the scope semantics of the "for" expression
>
> Its not really a "for expression" -- its a *comprehension*, which is
> much more than merely a for expression:
>
> # this isn't legal
> result = for x in seq
>
> One important difference is that unlike this proposed "with" expression,
> comprehensions have an obvious pair of delimiters which enclose the
> expression and give it a natural beginning and end. There's no need to
> memorise arcane operator precedences and parsing rules to work out where
> the "with...as" variable will be legal.
>
> Another important difference is that while there are good reasons for
> putting comprehension loop variables in their own sub-local scope, I
> don't see any such benefit to doing the same for this proposed with
> expression. I don't think we should encourage the proliferation of more
> and more layers of extra scopes. We already have six:
>
> sublocal (comprehensions)
> local
> nonlocal (enclosing functions)
> class
> global (module)
> builtins
>
>
> Let's be cautious about adding more varieties of sublocal scope.
>
>
> --
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] With expressions

[Cody Piersall]

On Thu, Aug 2, 2018 at 5:24 AM Thomas Nyberg via Python-ideas
 wrote:
>
> Is it true that Path('file').read_text() closes the file after the read?

A quick look at the source confirms that the file is closed:
https://github.com/python/cpython/blob/master/Lib/pathlib.py#L1174

The docstring is better than the official documentation, in my opinion.

Cody
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] With expressions

[Steven D'Aprano]

On Thu, Aug 02, 2018 at 11:35:11AM +0200, Ken Hilton wrote:

> Where this would benefit: I think the major use case is `f.read() with
> open('file') as f`.
[...]
> Therefore `f.read() with open('file') as f`, I think, would be much
> welcomed as the best way to read a file in an expression.

Perhaps so, but do we want to encourage that to the point of adding 
syntax to make it easier?

f.read() is a (mild) code-smell. Unless your file is guaranteed to be 
smaller than the amount of free memory, it risks starting your OS 
thrashing. IMO that makes this an idiom only suitable for quick and 
dirty scripts where the the user knows the limitations of the script and 
can abide by them.

(In these days of computers with multiple gigabytes of RAM, reading in 
an entire file is not as risky as it used to be. But on the other hand, 
in these days of terrabyte and even petabyte storage devices, there are 
more *really large* files too.)

Don't get me wrong -- f.read() is not necessarily bad. I often write 
scripts that slurp in an entire file at once, but they're typically 
throw-away scripts, and I'm also the user of the script and I know not 
to call it on files above a certain size. (As Miss Piggy once said, 
"Never eat more in one sitting than you can lift.")

But I'm not sure if this sort of thing is something we want to 
*encourage* rather than merely *allow*. Best practice for reading files 
is, after all, a with statement for a reason: we expect to read text 
files line by line, often wrapped in a try...except to handle 
exceptions.

For your use-case, I suspect the best thing is a utility function:

def read(name, size=-1, **kwargs):
with open(name, **kwargs) as f:
return f.read(size)

Not every three-line function needs to be a built-in, let alone 
given syntax :-)


> For those wondering about the scope semantics of the "as NAME", I think
> they would be identical to the scope semantics of the "for" expression 

Its not really a "for expression" -- its a *comprehension*, which is 
much more than merely a for expression:

# this isn't legal
result = for x in seq

One important difference is that unlike this proposed "with" expression, 
comprehensions have an obvious pair of delimiters which enclose the 
expression and give it a natural beginning and end. There's no need to 
memorise arcane operator precedences and parsing rules to work out where 
the "with...as" variable will be legal.

Another important difference is that while there are good reasons for 
putting comprehension loop variables in their own sub-local scope, I 
don't see any such benefit to doing the same for this proposed with 
expression. I don't think we should encourage the proliferation of more 
and more layers of extra scopes. We already have six:

sublocal (comprehensions)
local
nonlocal (enclosing functions)
class
global (module)
builtins


Let's be cautious about adding more varieties of sublocal scope.


-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] With expressions

[Thomas Nyberg via Python-ideas]

On 08/02/2018 12:43 PM, Paul Moore wrote:

But if someone wanted to raise a doc bug suggesting that we mention
this, I'm not going to bother objecting...
Paul



I opened a bug here:

https://bugs.python.org/issue34319

We can see what others think.

Cheers,
Thomas
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] With expressions

[Thomas Nyberg via Python-ideas]

On 08/02/2018 12:43 PM, Paul Moore wrote:

I'm not sure I see why you think it wouldn't - opening and closing the
file is a purely internal detail of the function. In any case, you
don't get given a file object, so how could anything *other* than the
read_text() close the file? So you're basically asking "does
Path.read_text() have a bug that causes it to leak a filehandle?" to
which my answer would be "I assume not, until someone demonstrates
such a bug".


To me the following look the same:

Path('file').read_text()
open('file').read()

The first presumably creates a Path object while the second creates a 
file object. Why should I assume that the Path object closes the 
underlying file object after the method is called? I mean maybe my 
assumption is bad, but I doubt I'd be the only one making it given how 
open() works and that it looks similar superficially.


Cheers,
Thomas
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] With expressions

[Paul Moore]

On Thu, 2 Aug 2018 at 11:25, Thomas Nyberg via Python-ideas
 wrote:
>
> Is it true that Path('file').read_text() closes the file after the read?
> I think that is the sort of functionality that Ken is asking for.
> It's not clear to me by your linked documentation that it does. If it
> does, maybe that should be made more clear in that linked documentation?
> (Of course, maybe it's written there somewhere and I'm just blind...)

I'm not sure I see why you think it wouldn't - opening and closing the
file is a purely internal detail of the function. In any case, you
don't get given a file object, so how could anything *other* than the
read_text() close the file? So you're basically asking "does
Path.read_text() have a bug that causes it to leak a filehandle?" to
which my answer would be "I assume not, until someone demonstrates
such a bug".

But if someone wanted to raise a doc bug suggesting that we mention
this, I'm not going to bother objecting...
Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] With expressions

[Andre Roberge]

On Thu, Aug 2, 2018 at 7:24 AM Thomas Nyberg via Python-ideas <
python-ideas@python.org> wrote:

> Is it true that Path('file').read_text() cl
> oses the file after the read?
> I think that is the sort of functionality that Ken is asking for.
> It's not clear to me by your linked documentation that it does. If it
> does, maybe that should be made more clear in that linked documentation?
>

Agreed. Then again, the documentation includes a link to the source at the
top and we find (
https://github.com/python/cpython/blob/3.7/Lib/pathlib.py#L1174)

docstring:  Open the file in text mode, read it, and close the file.

Or ...

>>> import pathlib >>> help(pathlib.Path.read_text) Help on function
read_text in module pathlib: read_text(self, encoding=None, errors=None)
Open the file in text mode, read it, and close the file.


The documentation would be improved if it used the text from the docstring
instead of its own one-line description.

André Roberge



> (Of course, maybe it's written there somewhere and I'm just blind...)
>
> Cheers,
> Thomas
>
> On 08/02/2018 11:53 AM, Paul Moore wrote:
> > On Thu, 2 Aug 2018 at 10:39, Ken Hilton  wrote:
> >
> >> With expressions allow using the enter/exit semantics of the with
> statement inside an expression context. Examples:
> >>
> >>  contents = f.read() with open('file') as f #the most obvious one
> >>  multiplecontents = [f.read() with open(name) as f for name in
> names] #reading multiple files
> >>
> >> I don't know if it's worth making the "as NAME" part of the with
> mandatory in an expression - is this a valid use case?
> >>
> >>  data = database.selectrows() with threadlock
> >>
> >> Where this would benefit: I think the major use case is `f.read() with
> open('file') as f`. Previous documentation has suggested
> `open('file').read()` and rely on garbage collection; as the disadvantages
> of that became obvious, it transitioned to a method that couldn't be done
> in an expression:
> >
> > That use case is satisfied by pathlib:
> >
> > Path('file').read_text()
> >
> > see
> https://docs.python.org/3.7/library/pathlib.html#pathlib.Path.read_text
> >
> > Are there any other use cases? I don't see any real advantage here
> > other than the non-advantage of being able to write one-liners.
> > Paul
> > ___
> > Python-ideas mailing list
> > Python-ideas@python.org
> > https://mail.python.org/mailman/listinfo/python-ideas
> > Code of Conduct: http://python.org/psf/codeofconduct/
> >
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] With expressions

[Thomas Nyberg via Python-ideas]

Is it true that Path('file').read_text() closes the file after the read? 
I think that is the sort of functionality that Ken is asking for.
It's not clear to me by your linked documentation that it does. If it 
does, maybe that should be made more clear in that linked documentation? 
(Of course, maybe it's written there somewhere and I'm just blind...)


Cheers,
Thomas

On 08/02/2018 11:53 AM, Paul Moore wrote:

On Thu, 2 Aug 2018 at 10:39, Ken Hilton  wrote:


With expressions allow using the enter/exit semantics of the with statement 
inside an expression context. Examples:

 contents = f.read() with open('file') as f #the most obvious one
 multiplecontents = [f.read() with open(name) as f for name in names] 
#reading multiple files

I don't know if it's worth making the "as NAME" part of the with mandatory in 
an expression - is this a valid use case?

 data = database.selectrows() with threadlock

Where this would benefit: I think the major use case is `f.read() with 
open('file') as f`. Previous documentation has suggested `open('file').read()` 
and rely on garbage collection; as the disadvantages of that became obvious, it 
transitioned to a method that couldn't be done in an expression:


That use case is satisfied by pathlib:

Path('file').read_text()

see https://docs.python.org/3.7/library/pathlib.html#pathlib.Path.read_text

Are there any other use cases? I don't see any real advantage here
other than the non-advantage of being able to write one-liners.
Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] With expressions

[Paul Moore]

On Thu, 2 Aug 2018 at 10:39, Ken Hilton  wrote:

> With expressions allow using the enter/exit semantics of the with statement 
> inside an expression context. Examples:
>
> contents = f.read() with open('file') as f #the most obvious one
> multiplecontents = [f.read() with open(name) as f for name in names] 
> #reading multiple files
>
> I don't know if it's worth making the "as NAME" part of the with mandatory in 
> an expression - is this a valid use case?
>
> data = database.selectrows() with threadlock
>
> Where this would benefit: I think the major use case is `f.read() with 
> open('file') as f`. Previous documentation has suggested 
> `open('file').read()` and rely on garbage collection; as the disadvantages of 
> that became obvious, it transitioned to a method that couldn't be done in an 
> expression:

That use case is satisfied by pathlib:

Path('file').read_text()

see https://docs.python.org/3.7/library/pathlib.html#pathlib.Path.read_text

Are there any other use cases? I don't see any real advantage here
other than the non-advantage of being able to write one-liners.
Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/