[Python-ideas] Re: A shortcut to load a JSON file into a dict : json.loadf
This reminds me of a previous proposal I can't remember if I hit the list with,
allowing
with open(filename.json, "r") as f:
my_dict = json.load(f)
to be spelt as a single expression:
my_dict = (json.load(f) with open(filename.json, "r") as f)
Obviously this would be more useful in comprehensions, but it might encourage
people not to lazily write `json.load(open("filename.json", "r"))` because they
want an expression, and end up leaving files open.
Eric
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/QR233SIBDHWDZLKWURUHXG37BDWJHA7Z/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Adding mixed positional and keyword to PEP 637
I can offer another example where the new syntax is likely to be of use to numpy. We currently have the following as synonyms: ``` np.copyto(a, b, casting='unsafe') a[...] = b ``` Obviously, it would be nice if we could allow other casting rules for `[]` assignment: ``` np.copyto(a, b, casting='safe') a[..., casting='safe'] = b ``` Eric ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/KO2EM52NWS2SGW45WROO45JG43XHZW6S/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PEP 637 and keyword only subscripts
Would another option be to just stop using the tuple-less index in the presence
of new syntax - so by example,
```
SYNTAXINDEX KWARGS
d[*[]]() {}
d[*[],] () {}
d[**{}] () {}
d[**{},] () {}
d[foo=1] () {'foo': 1}
d[foo=1,] () {'foo': 1}
d[x] x {} (valid existing syntax)
d[x,] (x,) {} (valid existing syntax)
d[x, *[]] (x,) {}
d[x, *[],](x,) {}
d[x, **{}](x,) {}
d[x, **{},] (x,) {}
d[x, foo=1] (x,) {'foo': 1}
d[x, foo=1,] (x,) {'foo': 1}
```
Essentially, the rules are:
* Does the indexing contain any of the new syntax proposed in this PEP (`*`,
`**`, or explicit kwarg)? If yes:
* Always pass a tuple as the index. This is new-style indexing, lets leave
behind the weird corner cases. `()` is a natural choice of singleton.
* If no:
* Use the existing rule of parsing the contents of `[]` as a tuple
The one downside of this approach is that perfect forwarding of `__getitem__`
requires a small dance
```
def __getitem__(self, args, **kwargs):
if not type(args) is tuple and not kwargs:
# this is pre-PEP-637 indexing
return self._wrapped[args]
else:
# the presence of `*` or `**` is enough to force PEP-637 indexing, even
# if those are empty
return self._wrapped[*args, **kwargs]
```
This would only come up in new code though - any forwarding `__getitem__`
without `**kwargs` would already be written correctly.
I'd argue it would be sufficient to note this a usage note in the PEP.
Eric
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/ZGPKU3EGXH72OGNGSBJO26ITHOREBGJ4/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PEP 9999 (provisional): support for indexing with keyword arguments
Thanks for the PEP, and for pinging the numpy list about it. Some comments: Sequence unpacking remains a syntax error inside subscripts: Reason: unpacking items would result it being immediately repacked into a tuple A simple counter-example is [:, *args, :], which could be treated as [(slice(None), *args, slice(None))]. When there are index objects to the left or right of *args, it enables : syntax in places that would otherwise be forbidden. I’d argue this should be folded into the scope of the PEP - if the parser for indexing is going to be overhauled, it would be nice to cross this one off. I think I requested this change independently at some point Keyword arguments must allow Ellipsis This is sort of uninteresting now, Ellipsis is not special-cased in the parser, all places where Ellipsis is allowed also allow ... - even __class__! a similar case occurs with setter notation It would perhaps be a good idea to recommend users declare their setter arguments as positional-only too,def __setitem__(self, index, value, /, **kwargs):, to avoid this problem Keyword-only subscripts are permitted. The positional index will be the empty tuple As discussed on the numpy mailing list, a better approach might be to not pass an argument at all, so that obj[spam=1, eggs=2] calls type(obj).__getitem__(obj, spam=1, eggs=2). Objects which want to support keyword-only indexing can provide a default of their own (and are free to choose between () and None, while objects which do not would just raise TypeError due to the missing positional argument the solution relies on the assumption that all keyword indices necessarily map into positional indices, or that they must have a name This seems like a non-issue to me. Keyword-only arguments already exist, and could be used in the relevant magic method def __getitem_ex__(self, *, x, y):. I would be inclined to remove this argument. Eric ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/ZQ4CHT37YUHYEFYFBQ423QNZMFCRULAW/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Make `del x` an expression evaluating to `x`
Yes, it would mean that, which is particularly nasty considering the fact that `_` in python and `Out` in IPython would then acquire the very reference that was being deleted. I think that my proposed change would be unacceptable without addressing this problem. To resolve this, `del` could be made both a statement and an expression. The parsing rules would be very similar to how the walrus operator is handled today, but instead of being a SyntaxError in statement mode, it would simply produce a different ast node: * `del x` -> `Delete(x)` (as `x := 0` -> SyntaxError) * `(del x)` -> `Expr(DeleteExpr(x))` (as `(x := 0)` -> `Expr(NamedExpr(...))`) Eric On Fri, 12 Jun 2020 at 17:21, Guido van Rossum wrote: > On Fri, Jun 12, 2020 at 4:55 AM Eric Wieser > wrote: > >> > He thought that the change of del he proposed will give him that >> behavior, but this is not true. >> >> Unless I'm forgetting part of the conversation, that's not true. Note >> that the numpy patch is merged. Today, you get the optimization with `z = a >> + b + c + d`. What you don't get is the same optimization if you use: >> ``` >> ab = a + b >> abc = ab + c >> z = abc + d >> ``` >> The language feature I propose is to allow you to _keep_ the optimization >> that was present in `z = a + b + c + d`, but write it as >> ``` >> ab = a + b >> abc = (del ab) + c >> z = (del abc) + d >> ``` >> > > But does that mean that typing `del x` at the REPL will print the value of > `x`? That seems wrong. If `del x` is an expression that returns the value > of `x` (and then unbinds it), then the REPL would seem to have no choice > but to print the returned value -- the REPL has no indication that it came > from `del`. > > -- > --Guido van Rossum (python.org/~guido) > *Pronouns: he/him **(why is my pronoun here?)* > <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/> > ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/ROHRO4GCH2TU7YXKGVTIFWNLLCEIYFNC/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Make `del x` an expression evaluating to `x`
> He thought that the change of del he proposed will give him that behavior, > but this is not true. Unless I'm forgetting part of the conversation, that's not true. Note that the numpy patch is merged. Today, you get the optimization with `z = a + b + c + d`. What you don't get is the same optimization if you use: ``` ab = a + b abc = ab + c z = abc + d ``` The language feature I propose is to allow you to _keep_ the optimization that was present in `z = a + b + c + d`, but write it as ``` ab = a + b abc = (del ab) + c z = (del abc) + d ``` ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/NJX7GPBY4E3EMNKGKL3W3Q5E37BWFBVX/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Make `del x` an expression evaluating to `x`
I've realized that I've actually seen code use a trick to enable exactly this optimization in the past, without needing language extensions (although I don't remember where). Taking two of my motivating examples from elsewhere in the thread: bc = b*c a = bc + d f = get_f_long_name(x) g = get_g_long_name(del f) h = get_h_long_name(del g) This can be written today with exactly the intended semantics by using lists and pop: bc = [b * c] a = bc.pop() f = [get_f_long_name(x)] g = [get_g_long_name(f.pop())] h = get_h_long_name(g.pop()) I'd argue that this is a little less readable, but it makes the incremental improvement provided by a language change less significant, which makes the change harder to justify. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/W2TGLWTHAQ34MGXBWB5STQR5FDS47GYV/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Make `del x` an expression evaluating to `x`
> And as I understand it (from a quick scan) the reason it can’t tell isn’t > that the refcount isn’t 1 (which is something CPython could maybe fix if it > were a problem, but it isn’t). Rather, it is already 1, but a refcount of 1 > doesn’t actually prove a temporary value This is at odds with my undestanding, which is that the reason it can’t tell _is_ that the refcount isn’t 1. When you do `(b * c) + d`, `(b * c).__add__(d)` is passed `self` with a refcount of 1 (on the stack). I believe this hits the optimization today in numpy. When you do `bc + d`, `bc.__add__(d)` is passed `self` with a refcount of 2 (one on the stack, and one in `locals()`). My proposal of del is that `(del bc) + d` would enter `bc.__add__(d)` with `self` passed with a refcount of 1. > Even if it could, surely the solution to “`a+b+c+d` is slow” can’t be “just > write `(del (del a+b)+c)+d` and it’ll be fast” That would be a syntax error under my proposal. `del` remains an operator only on names, not expressions. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/XZD4EIPTPRNKM6VFIF3GZTLOXH5M6AQ5/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Make `del x` an expression evaluating to `x`
This was exactly what I was thinking of when I said > This is unavoidable in a for loop without breaking existing code, but I think > could (and should?) be changed in a list comprehension Unfortunately, I was wrong in the last half of this statement. > since the comprehension control variable lives in an inner scope The inner scope can still be closed over via a lambda, such as in the following. I think that makes changing the semantics of either a breaking change. >>> funcs = [lambda: i for i in range(2)] >>> [f() for f in funcs()] [1, 1] # often not intended, but demonstrates the last-value-assigned semantics Eric ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/KPDOSMOB4YB5KUIM6WCCMGQIYGY3PGGO/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Make `del x` an expression evaluating to `x`
It looks like actually this can be be built as a function today:
def move(name):
return inspect.currentframe().f_back.f_locals.pop(name)
Which works as follows, but it feels awkward to pass variable names by strings
(and will confuse linters):
>>> for v in itertools.combinations([1, 2, 3], 1):
...print(id(move("v")))
1718903397008
1718903397008
1718903397008
Eric
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/HLQWDHLCEQCI75SBCCVNPUBLFT7R45QJ/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Make `del x` an expression evaluating to `x`
> You can get the same effect by not naming the value being thrown away This is absolutely true, although I don't think that's a particularly strong argument against it. The same can be said of `std::move` in C++. The purpose of this suggestion is primarily to allow introducing names without it coming at the cost of unwanted reference counts. Consider the following expression, with no intermediate names: h = get_h_long_name(get_g_long_name(get_f_long_name(x)) With my proposal, this would become exactly equivalent to the following in terms of reference counts f = get_f_long_name(del x) g = get_g_long_name(del f) h = get_h_long_name(del g) > Do you have an example of an application where this sort of > micro-optimisation is significant enough to justify a fairly major language > change? As an example of these optimizations being valuable, see https://github.com/numpy/numpy/pull/7997, which claims the optimization I described at the beginning resulted in a 1.5x-2x speedup. > but that didn't need a language change to implement... The language change I'm proposing isn't about _implementing_ these optimizations, it's about allowing users to exploit them without having to sacrifice naming their variables. Eric ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/ZR5LQTYVYQTC5C64WJDAETFFSVTL7D62/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Make `del x` an expression evaluating to `x`
Marco Sulla wrote: > I can be wrong, but for what I know, del variable_name does not > optimize the code, in the sense of performance. Correct, by itself `del variable_name` does not optimize the code - however, there exist functions implemented in C (which I gave examples of) with special cases for when `sys.getrefcount(x) == 1`, which means they are free to repurpose an existing object. > allowing the gc to free the memory if that variable was the last reference to > the object. The gc may not be relevant here - in CPython, objects are cleaned up immediately as part of reference counting. The GC only kicks in for circular object graphs. > In a for and a list comprehension, I suppose it's unneeded, since the > variable points automatically to another memory location, so the > reference is removed automatically. The nth reference is not removed until the n+1th reference has been created. This is unavoidable in a for loop without breaking existing code, but I think could (and should?) be changed in a list comprehension > Furthermore, if you want to delete a temporary variable, you could do: > > bc = b * c > a = bc + d > del bc This doesn't hit the optimization I was talking about, because `ndarray.__add__(bc, d)` contains `if sys.getrefcount(self) == 1`, but here the local variable results in a second refcount. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/GPQPNUJLRKD32734WQMX5ZIHEZVTOLH2/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Make `del x` an expression evaluating to `x`
TL;DR: should we make `del x` an expression that returns the value of `x`. ## Motivation I noticed yesterday that `itertools.combinations` has an optimization for when the returned tuple has no remaining ref-counts, and reuses it - namely, the following code: >>> for v in itertools.combinations([1, 2, 3], 1): ...print(id(v)) ...del v # without this, the optimization can't take place 2500926199840 2500926199840 2500926199840 will print the same id three times. However, when used as a list comprehension, the optimization can't step in, and I have no way of using the `del` keyword >>> [id(v) for v in itertools.combinations([1, 2, 3], 1)] [2500926200992, 2500926199072, 2500926200992] `itertools.combinations` is not the only place to make this optimization - parts of numpy use it too, allowing a = (b * c) + d to elide the temporary `b*c`. This elision can't happen with the spelling bc = b * c a = bc + d My suggestion would be to make `del x` an expression, with semantics "unbind the name `x`, and evaluate to its value". This would allow: >>> [id(del v) for v in itertools.combinations([1, 2, 3], 1)] [2500926200992, 2500926200992, 2500926200992] and bc = b * c a = (del bc) + d # in C++, this would be `std::move(bc) + d` ## Why a keyword Unbinding a name is not something that a user would expect a function to do. Functions operate on values, not names, so `move(bc)` would be surprising. ## Why `del` `del` already has "unbind this name" semantics, and allow it to be used as an expression would break no existing code. `move x` might be more understandable, but adding new keywords is expensive ## Optional extension For consistency, `x = (del foo.attr)` and `x = (del foo[i])` could also become legal expressions, and `__delete__`, `__delattr__`, and `__delitem__` would now have return values. Existing types would be free to continue to return `None`. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/WVQNATE7KYU5G64BQB5VEWALPYVS3QPV/ Code of Conduct: http://python.org/psf/codeofconduct/
