really do want is functional purity, which is a different
>>>>> concept and one that python as a language can't easily provide no matter
>>>>> what.
>>>>>
>>>>> --Josh
>>>>>
>>>>> On Mon, Feb 20, 2017 at 7:53 PM Abe Dillon
>
;>>> My thought was that if a compiler is capable of determining what needs
>>>>> to be compiled just in time, then an interpreter might be able to
>>>>> determine
>>>>> what expressions need to be evaluated just when their results are actually
>&
t expressions need to be evaluated just when their results are actually
>>>> used.
>>>>
>>>> So if you had code that looked like:
>>>>
>>>> >>> log.debug("data: %s", expensive())
>>>>
>>>> The interp
y require just in time compilation as well, otherwise the
>>> byte code that calls the "log.debug" function would be unaware of the byte
>>> code that implements the function.
>>>
>>> This is probably a pipe-dream, though; because the interpreter would
>
kip evaluating the expensive function if the
>>> result is never used. It would only evaluate it "just in time". This would
>>> almost certainly require just in time compilation as well, otherwise the
>>> byte code that calls the "log.debug" function would be un
robably a pipe-dream, though; because the interpreter would have
>> to be aware of side effects.
>>
>>
>>
>> On Mon, Feb 20, 2017 at 5:18 AM, wrote:
>>
>>
>>
>> > -Original Message-
>> > From: Python-ideas [mailto:p
gt; to be aware of side effects.
>
>
>
> On Mon, Feb 20, 2017 at 5:18 AM, wrote:
>
>
>
> > -Original Message-
> > From: Python-ideas [mailto:python-ideas-bounces+tritium-
> > list=sdamon@python.org] On Behalf Of Michel Desmoulin
> > Sent: Monday, Februa
t; list=sdamon@python.org] On Behalf Of Michel Desmoulin
> > Sent: Monday, February 20, 2017 3:30 AM
> > To: python-ideas@python.org
> > Subject: Re: [Python-ideas] Delayed Execution via Keyword
> >
> > I wrote a blog post about this, and someone asked me if it meant
> &g
> -Original Message-
> From: Python-ideas [mailto:python-ideas-bounces+tritium-
> list=sdamon@python.org] On Behalf Of Michel Desmoulin
> Sent: Monday, February 20, 2017 3:30 AM
> To: python-ideas@python.org
> Subject: Re: [Python-ideas] Delayed Execution via Keyw
I wrote a blog post about this, and someone asked me if it meant
allowing lazy imports to make optional imports easier.
Someting like:
lazy import foo
lazy from foo import bar
So now if I don't use the imports, the module is not loaded, which could
also significantly speed up applications starti
On 2/19/17, Michel Desmoulin wrote:
> Evnetually we also may need to allow this:
>
> a = lazy stuff
> if a is not lazy:
> print(a)
>
> But then lazy can't be used a var name to help with the transition.
What about this?
if not inspect.islazy(a):
print(a)
Next idea is probably obvious:
On 2/19/17, David Mertz wrote:
> On Sun, Feb 19, 2017 at 10:13 AM, Joseph Hackman
> wrote:
>>
>> My honest preference would be that the [] is evaluated fresh each time
>> the
>> function is called.
>> def stuff(arg=delayed f()):
>> would result in f() being called every time stuff() is. This seem
On Sun, Feb 19, 2017 at 10:47 AM, Joseph Hackman
wrote:
> Your argument has convinced me, and I now take (what i believe to be) your
>> position:
>>
>
> def stuff(arg = lazy f()):
>
> should result in a function where the default value of arg is not
> evaluated until first function call, and then
>
> This doesn't make sense. Function definition time is very different than
> function execution time. Changing that distinction is a WAY bigger change
> than I think we should contemplate.
> Moreover, there is a completely obvious way to spell the behavior you want:
> def stuff():
> arg = f
On Sun, Feb 19, 2017 at 10:13 AM, Joseph Hackman
wrote:
>
> My honest preference would be that the [] is evaluated fresh each time the
> function is called.
> def stuff(arg=delayed f()):
> would result in f() being called every time stuff() is. This seems more
> valuable to me than just doing it o
Michel-
Thanks for the feedback!
On 19 February 2017 at 11:24, Michel Desmoulin
wrote:
> A great proposal, although now I would have to explain to my students
> the subtle difference between:
>
> res = (print(i * i) for i in range(x))
> print('foo')
> print(res)
>
> And
>
> res = delayed [prin
>
> One last thing: my vote is not dropping the ":" in front of they
> keyword.
>
>
> I think the colon has parser problems, as I showed in some examples.
> Plus I don't like how it looks. But I'd much rather have `a = lazy:
> stuff` than not have the construct at all, nonetheless.
>
On Sun, Feb 19, 2017 at 8:24 AM, Michel Desmoulin wrote:
> A great proposal, although now I would have to explain to my students
> the subtle difference between:
>
> res = (print(i * i) for i in range(x))
> res = delayed [print(i * i) for i in range(x)]
> They seems doing something similar, bu
A great proposal, although now I would have to explain to my students
the subtle difference between:
res = (print(i * i) for i in range(x))
print('foo')
print(res)
And
res = delayed [print(i * i) for i in range(x)]
print('foo')
all(res)
They seems doing something similar, but they really don't.
I would like to add another view of this feature might be very useful
for cleaning up existing code bases:
Just have a look at https://pypi.org/project/xfork/ and specifically I
would like to point you to the following lines
https://github.com/srkunze/fork/blob/afecde0/fork.py#L216 till #419.
On 2/18/17, Steven D'Aprano wrote:
Sorry Steve that I use your words probably too much out of context!
I just want to reuse your examples to analyze if proposed "delayed
execution" is really necessary. Thanks for them! :)
> print("Start")
> result = delayed: get_nth_prime(10**6) # I dislike th
Using "delayed" in function signatures:
On Fri, Feb 17, 2017 at 09:27:35PM -0800, David Mertz wrote:
> That said, I think you are right that it makes no sense to declare a
> function signature with 'delayed' (or 'lazy', 'deferred', whatever word).
It makes perfect sense! That gives us function d
On Fri, Feb 17, 2017 at 9:34 PM, David Mertz wrote:
> On Fri, Feb 17, 2017 at 6:20 PM, Nathaniel Smith wrote:
>>
>> value = delayed: some_dict.get("whatever")
>> if value is None:
>> ...
>>
>> I.e., the question is, how does 'is' work on delayed objects? I guess
>> it has to force the promise
On Fri, Feb 17, 2017 at 9:45 PM, David Mertz wrote:
> That will make it pretty much impossible to tell whether something is a
>>
> delayed "thunk" or not, since *any* attempt to inspect it in any way
>> will cause it to reify. Maybe that's what we want.
>
>
> This feels like a disadvantage, and a
On Fri, Feb 17, 2017 at 5:23 PM, Steven D'Aprano
wrote:
>
> try:
> Aardvark
> except NameError:
> from backport import Aardvark
>
> No such thing is possible for new syntax. So that counts as a
> disadvantage of new syntax. Are we positive that there *must* be new
> syntax to solve this p
I'm not married to the colon. Does anyone else see any issue with dropping
it?
On 18 February 2017 at 00:27, David Mertz wrote:
> On Fri, Feb 17, 2017 at 2:35 PM, Joseph Hackman
> wrote:
>
>> I think we should use the colon to make the delayed word (or whatever
>> word is selected), unambiguous
Well, yes. I think the 'is' operator is where other attempts fall short,
and why it would require a change to Python. But yes, it would need to
force the promise.
On 17 February 2017 at 21:20, Nathaniel Smith wrote:
> On Thu, Feb 16, 2017 at 9:24 PM, Joseph Hackman
> wrote:
> > Howdy All!
> >
>
On Fri, Feb 17, 2017 at 6:20 PM, Nathaniel Smith wrote:
>
> value = delayed: some_dict.get("whatever")
> if value is None:
> ...
>
> I.e., the question is, how does 'is' work on delayed objects? I guess
> it has to force the promise and walk the proxy chain in each input and
> then do an 'is'
On Fri, Feb 17, 2017 at 2:35 PM, Joseph Hackman
wrote:
> I think we should use the colon to make the delayed word (or whatever word
> is selected), unambiguously used in this way (and to prevent any existing
> code from breaking).
>
> On 17 February 2017 at 17:09, David Mertz wrote:
>
>> That wa
On Thu, Feb 16, 2017 at 9:24 PM, Joseph Hackman wrote:
> Howdy All!
>
> This suggestion is inspired by the question on "Efficient debug logging".
>
> I propose a keyword to mark an expression for delayed/lazy execution, for
> the purposes of standardizing such behavior across the language.
>
> The
On 17 February 2017 at 20:23, Steven D'Aprano wrote:
>
> > I think it would be key, like async/await, to narrowly define the scope
> in
> > which the word delayed functions as a keyword.
>
> The PEP makes it clear that's just a transition phase: they will be
> turned into proper keywords in Pytho
On Fri, Feb 17, 2017 at 06:06:26PM -0500, Joseph Hackman wrote:
[...]
> I think it would be key, like async/await, to narrowly define the scope in
> which the word delayed functions as a keyword.
The PEP makes it clear that's just a transition phase: they will be
turned into proper keywords in P
On Fri, Feb 17, 2017 at 04:02:01PM -0600, Abe Dillon wrote:
> I'm fairly novice, so I could be way off base here, but it seems like the
> inevitable conclusion to this problem is something like JIT compilation,
> right? (admittedly, I know very little about JIT compilation)
No.
JIT compilation de
On Fri, 17 Feb 2017 at 23:14 Joshua Morton
wrote:
> @ Ed
>
> Its my understanding that d[k] is always d[k], even if d or k or both are
> delayed. On the other hand, `delayed d[k]` would not be, but you would need
> to explicitly state that. I think its worth expanding on the example Joseph
> made
On 17 February 2017 at 18:13, Joshua Morton
wrote:
> @ Joseph
>
> Function annotations can be arbitrary python expressions, it is completely
> legal to have something like
>
> >>> def foo(bar: lambda x: x + 1):
> ... pass
>
> Why you would want that I can't say, but it is legal. In th
@ Joseph
Function annotations can be arbitrary python expressions, it is completely
legal to have something like
>>> def foo(bar: lambda x: x + 1):
... pass
Why you would want that I can't say, but it is legal. In the same way, `def
foo(bar: delayed 1 + 1)` should probably be legal s
On 17 February 2017 at 06:10, Steven D'Aprano wrote:
> On Fri, Feb 17, 2017 at 12:24:53AM -0500, Joseph Hackman wrote:
>
> > I propose a keyword to mark an expression for delayed/lazy execution, for
> > the purposes of standardizing such behavior across the language.
> >
> > The proposed format i
I think we should use the colon to make the delayed word (or whatever word
is selected), unambiguously used in this way (and to prevent any existing
code from breaking).
On 17 February 2017 at 17:09, David Mertz wrote:
> That was a problem with the colon that occurred to me. I think it can't be
That was a problem with the colon that occurred to me. I think it can't be
tokenized in function annotations.
Plus I still think the no-colon looks better. But that's bikeshedding. Also
other words are plausible. I like lazy even more than delayed, I think.
Still, I'd love the construct whatever t
On Fri, 17 Feb 2017 at 21:58 Joshua Morton
wrote:
> Ed, I'm not seeing this perceived problem either.
>
> if we have
>
> >>> d = delayed {'a': 1, 'b': 2} # I'm not sure how this is delayed
> exactly, but sure
> >>> k = delayed string.ascii_lowercase[0]
> >>> d[k]
> 1
>
My probl
I'm fairly novice, so I could be way off base here, but it seems like the
inevitable conclusion to this problem is something like JIT compilation,
right? (admittedly, I know very little about JIT compilation)
Python seems to be accumulating a lot of different approaches to achieving
very similar t
On Fri, 17 Feb 2017 at 21:57 Joseph Jevnik wrote:
> > You should be able to pass the result to *any* existing code that
> expects a function and sometimes calls it, and the function should be
> called when that happens, rather than evaluated to a delayed object and
> then discarded.
>
> I disagre
Ed, I'm not seeing this perceived problem either.
if we have
>>> d = delayed {'a': 1, 'b': 2} # I'm not sure how this is delayed
exactly, but sure
>>> k = delayed string.ascii_lowercase[0]
>>> d[k]
1
I'm not sure how the delayedness of any of the subexpressions matter, since
e
> You should be able to pass the result to *any* existing code that expects
a function and sometimes calls it, and the function should be called when
that happens, rather than evaluated to a delayed object and then discarded.
I disagree with this claim because I do not think that you should have s
On Fri, 17 Feb 2017 at 21:21 Joseph Jevnik wrote:
> About the "whatever is d[k]" in five minutes comment: If I created an
> explict closure like: `thunk = lambda: d[k]` and then mutated `d` before
> evaluating the closure you would have the same issue. I don't think it is
> that confusing. If you
On Fri, 17 Feb 2017 at 21:18 Joseph Jevnik wrote:
> There is no existing code that uses delayed execution so we don't need to
> worry about breaking it.
>
I think you're missing the point here. This thing is transparent—that's
sort of the entire point—so you can pass delayed expressions to other
Abe-
You are correct. However I think it may still be salvageable.
In your code example, you could be either making a dict with a key of 1, or a
set of a delayed object. But there's no reason to build a set of a delayed
object because hashing it would immediately un-delay.
Similarly, I am not
About the "whatever is d[k]" in five minutes comment: If I created an
explict closure like: `thunk = lambda: d[k]` and then mutated `d` before
evaluating the closure you would have the same issue. I don't think it is
that confusing. If you need to know what `d[k]` evaluates to right now then
the or
There is no existing code that uses delayed execution so we don't need to
worry about breaking it. I think it would be much easier to reason about if
forcing an expression was always explicit. I am not sure what you mean with
the second case; why are you delaying a function if you care about the
ob
On Fri, 17 Feb 2017 at 19:38 Joseph Jevnik wrote:
> Delayed execution and respecting mutable semantics seems like a nightmare.
> For most indexers we assume hashability which implies immutability, why
> can't we also do that here? Also, why do we need to evaluate callables
> eagerly?
>
Respectin
>
> Couldn't the same thing be true of delayed if it is always followed by a
> colon?
No. Because there are other reasons you'd follow the variable `delayed`
with a colon:
>>> delayed = 1
>>> d = {delayed: "oops!"}
My earlier proposal (using unpacking syntax) doesn't work for the same
reason.
I think it could even be true without, but the colon may cause ambiguity
problems with function annotations.
def foo(delayed: delayed: 1 + 2)
is a bit odd, especially if `delayed` is chainable.
--Josh
On Fri, Feb 17, 2017 at 3:32 PM Joseph Hackman
wrote:
> Couldn't the same thing be true o
Couldn't the same thing be true of delayed if it is always followed by a colon?
I.e.
delayed=1
x= delayed: slow_function()
print(delayed) # prints 1
-Joseph
> On Feb 17, 2017, at 2:39 PM, Mark E. Haase wrote:
>
>> On Fri, Feb 17, 2017 at 1:55 PM, Joshua Morton
>> wrote:
>> but I'm wondering
On Fri, Feb 17, 2017 at 1:55 PM, Joshua Morton
wrote:
> but I'm wondering how common async and await were when that was proposed
> and accepted?
Actually, "async" and "await" are backwards compatible due to a clever
tokenizer hack. The "async" keyword may only appear in a few places (e.g.
async
Delayed execution and respecting mutable semantics seems like a nightmare.
For most indexers we assume hashability which implies immutability, why
can't we also do that here? Also, why do we need to evaluate callables
eagerly?
re the thunk replacing itself with the result instead of memoizing the
I think trying to eager-ify subexpressions is absurdly difficult to do
right, and also a problem that occurs in other places in Python already, so
solving it only for this new thing that might very well go no further is a
bit odd.
I don't think versions that aren't transparent are much use.
> Int
Hey! Excellent feedback!
In my mind, which word is selected doesn't matter much to me. I think the
technical term is 'thunk'? I think delayed is most clear.
I'm not sure if eager execution is so common in this framework it needs its own
keyword. Notably, default Python will handle that case
x
I did some quick thinking and a bit of research about some aspects of this
proposal:
There are a number of keyword options (delay, defer, lazy, delayed,
deferred, etc.), a quick look through github says that of these, "deferred"
seems to be the least used, but it still comes up quite a lot (350K t
Actually, following from the idea that packing and unpacking variables
should be delayed by default, it might make sense to use syntax like:
>>> a = *(2+2)
>>> b = a + 1
Instead of
>>> a = lazy 2+2 # or whatever you want the keyword to be
>>> b = a + 1
That syntax sort-of resembles generator e
I'd like to suggest a shorter keyword: `lazy`
This isn't an endorsement. I haven't had time to digest how big this change
would be.
If this is implemented, I'd also like to suggest that perhaps packing and
unpacking should be delayed by default and not evaluated until the contents
are used. It mi
Agreed. I think this may require some TLC to get right, but posting here for
feedback on the idea overall seemed like a good start. As far as I know, the
basic list and dict do not inspect what they contain. I.e.
d = {}
d['a']= delayed: stuff()
b=d['a']
b would end up as still the thunk, and s
On Sat, Feb 18, 2017 at 3:29 AM, Joseph Hackman wrote:
> ChrisA: I am not sure about collections. I think it may be fine to not
> special case it: if the act of putting it in the collection reads anything,
> then it is evaluated, and if it doesn't it isn't. The ideal design goal for
> this woul
Pavol: I think that some sort of magic string that is not a string and is
actually containing Python code could function, but is less elegant.
ChrisA: I am not sure about collections. I think it may be fine to not special
case it: if the act of putting it in the collection reads anything, then i
On Sat, Feb 18, 2017 at 2:12 AM, Joseph Hackman wrote:
> As for what triggers execution? I think everything except being on the right
> side of an assignment. Even identity. So if a delayed expression would
> evaluate to None, then code that checks is None should return true. I think
> this is
On 2/17/17, Chris Angelico wrote:
> Do delayed-expressions have identities or only values? For example:
>
> rand = delayed: random.randrange(10)
> otherrand = rand
> assert rand is otherrand # legal?
> randid = id(rand) # legal?
> print(rand) # force to concrete value
> assert any(rand is x for x
A few points for clarity:
Yes, I would expect each instance of delayed to result in a new delayed
expression, without caching, except for multiple calls to that same delayed
expression instance.
Also, I suggested the colon : because unlike async/await, the following
expression is NOT executed
On Fri, Feb 17, 2017 at 10:10 PM, Steven D'Aprano wrote:
>> the expression is executed and the delayed
>> expression is replaced with the result. (Thus, the delayed expression is
>> only every evaluated once).
>
> That's easily done by having the "delayed" keyword cache each expression
> it sees,
On Fri, Feb 17, 2017 at 12:24:53AM -0500, Joseph Hackman wrote:
> I propose a keyword to mark an expression for delayed/lazy execution, for
> the purposes of standardizing such behavior across the language.
>
> The proposed format is:
> delayed:
> i.e. log.info("info is %s", delayed: expensiveFu
Hi all,
If we want this it might be interesting to investigate what the Scheme
community
has been doing, since they have had this (under the name "promises") for
many years.
Basically:
Scheme: (delay expr)
<=>
proposed Python: delayed: expr
The Scheme community has experimented with what th
Agreed. But there might be cases where something occurring at most one—at
some unspecified time—is desirable behavior. In general though, I think
avoiding side effects should be programming recommendations, not anything
enforced.
This model isn't really so different from what we do with asyncio an
Even with the new syntax I would highly discourage delaying a function with
observable side effects. It would make reasoning about the behavior of the
program very difficult and debugging becomes much harder.
On Fri, Feb 17, 2017 at 3:31 AM, David Mertz wrote:
> I had forgotten about Daisy! It's
I had forgotten about Daisy! It's an interesting project too. The behavior
of 'autodask()' is closer to what I'd want in new syntax than is plain
dask.delayed(). I'm not sure of all the corners. But is definitely love to
have it for expressions generally, not only pure functions.
On Feb 17, 2017 1
You can let dask "see" into the function by entering it and wrapping all of
the operations in `delayed`; this is how daisy[0] builds up large compute
graphs. In this case, you could "inline" the identity function and the
delayed object would flow through the function and the call to identity
never
On Thu, Feb 16, 2017 at 11:15 PM, David Mertz wrote:
>
> This also means that a 'delayed' object needs to be idempotent. So
>
> x = delayed 2+2
>
> y = delayed x
>
> z = delayed delayed delayed y
>
>
> Wrapping more delays around an existing delayed object should probably
> just keep the same obj
I think maybe the idiomatic pattern should be assignment rather than just
bare name. E.g.
f = delayed 1 + 2
# We want to evaluate f before next op for some reason
*f = f*
# f is already a concrete value now, before calculating g
g = f * 7
I think if we follow my rule that "everything lazy w
On Thu, Feb 16, 2017 at 10:33 PM, Joshua Morton
wrote:
> David, can you elaborate on your example?
>
> if we replaced line four with
>
> >>> x = my_lazy_func(b, delayed c)
>
> what would the value of `x` be, and how would this differ from either
>
The value of the function would be whatever
David, can you elaborate on your example?
if we replaced line four with
>>> x = my_lazy_func(b, delayed c)
what would the value of `x` be, and how would this differ from either
>>> x = delayed my_lazy_func(b, delayed c)
or
>>> x = delayed my_lazy_func(b, c)
To put it another way,
Dask also has a function delayed() that may be used as an decorator and in
other ways like:
>>> from dask import delayed
>>> from operator import add, mul
>>> a = delayed(add)(1, 2)
>>> b = delayed(mul)(a, 3)
>>> b
Delayed('mul-1907f29b-60a4-48af-ba2a-938556555f9b')
>>> c = b.compute()
>>> c
9
You might be interested in https://github.com/ll/lazy_python, which
implements the features you describe but instead of a keyword it uses a
decorator.
On Fri, Feb 17, 2017 at 12:24 AM, Joseph Hackman
wrote:
> Howdy All!
>
> This suggestion is inspired by the question on "Efficient debug
I rather like this at first brush!
On Feb 16, 2017 9:25 PM, "Joseph Hackman" wrote:
> Howdy All!
>
> This suggestion is inspired by the question on "Efficient debug logging".
>
> I propose a keyword to mark an expression for delayed/lazy execution, for
> the purposes of standardizing such behavi
Howdy All!
This suggestion is inspired by the question on "Efficient debug logging".
I propose a keyword to mark an expression for delayed/lazy execution, for
the purposes of standardizing such behavior across the language.
The proposed format is:
delayed:
i.e. log.info("info is %s", delayed: e
81 matches
Mail list logo
