[Python-ideas] Re: Operator as first class citizens -- like in scala -- or yet another new operator?

[Yanghao Hua]

On Mon, Jun 10, 2019 at 8:57 PM Caleb Donovick  wrote:
>
> First off, I have admittedly not read all of this thread.  However, as 
> designer of DSL's in python, I wanted to jump in on a couple of things I have 
> seen suggested.  Sorry If I am repeating comments already made.  Over the 
> last few years I have thought about every suggestion I have seen in this 
> thread and they all don't work or are undesirable for one reason or another.

Glad to see I am not alone in the woods :-) So the argument that this
problem is only Yanghao -- a single person's problem -- can be gone.
You have to rephrase it to "this is only two person's problem" now.
;-)

> Regarding what code will become simpler if an assignment operator was 
> available.  I currently walk the AST to rewrite assignments into the form I 
> want.  This is code is really hard to read if you are not familiar with the 
> python AST, but the task it is performing is not hard to understand at all 
> (replace assignment nodes with calls to a function call to 
> dsl_assign(target_names, value, globals(), locals()).  The dsl_assign 
> function basically performs some type checking before doing the assignment.  
> Once again the code is much harder to understand than it should be as it 
> operates on names of variables and the globals / locals dictionaries instead 
> of on the variables themselves.  Also to get the hook into the AST I have to 
> have use an importer which further obscures my code and makes use kinda 
> annoying as one has to do the following:
> ```
> main.py:
> import dsl # sets up the importer to rewrite the AST
> import dsl_code # the code which should morally be the main but most be 
> imported after dsl
> ```
> Granted anything in a function or a class can be rewritten with a decorator 
> but module level code must be rewritten by an importer.

I thought about doing it a lot ... eventually manipulating AST is not
so much easier than actually re-develop the entire DSL from scratch
... and this suffers eventually similar isssues like @=/L[:]
overriding, it abuses a common understanding. I have also been looking
into MacroPy3 for some time now, and despite you still need something
like P[your customized expression] (the P[...] overhead which is
exposed to end-users), changing an existing python syntax to mean
something completely different, or translating a non-exist python
syntax into something else really makes me feel this will make things
mentally inconsistent, and actually makes python no longer python ...
And this haven't touched what it means for debugging later on ...

> The problem with overloading obj@=value:
> As Yanghao have pointed out @ comes with expectations of behavior.  Granted I 
> would gamble most developers are unaware that @ is python operator but it 
> still has meaning and as such I don't like abusing it.

Yep.

> The problem with using obj[:]=value:
> Similar to @ getitem and slices have meaning which I don't necessarily want 
> to override. Granted this is least objectionable solution I have seen 
> although, it creates weird requirements for types that have __getitem__ so it 
> can just be used by inheriting `TypedAssignment` or something similar.

Exactly. I will try to summarize all the pros and cons I saw people
posting for the L[:] case, and the thing for DSL is L[:] is confusing
on its own (not better than obj.next = ...).

> The problem with descriptors:
> They are hard to pass to functions, for example consider trying to fold 
> assignment.
> ```
> signals = [Signal('x'), Signal('y'), Signal('z')]
> out = functools.reduce(operator.iassign, signals)
> ```
> vs
> ```
> signals = SignalNamespace()
> signal_names =  ['x', 'y', 'z']
> out = functools.reduce(lambda v, name: settattr(signals, name, v))
> ```
> In general one has to pass the name of the signal and the namespace to a 
> function instead the signal itself which is problematic.
>
> The problem with exec:
> First off its totally unpythonic, but even if I hide the exec with importer 
> magic it still doesn't give me the behavior I want.
> Consider the following
> ```
> class TypeCheckDict(dict, MutableMapping): # dict needed to be used as globals
>   """
>   Dictionary which  binds keys to a type on first assignment then type checks 
> on future
>   assignement. Will infer type if not already bound. """
>   __slots__ = '_d'
>   def __init__(self, d=_MISSING):
> if d is _MISSING:
>   d = {}
> self._d = d
>
>   def __getitem__(self, name):
> v = self._d[name][1]
> if v is _MISSING:
>   raise ValueError()
> else:
>   return v
>
>   def __setitem__(self, name, value):
> if name not in self._d:
>   if isinstance(value, type):
> self._d[name] = [value, _MISSING]
>   else:
> self._d[name] = [type(value), _MISSING]
>   elif isinstance(value, self._d[name][0]):
> self._d[name][1] = value
>   else:
> raise TypeError(f'{value} is not a {self._d[name][0]}')
>
>   # __len__ __iter

[Python-ideas] Re: Assign-in-place operator

[Caleb Donovick]

The problem as I see it with slice assignment is that if we want to
operator to mean type defined assignment not necessary in place assignment.
It creates confusion for types which have __setitem__.

Caleb Donovick




On Thu, Jun 6, 2019 at 4:59 PM Greg Ewing 
wrote:

> Stephen J. Turnbull wrote:
> > L[m:m+k] specifies that a list operation will take
> > place on the k elements starting with m.  As a value, it makes a new
> > list of references to those elements.
>
> Even that is specific to lists. There's no requirement that a
> RHS slice has to create new references to elements. A type can
> define it so that it returns a mutable view of part of the
> original object. This is how numpy arrays behave, for example.
>
> As syntax, slice notation simply denotes a range of elements,
> and it does that the same way whether it's on the LHS or RHS.
>
> --
> Greg
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/HF3RYOEM3WG73IF5DACNBDVMBLD3PCBI/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/JQI2O6Y7KQM46AXNHZAC7UFAL737T2FZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Operator as first class citizens -- like in scala -- or yet another new operator?

[Barry Scott]

> On 11 Jun 2019, at 09:50, Yanghao Hua  wrote:
> 
> On Mon, Jun 10, 2019 at 8:57 PM Caleb Donovick  > wrote:
>> 
>> First off, I have admittedly not read all of this thread.  However, as 
>> designer of DSL's in python, I wanted to jump in on a couple of things I 
>> have seen suggested.  Sorry If I am repeating comments already made.  Over 
>> the last few years I have thought about every suggestion I have seen in this 
>> thread and they all don't work or are undesirable for one reason or another.
> 
> Glad to see I am not alone in the woods :-) So the argument that this
> problem is only Yanghao -- a single person's problem -- can be gone.
> You have to rephrase it to "this is only two person's problem" now.
> ;-)

Some times a DSL is usable within the python syntax and that is great.
I have use python for a number of DSL's.

But when the DSL is beyond what python can help with directly I'm wondering
why you do not parse the DSL with python and execute the results.

In that way you can have any semantics that you want from any syntax that you
wish to have. However you do not need the python language to be changed at all.

And it must be clear that you are making little to no progress on convincing 
people
that changing python is a good idea.

Barry


> 
>> Regarding what code will become simpler if an assignment operator was 
>> available.  I currently walk the AST to rewrite assignments into the form I 
>> want.  This is code is really hard to read if you are not familiar with the 
>> python AST, but the task it is performing is not hard to understand at all 
>> (replace assignment nodes with calls to a function call to 
>> dsl_assign(target_names, value, globals(), locals()).  The dsl_assign 
>> function basically performs some type checking before doing the assignment.  
>> Once again the code is much harder to understand than it should be as it 
>> operates on names of variables and the globals / locals dictionaries instead 
>> of on the variables themselves.  Also to get the hook into the AST I have to 
>> have use an importer which further obscures my code and makes use kinda 
>> annoying as one has to do the following:
>> ```
>> main.py:
>> import dsl # sets up the importer to rewrite the AST
>> import dsl_code # the code which should morally be the main but most be 
>> imported after dsl
>> ```
>> Granted anything in a function or a class can be rewritten with a decorator 
>> but module level code must be rewritten by an importer.
> 
> I thought about doing it a lot ... eventually manipulating AST is not
> so much easier than actually re-develop the entire DSL from scratch
> ... and this suffers eventually similar isssues like @=/L[:]
> overriding, it abuses a common understanding. I have also been looking
> into MacroPy3 for some time now, and despite you still need something
> like P[your customized expression] (the P[...] overhead which is
> exposed to end-users), changing an existing python syntax to mean
> something completely different, or translating a non-exist python
> syntax into something else really makes me feel this will make things
> mentally inconsistent, and actually makes python no longer python ...
> And this haven't touched what it means for debugging later on ...
> 
>> The problem with overloading obj@=value:
>> As Yanghao have pointed out @ comes with expectations of behavior.  Granted 
>> I would gamble most developers are unaware that @ is python operator but it 
>> still has meaning and as such I don't like abusing it.
> 
> Yep.
> 
>> The problem with using obj[:]=value:
>> Similar to @ getitem and slices have meaning which I don't necessarily want 
>> to override. Granted this is least objectionable solution I have seen 
>> although, it creates weird requirements for types that have __getitem__ so 
>> it can just be used by inheriting `TypedAssignment` or something similar.
> 
> Exactly. I will try to summarize all the pros and cons I saw people
> posting for the L[:] case, and the thing for DSL is L[:] is confusing
> on its own (not better than obj.next = ...).
> 
>> The problem with descriptors:
>> They are hard to pass to functions, for example consider trying to fold 
>> assignment.
>> ```
>> signals = [Signal('x'), Signal('y'), Signal('z')]
>> out = functools.reduce(operator.iassign, signals)
>> ```
>> vs
>> ```
>> signals = SignalNamespace()
>> signal_names =  ['x', 'y', 'z']
>> out = functools.reduce(lambda v, name: settattr(signals, name, v))
>> ```
>> In general one has to pass the name of the signal and the namespace to a 
>> function instead the signal itself which is problematic.
>> 
>> The problem with exec:
>> First off its totally unpythonic, but even if I hide the exec with importer 
>> magic it still doesn't give me the behavior I want.
>> Consider the following
>> ```
>> class TypeCheckDict(dict, MutableMapping): # dict needed to be used as 
>> globals
>>  """
>>  Dictionary which  binds keys to a type on first assig