subject:"\[Python\-Dev\] Re\: Hygenic macros PEP."

[Python-Dev] Re: Hygenic macros PEP.

2020-09-19 Thread Random832

On Tue, Sep 15, 2020, at 15:22, Mark Shannon wrote:
> 
> Hi all,
> 
> I'd like to propose a new PEP for hygienic macros.

To be clear, "hygienic macros" doesn't *just* mean AST-rewriting macros, it 
also means being able to define variables within the macro expansion that can't 
possibly collide with any names in the user code or in any other macros.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LHNATFDIXG5ZCWM7A2LEMDL6OQBG6WFA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Hygenic macros PEP.

2020-09-16 Thread Steve Holden

On Wed, Sep 16, 2020 at 3:53 PM Mark Shannon  wrote:

> [...]
>
> maybe!(a.b)
>
> which would translate to:
>
> $tmp = a; None if $tmp is None else ($tmp.b)
>
> This reminds me very much of PL/1's "compile-time  procedures," which
forty years ago were pretty much like PL/1 constructs with percent signs
sprinkled about. I didn't like those much, either ;D.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/INDHHUK5FXZ2MQFX6KMKIHDUPYYBMRGP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Hygenic macros PEP.

2020-09-16 Thread Mark Shannon

Hi Guido,

On 16/09/2020 5:54 am, Guido van Rossum wrote:
On Tue, Sep 15, 2020 at 7:31 PM Benjamin Peterson > wrote:

On Tue, Sep 15, 2020, at 20:10, Greg Ewing wrote:
> Maybe the PEP should propose an AST of its own, which would initially
> be a third thing separate from either of the existing ones, with
> the possibility of adopting it as the ast module representation
> some time in the future.

The rust approach to this problem to pass only a token stream to the
macro. Then, the macro writer brings their own parser.

And (I looked this up [1]) it returns another token stream which is then
re-parsed by the standard parser. I'm not quite sure who is responsible
for determining where the macro ends -- the standard parser or the macro
parser.

A nice side effect of this would be that (hopefully) the macro syntax
doesn't need to have the ugly special cases needed to allow `import! foo
as bar` and `from! mod import foo as bar` (though these still need to be
predefined macros).

I'm not entirely sure how this would preserve the hygiene of the macros
though, since the macro's parser could just do a token-by-token
substitution without checking the context, effectively what a macro in C
does. Forcing an AST on the macros restricts macros somewhat (e.g. it
requires the input arguments to look like well-formed expressions) but
that's probably a good thing.

(Hm, or not. How would you implement something like PEP 505 using
macros? It introduces new tokens like `?.` and `??`.

You can't implement any new operators with the PEP.

Having said that, a None-aware attribute operator could be implemented
something like this:

maybe!(a.b)

which would translate to:

$tmp = a; None if $tmp is None else ($tmp.b)

Cheers,
Mark.

[1] https://doc.rust-lang.org/book/ch19-06-macros.html

--
--Guido van Rossum (python.org/~guido )
/Pronouns: he/him //(why is my pronoun here?)/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/LRJCJN6LTA7Z65VBTPXIFLCX4LRXKLYN/
Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/3OS6K5YDVQ2CMSWV6CWBM3JK6UPYID3K/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Hygenic macros PEP.

2020-09-16 Thread Mark Shannon

Hi Benjamin,

I don't like the idea of just passing a stream of tokens to the macro
processor, for three reasons.

1. It makes life unnecessarily difficult for macro implementers by
forcing them to parse the stream of tokens.

Macros may contain difficult to parse constructs, and other macros.
If all that ends up happening is that macro processors reconstruct the
source and pass it to ast.parse(), then what's the point?

2. I want macros to have familiar syntactic structure, even if the
semantics are novel.

3. Pablo has already pointed out the potential performance impact of
exposing the AST at the Python level. Passing tokens around would mean
that both the AST nodes and lexer tokens would need to be Python
objects, making the performance impact much greater.

Cheers,
Mark.

On 16/09/2020 2:33 pm, Benjamin Peterson wrote:

On Tue, Sep 15, 2020, at 23:54, Guido van Rossum wrote:

On Tue, Sep 15, 2020 at 7:31 PM Benjamin Peterson wrote:

On Tue, Sep 15, 2020, at 20:10, Greg Ewing wrote:

Maybe the PEP should propose an AST of its own, which would initially
be a third thing separate from either of the existing ones, with
the possibility of adopting it as the ast module representation
some time in the future.

The rust approach to this problem to pass only a token stream to the macro.
Then, the macro writer brings their own parser.

And (I looked this up [1]) it returns another token stream which is
then re-parsed by the standard parser. I'm not quite sure who is
responsible for determining where the macro ends -- the standard parser
or the macro parser.

The standard parser determines the macro boundaries and passes the macro
implementation only the relevant tokens.

A nice side effect of this would be that (hopefully) the macro syntax
doesn't need to have the ugly special cases needed to allow `import!
foo as bar` and `from! mod import foo as bar` (though these still need
to be predefined macros).

I'm not entirely sure how this would preserve the hygiene of the macros
though, since the macro's parser could just do a token-by-token
substitution without checking the context, effectively what a macro in
C does. Forcing an AST on the macros restricts macros somewhat (e.g. it
requires the input arguments to look like well-formed expressions) but
that's probably a good thing.

The rust macro TokenStream is a bit higher level than a traditional token
stream. In particular, the macro implementation can request that new
identifiers it inserts into the stream are only resolvable within the macro for
hygiene. There's a bit about that here:
https://doc.rust-lang.org/proc_macro/struct.Ident.html#impl

(Hm, or not. How would you implement something like PEP 505 using
macros? It introduces new tokens like `?.` and `??`.

Is there any proposal that would let you add new tokens?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/GFDSVQ4Q7KBPUOKCIPGU724ANQZGVWV3/
Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/6RYU5Q5UMWACBSDLNHBPINCDLW5D4KWI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Hygenic macros PEP.

2020-09-16 Thread Batuhan Taskaya




On 16.09.2020 16:33, Benjamin Peterson wrote:

Is there any proposal that would let you add new tokens?


Not a proposal, but drafted a project a while back. By patching 
tokenizer and the parser at runtime (generated from lib2to3.pgen), 
allowed to introduce different tokens / rules, and worked like a 
transpiler (so that you can take the source code with creating a fake 
encoding, giving it as a raw source stream to the new parser with 
updated rules and then return the result as the decoded source);


https://github.com/isidentical/freesyntax/blob/master/example2.py
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SBQUKFSJDXOJBPXM42RHRNDJAB3J4I2J/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Hygenic macros PEP.

2020-09-16 Thread Benjamin Peterson

On Tue, Sep 15, 2020, at 23:54, Guido van Rossum wrote:
> On Tue, Sep 15, 2020 at 7:31 PM Benjamin Peterson  wrote:
> > On Tue, Sep 15, 2020, at 20:10, Greg Ewing wrote:
> > > Maybe the PEP should propose an AST of its own, which would initially
> > > be a third thing separate from either of the existing ones, with
> > > the possibility of adopting it as the ast module representation
> > > some time in the future.
> > 
> > The rust approach to this problem to pass only a token stream to the macro. 
> > Then, the macro writer brings their own parser.
> 
> And (I looked this up [1]) it returns another token stream which is 
> then re-parsed by the standard parser. I'm not quite sure who is 
> responsible for determining where the macro ends -- the standard parser 
> or the macro parser.

The standard parser determines the macro boundaries and passes the macro 
implementation only the relevant tokens.

> 
> A nice side effect of this would be that (hopefully) the macro syntax 
> doesn't need to have the ugly special cases needed to allow `import! 
> foo as bar` and `from! mod import foo as bar` (though these still need 
> to be predefined macros).
> 
> I'm not entirely sure how this would preserve the hygiene of the macros 
> though, since the macro's parser could just do a token-by-token 
> substitution without checking the context, effectively what a macro in 
> C does. Forcing an AST on the macros restricts macros somewhat (e.g. it 
> requires the input arguments to look like well-formed expressions) but 
> that's probably a good thing.

The rust macro TokenStream is a bit higher level than a traditional token 
stream. In particular, the macro implementation can request that new 
identifiers it inserts into the stream are only resolvable within the macro for 
hygiene. There's a bit about that here: 
https://doc.rust-lang.org/proc_macro/struct.Ident.html#impl

> 
> (Hm, or not. How would you implement something like PEP 505 using 
> macros? It introduces new tokens like `?.` and `??`.

Is there any proposal that would let you add new tokens?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GFDSVQ4Q7KBPUOKCIPGU724ANQZGVWV3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Hygenic macros PEP.

2020-09-16 Thread edwin

September 16, 2020 12:24 AM, "Guido van Rossum" mailto:gu...@python.org?to=%22Guido%20van%20Rossum%22%20)> 
wrote:
This may actually address the worry that was expressed that libraries will 
become too dependent on macros, by making it painful to maintain a macro 
processor across many versions. It will serve as a natural deterrent for 
libraries desiring stability.
This exactly addresses the worry that I have in a much better way that the 
command line flag that was mentioned. As long as it is inconvenient enough to 
prevent it from being THE WAY(tm) to code in Python, I have no other objections.

--Edwin
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/C55EZ7BNZCNFMBOOI7RFAASBALLS67EK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Hygenic macros PEP.

2020-09-16 Thread Pablo Galindo Salgado

> Would it complicate the parser much to allocate memory using a different
allocator? It's just a function call, presumably.

Is not the allocator per-se, is the fact that with the arena we can free
*all the memory* in one single call, instead of having to implement custom
free functions for the nodes or triggering a recursive chain of
deallocations. The
parser itself owns all the memory block (by owning the arena) so we don't
need
to add specific ownership models to the nodes, which makes the inner
workings
much easier to organise, including temporary allocations.

> As for performance, we don't know how much difference it will make until
we try.

True

> Deallocation doesn't have to be recursive.

But if we model the nodes as Python objects with owership via attributes,
it will be recursive because
it's a tree. You would need specific allocators to avoid it.



On Wed, 16 Sep 2020 at 12:48, Mark Shannon  wrote:

>
>
> On 16/09/2020 12:22 pm, Pablo Galindo Salgado wrote:
> > > Don't we need to do all of this in the _ast module, already?
> > > We already have an AST composed of Python objects
> >
> >
> > We have AST of Python objects, but the python versions are not used
> > internally, especially in the parser, where they are created. The parser
> > and the compiler
> > currently use exclusively the C versions of the nodes. Changing the
> parser
> > to use Python versions and especially not to use the arena allocator can
> > complicate
> > the parser substantially, and destroying the resulting tree can be
> > expensive (as destroying
> > a deep tree composed of Python objects currently is).
>
> Would it complicate the parser much to allocate memory using a different
> allocator? It's just a function call, presumably.
>
> As for performance, we don't know how much difference it will make until
> we try. Deallocation doesn't have to be recursive.
>
> Cheers,
> Mark.
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/U2QWBHZTLW7NSRH74FOAJO7Z2QZDYTR6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Hygenic macros PEP.

2020-09-16 Thread Mark Shannon

On 16/09/2020 12:22 pm, Pablo Galindo Salgado wrote:

> Don't we need to do all of this in the _ast module, already?
> We already have an AST composed of Python objects

We have AST of Python objects, but the python versions are not used
internally, especially in the parser, where they are created. The parser 
and the compiler

currently use exclusively the C versions of the nodes. Changing the parser
to use Python versions and especially not to use the arena allocator can 
complicate
the parser substantially, and destroying the resulting tree can be 
expensive (as destroying

a deep tree composed of Python objects currently is).

Would it complicate the parser much to allocate memory using a different 
allocator? It's just a function call, presumably.

As for performance, we don't know how much difference it will make until 
we try. Deallocation doesn't have to be recursive.

Cheers,
Mark.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ANUHAJT2MHG6ATUYTBSZYKYQYKP46PZP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Hygenic macros PEP.

2020-09-16 Thread Mark Shannon

Hi Matthias,

On 15/09/2020 11:25 pm, Matthias Bussonnier wrote:

Mark,

Thanks you this is an interesting proposal. I can see how many of the "magics"
provided by IPython/Jupyter could also be replaced by some of those macros.

A couple of questions – which I understand could be delegated to further
improvement to macros in a subsequent proposal.

- I'm not sure I see how these macros could take parameters ? For example
`timeit` in IPython takes a number of options on the number of repetitions and
the number of innerloop for really fast code. I'm not sure how this would be
done with your proposal, would
stmt_macro!(options):
multi_statement_body
Be something you consider for a later iteration? I might be missing how to do
it, apologies about this if its the case.
passing parameters also likely apply to siblings macros?

Macros already take arguments.

timeit! 1:
body

is legal syntax.
It is up to the timeit macro how to parse the arguments.

Alternatively you could use an inner macro:

timeit!:
repetitions! 1
body

- especially for statement_macros, have you considered to relax the syntax
inside the block, and pass the string of the dedented block to the macross
processor (or have it as an option). That would make it quite easier to embed
non-python syntax, and allow to convert a much larger chunk of the
IPython/Jupyter magics to macros.

I'm not sure that further relaxing the syntax would be acceptable. The
term "a nightmarish hodge-podge of different dialects" has already been
used :)

Cheers,
Mark.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/ALUNO4F4OFFRGNSCTYKT7AOZGSFPAE67/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Hygenic macros PEP.

2020-09-16 Thread Mark Shannon

Hi Victor,

On 15/09/2020 10:55 pm, Victor Stinner wrote:

Hi Mark,

In 2016, I wrote PEP 511 "API for code transformers" which proposed a
different implementation but also used "preprocessor" use case in the
rationale:

"""
A preprocessor can be easily implemented with an AST transformer. A
preprocessor has various and different usages.

PEP 511 seems to be entirely focused on performance.
My PEP is about semantics.

Some examples:

* Remove debug code like assertions and logs to make the code faster
to run it for production.
* Tail-call OptimizationTail-call recursion only, surely?

Tail-call elimination in general would need interpreter support.

* Add profiling code
* Lazy evaluation: see lazy_python (bytecode transformer) and lazy
macro of MacroPy (AST transformer)
* Change dictionary literals into collection.OrderedDict instances
* Declare constants: see @asconstants of codetransformer
* Domain Specific Language (DSL) like SQL queries. The Python language
itself doesn't need to be modified. Previous attempts to implement DSL
for SQL like PEP 335 - Overloadable Boolean Operators was rejected.
* Pattern Matching of functional languages
* String Interpolation, but PEP 498 -- Literal String Interpolation
was merged into Python 3.6.

MacroPy has a long list of examples and use cases.

This PEP does not add any new code transformer. Using a code
transformer will require an external module and to register it
manually.
"""
https://www.python.org/dev/peps/pep-0511/#usage-2-preprocessor

I rejected my PEP, extract of the rejection notice:

"""
This PEP was seen as blessing new Python-like programming languages
which are close but incompatible with the regular Python language. It
was decided to not promote syntaxes incompatible with Python.

This PEP was also seen as a nice tool to experiment new Python
features, but it is already possible to experiment them without the
PEP, only with importlib hooks. If a feature becomes useful, it should
be directly part of Python, instead of depending on an third party
Python module.
"""
https://www.python.org/dev/peps/pep-0511/#rejection-notice

Victor

Le mar. 15 sept. 2020 à 21:27, Mark Shannon a écrit :

Hi all,

I'd like to propose a new PEP for hygienic macros.

I'm attempting to reduce the controversy and pain of adding new syntax
to Python,
... by adding new syntax.

That might sound silly, but I hope to persuade that it isn't.

Adding a new feature to the language is always controversial.

Before a new module or functionality is added to the standard library,
it must usually prove itself as a package on PyPI.

(Not always the same package, the value of dataclasses was demonstrated
by "attrs".)

However, this isn't currently possible with new language features.

Any new language feature has pros and cons.
It adds expressive power for some (usually experienced) users,
but potentially confuses and inconveniences other users (especially
beginners).
However, it is very hard, if not impossible, to determine whether adding
new syntax will be be beneficial or harmful before the syntax is added.

Is there a way to add new syntax in a way that allows it be battle
tested before adding to the released version of the language?
Yes there is, Macros.

I'm not talking about C-style macros which do textual substitution, but
"hygienic" macros that rewrite the AST during compilation.

Macros allow domain specific new syntactic features, without bloating
the language or confusing newcomers.

In general, I would except macros to be used within libraries, so that
those libraries gain the power of their custom macros without making
Python ever larger and more complex.

However, there is no reason why they cannot be made more widely
available. Should a macro become widely used and popular, then it can be
considered for adoption into the language.

The PEP:

https://github.com/markshannon/peps/blob/hygenic-macros/pep-0700.rst

Cheers,
Mark.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/7BX4S6ROKPMTTJR64UPRJ4D66C23HMHI/
Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/QLI62HJNPCZYBTHHSYHN7GVBF5UAUUK7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Hygenic macros PEP.

2020-09-16 Thread Pablo Galindo Salgado

> > Don't we need to do all of this in the _ast module, already?
> > We already have an AST composed of Python objects


We have AST of Python objects, but the python versions are not used
internally, especially in the parser, where they are created. The parser
and the compiler
currently use exclusively the C versions of the nodes. Changing the parser
to use Python versions and especially not to use the arena allocator can
complicate
the parser substantially, and destroying the resulting tree can be
expensive (as destroying
a deep tree composed of Python objects currently is).

Cheers from cloudy London,
Pablo
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/Z3YSR5VDDVEEZZUND36VFXPS7W3SIAQW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Hygenic macros PEP.

2020-09-16 Thread Mark Shannon

Hi Pablo,

On 15/09/2020 10:34 pm, Pablo Galindo Salgado wrote:

Thanks for the proposal Mark!

I wanted to make some comments regarding converting AST nodes to
PyObjects internally. I see some challenges here:

* Not using an arena allocator for the nodes can introduce more
challenges than simplifications. The first is that deleting a deep tree
currently is just freeing the arena block, while if the nodes were
PyObjects it will involve recursive destruction. That could potentially
segfault so we would need to use some custom trashcan mechanism of
special deleters. All of this will certainly not simplify the code (at
least the parser code) and will impact performance (although just in the
parser/compiler phase).

Don't we need to do all of this in the _ast module, already?
We already have an AST composed of Python objects.

* We would need to (potentially) reimplement the AST sequences into
proper owning-containers. That will involve changing a considerable
amount of code and some slowdown due to having to use C-API calls.

I agree that the sequences would need additional code for them to be
Python objects.

They would be more expensive to create, but I don't the additional
cost being prohibitive.

* The proposal seems to imply that the AST will be a fully public and
stable API. This has some danger as any internal optimization regarding
AST can be braking changes to macro users. This will raise any problems
that now linter and static analyst tools could have to all users (of
macros), making of even more difficult to change it.

This is a good point.

I am going to change macros to take a testlist instead of an arglist.
Historically, function arguments have been an unstable part of the API.
Since the AST node for testlist is just a list of ast.expr, it should be
stable.

That way at least the top level of the AST will be stable.

Cheers,
Mark.

Regards from sunny London,
Pablo Galindo

On Tue, 15 Sep 2020, 20:28 Mark Shannon, > wrote:

Hi all,

I'd like to propose a new PEP for hygienic macros.

I'm attempting to reduce the controversy and pain of adding new syntax
to Python,
... by adding new syntax.

That might sound silly, but I hope to persuade that it isn't.

Adding a new feature to the language is always controversial.

Before a new module or functionality is added to the standard library,
it must usually prove itself as a package on PyPI.

(Not always the same package, the value of dataclasses was demonstrated
by "attrs".)

However, this isn't currently possible with new language features.

Any new language feature has pros and cons.
It adds expressive power for some (usually experienced) users,
but potentially confuses and inconveniences other users (especially
beginners).
However, it is very hard, if not impossible, to determine whether
adding
new syntax will be be beneficial or harmful before the syntax is added.

Is there a way to add new syntax in a way that allows it be battle
tested before adding to the released version of the language?
Yes there is, Macros.

I'm not talking about C-style macros which do textual substitution, but
"hygienic" macros that rewrite the AST during compilation.

Macros allow domain specific new syntactic features, without bloating
the language or confusing newcomers.

In general, I would except macros to be used within libraries, so that
those libraries gain the power of their custom macros without making
Python ever larger and more complex.

However, there is no reason why they cannot be made more widely
available. Should a macro become widely used and popular, then it
can be
considered for adoption into the language.

The PEP:

https://github.com/markshannon/peps/blob/hygenic-macros/pep-0700.rst

Cheers,
Mark.
___
Python-Dev mailing list -- python-dev@python.org

To unsubscribe send an email to python-dev-le...@python.org

https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at

https://mail.python.org/archives/list/python-dev@python.org/message/7BX4S6ROKPMTTJR64UPRJ4D66C23HMHI/
Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/DWDTBJQ6YANNPHRFIOEU4NZOPP4SVR3M/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Hygenic macros PEP.

2020-09-16 Thread Mark Shannon

Hi David,

On 15/09/2020 9:20 pm, David Mertz wrote:

Three-ish questions:

1. What could this do that Macropy does not already do? (yes, I know
"run as top-level script", but that's uninspiring for me).

Macros in MacroPy don't look like macros, they look
like normal Python code.

The proposed macros have distinct syntax, making it obvious at a glance
that they are macros. Since the macro name is a distinct token it makes
it obvious, especially once editors start highlighting them differently.

The proposed macros support additional syntactic forms.

The additional stmt_expr AST node make it possible to support complex
flow control like finally blocks, which MacroPy cannot do.

The import hooks that MacroPy uses have no means to support macro
versioning, resulting in either stale pyc files or excessive compilation.

I'm not sure which.

2. Do you have any evidence Numba developers would actually want this?!
(as claimed in draft FAQ). I know a lot of them, and teach Numba
sometimes, although have not directly contributed. I do not believe
they would prefer the AST to the bytecode for the analysis.

When writing a compiler you want the AST. It doesn't really matter what
the compiler is.

I'll change the example, so that I'm not attributing things to the numba
developers without their consent.

3. I guess not really a question, but I just can't see a language
feature like this as fully "opt-in". Any feature that is added is going
to occur in code I wind up having to maintain. That's already a pain
point with e.g. metaclasses/decorators, monkey patching, and other
constructs, where code often doesn't mean what it *looks like* it would
mean.

Whether a macro does what it looks like it does is largely down to
naming and design. The same could be said of library code in general.

... macros, even if hygienic, seem to make that need to learn a new DSL
or language for every code base much worse than those other constructs.
Looking at code with entirely novel flow control constructs adds a lot
of cognitive burden. Not least because there are likely to be a hundred
different implementations that make "case" or "do until" do very subtly
different things in each one.

On Tue, Sep 15, 2020 at 9:27 AM Mark Shannon > wrote: