I'm excited about the potential introduction of Lisp-style syntactic macros
into Python. 😁
It's unclear to me whether PEP 638 ("Syntactic Macros") is still being actively
developed, since the last activity I see on it is over a year ago (Sep & Oct
2020), but I thought I'd leave some initial comments I have anyway.
Quoting from the PEP text...
(1)
> Lexical analysis
> ~~~~~~~~~~~~~~~~
>
> Any sequence of identifier characters followed by an exclamation point
> (exclamation mark, UK English) will be tokenized as a ``MACRO_NAME``.
+1 to using the ! character to mark macros explicitly and loudly, since they
are very powerful and can completely customize the syntax used inside them.
I like the postfix syntax myself (as proposed) - so `macro_name!` is cool. I
wouldn't be a fan of prefix syntax; `!macro_name` looks ugly.
(2)
> Statement form
> ~~~~~~~~~~~~~~
>
> macro_stmt = MACRO_NAME testlist [ "import" NAME ] [ "as" NAME ] [ ":"
> NEWLINE suite ]
(2.1)
What is `testlist`? It is not defined in this PEP, nor is it defined in the
Python Grammar Specification [1].
Also, the `[ "import" NAME ]` and `[ "as" NAME ]` parts appear to be
special-purpose syntax only useful for supporting the `from!` and `with!`
macros mentioned later in the PEP. It feels odd that this syntax isn't more
general-purpose.
Perhaps the grammar could be made more general with something like:
> macro_stmt = MACRO_NAME macro_expr_parameters ( NEWLINE | ":" NEWLINE suite )
> macro_expr_parameters = ( expression | <keyword> )*
That would allow a statement macro to take some number of expressions and
keywords as arguments, in addition to a suite.
(2.2) +0 to the idea of prefixing a @ to a MACRO_NAME that is intended to be
used as a sibling-macro. That would look like:
> @do_nothing_marker!
> def foo(...):
> ...
(3)
> Expression form
> ~~~~~~~~~~~~~~~
>
> macro_expr = MACRO_NAME "(" testlist ")"
Again, what is `testlist`? Perhaps you're looking for something more like:
> macro_expr = MACRO_NAME "(" macro_parameters ")"
> macro_parameters = ( expression ( "," expression )* ","? )?
(4)
> Resolving ambiguity
> ~~~~~~~~~~~~~~~~~~~
>
> The statement form of a macro takes precedence, so that the code
> ``macro_name!(x)`` will be parsed as a macro statement,
> not as an expression statement containing a macro expression.
It seems to me that if you were to define an expression macro - like `cast!(T,
expression)` - that users would expect to be able to call such an expression
macro on a line by itself. The rule stated here suggests that the parser would
get confused by such a call, incorrectly treating the call as a call of a macro
*statement* rather than a macro *expression*.
(5)
> Compilation
> ~~~~~~~~~~~
>
> For macros with multiple names, [...]
Nit: "Multiple names"? I think you meant macros with "additional names", as
described later in the PEP.
(6)
> Defining macro processors
> ~~~~~~~~~~~~~~~~~~~~~~~~~
>
> A macro processor is defined by a four-tuple, consisting of
> ``(func, kind, version, additional_names)``:
>
> * [...]
> * ``additional_names`` are the names of the additional parts of the macro,
> and must be a tuple of strings.
Seems to me that "additional_names" might make more sense to call "additional
clause names" or just "clause names". That would be consistent with the
following "anatomy of a statement-macro" sketch:
try_!: # begin try_ statement; is try_ clause header
... # is try_ clause body
finally_!: # is finally_ clause header
... # is finally_ clause body; end of try_ statement
(7)
> Hygiene and debugging
> ~~~~~~~~~~~~~~~~~~~~~
>
> [...] No rules for naming will be enforced, but to ensure hygiene and help
> debugging, the following naming scheme is recommended: [...]
This appears to imply that it is up to the macro processor author to take
special care that their macro is hygenic. That is, *unhygenic* macros are the
default. Wouldn't it be safer to define a system where *hygenic* macros would
be the default instead? Seems like if we can head off the introduction of
Command Injection by default at the design level then we should do so.
(8)
> Examples
> ''''''''
Not a single example in this section defines an actual macro processor
function. Recommend implementing at least a toy macro end-to-end (including the
macro processor function) for full clarity, for each type of macro (i.e.
statements, sibling, and expression).
(9)
> Backwards Compatibility
> =======================
>
> This PEP is fully backwards compatible.
Nit: In the §"Implementation" section below, it is mentioned that nodes in
the`_ast` module would be made immutable. That sounds like a
backward-incompatible change to me.
(10)
> Currently, all AST nodes are allocated using an arena allocator.
> Changing to use the standard allocator might slow compilation down a little,
> but has advantages in terms of maintenance, as much code can be deleted.
I presume the arena allocator was introduced in the first place for a reason.
Perhaps to improve performance? By removing the arena allocator are there
potential downsides other than a performance regression?
(11)
> Reference Implementation
> ''''''''''''''''''''''''
>
> None as yet.
Seems like you could get a prototype off the ground by implementing an initial
version as a fake Python source file text encoding.
Then you could put something like `# coding=macros` at the top of a source file
to have it preprocessed by a prototype macro system.
(12)
Thanks for taking the time to read my comments.
--
David Foster | Seattle, WA, USA
Contributor to mypy, TypedDict, and Python's type system
[1]: https://docs.python.org/3/reference/grammar.html
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/AW6KOQ2V3TS6TQUHS4VGFIKSR3SOLJCJ/
Code of Conduct: http://python.org/psf/codeofconduct/
